title: Beyond Image Super-Resolution for Image Recognition with Task-Driven Perceptual Loss

abstract: In real-world scenarios, image recognition tasks, such as semantic segmentation and object detection, often pose greater challenges due to the lack of information available within low-resolution (LR) content. Image super-resolution (SR) is one of the promising solutions for addressing the challenges. However, due to the ill-posed property of SR, it is challenging for typical SR methods to restore task-relevant high-frequency contents, which may dilute the advantage of utilizing the SR method. Therefore, in this paper, we propose Super-Resolution for Image Recognition (SR4IR) that effectively guides the generation of SR images beneficial to achieving satisfactory image recognition performance when processing LR images. The critical component of our SR4IR is the task-driven perceptual (TDP) loss that enables the SR network to acquire task-specific knowledge from a network tailored for a specific task. Moreover, we propose a cross-quality patch mix and an alternate training framework that significantly enhances the efficacy of the TDP loss by addressing potential problems when employing the TDP loss. Through extensive experiments, we demonstrate that our SR4IR achieves outstanding task performance by generating SR images useful for a specific image recognition task, including semantic segmentation, object detection, and image classification. The implementation code is available at https://github.com/JaehaKim97/SR4IR.


title: HeMeNet: Heterogeneous Multichannel Equivariant Network for Protein Multitask Learning

abstract: Understanding and leveraging the 3D structures of proteins is central to a variety of biological and drug discovery tasks. While deep learning has been applied successfully for structure-based protein function prediction tasks, current methods usually employ distinct training for each task. However, each of the tasks is of small size, and such a single-task strategy hinders the models' performance and generalization ability. As some labeled 3D protein datasets are biologically related, combining multi-source datasets for larger-scale multi-task learning is one way to overcome this problem. In this paper, we propose a neural network model to address multiple tasks jointly upon the input of 3D protein structures. In particular, we first construct a standard structure-based multi-task benchmark called Protein-MT, consisting of 6 biologically relevant tasks, including affinity prediction and property prediction, integrated from 4 public datasets. Then, we develop a novel graph neural network for multi-task learning, dubbed Heterogeneous Multichannel Equivariant Network (HeMeNet), which is E(3) equivariant and able to capture heterogeneous relationships between different atoms. Besides, HeMeNet can achieve task-specific learning via the task-aware readout mechanism. Extensive evaluations on our benchmark verify the effectiveness of multi-task learning, and our model generally surpasses state-of-the-art models.


title: Accurate and precise quantum computation of valence two-neutron systems

abstract: Developing methods to solve nuclear many-body problems with quantum computers is an imperative pursuit within the nuclear physics community. Here, we introduce a quantum algorithm to accurately and precisely compute the ground state of valence two-neutron systems leveraging presently available Noisy Intermediate-Scale Quantum devices. Our focus lies on the nuclei having a doubly-magic core plus two valence neutrons in the $ p $, $ sd $, and $ pf $ shells, i.e. ${}^6$He, ${}^{18}$O, and ${}^{42}$Ca, respectively. Our ansatz, quantum circuit, is constructed in the pair-wise form, taking into account the symmetries of the system in an explicit manner, and enables us to reduce the number of qubits and the number of CNOT gates required. The results on a real quantum hardware by IBM Quantum Platform show that the proposed method gives very accurate results of the ground-state energies, which are typically within $ 0.1 \, \% $ error in the energy for ${}^6$He and ${}^{18}$O and at most $ 1 \, \% $ error for ${}^{42}$Ca. Furthermore, our experiments using real quantum devices also show the pivotal role of the circuit layout design, attuned to the connectivity of the qubits, in mitigating errors.


title: Selective Temporal Knowledge Graph Reasoning

abstract: Temporal Knowledge Graph (TKG), which characterizes temporally evolving facts in the form of (subject, relation, object, timestamp), has attracted much attention recently. TKG reasoning aims to predict future facts based on given historical ones. However, existing TKG reasoning models are unable to abstain from predictions they are uncertain, which will inevitably bring risks in real-world applications. Thus, in this paper, we propose an abstention mechanism for TKG reasoning, which helps the existing models make selective, instead of indiscriminate, predictions. Specifically, we develop a confidence estimator, called Confidence Estimator with History (CEHis), to enable the existing TKG reasoning models to first estimate their confidence in making predictions, and then abstain from those with low confidence. To do so, CEHis takes two kinds of information into consideration, namely, the certainty of the current prediction and the accuracy of historical predictions. Experiments with representative TKG reasoning models on two benchmark datasets demonstrate the effectiveness of the proposed CEHis.


title: Semi-leptonic $W$ decay to $B$ meson with lepton pairs in HQET factorization upto $\mathcal{O}(\alpha_s)$

abstract: Motivated by the study of heavy-light meson production within the framework of heavy quark effective theory (HQET) factorization, we extend the factorization formalism for rather a complicated process $W^+\to B^+\ell^+\ell^-$ in the limit of non-zero invariant squared-mass of dilepton, $q^2$, at the lowest order in $1/m_b$ up to $\mathcal{O}(\alpha_s)$. The purpose of the current study is to extend the HQET factorization formula for the $W^+\to B^+\ell^+\ell^-$ process and subsequently compute the form factors for this channel up to next-to-leading order (NLO) corrections in $\alpha_s$. We explicitly show the amplitude of the $W^+\to B^+\ell^+\ell^-$ process can also be factorized into a convolution between the perturbatively calculable hard-scattering kernel and the non-perturbative yet universal light-cone distribution amplitude (LCDA) defined in HQET. The validity of HQET factorization depends on the assumed scale hierarchy $m_W \sim m_b \gg \Lambda_{\mathrm{QCD}}$. Within the HQET framework, we evaluate the form factors associated with the $W^+ \rightarrow B^+\ell^+\ell^-$ process, providing insights into its phenomenology. In addition, we also perform an exploratory phenomenological study on $W^+ \rightarrow B^+\ell^+\ell^-$ by employing an exponential model for the LCDAs for $B^+$ meson. Our findings reveal that the branching ratio for $W^+ \rightarrow B^+\ell^+\ell^-$ is below $10^{-10}$. Although the branching ratios are small, this channel in high luminosity LHC experiments may serve to further constraints the value of $\lambda_B$.


title: Preventing Model Collapse in Gaussian Process Latent Variable Models

abstract: Gaussian process latent variable models (GPLVMs) are a versatile family of unsupervised learning models commonly used for dimensionality reduction. However, common challenges in modeling data with GPLVMs include inadequate kernel flexibility and improper selection of the projection noise, leading to a type of model collapse characterized by vague latent representations that do not reflect the underlying data structure. This paper addresses these issues by, first, theoretically examining the impact of projection variance on model collapse through the lens of a linear GPLVM. Second, we tackle model collapse due to inadequate kernel flexibility by integrating the spectral mixture (SM) kernel and a differentiable random Fourier feature (RFF) kernel approximation, which ensures computational scalability and efficiency through off-the-shelf automatic differentiation tools for learning the kernel hyperparameters, projection variance, and latent representations within the variational inference framework. The proposed GPLVM, named advisedRFLVM, is evaluated across diverse datasets and consistently outperforms various salient competing models, including state-of-the-art variational autoencoders (VAEs) and other GPLVM variants, in terms of informative latent representations and missing data imputation.


title: Detection of bromochloro alkanes in indoor dust using a novel CP-Seeker data integration tool

abstract: Bromochloro alkanes (BCAs) have been manufactured for use as flame retardants for decades and preliminary environmental risk screening suggests they are likely to behave similarly to polychlorinated alkanes (PCAs), subclasses of which are restricted as Stockholm Convention Persistent Organic Pollutants (POPs). BCAs have rarely been studied in the environment, though some evidence suggests they may migrate from treated-consumer materials into indoor dust, resulting in human exposure via inadvertent ingestion. In this study, BCA-C14 mixture standards were synthesized and used to validate an analytical method. This method relies on chloride-enhanced liquid chromatography-electrospray ionization-Orbitrap-high resolution mass spectrometry (LC-ESI-Orbitrap-HRMS) and a novel CP-Seeker integration software package for homologue detection and integration. Dust sample preparation via ultrasonic extraction, acidified silica clean-up and fractionation on neutral silica cartridges was found to be suitable for BCAs, with absolute recovery of individual homologues averaging 66 to 78% and coefficients of variation $\le$10% in replicated spiking experiments (n=3). In addition, a total of 59 indoor dust samples from six countries including Australia (n=10), Belgium (n=10), Colombia (n=10), Japan (n=10), Thailand (n=10) and the United States of America (n=9) were analysed for BCAs. BCAs were detected in seven samples from the USA, with carbon chain lengths of C8, C10, C12, C14, C16, C18, C24 to C28, C30 and C31 observed overall, though not detected in samples from any other countries. Bromination of detected homologues in the indoor dust samples ranged from Br1-4 as well as Br7, while chlorine numbers ranged from Cl2-11. BCA-C18 were the most frequently detected, observed in each of the USA samples, while the most prevalent halogenation degrees were homologues of Br2 and Cl4-5. Broad estimations of BCA concentrations in the dust samples indicated that levels may approach those of other flame retardants in at least some instances. These findings suggest that development of quantification strategies and further investigation of environmental occurrence and health implications are needed.


title: Task Integration Distillation for Object Detectors

abstract: Knowledge distillation is a widely adopted technique for model lightening. However, the performance of most knowledge distillation methods in the domain of object detection is not satisfactory. Typically, knowledge distillation approaches consider only the classification task among the two sub-tasks of an object detector, largely overlooking the regression task. This oversight leads to a partial understanding of the object detector's comprehensive task, resulting in skewed estimations and potentially adverse effects. Therefore, we propose a knowledge distillation method that addresses both the classification and regression tasks, incorporating a task significance strategy. By evaluating the importance of features based on the output of the detector's two sub-tasks, our approach ensures a balanced consideration of both classification and regression tasks in object detection. Drawing inspiration from real-world teaching processes and the definition of learning condition, we introduce a method that focuses on both key and weak areas. By assessing the value of features for knowledge distillation based on their importance differences, we accurately capture the current model's learning situation. This method effectively prevents the issue of biased predictions about the model's learning reality caused by an incomplete utilization of the detector's outputs.


title: MotionChain: Conversational Motion Controllers via Multimodal Prompts

abstract: Recent advancements in language models have demonstrated their adeptness in conducting multi-turn dialogues and retaining conversational context. However, this proficiency remains largely unexplored in other multimodal generative models, particularly in human motion models. By integrating multi-turn conversations in controlling continuous virtual human movements, generative human motion models can achieve an intuitive and step-by-step process of human task execution for humanoid robotics, game agents, or other embodied systems. In this work, we present MotionChain, a conversational human motion controller to generate continuous and long-term human motion through multimodal prompts. Specifically, MotionChain consists of multi-modal tokenizers that transform various data types such as text, image, and motion, into discrete tokens, coupled with a Vision-Motion-aware Language model. By leveraging large-scale language, vision-language, and vision-motion data to assist motion-related generation tasks, MotionChain thus comprehends each instruction in multi-turn conversation and generates human motions followed by these prompts. Extensive experiments validate the efficacy of MotionChain, demonstrating state-of-the-art performance in conversational motion generation, as well as more intuitive manners of controlling and interacting with virtual humans.


title: On the Role of Summary Content Units in Text Summarization Evaluation

abstract: At the heart of the Pyramid evaluation method for text summarization lie human written summary content units (SCUs). These SCUs are concise sentences that decompose a summary into small facts. Such SCUs can be used to judge the quality of a candidate summary, possibly partially automated via natural language inference (NLI) systems. Interestingly, with the aim to fully automate the Pyramid evaluation, Zhang and Bansal (2021) show that SCUs can be approximated by automatically generated semantic role triplets (STUs). However, several questions currently lack answers, in particular: i) Are there other ways of approximating SCUs that can offer advantages? ii) Under which conditions are SCUs (or their approximations) offering the most value? In this work, we examine two novel strategies to approximate SCUs: generating SCU approximations from AMR meaning representations (SMUs) and from large language models (SGUs), respectively. We find that while STUs and SMUs are competitive, the best approximation quality is achieved by SGUs. We also show through a simple sentence-decomposition baseline (SSUs) that SCUs (and their approximations) offer the most value when ranking short summaries, but may not help as much when ranking systems or longer summaries.


title: Tell and show: Combining multiple modalities to communicate manipulation tasks to a robot

abstract: As human-robot collaboration is becoming more widespread, there is a need for a more natural way of communicating with the robot. This includes combining data from several modalities together with the context of the situation and background knowledge. Current approaches to communication typically rely only on a single modality or are often very rigid and not robust to missing, misaligned, or noisy data. In this paper, we propose a novel method that takes inspiration from sensor fusion approaches to combine uncertain information from multiple modalities and enhance it with situational awareness (e.g., considering object properties or the scene setup). We first evaluate the proposed solution on simulated bimodal datasets (gestures and language) and show by several ablation experiments the importance of various components of the system and its robustness to noisy, missing, or misaligned observations. Then we implement and evaluate the model on the real setup. In human-robot interaction, we must also consider whether the selected action is probable enough to be executed or if we should better query humans for clarification. For these purposes, we enhance our model with adaptive entropy-based thresholding that detects the appropriate thresholds for different types of interaction showing similar performance as fine-tuned fixed thresholds.


title: Boosting Visual Recognition in Real-world Degradations via Unsupervised Feature Enhancement Module with Deep Channel Prior

abstract: The environmental perception of autonomous vehicles in normal conditions have achieved considerable success in the past decade. However, various unfavourable conditions such as fog, low-light, and motion blur will degrade image quality and pose tremendous threats to the safety of autonomous driving. That is, when applied to degraded images, state-of-the-art visual models often suffer performance decline due to the feature content loss and artifact interference caused by statistical and structural properties disruption of captured images. To address this problem, this work proposes a novel Deep Channel Prior (DCP) for degraded visual recognition. Specifically, we observe that, in the deep representation space of pre-trained models, the channel correlations of degraded features with the same degradation type have uniform distribution even if they have different content and semantics, which can facilitate the mapping relationship learning between degraded and clear representations in high-sparsity feature space. Based on this, a novel plug-and-play Unsupervised Feature Enhancement Module (UFEM) is proposed to achieve unsupervised feature correction, where the multi-adversarial mechanism is introduced in the first stage of UFEM to achieve the latent content restoration and artifact removal in high-sparsity feature space. Then, the generated features are transferred to the second stage for global correlation modulation under the guidance of DCP to obtain high-quality and recognition-friendly features. Evaluations of three tasks and eight benchmark datasets demonstrate that our proposed method can comprehensively improve the performance of pre-trained models in real degradation conditions. The source code is available at https://github.com/liyuhang166/Deep_Channel_Prior


title: Application of S-band for Protection in Multi-band Flexible-Grid Optical Networks

abstract: The core network is experiencing bandwidth capacity constraints as internet traffic grows. As a result, the notion of a Multi-band flexible-grid optical network was established to increase the lifespan of an optical core network. In this paper, we use the C+L band for working traffic transmission and the S-band for protection against failure. Furthermore, we compare the proposed method with the existing ones.


title: Samba: Semantic Segmentation of Remotely Sensed Images with State Space Model

abstract: High-resolution remotely sensed images pose a challenge for commonly used semantic segmentation methods such as Convolutional Neural Network (CNN) and Vision Transformer (ViT). CNN-based methods struggle with handling such high-resolution images due to their limited receptive field, while ViT faces challenges in handling long sequences. Inspired by Mamba, which adopts a State Space Model (SSM) to efficiently capture global semantic information, we propose a semantic segmentation framework for high-resolution remotely sensed images, named Samba. Samba utilizes an encoder-decoder architecture, with Samba blocks serving as the encoder for efficient multi-level semantic information extraction, and UperNet functioning as the decoder. We evaluate Samba on the LoveDA, ISPRS Vaihingen, and ISPRS Potsdam datasets, comparing its performance against top-performing CNN and ViT methods. The results reveal that Samba achieved unparalleled performance on commonly used remote sensing datasets for semantic segmentation. Our proposed Samba demonstrates for the first time the effectiveness of SSM in semantic segmentation of remotely sensed images, setting a new benchmark in performance for Mamba-based techniques in this specific application. The source code and baseline implementations are available at https://github.com/zhuqinfeng1999/Samba.


title: Polarity Calibration for Opinion Summarization

abstract: Opinion summarization is automatically generating summaries from a variety of subjective information, such as product reviews or political opinions. The challenge of opinions summarization lies in presenting divergent or even conflicting opinions. We conduct an analysis of previous summarization models, which reveals their inclination to amplify the polarity bias, emphasizing the majority opinions while ignoring the minority opinions. To address this issue and make the summarizer express both sides of opinions, we introduce the concept of polarity calibration, which aims to align the polarity of output summary with that of input text. Specifically, we develop a reinforcement training approach for polarity calibration. This approach feeds the polarity distance between output summary and input text as reward into the summarizer, and also balance polarity calibration with content preservation and language naturality. We evaluate our Polarity Calibration model (PoCa) on two types of opinions summarization tasks: summarizing product reviews and political opinions articles. Automatic and human evaluation demonstrate that our approach can mitigate the polarity mismatch between output summary and input text, as well as maintain the content semantic and language quality.


title: New Bellman induction and a weak version of $\mathrm{BMO}$

abstract: We enlarge the area of applicability of the Bellman function method to estimates in the spirit of the John--Nirenberg inequality abandoning certain convexity assumptions. As an application, we consider a characteristic of a function that is much smaller than the $\mathrm{BMO}$ norm, but whose finiteness leads to the exponential integrability of the function.


title: Equilibrium and Non-Equilibrium Molecular Dynamics Simulation of Thermo-Osmosis: Enhanced Effects on Polarized Graphene Surfaces

abstract: Thermo-osmotic flows, generated by applying a thermal gradient along a liquid-solid interface, could be harnessed to convert waste heat into electricity. While this phenomenon has been known for almost a century, there is a crucial need to gain a better understanding of the molecular origins of thermo-osmosis. In this paper, we start by detailing the multiple contributions to thermo-osmosis. We then showcase three approaches to compute the thermo-osmotic coefficient using molecular dynamics; a first method based on the computation of the interfacial enthalpy excess and Derjaguin's theoretical framework, a second approach based on the computation of the interfacial entropy excess using the so-called dry-surface method, and a novel non-equilibrium method to compute the thermo-osmotic coefficient in a periodic channel. We show that the three methods align with each other, in particular for smooth surfaces. In addition, for a polarized graphene-water interface, we observe large variations of thermo-osmotic responses, and multiple changes in flow direction with increasing surface charge. Overall, this study showcases the versatility of osmotic flows and calls for experimental investigation of thermo-osmotic behavior in the vicinity of charged surfaces.


title: Upsample Guidance: Scale Up Diffusion Models without Training

abstract: Diffusion models have demonstrated superior performance across various generative tasks including images, videos, and audio. However, they encounter difficulties in directly generating high-resolution samples. Previously proposed solutions to this issue involve modifying the architecture, further training, or partitioning the sampling process into multiple stages. These methods have the limitation of not being able to directly utilize pre-trained models as-is, requiring additional work. In this paper, we introduce upsample guidance, a technique that adapts pretrained diffusion model (e.g., $512^2$) to generate higher-resolution images (e.g., $1536^2$) by adding only a single term in the sampling process. Remarkably, this technique does not necessitate any additional training or relying on external models. We demonstrate that upsample guidance can be applied to various models, such as pixel-space, latent space, and video diffusion models. We also observed that the proper selection of guidance scale can improve image quality, fidelity, and prompt alignment.


title: Practical Persistent Multi-Word Compare-and-Swap Algorithms for Many-Core CPUs

abstract: In the last decade, academic and industrial researchers have focused on persistent memory because of the development of the first practical product, Intel Optane. One of the main challenges of persistent memory programming is to guarantee consistent durability over separate memory addresses, and Wang et al. proposed a persistent multi-word compare-and-swap (PMwCAS) algorithm to solve this problem. However, their algorithm contains redundant compare-and-swap (CAS) and cache flush instructions and does not achieve sufficient performance on many-core CPUs. This paper proposes a new algorithm to improve performance on many-core CPUs by removing useless CAS/flush instructions from PMwCAS operations. We also exclude dirty flags, which help ensure consistent durability in the original algorithm, from our algorithm using PMwCAS descriptors as write-ahead logs. Experimental results show that the proposed method is up to ten times faster than the original algorithm and suggests several productive uses of PMwCAS operations.


title: Temporal coherences of atomic chaotic light sources: the Siegert relation and beyond

abstract: Light is characterized by its electric field, yet quantum optics has revealed the importance of monitoring photon-photon correlations at all orders. We here present a comparative study of two experimental setups, composed of cold and warm Rubidium atoms, respectively, which allow us to probe and compare photon correlations up to the fourth order. The former operates in the quantum regime where spontaneous emission dominates, whereas the latter exhibits a temperature-limited coherence time. While both setups present almost-chaotic light statistics, we discuss how the access to different orders of photon correlations allows one to better characterize the mechanisms responsible for deviations from those statistics.


title: Efficient and Generalizable Certified Unlearning: A Hessian-free Recollection Approach

abstract: Machine unlearning strives to uphold the data owners' right to be forgotten by enabling models to selectively forget specific data. Recent advances suggest precomputing and storing statistics extracted from second-order information and implementing unlearning through Newton-style updates. However, the theoretical analysis of these works often depends on restrictive assumptions of convexity and smoothness, and those mentioned operations on Hessian matrix are extremely costly. As a result, applying these works to high-dimensional models becomes challenging. In this paper, we propose an efficient Hessian-free certified unlearning. We propose to maintain a statistical vector for each data, computed through affine stochastic recursion approximation of the difference between retrained and learned models. Our analysis does not involve inverting Hessian and thus can be extended to non-convex non-smooth objectives. Under same assumptions, we demonstrate advancements of proposed method beyond the state-of-the-art theoretical studies, in terms of generalization, unlearning guarantee, deletion capacity, and computation/storage complexity, and we show that the unlearned model of our proposed approach is close to or same as the retrained model. Based on the strategy of recollecting statistics for forgetting data, we develop an algorithm that achieves near-instantaneous unlearning as it only requires a vector addition operation. Experiments demonstrate that the proposed scheme surpasses existing results by orders of magnitude in terms of time/storage costs, while also enhancing accuracy.


title: Generative AI for Immersive Communication: The Next Frontier in Internet-of-Senses Through 6G

abstract: Over the past two decades, the Internet-of-Things (IoT) has been a transformative concept, and as we approach 2030, a new paradigm known as the Internet of Senses (IoS) is emerging. Unlike conventional Virtual Reality (VR), IoS seeks to provide multi-sensory experiences, acknowledging that in our physical reality, our perception extends far beyond just sight and sound; it encompasses a range of senses. This article explores existing technologies driving immersive multi-sensory media, delving into their capabilities and potential applications. This exploration includes a comparative analysis between conventional immersive media streaming and a proposed use case that leverages semantic communication empowered by generative Artificial Intelligence (AI). The focal point of this analysis is the substantial reduction in bandwidth consumption by 99.93% in the proposed scheme. Through this comparison, we aim to underscore the practical applications of generative AI for immersive media while addressing the challenges and outlining future trajectories.


title: Conjugate-Gradient-like Based Adaptive Moment Estimation Optimization Algorithm for Deep Learning

abstract: Training deep neural networks is a challenging task. In order to speed up training and enhance the performance of deep neural networks, we rectify the vanilla conjugate gradient as conjugate-gradient-like and incorporate it into the generic Adam, and thus propose a new optimization algorithm named CG-like-Adam for deep learning. Specifically, both the first-order and the second-order moment estimation of generic Adam are replaced by the conjugate-gradient-like. Convergence analysis handles the cases where the exponential moving average coefficient of the first-order moment estimation is constant and the first-order moment estimation is unbiased. Numerical experiments show the superiority of the proposed algorithm based on the CIFAR10/100 dataset.


title: EMONA: Event-level Moral Opinions in News Articles

abstract: Most previous research on moral frames has focused on social media short texts, little work has explored moral sentiment within news articles. In news articles, authors often express their opinions or political stance through moral judgment towards events, specifically whether the event is right or wrong according to social moral rules. This paper initiates a new task to understand moral opinions towards events in news articles. We have created a new dataset, EMONA, and annotated event-level moral opinions in news articles. This dataset consists of 400 news articles containing over 10k sentences and 45k events, among which 9,613 events received moral foundation labels. Extracting event morality is a challenging task, as moral judgment towards events can be very implicit. Baseline models were built for event moral identification and classification. In addition, we also conduct extrinsic evaluations to integrate event-level moral opinions into three downstream tasks. The statistical analysis and experiments show that moral opinions of events can serve as informative features for identifying ideological bias or subjective events.


title: Effective internal language model training and fusion for factorized transducer model

abstract: The internal language model (ILM) of the neural transducer has been widely studied. In most prior work, it is mainly used for estimating the ILM score and is subsequently subtracted during inference to facilitate improved integration with external language models. Recently, various of factorized transducer models have been proposed, which explicitly embrace a standalone internal language model for non-blank token prediction. However, even with the adoption of factorized transducer models, limited improvement has been observed compared to shallow fusion. In this paper, we propose a novel ILM training and decoding strategy for factorized transducer models, which effectively combines the blank, acoustic and ILM scores. Our experiments show a 17% relative improvement over the standard decoding method when utilizing a well-trained ILM and the proposed decoding strategy on LibriSpeech datasets. Furthermore, when compared to a strong RNN-T baseline enhanced with external LM fusion, the proposed model yields a 5.5% relative improvement on general-sets and an 8.9% WER reduction for rare words. The proposed model can achieve superior performance without relying on external language models, rendering it highly efficient for production use-cases. To further improve the performance, we propose a novel and memory-efficient ILM-fusion-aware minimum word error rate (MWER) training method which improves ILM integration significantly.


title: AddSR: Accelerating Diffusion-based Blind Super-Resolution with Adversarial Diffusion Distillation

abstract: Blind super-resolution methods based on stable diffusion showcase formidable generative capabilities in reconstructing clear high-resolution images with intricate details from low-resolution inputs. However, their practical applicability is often hampered by poor efficiency, stemming from the requirement of thousands or hundreds of sampling steps. Inspired by the efficient adversarial diffusion distillation (ADD), we design~\name~to address this issue by incorporating the ideas of both distillation and ControlNet. Specifically, we first propose a prediction-based self-refinement strategy to provide high-frequency information in the student model output with marginal additional time cost. Furthermore, we refine the training process by employing HR images, rather than LR images, to regulate the teacher model, providing a more robust constraint for distillation. Second, we introduce a timestep-adaptive ADD to address the perception-distortion imbalance problem introduced by original ADD. Extensive experiments demonstrate our~\name~generates better restoration results, while achieving faster speed than previous SD-based state-of-the-art models (e.g., $7$$\times$ faster than SeeSR).


title: Optical Spectra of Plasmon-Exciton Core-Shell Nanoparticles: An Anisotropic Classical Model Eliminates Discrepancies with Experiments

abstract: The optical properties of the hybrid core--shell nanostructures composed of a metallic core and an organic shell of molecular J-aggregates are determined by the electromagnetic coupling between plasmons localized at the surface of the metallic core and Frenkel excitons in the shell. In cases of strong and ultra-strong plasmon--exciton coupling, the use of the traditional isotropic classical oscillator model to describe the J-aggregate permittivity may lead to drastic discrepancies between theoretical predictions and the available experimental spectra of hybrid nanoparticles. We show that these discrepancies are not caused by limitations of the classical oscillator model itself, but by considering the organic shell as an optically isotropic material. By assuming a tangential orientation of the classical oscillators of the molecular J-aggregates in a shell, we obtain excellent agreement with experimental extinction spectra of TDBC-coated gold nanorods, which cannot be treated with the conventional isotropic shell model. Our results extend the understanding of the physical effects in the optics of metal--organic nanoparticles and suggest an approach for the theoretical description of such hybrid systems.


title: From the Albert algebra to Kac's ten-dimensional Jordan superalgebra via tensor categories in characteristic 5

abstract: Kac's ten-dimensional simple Jordan superalgebra over a field of characteristic 5 is obtained from a process of semisimplification, via tensor categories, from the exceptional simple Jordan algebra (or Albert algebra), together with a suitable order 5 automorphism. This explains McCrimmon's 'bizarre result' asserting that, in characteristic 5, Kac's superalgebra is a sort of 'degree 3 Jordan superalgebra'. As an outcome, the exceptional simple Lie superalgebra el(5;5), specific of characteristic 5, is obtained from the simple Lie algebra of type $E_8$ and an order 5 automorphism. In the process, precise recipes to obtain superalgebras from algebras in the category of representations of the cyclic group $C_p$, over a field of characteristic $p>2$, are given.


title: Self-Improvement Programming for Temporal Knowledge Graph Question Answering

abstract: Temporal Knowledge Graph Question Answering (TKGQA) aims to answer questions with temporal intent over Temporal Knowledge Graphs (TKGs). The core challenge of this task lies in understanding the complex semantic information regarding multiple types of time constraints (e.g., before, first) in questions. Existing end-to-end methods implicitly model the time constraints by learning time-aware embeddings of questions and candidate answers, which is far from understanding the question comprehensively. Motivated by semantic-parsing-based approaches that explicitly model constraints in questions by generating logical forms with symbolic operators, we design fundamental temporal operators for time constraints and introduce a novel self-improvement Programming method for TKGQA (Prog-TQA). Specifically, Prog-TQA leverages the in-context learning ability of Large Language Models (LLMs) to understand the combinatory time constraints in the questions and generate corresponding program drafts with a few examples given. Then, it aligns these drafts to TKGs with the linking module and subsequently executes them to generate the answers. To enhance the ability to understand questions, Prog-TQA is further equipped with a self-improvement strategy to effectively bootstrap LLMs using high-quality self-generated drafts. Extensive experiments demonstrate the superiority of the proposed Prog-TQA on MultiTQ and CronQuestions datasets, especially in the Hits@1 metric.


title: Dynamics on Markov surfaces: classification of stationary measures

abstract: Consider the four punctured sphere ${\mathbb{S}}_4^2$. Each choice of four traces, one for each puncture, determines a relative character variety for the representations of the fundamental group of ${\mathbb{S}}_4^2$ in ${\sf{SL}}_2(\mathbb{C})$. We classify the stationary probability measures for the action of the mapping class group ${\sf{Mod}}({\mathbb{S}}_4^2)$ on these character varieties.


title: Sentence-level Media Bias Analysis with Event Relation Graph

abstract: Media outlets are becoming more partisan and polarized nowadays. In this paper, we identify media bias at the sentence level, and pinpoint bias sentences that intend to sway readers' opinions. As bias sentences are often expressed in a neutral and factual way, considering broader context outside a sentence can help reveal the bias. In particular, we observe that events in a bias sentence need to be understood in associations with other events in the document. Therefore, we propose to construct an event relation graph to explicitly reason about event-event relations for sentence-level bias identification. The designed event relation graph consists of events as nodes and four common types of event relations: coreference, temporal, causal, and subevent relations. Then, we incorporate event relation graph for bias sentences identification in two steps: an event-aware language model is built to inject the events and event relations knowledge into the basic language model via soft labels; further, a relation-aware graph attention network is designed to update sentence embedding with events and event relations information based on hard labels. Experiments on two benchmark datasets demonstrate that our approach with the aid of event relation graph improves both precision and recall of bias sentence identification.


title: Contextual Embedding Learning to Enhance 2D Networks for Volumetric Image Segmentation

abstract: The segmentation of organs in volumetric medical images plays an important role in computer-aided diagnosis and treatment/surgery planning. Conventional 2D convolutional neural networks (CNNs) can hardly exploit the spatial correlation of volumetric data. Current 3D CNNs have the advantage to extract more powerful volumetric representations but they usually suffer from occupying excessive memory and computation nevertheless. In this study we aim to enhance the 2D networks with contextual information for better volumetric image segmentation. Accordingly, we propose a contextual embedding learning approach to facilitate 2D CNNs capturing spatial information properly. Our approach leverages the learned embedding and the slice-wisely neighboring matching as a soft cue to guide the network. In such a way, the contextual information can be transferred slice-by-slice thus boosting the volumetric representation of the network. Experiments on challenging prostate MRI dataset (PROMISE12) and abdominal CT dataset (CHAOS) show that our contextual embedding learning can effectively leverage the inter-slice context and improve segmentation performance. The proposed approach is a plug-and-play, and memory-efficient solution to enhance the 2D networks for volumetric segmentation. Our code is publicly available at https://github.com/JuliusWang-7/CE_Block.


title: Global existence and boundedness of solutions to a fully parabolic chemotaxis system with indirect signal production in $\mathbb{R}^4$

abstract: Global existence and boundedness of solutions to the Cauchy problem for the four dimensional fully parabolic chemotaxis system with indirect signal production are studied. We prove that solutions with initial mass below $(8\pi)^2$ exist globally in time. This value $(8\pi)^2$ is known as the four dimensional threshold value of the initial mass determining whether blow-up of solutions occurs or not. Furthermore, some condition on the initial mass guaranteeing that the solution remains uniformly bounded is also obtained.


title: Disentangled Pre-training for Human-Object Interaction Detection

abstract: Detecting human-object interaction (HOI) has long been limited by the amount of supervised data available. Recent approaches address this issue by pre-training according to pseudo-labels, which align object regions with HOI triplets parsed from image captions. However, pseudo-labeling is tricky and noisy, making HOI pre-training a complex process. Therefore, we propose an efficient disentangled pre-training method for HOI detection (DP-HOI) to address this problem. First, DP-HOI utilizes object detection and action recognition datasets to pre-train the detection and interaction decoder layers, respectively. Then, we arrange these decoder layers so that the pre-training architecture is consistent with the downstream HOI detection task. This facilitates efficient knowledge transfer. Specifically, the detection decoder identifies reliable human instances in each action recognition dataset image, generates one corresponding query, and feeds it into the interaction decoder for verb classification. Next, we combine the human instance verb predictions in the same image and impose image-level supervision. The DP-HOI structure can be easily adapted to the HOI detection task, enabling effective model parameter initialization. Therefore, it significantly enhances the performance of existing HOI detection models on a broad range of rare categories. The code and pre-trained weight are available at https://github.com/xingaoli/DP-HOI.


title: A Stability-Based Abstraction Framework for Reach-Avoid Control of Stochastic Dynamical Systems with Unknown Noise Distributions

abstract: Finite-state abstractions are widely studied for the automated synthesis of correct-by-construction controllers for stochastic dynamical systems. However, existing abstraction methods often lead to prohibitively large finite-state models. To address this issue, we propose a novel abstraction scheme for stochastic linear systems that exploits the system's stability to obtain significantly smaller abstract models. As a unique feature, we first stabilize the open-loop dynamics using a linear feedback gain. We then use a model-based approach to abstract a known part of the stabilized dynamics while using a data-driven method to account for the stochastic uncertainty. We formalize abstractions as Markov decision processes (MDPs) with intervals of transition probabilities. By stabilizing the dynamics, we can further constrain the control input modeled in the abstraction, which leads to smaller abstract models while retaining the correctness of controllers. Moreover, when the stabilizing feedback controller is aligned with the property of interest, then a good trade-off is achieved between the reduction in the abstraction size and the performance loss. The experiments show that our approach can reduce the size of the graph of abstractions by up to 90% with negligible performance loss.


title: Generalizing 6-DoF Grasp Detection via Domain Prior Knowledge

abstract: We focus on the generalization ability of the 6-DoF grasp detection method in this paper. While learning-based grasp detection methods can predict grasp poses for unseen objects using the grasp distribution learned from the training set, they often exhibit a significant performance drop when encountering objects with diverse shapes and structures. To enhance the grasp detection methods' generalization ability, we incorporate domain prior knowledge of robotic grasping, enabling better adaptation to objects with significant shape and structure differences. More specifically, we employ the physical constraint regularization during the training phase to guide the model towards predicting grasps that comply with the physical rule on grasping. For the unstable grasp poses predicted on novel objects, we design a contact-score joint optimization using the projection contact map to refine these poses in cluttered scenarios. Extensive experiments conducted on the GraspNet-1billion benchmark demonstrate a substantial performance gain on the novel object set and the real-world grasping experiments also demonstrate the effectiveness of our generalizing 6-DoF grasp detection method.


title: Formalizing Pfaffian in bounded arithmetic

abstract: We formalize algorithms computing Pfaffian in the theory of bounded arithmetic for sharpL which is based on Berkowitz algorithm for the determinant. We also prove relations among Pfaffian properties. Furthermore, we give an algorithm for Pfaffian pairs as well.


title: Photodriven Mott insulating heterostructures: A steady-state study of impact ionization processes

abstract: We investigate the photocurrent and spectral features in a simplified model of a Mott photovoltaic system consisting of a multilayered insulating heterostructure. The central correlated region is coupled to two metallic leads kept at different chemical potentials. A periodic drive applied to the correlated region produces excited doublons and holons across the Mott gap which are then separated by a potential gradient, which mimics the polarization-induced electric field present in oxyde heterostructures. The nonequilibrium Floquet steady-state is addressed by means of dynamical mean-field theory and its Floquet extension, while the so-called auxiliary master equation approach is employed as impurity solver. We find that impact ionization, identified by a kink in the photocurrent as function of the driving frequency, becomes significant and is generally favoured by weak, narrow-band hybridizations to the leads beyond a certain strength of the driving field. On the other hand, in the case of a direct coupling to metallic leads with a flat band, we observe a drastic reduction of impact ionization and of the photocurrent itself.


title: Asymptotics of Language Model Alignment

abstract: Let $p$ denote a generative language model. Let $r$ denote a reward model that returns a scalar that captures the degree at which a draw from $p$ is preferred. The goal of language model alignment is to alter $p$ to a new distribution $\phi$ that results in a higher expected reward while keeping $\phi$ close to $p.$ A popular alignment method is the KL-constrained reinforcement learning (RL), which chooses a distribution $\phi_\Delta$ that maximizes $E_{\phi_{\Delta}} r(y)$ subject to a relative entropy constraint $KL(\phi_\Delta || p) \leq \Delta.$ Another simple alignment method is best-of-$N$, where $N$ samples are drawn from $p$ and one with highest reward is selected. In this paper, we offer a closed-form characterization of the optimal KL-constrained RL solution. We demonstrate that any alignment method that achieves a comparable trade-off between KL divergence and reward must approximate the optimal KL-constrained RL solution in terms of relative entropy. To further analyze the properties of alignment methods, we introduce two simplifying assumptions: we let the language model be memoryless, and the reward model be linear. Although these assumptions may not reflect complex real-world scenarios, they enable a precise characterization of the asymptotic behavior of both the best-of-$N$ alignment, and the KL-constrained RL method, in terms of information-theoretic quantities. We prove that the reward of the optimal KL-constrained RL solution satisfies a large deviation principle, and we fully characterize its rate function. We also show that the rate of growth of the scaled cumulants of the reward is characterized by a proper Renyi cross entropy. Finally, we show that best-of-$N$ is asymptotically equivalent to KL-constrained RL solution by proving that their expected rewards are asymptotically equal, and concluding that the two distributions must be close in KL divergence.


title: Compact Binary Formation in Open Star Clusters II: Difficulty of Gaia NS formation in low-mass star clusters

abstract: Gaia mission offers opportunities to search for compact binaries not involved in binary interactions (hereafter inert compact binaries), and results in the discoveries of binaries containing one black hole (BH) or one neutron star (NS), called "Gaia BHs" and "Gaia NSs", respectively. We have assessed if Gaia BHs and NSs can be formed in open clusters through dynamical interactions. In order to obtain a large number of inert compact binaries similar to Gaia BHs and NSs, we have performed gravitational $N$-body simulations for a large number of open clusters whose total mass is $1.2 \times 10^8 M_\odot$. These clusters have various masses, metallicities, densities, and binary fractions. We have found that open clusters form Gaia BHs ($10^{-6}$-$10^{-5} M_\odot^{-1}$) much more efficiently than Gaia NSs ($\lesssim 10^{-7} M_\odot^{-1}$) for any cluster parameters. This is quite inconsistent with observational results, because the reported numbers of Gaia BHs and NSs are $3$ and $21$, respectively. Additionally, we have switched off NS natal kicks for $10^4$ open clusters each weighing $10^3 M_\odot$ in order to retain a large number of NSs in open clusters. Then, open clusters form inert NS binaries originating from primordial binaries rather than formed through dynamical interactions. This means that Gaia NSs are formed dominantly on isolated fields, not in open clusters, if there is no NS natal kick. We have concluded that Gaia BHs can be dominantly formed in open clusters, however Gaia NSs cannot.


title: A simple collocation-type approach to numerical stochastic homogenization

abstract: This paper proposes a novel collocation-type numerical stochastic homogenization method for prototypical stochastic homogenization problems with random coefficient fields of small correlation lengths. The presented method is based on a recently introduced localization technique that enforces a super-exponential decay of the basis functions relative to the underlying coarse mesh, resulting in considerable computational savings during the sampling phase. More generally, the collocation-type structure offers a particularly simple and computationally efficient construction in the stochastic setting with minimized communication between the patches where the basis functions of the method are computed. An error analysis that bridges numerical homogenization and the quantitative theory of stochastic homogenization is performed. In a series of numerical experiments, we study the effect of the correlation length and the discretization parameters on the approximation quality of the method.


title: The ZOO of combinatorial Banach spaces

abstract: We study Banach spaces induced by families of finite sets in the most natural (Schreier-like) way, that is, we consider the completion $X_\mc{F}$ of $c_{00}$ with respect to the norm $\sup\{\sum_{k\in F}|x(k)|:F\in\mc{F}\}$ where $\mc{F}$ is an arbitrary (not necessarily compact) family of finite sets covering $\mbb{N}$. Among other results, we discuss the following: (1) Structure theorems bonding the combinatorics of $\mc{F}$ and the geometry of $X_\mc{F}$ including possible characterizations and variants of the Schur property, $\ell_1$-saturation, and the lack of copies of $c_0$ in $X_\mc{F}$. (2) A plethora of examples including a relatively simple $\ell_1$-saturated combinatorial space which does not satisfy the Schur property, as well as a new presentation of Pe\l czy\'nski's universal space. (3) The complexity of the family $\{H\subseteq\NN:X_{\mc{F}\upharpoonright H}$ does not contain $c_0\}$.


title: Expansion of net correlations in terms of partial correlations

abstract: Graphical models are usually employed to represent statistical relationships between pairs of variables when all the remaining variables are fixed. In this picture, conditionally independent pairs are disconnected. In the real world, however, strict conditional independence is almost impossible to prove. Here we use a weaker version of the concept of graphical models, in which only the linear component of the conditional dependencies is represented. This notion enables us to relate the marginal Pearson correlation coefficient (a measure of linear marginal dependence) with the partial correlations (a measure of linear conditional dependence). Specifically, we use the graphical model to express the marginal Pearson correlation $\rho_{ij}$ between variables $X_i$ and $X_j$ as a sum of the efficacies with which messages propagate along all the paths connecting the variables in the graph. The expansion is convergent, and provides a mechanistic interpretation of how global correlations arise from local interactions. Moreover, by weighing the relevance of each path and of each intermediate node, an intuitive way to imagine interventions is enabled, revealing for example what happens when a given edge is pruned, or the weight of an edge is modified. The expansion is also useful to construct minimal equivalent models, in which latent variables are introduced to replace a larger number of marginalised variables. In addition, the expansion yields an alternative algorithm to calculate marginal Pearson correlations, particularly beneficial when partial correlation matrix inversion is difficult. Finally, for Gaussian variables, the mutual information is also related to message-passing efficacies along paths in the graph.

categories: stat.ME cond-mat.stat-mech math-ph math.MP math.ST physics.data-an stat.TH


title: CIRP: Cross-Item Relational Pre-training for Multimodal Product Bundling

abstract: Product bundling has been a prevailing marketing strategy that is beneficial in the online shopping scenario. Effective product bundling methods depend on high-quality item representations, which need to capture both the individual items' semantics and cross-item relations. However, previous item representation learning methods, either feature fusion or graph learning, suffer from inadequate cross-modal alignment and struggle to capture the cross-item relations for cold-start items. Multimodal pre-train models could be the potential solutions given their promising performance on various multimodal downstream tasks. However, the cross-item relations have been under-explored in the current multimodal pre-train models. To bridge this gap, we propose a novel and simple framework Cross-Item Relational Pre-training (CIRP) for item representation learning in product bundling. Specifically, we employ a multimodal encoder to generate image and text representations. Then we leverage both the cross-item contrastive loss (CIC) and individual item's image-text contrastive loss (ITC) as the pre-train objectives. Our method seeks to integrate cross-item relation modeling capability into the multimodal encoder, while preserving the in-depth aligned multimodal semantics. Therefore, even for cold-start items that have no relations, their representations are still relation-aware. Furthermore, to eliminate the potential noise and reduce the computational cost, we harness a relation pruning module to remove the noisy and redundant relations. We apply the item representations extracted by CIRP to the product bundling model ItemKNN, and experiments on three e-commerce datasets demonstrate that CIRP outperforms various leading representation learning methods.


title: Nonparametric efficient causal estimation of the intervention-specific expected number of recurrent events with continuous-time targeted maximum likelihood and highly adaptive lasso estimation

abstract: Longitudinal settings involving outcome, competing risks and censoring events occurring and recurring in continuous time are common in medical research, but are often analyzed with methods that do not allow for taking post-baseline information into account. In this work, we define statistical and causal target parameters via the g-computation formula by carrying out interventions directly on the product integral representing the observed data distribution in a continuous-time counting process model framework. In recurrent events settings our target parameter identifies the expected number of recurrent events also in settings where the censoring mechanism or post-baseline treatment decisions depend on past information of post-baseline covariates such as the recurrent event process. We propose a flexible estimation procedure based on targeted maximum likelihood estimation coupled with highly adaptive lasso estimation to provide a novel approach for double robust and nonparametric inference for the considered target parameter. We illustrate the methods in a simulation study.


title: Transfer Learning from Whisper for Microscopic Intelligibility Prediction

abstract: Macroscopic intelligibility models predict the expected human word-error-rate for a given speech-in-noise stimulus. In contrast, microscopic intelligibility models aim to make fine-grained predictions about listeners' perception, e.g. predicting phonetic or lexical responses. State-of-the-art macroscopic models use transfer learning from large scale deep learning models for speech processing, whereas such methods have rarely been used for microscopic modeling. In this paper, we study the use of transfer learning from Whisper, a state-of-the-art deep learning model for automatic speech recognition, for microscopic intelligibility prediction at the level of lexical responses. Our method outperforms the considered baselines, even in a zero-shot setup, and yields a relative improvement of up to 66\% when fine-tuned to predict listeners' responses. Our results showcase the promise of large scale deep learning based methods for microscopic intelligibility prediction.


title: A Posteriori Single- and Multi-Goal Error Control and Adaptivity for Partial Differential Equations

abstract: This work reviews goal-oriented a posteriori error control, adaptivity and solver control for finite element approximations to boundary and initial-boundary value problems for stationary and non-stationary partial differential equations, respectively. In particular, coupled field problems with different physics may require simultaneously the accurate evaluation of several quantities of interest, which is achieved with multi-goal oriented error control. Sensitivity measures are obtained by solving an adjoint problem. Error localization is achieved with the help of a partition-of-unity. We also review and extend theoretical results for efficiency and reliability by employing a saturation assumption. The resulting adaptive algorithms allow to balance discretization and non-linear iteration errors, and are demonstrated for four applications: Poisson's problem, non-linear elliptic boundary value problems, stationary incompressible Navier-Stokes equations, and regularized parabolic $p$-Laplace initial-boundary value problems. Therein, different finite element discretizations in two different software libraries are utilized, which are partially accompanied with open-source implementations on GitHub.


title: Real-space Calculation of Orbital Hall Responses in Disordered Materials

abstract: We developed an efficient numerical approach to compute the different components of the orbital Hall responses in disordered materials from the Berry phase theory of magnetization. We propose a theoretical framework based on the Chebyshev expansion of Green's functions and the position operator for systems under arbitrary boundary conditions. The capability of this scheme is illustrated by computing the orbital Hall conductivity for gapped graphene and Haldane model in the presence of nonperturbative disorder effects. This methodology opens the door to realistic simulations of orbital Hall responses in arbitrary complex models of disordered materials.


title: Weakly-supervised Audio Separation via Bi-modal Semantic Similarity

abstract: Conditional sound separation in multi-source audio mixtures without having access to single source sound data during training is a long standing challenge. Existing mix-and-separate based methods suffer from significant performance drop with multi-source training mixtures due to the lack of supervision signal for single source separation cases during training. However, in the case of language-conditional audio separation, we do have access to corresponding text descriptions for each audio mixture in our training data, which can be seen as (rough) representations of the audio samples in the language modality. To this end, in this paper, we propose a generic bi-modal separation framework which can enhance the existing unsupervised frameworks to separate single-source signals in a target modality (i.e., audio) using the easily separable corresponding signals in the conditioning modality (i.e., language), without having access to single-source samples in the target modality during training. We empirically show that this is well within reach if we have access to a pretrained joint embedding model between the two modalities (i.e., CLAP). Furthermore, we propose to incorporate our framework into two fundamental scenarios to enhance separation performance. First, we show that our proposed methodology significantly improves the performance of purely unsupervised baselines by reducing the distribution shift between training and test samples. In particular, we show that our framework can achieve 71% boost in terms of Signal-to-Distortion Ratio (SDR) over the baseline, reaching 97.5% of the supervised learning performance. Second, we show that we can further improve the performance of the supervised learning itself by 17% if we augment it by our proposed weakly-supervised framework, that enables a powerful semi-supervised framework for audio separation.


title: Intrusion Tolerance for Networked Systems through Two-Level Feedback Control

abstract: We formulate intrusion tolerance for a system with service replicas as a two-level optimal control problem. On the local level node controllers perform intrusion recovery, and on the global level a system controller manages the replication factor. The local and global control problems can be formulated as classical problems in operations research, namely, the machine replacement problem and the inventory replenishment problem. Based on this formulation, we design TOLERANCE, a novel control architecture for intrusion-tolerant systems. We prove that the optimal control strategies on both levels have threshold structure and design efficient algorithms for computing them. We implement and evaluate TOLERANCE in an emulation environment where we run 10 types of network intrusions. The results show that TOLERANCE can improve service availability and reduce operational cost compared with state-of-the-art intrusion-tolerant systems.


title: Cosmological study of a symmetric teleparallel gravity model

abstract: We study a symmetric teleparallel gravity with a Lagrangian of logarithmic form. The full model leads to an accelerated universe and for specific values of the free parameters the Hubble rate reduces to the well-known Dvali-Gabadadze-Porrati model, though the evolution of the gravitational potentials are different. We consider different branches of the logarithmic model, among which are self-accelerated branch and normal branch. The phenomenology of both the background and linear perturbations is discussed, including all the relevant effects on cosmic microwave background radiation (CMB) angular power spectrum, lensing and matter power spectra. To this purpose, we modified the Einstein-Boltzmann code mgcamb. Finally, we derive bounds on the free parameters which are in agreement with early dark energy constraint from CMB and big bang nucleosynthesis constraint on the helium abundance.


title: Atom-Level Optical Chemical Structure Recognition with Limited Supervision

abstract: Identifying the chemical structure from a graphical representation, or image, of a molecule is a challenging pattern recognition task that would greatly benefit drug development. Yet, existing methods for chemical structure recognition do not typically generalize well, and show diminished effectiveness when confronted with domains where data is sparse, or costly to generate, such as hand-drawn molecule images. To address this limitation, we propose a new chemical structure recognition tool that delivers state-of-the-art performance and can adapt to new domains with a limited number of data samples and supervision. Unlike previous approaches, our method provides atom-level localization, and can therefore segment the image into the different atoms and bonds. Our model is the first model to perform OCSR with atom-level entity detection with only SMILES supervision. Through rigorous and extensive benchmarking, we demonstrate the preeminence of our chemical structure recognition approach in terms of data efficiency, accuracy, and atom-level entity prediction.


title: Octopus v2: On-device language model for super agent

abstract: Language models have shown effectiveness in a variety of software applications, particularly in tasks related to automatic workflow. These models possess the crucial ability to call functions, which is essential in creating AI agents. Despite the high performance of large-scale language models in cloud environments, they are often associated with concerns over privacy and cost. Current on-device models for function calling face issues with latency and accuracy. Our research presents a new method that empowers an on-device model with 2 billion parameters to surpass the performance of GPT-4 in both accuracy and latency, and decrease the context length by 95\%. When compared to Llama-7B with a RAG-based function calling mechanism, our method enhances latency by 35-fold. This method reduces the latency to levels deemed suitable for deployment across a variety of edge devices in production environments, aligning with the performance requisites for real-world applications.


title: Unleash the Potential of CLIP for Video Highlight Detection

abstract: Multimodal and large language models (LLMs) have revolutionized the utilization of open-world knowledge, unlocking novel potentials across various tasks and applications. Among these domains, the video domain has notably benefited from their capabilities. In this paper, we present Highlight-CLIP (HL-CLIP), a method designed to excel in the video highlight detection task by leveraging the pre-trained knowledge embedded in multimodal models. By simply fine-tuning the multimodal encoder in combination with our innovative saliency pooling technique, we have achieved the state-of-the-art performance in the highlight detection task, the QVHighlight Benchmark, to the best of our knowledge.


title: Towards Scalable & Efficient Interaction-Aware Planning in Autonomous Vehicles using Knowledge Distillation

abstract: Real-world driving involves intricate interactions among vehicles navigating through dense traffic scenarios. Recent research focuses on enhancing the interaction awareness of autonomous vehicles to leverage these interactions in decision-making. These interaction-aware planners rely on neural-network-based prediction models to capture inter-vehicle interactions, aiming to integrate these predictions with traditional control techniques such as Model Predictive Control. However, this integration of deep learning-based models with traditional control paradigms often results in computationally demanding optimization problems, relying on heuristic methods. This study introduces a principled and efficient method for combining deep learning with constrained optimization, employing knowledge distillation to train smaller and more efficient networks, thereby mitigating complexity. We demonstrate that these refined networks maintain the problem-solving efficacy of larger models while significantly accelerating optimization. Specifically, in the domain of interaction-aware trajectory planning for autonomous vehicles, we illustrate that training a smaller prediction network using knowledge distillation speeds up optimization without sacrificing accuracy.


title: Improving the accuracy and consistency of the energy quadratization method with an energy-optimized technique

abstract: We propose an energy-optimized invariant energy quadratization method to solve the gradient flow models in this paper, which requires only one linear energy-optimized step to correct the auxiliary variables on each time step. In addition to inheriting the benefits of the baseline and relaxed invariant energy quadratization method, our approach has several other advantages. Firstly, in the process of correcting auxiliary variables, we can directly solve linear programming problem by the energy-optimized technique, which greatly simplifies the nonlinear optimization problem in the previous relaxed invariant energy quadratization method. Secondly, we construct new linear unconditionally energy stable schemes by applying backward Euler formulas and Crank-Nicolson formula, so that the accuracy in time can reach the first- and second-order. Thirdly, comparing with relaxation technique, the modified energy obtained by energy-optimized technique is closer to the original energy, meanwhile the accuracy and consistency of the numerical solutions can be improved. Ample numerical examples have been presented to demonstrate the accuracy, efficiency and energy stability of the proposed schemes.


title: Global Mapping of Exposure and Physical Vulnerability Dynamics in Least Developed Countries using Remote Sensing and Machine Learning

abstract: As the world marked the midterm of the Sendai Framework for Disaster Risk Reduction 2015-2030, many countries are still struggling to monitor their climate and disaster risk because of the expensive large-scale survey of the distribution of exposure and physical vulnerability and, hence, are not on track in reducing risks amidst the intensifying effects of climate change. We present an ongoing effort in mapping this vital information using machine learning and time-series remote sensing from publicly available Sentinel-1 SAR GRD and Sentinel-2 Harmonized MSI. We introduce the development of "OpenSendaiBench" consisting of 47 countries wherein most are least developed (LDCs), trained ResNet-50 deep learning models, and demonstrated the region of Dhaka, Bangladesh by mapping the distribution of its informal constructions. As a pioneering effort in auditing global disaster risk over time, this paper aims to advance the area of large-scale risk quantification in informing our collective long-term efforts in reducing climate and disaster risk.


title: Curvature conditions, Liouville-type theorems and Harnack inequalities for a nonlinear parabolic equation on smooth metric measure spaces

abstract: In this paper we prove gradient estimates of both elliptic and parabolic types, specifically, of Souplet-Zhang, Hamilton and Li-Yau types for positive smooth solutions to a class of nonlinear parabolic equations involving the Witten or drifting Laplacian on smooth metric measure spaces. These estimates are established under various curvature conditions and lower bounds on the generalised Bakry-\'Emery Ricci tensor and find utility in proving elliptic and parabolic Harnack-type inequalities as well as general elliptic and parabolic Liouville-type and other global constancy results. Several applications and consequences are presented and discussed.


title: Exploring Latent Pathways: Enhancing the Interpretability of Autonomous Driving with a Variational Autoencoder

abstract: Autonomous driving presents a complex challenge, which is usually addressed with artificial intelligence models that are end-to-end or modular in nature. Within the landscape of modular approaches, a bio-inspired neural circuit policy model has emerged as an innovative control module, offering a compact and inherently interpretable system to infer a steering wheel command from abstract visual features. Here, we take a leap forward by integrating a variational autoencoder with the neural circuit policy controller, forming a solution that directly generates steering commands from input camera images. By substituting the traditional convolutional neural network approach to feature extraction with a variational autoencoder, we enhance the system's interpretability, enabling a more transparent and understandable decision-making process. In addition to the architectural shift toward a variational autoencoder, this study introduces the automatic latent perturbation tool, a novel contribution designed to probe and elucidate the latent features within the variational autoencoder. The automatic latent perturbation tool automates the interpretability process, offering granular insights into how specific latent variables influence the overall model's behavior. Through a series of numerical experiments, we demonstrate the interpretative power of the variational autoencoder-neural circuit policy model and the utility of the automatic latent perturbation tool in making the inner workings of autonomous driving systems more transparent.


title: T-VSL: Text-Guided Visual Sound Source Localization in Mixtures

abstract: Visual sound source localization poses a significant challenge in identifying the semantic region of each sounding source within a video. Existing self-supervised and weakly supervised source localization methods struggle to accurately distinguish the semantic regions of each sounding object, particularly in multi-source mixtures. These methods often rely on audio-visual correspondence as guidance, which can lead to substantial performance drops in complex multi-source localization scenarios. The lack of access to individual source sounds in multi-source mixtures during training exacerbates the difficulty of learning effective audio-visual correspondence for localization. To address this limitation, in this paper, we propose incorporating the text modality as an intermediate feature guide using tri-modal joint embedding models (e.g., AudioCLIP) to disentangle the semantic audio-visual source correspondence in multi-source mixtures. Our framework, dubbed T-VSL, begins by predicting the class of sounding entities in mixtures. Subsequently, the textual representation of each sounding source is employed as guidance to disentangle fine-grained audio-visual source correspondence from multi-source mixtures, leveraging the tri-modal AudioCLIP embedding. This approach enables our framework to handle a flexible number of sources and exhibits promising zero-shot transferability to unseen classes during test time. Extensive experiments conducted on the MUSIC, VGGSound, and VGGSound-Instruments datasets demonstrate significant performance improvements over state-of-the-art methods.


title: Safe Interval RRT* for Scalable Multi-Robot Path Planning in Continuous Space

abstract: In this paper, we consider the problem of Multi-Robot Path Planning (MRPP) in continuous space to find conflict-free paths. The difficulty of the problem arises from two primary factors. First, the involvement of multiple robots leads to combinatorial decision-making, which escalates the search space exponentially. Second, the continuous space presents potentially infinite states and actions. For this problem, we propose a two-level approach where the low level is a sampling-based planner Safe Interval RRT* (SI-RRT*) that finds a collision-free trajectory for individual robots. The high level can use any method that can resolve inter-robot conflicts where we employ two representative methods that are Prioritized Planning (SI-CPP) and Conflict Based Search (SI-CCBS). Experimental results show that SI-RRT* can find a high-quality solution quickly with a small number of samples. SI-CPP exhibits improved scalability while SI-CCBS produces higher-quality solutions compared to the state-of-the-art planners for continuous space. Compared to the most scalable existing algorithm, SI-CPP achieves a success rate that is up to 94% higher with 100 robots while maintaining solution quality (i.e., flowtime, the sum of travel times of all robots) without significant compromise. SI-CPP also decreases the makespan up to 45%. SI-CCBS decreases the flowtime by 9% compared to the competitor, albeit exhibiting a 14% lower success rate.


title: M2SA: Multimodal and Multilingual Model for Sentiment Analysis of Tweets

abstract: In recent years, multimodal natural language processing, aimed at learning from diverse data types, has garnered significant attention. However, there needs to be more clarity when it comes to analysing multimodal tasks in multi-lingual contexts. While prior studies on sentiment analysis of tweets have predominantly focused on the English language, this paper addresses this gap by transforming an existing textual Twitter sentiment dataset into a multimodal format through a straightforward curation process. Our work opens up new avenues for sentiment-related research within the research community. Additionally, we conduct baseline experiments utilising this augmented dataset and report the findings. Notably, our evaluations reveal that when comparing unimodal and multimodal configurations, using a sentiment-tuned large language model as a text encoder performs exceptionally well.


title: Peer-aided Repairer: Empowering Large Language Models to Repair Advanced Student Assignments

abstract: Automated generation of feedback on programming assignments holds significant benefits for programming education, especially when it comes to advanced assignments. Automated Program Repair techniques, especially Large Language Model based approaches, have gained notable recognition for their potential to fix introductory assignments. However, the programs used for evaluation are relatively simple. It remains unclear how existing approaches perform in repairing programs from higher-level programming courses. To address these limitations, we curate a new advanced student assignment dataset named Defects4DS from a higher-level programming course. Subsequently, we identify the challenges related to fixing bugs in advanced assignments. Based on the analysis, we develop a framework called PaR that is powered by the LLM. PaR works in three phases: Peer Solution Selection, Multi-Source Prompt Generation, and Program Repair. Peer Solution Selection identifies the closely related peer programs based on lexical, semantic, and syntactic criteria. Then Multi-Source Prompt Generation adeptly combines multiple sources of information to create a comprehensive and informative prompt for the last Program Repair stage. The evaluation on Defects4DS and another well-investigated ITSP dataset reveals that PaR achieves a new state-of-the-art performance, demonstrating impressive improvements of 19.94% and 15.2% in repair rate compared to prior state-of-the-art LLM- and symbolic-based approaches, respectively


title: Soliton Amplification in the Korteweg-de Vries Equation by Multiplicative Forcing

abstract: We study the stability and dynamics of solitons in the Korteweg de-Vries (KdV) equation with small multiplicative forcing. Forcing breaks the conservative structure of the KdV equation, leading to substantial changes in energy over long time. We show that, for small forcing, the inserted energy is almost fully absorbed by the soliton, resulting in a drastically changed amplitude and velocity. We decompose the solution to the forced equation into a modulated soliton and an infinite dimensional perturbation. Assuming slow exponential decay of the forcing, we show that the perturbation decays at the same exponential rate in a weighted Sobolev norm centered around the soliton.


title: Quantum charges of harmonic oscillators

abstract: We discuss Riemannian geometry of one-dimensional quantum harmonic oscillator. Its wavefunction is a holomorphic section of the complex line bundle $L_{\sf{v}}$ over the phase space $\mathbb{R}^2$. We show that the energy eigenfunctions $\psi_n$ with $n\ge 1$, corresponding to the energy levels $E_n$, are complex coordinates on orbifolds $\mathbb{R}^2/\mathbb{Z}_n$ embedded into $L_{\sf{v}}$, where $\mathbb{Z}_n$ is the cyclic group of order $n$. In fact, $\psi_n (t,z)$ is a standing wave on $\mathbb{R}^2/\mathbb{Z}_n$, where $z$ is a complex coordinate on the phase space $\mathbb{R}^2\cong\mathbb{C}$. Oscillators are characterized by two quantum charges $(q_l^{}, q_{\sf{v}})=(n,1)$, where $q_l^{}=n$ is the winding number for the group U(1) acting on $\mathbb{R}^2/\mathbb{Z}_n$ and $q_{\sf{v}}^{}=1$ is the winding number for the U(1)-rotations on fibres of the bundle $L_{\sf{v}}\to\mathbb{R}^2$, and $E_n=\hbar\omega(q_l^{}+\frac{1}{2} q_{\sf{v}}).$ We also discuss "antioscillators" with opposite quantum charges and the same positive energy.


title: Analyzing the Single Event Upset Vulnerability of Binarized Neural Networks on SRAM FPGAs

abstract: Neural Networks (NNs) are increasingly used in the last decade in several demanding applications, such as object detection and classification, autonomous driving, etc. Among different computing platforms for implementing NNs, FPGAs have multiple advantages due to design flexibility and high performance-to-watt ratio. Moreover, approximation techniques, such as quantization, have been introduced, which reduce the computational and storage requirements, thus enabling the integration of larger NNs into FPGA devices. On the other hand, FPGAs are sensitive to radiation-induced Single Event Upsets (SEUs). In this work, we perform an in-depth reliability analysis in an FPGA-based Binarized Fully Connected Neural Network (BNN) accelerator running a statistical fault injection campaign. The BNN benchmark has been produced by FINN, an open-source framework that provides an end-to-end flow from abstract level to design, making it easy to design customized FPGA NN accelerators, while it also supports various approximation techniques. The campaign includes the injection of faults in the configuration memory of a state-of-the-art Xilinx Ultrascale+ FPGA running the BNN, as well an exhaustive fault injection in the user flip flops. We have analyzed the fault injection results characterizing the SEU vulnerability of the circuit per network layer, per clock cycle, and register. In general, the results show that the BNNs are inherently resilient to soft errors, since a low portion of SEUs in the configuration memory and the flip flops, cause system crashes or misclassification errors.


title: GEARS: Local Geometry-aware Hand-object Interaction Synthesis

abstract: Generating realistic hand motion sequences in interaction with objects has gained increasing attention with the growing interest in digital humans. Prior work has illustrated the effectiveness of employing occupancy-based or distance-based virtual sensors to extract hand-object interaction features. Nonetheless, these methods show limited generalizability across object categories, shapes and sizes. We hypothesize that this is due to two reasons: 1) the limited expressiveness of employed virtual sensors, and 2) scarcity of available training data. To tackle this challenge, we introduce a novel joint-centered sensor designed to reason about local object geometry near potential interaction regions. The sensor queries for object surface points in the neighbourhood of each hand joint. As an important step towards mitigating the learning complexity, we transform the points from global frame to hand template frame and use a shared module to process sensor features of each individual joint. This is followed by a spatio-temporal transformer network aimed at capturing correlation among the joints in different dimensions. Moreover, we devise simple heuristic rules to augment the limited training sequences with vast static hand grasping samples. This leads to a broader spectrum of grasping types observed during training, in turn enhancing our model's generalization capability. We evaluate on two public datasets, GRAB and InterCap, where our method shows superiority over baselines both quantitatively and perceptually.


title: Maximum principles and moving planes method for the fractional $p(x,\cdot)$-Laplacian

abstract: In this paper, we investigate the monotonicity of solutions for a nonlinear equations involving the fractional Laplacian with variable exponent. We first prove different maximum principles involving this operator. Then we employ the direct moving planes method to obtain monotonicity of solutions to a nonlinear equations in which the fractional laplacian with variable exponent is present. Note that, there are no results studying the monotonicity of solutions for local or nonlocal equations with variables exponent. Our results are new in this setting and includes a self-contained techniques.


title: Security for adversarial wiretap channels

abstract: We consider the wiretap channel, where the individual channel uses have memory or are influenced by an adversary. We analyze the explicit and computationally efficient construction of information-theoretically secure coding schemes which use the inverse of an extractor and an error-correcting code. These schemes are known to achieve secrecy capacity on a large class of memoryless wiretap channels. We show that this also holds for certain channel types with memory. In particular, they can achieve secrecy capacity on channels where an adversary can pick a sequence of ``states'' governing the channel's behavior, as long as, given every possible state, the channel is strongly symmetric.


title: A Formal Proof of R(4,5)=25

abstract: In 1995, McKay and Radziszowski proved that the Ramsey number R(4,5) is equal to 25. Their proof relies on a combination of high-level arguments and computational steps. The authors have performed the computational parts of the proof with different implementations in order to reduce the possibility of an error in their programs. In this work, we prove this theorem in the interactive theorem prover HOL4 limiting the uncertainty to the small HOL4 kernel. Instead of verifying their algorithms directly, we rely on the HOL4 interface to MiniSat SAT to prove gluing lemmas. To reduce the number of such lemmas and thus make the computational part of the proof feasible, we implement a generalization algorithm. We verify that its output covers all the possible cases by implementing a custom SAT-solver extended with a graph isomorphism checker.


title: Conditionings to avoid points with various clocks for L\'{e}vy processes

abstract: We discuss conditionings to avoid two points and one-point local time penalizations with conditioning to avoid another point, for which we adopt various clocks. We also give corrections to some of the previous results of Takeda--Yano for one-point local time penalizations.


title: Diagnostics of 3D explosion asymmetries of stripped-envelope supernovae by nebular line profiles

abstract: Understanding the explosion mechanism and hydrodynamic evolution of core-collapse supernovae is a long-standing quest in astronomy. The asymmetries caused by the explosion are encoded into the line profiles which appear in the nebular phase of the SN evolution -- with particularly clean imprints in He star explosions. Here, we carry out nine different supernova simulations of He-core progenitors, exploding them in 3D with parametrically varied neutrino luminosities using the $\texttt{Prometheus-HotB}$ code, hydrodynamically evolving the models to the homologeous phase. We then compute nebular phase spectra with the 3D NLTE spectral synthesis code $\texttt{ExTraSS}$ (EXplosive TRAnsient Spectral Simulator). We study how line widths and shifts depend on progenitor mass, explosion energy, and viewing angle. We compare the predicted line profile properties against a large set of Type Ib observations, and discuss the degree to which current neutrino-driven explosions can match observationally inferred asymmetries. With self-consistent 3D modelling -- circumventing the difficulties of representing $^{56}$Ni mixing and clumping accurately in 1D models -- we find that neither low-mass He cores exploding with high energies nor high-mass cores exploding with low energies contribute to the Type Ib SN population. Models which have line profile widths in agreement with this population give sufficiently large centroid shifts for calcium emission lines. Calcium is more strongly affected by explosion asymmetries connected to the neutron star kicks than oxygen and magnesium. Lastly, we turn to the NIR spectra from our models to investigate the potential of using this regime to look for the presence of He in the nebular phase.


title: Experimental Demonstration of Back-Linked Fabry-Perot Interferometer for the Space Gravitational Wave Antenna

abstract: The back-linked Fabry-Perot interferometer (BLFPI) is an interferometer topology proposed for space gravitational wave antennas with the use of inter-satellite Fabry-Perot interferometers. The BLFPI offers simultaneous and independent control over all interferometer length degrees of freedom by controlling the laser frequencies. Therefore, BLFPI does not require an active control system for the physical lengths of the inter-satellite Fabry-Perot interferometers. To achieve a high sensitivity, the implementation must rely on an offline signal process for subtracting laser frequency noises. However, the subtraction has not been experimentally verified to date. This paper reports a demonstration of the frequency noise subtraction in the frequency band of 100 Hz-50 kHz, including the cavity pole frequency, using Fabry-Perot cavities with a length of 46 cm. The highest reduction ratio of approximately 200 was achieved. This marks the first experimental verification of the critical function in the BLFPI.


title: Guidelines for Cerebrovascular Segmentation: Managing Imperfect Annotations in the context of Semi-Supervised Learning

abstract: Segmentation in medical imaging is an essential and often preliminary task in the image processing chain, driving numerous efforts towards the design of robust segmentation algorithms. Supervised learning methods achieve excellent performances when fed with a sufficient amount of labeled data. However, such labels are typically highly time-consuming, error-prone and expensive to produce. Alternatively, semi-supervised learning approaches leverage both labeled and unlabeled data, and are very useful when only a small fraction of the dataset is labeled. They are particularly useful for cerebrovascular segmentation, given that labeling a single volume requires several hours for an expert. In addition to the challenge posed by insufficient annotations, there are concerns regarding annotation consistency. The task of annotating the cerebrovascular tree is inherently ambiguous. Due to the discrete nature of images, the borders and extremities of vessels are often unclear. Consequently, annotations heavily rely on the expert subjectivity and on the underlying clinical objective. These discrepancies significantly increase the complexity of the segmentation task for the model and consequently impair the results. Consequently, it becomes imperative to provide clinicians with precise guidelines to improve the annotation process and construct more uniform datasets. In this article, we investigate the data dependency of deep learning methods within the context of imperfect data and semi-supervised learning, for cerebrovascular segmentation. Specifically, this study compares various state-of-the-art semi-supervised methods based on unsupervised regularization and evaluates their performance in diverse quantity and quality data scenarios. Based on these experiments, we provide guidelines for the annotation and training of cerebrovascular segmentation models.


title: Coefficient control of the regularized p-Stokes equations

abstract: The Antarctic and Greenland ice sheet simulation is challenging due to unknown parameters in the p-Stokes equations. In this work, we prove the existence of a solution to a parameter identification for the ice rheology and the friction coefficient. Additionally, we verify G\^ateaux differentiability of the control-to-state operator by extending a similar result for distributed control. Moreover, we have more complicated boundary conditions. We only have to add a small diffusion term and assume the nonlinear exponent, which is given in applications, to be small enough to obtain the results. Finally, we state the adjoint equation and prove existence and uniqueness of a solution for this equation.


title: Class-Incremental Few-Shot Event Detection

abstract: Event detection is one of the fundamental tasks in information extraction and knowledge graph. However, a realistic event detection system often needs to deal with new event classes constantly. These new classes usually have only a few labeled instances as it is time-consuming and labor-intensive to annotate a large number of unlabeled instances. Therefore, this paper proposes a new task, called class-incremental few-shot event detection. Nevertheless, this task faces two problems, i.e., old knowledge forgetting and new class overfitting. To solve these problems, this paper further presents a novel knowledge distillation and prompt learning based method, called Prompt-KD. Specifically, to handle the forgetting problem about old knowledge, Prompt-KD develops an attention based multi-teacher knowledge distillation framework, where the ancestor teacher model pre-trained on base classes is reused in all learning sessions, and the father teacher model derives the current student model via adaptation. On the other hand, in order to cope with the few-shot learning scenario and alleviate the corresponding new class overfitting problem, Prompt-KD is also equipped with a prompt learning mechanism. Extensive experiments on two benchmark datasets, i.e., FewEvent and MAVEN, demonstrate the superior performance of Prompt-KD.


title: Auditing Large Language Models for Enhanced Text-Based Stereotype Detection and Probing-Based Bias Evaluation

abstract: Recent advancements in Large Language Models (LLMs) have significantly increased their presence in human-facing Artificial Intelligence (AI) applications. However, LLMs could reproduce and even exacerbate stereotypical outputs from training data. This work introduces the Multi-Grain Stereotype (MGS) dataset, encompassing 51,867 instances across gender, race, profession, religion, and stereotypical text, collected by fusing multiple previously publicly available stereotype detection datasets. We explore different machine learning approaches aimed at establishing baselines for stereotype detection, and fine-tune several language models of various architectures and model sizes, presenting in this work a series of stereotypes classifier models for English text trained on MGS. To understand whether our stereotype detectors capture relevant features (aligning with human common sense) we utilise a variety of explanainable AI tools, including SHAP, LIME, and BertViz, and analyse a series of example cases discussing the results. Finally, we develop a series of stereotype elicitation prompts and evaluate the presence of stereotypes in text generation tasks with popular LLMs, using one of our best performing previously presented stereotypes detectors. Our experiments yielded several key findings: i) Training stereotype detectors in a multi-dimension setting yields better results than training multiple single-dimension classifiers.ii) The integrated MGS Dataset enhances both the in-dataset and cross-dataset generalisation ability of stereotype detectors compared to using the datasets separately. iii) There is a reduction in stereotypes in the content generated by GPT Family LLMs with newer versions.