title: Ultrafast electron diffraction of photoexcited gas-phase cyclobutanone predicted by ab initio multiple cloning simulations

abstract: We present the result of our calculations of ultrafast electron diffraction (UED) for cyclobutanone excited into $S_2$ electronic state, which are based on the non-adiabatic dynamics simulations with \textit{Ab Initio} Multiple Cloning (AIMC) method with the electronic structure calculated at the SA(3)-CASSCF(12,12)/aug-cc-pVDZ level of theory. The key features in the UED pattern were identified that can be used to distinguish between the reaction pathways observed in the AIMC dynamics, although there is a significant overlap between representative signals due to structural similarity of the products. The calculated UED pattern can be compared with experiment.


title: Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review

abstract: This systematic literature review comprehensively examines the application of Large Language Models (LLMs) in forecasting and anomaly detection, highlighting the current state of research, inherent challenges, and prospective future directions. LLMs have demonstrated significant potential in parsing and analyzing extensive datasets to identify patterns, predict future events, and detect anomalous behavior across various domains. However, this review identifies several critical challenges that impede their broader adoption and effectiveness, including the reliance on vast historical datasets, issues with generalizability across different contexts, the phenomenon of model hallucinations, limitations within the models' knowledge boundaries, and the substantial computational resources required. Through detailed analysis, this review discusses potential solutions and strategies to overcome these obstacles, such as integrating multimodal data, advancements in learning methodologies, and emphasizing model explainability and computational efficiency. Moreover, this review outlines critical trends that are likely to shape the evolution of LLMs in these fields, including the push toward real-time processing, the importance of sustainable modeling practices, and the value of interdisciplinary collaboration. Conclusively, this review underscores the transformative impact LLMs could have on forecasting and anomaly detection while emphasizing the need for continuous innovation, ethical considerations, and practical solutions to realize their full potential.


title: Quantum Automating $\mathbf{TC}^0$-Frege Is LWE-Hard

abstract: We prove the first hardness results against efficient proof search by quantum algorithms. We show that under Learning with Errors (LWE), the standard lattice-based cryptographic assumption, no quantum algorithm can weakly automate $\mathbf{TC}^0$-Frege. This extends the line of results of Kraj\'i\v{c}ek and Pudl\'ak (Information and Computation, 1998), Bonet, Pitassi, and Raz (FOCS, 1997), and Bonet, Domingo, Gavald\`a, Maciel, and Pitassi (Computational Complexity, 2004), who showed that Extended Frege, $\mathbf{TC}^0$-Frege and $\mathbf{AC}^0$-Frege, respectively, cannot be weakly automated by classical algorithms if either the RSA cryptosystem or the Diffie-Hellman key exchange protocol are secure. To the best of our knowledge, this is the first interaction between quantum computation and propositional proof search.


title: Subspace Tracking with Dynamical Models on the Grassmannian

abstract: Tracking signals in dynamic environments presents difficulties in both analysis and implementation. In this work, we expand on a class of subspace tracking algorithms which utilize the Grassmann manifold -- the set of linear subspaces of a high-dimensional vector space. We design regularized least squares algorithms based on common manifold operations and intuitive dynamical models. We demonstrate the efficacy of the approach for a narrowband beamforming scenario, where the dynamics of multiple signals of interest are captured by motion on the Grassmannian.


title: Prompt-Based Bias Calibration for Better Zero/Few-Shot Learning of Language Models

abstract: Prompt learning is susceptible to intrinsic bias present in pre-trained language models (LMs), resulting in sub-optimal performance of prompt-based zero/few-shot learning. In this work, we propose a null-input prompting method to calibrate intrinsic bias encoded in pre-trained LMs. Different from prior efforts that address intrinsic bias primarily for social fairness and often involve excessive computational cost, our objective is to explore enhancing LMs' performance in downstream zero/few-shot learning while emphasizing the efficiency of intrinsic bias calibration. Specifically, we leverage a diverse set of auto-selected null-meaning inputs generated from GPT-4 to prompt pre-trained LMs for intrinsic bias probing. Utilizing the bias-reflected probability distribution, we formulate a distribution disparity loss for bias calibration, where we exclusively update bias parameters ($0.1\%$ of total parameters) of LMs towards equal probability distribution. Experimental results show that the calibration promotes an equitable starting point for LMs while preserving language modeling abilities. Across a wide range of datasets, including sentiment analysis and topic classification, our method significantly improves zero/few-shot learning performance of LMs for both in-context learning and prompt-based fine-tuning (on average $9\%$ and $2\%$, respectively).


title: A Heterotic Hermitian--Yang--Mills Equivalence

abstract: We consider N=1, d=4 vacua of heterotic theories in the large radius limit in which alpha' << 1. We construct a real differential operator $\mathcal{D}= D+\bar{D}$ on an extension bundle $(Q, \mathcal{D})$ with underlying topology $Q=(T^{1,0}X)^* \oplus {\rm End} \, E \oplus T^{1,0} X$ whose curvature is holomorphic and Hermitian-Yang-Mills with respect to the complex structure and metric on the underlying non-Kahler complex 3-fold X if and only if the heterotic supersymmetry equations and Bianchi identity are satisfied. This is suggestive of an analogue of the Donaldson--Uhlenbeck--Yau correspondence for heterotic vacua of this type.


title: On rational points on classifying stacks and Malle's conjecture

abstract: In this expository article, we compare Malle's conjecture on counting number fields of bounded discriminant with recent conjectures of Ellenberg--Satriano--Zureick-Brown and Darda--Yasuda on counting points of bounded height on classifying stacks. We illustrate the comparisons via the classifying stacks $B(\mathbb{Z}/n\mathbb{Z})$ and $B{\mu_n}$.


title: Fermion exchange in ring polymer self-consistent field theory

abstract: A mapping is made between fermion exchange and excluded volume in the quantum-classical isomorphism using polymer self-consistent field theory. Apart from exchange, quantum particles are known to be exactly representable in classical statistical mechanics as ring polymers, with contours that are parametrized by the inverse thermal energy, often called the imaginary time. Evidence in support of a previously used approximation for fermion exchange in ring polymer self-consistent field theory is given, specifically, that the use of all-contour interactions in the mean field picture instead of equal imaginary time interactions is justified based on the symmetry of ring polymers. It is also shown that the removal of forbidden thermal trajectories, both those that violate excluded volume directly and those that represent topologically inaccessible microstates, is equivalent to antisymmetric exchange. The electron density of the beryllium atom is calculated with ring polymer self-consistent field theory ignoring classical correlations, and very good agreement is found with Hartree-Fock theory which also neglects Coulomb correlations. The total binding energies agree to within less than 6%, which while still far from chemical accuracy, is remarkable given that the field theory equations are derived from first principles with zero free parameters. The discrepancy between self-consistent field theory and Hartree-Fock theory is attributed to classical Coulomb self-interactions which are included in Hartree-Fock theory but not in self-consistent field theory. A potential method to improve the agreement by more accurately representing electron-electron self-interactions in self-consistent field theory is discussed, as are the implications for quantum foundations of the quantum-classical mapping between fermion exchange and thermal trajectory excluded volume.


title: Efficient Sampling on Riemannian Manifolds via Langevin MCMC

abstract: We study the task of efficiently sampling from a Gibbs distribution $d \pi^* = e^{-h} d {vol}_g$ over a Riemannian manifold $M$ via (geometric) Langevin MCMC; this algorithm involves computing exponential maps in random Gaussian directions and is efficiently implementable in practice. The key to our analysis of Langevin MCMC is a bound on the discretization error of the geometric Euler-Murayama scheme, assuming $\nabla h$ is Lipschitz and $M$ has bounded sectional curvature. Our error bound matches the error of Euclidean Euler-Murayama in terms of its stepsize dependence. Combined with a contraction guarantee for the geometric Langevin Diffusion under Kendall-Cranston coupling, we prove that the Langevin MCMC iterates lie within $\epsilon$-Wasserstein distance of $\pi^*$ after $\tilde{O}(\epsilon^{-2})$ steps, which matches the iteration complexity for Euclidean Langevin MCMC. Our results apply in general settings where $h$ can be nonconvex and $M$ can have negative Ricci curvature. Under additional assumptions that the Riemannian curvature tensor has bounded derivatives, and that $\pi^*$ satisfies a $CD(\cdot,\infty)$ condition, we analyze the stochastic gradient version of Langevin MCMC, and bound its iteration complexity by $\tilde{O}(\epsilon^{-2})$ as well.

categories: math.ST cs.LG math.PR stat.CO stat.ML stat.TH


title: Evaluating the transport properties of interface-type, analog memristors

abstract: Interface-type, analog memristors have quite a reputation for real-time applications in edge sensorics, edge computing, and neuromorphic computing. The n-type conducting BiFeO3 (BFO) is such an interface-type, analog memristor which is also nonlinear and can therefore not only store, but also process data in the same memristor cell without data transfer between the data storage unit and the data processing unit. Here we present a physical memristor model which describes the hysteretic current-voltage curves of the BFO memristor in the small and large current-voltage range. Extracted internal state variables are reconfigured by the ion drift in the two write branches and are determining the electron transport in the two read branches. Simulation of electronic circuits with the BFO interface-type, analog memristors was not possible so far because previous physical memristor models have not captured the full range of internal state variables. We show quantitative agreement between modeled and experimental current-voltage curves exemplarily of three different BFO memristors in the small and large current-voltage ranges. Extracted dynamic and static internal state variables in the two full write branches and in the two full read branches, respectively, can be used for simulating electronic circuits with BFO memristors, e.g. in edge sensorics, edge computing, and neuromorphic computing.

categories: cond-mat.dis-nn math-ph math.MP physics.app-ph


title: Can we Soft Prompt LLMs for Graph Learning Tasks?

abstract: Graph plays an important role in representing complex relationships in real-world applications such as social networks, biological data and citation networks. In recent years, Large Language Models (LLMs) have achieved tremendous success in various domains, which makes applying LLMs to graphs particularly appealing. However, directly applying LLMs to graph modalities presents unique challenges due to the discrepancy and mismatch between the graph and text modalities. Hence, to further investigate LLMs' potential for comprehending graph information, we introduce GraphPrompter, a novel framework designed to align graph information with LLMs via soft prompts. Specifically, GraphPrompter consists of two main components: a graph neural network to encode complex graph information and an LLM that effectively processes textual information. Comprehensive experiments on various benchmark datasets under node classification and link prediction tasks demonstrate the effectiveness of our proposed method. The GraphPrompter framework unveils the substantial capabilities of LLMs as predictors in graph-related tasks, enabling researchers to utilize LLMs across a spectrum of real-world graph scenarios more effectively.


title: Transductive Sample Complexities Are Compact

abstract: We demonstrate a compactness result holding broadly across supervised learning with a general class of loss functions: Any hypothesis class $H$ is learnable with transductive sample complexity $m$ precisely when all of its finite projections are learnable with sample complexity $m$. We prove that this exact form of compactness holds for realizable and agnostic learning with respect to any proper metric loss function (e.g., any norm on $\mathbb{R}^d$) and any continuous loss on a compact space (e.g., cross-entropy, squared loss). For realizable learning with improper metric losses, we show that exact compactness of sample complexity can fail, and provide matching upper and lower bounds of a factor of 2 on the extent to which such sample complexities can differ. We conjecture that larger gaps are possible for the agnostic case. Furthermore, invoking the equivalence between sample complexities in the PAC and transductive models (up to lower order factors, in the realizable case) permits us to directly port our results to the PAC model, revealing an almost-exact form of compactness holding broadly in PAC learning.


title: Stability of asymptotic waves in the Fisher-Stefan equation

abstract: We establish spectral, linear, and nonlinear stability of the vanishing and slow-moving travelling waves that arise as time asymptotic solutions to the Fisher-Stefan equation. Nonlinear stability is in terms of the limiting equations that the asymptotic waves satisfy.


title: Better bounds for low-energy product formulas

abstract: Product formulas are one of the main approaches for quantum simulation of the Hamiltonian dynamics of a quantum system. Their implementation cost is computed based on error bounds which are often pessimistic, resulting in overestimating the total runtime. In this work, we rigorously consider the error induced by product formulas when the state undergoing time evolution lies in the low-energy sector with respect to the Hamiltonian of the system. We show that in such a setting, the usual error bounds based on the operator norm of nested commutators can be replaced by those restricted to suitably chosen low-energy subspaces, yielding tighter error bounds. Furthermore, under some locality and positivity assumptions, we show that the simulation of generic product formulas acting on low-energy states can be done asymptotically more efficiently when compared with previous results.


title: Symplectic fillings of unit cotangent bundles of spheres and applications

abstract: We prove the uniqueness, up to diffeomorphism, of symplectically aspherical fillings of the unit cotangent bundle of the $3$-sphere $S^3$ under a certain topological assumption, which Stein fillings automatically satisfy. In the course of the proof, we show that any symplectically aspherical filling of the unit cotangent bundle of the $n$-sphere $S^n$ ($n \geq 3$) is simply-connected. As applications, we first show the non-existence of exact symplectic cobordisms between some $5$-dimensional Brieskorn manifolds. We also determine the diffeomorphism types of closed symplectic $6$-manifolds with certain codimension $2$ symplectic submanifolds.


title: The Dirichlet problem for the $p(x)$-Laplacian with unbounded exponent $p(x)$

abstract: We prove the solvability of the Dirichlet problem for the variable exponent $p$-Laplacian with boundary data in $W^{1,p(x)}(\Omega)$ on a bounded, smooth domain $\Omega \subset {\mathbb R}^n$. Our main focus will be on an a.e. finite variable exponent $p(\cdot)$ with $n < \inf\limits_{x\in \Omega}p(x)$ and $\sup\limits_{x\in \Omega}p(x) = \infty$ under the sole assumption that $p\in C(\Omega)$.


title: Deep Spectral Meshes: Multi-Frequency Facial Mesh Processing with Graph Neural Networks

abstract: With the rising popularity of virtual worlds, the importance of data-driven parametric models of 3D meshes has grown rapidly. Numerous applications, such as computer vision, procedural generation, and mesh editing, vastly rely on these models. However, current approaches do not allow for independent editing of deformations at different frequency levels. They also do not benefit from representing deformations at different frequencies with dedicated representations, which would better expose their properties and improve the generated meshes' geometric and perceptual quality. In this work, spectral meshes are introduced as a method to decompose mesh deformations into low-frequency and high-frequency deformations. These features of low- and high-frequency deformations are used for representation learning with graph convolutional networks. A parametric model for 3D facial mesh synthesis is built upon the proposed framework, exposing user parameters that control disentangled high- and low-frequency deformations. Independent control of deformations at different frequencies and generation of plausible synthetic examples are mutually exclusive objectives. A Conditioning Factor is introduced to leverage these objectives. Our model takes further advantage of spectral partitioning by representing different frequency levels with disparate, more suitable representations. Low frequencies are represented with standardised Euclidean coordinates, and high frequencies with a normalised deformation representation (DR). This paper investigates applications of our proposed approach in mesh reconstruction, mesh interpolation, and multi-frequency editing. It is demonstrated that our method improves the overall quality of generated meshes on most datasets when considering both the $L_1$ norm and perceptual Dihedral Angle Mesh Error (DAME) metrics.


title: Seiberg-Witten Equations and Einstein Metrics on Finite Volume 4-Manifolds with Asymptotically Hyperbolic Ends

abstract: We construct infinitely many examples of finite volume 4-manifolds with $T^3$ ends that do not admit any cusped asymptotically hyperbolic Einstein metrics yet satisfy a strict logarithmic version of the Hitchin-Thorpe inequality due to Dai-Wei. This is done by using estimates from Seiberg-Witten theory due to LeBrun as well as a method for constructing solutions to the Seiberg-Witten equations on noncompact manifolds due to Biquard. We also use constructions coming from the $Pin^-(2)$ monopole equations to obtain a larger class of manifolds where these techniques apply.


title: Parametric All-Optical Modulation on Chip

abstract: We demonstrate parametric all-optical modulation in a periodically-poled lithium niobate microring resonator on chip. It employs quantum Zeno blockade between two distinct waves, a signal and a pump, through their sum-frequency generation at a large per-photon efficiency of 8.2 MHz. With nanosecond pump pulses at 6 mW peak power, 85.7% modulation extinction is observed, marking over 30~times efficiency improvement across various previous implementations. With only 2 mW pump peak power, 43.0% modulation extinction is observed for a doubly-stronger signal at 4 mW. This demonstrates, for the first time, that optical transistors with cascadability and fan-out are possible with just parametric nonlinear optics. These results, together with inherent advantages in such photonic integrated circuits, open the door to scalable technology for all-optical and quantum information processing.


title: Beam Squinting Compensation: An NCR-Assisted Scenario

abstract: Millimeter wave (mmWave) and sub-THz communications, foreseen for sixth generation (6G), suffer from high propagation losses which affect the network coverage. To address this point, smart entities such as network-controlled repeaters (NCRs) have been considered as cost-efficient solutions for coverage extension. NCRs, which have been standardized in 3rd generation partnership project Release 18, are radio frequency repeaters with beamforming capability controlled by the network through side control information. Another challenge raised by the adoption of high frequency bands is the use of large bandwidths. Here, a common configuration is to divide a large frequency band into multiple smaller subbands. In this context, we consider a scenario with NCRs where signaling related to measurements used for radio resource management is transmitted in one subband centered at frequency $f_c$ and data transmission is performed at a different frequency $f_c + \Delta f$ based on the measurements taken at $f_c$. Here, a challenge is that the array radiation pattern can be frequency dependent and, therefore, lead to beam misalignment, called beam squinting. We characterize beam squinting in the context of subband operation and propose a solution where the beam patterns to be employed at a given subband can be adjusted/compensated to mitigate beam squinting. Our results show that, without compensation, the perceived signal to interference-plus-noise ratio (SINR) and so the throughput can be substantially decreased due to beam squinting. However, with our proposed compensation method, the system is able to support NCR subband signaling operation with similar performance as if signaling and data were transmitted at the same frequency.


title: Infinitesimal jet spaces of $\text{Bun}_G$ in positive characteristic

abstract: Given a semisimple reductive group $G$ and a smooth projective curve $X$ over an algebraically closed field $k$ of arbitrary characteristic, let $\text{Bun}_G$ denote the moduli space of principal $G$-bundles over $X$. For a bundle $P\in\text{Bun}_G$ without infinitesimal symmetries, we provide a description of all divided-power infinitesimal jet spaces, $J_P^{n,PD}(\text{Bun}_G)$, of $\text{Bun}_G$ at $P$. The description is in terms of differential forms on $X^n$ with logarithmic singularities along the diagonals and with coefficients in $(\mathfrak{g}_P^*)^{\boxtimes n}$. Furthermore, we show the pullback of these differential forms to the Fulton-Macpherson compactification of the configuration space, $\hat{X}^n$, is an isomorphism. Thus, we relate the two constructions of Beilinson-Drinfeld and Beilinson-Ginzburg, and as a consequence, give a connection between divided-power infinitesimal jet spaces of $\text{Bun}_G$ and the $\mathcal{L}ie$ operad.


title: Field Line Universal relaXer (FLUX): A Fluxon Approach to Coronal Magnetic Field Modeling

abstract: We describe a novel method for modeling the global, steady solar wind using photospheric magnetic fields as a driving boundary condition. Prior wind models in this class include both rapid heuristic methods that use potential field extrapolation and variants thereof, trading rigor for computation speed, and detailed 3D magnetohydrodynamic (MHD) models that attempt to simulate the entire solar corona with a degree of physical rigor, but require large amounts of computation. The Field Line Universal relaXer (FLUX), an open-source numerical code which implements the 'fluxon' semi-lagrangian approach to MHD modeling, provides an intermediate approach between these two general classes. In particular, the fluxon approach to MHD describes the magnetic field through discrete analogues of magnetic field lines, relaxing these structures to a stationary state of force balance. In this work we introduce a one-dimensional solar wind solution along each fieldline, providing an ensemble of solutions that are interpolated back onto a uniform grid at an outer boundary surface. This provides advantages in physical rigor over heuristic semi-analytic techniques, and in computational efficiency over full 3D MHD techniques. Here we describe the underlying methodology and the FLUXPipe modeling pipeline process.


title: First Year of Stellar-Mass Black Hole Observations with the Imaging X-ray Polarimetry Explorer

abstract: The Imaging X-ray Polarimetry Explorer (IXPE), launched on December 9, 2021, enables X-ray polarimetric observations with unprecedented sensitivity in the 2-8 keV energy range. X-ray polarization allows us to test accretion disk, corona, and emission models of stellar-mass black holes in X-ray binaries found predominantly in soft and hard states of accretion. Every state is dominated by a combination of thermal disk, coronal, or reflected emission$\unicode{x2014}$each type containing different information about the environment around the black hole that can be understood through polarization. In 2022, IXPE measured the polarization signatures of four stellar-mass black holes: Cygnus X-1, 4U 1630-47, Cygnus X-3, and LMC X-1. We report on the physical consequences of these first IXPE observations and the science driven by this new chapter in X-ray polarimetry.


title: Sequential Manipulation of Deformable Linear Object Networks with Endpoint Pose Measurements using Adaptive Model Predictive Control

abstract: Robotic manipulation of deformable linear objects (DLOs) is an active area of research, though emerging applications, like automotive wire harness installation, introduce constraints that have not been considered in prior work. Confined workspaces and limited visibility complicate prior assumptions of multi-robot manipulation and direct measurement of DLO configuration (state). This work focuses on single-arm manipulation of stiff DLOs (StDLOs) connected to form a DLO network (DLON), for which the measurements (output) are the endpoint poses of the DLON, which are subject to unknown dynamics during manipulation. To demonstrate feasibility of output-based control without state estimation, direct input-output dynamics are shown to exist by training neural network models on simulated trajectories. Output dynamics are then approximated with polynomials and found to contain well-known rigid body dynamics terms. A composite model consisting of a rigid body model and an online data-driven residual is developed, which predicts output dynamics more accurately than either model alone, and without prior experience with the system. An adaptive model predictive controller is developed with the composite model for DLON manipulation, which completes DLON installation tasks, both in simulation and with a physical automotive wire harness.


title: BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains

abstract: Large Language Models (LLMs) have demonstrated remarkable versatility in recent years, offering potential applications across specialized domains such as healthcare and medicine. Despite the availability of various open-source LLMs tailored for health contexts, adapting general-purpose LLMs to the medical domain presents significant challenges. In this paper, we introduce BioMistral, an open-source LLM tailored for the biomedical domain, utilizing Mistral as its foundation model and further pre-trained on PubMed Central. We conduct a comprehensive evaluation of BioMistral on a benchmark comprising 10 established medical question-answering (QA) tasks in English. We also explore lightweight models obtained through quantization and model merging approaches. Our results demonstrate BioMistral's superior performance compared to existing open-source medical models and its competitive edge against proprietary counterparts. Finally, to address the limited availability of data beyond English and to assess the multilingual generalization of medical LLMs, we automatically translated and evaluated this benchmark into 7 other languages. This marks the first large-scale multilingual evaluation of LLMs in the medical domain. Datasets, multilingual evaluation benchmarks, scripts, and all the models obtained during our experiments are freely released.


title: Revisiting Experience Replayable Conditions

abstract: Experience replay (ER) used in (deep) reinforcement learning is considered to be applicable only to off-policy algorithms. However, there have been some cases in which ER has been applied for on-policy algorithms, suggesting that off-policyness might be a sufficient condition for applying ER. This paper reconsiders more strict "experience replayable conditions" (ERC) and proposes the way of modifying the existing algorithms to satisfy ERC. To this end, instability of policy improvements is assumed to be a key in ERC. The instability factors are revealed from the viewpoint of metric learning as i) repulsive forces from negative samples and ii) replays of inappropriate experiences. Accordingly, the corresponding stabilization tricks are derived. As a result, it is confirmed through numerical simulations that the proposed stabilization tricks make ER applicable to an advantage actor-critic, an on-policy algorithm. In addition, its learning performance is comparable to that of a soft actor-critic, a state-of-the-art off-policy algorithm.


title: Incompressible limit for weakly asymmetric simple exclusion processes coupled through collision

abstract: We establish the incompressible limit of weakly asymmetric simple exclusion processes coupled through particle collisions. The incompressible limit depends on various parameters in the particle system and is linked to fluid dynamics equations. Our main contributions to previous results are the extension of the parameter space and the focus on local particle jumps. Our proof uses the relative entropy method. The main novelties in the proof are a Boltzmann-Gibbs principle (a replacement lemma) and a spectral gap estimate.


title: Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)

abstract: CLIP embeddings have demonstrated remarkable performance across a wide range of computer vision tasks. However, these high-dimensional, dense vector representations are not easily interpretable, restricting their usefulness in downstream applications that require transparency. In this work, we empirically show that CLIP's latent space is highly structured, and consequently that CLIP representations can be decomposed into their underlying semantic components. We leverage this understanding to propose a novel method, Sparse Linear Concept Embeddings (SpLiCE), for transforming CLIP representations into sparse linear combinations of human-interpretable concepts. Distinct from previous work, SpLiCE does not require concept labels and can be applied post hoc. Through extensive experimentation with multiple real-world datasets, we validate that the representations output by SpLiCE can explain and even replace traditional dense CLIP representations, maintaining equivalent downstream performance while significantly improving their interpretability. We also demonstrate several use cases of SpLiCE representations including detecting spurious correlations, model editing, and quantifying semantic shifts in datasets.


title: Quasilinear Lane-Emden type systems with sub-natural growth terms

abstract: Global pointwise estimates are obtained for quasilinear Lane-Emden-type systems involving measures in the "sublinear growth" rate. We give necessary and sufficient conditions for existence expressed in terms of Wolff's potential. Our approach is based on recent advances due to Kilpel\"{a}inen and Mal\'{y} in the potential theory. This method enables us to treat several problems, such as equations involving general quasilinear operators and fractional Laplacian.


title: A local to global question for linear functionals

abstract: Let $F$ be an algebraically closed field and let $n\geq 3$. Consider $V=F^n$ with standard basis $\{\vec{e}_1,\ldots,\vec{e}_n\}$ and its dual space $V^*= {\mathrm{Hom}}_{F-{\mathrm{lin}}}(V,F)$ with dual basis $\{y_1,\ldots,y_n\}\subseteq V^*$ and let $\vec{y} = \sum_i y_i\otimes \vec{e}_i\in V^*\otimes V$. Let $d<n$ and consider the vectors $\vec{q}_1,\ldots,\vec{q}_d\in V^*\otimes V$. In this note we consider the question of whether $\vec{y}(\vec{v}) = \vec{v} \in Span_F(\vec{q}_1(\vec{v}),\ldots,\vec{q}_d(\vec{v}))$ for all $\vec{v}\in V$ implies that $\vec{y}\in Span_F(\vec{q}_1,\ldots,\vec{q}_d)$. We show this is true for $d=1$ or $d=2$, but that additional properties are needed for $d\geq 3$. We then interpret this result in terms of subspaces of $M_n(F)$ that do not contain any rank 1 idempotents.


title: DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows

abstract: Large language models (LLMs) have become a dominant and important tool for NLP researchers in a wide range of tasks. Today, many researchers use LLMs in synthetic data generation, task evaluation, fine-tuning, distillation, and other model-in-the-loop research workflows. However, challenges arise when using these models that stem from their scale, their closed source nature, and the lack of standardized tooling for these new and emerging workflows. The rapid rise to prominence of these models and these unique challenges has had immediate adverse impacts on open science and on the reproducibility of work that uses them. In this paper, we introduce DataDreamer, an open source Python library that allows researchers to write simple code to implement powerful LLM workflows. DataDreamer also helps researchers adhere to best practices that we propose to encourage open science and reproducibility. The library and documentation are available at https://github.com/datadreamer-dev/DataDreamer .


title: Subgraph-level Universal Prompt Tuning

abstract: In the evolving landscape of machine learning, the adaptation of pre-trained models through prompt tuning has become increasingly prominent. This trend is particularly observable in the graph domain, where diverse pre-training strategies present unique challenges in developing effective prompt-based tuning methods for graph neural networks. Previous approaches have been limited, focusing on specialized prompting functions tailored to models with edge prediction pre-training tasks. These methods, however, suffer from a lack of generalizability across different pre-training strategies. Recently, a simple prompt tuning method has been designed for any pre-training strategy, functioning within the input graph's feature space. This allows it to theoretically emulate any type of prompting function, thereby significantly increasing its versatility for a range of downstream applications. Nevertheless, the capacity of such simple prompts to fully grasp the complex contexts found in graphs remains an open question, necessitating further investigation. Addressing this challenge, our work introduces the Subgraph-level Universal Prompt Tuning (SUPT) approach, focusing on the detailed context within subgraphs. In SUPT, prompt features are assigned at the subgraph-level, preserving the method's universal capability. This requires extremely fewer tuning parameters than fine-tuning-based methods, outperforming them in 42 out of 45 full-shot scenario experiments with an average improvement of over 2.5%. In few-shot scenarios, it excels in 41 out of 45 experiments, achieving an average performance increase of more than 6.6%.


title: UMAIR-FPS: User-aware Multi-modal Animation Illustration Recommendation Fusion with Painting Style

abstract: The rapid advancement of high-quality image generation models based on AI has generated a deluge of anime illustrations. Recommending illustrations to users within massive data has become a challenging and popular task. However, existing anime recommendation systems have focused on text features but still need to integrate image features. In addition, most multi-modal recommendation research is constrained by tightly coupled datasets, limiting its applicability to anime illustrations. We propose the User-aware Multi-modal Animation Illustration Recommendation Fusion with Painting Style (UMAIR-FPS) to tackle these gaps. In the feature extract phase, for image features, we are the first to combine image painting style features with semantic features to construct a dual-output image encoder for enhancing representation. For text features, we obtain text embeddings based on fine-tuning Sentence-Transformers by incorporating domain knowledge that composes a variety of domain text pairs from multilingual mappings, entity relationships, and term explanation perspectives, respectively. In the multi-modal fusion phase, we novelly propose a user-aware multi-modal contribution measurement mechanism to weight multi-modal features dynamically according to user features at the interaction level and employ the DCN-V2 module to model bounded-degree multi-modal crosses effectively. UMAIR-FPS surpasses the stat-of-the-art baselines on large real-world datasets, demonstrating substantial performance enhancements.


title: Making Short-Form Videos Accessible with Hierarchical Video Summaries

abstract: Short videos on platforms such as TikTok, Instagram Reels, and YouTube Shorts (i.e. short-form videos) have become a primary source of information and entertainment. Many short-form videos are inaccessible to blind and low vision (BLV) viewers due to their rapid visual changes, on-screen text, and music or meme-audio overlays. In our formative study, 7 BLV viewers who regularly watched short-form videos reported frequently skipping such inaccessible content. We present ShortScribe, a system that provides hierarchical visual summaries of short-form videos at three levels of detail to support BLV viewers in selecting and understanding short-form videos. ShortScribe allows BLV users to navigate between video descriptions based on their level of interest. To evaluate ShortScribe, we assessed description accuracy and conducted a user study with 10 BLV participants comparing ShortScribe to a baseline interface. When using ShortScribe, participants reported higher comprehension and provided more accurate summaries of video content.


title: Interpolation between domains of powers of operators in quaternionic Banach spaces

abstract: In contrast to the classical complex spectral theory, where the spectrum is related to the invertibility of $\lambda-A:D(A)\subseteq X_\mathbb{C}\rightarrow X_\mathbb{C}$, in the noncommutative quaternionic $S$-spectral theory one uses the invertibility of the second order polynomial $Q_s(T):=T^2-2\text{Re}(s)T+|s|^2:D(T^2)\subseteq X\rightarrow X$ to define the $S$-spectrum, where $X$ is a quaternionic Banach space. In this paper we will consider quaternionic operators $T$, for which at least one ray $\{te^{i\omega}\;|\;t>0\}$, $\omega\in[0,\pi]$, $i\in\mathbb{S}$ is contained in the $S$-resolvent set, and the inverse operator $Q_s^{-1}(T)$ admits certain decay properties on this ray. Utilizing the $K$-interpolation method, we then demonstrate that the domain $D(T^k)$ of the $k$-th power of $T$ is an intermediate space between $D(T^n)$ and $D(T^m)$, whenever $n<k<m\in\mathbb{N}_0$. Moreover, also a characterization of the interpolation space $(X,D(T^n))_{\theta,p}$, $\theta\in(0,1)$, $p\in[1,\infty]$, in is given in terms of integrability conditions on the pseudo $S$-resolvent $Q_s^{-1}(T)$.


title: Catalytic enhancement in the performance of the microscopic two-stroke heat engine

abstract: We consider a model of a heat engine operating in the microscopic regime: the two-stroke engine. It produces work and exchanges heat in two discrete strokes that are separated in time. The engine consists of two $d$-level systems initialized in thermal states at two distinct temperatures. Additionally, an auxiliary non-equilibrium system called catalyst may be incorporated into the engine, provided the state of the catalyst remains unchanged after the completion of a thermodynamic cycle. This ensures that the work produced arises solely from the temperature difference, Upon establishing the rigorous thermodynamic framework, we characterize two-fold improvement stemming from the inclusion of a catalyst. Firstly, we show that the presence of a catalyst allows for surpassing the optimal efficiency of two-stroke heat engines which are not assisted by a catalyst. In particular, we prove that the optimal efficiency for two-stroke heat engine consisting of two-level systems is given by the Otto efficiency, and that it can be surpassed via incorporating a catalyst. Secondly, we show that incorporating a catalyst allows the engine to operate in frequency and temperature regimes that are not accessible for non-catalytic two-stroke engines.


title: Middleware-based multi-agent development environment for building and testing distributed intelligent systems

abstract: The spread of the Internet of Things (IoT) is demanding new, powerful architectures for handling the huge amounts of data produced by the IoT devices. In many scenarios, many existing isolated solutions applied to IoT devices use a set of rules to detect, report and mitigate malware activities or threats. This paper describes a development environment that allows the programming and debugging of such rule-based multi-agent solutions. The solution consists of the integration of a rule engine into the agent, the use of a specialized, wrapping agent class with a graphical user interface for programming and testing purposes, and a mechanism for the incremental composition of behaviors. Finally, a set of examples and a comparative study were accomplished to test the suitability and validity of the approach. The JADE multi-agent middleware has been used for the practical implementation of the approach.


title: Exploring RIS Coverage Enhancement in Factories: From Ray-Based Modeling to Use-Case Analysis

abstract: Reconfigurable Intelligent Surfaces (RISs) have risen to the forefront of wireless communications research due to their proactive ability to alter the wireless environment intelligently, promising improved wireless network capacity and coverage. Thus, RISs are a pivotal technology in evolving next-generation communication networks. This paper demonstrates a system-level modeling approach for RIS. The RIS model, integrated with the Volcano ray-tracing (RT) tool, is used to analyze the far-field (FF) RIS channel properties in a typical factory environment and explore coverage enhancement at sub-6 GHz and mmWave frequencies. The results obtained in non-line-of-sight (NLoS) scenarios confirm that RIS application is relevant for 5G industrial networks.


title: MFBind: a Multi-Fidelity Approach for Evaluating Drug Compounds in Practical Generative Modeling

abstract: Current generative models for drug discovery primarily use molecular docking to evaluate the quality of generated compounds. However, such models are often not useful in practice because even compounds with high docking scores do not consistently show experimental activity. More accurate methods for activity prediction exist, such as molecular dynamics based binding free energy calculations, but they are too computationally expensive to use in a generative model. We propose a multi-fidelity approach, Multi-Fidelity Bind (MFBind), to achieve the optimal trade-off between accuracy and computational cost. MFBind integrates docking and binding free energy simulators to train a multi-fidelity deep surrogate model with active learning. Our deep surrogate model utilizes a pretraining technique and linear prediction heads to efficiently fit small amounts of high-fidelity data. We perform extensive experiments and show that MFBind (1) outperforms other state-of-the-art single and multi-fidelity baselines in surrogate modeling, and (2) boosts the performance of generative models with markedly higher quality compounds.


title: Improvising Age Verification Technologies in Canada: Technical, Regulatory and Social Dynamics

abstract: Age verification, which is a mandatory legal requirement for delivering certain age-appropriate services or products, has recently been emphasized around the globe to ensure online safety for children. The rapid advancement of artificial intelligence has facilitated the recent development of some cutting-edge age-verification technologies, particularly using biometrics. However, successful deployment and mass acceptance of these technologies are significantly dependent on the corresponding socio-economic and regulatory context. This paper reviews such key dynamics for improvising age-verification technologies in Canada. It is particularly essential for such technologies to be inclusive, transparent, adaptable, privacy-preserving, and secure. Effective collaboration between academia, government, and industry entities can help to meet the growing demands for age-verification services in Canada while maintaining a user-centric approach.


title: Enabling Zero Trust Security in IoMT Edge Network

abstract: Internet of Medical Things (IoMT) deals with a patient-data-rich segment, which makes security and privacy a severe concern for patients. Therefore, access control is a significant aspect of ensuring trust in the IoMT. However, deploying existing authentication and authorization solutions to the Internet of Medical Things (IoMT) is not straightforward because of highly dynamic and possibly unprotected environments and untrusted supply chain for the IoT devices. In this article, we propose Soter, a Zero-Trust based authentication system for the IoMT. Soter Incorporates trust negotiation mechanisms within the Zero Trust framework to enable dynamic trust establishment. When a user or device seeks access to a resource, initiate a trust negotiation process. During this process, credentials, attributes, and contextual information are exchanged between the requester and the resource owner. Soter defines access rules based on various factors, including user identity, device health, and location. Access is granted or denied based on these conditions.


title: The physical properties of Fermi-4LAC low-synchrotron-peaked BL Lac objects

abstract: Previous studies on the fitting of spectral energy distributions (SEDs) often apply the external-Compton process to interpret the high-energy peak of low-synchrotron-peaked (LSP) BL Lac objects (LBLs), despite the lack of strong broad emission lines observed for LBLs. In this work, we collect quasi-simultaneous multi-wavelength data of 15 LBLs from the Fermi fourth LAT AGN catalog (4LAC). We propose an analytical method to assess the necessity of external photon fields in the framework of one-zone scenario. Following derived analytical results, we fit the SEDs of these LBLs with the conventional one-zone leptonic model and study their jet physical properties. Our main results can be summarized as follows. (1)We find that most LBLs cannot be fitted by the one-zone synchrotron self-Compton (SSC) model. This indicates that external photons play a crucial role in the high-energy emission of LBLs, therefore we suggest that LBLs are masquerading BL Lacs. (2) We suggest that the $\gamma$-ray emitting regions of LBLs are located outside the broad-line region and within the dusty torus. (3) By extending the analytical method to all types of LSPs in Fermi-4LAC (using historical data), we find that the high-energy peaks of some flat spectrum radio quasars and blazar candidates of unknown types can be attributed to the SSC emission, implying that the importance of external photons could be minor. We suggest that the variability timescale may help distinguish the origin of the high-energy peak.


title: Requirements for probing chiral Casimir-Polder forces in a molecular Talbot-Lau interferometer

abstract: We theoretically investigate the influence of chiral Casimir-Polder (CP) forces in Talbot-Lau interferometry, based on three nanomechanical gratings. We study scenarios where the second grating is either directly written into a chiral material or where the nanomask is coated with chiral substances. We show requirements for probing enantiospecific effects in matter-wave interferometry in the transmission signal and the interference fringe visibility, which depend on the de Broglie wavelength and the molecular chirality. The proposed setup is particularly sensitive to CP forces in the non-retarded regime where chiral effects can be comparable in magnitude to their electric and magnetic counterparts. While the first and third gratings do not change the phase of the matter wave, applying a coating of chiral substances to them enhances the instrument's chiral selectivity.


title: Pretext Training Algorithms for Event Sequence Data

abstract: Pretext training followed by task-specific fine-tuning has been a successful approach in vision and language domains. This paper proposes a self-supervised pretext training framework tailored to event sequence data. We introduce a novel alignment verification task that is specialized to event sequences, building on good practices in masked reconstruction and contrastive learning. Our pretext tasks unlock foundational representations that are generalizable across different down-stream tasks, including next-event prediction for temporal point process models, event sequence classification, and missing event interpolation. Experiments on popular public benchmarks demonstrate the potential of the proposed method across different tasks and data domains.


title: Darwin Turing Dawkins: Building a General Theory of Evolution

abstract: Living things, computers, societies, and even books are part of a grand evolutionary struggle to survive. That struggle shapes nature, nations, religions, art, science, and you. What you think, feel, and do is determined by it. Darwinian evolution does not apply solely to the genes that are stored in DNA. Using the insights of Alan Turing and Richard Dawkins, we will see that it also applies to the memes we store in our brains and the information we store in our computers. The next time you run for president, fight a war, or just deal with the ordinary problems humans are heir to, perhaps this book will be of use. If you want to understand why and when you will die, or if you want to achieve greatness this book may help. If you are concerned about where the computer revolution is headed, this book may provide some answers.


title: Ion Transport through Short Nanopores Modulated by Charged Exterior Surfaces

abstract: Short nanopores find extensive applications capitalizing on their high throughput and detection resolution. Ionic behaviors through long nanopores are mainly determined by charged inner-pore walls. When pore lengths decrease to sub-200 nm, charged exterior surfaces provide considerable modulation to ion current. We find that the charge status of inner-pore walls affects the modulation of ion current from charged exterior surfaces. For 50-nm-long nanopores with neutral inner-pore walls, charged exterior surfaces on the voltage (surfaceV) and ground (surfaceG) sides enhance and inhibit ion transport by forming ion enrichment and depletion zones inside nanopores, respectively. For nanopores with both charged inner-pore and exterior surfaces, continuous electric double layers enhance ion transport through nanopores significantly. The charged surfaceV results in higher ion current by simultaneously weakening ion depletion at pore entrances and enhancing the intra-pore ion enrichment. The charged surfaceG expedites the exit of ions from nanopores, resulting in a decrease in ion enrichment at pore exits. Through adjustment in the width of charged-ring regions near pore boundaries, the effective charged width of the charged exterior is explored at ~20nm. Our results may provide a theoretical guide for further optimizing the performance of nanopore-based applications, like seawater desalination, biosensing, and osmotic energy conversion.


title: Assessing the Performance of OpenTitan as Cryptographic Accelerator in Secure Open-Hardware System-on-Chips

abstract: RISC-V open-source systems are emerging in deployment scenarios where safety and security are critical. OpenTitan is an open-source silicon root-of-trust designed to be deployed in a wide range of systems, from high-end to deeply embedded secure environments. Despite the availability of various cryptographic hardware accelerators that make OpenTitan suitable for offloading cryptographic workloads from the main processor, there has been no accurate and quantitative establishment of the benefits derived from using OpenTitan as a secure accelerator. This paper addresses this gap by thoroughly analysing strengths and inefficiencies when offloading cryptographic workloads to OpenTitan. The focus is on three key IPs - HMAC, AES, and OpenTitan Big Number accelerator (OTBN) - which can accelerate four security workloads: Secure Hash Functions, Message Authentication Codes, Symmetric cryptography, and Asymmetric cryptography. For every workload, we develop a bare-metal driver for the OpenTitan accelerator and analyze its efficiency when computation is offloaded from a RISC-V application core within a System-on-Chip designed for secure Cyber-Physical Systems applications. Finally, we assess it against a software implementation on the application core. The characterization was conducted on a cycle-accurate RTL simulator of the System-on-Chip (SoC). Our study demonstrates that OpenTitan significantly outperforms software implementations, with speedups ranging from 4.3x to 12.5x. However, there is potential for even greater gains as the current OpenTitan utilizes a fraction of the accelerator bandwidths, which ranges from 16% to 61%, depending on the memory being accessed and the accelerator used. Our results open the way to the optimization of OpenTitan-based secure platforms, providing design guidelines to unlock the full potential of its accelerators in secure applications.


title: Improved SQP and SLSQP Algorithms for Feasible Path-based Process Optimisation

abstract: The feasible path algorithm has been widely used for process optimisation due to its good convergence. The sequential quadratic programming (SQP) algorithm is usually used to drive the feasible path algorithm towards optimality. However, existing SQP algorithms may suffer from inconsistent quadratic programming (QP) subproblems and numerical noise, especially for ill-conditioned optimisation problems, leading to a suboptimal or infeasible solution. In this work, we propose an improved SQP algorithm (I-SQP) and an improved sequential least squares programming algorithm (I-SLSQP) that solves a least squares (LSQ) subproblem at each major iteration. A hybrid method through the combination of two relaxation formulations is proposed to solve inconsistent subproblems for better convergence and higher efficiency. We analyse a certain part of the dual LSQ solution algorithm and find it suffers from serious cancellation errors, resulting in an inaccurate search direction or no viable search direction generated. Therefore, the QP solver is used to solve LSQ subproblems in such a situation. Several challenging process optimisation problems are solved to demonstrate the advantages of the proposed algorithms over existing solvers including the SLSQP solver in SciPy, fmincon in Matlab and IPOPT.


title: LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logs

abstract: System logs are some of the most important information for the maintenance of software systems, which have become larger and more complex in recent years. The goal of log-based anomaly detection is to automatically detect system anomalies by analyzing the large number of logs generated in a short period of time, which is a critical challenge in the real world. Previous studies have used a log parser to extract templates from unstructured log data and detect anomalies on the basis of patterns of the template occurrences. These methods have limitations for logs with unknown templates. Furthermore, since most log anomalies are known to be point anomalies rather than contextual anomalies, detection methods based on occurrence patterns can cause unnecessary delays in detection. In this paper, we propose LogELECTRA, a new log anomaly detection model that analyzes a single line of log messages more deeply on the basis of self-supervised anomaly detection. LogELECTRA specializes in detecting log anomalies as point anomalies by applying ELECTRA, a natural language processing model, to analyze the semantics of a single line of log messages. LogELECTRA outperformed existing state-of-the-art methods in experiments on the public benchmark log datasets BGL, Sprit, and Thunderbird.


title: Prompt Learning for Multi-Label Code Smell Detection: A Promising Approach

abstract: Code smells indicate the potential problems of software quality so that developers can identify refactoring opportunities by detecting code smells. State-of-the-art approaches leverage heuristics, machine learning, and deep learning to detect code smells. However, existing approaches have not fully explored the potential of large language models (LLMs). In this paper, we propose \textit{PromptSmell}, a novel approach based on prompt learning for detecting multi-label code smell. Firstly, code snippets are acquired by traversing abstract syntax trees. Combined code snippets with natural language prompts and mask tokens, \textit{PromptSmell} constructs the input of LLMs. Secondly, to detect multi-label code smell, we leverage a label combination approach by converting a multi-label problem into a multi-classification problem. A customized answer space is added to the word list of pre-trained language models, and the probability distribution of intermediate answers is obtained by predicting the words at the mask positions. Finally, the intermediate answers are mapped to the target class labels by a verbalizer as the final classification result. We evaluate the effectiveness of \textit{PromptSmell} by answering six research questions. The experimental results demonstrate that \textit{PromptSmell} obtains an improvement of 11.17\% in $precision_{w}$ and 7.4\% in $F1_{w}$ compared to existing approaches.


title: Distributed Model Predictive Control for Cooperative Multirotor Landing on Uncrewed Surface Vessel in Waves

abstract: Heterogeneous autonomous robot teams consisting of multirotor and uncrewed surface vessels (USVs) have the potential to enable various maritime applications, including advanced search-and-rescue operations. A critical requirement of these applications is the ability to land a multirotor on a USV for tasks such as recharging. This paper addresses the challenge of safely landing a multirotor on a cooperative USV in harsh open waters. To tackle this problem, we propose a novel sequential distributed model predictive control (MPC) scheme for cooperative multirotor-USV landing. Our approach combines standard tracking MPCs for the multirotor and USV with additional artificial intermediate goal locations. These artificial goals enable the robots to coordinate their cooperation without prior guidance. Each vehicle solves an individual optimization problem for both the artificial goal and an input that tracks it but only communicates the former to the other vehicle. The artificial goals are penalized by a suitable coupling cost. Furthermore, our proposed distributed MPC scheme utilizes a spatial-temporal wave model to coordinate in real-time a safer landing location and time the multirotor's landing to limit severe tilt of the USV.


title: Chain of Logic: Rule-Based Reasoning with Large Language Models

abstract: Rule-based reasoning, a fundamental type of legal reasoning, enables us to draw conclusions by accurately applying a rule to a set of facts. We explore causal language models as rule-based reasoners, specifically with respect to compositional rules - rules consisting of multiple elements which form a complex logical expression. Reasoning about compositional rules is challenging because it requires multiple reasoning steps, and attending to the logical relationships between elements. We introduce a new prompting method, Chain of Logic, which elicits rule-based reasoning through decomposition (solving elements as independent threads of logic), and recomposition (recombining these sub-answers to resolve the underlying logical expression). This method was inspired by the IRAC (Issue, Rule, Application, Conclusion) framework, a sequential reasoning approach used by lawyers. We evaluate chain of logic across eight rule-based reasoning tasks involving three distinct compositional rules from the LegalBench benchmark and demonstrate it consistently outperforms other prompting methods, including chain of thought and self-ask, using open-source and commercial language models.


title: ManiFPT: Defining and Analyzing Fingerprints of Generative Models

abstract: Recent works have shown that generative models leave traces of their underlying generative process on the generated samples, broadly referred to as fingerprints of a generative model, and have studied their utility in detecting synthetic images from real ones. However, the extend to which these fingerprints can distinguish between various types of synthetic image and help identify the underlying generative process remain under-explored. In particular, the very definition of a fingerprint remains unclear, to our knowledge. To that end, in this work, we formalize the definition of artifact and fingerprint in generative models, propose an algorithm for computing them in practice, and finally study its effectiveness in distinguishing a large array of different generative models. We find that using our proposed definition can significantly improve the performance on the task of identifying the underlying generative process from samples (model attribution) compared to existing methods. Additionally, we study the structure of the fingerprints, and observe that it is very predictive of the effect of different design choices on the generative process.


title: Non-convex optimization problems for maximum hands-off control

abstract: The maximum hands-off control is the optimal solution to the L0 optimal control problem. It has the minimum support length among all feasible control inputs. To avoid computational difficulties arising from its combinatorial nature, the convex approximation method that replaces the L0 norm by the L1 norm in the cost function has been employed on standard. However, this approximation method does not necessarily obtain the maximum hands-off control. In response to this limitation, this paper newly introduces a non-convex approximation method and formulates a class of non-convex optimal control problems that are always equivalent to the maximum hands-off control problem. Based on the results, this paper describes the computation method that quotes algorithms designed for the difference of convex functions optimization. Finally, this paper confirms the effectiveness of the non-convex approximation method with a numerical example.


title: Polyhedral Complex Derivation from Piecewise Trilinear Networks

abstract: Recent advancements in visualizing deep neural networks provide insights into their structures and mesh extraction from Continuous Piecewise Affine (CPWA) functions. Meanwhile, developments in neural surface representation learning incorporate non-linear positional encoding, addressing issues like spectral bias; however, this poses challenges in applying mesh extraction techniques based on CPWA functions. Focusing on trilinear interpolating methods as positional encoding, we present theoretical insights and an analytical mesh extraction, showing the transformation of hypersurfaces to flat planes within the trilinear region under the eikonal constraint. Moreover, we introduce a method for approximating intersecting points among three hypersurfaces contributing to broader applications. We empirically validate correctness and parsimony through chamfer distance and efficiency, and angular distance, while examining the correlation between the eikonal loss and the planarity of the hypersurfaces.


title: Explaining generative diffusion models via visual analysis for interpretable decision-making process

abstract: Diffusion models have demonstrated remarkable performance in generation tasks. Nevertheless, explaining the diffusion process remains challenging due to it being a sequence of denoising noisy images that are difficult for experts to interpret. To address this issue, we propose the three research questions to interpret the diffusion process from the perspective of the visual concepts generated by the model and the region where the model attends in each time step. We devise tools for visualizing the diffusion process and answering the aforementioned research questions to render the diffusion process human-understandable. We show how the output is progressively generated in the diffusion process by explaining the level of denoising and highlighting relationships to foundational visual concepts at each time step through the results of experiments with various visual analyses using the tools. Throughout the training of the diffusion model, the model learns diverse visual concepts corresponding to each time-step, enabling the model to predict varying levels of visual concepts at different stages. We substantiate our tools using Area Under Cover (AUC) score, correlation quantification, and cross-attention mapping. Our findings provide insights into the diffusion process and pave the way for further research into explainable diffusion mechanisms.


title: Theory of Wetting Dynamics with Surface Binding

abstract: Biomolecules, such as proteins and RNAs, can phase separate in the cytoplasm of cells to form biological condensates. Such condensates are liquid-like droplets that can wet biological surfaces such as membranes. Many molecules that can participate in phase separation can also reversibly bind to membrane surfaces. When a droplet wets such a surface, these molecules can diffuse both inside the droplet or in the bound state on the surface. How the interplay between surface binding and surface diffusion affects the wetting kinetics is not well understood. Here, we derive the governing equations using non-equilibrium thermodynamics by relating the diffusive fluxes and forces at the surface coupled to the bulk. We use our theory to study the spreading kinetics in the presence of surface binding and find that binding speeds up wetting by nucleating a droplet inside the surface. Our results are relevant both to artificial systems and to condensates in cells. They suggest that the wetting of droplets in living cells could be regulated by two-dimensional droplets in the surface-bound layer changing the binding affinity to biological surfaces.


title: Signal of phase transition hidden in quasinormal modes of regular AdS black holes

abstract: We discuss the intrinsic relations between thermodynamic phase transitions and quasinormal modes in regular AdS black holes, specifically in the Bardeen and Hayward AdS classes. To this end, we calculate the quasinormal modes of massless scalar field perturbations around small and large black holes via the Horowitz-Hubeny method. By investigating the isobaric and isothermal phase transitions for Bardeen and Hayward AdS black holes in detail, we observe that a dramatic change of quasinormal modes appears near the phase transition point of small and large black holes, and that it corresponds to the swallow tail structure in the plane of Gibbs free energy with respect to pressure. Moreover, by analyzing the evolution of black holes along the coexistence curve of small and large black hole phases, we also observe the dramatic change in quasinormal modes. Such a phenomenon confirms the signal of the phase transition in the quasinormal mode spectrum, which can be understood as a thermodynamic signal hidden in the dynamical spectrum.


title: Nonradiating sources of Maxwell's equations

abstract: This paper presents a thorough investigation into nonradiating sources of Maxwell's equations. Various characterizations are developed to clarify the properties of nonradiating sources, considering their varying degrees of regularity. Furthermore, the characterizations are examined on far-field patterns and near-field data of the electric field, along with the null spaces of integral operators. The study includes the explicit construction of several illustrative examples to demonstrate the presence of nonradiating sources in different null spaces.


title: Study of $B_s \to \phi (\rho^0, \omega)$ Decays in Standard Model and Family Non-universal $Z^\prime$ Model

abstract: Within QCD factorization approach, we employ the new results of form factors of $B_s \to \phi$ and calculate the branching fractions, CP asymmetries (CPAs) and polarization fractions of the decay modes $B_s \to \phi (\rho^0, \omega)$ in both the standard model (SM) and the family non-universal $Z^\prime$ model. We find that in SM the above observables are relate to two parameters $\rho_H$ and $\phi_H$, which characterize the end-point divergence in the hard spectator-scattering amplitudes. When setting $\rho_H=0.5$ and $\phi_H\in [-180,180]^\circ$, the theoretical uncertainties are large. By combining the experimental data, our results can be used to constrain these two parameters. Supposing $\rho_H=0$, we study the effects of $Z^\prime$ boson on the concerned observables. With the available branching fraction of $B_s \to \phi \rho^0$, the possible ranges of parameter spaces are obtained. Within the allowed parameter spaces, the branching fraction of $B_s \to \phi \omega$ can be enlarged remarkably. Furthermore, the new introduced weak phase plays important roles in significantly affecting the CPAs and polarization fractions, which are important observables for probing the effects of NP. If these decay modes were measured in the on-going LHC-b and Belle-II experiments in future, the peculiar deviation from SM could provide a signal of the family non-universal $Z^\prime$ model, which can also be used to constrain the mass of $Z^\prime$ boson in turn.


title: Understanding Survey Paper Taxonomy about Large Language Models via Graph Representation Learning

abstract: As new research on Large Language Models (LLMs) continues, it is difficult to keep up with new research and models. To help researchers synthesize the new research many have written survey papers, but even those have become numerous. In this paper, we develop a method to automatically assign survey papers to a taxonomy. We collect the metadata of 144 LLM survey papers and explore three paradigms to classify papers within the taxonomy. Our work indicates that leveraging graph structure information on co-category graphs can significantly outperform the language models in two paradigms; pre-trained language models' fine-tuning and zero-shot/few-shot classifications using LLMs. We find that our model surpasses an average human recognition level and that fine-tuning LLMs using weak labels generated by a smaller model, such as the GCN in this study, can be more effective than using ground-truth labels, revealing the potential of weak-to-strong generalization in the taxonomy classification task.


title: A MASH simulation of the photoexcited dynamics of cyclobutanone

abstract: In response to a community prediction challenge, we simulate the nonadiabatic dynamics of cyclobutanone using the mapping approach to surface hopping (MASH). We consider the first 500 fs of relaxation following photo-excitation to the S2 state and predict the corresponding time-resolved electron-diffraction signal that will be measured by the planned experiment. 397 ab-initio trajectories were obtained on the fly with state-averaged complete active space self-consistent field (SA-CASSCF) using a (12,11) active space. To obtain an estimate of the potential systematic error 198 of the trajectories were calculated using an aug-cc-pVDZ basis set and 199 with a 6-31+G* basis set. MASH is a recently proposed independent trajectory method for simulating nonadiabatic dynamics, originally derived for two-state problems. As there are three relevant electronic states in this system, we used a newly developed multi-state generalisation of MASH for the simulation: the uncoupled spheres multi-state MASH method (unSMASH). This study therefore serves both as an investigation of the photo-dissociation dynamics of cyclobutanone, and also as a demonstration of the applicability of unSMASH to ab-initio simulations. In line with previous experimental studies, we observe that the simulated dynamics is dominated by three sets of dissociation products, C3H6+CO, C2H4+C2H2O and C2H4+CH2+CO, and we interpret our predicted electron-diffraction signal in terms of the key features of the associated dissociation pathways.


title: Continuous-variable quantum key distribution over 28.6 km fiber with an integrated silicon photonic receiver chip

abstract: Quantum key distribution, which ensures information-theoretically secret key generation, is currently advancing through photonic integration to achieve high performance, cost reduction and compact size, thereby facilitating the large-scale deployment. Continuous-variable quantum key distribution is an attractive approach for photonic integrations due to its compatibility with off-the-shelf optical communication devices. However, its chip-based systems have encountered significant limitations primarily related to the shot-noise-limited receiver design, which demands low noise, wide bandwidth, high clearance and well stability. Here, we report the implementation of a real local oscillator continuous-variable quantum key distribution system with an integrated silicon photonic receiver chip. Thanks to the well-designed chip-based homodyne detectors with a bandwidth up to 1.5 GHz and a clearance up to 7.42 dB, the transmission distance of the system has been extended to 28.6 km, achieving a secret key generation rate of Mbps level. This technological advancement enables the quantum key distribution systems with photonic integrated receivers to achieve the coverage in both access network scenarios and short-distance metropolitan interconnections, paving the way for the development of the next-generation quantum key distribution networks on a large scale.


title: Measuring and Reducing LLM Hallucination without Gold-Standard Answers

abstract: LLM hallucination, i.e. generating factually incorrect yet seemingly convincing answers, is currently a major threat to the trustworthiness and reliability of LLMs. The first step towards solving this complicated problem is to measure it. However, existing hallucination metrics require having a benchmark dataset with gold-standard answers, i.e. "best" or "correct" answers written by humans. Such requirements make hallucination measurement costly and prone to human errors. In this work, we propose Factualness Evaluations via Weighting LLMs (FEWL), an innovative hallucination metric that is specifically designed for the scenario when gold-standard answers are absent. FEWL leverages the answers from off-the-shelf LLMs that serve as a proxy of gold-standard answers. The key challenge is how to quantify the expertise of reference LLMs resourcefully. We show FEWL has certain theoretical guarantees and demonstrate empirically it gives more accurate hallucination measures than naively using reference LLMs. We also show how to leverage FEWL to reduce hallucination through both in-context learning and supervised fine-tuning. Extensive experiment results on Truthful-QA, CHALE, and HaluEval datasets demonstrate the effectiveness of FEWL.


title: Well-posedness of a Pseudo-Parabolic KWC System in Materials Science

abstract: The original KWC-system is widely used in materials science. It was proposed in [Kobayashi et al, Physica D, 140, 141--150 (2000)] and is based on the phase field model of planar grain boundary motion. This model suffers from two key challenges. Firstly, it is difficult to establish its relation to physics, in particular, a variational model. Secondly, it lacks uniqueness. The former has been recently studied within the realm of BV-theory. The latter only holds under various simplifications. This article introduces a pseudo-parabolic version of the KWC-system. A direct relationship with variational model (as gradient-flow) and uniqueness are established without making any unrealistic simplifications. Namely, this is the first KWC-system which is both physically and mathematically valid. The proposed model overcomes the well-known open issues.


title: The weakness of soft X-ray intensity: possible physical reason for weak line quasars

abstract: Weak-line quasars (WLQs) are a notable group of active galactic nuclei (AGNs) that show unusually weak UV lines even though their optical-UV continuum shapes are similar to those of typical quasars. The physical mechanism for WLQs is an unsolved puzzle in the AGN unified model. We explore the properties of UV emission lines by performing extensive photoionization calculations based on Cloudy simulation with different spectral energy distributions (SEDs) of AGNs. The AGN continua are built from several observational empirical correlations, where the black-body emission from the cold disk, the power-law emission from the hot corona, and a soft X-ray excess component are considered. We find that the equivalent width (EW) of C {\footnotesize IV} from our models is systematically lower than observational values if the component of soft X-ray excess is neglected. The EW will increase several times and is roughly consistent with the observations after considering the soft X-ray excess component as constrained from normal type I AGNs. We find that the UV lines are weak for QSOs with quite large BH mass (e.g., $M_{\rm BH}>10^9M_{\odot}$) and weak soft X-ray emission due to the deficit of ionizing photons. As an example, we present the strength of C {\footnotesize IV} based on the multi-band SEDs for three nearby weak-line AGNs, where the weaker soft X-ray emission normally predicts the weaker lines.


title: GTC Spectroscopic Surveys of Planetary Nebulae in the Milky Way and M31

abstract: We report spectroscopic surveys of planetary nebulae (PNe) in the Milky Way and Andromeda (M31), using the 10.4-m Gran Telescopio Canarias (GTC). The spectra are of high quality and cover the whole optical range, mostly from 3650 \r{A} to beyond 1 micron, enabling detection of nebular emission lines critical for spectral analysis as well as photoionization modeling. We obtained GTC spectra of 24 compact (angular diameter <5 arcsec) PNe located in the Galactic disk, ~3-20 kpc from the Galactic centre, and can be used to constrain stellar evolution models and derive radial abundance gradients of the Milky Way. We have observed 30 PNe in the outer halo of M31 using the GTC. These halo PNe are uniformly metal-rich and probably all evolved from low-mass stars, consistent with the conjecture that they all formed from the metal-rich gas in M31 disk but displaced to their present locations due to galaxy interactions.


title: Grounding Language about Belief in a Bayesian Theory-of-Mind

abstract: Despite the fact that beliefs are mental states that cannot be directly observed, humans talk about each others' beliefs on a regular basis, often using rich compositional language to describe what others think and know. What explains this capacity to interpret the hidden epistemic content of other minds? In this paper, we take a step towards an answer by grounding the semantics of belief statements in a Bayesian theory-of-mind: By modeling how humans jointly infer coherent sets of goals, beliefs, and plans that explain an agent's actions, then evaluating statements about the agent's beliefs against these inferences via epistemic logic, our framework provides a conceptual role semantics for belief, explaining the gradedness and compositionality of human belief attributions, as well as their intimate connection with goals and plans. We evaluate this framework by studying how humans attribute goals and beliefs while watching an agent solve a doors-and-keys gridworld puzzle that requires instrumental reasoning about hidden objects. In contrast to pure logical deduction, non-mentalizing baselines, and mentalizing that ignores the role of instrumental plans, our model provides a much better fit to human goal and belief attributions, demonstrating the importance of theory-of-mind for a semantics of belief.


title: Entanglement Degradation in Causal Diamonds

abstract: Entanglement degradation appears to be a generic prediction in relativistic quantum information whenever horizons restrict access to a region of spacetime. This property has been previously explored in connection with the Unruh effect, where a bipartite entangled system composed of an inertial observer (Alice) and a uniformly accelerated observer (Rob) was studied, with entanglement degradation caused by the relative acceleration -- and with equivalent results for the case when Alice is freely falling into a black hole and Rob experiences a constant proper acceleration as a stationary near-horizon observer. In this work, we show that a similar degradation also occurs in the case of an entangled system composed of an inertial observer (Alice) and a "diamond observer" (Dave) with a finite lifetime. The condition of a finite lifetime is equivalent to the restriction of Dave's access within a causal diamond. Specifically, if the system starts in a maximally entangled state, prepared from Alice's perspective, entanglement degradation is enforced by the presence of the diamond's causal horizons.


title: A Distributionally Robust Estimator that Dominates the Empirical Average

abstract: We leverage the duality between risk-averse and distributionally robust optimization (DRO) to devise a distributionally robust estimator that strictly outperforms the empirical average for all probability distributions with negative excess kurtosis. The aforesaid estimator solves the $\chi^{2}-$robust mean squared error problem in closed form.


title: Path Loss Modeling for RIS-Assisted Wireless System with Direct Link and Elevation Factors

abstract: The present path loss models for wireless systems employing reconfigurable intelligent surfaces (RIS) do not account for the elevation of the transmitter, receiver, and RIS module. In this paper, we develop an analytical model for path loss of a wireless system utilizing an NxM-element RIS module positioned above the ground surface with elevated transmitter and receiver configurations. Furthermore, we integrate the direct link into the path loss model to enhance its applicability, a crucial aspect often neglected in previous research. We also present simplified analytical expressions for path loss under various configurations, including near-field and far-field scenarios. These expressions elucidate the impact of elevation factors on path loss, facilitating more accurate signal quality estimation at the receiver. Simulation results corroborate that accounting for elevated RIS modules and transceiver units can yield improved deployment strategies for RIS-based wireless systems.


title: The universal Lipschitz path space of the Heisenberg group $\mathbb{H}^1$

abstract: The goal of this paper is to define and inspect a metric version of the universal path space and study its application to purely 2-unrectifiable spaces, in particular the Heisenberg group $\mathbb{H}^1$. The construction of the universal Lipschitz path space, as the metric version is called, echoes the construction of the universal cover for path-connected, locally path-connected, and semilocally simply connected spaces. We prove that the universal Lipschitz path space of a purely 2-unrectifiable space, much like the universal cover, satisfies a unique lifting property, a universal property, and is Lipschitz simply connected. The existence of such a universal Lipschitz path space of $\mathbb{H}^1$ will be used to prove that $\pi_{1}^{\text{Lip}}(\mathbb{H}^1)$ is torsion-free in a subsequent paper.


title: Recurrent Neural Networks for Multivariate Loss Reserving and Risk Capital Analysis

abstract: Reserves comprise most of the liabilities of a property and casualty (P&C) company and are actuaries' best estimate for unpaid future claims. Notably, the reserves for different lines of business (LOB) are related, as there may be dependence between events related to claims. There have been parametric and non-parametric methods in the actuarial industry for loss reserving; only a few tools have been developed to use the recurrent neural network (RNN) for multivariate loss reserving and risk capital analyses. This paper aims to study RNN methods to model dependence between loss triangles and develop predictive distribution for reserves using machine learning. Thus, we create an RNN model to capture dependence between LOBs by extending the Deep Triangle (DT) model from Kuo (2019). In the extended Deep Triangle (EDT), we use the incremental paid loss from two LOBs as input and the symmetric squared loss of two LOBs as the loss function. Then, we extend generative adversarial networks (GANs) by transforming the two loss triangles into a tabular format and generating synthetic loss triangles to obtain the predictive distribution for reserves. To illustrate our method, we apply and calibrate these methods on personal and commercial automobile lines from a large US P&C insurance company and compare the results with copula regression models. The results show that the EDT model performs better than the copula regression models in predicting total loss reserve. In addition, with the obtained predictive distribution for reserves, we show that risk capitals calculated from EDT combined with GAN are smaller than that of the copula regression models, which implies a more considerable diversification benefit. Finally, these findings are also confirmed in a simulation study.


title: Pushing the Limits of Zero-shot End-to-End Speech Translation

abstract: Data scarcity and the modality gap between the speech and text modalities are two major obstacles of end-to-end Speech Translation (ST) systems, thus hindering their performance. Prior work has attempted to mitigate these challenges by leveraging external MT data and optimizing distance metrics that bring closer the speech-text representations. However, achieving competitive results typically requires some ST data. For this reason, we introduce ZeroSwot, a method for zero-shot ST that bridges the modality gap without any paired ST data. Leveraging a novel CTC compression and Optimal Transport, we train a speech encoder using only ASR data, to align with the representation space of a massively multilingual MT model. The speech encoder seamlessly integrates with the MT model at inference, enabling direct translation from speech to text, across all languages supported by the MT model. Our experiments show that we can effectively close the modality gap without ST data, while our results on MuST-C and CoVoST demonstrate our method's superiority over not only previous zero-shot models, but also supervised ones, achieving state-of-the-art results.


title: Connect the dots: Dataset Condensation, Differential Privacy, and Adversarial Uncertainty

abstract: Our work focuses on understanding the underpinning mechanism of dataset condensation by drawing connections with ($\epsilon$, $\delta$)-differential privacy where the optimal noise, $\epsilon$, is chosen by adversarial uncertainty \cite{Grining2017}. We can answer the question about the inner workings of the dataset condensation procedure. Previous work \cite{dong2022} proved the link between dataset condensation (DC) and ($\epsilon$, $\delta$)-differential privacy. However, it is unclear from existing works on ablating DC to obtain a lower-bound estimate of $\epsilon$ that will suffice for creating high-fidelity synthetic data. We suggest that adversarial uncertainty is the most appropriate method to achieve an optimal noise level, $\epsilon$. As part of the internal dynamics of dataset condensation, we adopt a satisfactory scheme for noise estimation that guarantees high-fidelity data while providing privacy.


title: Understanding In-Context Learning with a Pelican Soup Framework

abstract: Many existing theoretical analyses of in-context learning for natural language processing are based on latent variable models that leaves gaps between theory and practice. We aim to close these gaps by proposing a theoretical framework, the Pelican Soup Framework. In this framework, we introduce (1) the notion of a common sense knowledge base, (2) a general formalism for natural language classification tasks, and the notion of (3) meaning association. Under this framework, we can establish a $\mathcal{O}(1/T)$ loss bound for in-context learning, where $T$ is the number of example-label pairs in the demonstration. Compared with previous works, our bound reflects the effect of the choice of verbalizers and the effect of instruction tuning. An additional notion of \textit{atom concepts} makes our framework possible to explain the generalization to tasks unseen in the language model training data. Finally, we propose a toy setup, Calcutec, and a digit addition task that mimics types of distribution shifts a model needs to overcome to perform in-context learning. We also experiment with GPT2-Large on real-world NLP tasks. Our empirical results demonstrate the efficacy of our framework to explain in-context learning.


title: DABS-LS: Deep Atlas-Based Segmentation Using Regional Level Set Self-Supervision

abstract: Cochlear implants (CIs) are neural prosthetics used to treat patients with severe-to-profound hearing loss. Patient-specific modeling of CI stimulation of the auditory nerve fiber (ANFs) can help audiologists improve the CI programming. These models require localization of the ANFs relative to surrounding anatomy and the CI. Localization is challenging because the ANFs are so small they are not directly visible in clinical imaging. In this work, we hypothesize the position of the ANFs can be accurately inferred from the location of the internal auditory canal (IAC), which has high contrast in CT, since the ANFs pass through this canal between the cochlea and the brain. Inspired by VoxelMorph, in this paper we propose a deep atlas-based IAC segmentation network. We create a single atlas in which the IAC and ANFs are pre-localized. Our network is trained to produce deformation fields (DFs) mapping coordinates from the atlas to new target volumes and that accurately segment the IAC. We hypothesize that DFs that accurately segment the IAC in target images will also facilitate accurate atlas-based localization of the ANFs. As opposed to VoxelMorph, which aims to produce DFs that accurately register the entire volume, our novel contribution is an entirely self-supervised training scheme that aims to produce DFs that accurately segment the target structure. This self-supervision is facilitated using a regional level set (LS) inspired loss function. We call our method Deep Atlas Based Segmentation using Level Sets (DABS-LS). Results show that DABS-LS outperforms VoxelMorph for IAC segmentation. Tests with publicly available datasets for trachea and kidney segmentation also show significant improvement in segmentation accuracy, demonstrating the generalizability of the method.


title: DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection

abstract: Large language models are limited by challenges in factuality and hallucinations to be directly employed off-the-shelf for judging the veracity of news articles, where factual accuracy is paramount. In this work, we propose DELL that identifies three key stages in misinformation detection where LLMs could be incorporated as part of the pipeline: 1) LLMs could \emph{generate news reactions} to represent diverse perspectives and simulate user-news interaction networks; 2) LLMs could \emph{generate explanations} for proxy tasks (e.g., sentiment, stance) to enrich the contexts of news articles and produce experts specializing in various aspects of news understanding; 3) LLMs could \emph{merge task-specific experts} and provide an overall prediction by incorporating the predictions and confidence scores of varying experts. Extensive experiments on seven datasets with three LLMs demonstrate that DELL outperforms state-of-the-art baselines by up to 16.8\% in macro f1-score. Further analysis reveals that the generated reactions and explanations are greatly helpful in misinformation detection, while our proposed LLM-guided expert merging helps produce better-calibrated predictions.


title: Evaluating and Improving Continual Learning in Spoken Language Understanding

abstract: Continual learning has emerged as an increasingly important challenge across various tasks, including Spoken Language Understanding (SLU). In SLU, its objective is to effectively handle the emergence of new concepts and evolving environments. The evaluation of continual learning algorithms typically involves assessing the model's stability, plasticity, and generalizability as fundamental aspects of standards. However, existing continual learning metrics primarily focus on only one or two of the properties. They neglect the overall performance across all tasks, and do not adequately disentangle the plasticity versus stability/generalizability trade-offs within the model. In this work, we propose an evaluation methodology that provides a unified evaluation on stability, plasticity, and generalizability in continual learning. By employing the proposed metric, we demonstrate how introducing various knowledge distillations can improve different aspects of these three properties of the SLU model. We further show that our proposed metric is more sensitive in capturing the impact of task ordering in continual learning, making it better suited for practical use-case scenarios.


title: Inversion of Adjunction for the minimal exponent

abstract: We prove that the minimal exponent for local complete intersections satisfies an Inversion-of-Adjunction property. As a result, we also obtain the Inversion of Adjunction for higher Du Bois and higher rational singularities for local complete intersections.


title: Fixed Confidence Best Arm Identification in the Bayesian Setting

abstract: We consider the fixed-confidence best arm identification (FC-BAI) problem in the Bayesian setting. This problem aims to find the arm of the largest mean with a fixed confidence level when the bandit model has been sampled from the known prior. Most studies on the FC-BAI problem have been conducted in the frequentist setting, where the bandit model is predetermined before the game starts. We show that the traditional FC-BAI algorithms studied in the frequentist setting, such as track-and-stop and top-two algorithms, result in arbitrarily suboptimal performances in the Bayesian setting. We also obtain a lower bound of the expected number of samples in the Bayesian setting and introduce a variant of successive elimination that has a matching performance with the lower bound up to a logarithmic factor. Simulations verify the theoretical results.


title: Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models

abstract: Instruction-tuning language models has become a crucial step in aligning them for general use. Typically, this process involves extensive training on large datasets, incurring high training costs. In this paper, we introduce a novel training data selection based on the learning percentage of the samples. We assert that current language models possess the capability to autonomously select high-quality training data, leading to comparable or improved performance compared to training on the entire dataset. Our experiments span different-sized models, revealing that this characteristic holds for models ranging from 1B (small) to 13B (large) in size. Moreover, we demonstrate an interesting finding that the data hardness transfers across model sizes, and a smaller 350M model can effectively curate high-quality training data with hard samples for a larger 13B model, resulting in an equally or superior instruction-tuned model compared to training on the complete dataset. Utilizing open-sourced OPT and Llama-2 models up to 13B in size, two publicly available instruction-tuning training datasets and evaluated by both automatic metrics & humans, our paper introduces a novel approach to training data selection, showcasing a more efficient alternative.


title: Spatial Variations of Dust Opacity and Grain Growth in Dark Clouds: L1689, L1709 and L1712

abstract: The far-infrared (FIR) opacity of dust in dark clouds within the Ophiuchus molecular cloud is investigated through multi-wavelength infrared observations from UKIDSS, Spitzer and Herschel. Employing the infrared color excess technique with both near-infrared (NIR) and mid-infrared (MIR) photometric data, a high-resolution extinction map in the $K$ band ($A_K$) is constructed for three dark clouds: L1689, L1709, and L1712. The derived extinction map has a resolution of $1'$ and reaches a depth of $A_K\sim3$ mag. The FIR optical depths $\tau_{250}$ at a reference wavelength of $250\,\rm \mu m$ are obtained by fitting the Herschel PACS and SPIRE continuum data at 100, 160, 250, 350 and 500 $\rm \mu m$ using a modified blackbody model. The average dust opacity per unit gas mass at $250\rm \mu m$, $r\kappa_{250}$ is determined through a pixel-by-pixel correlation of $\tau_{250}$ with $A_K$, yielding a value of approximately $0.09\,\rm cm^2\,g^{-1}$, which is about 2-3 times higher than the typical value in the diffuse interstellar medium (ISM). Additionally, an independent analysis across 16 sub-regions within the Ophiuchus cloud indicates spatial variations in dust opacity, with values ranging from 0.07-0.12$\,\rm cm^2\,g^{-1}$. Although the observed trend of increasing dust opacity with higher extinction implies grain growth, our findings indicate that rapid grain growth clearly not yet occurred in the dark clouds studied in this work.