title: Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition

abstract: In this article, we introduce a novel problem of audio-visual autism behavior recognition, which includes social behavior recognition, an essential aspect previously omitted in AI-assisted autism screening research. We define the task at hand as one that is audio-visual autism behavior recognition, which uses audio and visual cues, including any speech present in the audio, to recognize autism-related behaviors. To facilitate this new research direction, we collected an audio-visual autism spectrum dataset (AV-ASD), currently the largest video dataset for autism screening using a behavioral approach. It covers an extensive range of autism-associated behaviors, including those related to social communication and interaction. To pave the way for further research on this new problem, we intensively explored leveraging foundation models and multimodal large language models across different modalities. Our experiments on the AV-ASD dataset demonstrate that integrating audio, visual, and speech modalities significantly enhances the performance in autism behavior recognition. Additionally, we explored the use of a post-hoc to ad-hoc pipeline in a multimodal large language model to investigate its potential to augment the model's explanatory capability during autism behavior recognition. We will release our dataset, code, and pre-trained models.


title: PhoWhisper: Automatic Speech Recognition for Vietnamese

abstract: We introduce PhoWhisper in five versions for Vietnamese automatic speech recognition. PhoWhisper's robustness is achieved through fine-tuning the Whisper model on an 844-hour dataset that encompasses diverse Vietnamese accents. Our experimental study demonstrates state-of-the-art performances of PhoWhisper on benchmark Vietnamese ASR datasets. We have open-sourced PhoWhisper at: https://github.com/VinAIResearch/PhoWhisper


title: Review on the minimally extended varying speed of light model

abstract: It has been known that dimensional constants such as $\hbar$, $c$, $G$, $e$, and $k$ are merely human constructs whose values and units vary depending on the chosen system of measurement. Therefore, the time variation of dimensional constants lacks operational significance due to their dependence on them. It is well-structured and represents a valid discussion. However, this fact only becomes a meaningful debate within the context of a static or present universe. As well-established theoretically and observationally, the current universe is undergoing accelerated expansion, wherein dimensional quantities, like the wavelength of light, also experience redshift phenomena elongating over cosmic time. In other words, in an expanding universe, dimensional quantities of physical parameters vary with cosmic time. From this perspective, there exists the possibility that dimensional constants, such as the speed of light, could vary with the expansion of the universe. In this review paper, we contemplate under what circumstances the speed of light may change or remain constant over cosmic time, and discuss the potential for distinguishing these cases observationally.


title: EVAN: Evolutional Video Streaming Adaptation via Neural Representation

abstract: Adaptive bitrate (ABR) using conventional codecs cannot further modify the bitrate once a decision has been made, exhibiting limited adaptation capability. This may result in either overly conservative or overly aggressive bitrate selection, which could cause either inefficient utilization of the network bandwidth or frequent re-buffering, respectively. Neural representation for video (NeRV), which embeds the video content into neural network weights, allows video reconstruction with incomplete models. Specifically, the recovery of one frame can be achieved without relying on the decoding of adjacent frames. NeRV has the potential to provide high video reconstruction quality and, more importantly, pave the way for developing more flexible ABR strategies for video transmission. In this work, a new framework, named Evolutional Video streaming Adaptation via Neural representation (EVAN), which can adaptively transmit NeRV models based on soft actor-critic (SAC) reinforcement learning, is proposed. EVAN is trained with a more exploitative strategy and utilizes progressive playback to avoid re-buffering. Experiments showed that EVAN can outperform existing ABRs with 50% reduction in re-buffering and achieve nearly 20% .


title: Evaluation of Vortex Criteria by Virtue of the Quadruple Decomposition of Velocity Gradient Tensor

abstract: Based on the analysis of the velocity gradient tensor, we investigate in this paper the physical interpretation and limitations of four vortex criteria: $\omega$, $Q$, $\varDelta$ and $\lambda_{ci}$, and reveal the actual physical meaning of vortex patterns which are usually illustrated by level sets of various vortex criteria. A quadruple decomposition based on the normality of the velocity gradient tensor is proposed for the first time, which resolves the motion of a fluid element into dilation, axial stretch along the normal frame, in-plane distortion, and simple shear, in order to clarify the kinematical interpretation of various vortex criteria. The mean rotation characterized by the vorticity $\omega$ always consists of simple shear; the $Q$-criterion can reflect the strength of net rotation within the invariant plane relative to the axial stretch of a fluid element, and it is a sufficient but unnecessary condition for the existence of net rotation; the $\varDelta$-criterion can exactly identify the existence of net rotation, but it is not the strength of net rotation; in the case that net rotation exists, $\lambda_{ci}$ characterize its absolute strength. net rotation is the total effect of the normal rotation within the invariant plane and the simple shear, where the former is the most basic rotation. The newly introduced quadruple decomposition can improve our understanding of vortices in fluids and their motions.


title: ShadowRefiner: Towards Mask-free Shadow Removal via Fast Fourier Transformer

abstract: Shadow-affected images often exhibit pronounced spatial discrepancies in color and illumination, consequently degrading various vision applications including object detection and segmentation systems. To effectively eliminate shadows in real-world images while preserving intricate details and producing visually compelling outcomes, we introduce a mask-free Shadow Removal and Refinement network (ShadowRefiner) via Fast Fourier Transformer. Specifically, the Shadow Removal module in our method aims to establish effective mappings between shadow-affected and shadow-free images via spatial and frequency representation learning. To mitigate the pixel misalignment and further improve the image quality, we propose a novel Fast-Fourier Attention based Transformer (FFAT) architecture, where an innovative attention mechanism is designed for meticulous refinement. Our method wins the championship in the Perceptual Track and achieves the second best performance in the Fidelity Track of NTIRE 2024 Image Shadow Removal Challenge. Besides, comprehensive experiment result also demonstrate the compelling effectiveness of our proposed method. The code is publicly available: https://github.com/movingforward100/Shadow_R.


title: Less Peaky and More Accurate CTC Forced Alignment by Label Priors

abstract: Connectionist temporal classification (CTC) models are known to have peaky output distributions. Such behavior is not a problem for automatic speech recognition (ASR), but it can cause inaccurate forced alignments (FA), especially at finer granularity, e.g., phoneme level. This paper aims at alleviating the peaky behavior for CTC and improve its suitability for forced alignment generation, by leveraging label priors, so that the scores of alignment paths containing fewer blanks are boosted and maximized during training. As a result, our CTC model produces less peaky posteriors and is able to more accurately predict the offset of the tokens besides their onset. It outperforms the standard CTC model and a heuristics-based approach for obtaining CTC's token offset timestamps by 12-40% in phoneme and word boundary errors (PBE and WBE) measured on the Buckeye and TIMIT data. Compared with the most widely used FA toolkit Montreal Forced Aligner (MFA), our method performs similarly on PBE/WBE on Buckeye, yet falls behind MFA on TIMIT. Nevertheless, our method has a much simpler training pipeline and better runtime efficiency. Our training recipe and pretrained model are released in TorchAudio.


title: Breaking Walls: Pioneering Automatic Speech Recognition for Central Kurdish: End-to-End Transformer Paradigm

abstract: Automatic Speech Recognition (ASR), as an interesting field of speech processing, is utilized in real applications and is implemented using various techniques amongst which the artificial neural network is the most popular. Increasing the performance, making these systems robust to noise and developing this technology for low-resource languages is among the current challenges. This paper addresses the development of an ASR system for the Central Kurdish language (CKB), as a low-resource language, using end to end transformers. Kurdish, as an Indo-European language, is categorized into three main dialects, i.e., Central Kurdish (i.e., Sorani), North Kurdish (Kirmanji), and South Kurdish which is spoken by more than 30 million people. In this research, a speech corpus of size 224 hours is collected using various sources. Then, this corpus is used to train the transformer-based acoustic model. A transfer learning technique is also utilized in training acoustic models. As a result of these efforts, our optimal model attains state-of-the-art results on the Asosoft test set, achieving a Word Error Rate (WER) of 13%. This accomplishment signifies a notable advancement in ASR technology for the Central Kurdish language, particularly in the context of low-resource languages.


title: Gated Low-rank Adaptation for personalized Code-Switching Automatic Speech Recognition on the low-spec devices

abstract: In recent times, there has been a growing interest in utilizing personalized large models on low-spec devices, such as mobile and CPU-only devices. However, utilizing a personalized large model in the on-device is inefficient, and sometimes limited due to computational cost. To tackle the problem, this paper presents the weights separation method to minimize on-device model weights using parameter-efficient fine-tuning methods. Moreover, some people speak multiple languages in an utterance, as known as code-switching, the personalized ASR model is necessary to address such cases. However, current multilingual speech recognition models are limited to recognizing a single language within each utterance. To tackle this problem, we propose code-switching speech recognition models that incorporate fine-tuned monolingual and multilingual speech recognition models. Additionally, we introduce a gated low-rank adaptation(GLoRA) for parameter-efficient fine-tuning with minimal performance degradation. Our experiments, conducted on Korean-English code-switching datasets, demonstrate that fine-tuning speech recognition models for code-switching surpasses the performance of traditional code-switching speech recognition models trained from scratch. Furthermore, GLoRA enhances parameter-efficient fine-tuning performance compared to conventional LoRA.


title: A cost minimization approach to fix the vocabulary size in a tokenizer for an End-to-End ASR system

abstract: Unlike hybrid speech recognition systems where the use of tokens was restricted to phones, biphones or triphones the choice of tokens in the end-to-end ASR systems is derived from the text corpus of the training data. The use of tokenization algorithms like Byte Pair Encoding (BPE) and WordPiece is popular in identifying the tokens that are used in the overall training process of the speech recognition system. Popular toolkits, like ESPNet use a pre-defined vocabulary size (number of tokens) for these tokenization algorithms, but there is no discussion on how vocabulary size was derived. In this paper, we build a cost function, assuming the tokenization process to be a black-box to enable choosing the number of tokens which might most benefit building an end-to-end ASR. We show through experiments on LibriSpeech 100 hour set that the performance of an end-to-end ASR system improves when the number of tokens are chosen carefully.


title: Two-layer baroclinic turbulence with arbitrary layer depths

abstract: While heat transport by baroclinic turbulence in oceans and planetary atmospheres is well described by a two-layer model, the relative depth of the two layers varies greatly depending on the situation of interest, making it an important parameter governing the transport properties of the system. Focusing on the low-drag turbulent regime, we extend the vortex-gas scaling theory to address the case of arbitrary layer depths. To wit, we map the arbitrary-layer-depth system onto an equivalent equal-depth system with rescaled parameters, establishing the asymptotic validity of the mapping for weak bottom drag. This approach leads to quantitative predictions for the turbulent transport by two-layer baroclinic turbulence with arbitrary layer depths, without additional free parameters. We validate these predictions using an extended suite of numerical simulations with either linear or quadratic bottom drag.


title: Sequence-to-sequence models in peer-to-peer learning: A practical application

abstract: This paper explores the applicability of sequence-to-sequence (Seq2Seq) models based on LSTM units for Automatic Speech Recognition (ASR) task within peer-to-peer learning environments. Leveraging two distinct peer-to-peer learning methods, the study simulates the learning process of agents and evaluates their performance in ASR task using two different ASR datasets. In a centralized training setting, utilizing a scaled-down variant of the Deep Speech 2 model, a single model achieved a Word Error Rate (WER) of 84\% when trained on the UserLibri dataset, and 38\% when trained on the LJ Speech dataset. Conversely, in a peer-to-peer learning scenario involving 55 agents, the WER ranged from 87\% to 92\% for the UserLibri dataset, and from 52\% to 56\% for the LJ Speech dataset. The findings demonstrate the feasibility of employing Seq2Seq models in decentralized settings, albeit with slightly higher Word Error Rates (WER) compared to centralized training methods.


title: Combining X-Vectors and Bayesian Batch Active Learning: Two-Stage Active Learning Pipeline for Speech Recognition

abstract: Emphasizing a data-centric AI approach, this paper introduces a novel two-stage active learning (AL) pipeline for automatic speech recognition (ASR), combining unsupervised and supervised AL methods. The first stage utilizes unsupervised AL by using x-vectors clustering for diverse sample selection from unlabeled speech data, thus establishing a robust initial dataset for the subsequent supervised AL. The second stage incorporates a supervised AL strategy, with a batch AL method specifically developed for ASR, aimed at selecting diverse and informative batches of samples. Here, sample diversity is also achieved using x-vectors clustering, while the most informative samples are identified using a Bayesian AL method tailored for ASR with an adaptation of Monte Carlo dropout to approximate Bayesian inference. This approach enables precise uncertainty estimation, thereby enhancing ASR model training with significantly reduced data requirements. Our method has shown superior performance compared to competing methods on homogeneous, heterogeneous, and OOD test sets, demonstrating that strategic sample selection and innovative Bayesian modeling can substantially optimize both labeling effort and data utilization in deep learning-based ASR applications.


title: Tunable drag drop via flow-induced snap-through in origami

abstract: We leverage the snap-through response of a bistable origami mechanism to induce a discontinuous evolution of drag with flow speed. The transition between equilibrium states is passively actuated by airflow, and we demonstrate that large shape reconfiguration over a small increment of flow velocity leads to a pronounced and sudden drop in drag. Moreover, we show that systematically varying the geometrical and mechanical properties of the origami unit enables the tuning of this drag discontinuity and the critical speed and loading at which it occurs. Experimental results are supported by a theoretical aero-elastic model, which further guides inverse design to identify the combination of structural origami parameters for targeted drag collapse. This approach sheds light on harnessing origami-inspired mechanisms for efficient passive drag control in a fluid environment, applicable for load alleviation or situations requiring swift transitions in aerodynamic performances.


title: Optimising microstructural characterisation of white-matter phantoms: impact of gradient waveform modulation on Non-uniform Oscillating Gradient Spin-Echo sequences

abstract: Changes in the nervous system due to neurological diseases take place at very small spatial scales, on the order of the micro and nanometers. Developing non-invasive imaging methods for obtaining this microscopic information as quantitative biomarkers is therefore crucial for improved medical diagnosis. In this context, diffusion-weighted magnetic resonance imaging has shown significant advances in revealing tissue microstructural features by probing molecular diffusion processes. Implementing modulated gradient spin-echo sequences allows monitoring time-dependent diffusion processes to reveal such detailed information. In particular, one of those sequences termed Non-uniform Oscillating Gradient Spin-Echo (NOGSE), has shown to selectively characterise microstructure sizes by generating an image contrast based on a signal decay-shift rather than on the conventionally used signal decay rate. In this work, we prove that such decay-shift is more pronounced with instantaneous switches of the magnetic field gradient strength sign. As fast gradient ramps need to be avoided in clinical settings, due to potential patient discomfort and artefacts in imaging, we evaluate the method's efficacy for estimating microstructure sizes using both idealised, sharp gradient modulations and more realistic, smooth modulations. In this more realistic scenario, we find that the signal decay shift might be lost as the diffusion time increases, likely hindering the accurate estimation of microstructural characteristics. We demonstrate, by a combination of numerical simulations, information theory analysis and proof-of-principle experiments with white-matter phantoms, that optimal sequence design to estimate microstructure size distributions can be achieved using either sharp or smooth gradient spin-echo modulations. This approach simplifies the translation of the NOGSE method for its use in clinical settings.


title: Cluster-to-Predict Affect Contours from Speech

abstract: Continuous emotion recognition (CER) aims to track the dynamic changes in a person's emotional state over time. This paper proposes a novel approach to translating CER into a prediction problem of dynamic affect-contour clusters from speech, where the affect-contour is defined as the contour of annotated affect attributes in a temporal window. Our approach defines a cluster-to-predict (C2P) framework that learns affect-contour clusters, which are predicted from speech with higher precision. To achieve this, C2P runs an unsupervised iterative optimization process to learn affect-contour clusters by minimizing both clustering loss and speech-driven affect-contour prediction loss. Our objective findings demonstrate the value of speech-driven clustering for both arousal and valence attributes. Experiments conducted on the RECOLA dataset yielded promising classification results, with F1 scores of 0.84 for arousal and 0.75 for valence in our four-class speech-driven affect-contour prediction model.


title: Conductivity and Electrode Placement Variability: Evaluating the Cramer-Rao Lower Bound in 2D Electrical Impedance Tomography

abstract: Electrical Impedance Tomography (EIT) is a non-invasive imaging modality that has earned significant attention for its potential in real-time monitoring of various physiological parameters. Despite its promise, the sensitivity and resolution of EIT remain areas of active research. This article presents a new way to analyse the sensitivity of EIT systems using the Cramer-Rao Lower Bound (CRLB). This statistical tool provides a theoretical lower bound on the variance of unbiased estimators. To simulate the EIT full cycle, we compared two forward models: standard Dirichlet-to-Neumann (DtN) and restricted DtN. By taking the CRLB over modelled with Finite Elements Method (FEM) 2D circular area with various conductivity patterns, we showed the influence of electrode configuration on each basic FEM element sensitivity. Also, the sensitivity on small perturbations for different conductivity patterns was offered. This article aims to bridge the gap between the research of widely used hardware implementations standard forward DtN model and theoretically approved restricted DtN. We conclude with a discussion on the motivation behind this research and the objectives our work introduces for the inverse restricted DtN model in practice.


title: Designing the system to measure the depth-dose profile of a proton beam using CsI(Tl) scintillator

abstract: In this work, the standard CsI(Tl) scintillator was used to determine the characteristics of a proton beam. By irradiating the scintillator with a proton beam, it was able to subsequently measure the emitted light using a spectrometer. This work presents the dose response of a scintillator and its use to measure the depth-dose profile of a proton beam. The measurement of the dose response showed that the emission light of the scintillator depended on the total dose of the proton beam. For the design of the depth-dose measurement, the setup of the water phantom included a scintillator and a water tank. The depth-dose profile was determined through varying the water depth at various positions along the path of the proton beam until the end of the Bragg peak region. To compare the measured depth-dose profile, the deposited energy was simulated and fitted Birks constant for collecting quenching. Furthermore, the recommended measurement design suggested using a thin scintillator, as it resulted in a narrower shape of the depth-dose profile. Based on experiment and simulation data, a standard CsI(Tl) scintillator was promising for determining the characteristic proton beam.


title: Selfsupervised learning for pathological speech detection

abstract: Speech production is a complex phenomenon, wherein the brain orchestrates a sequence of processes involving thought processing, motor planning, and the execution of articulatory movements. However, this intricate execution of various processes is susceptible to influence and disruption by various neurodegenerative pathological speech disorders, such as Parkinsons' disease, resulting in dysarthria, apraxia, and other conditions. These disorders lead to pathological speech characterized by abnormal speech patterns and imprecise articulation. Diagnosing these speech disorders in clinical settings typically involves auditory perceptual tests, which are time-consuming, and the diagnosis can vary among clinicians based on their experiences, biases, and cognitive load during the diagnosis. Additionally, unlike neurotypical speakers, patients with speech pathologies or impairments are unable to access various virtual assistants such as Alexa, Siri, etc. To address these challenges, several automatic pathological speech detection (PSD) approaches have been proposed. These approaches aim to provide efficient and accurate detection of speech disorders, thereby facilitating timely intervention and support for individuals affected by these conditions. These approaches mainly vary in two aspects: the input representations utilized and the classifiers employed. Due to the limited availability of data, the performance of detection remains subpar. Self-supervised learning (SSL) embeddings, such as wav2vec2, and their multilingual versions, are being explored as a promising avenue to improve performance. These embeddings leverage self-supervised learning techniques to extract rich representations from audio data, thereby offering a potential solution to address the limitations posed by the scarcity of labeled data.


title: On equivalence of gauge-invariant models for massive integer-spin fields

abstract: There are several approaches to formulate gauge-invariant models for massive integer-spin field in $d$ dimensions including the following: (i) in terms of symmetric tensor fields $\phi_{\mu_1 \dots \mu_k} $, with $k = s, s-1, \dots , 0$, restricted to be double traceless for $k\geq 4$; and (ii) in terms of a quartet of $traceful$ symmetric tensor fields $\psi_{\mu_1 \dots \mu_k} $, of rank $k=s,s-1,s-2, s-3$. We demonstrate that these formulations in Minkowski space ${\mathbb M}^d$ are equivalent to the gauge-invariant theory for a massive integer-spin field proposed in 1989 by Pashnev. We also make use of the Klishevich-Zinoviev theory in ${\mathbb M}^d$ to derive a generalisation of the Singh-Hagen model for a massive integer-spin field in $d>4 $ dimensions.


title: Higher DNS-resolution requirements for expanded overlap region and confirmation of a convergence criterion

abstract: Direct numerical simulations (DNS) stand out as formidable tools in studying turbulent flows. Despite the fact that the achievable Reynolds number remains lower than those available through experimental methods, DNS offers a distinct advantage: the complete knowledge of the velocity field, facilitating the evaluation of any desired quantity. This capability should extend to compute derivatives. Among the classic functions requiring derivatives is the indicator function, $\Xi(y^+) = y^+\frac{{\rm d}\overline{U}_x^+}{{\rm d}y^+}$. This function encapsulates the wall-normal derivative of the streamwise velocity, with its value possibly influenced by mesh size and its spatial distribution. The indicator function serves as a fundamental element in unraveling the interplay between inner and outer layers in wall-bounded flows, including the overlap region, and forms a critical underpinning in turbulence modeling. Our investigation reveals a sensitivity of the indicator function on the utilized mesh distributions, prompting inquiries into the conventional mesh-sizing paradigms for DNS applications.


title: Cross-Modal Safety Alignment: Is textual unlearning all you need?

abstract: Recent studies reveal that integrating new modalities into Large Language Models (LLMs), such as Vision-Language Models (VLMs), creates a new attack surface that bypasses existing safety training techniques like Supervised Fine-tuning (SFT) and Reinforcement Learning with Human Feedback (RLHF). While further SFT and RLHF-based safety training can be conducted in multi-modal settings, collecting multi-modal training datasets poses a significant challenge. Inspired by the structural design of recent multi-modal models, where, regardless of the combination of input modalities, all inputs are ultimately fused into the language space, we aim to explore whether unlearning solely in the textual domain can be effective for cross-modality safety alignment. Our evaluation across six datasets empirically demonstrates the transferability -- textual unlearning in VLMs significantly reduces the Attack Success Rate (ASR) to less than 8\% and in some cases, even as low as nearly 2\% for both text-based and vision-text-based attacks, alongside preserving the utility. Moreover, our experiments show that unlearning with a multi-modal dataset offers no potential benefits but incurs significantly increased computational demands, possibly up to 6 times higher.


title: Towards high-order consistency and convergence of conservative SPH approximations

abstract: Smoothed particle hydrodynamics (SPH) offers distinct advantages for modeling many engineering problems, yet achieving high-order consistency in its conservative formulation remains to be addressed. While zero- and higher-order consistencies can be obtained using particle-pair differences and the kernel gradient correction (KGC) approaches, respectively, for SPH gradient approximations, their applicability for discretizing conservation laws in practical simulations is limited due to their non-conservative feature. Although the standard anti-symmetric SPH approximation is able to achieve conservative zero-order consistency through particle relaxation, its straightforward extensions with the KGC fail to satisfy either zero- or higher-order consistency. In this paper, we propose the reverse KGC (RKGC) formulation, which is conservative and able to precisely satisfy both zero- and first-order consistencies when particles are relaxed based on the KGC matrix. Extensive numerical examples show that the new formulation considerably improves the accuracy of the Lagrangian SPH method. In particular, it is able to resolve the long-standing high-dissipation issue for simulating free-surface flows. Furthermore, with fully relaxed particles, it enhances the accuracy of the Eulerian SPH method even when the ratio between the smoothing length and the particle spacing is considerably reduced. Indeed, the reverse KGC formulation holds the potential for the extension to even higher-order consistencies. However, addressing the corresponding particle relaxation problem remains a pending challenge.


title: Are PPO-ed Language Models Hackable?

abstract: Numerous algorithms have been proposed to $\textit{align}$ language models to remove undesirable behaviors. However, the challenges associated with a very large state space and creating a proper reward function often result in various jailbreaks. Our paper aims to examine this effect of reward in the controlled setting of positive sentiment language generation. Instead of online training of a reward model based on human feedback, we employ a statically learned sentiment classifier. We also consider a setting where our model's weights and activations are exposed to an end-user after training. We examine a pretrained GPT-2 through the lens of mechanistic interpretability before and after proximal policy optimization (PPO) has been applied to promote positive sentiment responses. Using these insights, we (1) attempt to "hack" the PPO-ed model to generate negative sentiment responses and (2) add a term to the reward function to try and alter `negative' weights.


title: Pretrained Mobility Transformer: A Foundation Model for Human Mobility

abstract: Ubiquitous mobile devices are generating vast amounts of location-based service data that reveal how individuals navigate and utilize urban spaces in detail. In this study, we utilize these extensive, unlabeled sequences of user trajectories to develop a foundation model for understanding urban space and human mobility. We introduce the \textbf{P}retrained \textbf{M}obility \textbf{T}ransformer (PMT), which leverages the transformer architecture to process user trajectories in an autoregressive manner, converting geographical areas into tokens and embedding spatial and temporal information within these representations. Experiments conducted in three U.S. metropolitan areas over a two-month period demonstrate PMT's ability to capture underlying geographic and socio-demographic characteristics of regions. The proposed PMT excels across various downstream tasks, including next-location prediction, trajectory imputation, and trajectory generation. These results support PMT's capability and effectiveness in decoding complex patterns of human mobility, offering new insights into urban spatial functionality and individual mobility preferences.


title: An Open-Source Framework for Efficient Numerically-Tailored Computations

abstract: We present a versatile open-source framework designed to facilitate efficient, numerically-tailored Matrix-Matrix Multiplications (MMMs). The framework offers two primary contributions: first, a fine-tuned, automated pipeline for arithmetic datapath generation, enabling highly customizable systolic MMM kernels; second, seamless integration of the generated kernels into user code, irrespective of the programming language employed, without necessitating modifications. The framework demonstrates a systematic enhancement in accuracy per energy cost across diverse High Performance Computing (HPC) workloads displaying a variety of numerical requirements, such as Artificial Intelligence (AI) inference and Sea Surface Height (SSH) computation. For AI inference, we consider a set of state-of-the-art neural network models, namely ResNet18, ResNet34, ResNet50, DenseNet121, DenseNet161, DenseNet169, and VGG11, in conjunction with two datasets, two computer formats, and 27 distinct intermediate arithmetic datapaths. Our approach consistently reduces energy consumption across all cases, with a notable example being the reduction by factors of $3.3\times$ for IEEE754-32 and $1.4\times$ for Bfloat16 during ImageNet inference with ResNet50. This is accomplished while maintaining accuracies of $82.3\%$ and $86\%$, comparable to those achieved with conventional Floating-Point Units (FPUs). In the context of SSH computation, our method achieves fully-reproducible results using double-precision words, surpassing the accuracy of conventional double- and quad-precision arithmetic in FPUs. Our approach enhances SSH computation accuracy by a minimum of $5\times$ and $27\times$ compared to IEEE754-64 and IEEE754-128, respectively, resulting in $5.6\times$ and $15.1\times$ improvements in accuracy per power cost.


title: Exploiting Chaotic Dynamics as Deep Neural Networks

abstract: Chaos presents complex dynamics arising from nonlinearity and a sensitivity to initial states. These characteristics suggest a depth of expressivity that underscores their potential for advanced computational applications. However, strategies to effectively exploit chaotic dynamics for information processing have largely remained elusive. In this study, we reveal that the essence of chaos can be found in various state-of-the-art deep neural networks. Drawing inspiration from this revelation, we propose a novel method that directly leverages chaotic dynamics for deep learning architectures. Our approach is systematically evaluated across distinct chaotic systems. In all instances, our framework presents superior results to conventional deep neural networks in terms of accuracy, convergence speed, and efficiency. Furthermore, we found an active role of transient chaos formation in our scheme. Collectively, this study offers a new path for the integration of chaos, which has long been overlooked in information processing, and provides insights into the prospective fusion of chaotic dynamics within the domains of machine learning and neuromorphic computation.


title: Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data

abstract: Throughout many fields, practitioners often rely on differential equations to model systems. Yet, for many applications, the theoretical derivation of such equations and/or accurate resolution of their solutions may be intractable. Instead, recently developed methods, including those based on parameter estimation, operator subset selection, and neural networks, allow for the data-driven discovery of both ordinary and partial differential equations (PDEs), on a spectrum of interpretability. The success of these strategies is often contingent upon the correct identification of representative equations from noisy observations of state variables and, as importantly and intertwined with that, the mathematical strategies utilized to enforce those equations. Specifically, the latter has been commonly addressed via unconstrained optimization strategies. Representing the PDE as a neural network, we propose to discover the PDE by solving a constrained optimization problem and using an intermediate state representation similar to a Physics-Informed Neural Network (PINN). The objective function of this constrained optimization problem promotes matching the data, while the constraints require that the PDE is satisfied at several spatial collocation points. We present a penalty method and a widely used trust-region barrier method to solve this constrained optimization problem, and we compare these methods on numerical examples. Our results on the Burgers' and the Korteweg-De Vreis equations demonstrate that the latter constrained method outperforms the penalty method, particularly for higher noise levels or fewer collocation points. For both methods, we solve these discovered neural network PDEs with classical methods, such as finite difference methods, as opposed to PINNs-type methods relying on automatic differentiation. We briefly highlight other small, yet crucial, implementation details.


title: Spatiotemporal Predictions of Toxic Urban Plumes Using Deep Learning

abstract: Industrial accidents, chemical spills, and structural fires can release large amounts of harmful materials that disperse into urban atmospheres and impact populated areas. Computer models are typically used to predict the transport of toxic plumes by solving fluid dynamical equations. However, these models can be computationally expensive due to the need for many grid cells to simulate turbulent flow and resolve individual buildings and streets. In emergency response situations, alternative methods are needed that can run quickly and adequately capture important spatiotemporal features. Here, we present a novel deep learning model called ST-GasNet that was inspired by the mathematical equations that govern the behavior of plumes as they disperse through the atmosphere. ST-GasNet learns the spatiotemporal dependencies from a limited set of temporal sequences of ground-level toxic urban plumes generated by a high-resolution large eddy simulation model. On independent sequences, ST-GasNet accurately predicts the late-time spatiotemporal evolution, given the early-time behavior as an input, even for cases when a building splits a large plume into smaller plumes. By incorporating large-scale wind boundary condition information, ST-GasNet achieves a prediction accuracy of at least 90% on test data for the entire prediction period.


title: Exploring the Potential of Polynomial Basis Functions in Kolmogorov-Arnold Networks: A Comparative Study of Different Groups of Polynomials

abstract: This paper presents a comprehensive survey of 18 distinct polynomials and their potential applications in Kolmogorov-Arnold Network (KAN) models as an alternative to traditional spline-based methods. The polynomials are classified into various groups based on their mathematical properties, such as orthogonal polynomials, hypergeometric polynomials, q-polynomials, Fibonacci-related polynomials, combinatorial polynomials, and number-theoretic polynomials. The study aims to investigate the suitability of these polynomials as basis functions in KAN models for complex tasks like handwritten digit classification on the MNIST dataset. The performance metrics of the KAN models, including overall accuracy, Kappa, and F1 score, are evaluated and compared. The Gottlieb-KAN model achieves the highest performance across all metrics, suggesting its potential as a suitable choice for the given task. However, further analysis and tuning of these polynomials on more complex datasets are necessary to fully understand their capabilities in KAN models. The source code for the implementation of these KAN models is available at https://github.com/seydi1370/Basis_Functions .


title: Planetary Causal Inference: Implications for the Geography of Poverty

abstract: Earth observation data such as satellite imagery can, when combined with machine learning, have profound impacts on our understanding of the geography of poverty through the prediction of living conditions, especially where government-derived economic indicators are either unavailable or potentially untrustworthy. Recent work has progressed in using EO data not only to predict spatial economic outcomes, but also to explore cause and effect, an understanding which is critical for downstream policy analysis. In this review, we first document the growth of interest in EO-ML analyses in the causal space. We then trace the relationship between spatial statistics and EO-ML methods before discussing the four ways in which EO data has been used in causal ML pipelines -- (1.) poverty outcome imputation for downstream causal analysis, (2.) EO image deconfounding, (3.) EO-based treatment effect heterogeneity, and (4.) EO-based transportability analysis. We conclude by providing a workflow for how researchers can incorporate EO data in causal ML analysis going forward.


title: Contextual Counting: A Mechanistic Study of Transformers on a Quantitative Task

abstract: Transformers have revolutionized machine learning across diverse domains, yet understanding their behavior remains crucial, particularly in high-stakes applications. This paper introduces the contextual counting task, a novel toy problem aimed at enhancing our understanding of Transformers in quantitative and scientific contexts. This task requires precise localization and computation within datasets, akin to object detection or region-based scientific analysis. We present theoretical and empirical analysis using both causal and non-causal Transformer architectures, investigating the influence of various positional encodings on performance and interpretability. In particular, we find that causal attention is much better suited for the task, and that no positional embeddings lead to the best accuracy, though rotary embeddings are competitive and easier to train. We also show that out of distribution performance is tightly linked to which tokens it uses as a bias term.


title: The potential fluctuation and its interfacial phenomena in molecular, micro, macro, and cosmic flow instabilities

abstract: Flow instabilities play important roles in a wide range of engineering, geophysical, and astrophysical flows, ranging from supernova explosion in crab nebula, formation of clouds in sky, waves on ocean, to inertial confinement fusion capsules, making fusion energy a viable alternative energy source in the future. The potential for life is directly related to flow instability mixing as well. Previous researchers have focused on developed stages of flow instabilities by assuming sine wave interface between fluids in the flow instabilities. No scientific research has been reported to investigate the origin of flow instabilities. The paper advances a new physics concept, potential fluctuation in flow based on the conservation of mass, which presents potential oscillatory sine wave surface in the spatial and temporal dimensions at the interface of flow instabilities. Potential fluctuation is decided by the two densities and velocities in the flow as indicated by the relation of continuity. Even before the flow instabilities start to develop, potential fluctuation has already internally existed in flow. It is only decided by the densities and velocities of the two moving fluids and is not related to the surface topography of the boundary of flows in the flow instability. It is the gene of flow instability. The paper presents breakthrough of understanding of micro, macro, and cosmic flow instabilities.


title: Capturing Climatic Variability: Using Deep Learning for Stochastic Downscaling

abstract: Adapting to the changing climate requires accurate local climate information, a computationally challenging problem. Recent studies have used Generative Adversarial Networks (GANs), a type of deep learning, to learn complex distributions and downscale climate variables efficiently. Capturing variability while downscaling is crucial for estimating uncertainty and characterising extreme events - critical information for climate adaptation. Since downscaling is an undetermined problem, many fine-scale states are physically consistent with the coarse-resolution state. To quantify this ill-posed problem, downscaling techniques should be stochastic, able to sample realisations from a high-resolution distribution conditioned on low-resolution input. Previous stochastic downscaling attempts have found substantial underdispersion, with models failing to represent the full distribution. We propose approaches to improve the stochastic calibration of GANs in three ways: a) injecting noise inside the network, b) adjusting the training process to explicitly account for the stochasticity, and c) using a probabilistic loss metric. We tested our models first on a synthetic dataset with known distributional properties, and then on a realistic downscaling scenario, predicting high-resolution wind components from low-resolution climate covariates. Injecting noise, on its own, substantially improved the quality of conditional and full distributions in tests with synthetic data, but performed less well for wind field downscaling, where models remained underdispersed. For wind downscaling, we found that adjusting the training method and including the probabilistic loss improved calibration. The best model, with all three changes, showed much improved skill at capturing the full variability of the high-resolution distribution and thus at characterising extremes.


title: Production planning in 3DPrinting factories

abstract: Production planning in 3D printing factories brings new challenges among which the scheduling of parts to be produced stands out. A main issue is to increase the efficiency of the plant and 3D printers productivity. Planning, scheduling, and nesting in 3D printing are recurrent problems in the search for new techniques to promote the development of this technology. In this work, we address the problem for the suppliers that have to schedule their daily production. This problem is part of the LONJA3D model, a managed 3D printing market where the parts ordered by the customers are reorganized into new batches so that suppliers can optimize their production capacity. In this paper, we propose a method derived from the design of combinatorial auctions to solve the nesting problem in 3D printing. First, we propose the use of a heuristic to create potential manufacturing batches. Then, we compute the expected return for each batch. The selected batch should generate the highest income. Several experiments have been tested to validate the process. This method is a first approach to the planning problem in 3D printing and further research is proposed to improve the procedure


title: Stochastic Earned Value Analysis using Monte Carlo Simulation and Statistical Learning Techniques

abstract: The aim of this paper is to describe a new an integrated methodology for project control under uncertainty. This proposal is based on Earned Value Methodology and risk analysis and presents several refinements to previous methodologies. More specifically, the approach uses extensive Monte Carlo simulation to obtain information about the expected behavior of the project. This dataset is exploited in several ways using different statistical learning methodologies in a structured fashion. Initially, simulations are used to detect if project deviations are a consequence of the expected variability using Anomaly Detection algorithms. If the project follows this expected variability, probabilities of success in cost and time and expected cost and total duration of the project can be estimated using classification and regression approaches.


title: Progenitor with small reaction networks should not be used as initial conditions for core collapse

abstract: Core collapse initial conditions are a bottleneck in understanding the explosion mechanism(s) of massive stars. Stellar evolution codes struggle after carbon burning, and either stop or adopt numerical simplifications missing crucial physics. The use of small nuclear reaction networks (NRN) that account for energy production but bypass weak reactions is typical, but insufficient to study the dynamics of the collapse. We advise against the use of progenitors computed with small NRN in expensive multi-dimensional simulations of core collapse, bounce, (jet formation), and explosion.


title: Unveiling the Potential of AI for Nanomaterial Morphology Prediction

abstract: Creation of nanomaterials with specific morphology remains a complex experimental process, even though there is a growing demand for these materials in various industry sectors. This study explores the potential of AI to predict the morphology of nanoparticles within the data availability constraints. For that, we first generated a new multi-modal dataset that is double the size of analogous studies. Then, we systematically evaluated performance of classical machine learning and large language models in prediction of nanomaterial shapes and sizes. Finally, we prototyped a text-to-image system, discussed the obtained empirical results, as well as the limitations and promises of existing approaches.


title: LOLAMEME: Logic, Language, Memory, Mechanistic Framework

abstract: The performance of Large Language Models has achieved superhuman breadth with unprecedented depth. At the same time, the language models are mostly black box models and the underlying mechanisms for performance have been evaluated using synthetic or mechanistic schemes. We extend current mechanistic schemes to incorporate Logic, memory, and nuances of Language such as latent structure. The proposed framework is called LOLAMEME and we provide two instantiations of LOLAMEME: LoLa and MeMe languages. We then consider two generative language model architectures: transformer-based GPT-2 and convolution-based Hyena. We propose the hybrid architecture T HEX and use LOLAMEME framework is used to compare three architectures. T HEX outperforms GPT-2 and Hyena on select tasks.


title: Construction and Observation of Flexibly Controllable High-Dimensional Non-Hermitian Skin Effects

abstract: Non-Hermitian skin effect (NHSE) is one of the most fundamental phenomena in non-Hermitian physics. Although it is established that one-dimensional NHSE originates from the nontrivial spectral winding topology, the topological origin behind the higher-dimensional NHSE remains unclear so far. This poses a substantial challenge in constructing and manipulating high-dimensional NHSEs. Here, an intuitive bottom-to-top scheme to construct high-dimensional NHSEs is proposed, through assembling multiple independent one-dimensional NHSEs. Not only the elusive high-dimensional NHSEs can be effectively predicted from the well-defined one-dimensional spectral winding topologies, but also the high-dimensional generalized Brillouin zones can be directly synthesized from the one-dimensional counterparts. As examples, two two-dimensional nonreciprocal acoustic metamaterials are experimentally implemented to demonstrate highly controllable multi-polar NHSEs and hybrid skin-topological effects, where the sound fields can be frequency-selectively localized at any desired corners and boundaries. These results offer a practicable strategy for engineering high-dimensional NHSEs, which could boost advanced applications such as selective filters and directional amplifiers.

categories: physics.class-ph cond-mat.mtrl-sci cond-mat.other physics.app-ph


title: Graph Neural Networks for Brain Graph Learning: A Survey

abstract: Exploring the complex structure of the human brain is crucial for understanding its functionality and diagnosing brain disorders. Thanks to advancements in neuroimaging technology, a novel approach has emerged that involves modeling the human brain as a graph-structured pattern, with different brain regions represented as nodes and the functional relationships among these regions as edges. Moreover, graph neural networks (GNNs) have demonstrated a significant advantage in mining graph-structured data. Developing GNNs to learn brain graph representations for brain disorder analysis has recently gained increasing attention. However, there is a lack of systematic survey work summarizing current research methods in this domain. In this paper, we aim to bridge this gap by reviewing brain graph learning works that utilize GNNs. We first introduce the process of brain graph modeling based on common neuroimaging data. Subsequently, we systematically categorize current works based on the type of brain graph generated and the targeted research problems. To make this research accessible to a broader range of interested researchers, we provide an overview of representative methods and commonly used datasets, along with their implementation sources. Finally, we present our insights on future research directions. The repository of this survey is available at \url{https://github.com/XuexiongLuoMQ/Awesome-Brain-Graph-Learning-with-GNNs}.


title: High-order Shallow Water Equations (HSWE)

abstract: Most coastal flows are turbulent and cannot be accurately described by shallow water equations. This paper presents a novel theory for characterizing accelerated coastal flows, offering improved representation of turbulent flows. The theory builds on second-order approximation long-wave theory, utilizing a dual-velocity vector approach (comprising a mean velocity and a turbulence-induced distortion velocity) in the primary flow directions. The resulting second-order long-wave model consists of five equations, which is one fewer than the number derived from averaged models that incorporate turbulence as reported in the literature, yet it retains equivalent physical and hydrodynamic characteristics. The eigenstructure of the model is presented. This modeling approach shows promise for addressing a variety of engineering problems.


title: Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networks

abstract: This study investigates the loss of generalization ability in neural networks, revisiting warm-starting experiments from Ash & Adams. Our empirical analysis reveals that common methods designed to enhance plasticity by maintaining trainability provide limited benefits to generalization. While reinitializing the network can be effective, it also risks losing valuable prior knowledge. To this end, we introduce the Hare & Tortoise, inspired by the brain's complementary learning system. Hare & Tortoise consists of two components: the Hare network, which rapidly adapts to new information analogously to the hippocampus, and the Tortoise network, which gradually integrates knowledge akin to the neocortex. By periodically reinitializing the Hare network to the Tortoise's weights, our method preserves plasticity while retaining general knowledge. Hare & Tortoise can effectively maintain the network's ability to generalize, which improves advanced reinforcement learning algorithms on the Atari-100k benchmark. The code is available at https://github.com/dojeon-ai/hare-tortoise.


title: CoNO: Complex Neural Operator for Continous Dynamical Physical Systems

abstract: Neural operators extend data-driven models to map between infinite-dimensional functional spaces. While these operators perform effectively in either the time or frequency domain, their performance may be limited when applied to non-stationary spatial or temporal signals whose frequency characteristics change with time. Here, we introduce Complex Neural Operator (CoNO) that parameterizes the integral kernel using Fractional Fourier Transform (FrFT), better representing non-stationary signals in a complex-valued domain. Theoretically, we prove the universal approximation capability of CoNO. We perform an extensive empirical evaluation of CoNO on seven challenging partial differential equations (PDEs), including regular grids, structured meshes, and point clouds. Empirically, CoNO consistently attains state-of-the-art performance, showcasing an average relative gain of 10.9%. Further, CoNO exhibits superior performance, outperforming all other models in additional tasks such as zero-shot super-resolution and robustness to noise. CoNO also exhibits the ability to learn from small amounts of data -- giving the same performance as the next best model with just 60% of the training data. Altogether, CoNO presents a robust and superior model for modeling continuous dynamical systems, providing a fillip to scientific machine learning.


title: Towards Learning Foundation Models for Heuristic Functions to Solve Pathfinding Problems

abstract: Pathfinding problems are found throughout robotics, computational science, and natural sciences. Traditional methods to solve these require training deep neural networks (DNNs) for each new problem domain, consuming substantial time and resources. This study introduces a novel foundation model, leveraging deep reinforcement learning to train heuristic functions that seamlessly adapt to new domains without further fine-tuning. Building upon DeepCubeA, we enhance the model by providing the heuristic function with the domain's state transition information, improving its adaptability. Utilizing a puzzle generator for the 15-puzzle action space variation domains, we demonstrate our model's ability to generalize and solve unseen domains. We achieve a strong correlation between learned and ground truth heuristic values across various domains, as evidenced by robust R-squared and Concordance Correlation Coefficient metrics. These results underscore the potential of foundation models to establish new standards in efficiency and adaptability for AI-driven solutions in complex pathfinding problems.


title: Privacy-Aware Randomized Quantization via Linear Programming

abstract: Differential privacy mechanisms such as the Gaussian or Laplace mechanism have been widely used in data analytics for preserving individual privacy. However, they are mostly designed for continuous outputs and are unsuitable for scenarios where discrete values are necessary. Although various quantization mechanisms were proposed recently to generate discrete outputs under differential privacy, the outcomes are either biased or have an inferior accuracy-privacy trade-off. In this paper, we propose a family of quantization mechanisms that is unbiased and differentially private. It has a high degree of freedom and we show that some existing mechanisms can be considered as special cases of ours. To find the optimal mechanism, we formulate a linear optimization that can be solved efficiently using linear programming tools. Experiments show that our proposed mechanism can attain a better privacy-accuracy trade-off compared to baselines.


title: Data Quality in Edge Machine Learning: A State-of-the-Art Survey

abstract: Data-driven Artificial Intelligence (AI) systems trained using Machine Learning (ML) are shaping an ever-increasing (in size and importance) portion of our lives, including, but not limited to, recommendation systems, autonomous driving technologies, healthcare diagnostics, financial services, and personalized marketing. On the one hand, the outsized influence of these systems imposes a high standard of quality, particularly in the data used to train them. On the other hand, establishing and maintaining standards of Data Quality (DQ) becomes more challenging due to the proliferation of Edge Computing and Internet of Things devices, along with their increasing adoption for training and deploying ML models. The nature of the edge environment -- characterized by limited resources, decentralized data storage, and processing -- exacerbates data-related issues, making them more frequent, severe, and difficult to detect and mitigate. From these observations, it follows that DQ research for edge ML is a critical and urgent exploration track for the safety and robust usefulness of present and future AI systems. Despite this fact, DQ research for edge ML is still in its infancy. The literature on this subject remains fragmented and scattered across different research communities, with no comprehensive survey to date. Hence, this paper aims to fill this gap by providing a global view of the existing literature from multiple disciplines that can be grouped under the umbrella of DQ for edge ML. Specifically, we present a tentative definition of data quality in Edge computing, which we use to establish a set of DQ dimensions. We explore each dimension in detail, including existing solutions for mitigation.


title: Multimodal Deep Learning for Low-Resource Settings: A Vector Embedding Alignment Approach for Healthcare Applications

abstract: Large-scale multi-modal deep learning models have revolutionized domains such as healthcare, highlighting the importance of computational power. However, in resource-constrained regions like Low and Middle-Income Countries (LMICs), limited access to GPUs and data poses significant challenges, often leaving CPUs as the sole resource. To address this, we advocate for leveraging vector embeddings to enable flexible and efficient computational methodologies, democratizing multimodal deep learning across diverse contexts. Our paper investigates the efficiency and effectiveness of using vector embeddings from single-modal foundation models and multi-modal Vision-Language Models (VLMs) for multimodal deep learning in low-resource environments, particularly in healthcare. Additionally, we propose a simple yet effective inference-time method to enhance performance by aligning image-text embeddings. Comparing these approaches with traditional methods, we assess their impact on computational efficiency and model performance using metrics like accuracy, F1-score, inference time, training time, and memory usage across three medical modalities: BRSET (ophthalmology), HAM10000 (dermatology), and SatelliteBench (public health). Our findings show that embeddings reduce computational demands without compromising model performance. Furthermore, our alignment method improves performance in medical tasks. This research promotes sustainable AI practices by optimizing resources in constrained environments, highlighting the potential of embedding-based approaches for efficient multimodal learning. Vector embeddings democratize multimodal deep learning in LMICs, particularly in healthcare, enhancing AI adaptability in varied use cases.


title: D-FaST: Cognitive Signal Decoding with Disentangled Frequency-Spatial-Temporal Attention

abstract: Cognitive Language Processing (CLP), situated at the intersection of Natural Language Processing (NLP) and cognitive science, plays a progressively pivotal role in the domains of artificial intelligence, cognitive intelligence, and brain science. Among the essential areas of investigation in CLP, Cognitive Signal Decoding (CSD) has made remarkable achievements, yet there still exist challenges related to insufficient global dynamic representation capability and deficiencies in multi-domain feature integration. In this paper, we introduce a novel paradigm for CLP referred to as Disentangled Frequency-Spatial-Temporal Attention(D-FaST). Specifically, we present an novel cognitive signal decoder that operates on disentangled frequency-space-time domain attention. This decoder encompasses three key components: frequency domain feature extraction employing multi-view attention, spatial domain feature extraction utilizing dynamic brain connection graph attention, and temporal feature extraction relying on local time sliding window attention. These components are integrated within a novel disentangled framework. Additionally, to encourage advancements in this field, we have created a new CLP dataset, MNRED. Subsequently, we conducted an extensive series of experiments, evaluating D-FaST's performance on MNRED, as well as on publicly available datasets including ZuCo, BCIC IV-2A, and BCIC IV-2B. Our experimental results demonstrate that D-FaST outperforms existing methods significantly on both our datasets and traditional CSD datasets including establishing a state-of-the-art accuracy score 78.72% on MNRED, pushing the accuracy score on ZuCo to 78.35%, accuracy score on BCIC IV-2A to 74.85% and accuracy score on BCIC IV-2B to 76.81%.


title: Distortion-free Watermarks are not Truly Distortion-free under Watermark Key Collisions

abstract: Language model (LM) watermarking techniques inject a statistical signal into LM-generated content by substituting the random sampling process with pseudo-random sampling, using watermark keys as the random seed. Among these statistical watermarking approaches, distortion-free watermarks are particularly crucial because they embed watermarks into LM-generated content without compromising generation quality. However, one notable limitation of pseudo-random sampling compared to true-random sampling is that, under the same watermark keys (i.e., key collision), the results of pseudo-random sampling exhibit correlations. This limitation could potentially undermine the distortion-free property. Our studies reveal that key collisions are inevitable due to the limited availability of watermark keys, and existing distortion-free watermarks exhibit a significant distribution bias toward the original LM distribution in the presence of key collisions. Moreover, achieving a perfect distortion-free watermark is impossible as no statistical signal can be embedded under key collisions. To reduce the distribution bias caused by key collisions, we introduce a new family of distortion-free watermarks--beta-watermark. Experimental results support that the beta-watermark can effectively reduce the distribution bias under key collisions.


title: Gated recurrent neural network with TPE Bayesian optimization for enhancing stock index prediction accuracy

abstract: The recent advancement of deep learning architectures, neural networks, and the combination of abundant financial data and powerful computers are transforming finance, leading us to develop an advanced method for predicting future stock prices. However, the accessibility of investment and trading at everyone's fingertips made the stock markets increasingly intricate and prone to volatility. The increased complexity and volatility of the stock market have driven demand for more models, which would effectively capture high volatility and non-linear behavior of the different stock prices. This study explored gated recurrent neural network (GRNN) algorithms such as LSTM (long short-term memory), GRU (gated recurrent unit), and hybrid models like GRU-LSTM, LSTM-GRU, with Tree-structured Parzen Estimator (TPE) Bayesian optimization for hyperparameter optimization (TPE-GRNN). The aim is to improve the prediction accuracy of the next day's closing price of the NIFTY 50 index, a prominent Indian stock market index, using TPE-GRNN. A combination of eight influential factors is carefully chosen from fundamental stock data, technical indicators, crude oil price, and macroeconomic data to train the models for capturing the changes in the price of the index with the factors of the broader economy. Single-layer and multi-layer TPE-GRNN models have been developed. The models' performance is evaluated using standard matrices like R2, MAPE, and RMSE. The analysis of models' performance reveals the impact of feature selection and hyperparameter optimization (HPO) in enhancing stock index price prediction accuracy. The results show that the MAPE of our proposed TPE-LSTM method is the lowest (best) with respect to all the previous models for stock index price prediction.


title: A Novel Defense Against Poisoning Attacks on Federated Learning: LayerCAM Augmented with Autoencoder

abstract: Recent attacks on federated learning (FL) can introduce malicious model updates that circumvent widely adopted Euclidean distance-based detection methods. This paper proposes a novel defense strategy, referred to as LayerCAM-AE, designed to counteract model poisoning in federated learning. The LayerCAM-AE puts forth a new Layer Class Activation Mapping (LayerCAM) integrated with an autoencoder (AE), significantly enhancing detection capabilities. Specifically, LayerCAM-AE generates a heat map for each local model update, which is then transformed into a more compact visual format. The autoencoder is designed to process the LayerCAM heat maps from the local model updates, improving their distinctiveness and thereby increasing the accuracy in spotting anomalous maps and malicious local models. To address the risk of misclassifications with LayerCAM-AE, a voting algorithm is developed, where a local model update is flagged as malicious if its heat maps are consistently suspicious over several rounds of communication. Extensive tests of LayerCAM-AE on the SVHN and CIFAR-100 datasets are performed under both Independent and Identically Distributed (IID) and non-IID settings in comparison with existing ResNet-50 and REGNETY-800MF defense models. Experimental results show that LayerCAM-AE increases detection rates (Recall: 1.0, Precision: 1.0, FPR: 0.0, Accuracy: 1.0, F1 score: 1.0, AUC: 1.0) and test accuracy in FL, surpassing the performance of both the ResNet-50 and REGNETY-800MF. Our code is available at: https://github.com/jjzgeeks/LayerCAM-AE


title: Know Your Neighborhood: General and Zero-Shot Capable Binary Function Search Powered by Call Graphlets

abstract: Binary code similarity detection is an important problem with applications in areas like malware analysis, vulnerability research and plagiarism detection. This paper proposes a novel graph neural network architecture combined with a novel graph data representation called call graphlets. A call graphlet encodes the neighborhood around each function in a binary executable, capturing the local and global context through a series of statistical features. A specialized graph neural network model is then designed to operate on this graph representation, learning to map it to a feature vector that encodes semantic code similarities using deep metric learning. The proposed approach is evaluated across four distinct datasets covering different architectures, compiler toolchains, and optimization levels. Experimental results demonstrate that the combination of call graphlets and the novel graph neural network architecture achieves state-of-the-art performance compared to baseline techniques across cross-architecture, mono-architecture and zero shot tasks. In addition, our proposed approach also performs well when evaluated against an out-of-domain function inlining task. Overall, the work provides a general and effective graph neural network-based solution for conducting binary code similarity detection.


title: Flow-Induced Vibration of Flexible Hydrofoil Within Cavitating Turbulent Flow

abstract: The flow-induced vibration and cavitation dynamics of three-dimensional flow past a cantilever flexible hydrofoil are investigated using a large eddy simulation (LES) model, a homogeneous mixture cavitation model and the structural modes superposition method. The present work aims to explore a potential mechanism responsible for a propeller singing behavior, and thus focuses on the synchronized hydroelastic coupling among the pressure pulsation inside the flow field, the cavitation generation and the structural vibration. To begin, we validate the tip vortex dynamics of a flexible hydrofoil against the available experimental. Our results demonstrate that the tip vortex shedding and the blade vibration are responsible for the intense peak in the low-frequency tonal components of the noise source, and the trailing-edge vortex shedding induces broadband components. Additionally, the generation of sheet cavitation induces considerable synchronized hydrofoil vibration (subjected to a flutter-like response), and affects the pressure fluctuations in the flow field, which further dominate the features of the underwater noise sources. It is suggested that the cavitation behavior and structural vibrations co-dominate the characteristics of singing noise from a propeller blade.


title: PPINtonus: Early Detection of Parkinson's Disease Using Deep-Learning Tonal Analysis

abstract: PPINtonus is a system for the early detection of Parkinson's Disease (PD) utilizing deep-learning tonal analysis, providing a cost-effective and accessible alternative to traditional neurological examinations. Partnering with the Parkinson's Voice Project (PVP), PPINtonus employs a semi-supervised conditional generative adversarial network to generate synthetic data points, enhancing the training dataset for a multi-layered deep neural network. Combined with PRAAT phonetics software, this network accurately assesses biomedical voice measurement values from a simple 120-second vocal test performed with a standard microphone in typical household noise conditions. The model's performance was validated using a confusion matrix, achieving an impressive 92.5 \% accuracy with a low false negative rate. PPINtonus demonstrated a precision of 92.7 \%, making it a reliable tool for early PD detection. The non-intrusive and efficient methodology of PPINtonus can significantly benefit developing countries by enabling early diagnosis and improving the quality of life for millions of PD patients through timely intervention and management.


title: Less is More: Pseudo-Label Filtering for Continual Test-Time Adaptation

abstract: Continual Test-Time Adaptation (CTTA) aims to adapt a pre-trained model to a sequence of target domains during the test phase without accessing the source data. To adapt to unlabeled data from unknown domains, existing methods rely on constructing pseudo-labels for all samples and updating the model through self-training. However, these pseudo-labels often involve noise, leading to insufficient adaptation. To improve the quality of pseudo-labels, we propose a pseudo-label selection method for CTTA, called Pseudo Labeling Filter (PLF). The key idea of PLF is to keep selecting appropriate thresholds for pseudo-labels and identify reliable ones for self-training. Specifically, we present three principles for setting thresholds during continuous domain learning, including initialization, growth and diversity. Based on these principles, we design Self-Adaptive Thresholding to filter pseudo-labels. Additionally, we introduce a Class Prior Alignment (CPA) method to encourage the model to make diverse predictions for unknown domain samples. Through extensive experiments, PLF outperforms current state-of-the-art methods, proving its effectiveness in CTTA.


title: MoFormer: Multi-objective Antimicrobial Peptide Generation Based on Conditional Transformer Joint Multi-modal Fusion Descriptor

abstract: Deep learning holds a big promise for optimizing existing peptides with more desirable properties, a critical step towards accelerating new drug discovery. Despite the recent emergence of several optimized Antimicrobial peptides(AMP) generation methods, multi-objective optimizations remain still quite challenging for the idealism-realism tradeoff. Here, we establish a multi-objective AMP synthesis pipeline (MoFormer) for the simultaneous optimization of multi-attributes of AMPs. MoFormer improves the desired attributes of AMP sequences in a highly structured latent space, guided by conditional constraints and fine-grained multi-descriptor.We show that MoFormer outperforms existing methods in the generation task of enhanced antimicrobial activity and minimal hemolysis. We also utilize a Pareto-based non-dominated sorting algorithm and proxies based on large model fine-tuning to hierarchically rank the candidates. We demonstrate substantial property improvement using MoFormer from two perspectives: (1) employing molecular simulations and scoring interactions among amino acids to decipher the structure and functionality of AMPs; (2) visualizing latent space to examine the qualities and distribution features, verifying an effective means to facilitate multi-objective optimization AMPs with design constraints


title: LOLA: LLM-Assisted Online Learning Algorithm for Content Experiments

abstract: In the rapidly evolving digital content landscape, media firms and news publishers require automated and efficient methods to enhance user engagement. This paper introduces the LLM-Assisted Online Learning Algorithm (LOLA), a novel framework that integrates Large Language Models (LLMs) with adaptive experimentation to optimize content delivery. Leveraging a large-scale dataset from Upworthy, which includes 17,681 headline A/B tests aimed at evaluating the performance of various headlines associated with the same article content, we first investigate three broad pure-LLM approaches: prompt-based methods, embedding-based classification models, and fine-tuned open-source LLMs. Our findings indicate that prompt-based approaches perform poorly, achieving no more than 65% accuracy in identifying the catchier headline among two options. In contrast, OpenAI-embedding-based classification models and fine-tuned Llama-3-8b models achieve comparable accuracy, around 82-84%, though still falling short of the performance of experimentation with sufficient traffic. We then introduce LOLA, which combines the best pure-LLM approach with the Upper Confidence Bound algorithm to adaptively allocate traffic and maximize clicks. Our numerical experiments on Upworthy data show that LOLA outperforms the standard A/B testing method (the current status quo at Upworthy), pure bandit algorithms, and pure-LLM approaches, particularly in scenarios with limited experimental traffic or numerous arms. Our approach is both scalable and broadly applicable to content experiments across a variety of digital settings where firms seek to optimize user engagement, including digital advertising and social media recommendations.


title: Is Data Valuation Learnable and Interpretable?

abstract: Measuring the value of individual samples is critical for many data-driven tasks, e.g., the training of a deep learning model. Recent literature witnesses the substantial efforts in developing data valuation methods. The primary data valuation methodology is based on the Shapley value from game theory, and various methods are proposed along this path. {Even though Shapley value-based valuation has solid theoretical basis, it is entirely an experiment-based approach and no valuation model has been constructed so far.} In addition, current data valuation methods ignore the interpretability of the output values, despite an interptable data valuation method is of great helpful for applications such as data pricing. This study aims to answer an important question: is data valuation learnable and interpretable? A learned valuation model have several desirable merits such as fixed number of parameters and knowledge reusability. An intrepretable data valuation model can explain why a sample is valuable or invaluable. To this end, two new data value modeling frameworks are proposed, in which a multi-layer perception~(MLP) and a new regression tree are utilized as specific base models for model training and interpretability, respectively. Extensive experiments are conducted on benchmark datasets. {The experimental results provide a positive answer for the question.} Our study opens up a new technical path for the assessing of data values. Large data valuation models can be built across many different data-driven tasks, which can promote the widespread application of data valuation.


title: ACCO: Accumulate while you Communicate, Hiding Communications in Distributed LLM Training

abstract: Training Large Language Models (LLMs) relies heavily on distributed implementations, employing multiple GPUs to compute stochastic gradients on model replicas in parallel. However, synchronizing gradients in data parallel settings induces a communication overhead increasing with the number of distributed workers, which can impede the efficiency gains of parallelization. To address this challenge, optimization algorithms reducing inter-worker communication have emerged, such as local optimization methods used in Federated Learning. While effective in minimizing communication overhead, these methods incur significant memory costs, hindering scalability: in addition to extra momentum variables, if communications are only allowed between multiple local optimization steps, then the optimizer's states cannot be sharded among workers. In response, we propose $\textbf{AC}$cumulate while $\textbf{CO}$mmunicate ($\texttt{ACCO}$), a memory-efficient optimization algorithm tailored for distributed training of LLMs. $\texttt{ACCO}$ allows to shard optimizer states across workers, overlaps gradient computations and communications to conceal communication costs, and accommodates heterogeneous hardware. Our method relies on a novel technique to mitigate the one-step delay inherent in parallel execution of gradient computations and communications, eliminating the need for warmup steps and aligning with the training dynamics of standard distributed optimization while converging faster in terms of wall-clock time. We demonstrate the effectiveness of $\texttt{ACCO}$ on several LLMs training and fine-tuning tasks.


title: Frequency Enhanced Pre-training for Cross-city Few-shot Traffic Forecasting

abstract: The field of Intelligent Transportation Systems (ITS) relies on accurate traffic forecasting to enable various downstream applications. However, developing cities often face challenges in collecting sufficient training traffic data due to limited resources and outdated infrastructure. Recognizing this obstacle, the concept of cross-city few-shot forecasting has emerged as a viable approach. While previous cross-city few-shot forecasting methods ignore the frequency similarity between cities, we have made an observation that the traffic data is more similar in the frequency domain between cities. Based on this fact, we propose a \textbf{F}requency \textbf{E}nhanced \textbf{P}re-training Framework for \textbf{Cross}-city Few-shot Forecasting (\textbf{FEPCross}). FEPCross has a pre-training stage and a fine-tuning stage. In the pre-training stage, we propose a novel Cross-Domain Spatial-Temporal Encoder that incorporates the information of the time and frequency domain and trains it with self-supervised tasks encompassing reconstruction and contrastive objectives. In the fine-tuning stage, we design modules to enrich training samples and maintain a momentum-updated graph structure, thereby mitigating the risk of overfitting to the few-shot training data. Empirical evaluations performed on real-world traffic datasets validate the exceptional efficacy of FEPCross, outperforming existing approaches of diverse categories and demonstrating characteristics that foster the progress of cross-city few-shot forecasting.


title: A hybrid numerical methodology coupling Reduced Order Modeling and Graph Neural Networks for non-parametric geometries: applications to structural dynamics problems

abstract: This work introduces a new approach for accelerating the numerical analysis of time-domain partial differential equations (PDEs) governing complex physical systems. The methodology is based on a combination of a classical reduced-order modeling (ROM) framework and recently-introduced Graph Neural Networks (GNNs), where the latter is trained on highly heterogeneous databases of varying numerical discretization sizes. The proposed techniques are shown to be particularly suitable for non-parametric geometries, ultimately enabling the treatment of a diverse range of geometries and topologies. Performance studies are presented in an application context related to the design of aircraft seats and their corresponding mechanical responses to shocks, where the main motivation is to reduce the computational burden and enable the rapid design iteration for such problems that entail non-parametric geometries. The methods proposed here are straightforwardly applicable to other scientific or engineering problems requiring a large number of finite element-based numerical simulations, with the potential to significantly enhance efficiency while maintaining reasonable accuracy.


title: Adaptive Layer Splitting for Wireless LLM Inference in Edge Computing: A Model-Based Reinforcement Learning Approach

abstract: Optimizing the deployment of large language models (LLMs) in edge computing environments is critical for enhancing privacy and computational efficiency. Toward efficient wireless LLM inference in edge computing, this study comprehensively analyzes the impact of different splitting points in mainstream open-source LLMs. On this basis, this study introduces a framework taking inspiration from model-based reinforcement learning (MBRL) to determine the optimal splitting point across the edge and user equipment (UE). By incorporating a reward surrogate model, our approach significantly reduces the computational cost of frequent performance evaluations. Extensive simulations demonstrate that this method effectively balances inference performance and computational load under varying network conditions, providing a robust solution for LLM deployment in decentralized settings.


title: Fine Structures Roll up in the Flow of Film Boiling at High Density Ratios

abstract: Film boiling has practical applications in the current technology including steam power plants, cooling of electronic devices and emergency cooling systems. A finite difference/front tracking method is used to simulate film boiling at high density ratios on a horizontal plate subject to a constant wall heat flux. The grid resolution is relatively high (768 grids per width of the domain). The flow is dominated by Rayleigh-Taylor instability as well as Kelvin-Helmholtz instability. The flow structure includes the roll up of the interface between the gas and liquid. This happens at high density ratio (1000) where the difference between the gas and the liquid velocities across the interface is large. The jump in tangential velocity is an order of magnitude smaller at a lower density ratio (100). Hence, there is no roll up at lower density ratio. The flow is also influenced by vortex development as a result of the baroclinic term in the vorticity transport equation. The density gradient is large at the interface at high density ratio which tends to increase the baroclinic term. The plot of pressure gradient and density gradient shows that they are not parallel in the roll up regions. As a result, vortices in small scales develop that shed in the gas phase. The plot of the enstrophy with time shows that it shows smooth variation that match for two grid resolutions, however at a specific time enstrophies become spiky, and they depart from each other at two grid resolutions. The spiky behavior of enstrophy is due to vortex shedding in the roll up region.


title: Immunocto: a massive immune cell database auto-generated for histopathology

abstract: With the advent of novel cancer treatment options such as immunotherapy, studying the tumour immune micro-environment is crucial to inform on prognosis and understand response to therapeutic agents. A key approach to characterising the tumour immune micro-environment may be through combining (1) digitised microscopic high-resolution optical images of hematoxylin and eosin (H&E) stained tissue sections obtained in routine histopathology examinations with (2) automated immune cell detection and classification methods. However, current individual immune cell classification models for digital pathology present relatively poor performance. This is mainly due to the limited size of currently available datasets of individual immune cells, a consequence of the time-consuming and difficult problem of manually annotating immune cells on digitised H&E whole slide images. In that context, we introduce Immunocto, a massive, multi-million automatically generated database of 6,848,454 human cells, including 2,282,818 immune cells distributed across 4 subtypes: CD4$^+$ T cell lymphocytes, CD8$^+$ T cell lymphocytes, B cell lymphocytes, and macrophages. For each cell, we provide a 64$\times$64 pixels H&E image at $\mathbf{40}\times$ magnification, along with a binary mask of the nucleus and a label. To create Immunocto, we combined open-source models and data to automatically generate the majority of contours and labels. The cells are obtained from a matched H&E and immunofluorescence colorectal dataset from the Orion platform, while contours are obtained using the Segment Anything Model. A classifier trained on H&E images from Immunocto produces an average F1 score of 0.74 to differentiate the 4 immune cell subtypes and other cells. Immunocto can be downloaded at: https://zenodo.org/uploads/11073373.


title: Unelicitable Backdoors in Language Models via Cryptographic Transformer Circuits

abstract: The rapid proliferation of open-source language models significantly increases the risks of downstream backdoor attacks. These backdoors can introduce dangerous behaviours during model deployment and can evade detection by conventional cybersecurity monitoring systems. In this paper, we introduce a novel class of backdoors in autoregressive transformer models, that, in contrast to prior art, are unelicitable in nature. Unelicitability prevents the defender from triggering the backdoor, making it impossible to evaluate or detect ahead of deployment even if given full white-box access and using automated techniques, such as red-teaming or certain formal verification methods. We show that our novel construction is not only unelicitable thanks to using cryptographic techniques, but also has favourable robustness properties. We confirm these properties in empirical investigations, and provide evidence that our backdoors can withstand state-of-the-art mitigation strategies. Additionally, we expand on previous work by showing that our universal backdoors, while not completely undetectable in white-box settings, can be harder to detect than some existing designs. By demonstrating the feasibility of seamlessly integrating backdoors into transformer models, this paper fundamentally questions the efficacy of pre-deployment detection strategies. This offers new insights into the offence-defence balance in AI safety and security.


title: Hot spot as hallmark of transition from dielectric disk to ring

abstract: Topological transformations of dielectric structures radically change the eigenvalues and eigenfunctions of photonic resonances. Moreover, optical effects may arise that characterize the moment of transition from one structure to another, but are not inherent in either the initial or final structure. We demonstrate that such a hallmark of the disk-ring transition is a hot spot of a special nature that arises at the moment a central hole appears in the disk. The hot spot in the air hole is caused by the Mie resonance of the disk with azimuthal number $m$ = 1, while other Mie resonances do not contribute to the effect due to symmetry. As the hole increases, the hot spot fades out, and we theoretically and experimentally observe the formation of photonic resonances of the ring from resonances of the disk. Using near-field and far-field measurements, we discovered clustering of disk photonic modes into distinct galleries of ring eigenmodes that are formed by the inner and outer walls of the cavity. Thus, we demonstrate both the beginning and the end of the rearrangement of photonic eigenmodes during the transition from a dielectric disk to a narrow ring.


title: A new conformal quasi-local energy in general relativity

abstract: We construct new conserved quasi-local energies in general relativity using the formalism developed by \cite{CWY}. In particular, we use the optimal isometric embedding defined in \cite{yau,yau1} to transplant the conformal Killing fields of the Minkowski space back to the $ 2-$ surface of interest in the physical spacetime. For an asymptotically flat spacetime of order $1$, we show that these energies are always finite. Their limit as the total energies of an isolated system is evaluated and a conservation law under Einsteinian evolution is deduced.


title: Safeguarding Large Language Models: A Survey

abstract: In the burgeoning field of Large Language Models (LLMs), developing a robust safety mechanism, colloquially known as "safeguards" or "guardrails", has become imperative to ensure the ethical use of LLMs within prescribed boundaries. This article provides a systematic literature review on the current status of this critical mechanism. It discusses its major challenges and how it can be enhanced into a comprehensive mechanism dealing with ethical issues in various contexts. First, the paper elucidates the current landscape of safeguarding mechanisms that major LLM service providers and the open-source community employ. This is followed by the techniques to evaluate, analyze, and enhance some (un)desirable properties that a guardrail might want to enforce, such as hallucinations, fairness, privacy, and so on. Based on them, we review techniques to circumvent these controls (i.e., attacks), to defend the attacks, and to reinforce the guardrails. While the techniques mentioned above represent the current status and the active research trends, we also discuss several challenges that cannot be easily dealt with by the methods and present our vision on how to implement a comprehensive guardrail through the full consideration of multi-disciplinary approach, neural-symbolic method, and systems development lifecycle.


title: The Vertebrate Breed Ontology: Towards Effective Breed Data Standardization

abstract: Background: Limited universally adopted data standards in veterinary science hinders data interoperability and therefore integration and comparison; this ultimately impedes application of existing information-based tools to support advancement in veterinary diagnostics, treatments, and precision medicine. Objectives: Creation of a Vertebrate Breed Ontology (VBO) as a single, coherent logic-based standard for documenting breed names in animal health, production and research-related records will improve data use capabilities in veterinary and comparative medicine. Animals: No live animals were used in this study. Methods: A list of breed names and related information was compiled from relevant sources, organizations, communities, and experts using manual and computational approaches to create VBO. Each breed is represented by a VBO term that includes all provenance and the breed's related information as metadata. VBO terms are classified using description logic to allow computational applications and Artificial Intelligence-readiness. Results: VBO is an open, community-driven ontology representing over 19,000 livestock and companion animal breeds covering 41 species. Breeds are classified based on community and expert conventions (e.g., horse breed, cattle breed). This classification is supported by relations to the breeds' genus and species indicated by NCBI Taxonomy terms. Relationships between VBO terms, e.g. relating breeds to their foundation stock, provide additional context to support advanced data analytics. VBO term metadata includes common names and synonyms, breed identifiers or codes, and attributed cross-references to other databases. Conclusion and clinical importance: Veterinary data interoperability and computability can be enhanced by the adoption of VBO as a source of standard breed names in databases and veterinary electronic health records.


title: Take a Step Further: Understanding Page Spray in Linux Kernel Exploitation

abstract: Recently, a novel method known as Page Spray emerges, focusing on page-level exploitation for kernel vulnerabilities. Despite the advantages it offers in terms of exploitability, stability, and compatibility, comprehensive research on Page Spray remains scarce. Questions regarding its root causes, exploitation model, comparative benefits over other exploitation techniques, and possible mitigation strategies have largely remained unanswered. In this paper, we conduct a systematic investigation into Page Spray, providing an in-depth understanding of this exploitation technique. We introduce a comprehensive exploit model termed the \sys model, elucidating its fundamental principles. Additionally, we conduct a thorough analysis of the root causes underlying Page Spray occurrences within the Linux Kernel. We design an analyzer based on the Page Spray analysis model to identify Page Spray callsites. Subsequently, we evaluate the stability, exploitability, and compatibility of Page Spray through meticulously designed experiments. Finally, we propose mitigation principles for addressing Page Spray and introduce our own lightweight mitigation approach. This research aims to assist security researchers and developers in gaining insights into Page Spray, ultimately enhancing our collective understanding of this emerging exploitation technique and making improvements to the community.


title: Progressive Inference: Explaining Decoder-Only Sequence Classification Models Using Intermediate Predictions

abstract: This paper proposes Progressive Inference - a framework to compute input attributions to explain the predictions of decoder-only sequence classification models. Our work is based on the insight that the classification head of a decoder-only Transformer model can be used to make intermediate predictions by evaluating them at different points in the input sequence. Due to the causal attention mechanism, these intermediate predictions only depend on the tokens seen before the inference point, allowing us to obtain the model's prediction on a masked input sub-sequence, with negligible computational overheads. We develop two methods to provide sub-sequence level attributions using this insight. First, we propose Single Pass-Progressive Inference (SP-PI), which computes attributions by taking the difference between consecutive intermediate predictions. Second, we exploit a connection with Kernel SHAP to develop Multi Pass-Progressive Inference (MP-PI). MP-PI uses intermediate predictions from multiple masked versions of the input to compute higher quality attributions. Our studies on a diverse set of models trained on text classification tasks show that SP-PI and MP-PI provide significantly better attributions compared to prior work.