title: Estimating and Implementing Conventional Fairness Metrics With Probabilistic Protected Features

abstract: The vast majority of techniques to train fair models require access to the protected attribute (e.g., race, gender), either at train time or in production. However, in many important applications this protected attribute is largely unavailable. In this paper, we develop methods for measuring and reducing fairness violations in a setting with limited access to protected attribute labels. Specifically, we assume access to protected attribute labels on a small subset of the dataset of interest, but only probabilistic estimates of protected attribute labels (e.g., via Bayesian Improved Surname Geocoding) for the rest of the dataset. With this setting in mind, we propose a method to estimate bounds on common fairness metrics for an existing model, as well as a method for training a model to limit fairness violations by solving a constrained non-convex optimization problem. Unlike similar existing approaches, our methods take advantage of contextual information -- specifically, the relationships between a model's predictions and the probabilistic prediction of protected attributes, given the true protected attribute, and vice versa -- to provide tighter bounds on the true disparity. We provide an empirical illustration of our methods using voting data. First, we show our measurement method can bound the true disparity up to 5.5x tighter than previous methods in these applications. Then, we demonstrate that our training technique effectively reduces disparity while incurring lesser fairness-accuracy trade-offs than other fair optimization methods with limited access to protected attributes.


title: Keypoint-Augmented Self-Supervised Learning for Medical Image Segmentation with Limited Annotation

abstract: Pretraining CNN models (i.e., UNet) through self-supervision has become a powerful approach to facilitate medical image segmentation under low annotation regimes. Recent contrastive learning methods encourage similar global representations when the same image undergoes different transformations, or enforce invariance across different image/patch features that are intrinsically correlated. However, CNN-extracted global and local features are limited in capturing long-range spatial dependencies that are essential in biological anatomy. To this end, we present a keypoint-augmented fusion layer that extracts representations preserving both short- and long-range self-attention. In particular, we augment the CNN feature map at multiple scales by incorporating an additional input that learns long-range spatial self-attention among localized keypoint features. Further, we introduce both global and local self-supervised pretraining for the framework. At the global scale, we obtain global representations from both the bottleneck of the UNet, and by aggregating multiscale keypoint features. These global features are subsequently regularized through image-level contrastive objectives. At the local scale, we define a distance-based criterion to first establish correspondences among keypoints and encourage similarity between their features. Through extensive experiments on both MRI and CT segmentation tasks, we demonstrate the architectural advantages of our proposed method in comparison to both CNN and Transformer-based UNets, when all architectures are trained with randomly initialized weights. With our proposed pretraining strategy, our method further outperforms existing SSL methods by producing more robust self-attention and achieving state-of-the-art segmentation results. The code is available at https://github.com/zshyang/kaf.git.


title: Decentralized Micro Water-Energy Co-Optimization for Small Communities

abstract: The water-energy nexus encompasses the interdependencies between water and energy resources identifying the existing links between the production and distribution of these resources. Therefore, understanding the water-energy nexus is crucial for developing sustainable and integrated resource management approaches. This paper proposes a decentralized co-optimization model for a micro water-energy nexus system (MWEN), aiming to optimize the combined supply of both resources to end consumers. The approach respects the separate ownership and management of the water and energy sectors while bridging the gap between their optimized operations. An enhanced version of the alternating direction method of multipliers (ADMM) is proposed, the objective-based ADMM (OB-ADMM), which is able to robustly optimize each system independently towards a common objective, only sharing information about the power consumption of water management, providing privacy for each resource provider.


title: Solving Two-Player General-Sum Games Between Swarms

abstract: Hamilton-Jacobi-Isaacs (HJI) PDEs are the governing equations for the two-player general-sum games. Unlike Reinforcement Learning (RL) methods, which are data-intensive methods for learning value function, learning HJ PDEs provide a guaranteed convergence to the Nash Equilibrium value of the game when it exists. However, a caveat is that solving HJ PDEs becomes intractable when the state dimension increases. To circumvent the curse of dimensionality (CoD), physics-informed machine learning methods with supervision can be used and have been shown to be effective in generating equilibrial policies in two-player general-sum games. In this work, we extend the existing work on agent-level two-player games to a two-player swarm-level game, where two sub-swarms play a general-sum game. We consider the \textit{Kolmogorov forward equation} as the dynamic model for the evolution of the densities of the swarms. Results show that policies generated from the physics-informed neural network (PINN) result in a higher payoff than a Nash Double Deep Q-Network (Nash DDQN) agent and have comparable performance with numerical solvers.


title: Commutative Width and Depth Scaling in Deep Neural Networks

abstract: This paper is the second in the series Commutative Scaling of Width and Depth (WD) about commutativity of infinite width and depth limits in deep neural networks. Our aim is to understand the behaviour of neural functions (functions that depend on a neural network model) as width and depth go to infinity (in some sense), and eventually identify settings under which commutativity holds, i.e. the neural function tends to the same limit no matter how width and depth limits are taken. In this paper, we formally introduce and define the commutativity framework, and discuss its implications on neural network design and scaling. We study commutativity for the neural covariance kernel which reflects how network layers separate data. Our findings extend previous results established in [55] by showing that taking the width and depth to infinity in a deep neural network with skip connections, when branches are suitably scaled to avoid exploding behaviour, result in the same covariance structure no matter how that limit is taken. This has a number of theoretical and practical implications that we discuss in the paper. The proof techniques in this paper are novel and rely on tools that are more accessible to readers who are not familiar with stochastic calculus (used in the proofs of WD(I))).


title: Designing User-Centric Behavioral Interventions to Prevent Dysglycemia with Novel Counterfactual Explanations

abstract: Maintaining normal blood glucose levels through lifestyle behaviors is central to maintaining health and preventing disease. Frequent exposure to dysglycemia (i.e., abnormal glucose events such as hyperlycemia and hypoglycemia) leads to chronic complications including diabetes, kidney disease and need for dialysis, myocardial infarction, stroke, amputation, and death. Therefore, a tool capable of predicting dysglycemia and offering users actionable feedback about how to make changes in their diet, exercise, and medication to prevent abnormal glycemic events could have significant societal impacts. Counterfactual explanations can provide insights into why a model made a particular prediction by generating hypothetical instances that are similar to the original input but lead to a different prediction outcome. Therefore, counterfactuals can be viewed as a means to design AI-driven health interventions to prevent adverse health outcomes such as dysglycemia. In this paper, we design GlyCoach, a framework for generating counterfactual explanations for glucose control. Leveraging insights from adversarial learning, GlyCoach characterizes the decision boundary for high-dimensional health data and performs a grid search to generate actionable interventions. GlyCoach is unique in integrating prior knowledge about user preferences of plausible explanations into the process of counterfactual generation. We evaluate GlyCoach extensively using two real-world datasets and external simulators from prior studies that predict glucose response. GlyCoach achieves 87\% sensitivity in the simulation-aided validation, surpassing the state-of-the-art techniques for generating counterfactual explanations by at least $10\%$. Besides, counterfactuals from GlyCoach exhibit a $32\%$ improved normalized distance compared to previous research.


title: A Framework for Interpretability in Machine Learning for Medical Imaging

abstract: Interpretability for machine learning models in medical imaging (MLMI) is an important direction of research. However, there is a general sense of murkiness in what interpretability means. Why does the need for interpretability in MLMI arise? What goals does one actually seek to address when interpretability is needed? To answer these questions, we identify a need to formalize the goals and elements of interpretability in MLMI. By reasoning about real-world tasks and goals common in both medical image analysis and its intersection with machine learning, we identify five core elements of interpretability: localization, visual recognizability, physical attribution, model transparency, and actionability. From this, we arrive at a framework for interpretability in MLMI, which serves as a step-by-step guide to approaching interpretability in this context. Overall, this paper formalizes interpretability needs in the context of medical imaging, and our applied perspective clarifies concrete MLMI-specific goals and considerations in order to guide method design and improve real-world usage. Our goal is to provide practical and didactic information for model designers and practitioners, inspire developers of models in the medical imaging field to reason more deeply about what interpretability is achieving, and suggest future directions of interpretability research.


title: OFDM-RSMA: Robust Transmission under Inter-Carrier Interference

abstract: Rate-splitting multiple access (RSMA) is a multiple access scheme to mitigate the effects of the multi-user interference (MUI) in multi-antenna systems. In this study, we leverage the interference management capabilities of RSMA to tackle the issue of inter-carrier interference (ICI) in orthogonal frequency division multiplexing (OFDM) waveform. We formulate a sum-rate maximization problem to find the optimal subcarrier and power allocation for downlink transmission in a two-user system using RSMA and OFDM. A weighted minimum mean-square error (WMMSE)-based algorithm is proposed to obtain a solution for the formulated non-convex problem. We show that the marriage of rate-splitting (RS) with OFDM provides complementary strengths to cope with peculiar characteristic of wireless medium and its performance-limiting challenges including inter-symbol interference (ISI), MUI, ICI, and inter-numerology interference (INI). The sum-rate performance of the proposed OFDM-RSMA scheme is numerically compared with that of conventional orthogonal frequency division multiple access (OFDMA) and OFDM-non-orthogonal multiple access (NOMA). It is shown that the proposed OFDM-RSMA outperforms OFDM-NOMA and OFDMA in diverse propagation channel conditions owing to its flexible structure and robust interference management capabilities.


title: From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression

abstract: We conduct a comprehensive investigation into the dynamics of gradient descent using large-order constant step-sizes in the context of quadratic regression models. Within this framework, we reveal that the dynamics can be encapsulated by a specific cubic map, naturally parameterized by the step-size. Through a fine-grained bifurcation analysis concerning the step-size parameter, we delineate five distinct training phases: (1) monotonic, (2) catapult, (3) periodic, (4) chaotic, and (5) divergent, precisely demarcating the boundaries of each phase. As illustrations, we provide examples involving phase retrieval and two-layer neural networks employing quadratic activation functions and constant outer-layers, utilizing orthogonal training data. Our simulations indicate that these five phases also manifest with generic non-orthogonal data. We also empirically investigate the generalization performance when training in the various non-monotonic (and non-divergent) phases. In particular, we observe that performing an ergodic trajectory averaging stabilizes the test error in non-monotonic (and non-divergent) phases.


title: One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition

abstract: This paper presents a novel framework for joint speaker diarization (SD) and automatic speech recognition (ASR), named SLIDAR (sliding-window diarization-augmented recognition). SLIDAR can process arbitrary length inputs and can handle any number of speakers, effectively solving ``who spoke what, when'' concurrently. SLIDAR leverages a sliding window approach and consists of an end-to-end diarization-augmented speech transcription (E2E DAST) model which provides, locally, for each window: transcripts, diarization and speaker embeddings. The E2E DAST model is based on an encoder-decoder architecture and leverages recent techniques such as serialized output training and ``Whisper-style" prompting. The local outputs are then combined to get the final SD+ASR result by clustering the speaker embeddings to get global speaker identities. Experiments performed on monaural recordings from the AMI corpus confirm the effectiveness of the method in both close-talk and far-field speech scenarios.


title: Threat Modelling in Internet of Things (IoT) Environment Using Dynamic Attack Graphs

abstract: This work presents a threat modelling approach to represent changes to the attack paths through an Internet of Things (IoT) environment when the environment changes dynamically, i.e., when new devices are added or removed from the system or when whole sub-systems join or leave. The proposed approach investigates the propagation of threats using attack graphs. However, traditional attack graph approaches have been applied in static environments that do not continuously change such as the Enterprise networks, leading to static and usually very large attack graphs. In contrast, IoT environments are often characterised by dynamic change and interconnections; different topologies for different systems may interconnect with each other dynamically and outside the operator control. Such new interconnections lead to changes in the reachability amongst devices according to which their corresponding attack graphs change. This requires dynamic topology and attack graphs for threat and risk analysis. In this paper, a threat modelling approach is developed that copes with dynamic system changes that may occur in IoT environments and enables identifying attack paths whilst allowing for system dynamics. Dynamic topology and attack graphs were developed that are able to cope with the changes in the IoT environment rapidly by maintaining their associated graphs. To motivate the work and illustrate the proposed approach, an example scenario based on healthcare systems is introduced. The proposed approach is implemented using a Graph Database Management Tool (GDBM)- Neo4j- which is a popular tool for mapping, visualising and querying the graphs of highly connected data, and is efficient in providing a rapid threat modelling mechanism, which makes it suitable for capturing security changes in the dynamic IoT environment.


title: Forecasting Tropical Cyclones with Cascaded Diffusion Models

abstract: As tropical cyclones become more intense due to climate change, the rise of Al-based modelling provides a more affordable and accessible approach compared to traditional methods based on mathematical models. This work leverages generative diffusion models to forecast cyclone trajectories and precipitation patterns by integrating satellite imaging, remote sensing, and atmospheric data. It employs a cascaded approach that incorporates three main tasks: forecasting, super-resolution, and precipitation modelling. The training dataset includes 51 cyclones from six major tropical cyclone basins from January 2019 - March 2023. Experiments demonstrate that the final forecasts from the cascaded models show accurate predictions up to a 36-hour rollout, with excellent Structural Similarity (SSIM) and Peak-Singal-To-Noise Ratio (PSNR) values exceeding 0.5 and 20 dB, respectively, for all three tasks. The 36-hour forecasts can be produced in as little as 30 mins on a single Nvidia A30/RTX 2080 Ti. This work also highlights the promising efficiency of Al methods such as diffusion models for high-performance needs in weather forecasting, such as tropical cyclone forecasting, while remaining computationally affordable, making them ideal for highly vulnerable regions with critical forecasting needs and financial limitations. Code accessible at \url{https://github.com/nathzi1505/forecast-diffmodels}.


title: Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models

abstract: Prompt tuning in natural language processing (NLP) has become an increasingly popular method for adapting large language models to specific tasks. However, the transferability of these prompts, especially continuous prompts, between different models remains a challenge. In this work, we propose a zero-shot continuous prompt transfer method, where source prompts are encoded into relative space and the corresponding target prompts are searched for transferring to target models. Experimental results confirm the effectiveness of our method, showing that 'task semantics' in continuous prompts can be generalized across various language models. Moreover, we find that combining 'task semantics' from multiple source models can further enhance the generalizability of transfer.


title: Synthesis of infrared Stokes spectra in an evolving solar chromospheric jet

abstract: Chromospheric jets are plausible agents of energy and mass transport in the solar chromosphere, although their driving mechanisms have not yet been elucidated. Magnetic field measurements are key for distinguishing the driving mechanisms of chromospheric jets. We performed a full Stokes synthesis in the infrared range with a realistic radiative magnetohydrodynamics simulation that generated a chromospheric jet to predict spectro-polarimetric observations from the Sunrise Chromospheric Infrared spectro-Polarimeter (SCIP) onboard the SUNRISE III balloon telescope. The jet was launched by the collision between the transition region and an upflow driven by the ascending motion of the twisted magnetic field at the envelope of the flux tube. This motion is consistent with upwardly propagating non-linear Alfvenic waves. The upflow could be detected as continuous Doppler signals in the CaII 849.8 nm line at the envelope where the dark line core intensity and strong linear polarisation coexist. The axis of the flux tube was bright in both FeI 846.8 nm and CaII 849.8 nm lines with down-flowing plasma inside it. The structure, time evolution, and Stokes signals predicted in our study will improve the physical interpretation of future spectro-polarimetric observations with SUNRISE III/SCIP.


title: Closing the Curious Case of Neural Text Degeneration

abstract: Despite their ubiquity in language generation, it remains unknown why truncation sampling heuristics like nucleus sampling are so effective. We provide a theoretical explanation for the effectiveness of the truncation sampling by proving that truncation methods that discard tokens below some probability threshold (the most common type of truncation) can guarantee that all sampled tokens have nonzero true probability. However, thresholds are a coarse heuristic, and necessarily discard some tokens with nonzero true probability as well. In pursuit of a more precise sampling strategy, we show that we can leverage a known source of model errors, the softmax bottleneck, to prove that certain tokens have nonzero true probability, without relying on a threshold. Based on our findings, we develop an experimental truncation strategy and the present pilot studies demonstrating the promise of this type of algorithm. Our evaluations show that our method outperforms its threshold-based counterparts under automatic and human evaluation metrics for low-entropy (i.e., close to greedy) open-ended text generation. Our theoretical findings and pilot experiments provide both insight into why truncation sampling works, and make progress toward more expressive sampling algorithms that better surface the generative capabilities of large language models.


title: Segmented zero-inflated Poisson mixed effects model with random changepoint

abstract: The COVID-19 pandemic has had a substantial impact on hospital services, as many institutions have observed a surge in healthcare-associated infections (HAIs) despite heightened adherence to isolation protocols and hand hygiene. According to the World Health Organization (WHO), HAIs are among the leading causes of mortality and morbidity of hospitalized patients. This study aims to examine the effect of the COVID-19 pandemic on the incidence of central venous catheter-related bloodstream infections (CR-BSIs) of hospitals in the city of S\~ao Paulo. Initially we considered segmented zero-inflated Poisson (ZIP) mixed-effects models with known changepoint, which can be estimated applying the standard framework of ZIP mixed-effects models. However, we found that the changepoint could occur at varying times across different hospitals. We present an effective iterative procedure to estimate segmented ZIP mixed-effects models with random changepoints in a likelihood-based framework. The suggested procedure is a practical approach employing conventional computational tools for estimating standard mixed-effects zero-inflated Poisson (ZIP) models. Prior to its implementation to the CR-BSI data, simulation studies were conducted to examine the accuracy of the estimation under various scenarios.


title: DynAMO: Multi-agent reinforcement learning for dynamic anticipatory mesh optimization with applications to hyperbolic conservation laws

abstract: We introduce DynAMO, a reinforcement learning paradigm for Dynamic Anticipatory Mesh Optimization. Adaptive mesh refinement is an effective tool for optimizing computational cost and solution accuracy in numerical methods for partial differential equations. However, traditional adaptive mesh refinement approaches for time-dependent problems typically rely only on instantaneous error indicators to guide adaptivity. As a result, standard strategies often require frequent remeshing to maintain accuracy. In the DynAMO approach, multi-agent reinforcement learning is used to discover new local refinement policies that can anticipate and respond to future solution states by producing meshes that deliver more accurate solutions for longer time intervals. By applying DynAMO to discontinuous Galerkin methods for the linear advection and compressible Euler equations in two dimensions, we demonstrate that this new mesh refinement paradigm can outperform conventional threshold-based strategies while also generalizing to different mesh sizes, remeshing and simulation times, and initial conditions.


title: DANI: Fast Diffusion Aware Network Inference with Preserving Topological Structure Property

abstract: The fast growth of social networks and their data access limitations in recent years has led to increasing difficulty in obtaining the complete topology of these networks. However, diffusion information over these networks is available, and many algorithms have been proposed to infer the underlying networks using this information. The previously proposed algorithms only focus on inferring more links and ignore preserving the critical topological characteristics of the underlying social networks. In this paper, we propose a novel method called DANI to infer the underlying network while preserving its structural properties. It is based on the Markov transition matrix derived from time series cascades, as well as the node-node similarity that can be observed in the cascade behavior from a structural point of view. In addition, the presented method has linear time complexity (increases linearly with the number of nodes, number of cascades, and square of the average length of cascades), and its distributed version in the MapReduce framework is also scalable. We applied the proposed approach to both real and synthetic networks. The experimental results showed that DANI has higher accuracy and lower run time while maintaining structural properties, including modular structure, degree distribution, connected components, density, and clustering coefficients, than well-known network inference methods.


title: Some Probabilistic Properties of General Topological Markov Chains

abstract: In this paper, we utilize the framework of Markov processes to attain a more probabilistic perspective on the theory of transfer operators. In doing so, we establish a functional central limit theorem (FLCT) for an $O(N)$ model associated with Dyson potential on the one-dimensional lattice. We also proof a FLCT on a non-compact alphabet setting, for a model associated with a Dyson type potential on the one-dimensional lattice. A Breimann ergodic theorem for equilibrium measures arising from the Ruelle Operator Formalism is proved. Furthermore, we obtain a qualitative criterion (strong transitivity) to determine when the conformal measure has full support. As an application, we show how to connect the Ruelle operator framework with the perspective of the Hopf theory of Markov Processes.


title: Robustifying State-space Models for Long Sequences via Approximate Diagonalization

abstract: State-space models (SSMs) have recently emerged as a framework for learning long-range sequence tasks. An example is the structured state-space sequence (S4) layer, which uses the diagonal-plus-low-rank structure of the HiPPO initialization framework. However, the complicated structure of the S4 layer poses challenges; and, in an effort to address these challenges, models such as S4D and S5 have considered a purely diagonal structure. This choice simplifies the implementation, improves computational efficiency, and allows channel communication. However, diagonalizing the HiPPO framework is itself an ill-posed problem. In this paper, we propose a general solution for this and related ill-posed diagonalization problems in machine learning. We introduce a generic, backward-stable "perturb-then-diagonalize" (PTD) methodology, which is based on the pseudospectral theory of non-normal operators, and which may be interpreted as the approximate diagonalization of the non-normal matrices defining SSMs. Based on this, we introduce the S4-PTD and S5-PTD models. Through theoretical analysis of the transfer functions of different initialization schemes, we demonstrate that the S4-PTD/S5-PTD initialization strongly converges to the HiPPO framework, while the S4D/S5 initialization only achieves weak convergences. As a result, our new models show resilience to Fourier-mode noise-perturbed inputs, a crucial property not achieved by the S4D/S5 models. In addition to improved robustness, our S5-PTD model averages 87.6% accuracy on the Long-Range Arena benchmark, demonstrating that the PTD methodology helps to improve the accuracy of deep learning models.


title: Quantum Entanglement Phase Transitions and Computational Complexity: Insights from Ising Models

abstract: In this paper, we construct 2-dimensional bipartite cluster states and perform single-qubit measurements on the bulk qubits. We explore the entanglement scaling of the unmeasured 1-dimensional boundary state and show that under certain conditions, the boundary state can undergo a volume-law to an area-law entanglement transition driven by variations in the measurement angle. We bridge this boundary state entanglement transition and the measurement-induced phase transition in the non-unitary 1+1-dimensional circuit via the transfer matrix method. We also explore the application of this entanglement transition on the computational complexity problems. Specifically, we establish a relation between the boundary state entanglement transition and the sampling complexity of the bipartite $2$d cluster state, which is directly related to the computational complexity of the corresponding Ising partition function with complex parameters. By examining the boundary state entanglement scaling, we numerically identify the parameter regime for which the $2$d quantum state can be efficiently sampled, which indicates that the Ising partition function can be evaluated efficiently in such a region.


title: Rooted labeled trees and exceptional sequences of type $B_n/C_n$

abstract: We show that exceptional sequences in the abelian tube of rank $n$, which we denote $\mathscr{ W}_n$, are related to exceptional sequences of type $B_n$ and $C_n$ and to those of type $B_{n-1}$ and $C_{n-1}$. $\mathscr{W}_n$ has $n^{n-1}$ exceptional sequences. These are in $1$-to-$n$ correspondence from the $n^n$ "augmented" rooted labeled trees with $n$ vertices which, in turn, are in bijection with exceptional sequences of type $B_n$ and $C_n$. By determining the probability distribution of relative projectives in these exceptional sequences, we show there are $(2n)!/n!$ signed exceptional sequences in $\mathscr{W}_n$ and we show that these are in bijection with signed exceptional sequences of type $B_{n-1}$ and $C_{n-1}$ by combining the results of Buan-Marsh-Vatne and Igusa-Todorov.


title: Transcending Domains through Text-to-Image Diffusion: A Source-Free Approach to Domain Adaptation

abstract: Domain Adaptation (DA) is a method for enhancing a model's performance on a target domain with inadequate annotated data by applying the information the model has acquired from a related source domain with sufficient labeled data. The escalating enforcement of data-privacy regulations like HIPAA, COPPA, FERPA, etc. have sparked a heightened interest in adapting models to novel domains while circumventing the need for direct access to the source data, a problem known as Source-Free Domain Adaptation (SFDA). In this paper, we propose a novel framework for SFDA that generates source data using a text-to-image diffusion model trained on the target domain samples. Our method starts by training a text-to-image diffusion model on the labeled target domain samples, which is then fine-tuned using the pre-trained source model to generate samples close to the source data. Finally, we use Domain Adaptation techniques to align the artificially generated source data with the target domain data, resulting in significant performance improvements of the model on the target domain. Through extensive comparison against several baselines on the standard Office-31, Office-Home, and VisDA benchmarks, we demonstrate the effectiveness of our approach for the SFDA task.


title: New conformal-like symmetry of strictly massless fermions in four-dimensional de Sitter space

abstract: We present new infinitesimal `conformal-like' symmetries for the field equations of strictly massless spin-$s \geq 3/2$ totally symmetric tensor-spinors (i.e. gauge potentials) on 4-dimensional de Sitter spacetime ($dS_{4}$). The corresponding symmetry transformations are generated by the five closed conformal Killing vectors of $dS_{4}$, but they are not conventional conformal transformations. We show that the algebra generated by the ten de Sitter (dS) symmetries and the five conformal-like symmetries closes on the conformal-like algebra $so(4,2)$ up to gauge transformations of the gauge potentials. The transformations of the gauge-invariant field strength tensor-spinors under the conformal-like symmetries are given by the product of $\gamma^{5}$ times a usual infinitesimal conformal transformation of the field strengths. Furthermore, we demonstrate that the two sets of physical mode solutions, corresponding to the two helicities $\pm s$ of the strictly massless theories, form a direct sum of Unitary Irreducible Representations (UIRs) of the conformal-like algebra. We also fill a gap in the literature by explaining how these physical modes form a direct sum of Discrete Series UIRs of the dS algebra $so(4,1)$.


title: Spatially-Resolved Temperature and Density Structures of Nearby HII regions

abstract: Photoionization models frequently assume constant temperature or density within HII regions. We investigate this assumption by measuring the detailed temperature and density structures of four HII regions in the Large Magellanic Cloud and the Small Magellanic Cloud, using integral-field spectroscopic data from the Wide-Field Spectrograph on the ANU 2.3m telescope. We analyse the distribution of emission-lines of low-ionization species, intermediate-ionization species and high-ionization species. We present the complex electron temperature and density structures within HII regions. All four nebulae present a negative gradient in the electron density profile. Both positive and negative temperature gradients are observed in the nebulae. We create a series of nebula models with a constant ISM pressure and varying temperature and density distributions. Comparison of the line ratios between our HII regions and models suggests that none of the simple nebula models can reproduce the observed temperature and density structures. Comparison between the models and the data suggests that the ISM pressure of nebulae in LMC and SMC is between log(P/k)=6-7.5. Complex internal structures of the nebulae highlight the importance of future Monte-Carlo photoionization codes for accurate nebula modeling, which include a comprehensive consideration of arbitrary geometries of HII regions.


title: Transformers are efficient hierarchical chemical graph learners

abstract: Transformers, adapted from natural language processing, are emerging as a leading approach for graph representation learning. Contemporary graph transformers often treat nodes or edges as separate tokens. This approach leads to computational challenges for even moderately-sized graphs due to the quadratic scaling of self-attention complexity with token count. In this paper, we introduce SubFormer, a graph transformer that operates on subgraphs that aggregate information by a message-passing mechanism. This approach reduces the number of tokens and enhances learning long-range interactions. We demonstrate SubFormer on benchmarks for predicting molecular properties from chemical structures and show that it is competitive with state-of-the-art graph transformers at a fraction of the computational cost, with training times on the order of minutes on a consumer-grade graphics card. We interpret the attention weights in terms of chemical structures. We show that SubFormer exhibits limited over-smoothing and avoids over-squashing, which is prevalent in traditional graph neural networks.


title: Obstructions to free periodicity and symmetric L-space knots

abstract: We investigate a polynomial factorization problem which naturally arises in Hartley's factorization condition on the Alexander polynomial of freely periodic knots. We give a number-theoretic interpretation of this factorization condition, which allows for efficient computation. As an application, we prove that any polynomial which is not a product of cyclotomic polynomials can be the Alexander polynomial of a freely $p$-periodic knot for only finitely many $p$. As a demonstration of the computational efficiency of these methods, we also show that the Alexander polynomial of any freely-periodic L-space knot with genus at most 16 must be a product of cyclotomic polynomials. We conjecture that any periodic or freely periodic L-space knot must be an iterated torus knot.


title: On Representation Complexity of Model-based and Model-free Reinforcement Learning

abstract: We study the representation complexity of model-based and model-free reinforcement learning (RL) in the context of circuit complexity. We prove theoretically that there exists a broad class of MDPs such that their underlying transition and reward functions can be represented by constant depth circuits with polynomial size, while the optimal $Q$-function suffers an exponential circuit complexity in constant-depth circuits. By drawing attention to the approximation errors and building connections to complexity theory, our theory provides unique insights into why model-based algorithms usually enjoy better sample complexity than model-free algorithms from a novel representation complexity perspective: in some cases, the ground-truth rule (model) of the environment is simple to represent, while other quantities, such as $Q$-function, appear complex. We empirically corroborate our theory by comparing the approximation error of the transition kernel, reward function, and optimal $Q$-function in various Mujoco environments, which demonstrates that the approximation errors of the transition kernel and reward function are consistently lower than those of the optimal $Q$-function. To the best of our knowledge, this work is the first to study the circuit complexity of RL, which also provides a rigorous framework for future research.


title: On-Chip Stimulated Brillouin Scattering via Surface Acoustic Waves

abstract: Surface acoustic wave (SAW) devices are ubiquitously used for signal processing and filtering, as well as mechanical, chemical and biological sensing, and show promise as quantum transducers. However, nowadays most SAWs are excited and driven via electromechanical coupling and interdigital transducers (IDTs), limiting operation bandwidth and flexibility. Novel ways to coherently excite and detect SAWs all-optically interfaced with photonic integrated circuits are yet elusive. Backward Stimulated Brillouin scattering (SBS) provides strong coherent interactions between optical and acoustic waves in chip-scale waveguides, however, demonstrations have been limited to single longitudinal waves in the waveguide core. Here, we numerically model and experimentally demonstrate surface acoustic wave stimulated Brillouin scattering (SAW-SBS) on a photonic chip. We designed and fabricated tailored waveguides made out of GeAsSe glass that show good overlap between SAWs at 3.81 GHz and guided optical modes, without requiring a top cladding. We measure a 225 W$^{-1}$m$^{-1}$ Brillouin gain coefficient of the surface acoustic resonance and linewidth narrowing to 40 MHz. Experimentally accessing this new regime of stimulated Brillouin scattering opens the door for novel on-chip sensing and signal processing applications, strong Brillouin interactions in materials that do not provide sufficient acoustic guidance in the waveguide core as well as excitation of surface acoustic waves in non-piezoelectric materials.


title: Deciphering Diagnoses: How Large Language Models Explanations Influence Clinical Decision Making

abstract: Clinical Decision Support Systems (CDSS) utilize evidence-based knowledge and patient data to offer real-time recommendations, with Large Language Models (LLMs) emerging as a promising tool to generate plain-text explanations for medical decisions. This study explores the effectiveness and reliability of LLMs in generating explanations for diagnoses based on patient complaints. Three experienced doctors evaluated LLM-generated explanations of the connection between patient complaints and doctor and model-assigned diagnoses across several stages. Experimental results demonstrated that LLM explanations significantly increased doctors' agreement rates with given diagnoses and highlighted potential errors in LLM outputs, ranging from 5% to 30%. The study underscores the potential and challenges of LLMs in healthcare and emphasizes the need for careful integration and evaluation to ensure patient safety and optimal clinical utility.


title: Investigation of flow patterns in two-phase carbon dioxide in horizontal and vertical pipes

abstract: Modern particle detectors crucially depend on efficient cooling systems. Two-phase carbon dioxide (CO$_2$) is a suitable solution as a cooling agent. This publication presents the observations and results of investigations of horizontal and vertical flow of two-phase CO$_2$ at a temperature of $T=-15\,^\circ$C and a pressure of approximately $23\,$bar. Heat fluxes between $98.5\,$kW/m$^2$ and $200\,$kW/m$^2$ were applied to the CO$_2$, covering the range expected to occur in the future ATLAS Pixel detector being built for the high-luminosity phase of the Large Hadron Collider. Flow speeds ranged from $11.8\,$m/s to $28.1\,$m/s. Dedicated sensors were used to measure the temperature and pressure before and after heating the CO$_2$. Two-phase flow patterns occuring in the pipe after heating the CO$_2$ were recorded with a high-speed camera. Stratified, wavy and slug flow are found to be the predominant patterns for horizontal flow, while upward vertical flow is mainly found to be slug or churn. Based on the recorded images the void fraction of the CO$_2$ after heating is determined and compared for the different setups. The results are summarised in a flow-pattern map. A clear distinction between vertical and horizontal flow is found, with horizontal flow exhibiting a significantly higher void fraction than vertical flow. Based on the pressure measurements, the pressure drop after heating the CO$_2$ is measured and the corresponding Euler number is computed. While the pressure drop increases with the heat flux for horizontal flow due to frictional losses, the pressure drop reduces with the heat flux in case of upward vertical flow, since static pressure is important in this case.


title: Symplectic structure, product structures and complex structures on Leibniz algebras

abstract: In this paper, a symplectic structure on a Leibniz algebra is defined to be a {\em symmetric} nondegenerate bilinear form satisfying certain compatibility condition, and a phase space of a Leibniz algebra is defined to be a symplectic Leibniz algebra satisfying certain conditions. We show that a Leibniz algebra has a phase space if and only if there is a compatible Leibniz-dendriform algebra, and phase spaces of Leibniz algebras one-to-one corresponds to Manin triples of Leibniz-dendriform algebras. Product (paracomplex) structures and complex structures on Leibniz algebras are studied in terms of decompositions of Leibniz algebras. A para-K\"{a}hler structure on a Leibniz algebra is defined to be a symplectic structure and a paracomplex structure satisfying a compatibility condition. We show that a symplectic Leibniz algebra admits a para-K\"{a}hler structure if and only if the Leibniz algebra is the direct sum of two isotropic subalgebras as vector spaces. A complex product structure on a Leibniz algebra consists of a complex structure and a product structure satisfying a compatibility condition. A pseudo-K\"{a}hler structure on a Leibniz algebra is defined to be a symplectic structure and a complex structure satisfying a compatibility condition. Various properties and relations of complex product structures and pseudo-K\"{a}hler structures are studied. In particular, Leibniz-dendriform algebras give rise to complex product structures and pseudo-K\"{a}hler structures naturally.


title: Learning Class-Specific Spectral Patterns to Improve Deep Learning Based Scene-Level Fire Smoke Detection from Multi-Spectral Satellite Imagery

abstract: Detecting fire smoke is crucial for the timely identification of early wildfires using satellite imagery. However, the spatial and spectral similarity of fire smoke to other confounding aerosols, such as clouds and haze, often confuse even the most advanced deep-learning (DL) models. Nonetheless, these aerosols also present distinct spectral characteristics in some specific bands, and such spectral patterns are useful for distinguishing the aerosols more accurately. Early research tried to derive various threshold values from the reflectance and brightness temperature in specific spectral bands to differentiate smoke and cloud pixels. However, such threshold values were determined based on domain knowledge and are hard to generalise. In addition, such threshold values were manually derived from specific combinations of bands to infer spectral patterns, making them difficult to employ in deep-learning models. In this paper, we introduce a DL module called input amplification (InAmp) which is designed to enable DL models to learn class-specific spectral patterns automatically from multi-spectral satellite imagery and improve the fire smoke detection accuracy. InAmp can be conveniently integrated with different DL architectures. We evaluate the effectiveness of the InAmp module on different Convolutional neural network (CNN) architectures using two satellite imagery datasets: USTC_SmokeRS, derived from Moderate Resolution Imaging Spectroradiometer (MODIS) with three spectral bands; and Landsat_Smk, derived from Landsat 5/8 with six spectral bands. Our experimental results demonstrate that the InAmp module improves the fire smoke detection accuracy of the CNN models. Additionally, we visualise the spectral patterns extracted by the InAmp module using test imagery and demonstrate that the InAmp module can effectively extract class-specific spectral patterns.


title: Generative Autoencoding of Dropout Patterns

abstract: We propose a generative model termed Deciphering Autoencoders. In this model, we assign a unique random dropout pattern to each data point in the training dataset and then train an autoencoder to reconstruct the corresponding data point using this pattern as information to be encoded. Since the training of Deciphering Autoencoders relies solely on reconstruction error, it offers more stable training than other generative models. Despite its simplicity, Deciphering Autoencoders show comparable sampling quality to DCGAN on the CIFAR-10 dataset.


title: Greedy invariant-domain preserving approximation for hyperbolic systems

abstract: The paper focuses on invariant-domain preserving approximations of hyperbolic systems. We propose a new way to estimate the artificial viscosity that has to be added to make explicit, conservative, consistent numerical methods invariant-domain preserving and entropy inequality compliant. Instead of computing an upper bound on the maximum wave speed in Riemann problems, we estimate a minimum wave speed in the said Riemann problems such that the approximation satisfies predefined invariant-domain properties and predefined entropy inequalities. This technique eliminates non-essential fast waves from the construction of the artificial viscosity, while preserving pre-assigned invariant-domain properties and entropy inequalities.


title: Large Language Models as Analogical Reasoners

abstract: Chain-of-thought (CoT) prompting for language models demonstrates impressive performance across reasoning tasks, but typically needs labeled exemplars of the reasoning process. In this work, we introduce a new prompting approach, analogical prompting, designed to automatically guide the reasoning process of large language models. Inspired by analogical reasoning, a cognitive process in which humans draw from relevant past experiences to tackle new problems, our approach prompts language models to self-generate relevant exemplars or knowledge in the context, before proceeding to solve the given problem. This method presents several advantages: it obviates the need for labeling or retrieving exemplars, offering generality and convenience; it can also tailor the generated exemplars and knowledge to each problem, offering adaptability. Experimental results show that our approach outperforms 0-shot CoT and manual few-shot CoT in a variety of reasoning tasks, including math problem solving in GSM8K and MATH, code generation in Codeforces, and other reasoning tasks in BIG-Bench.


title: Broadband terahertz near-field excitation and detection of silicon photonic crystal modes

abstract: Chip-based terahertz (THz) devices are emerging as versatile tools for manipulating mm-wave frequencies in the context of integrated high-speed communication technologies for potential sixth-generation (6G) wireless applications. The characterization of THz devices is typically performed using far-field techniques that provide limited information about the underlying physical mechanisms producing them. As the library of chip-based functionalities expands, e.g., for tailoring the emission and directional propagation properties of THz antennas and waveguides, novel characterization techniques will likely be beneficial for observing subtle effects that are sensitive to a device's structural parameters. Here we present near-field measurements showing the emission properties of a broadband THz emitter placed in the vicinity of a photonic crystal (PHC) slab. These experiments reveal long-predicted emission properties, but which to our knowledge have yet to be experimentally observed at THz frequencies. We demonstrate three distinct effects between 0.3-0.5 THz: (i) field suppression at frequencies corresponding to quasi-TE bandgaps (ii) a frequency-dependent directed emission along two distinct pathways for two neighboring frequencies, resulting in a local field concentration; (iii) a re-direction of the emission, achieved by rotating the PHC with respect to the dipole orientation. Simulations reveal that the observed behavior can be predicted from the underlying band structure. Our results highlight the opportunities that PHCs can potentially provide for alignment-free, chip-based 6G technologies. Our experimental technique extends the applicability realms of THz spectroscopy and will find use for characterizing the THz modes supported by true samples, whose inherent imperfections cannot realistically be accounted for by simulations, particularly in highly dispersive frequency bands.


title: Active motion of tangentially-driven polymers in periodic array of obstacles

abstract: We computationally investigate the active transport of tangentially-driven polymers with varying degrees of flexibility and activity in two-dimensional square lattices of obstacles. Tight periodic confinement induces notable conformational changes and distinct modes of transport for flexible and stiff active filaments. It leads to localization and caging of flexible polymers inside the inter-obstacle pores, while promoting more elongated conformations and enhanced diffusion for stiff polymers at low to moderate activity levels. The migration of flexible active polymers occurs via hopping events, where they unfold to move from one cage to another. In contrast, stiff chains travel mainly in straight paths within inter-obstacle channels, while occasionally changing their direction of motion. Both the duration of caging and persistent directed migration within the channels decrease with increasing the activity level. As a consequence, at high active forces polymers overcome confinement effects and transport within inter-obstacle pores as swiftly as those in free space. We explain the center of mass dynamics of semiflexible polymers in terms of active force and obstacle packing fraction by developing an approximate analytical theory.


title: Ensemble Distillation for Unsupervised Constituency Parsing

abstract: We investigate the unsupervised constituency parsing task, which organizes words and phrases of a sentence into a hierarchical structure without using linguistically annotated data. We observe that existing unsupervised parsers capture differing aspects of parsing structures, which can be leveraged to enhance unsupervised parsing performance. To this end, we propose a notion of "tree averaging," based on which we further propose a novel ensemble method for unsupervised parsing. To improve inference efficiency, we further distill the ensemble knowledge into a student model; such an ensemble-then-distill process is an effective approach to mitigate the over-smoothing problem existing in common multi-teacher distilling methods. Experiments show that our method surpasses all previous approaches, consistently demonstrating its effectiveness and robustness across various runs, with different ensemble components, and under domain-shift conditions.


title: On the Peak-to-Average Power Ratio of Vibration Signals: Analysis and Signal Companding for an Efficient Remote Vibration-Based Condition Monitoring

abstract: Vibration-based condition monitoring (VBCM) is widely utilized in various applications due to its non-destructive nature. Recent advancements in sensor technology, the Internet of Things (IoT), and computing have enabled the facilitation of reliable distributed VBCM where sensor nodes are deployed at multiple locations and connected wirelessly to monitoring centers. However, sensor nodes are typically constrained by limited power resources, necessitating control over the peak-to-average power ratio (PAPR) of the generated vibration signals. Effective control of PAPR is crucial to prevent nonlinear distortion and reduce power consumption within the node. Additionally, avoiding nonlinear distortion in the vibration signal and preserving its waveform is essential to ensure the reliability of condition monitoring. This paper conducts an in-depth analysis of the PAPR of vibration signals in VBCM systems, evaluates the impact of nonlinear power amplification on the system performance, and proposes a lightweight autoencoder-based signal companding scheme to control the PAPR to improve power efficiency and mitigate the impact of nonlinear distortion. The proposed scheme employs a lightweight reconstruction autoencoder with a compression-based activation function in the source to compress the vibration signals and avoid increasing the average power of the compressed signal. In the destination, the proposed scheme uses a denoising-expansion autoencoder to expand the compressed signals while minimizing noise enhancement during the expansion process. The experimental results demonstrate the effectiveness of the proposed companding scheme in preventing nonlinear distortion, improving the efficiency of power amplification in the source, and restoring the PAPR characteristics in the destination while avoiding the undesired effect of noise expansion.


title: Software Testing and Code Refactoring: A Survey with Practitioners

abstract: Nowadays, software testing professionals are commonly required to develop coding skills to work on test automation. One essential skill required from those who code is the ability to implement code refactoring, a valued quality aspect of software development; however, software developers usually encounter obstacles in successfully applying this practice. In this scenario, the present study aims to explore how software testing professionals (e.g., software testers, test engineers, test analysts, and software QAs) deal with code refactoring to understand the benefits and limitations of this practice in the context of software testing. We followed the guidelines to conduct surveys in software engineering and applied three sampling techniques, namely convenience sampling, purposive sampling, and snowballing sampling, to collect data from testing professionals. We received answers from 80 individuals reporting their experience refactoring the code of automated tests. We concluded that in the context of software testing, refactoring offers several benefits, such as supporting the maintenance of automated tests and improving the performance of the testing team. However, practitioners might encounter barriers in effectively implementing this practice, in particular, the lack of interest from managers and leaders. Our study raises discussions on the importance of having testing professionals implement refactoring in the code of automated tests, allowing them to improve their coding abilities.


title: Perceiver-based CDF Modeling for Time Series Forecasting

abstract: Transformers have demonstrated remarkable efficacy in forecasting time series data. However, their extensive dependence on self-attention mechanisms demands significant computational resources, thereby limiting their practical applicability across diverse tasks, especially in multimodal problems. In this work, we propose a new architecture, called perceiver-CDF, for modeling cumulative distribution functions (CDF) of time series data. Our approach combines the perceiver architecture with a copula-based attention mechanism tailored for multimodal time series prediction. By leveraging the perceiver, our model efficiently transforms high-dimensional and multimodal data into a compact latent space, thereby significantly reducing computational demands. Subsequently, we implement a copula-based attention mechanism to construct the joint distribution of missing data for prediction. Further, we propose an output variance testing mechanism to effectively mitigate error propagation during prediction. To enhance efficiency and reduce complexity, we introduce midpoint inference for the local attention mechanism. This enables the model to efficiently capture dependencies within nearby imputed samples without considering all previous samples. The experiments on the unimodal and multimodal benchmarks consistently demonstrate a 20% improvement over state-of-the-art methods while utilizing less than half of the computational resources.


title: Stray magnetic fields from elliptical-shaped and stadium-shaped ferromagnets

abstract: An artificial spin ice consisting of numerous ferromagnets has attracted attention because of its applicability to practical devices. The ferromagnets interact through their stray magnetic field and show various functionality. The ferromagnetic element in the spin ice was recently made in elliptical-shape or stadium-shape. The former has a narrow edge, expecting to generate a large stray magnetic field. The latter has a large volume and is also expected to generate a large stray magnetic field. Here, we estimate the stray magnetic field by numerically integrating the solution of the Poisson equation. When magnetization is parallel to an easy axis, the elliptical-shaped ferromagnet generates a larger stray magnetic field than the stadium-shaped ferromagnet. The stray magnetic fields from both ferromagnets for arbitrary magnetization directions are also investigated.


title: Vortex in superconducting thin-film strips of arbitrary width

abstract: The currents and field distributions of a vortex in a thin superconducting strip of a width $W$ is considered. It is shown that unlike infinite films where the vortex field crosses the film only in one direction (say, from the half-space under the film to the half-space above it), in strips (as well as in finite size film samples of any shape) the field lines go back to the lower half-space not only out of the sample but also through certain strip parts. The strip patches where the magnetic lines ``dive" back to the space under the strip are situated mostly near the strip edges at the strip and out of it. The magnetic flux through the strip associated with the vortex is shown to be less than the flux quantum and depends on the vortex position. The suppression of the vortex flux is strong in narrow strips with $W<<\Lambda=2\lambda^2/d$ where $\lambda$ is the bulk London penetration depth and $d$ is the film thickness. The vortex energy scales roughly with $W/\Lambda$ and depends on vortex position.


title: Predicting Future Spatiotemporal Occupancy Grids with Semantics for Autonomous Driving

abstract: For autonomous vehicles to proactively plan safe trajectories and make informed decisions, they must be able to predict the future occupancy states of the local environment. However, common issues with occupancy prediction include predictions where moving objects vanish or become blurred, particularly at longer time horizons. We propose an environment prediction framework that incorporates environment semantics for future occupancy prediction. Our method first semantically segments the environment and uses this information along with the occupancy information to predict the spatiotemporal evolution of the environment. We validate our approach on the real-world Waymo Open Dataset. Compared to baseline methods, our model has higher prediction accuracy and is capable of maintaining moving object appearances in the predictions for longer prediction time horizons.


title: A Strategy for Identifying Periodic Sources Contributing to the Galactic Center Excess

abstract: The origin of the Galactic Center gamma-ray excess has not been conclusively determined after over a decade of careful study. The two most widely discussed possibilities are a population of millisecond pulsars (MSPs), and annihilation of dark matter particles. In contrast with annihilating dark matter, MSPs are expected to produce periodic emission. We show that even though the number of photons contributing to the excess is small, there is potentially sufficient information in the data from Fermi to detect a periodic MSP signal. Such a detection would definitively prove that at least some fraction of the excess is due to MSPs. We argue that this conclusion is robust to potential timing perturbations of the gamma-ray photons, such as those due to Earth's orbit, even if the number of parameters that must be used to model the perturbations is $\sim 7$.


title: Super-Earth LHS3844b is tidally locked

abstract: Short period exoplanets on circular orbits are thought to be tidally locked into synchronous rotation. If tidally locked, these planets must possess permanent day- and nightsides, with extreme irradiation on the dayside and none on the nightside. However, so far the tidal locking hypothesis for exoplanets is supported by little to no empirical evidence. Previous work showed that the super-Earth LHS 3844b likely has no atmosphere, which makes it ideal for constraining the planet's rotation. Here we revisit the Spitzer phase curve of LHS 3844b with a thermal model of an atmosphere-less planet and analyze the impact of non-synchronous rotation, eccentricity, tidal dissipation, and surface composition. Based on the lack of observed strong tidal heating we rule out rapid non-synchronous rotation (including a Mercury-like 3:2 spin-orbit resonance) and constrain the planet's eccentricity to less than 0.001 (more circular than Io's orbit). In addition, LHS 3844b's phase curve implies that the planet either still experiences weak tidal heating via a small-but-nonzero eccentricity (requiring an undetected orbital companion), or that its surface has been darkened by space weathering; of these two scenarios we consider space weathering more likely. Our results thus support the hypothesis that short period rocky exoplanets are tidally locked, and further show that space weathering can significantly modify the surfaces of atmosphere-less exoplanets.


title: Large Language Models for Test-Free Fault Localization

abstract: Fault Localization (FL) aims to automatically localize buggy lines of code, a key first step in many manual and automatic debugging tasks. Previous FL techniques assume the provision of input tests, and often require extensive program analysis, program instrumentation, or data preprocessing. Prior work on deep learning for APR struggles to learn from small datasets and produces limited results on real-world programs. Inspired by the ability of large language models (LLMs) of code to adapt to new tasks based on very few examples, we investigate the applicability of LLMs to line level fault localization. Specifically, we propose to overcome the left-to-right nature of LLMs by fine-tuning a small set of bidirectional adapter layers on top of the representations learned by LLMs to produce LLMAO, the first language model based fault localization approach that locates buggy lines of code without any test coverage information. We fine-tune LLMs with 350 million, 6 billion, and 16 billion parameters on small, manually curated corpora of buggy programs such as the Defects4J corpus. We observe that our technique achieves substantially more confidence in fault localization when built on the larger models, with bug localization performance scaling consistently with the LLM size. Our empirical evaluation shows that LLMAO improves the Top-1 results over the state-of-the-art machine learning fault localization (MLFL) baselines by 2.3%-54.4%, and Top-5 results by 14.4%-35.6%. LLMAO is also the first FL technique trained using a language model architecture that can detect security vulnerabilities down to the code line level.


title: Can GPT-4 Replicate Empirical Software Engineering Research?

abstract: Empirical software engineering research on production systems has brought forth a better understanding of the software engineering process for practitioners and researchers alike. However, only a small subset of production systems is studied, limiting the impact of this research. While software engineering practitioners could benefit from replicating research on their own data, this poses its own set of challenges, since performing replications requires a deep understanding of research methodologies and subtle nuances in software engineering data. Given that large language models (LLMs), such as GPT-4, show promise in tackling both software engineering- and science-related tasks, these models could help replicate and thus democratize empirical software engineering research. In this paper, we examine GPT-4's abilities to perform replications of empirical software engineering research on new data. We study their ability to surface assumptions made in empirical software engineering research methodologies, as well as their ability to plan and generate code for analysis pipelines on seven empirical software engineering papers. We perform a user study with 14 participants with software engineering research expertise, who evaluate GPT-4-generated assumptions and analysis plans (i.e., a list of module specifications) from the papers. We find that GPT-4 is able to surface correct assumptions, but struggles to generate ones that apply common knowledge about software engineering data. In a manual analysis of the generated code, we find that the GPT-4-generated code contains correct high-level logic, given a subset of the methodology. However, the code contains many small implementation-level errors, reflecting a lack of software engineering knowledge. Our findings have implications for leveraging LLMs for software engineering research as well as practitioner data scientists in software teams.


title: Time-LLM: Time Series Forecasting by Reprogramming Large Language Models

abstract: Time series forecasting holds significant importance in many real-world dynamic systems and has been extensively studied. Unlike natural language process (NLP) and computer vision (CV), where a single large model can tackle multiple tasks, models for time series forecasting are often specialized, necessitating distinct designs for different tasks and applications. While pre-trained foundation models have made impressive strides in NLP and CV, their development in time series domains has been constrained by data sparsity. Recent studies have revealed that large language models (LLMs) possess robust pattern recognition and reasoning abilities over complex sequences of tokens. However, the challenge remains in effectively aligning the modalities of time series data and natural language to leverage these capabilities. In this work, we present Time-LLM, a reprogramming framework to repurpose LLMs for general time series forecasting with the backbone language models kept intact. We begin by reprogramming the input time series with text prototypes before feeding it into the frozen LLM to align the two modalities. To augment the LLM's ability to reason with time series data, we propose Prompt-as-Prefix (PaP), which enriches the input context and directs the transformation of reprogrammed input patches. The transformed time series patches from the LLM are finally projected to obtain the forecasts. Our comprehensive evaluations demonstrate that Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models. Moreover, Time-LLM excels in both few-shot and zero-shot learning scenarios.


title: Error Correction for DNA Storage

abstract: DNA-based storage is an emerging storage technology that provides high information density and long duration. Due to the physical constraints in the reading and writing processes, error correction in DNA storage poses several interesting coding theoretic challenges, some of which are new. In this paper, we give a brief introduction to some of the coding challenges for DNA-based storage, including deletion/insertion correcting codes, codes over sliced channels, and duplication correcting codes.


title: Analysis of Local Anisotropy Fluctuations in Compact Objects

abstract: Mathematical modeling within the framework of the general theory of relativity has been used to explain the behavior and structure of massive objects as neutron stars, quasars, black holes, pulsars and white dwarfs and requires finding the exact solutions of the Einstein-Maxwell system. In this paper we study the effects induced by fluctuations of local anisotropy in a new family of anisotropic solutions depending on a parameter, with a specific value that provides a radial pressure having the same functional dependence on the radial coordinate as the Schwarzschild solution. It is shown the effect the functional dependence on the radial coordinate has in the occurrence of cracking within the sphere when anisotropy fluctuations are allowed.


title: Phase transition and universality in the majority-rule model with social noises on various network topologies

abstract: This paper explores the impact of social noise, characterized by either independence or anticonformity, on order-disorder phase transitions within the majority-rule model framework. The study spans diverse network topologies, including the complete graph, two-dimensional (2-D) square lattice, three-dimensional (3-D) square lattice, and heterogeneous networks such as Watts-Strogatz, Barab\'asi-Albert, and Erd\H{o}s-Renyi networks. The social noise is represented by the fundamental parameter $p$, denoting the likelihood of agents displaying independent or anticonformist tendencies. The nature of the phase transitions observed depends on both the probability parameter $p$ and the network topologies. Specifically, the models show continuous phase transitions on the complete graph and the 2-D square lattice, with critical points subject to parameter variations. However, in the 3-D lattice, models featuring independence noise notably lack a distinct phase transition, whereas those with anticonformity noise still exhibit continuous phase transitions. Through finite-size scaling analysis and critical exponents evaluation, we confirm that all models align with the Ising model universality class, except in the 3-D model.


title: Nugget: Neural Agglomerative Embeddings of Text

abstract: Embedding text sequences is a widespread requirement in modern language understanding. Existing approaches focus largely on constant-size representations. This is problematic, as the amount of information contained in text often varies with the length of the input. We propose a solution called Nugget, which encodes language into a representation based on a dynamically selected subset of input tokens. These nuggets are learned through tasks like autoencoding and machine translation, and intuitively segment language into meaningful units. We demonstrate Nugget outperforms related approaches in tasks involving semantic comparison. Finally, we illustrate these compact units allow for expanding the contextual window of a language model (LM), suggesting new future LMs that can condition on significantly larger amounts of content.


title: Health Guardian: Using Multi-modal Data to Understand Individual Health

abstract: Artificial intelligence (AI) has shown great promise in revolutionizing the field of digital health by improving disease diagnosis, treatment, and prevention. This paper describes the Health Guardian platform, a non-commercial, scientific research-based platform developed by the IBM Digital Health team to rapidly translate AI research into cloud-based microservices. The platform can collect health-related data from various digital devices, including wearables and mobile applications. Its flexible architecture supports microservices that accept diverse data types such as text, audio, and video, expanding the range of digital health assessments and enabling holistic health evaluations by capturing voice, facial, and motion bio-signals. These microservices can be deployed to a clinical cohort specified through the Clinical Task Manager (CTM). The CTM then collects multi-modal, clinical data that can iteratively improve the accuracy of AI predictive models, discover new disease mechanisms, or identify novel biomarkers. This paper highlights three microservices with different input data types, including a text-based microservice for depression assessment, a video-based microservice for sit-to-stand mobility assessment, and a wearable-based microservice for functional mobility assessment. The CTM is also discussed as a tool to help design and set up clinical studies to unlock the full potential of the platform. Today, the Health Guardian platform is being leveraged in collaboration with research partners to optimize the development of AI models by utilizing a multitude of input sources. This approach streamlines research efforts, enhances efficiency, and facilitates the development and validation of digital health applications.


title: New models of d-dimensional black holes without inner horizon and with an integrable singularity

abstract: Theoretically, it has been proposed that objects traveling radially along regular black holes (RBHs) would not be destroyed because of finite tidal forces and the absence of a singularity. However, the matter source allows the creation of an inner horizon linked to an unstable de Sitter core due to mass inflation instability. This inner horizon also gives rise to the appearance of a remnant, inhibiting complete evaporation. We introduce here a $d$-dimensional black hole model with Localized Sources of Matter (LSM), characterized by the absence of an inner horizon and featuring a central integrable singularity instead of an unstable de Sitter core. In our model, any object tracing a radial and timelike world-line would not be crushed by the singularity. This is attributed to finite tidal forces, the extendability of radial geodesics, and the weak nature of the singularity. Our LSM model enables the potential complete evaporation down to $r_h=0$ without forming a remnant. In higher dimensions, complete evaporation occurs through a phase transition, which could occur at Planck scales and be speculatively driven by the Generalized Uncertainty Principle (GUP). Unlike RBHs, our model satisfies the energy conditions. We demonstrate a linear correction to the conventional area law of entropy, distinct from the RBH's correction. Additionally, we investigate the stability of the solutions through the speed of sound.


title: Learning Expected Appearances for Intraoperative Registration during Neurosurgery

abstract: We present a novel method for intraoperative patient-to-image registration by learning Expected Appearances. Our method uses preoperative imaging to synthesize patient-specific expected views through a surgical microscope for a predicted range of transformations. Our method estimates the camera pose by minimizing the dissimilarity between the intraoperative 2D view through the optical microscope and the synthesized expected texture. In contrast to conventional methods, our approach transfers the processing tasks to the preoperative stage, reducing thereby the impact of low-resolution, distorted, and noisy intraoperative images, that often degrade the registration accuracy. We applied our method in the context of neuronavigation during brain surgery. We evaluated our approach on synthetic data and on retrospective data from 6 clinical cases. Our method outperformed state-of-the-art methods and achieved accuracies that met current clinical standards.


title: Exact results for some extremal problems on expansions I

abstract: The expansion of a graph $F$, denoted by $F^3$, is the $3$-graph obtained from $F$ by adding a new vertex to each edge such that different edges receive different vertices. For large $n$, we establish tight upper bounds for: The maximum number of edges in an $n$-vertex $3$-graph that does not contain $T^3$ for certain class $\mathcal{T}$ of trees, sharpening (partially) a result of Kostochka--Mubayi--Verstra\"{e}te. The minimum number of colors needed to color the complete $n$-vertex $3$-graph to ensure the existence of a rainbow copy of $F^3$ when $F$ is a graph obtained from some tree $T\in \mathcal{T}$ by adding a new edge, extending anti-Ramsey results on $P_{2t}^3$ by Gu--Li--Shi and $C_{2t}^3$ by Tang--Li--Yan. The maximum number of edges in an $n$-vertex $3$-graph whose shadow does not contain the shadow of $C_{k}^3$ or $T^3$ for $T\in \mathcal{T}$, answering a question of Lv \etal on generalized Tur\'{a}n problems.


title: Blending Imitation and Reinforcement Learning for Robust Policy Improvement

abstract: While reinforcement learning (RL) has shown promising performance, its sample complexity continues to be a substantial hurdle, restricting its broader application across a variety of domains. Imitation learning (IL) utilizes oracles to improve sample efficiency, yet it is often constrained by the quality of the oracles deployed. which actively interleaves between IL and RL based on an online estimate of their performance. RPI draws on the strengths of IL, using oracle queries to facilitate exploration, an aspect that is notably challenging in sparse-reward RL, particularly during the early stages of learning. As learning unfolds, RPI gradually transitions to RL, effectively treating the learned policy as an improved oracle. This algorithm is capable of learning from and improving upon a diverse set of black-box oracles. Integral to RPI are Robust Active Policy Selection (RAPS) and Robust Policy Gradient (RPG), both of which reason over whether to perform state-wise imitation from the oracles or learn from its own value function when the learner's performance surpasses that of the oracles in a specific state. Empirical evaluations and theoretical analysis validate that RPI excels in comparison to existing state-of-the-art methodologies, demonstrating superior performance across various benchmark domains.


title: RETRO: Reactive Trajectory Optimization for Real-Time Robot Motion Planning in Dynamic Environments

abstract: Reactive trajectory optimization for robotics presents formidable challenges, demanding the rapid generation of purposeful robot motion in complex and swiftly changing dynamic environments. While much existing research predominantly addresses robotic motion planning with predefined objectives, emerging problems in robotic trajectory optimization frequently involve dynamically evolving objectives and stochastic motion dynamics. However, effectively addressing such reactive trajectory optimization challenges for robot manipulators proves difficult due to inefficient, high-dimensional trajectory representations and a lack of consideration for time optimization. In response, we introduce a novel trajectory optimization framework called RETRO. RETRO employs adaptive optimization techniques that span both spatial and temporal dimensions. As a result, it achieves a remarkable computing complexity of $O(T^{2.4}) + O(Tn^{2})$, a significant improvement over the traditional application of DDP, which leads to a complexity of $O(n^{4})$ when reasonable time step sizes are used. To evaluate RETRO's performance in terms of error, we conducted a comprehensive analysis of its regret bounds, comparing it to an Oracle value function obtained through an Oracle trajectory optimization algorithm. Our analytical findings demonstrate that RETRO's total regret can be upper-bounded by a function of the chosen time step size. Moreover, our approach delivers smoothly optimized robot trajectories within the joint space, offering flexibility and adaptability for various tasks. It can seamlessly integrate task-specific requirements such as collision avoidance while maintaining real-time control rates. We validate the effectiveness of our framework through extensive simulations and real-world robot experiments in closed-loop manipulation scenarios.


title: Randomized Dimension Reduction with Statistical Guarantees

abstract: Large models and enormous data are essential driving forces of the unprecedented successes achieved by modern algorithms, especially in scientific computing and machine learning. Nevertheless, the growing dimensionality and model complexity, as well as the non-negligible workload of data pre-processing, also bring formidable costs to such successes in both computation and data aggregation. As the deceleration of Moore's Law slackens the cost reduction of computation from the hardware level, fast heuristics for expensive classical routines and efficient algorithms for exploiting limited data are increasingly indispensable for pushing the limit of algorithm potency. This thesis explores some of such algorithms for fast execution and efficient data utilization. From the computational efficiency perspective, we design and analyze fast randomized low-rank decomposition algorithms for large matrices based on "matrix sketching", which can be regarded as a dimension reduction strategy in the data space. These include the randomized pivoting-based interpolative and CUR decomposition discussed in Chapter 2 and the randomized subspace approximations discussed in Chapter 3. From the sample efficiency perspective, we focus on learning algorithms with various incorporations of data augmentation that improve generalization and distributional robustness provably. Specifically, Chapter 4 presents a sample complexity analysis for data augmentation consistency regularization where we view sample efficiency from the lens of dimension reduction in the function space. Then in Chapter 5, we introduce an adaptively weighted data augmentation consistency regularization algorithm for distributionally robust optimization with applications in medical image segmentation.


title: Control of Soft Pneumatic Actuators with Approximated Dynamical Modeling

abstract: This paper introduces a full system modeling strategy for a syringe pump and soft pneumatic actuators(SPAs). The soft actuator is conceptualized as a beam structure, utilizing a second-order bending model. The equation of natural frequency is derived from Euler's bending theory, while the damping ratio is estimated by fitting step responses of soft pneumatic actuators. Evaluation of model uncertainty underscores the robustness of our modeling methodology. To validate our approach, we deploy it across four prototypes varying in dimensional parameters. Furthermore, a syringe pump is designed to drive the actuator, and a pressure model is proposed to construct a full system model. By employing this full system model, the Linear-Quadratic Regulator (LQR) controller is implemented to control the soft actuator, achieving high-speed responses and high accuracy in both step response and square wave function response tests. Both the modeling method and the LQR controller are thoroughly evaluated through experiments. Lastly, a gripper, consisting of two actuators with a feedback controller, demonstrates stable grasping of delicate objects with a significantly higher success rate.


title: Deformations and desingularizations of conically singular associative submanifolds

abstract: The proposals of Joyce [Joy18], and Doan and Walpuski [DW19] on counting closed associative submanifolds of $G_2$-manifolds depend on various conjectural transitions. This article contributes to the transitions arising from the degenerations of associative submanifolds into conically singular (CS) associative submanifolds. First, we study the moduli space of CS associative submanifolds with singularities at a finite number of points locally modeled on associative cones in $\mathbf R^7$ and their transversality in a co-closed $G_2$-manifold as well as in a path of co-closed $G_2$-structures. We prove that for a generic co-closed $G_2$-structure and for a generic path of co-closed $G_2$-structures, there are no CS associative submanifolds having singularities modeled on cones with stability-index greater than $0$ and $1$, respectively. We establish that associative cones with links null-torsion holomorphic curves in $S^6$ have stability-index greater than $4$, and all special Lagrangian cones in $\mathbf C^3$ have stability-index greater than equal to $1$ with equality only holding for the Harvey-Lawson $T^2$-cone and a transverse pair of planes. Second, we study the desingularizations of CS associative submanifolds in a one parameter family of co-closed $G_2$-structures. Consequently, we derive desingularization results relating the above transitions for CS associative submanifolds with Harvey-Lawson $T^2$-cone singularity and for associative submanifolds with transverse self-intersection.


title: Intelligent Reflecting Surface Aided MIMO Networks: Distributed or Centralized Architecture?

abstract: We investigate the capacity of a broadcast channel with a multi-antenna base station (BS) sending independent messages to multiple users, aided by IRSs with N elements. In particular, both the distributed and centralized IRS deployment architectures are considered. Regarding the distributed IRS, the N IRS elements form multiple IRSs and each of them is installed near a user cluster; while for the centralized IRS, all IRS elements are located in the vicinity of the BS. To draw essential insights, we first derive the maximum capacity achieved by the distributed IRS and centralized IRS, respectively, under the assumption of line-of-sight propagation and homogeneous channel setups. By capturing the fundamental tradeoff between the spatial multiplexing gain and passive beamforming gain, we rigourously prove that the capacity of the distributed IRS is higher than that of the centralized IRS provided that the total number of IRS elements is above a threshold. Motivated by the superiority of the distributed IRS, we then focus on the transmission and element allocation design under the distributed IRS. By exploiting the user channel correlation of intra-clusters and inter-clusters, an efficient hybrid multiple access scheme relying on both spatial and time domains is proposed to fully exploit both the passive beamforming gain and spatial DoF. Moreover, the IRS element allocation problem is investigated for the objectives of sum-rate maximization and minimum user rate maximization, respectively. Finally, extensive numerical results are provided to validate our theoretical finding and also to unveil the effectiveness of the distributed IRS for improving the system capacity under various system setups.


title: Sex-specific ultraviolet radiation tolerance across Drosophila

abstract: The genetic basis of phenotypic differences between species is among the most longstanding questions in evolutionary biology. How new genes form and the processes selection acts to produce differences across species are fundamental to understand how species persist and evolve in an ever-changing environment. Adaptation and genetic innovation arise in the genome by a variety of sources. Functional genomics requires both intrinsic genetic discoveries, as well as empirical testing to observe adaptation between lineages. Here we explore two species of Drosophila on the island of Sao Tome and mainland Africa, D. santomea and D. yakuba. These two species both inhabit the island, but occupy differing species distributions based on elevation, with D. yakuba also having populations on mainland Africa. Intrinsic evidence shows genes between species may have a role in adaptation to higher UV tolerance with DNA repair mechanisms (PARP) and resistance to humeral stress lethal effects (Victoria). We conducted empirical assays between island D. santomea, D. yakuba, and mainland D. yakuba. Flies were shocked with UVB radiation (@ 302 nm) at 1650-1990 mW/cm2 for 30 minutes on a transilluminator apparatus. Custom 5-wall acrylic enclosures were constructed for viewing and containment of flies. All assays were filmed. Island groups did show significant differences between fall-time under UV stress and recovery time post-UV stress test between regions and sex. This study shows evidence that mainland flies are less resistant to UV radiation than their island counterparts. Further work exploring the genetic basis for UV tolerance will be conducted from empirical assays. Understanding the mechanisms and processes that promote adaptation and testing extrinsic traits within the context of the genome is crucially important to understand evolutionary machinery.


title: Ultra-High-Energy Gamma-Ray Astronomy

abstract: Ultra-High Energy (UHE, $>$0.1\,PeV) $\gamma$-ray Astronomy is rapidly evolving into an expanding branch of the $\gamma$-ray astronomy with the surprising discovery of 12 PeVatrons and the detection of a handful of photons above 1 PeV. Nearly all known celestial object types that have emissions in the TeV band are found also emitting UHE photons. UHE $\gamma$-rays have a well-defined horizon inside our galaxy due to the absorption of infrared and cosmic microwave backgrounds in the universe. With the last 30 years, traditional cosmic ray (CR) detection techniques allow the detection of UHE $\gamma$-rays, and opened up the last observation window. For leptonic sources, UHE radiation is in the deep Klein-Nishina regime which is largely suppressed. Therefore UHE $\gamma$-ray detection will help to locate and identify hadronic radiation sources, tracing the historic pursuit for the origin of CRs around the knee of the spectrum. The Crab Nebula is again the focus of attention with measured photon emissions above 1\,PeV. In the absence of hadronic processes, this may indicate the existence of an extreme accelerator of e$^+$/e$^-$. Utilization of the CR extensive air shower detection techniques broadens the field of view of the source observations, enabling the measurement of UHE radiation surrounding the sources. These observations can probe the particle propagation inside and outside the accelerators and the subsequent injection/escape into the interstellar medium.


title: A Volumetric Approach to Monge's Optimal Transport on Surfaces

abstract: We propose a volumetric formulation for computing the Optimal Transport problem defined on surfaces in $\mathbb{R}^3$, found in disciplines like optics, computer graphics, and computational methodologies. Instead of directly tackling the original problem on the surface, we define a new Optimal Transport problem on a thin tubular region, $T_{\epsilon}$, adjacent to the surface. This extension offers enhanced flexibility and simplicity for numerical discretization on Cartesian grids. The Optimal Transport mapping and potential function computed on $T_{\epsilon}$ are consistent with the original problem on surfaces. We demonstrate that, with the proposed volumetric approach, it is possible to use simple and straightforward numerical methods to solve Optimal Transport for $\Gamma = \mathbb{S}^2$ and the $2$-torus.


title: An approach to study interactions of antineutrons with CsI at a $J/\psi$ factory

abstract: Cesium Iodide (CsI) crystals are widely used in high-energy physics for their scintillation properties that enable detection of charged and neutral particles via direct and indirect ionization and form the basis of electromagnetic calorimeters. However, knowledge of antineutron interactions with CsI is limited due to the difficulty of obtaining sources of antineutron of sufficient intensity and energy definition. As antineutron are abundantly produced by many processes it would be particularly useful to improve understanding of the interactions of antineutrons with CsI crystals. We propose to use the decay $J/\psi\to p\pi^-\bar{n}$ at the BEPCII $J/\psi$ factory as a source of antineutrons using the BESIII detector with a CsI target added between the beam pipe and the detector. The BESIII Monte Carlo simulation with varying thicknesses of CsI target is used to validate the approach and optimize the target thickness. Selecting $p\pi^-$ charged particle tracks from the Monte Carlo we obtain clean antineutron samples with well defined momentum and direction. The selection efficiency, momentum and angular resolutions, as well as the interaction probability between antineutron and the CsI target are estimated. As the CsI thickness is increased more antineutron CsI interactions are obtained,however the quality of the $p\pi^-$ selection is degraded. The Monte Carlo study yields an optimum thickness that balances these effects. This approach can be applied to similar experiments with other types of target materials to measure baryons such as liquid hydrogen/deuterium and $\Lambda/\Xi$ hyperons.


title: 5G Network Slicing: Analysis of Multiple Machine Learning Classifiers

abstract: The division of one physical 5G communications infrastructure into several virtual network slices with distinct characteristics such as bandwidth, latency, reliability, security, and service quality is known as 5G network slicing. Each slice is a separate logical network that meets the requirements of specific services or use cases, such as virtual reality, gaming, autonomous vehicles, or industrial automation. The network slice can be adjusted dynamically to meet the changing demands of the service, resulting in a more cost-effective and efficient approach to delivering diverse services and applications over a shared infrastructure. This paper assesses various machine learning techniques, including the logistic regression model, linear discriminant model, k-nearest neighbor's model, decision tree model, random forest model, SVC BernoulliNB model, and GaussianNB model, to investigate the accuracy and precision of each model on detecting network slices. The report also gives an overview of 5G network slicing.


title: A generative approach to frame-level multi-competitor races

abstract: Multi-competitor races often feature complicated within-race strategies that are difficult to capture when training data on race outcome level data. Further, models which do not account for such strategic effects may suffer from confounded inferences and predictions. In this work we develop a general generative model for multi-competitor races which allows analysts to explicitly model certain strategic effects such as changing lanes or drafting and separate these impacts from competitor ability. The generative model allows one to simulate full races from any real or created starting position which opens new avenues for attributing value to within-race actions and to perform counter-factual analyses. This methodology is sufficiently general to apply to any track based multi-competitor races where both tracking data is available and competitor movement is well described by simultaneous forward and lateral movements. We apply this methodology to one-mile horse races using data provided by the New York Racing Association (NYRA) and the New York Thoroughbred Horsemen's Association (NYTHA) for the Big Data Derby 2022 Kaggle Competition. This data features granular tracking data for all horses at the frame-level (occurring at approximately 4hz). We demonstrate how this model can yield new inferences, such as the estimation of horse-specific speed profiles which vary over phases of the race, and examples of posterior predictive counterfactual simulations to answer questions of interest such as starting lane impacts on race outcomes.


title: Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns

abstract: Attention, specifically scaled dot-product attention, has proven effective for natural language, but it does not have a mechanism for handling hierarchical patterns of arbitrary nesting depth, which limits its ability to recognize certain syntactic structures. To address this shortcoming, we propose stack attention: an attention operator that incorporates stacks, inspired by their theoretical connections to context-free languages (CFLs). We show that stack attention is analogous to standard attention, but with a latent model of syntax that requires no syntactic supervision. We propose two variants: one related to deterministic pushdown automata (PDAs) and one based on nondeterministic PDAs, which allows transformers to recognize arbitrary CFLs. We show that transformers with stack attention are very effective at learning CFLs that standard transformers struggle on, achieving strong results on a CFL with theoretically maximal parsing difficulty. We also show that stack attention is more effective at natural language modeling under a constrained parameter budget, and we include results on machine translation.


title: Two-color exchangeability does not imply exchangeability

abstract: We provide simple examples of two-color exchangeable sequences $\xi=(\xi_1, \xi_2, \ldots, \xi_n)$ that are not exchangeable. This answers a question of Bladt and Shaiderman~\cite[Question 2.6]{bladt2019characterisation} for finite two-color exchangeable sequences.