title: UGAE: A Novel Approach to Non-exponential Discounting

abstract: The discounting mechanism in Reinforcement Learning determines the relative importance of future and present rewards. While exponential discounting is widely used in practice, non-exponential discounting methods that align with human behavior are often desirable for creating human-like agents. However, non-exponential discounting methods cannot be directly applied in modern on-policy actor-critic algorithms. To address this issue, we propose Universal Generalized Advantage Estimation (UGAE), which allows for the computation of GAE advantage values with arbitrary discounting. Additionally, we introduce Beta-weighted discounting, a continuous interpolation between exponential and hyperbolic discounting, to increase flexibility in choosing a discounting method. To showcase the utility of UGAE, we provide an analysis of the properties of various discounting methods. We also show experimentally that agents with non-exponential discounting trained via UGAE outperform variants trained with Monte Carlo advantage estimation. Through analysis of various discounting methods and experiments, we demonstrate the superior performance of UGAE with Beta-weighted discounting over the Monte Carlo baseline on standard RL benchmarks. UGAE is simple and easily integrated into any advantage-based algorithm as a replacement for the standard recursive GAE.


title: Languages with Decidable Learning: A Meta-theorem

abstract: We study expression learning problems with syntactic restrictions and introduce the class of finite-aspect checkable languages to characterize symbolic languages that admit decidable learning. The semantics of such languages can be defined using a bounded amount of auxiliary information that is independent of expression size but depends on a fixed structure over which evaluation occurs. We introduce a generic programming language for writing programs that evaluate expression syntax trees, and we give a meta-theorem that connects such programs for finite-aspect checkable languages to finite tree automata, which allows us to derive new decidable learning results and decision procedures for several expression learning problems by writing programs in the programming language.


title: A single player and a mass of agents: a pursuit evasion-like game

abstract: We study a finite-horizon differential game of pursuit-evasion like, between a single player and a mass of agents. The player and the mass directly control their own evolution, which for the mass is given by a first order PDE of transport equation type. Using also an adapted concept of non-anticipating strategies, we derive an infinite dimensional Isaacs equation, and by dynamic programming techniques we prove that the value function is the unique viscosity solution on a suitable invariant subset of a Hilbert space.


title: Is Distance Matrix Enough for Geometric Deep Learning?

abstract: Graph Neural Networks (GNNs) are often used for tasks involving the 3D geometry of a given graph, such as molecular dynamics simulation. While incorporating Euclidean distance into Message Passing Neural Networks (referred to as Vanilla DisGNN) is a straightforward way to learn the geometry, it has been demonstrated that Vanilla DisGNN is geometrically incomplete. In this work, we first construct families of novel and symmetric geometric graphs that Vanilla DisGNN cannot distinguish even when considering all-pair distances, which greatly expands the existing counterexample families. Our counterexamples show the inherent limitation of Vanilla DisGNN to capture symmetric geometric structures. We then propose $k$-DisGNNs, which can effectively exploit the rich geometry contained in the distance matrix. We demonstrate the high expressive power of $k$-DisGNNs from three perspectives: 1. They can learn high-order geometric information that cannot be captured by Vanilla DisGNN. 2. They can unify some existing well-designed geometric models. 3. They are universal function approximators from geometric graphs to scalars (when $k\geq 2$) and vectors (when $k\geq 3$). Most importantly, we establish a connection between geometric deep learning (GDL) and traditional graph representation learning (GRL), showing that those highly expressive GNN models originally designed for GRL can also be applied to GDL with impressive performance, and that existing complicated, equivariant models are not the only solution. Experiments verify our theory. Our $k$-DisGNNs achieve many new state-of-the-art results on MD17.


title: Rethinking Vision Transformer and Masked Autoencoder in Multimodal Face Anti-Spoofing

abstract: Recently, vision transformer (ViT) based multimodal learning methods have been proposed to improve the robustness of face anti-spoofing (FAS) systems. However, there are still no works to explore the fundamental natures (\textit{e.g.}, modality-aware inputs, suitable multimodal pre-training, and efficient finetuning) in vanilla ViT for multimodal FAS. In this paper, we investigate three key factors (i.e., inputs, pre-training, and finetuning) in ViT for multimodal FAS with RGB, Infrared (IR), and Depth. First, in terms of the ViT inputs, we find that leveraging local feature descriptors benefits the ViT on IR modality but not RGB or Depth modalities. Second, in observation of the inefficiency on direct finetuning the whole or partial ViT, we design an adaptive multimodal adapter (AMA), which can efficiently aggregate local multimodal features while freezing majority of ViT parameters. Finally, in consideration of the task (FAS vs. generic object classification) and modality (multimodal vs. unimodal) gaps, ImageNet pre-trained models might be sub-optimal for the multimodal FAS task. To bridge these gaps, we propose the modality-asymmetric masked autoencoder (M$^{2}$A$^{2}$E) for multimodal FAS self-supervised pre-training without costly annotated labels. Compared with the previous modality-symmetric autoencoder, the proposed M$^{2}$A$^{2}$E is able to learn more intrinsic task-aware representation and compatible with modality-agnostic (e.g., unimodal, bimodal, and trimodal) downstream settings. Extensive experiments with both unimodal (RGB, Depth, IR) and multimodal (RGB+Depth, RGB+IR, Depth+IR, RGB+Depth+IR) settings conducted on multimodal FAS benchmarks demonstrate the superior performance of the proposed methods. We hope these findings and solutions can facilitate the future research for ViT-based multimodal FAS.


title: Verifying Generalization in Deep Learning

abstract: Deep neural networks (DNNs) are the workhorses of deep learning, which constitutes the state of the art in numerous application domains. However, DNN-based decision rules are notoriously prone to poor generalization, i.e., may prove inadequate on inputs not encountered during training. This limitation poses a significant obstacle to employing deep learning for mission-critical tasks, and also in real-world environments that exhibit high variability. We propose a novel, verification-driven methodology for identifying DNN-based decision rules that generalize well to new input domains. Our approach quantifies generalization to an input domain by the extent to which decisions reached by independently trained DNNs are in agreement for inputs in this domain. We show how, by harnessing the power of DNN verification, our approach can be efficiently and effectively realized. We evaluate our verification-based approach on three deep reinforcement learning (DRL) benchmarks, including a system for Internet congestion control. Our results establish the usefulness of our approach. More broadly, our work puts forth a novel objective for formal verification, with the potential for mitigating the risks associated with deploying DNN-based systems in the wild.


title: Removing Image Artifacts From Scratched Lens Protectors

abstract: A protector is placed in front of the camera lens for mobile devices to avoid damage, while the protector itself can be easily scratched accidentally, especially for plastic ones. The artifacts appear in a wide variety of patterns, making it difficult to see through them clearly. Removing image artifacts from the scratched lens protector is inherently challenging due to the occasional flare artifacts and the co-occurring interference within mixed artifacts. Though different methods have been proposed for some specific distortions, they seldom consider such inherent challenges. In our work, we consider the inherent challenges in a unified framework with two cooperative modules, which facilitate the performance boost of each other. We also collect a new dataset from the real world to facilitate training and evaluation purposes. The experimental results demonstrate that our method outperforms the baselines qualitatively and quantitatively. The code and datasets will be released after acceptance.


title: Individualized Treatment Allocation in Sequential Network Games

abstract: Designing individualized allocation of treatments so as to maximize the equilibrium welfare of interacting agents has many policy-relevant applications. Focusing on sequential decision games of interacting agents, this paper develops a method to obtain optimal treatment assignment rules that maximize a social welfare criterion by evaluating stationary distributions of outcomes. Stationary distributions in sequential decision games are given by Gibbs distributions, which are difficult to optimize with respect to a treatment allocation due to analytical and computational complexity. We apply a variational approximation to the stationary distribution and optimize the approximated equilibrium welfare with respect to treatment allocation using a greedy optimization algorithm. We characterize the performance of the variational approximation, deriving a performance guarantee for the greedy optimization algorithm via a welfare regret bound. We implement our proposed method in simulation exercises and an empirical application using the Indian microfinance data (Banerjee et al., 2013), and show it delivers significant welfare gains.


title: Iwasawa invariants of modular forms with $a_p=0$

abstract: Fix a prime $p$ and a cuspidal newform $f$ of level coprime to $p$ with $a_p=0$. Attached to $f$ are signed $p$-adic $L$-functions $L_p^\pm(f)$ and Mazur-Tate elements $\theta_n(f)$, both of which encode arithmetic data about $f$ along the cyclotomic $\mathbf{Z}_p$-extension of $\mathbf{Q}$. We compute the Iwasawa invariants of Mazur-Tate elements in terms of the corresponding invariants of the signed $p$-adic $L$-functions. As corollaries, we determine the $p$-adic valuation of critical values of the $L$-function of $f$, and describe a relation between the Iwasawa invariants of congruent modular forms of weights 2 and $p+1$. Our results provide an asymptotic method for computing the signed Iwasawa invariants attached to newforms of any weight $k\geq 2$ with $a_p=0$.


title: Characterizing the Influence of Charge Extraction Layers on the Performance of Triple-Cation Perovskite Solar Cells

abstract: Selecting suitable charge transport layers and suppressing non-radiative recombination at interfaces to the absorber layer are vital to maximize the efficiency of halide perovskite solar cells. In this work, high-quality perovskite thin films and devices are fabricated with different fullerene-based electron transport layers and different self-assembled monolayers as hole transport layers. We then perform a comparative study of a significant variety of different electrical, optical and photoemission-based characterization techniques to quantify the properties of the solar cells, the individual layers and importantly the interfaces between them. In addition, we highlight the limitations and problems of the different measurements, the insights gained by combining different methods and the different strategies to extract information from the experimental raw data.


title: Matrix valued discrete-continuous functions with the prolate spheroidal property and bispectrality

abstract: Classical prolate spheroidal functions play an important role in the study of time-band limiting, scaling limits of random matrices, and the distribution of the zeros of the Riemann zeta function. We establish an intrinsic relationship between discrete-continuous bispectral functions and the prolate spheroidal phenomenon. The former functions form a vast class, parametrized by an infinite dimensional manifold, and are constructed by Darboux transformations from classical bispectral functions associated to orthogonal polynomials. Special cases include spherical functions. We prove that all such Darboux transformations which are self-adjoint in a certain sense give rise to integral operators possessing commuting differential operators and to discrete integral operators possessing commuting shift operators. One particularly striking implication of this is the correspondence between discrete and continuous pairs of commuting operators. Moreover, all results are proved in the setting of matrix valued functions, which provides further advantages for applications. Our methods rely on the use of noncommutative matrix valued Fourier algebras associated to discrete-continuous bispectral functions. We produce the commuting differential and shift operators in a constructive way with explicit upper bounds on their orders and bandwidths, which is illustrated with many concrete examples.


title: Reflexive polygons and rational elliptic surfaces

abstract: In this note we study in detail the geometry of eight rational elliptic surfaces naturally associated to the sixteen reflexive polygons. The elliptic fibrations supported by these surfaces correspond under mirror symmetry to the eight families of smooth del Pezzo surfaces with very ample anticanonical bundle.


title: Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetes

abstract: Medical experts may use Artificial Intelligence (AI) systems with greater trust if these are supported by contextual explanations that let the practitioner connect system inferences to their context of use. However, their importance in improving model usage and understanding has not been extensively studied. Hence, we consider a comorbidity risk prediction scenario and focus on contexts regarding the patients clinical state, AI predictions about their risk of complications, and algorithmic explanations supporting the predictions. We explore how relevant information for such dimensions can be extracted from Medical guidelines to answer typical questions from clinical practitioners. We identify this as a question answering (QA) task and employ several state-of-the-art LLMs to present contexts around risk prediction model inferences and evaluate their acceptability. Finally, we study the benefits of contextual explanations by building an end-to-end AI pipeline including data cohorting, AI risk modeling, post-hoc model explanations, and prototyped a visual dashboard to present the combined insights from different context dimensions and data sources, while predicting and identifying the drivers of risk of Chronic Kidney Disease - a common type-2 diabetes comorbidity. All of these steps were performed in engagement with medical experts, including a final evaluation of the dashboard results by an expert medical panel. We show that LLMs, in particular BERT and SciBERT, can be readily deployed to extract some relevant explanations to support clinical usage. To understand the value-add of the contextual explanations, the expert panel evaluated these regarding actionable insights in the relevant clinical setting. Overall, our paper is one of the first end-to-end analyses identifying the feasibility and benefits of contextual explanations in a real-world clinical use case.


title: DaliID: Distortion-Adaptive Learned Invariance for Identification Models

abstract: In unconstrained scenarios, face recognition and person re-identification are subject to distortions such as motion blur, atmospheric turbulence, or upsampling artifacts. To improve robustness in these scenarios, we propose a methodology called Distortion-Adaptive Learned Invariance for Identification (DaliID) models. We contend that distortion augmentations, which degrade image quality, can be successfully leveraged to a greater degree than has been shown in the literature. Aided by an adaptive weighting schedule, a novel distortion augmentation is applied at severe levels during training. This training strategy increases feature-level invariance to distortions and decreases domain shift to unconstrained scenarios. At inference, we use a magnitude-weighted fusion of features from parallel models to retain robustness across the range of images. DaliID models achieve state-of-the-art (SOTA) for both face recognition and person re-identification on seven benchmark datasets, including IJB-S, TinyFace, DeepChange, and MSMT17. Additionally, we provide recaptured evaluation data at a distance of 750+ meters and further validate on real long-distance face imagery.


title: Connected coalitions in graphs

abstract: The connected coalition in a graph $G=(V,E)$ consists of two disjoint sets of vertices $V_{1}$ and $V_{2}$, neither of which is a connected dominating set but whose union $V_{1}\cup V_{2}$, is a connected dominating set. A connected coalition partition in a graph $G$ of order $n=|V|$ is a vertex partition $\psi$ = $\{V_1, V_2,..., V_k \}$ such that every set $V_i \in \psi$ either is a connected dominating set consisting of a single vertex of degree $n-1$, or is not a connected dominating set but forms a connected coalition with another set $V_j\in \psi$ which is not a connected dominating set. The connected coalition number, denoted by $CC(G)$, is the maximum cardinality of a connected coalition partition of $G$. In this paper, we initiate the study of connected coalition in graphs and present some basic results. Precisely, we characterize all graphs that have a connected coalition partition. Moreover, we show that for any graph $G$ of order $n$ with $\delta(G)=1$ and with no full vertex, it holds that $CC(G)<n$. Furthermore, we show that for any tree $T$, $CC(T)=2$. Finally, we present two polynomial-time algorithms that for a given connected graph $G$ of order $n$ determine whether $CC(G)=n$ or $CC(G)=n-1$.


title: Coherence by Normalization for Linear Multicategorical Structures

abstract: We establish a formal correspondence between resource calculi an appropriate linear multicategories. We consider the cases of (symmetric) representable, symmetric closed and autonomous multicategories. For all these structures, we prove that morphisms of the corresponding free constructions can be presented by means of typed resource terms, up to a reduction relation and a structural equivalence. Thanks to the linearity of the calculi, we can prove strong normalization of the reduction by combinatorial methods, defining appropriate decreasing measures. From this, we achieve a general coherence result: morphisms that live in the free multicategorical structures are the same whenever the normal forms of the associated terms are equal. As further application, we obtain syntactic proofs of Mac Lane's coherence theorems for (symmetric) monoidal categories.


title: Improved Decoding of Attentional Selection in Multi-Talker Environments with Self-Supervised Learned Speech Representation

abstract: Auditory attention decoding (AAD) is a technique used to identify and amplify the talker that a listener is focused on in a noisy environment. This is done by comparing the listener's brainwaves to a representation of all the sound sources to find the closest match. The representation is typically the waveform or spectrogram of the sounds. The effectiveness of these representations for AAD is uncertain. In this study, we examined the use of self-supervised learned speech representation in improving the accuracy and speed of AAD. We recorded the brain activity of three subjects using invasive electrocorticography (ECoG) as they listened to two conversations and focused on one. We used WavLM to extract a latent representation of each talker and trained a spatiotemporal filter to map brain activity to intermediate representations of speech. During the evaluation, the reconstructed representation is compared to each speaker's representation to determine the target speaker. Our results indicate that speech representation from WavLM provides better decoding accuracy and speed than the speech envelope and spectrogram. Our findings demonstrate the advantages of self-supervised learned speech representation for auditory attention decoding and pave the way for developing brain-controlled hearable technologies.


title: Multispectral Contrastive Learning with Viewmaker Networks

abstract: Contrastive learning methods have been applied to a range of domains and modalities by training models to identify similar "views" of data points. However, specialized scientific modalities pose a challenge for this paradigm, as identifying good views for each scientific instrument is complex and time-intensive. In this paper, we focus on applying contrastive learning approaches to a variety of remote sensing datasets. We show that Viewmaker networks, a recently proposed method for generating views, are promising for producing views in this setting without requiring extensive domain knowledge and trial and error. We apply Viewmaker to four multispectral imaging problems, each with a different format, finding that Viewmaker can outperform cropping- and reflection-based methods for contrastive learning in every case when evaluated on downstream classification tasks. This provides additional evidence that domain-agnostic methods can empower contrastive learning to scale to real-world scientific domains. Open source code can be found at https://github.com/jbayrooti/divmaker.


title: On consecutive greedy and other greedy-like type of bases

abstract: We continue our study of the Thresholding Greedy Algorithm when we restrict the vectors involved in our approximations so that they either are supported on intervals of $\mathbb N$ or have constant coefficients. We introduce and characterize what we call consecutive greedy bases and provide new characterizations of almost greedy and squeeze symmetric Schauder bases. Moreover, we investigate some cases involving greedy-like properties with constant 1 and study the related notion of Property (A,$\tau$).


title: Improving Sign Recognition with Phonology

abstract: We use insights from research on American Sign Language (ASL) phonology to train models for isolated sign language recognition (ISLR), a step towards automatic sign language understanding. Our key insight is to explicitly recognize the role of phonology in sign production to achieve more accurate ISLR than existing work which does not consider sign language phonology. We train ISLR models that take in pose estimations of a signer producing a single sign to predict not only the sign but additionally its phonological characteristics, such as the handshape. These auxiliary predictions lead to a nearly 9% absolute gain in sign recognition accuracy on the WLASL benchmark, with consistent improvements in ISLR regardless of the underlying prediction model architecture. This work has the potential to accelerate linguistic research in the domain of signed languages and reduce communication barriers between deaf and hearing people.


title: A simple trick to improve the accuracy of PIV/PTV data

abstract: Particle Image Velocimetry (PIV) estimates velocities through correlations of particle images within interrogation windows, leading to a spatial modulation of the velocity field. Although in principle Particle Tracking Velocimetry (PTV) estimates locally a non-modulated particle displacement, to exploit the scattered data from PTV it is necessary to interpolate these data on a structured grid, which implies a spatial modulation effect that biases the resulting velocity field. This systematic error due to finite spatial resolution inevitably depends on the interrogation window size and on the interparticle spacing. It must be observed that all these operations (cross-correlation, direct interpolation or averaging in windows) induce modulation on both the mean and the fluctuating part. We introduce a simple trick to reduce this systematic error source of PIV/PTV measurements exploiting ensemble statistics. Ensemble Particle Tracking Velocimetry (EPTV) can be leveraged to obtain the high-resolution mean flow by merging the different instantaneous realisations. The mean flow can be estimated with EPTV, and the fluctuating part can be measured from PIV/PTV. The high-resolution mean can then be superposed to the instantaneous fluctuating part to obtain velocity fields with lower systematic error. The methodology is validated against datasets with a progressively increasing level of complexity: two virtual experiments based on direct numerical simulations (DNS) of the wake of a fluidic pinball and a channel flow and the experimental data of a turbulent boundary layer. For all the cases both PTV and PIV are analysed.


title: Confidence and Uncertainty Assessment for Distributional Random Forests

abstract: The Distributional Random Forest (DRF) is a recently introduced Random Forest algorithm to estimate multivariate conditional distributions. Due to its general estimation procedure, it can be employed to estimate a wide range of targets such as conditional average treatment effects, conditional quantiles, and conditional correlations. However, only results about the consistency and convergence rate of the DRF prediction are available so far. We characterize the asymptotic distribution of DRF and develop a bootstrap approximation of it. This allows us to derive inferential tools for quantifying standard errors and the construction of confidence regions that have asymptotic coverage guarantees. In simulation studies, we empirically validate the developed theory for inference of low-dimensional targets and for testing distributional differences between two populations.


title: Interpretable Deep Learning for Forecasting Online Advertising Costs: Insights from the Competitive Bidding Landscape

abstract: As advertisers increasingly shift their budgets toward digital advertising, forecasting advertising costs is essential for making budget plans to optimize marketing campaign returns. In this paper, we perform a comprehensive study using a variety of time-series forecasting methods to predict daily average cost-per-click (CPC) in the online advertising market. We show that forecasting advertising costs would benefit from multivariate models using covariates from competitors' CPC development identified through time-series clustering. We further interpret the results by analyzing feature importance and temporal attention. Finally, we show that our approach has several advantages over models that individual advertisers might build based solely on their collected data.


title: Towards Multi-User Activity Recognition through Facilitated Training Data and Deep Learning for Human-Robot Collaboration Applications

abstract: Human-robot interaction (HRI) research is progressively addressing multi-party scenarios, where a robot interacts with more than one human user at the same time. Conversely, research is still at an early stage for human-robot collaboration. The use of machine learning techniques to handle such type of collaboration requires data that are less feasible to produce than in a typical HRC setup. This work outlines scenarios of concurrent tasks for non-dyadic HRC applications. Based upon these concepts, this study also proposes an alternative way of gathering data regarding multi-user activity, by collecting data related to single users and merging them in post-processing, to reduce the effort involved in producing recordings of pair settings. To validate this statement, 3D skeleton poses of activity of single users were collected and merged in pairs. After this, such datapoints were used to separately train a long short-term memory (LSTM) network and a variational autoencoder (VAE) composed of spatio-temporal graph convolutional networks (STGCN) to recognise the joint activities of the pairs of people. The results showed that it is possible to make use of data collected in this way for pair HRC settings and get similar performances compared to using training data regarding groups of users recorded under the same settings, relieving from the technical difficulties involved in producing these data. The related code and collected data are publicly available.


title: On the fluctuations of an SDE system modelling grid cells

abstract: Several differential equation models have been proposed to explain the formation of patterns characteristic of the grid cell network. Understanding the effect of noise on these models is one of the key open questions in computational neuroscience. In the present work, we continue the analysis of an SDE system commonly proposed for this aim, which we initiated in a previous paper. We show that the fluctuations of the empirical measure associated to the particle system around its mean field limit converge to the solution of a Langevin SPDE. The interaction between different columns of neurons along the cortex prescribes a peculiar scaling regime.


title: Adversarial Online Collaborative Filtering

abstract: We investigate the problem of online collaborative filtering under no-repetition constraints, whereby users need to be served content in an online fashion and a given user cannot be recommended the same content item more than once. We start by designing and analyzing an algorithm that works under biclustering assumptions on the user-item preference matrix, and show that this algorithm exhibits an optimal regret guarantee, while being fully adaptive, in that it is oblivious to any prior knowledge about the sequence of users, the universe of items, as well as the biclustering parameters of the preference matrix. We then propose a more robust version of this algorithm which operates with general matrices. Also this algorithm is parameter free, and we prove regret guarantees that scale with the amount by which the preference matrix deviates from a biclustered structure. To our knowledge, these are the first results on online collaborative filtering that hold at this level of generality and adaptivity under no-repetition constraints. Finally, we complement our theoretical findings with simple experiments on real-world datasets aimed at both validating the theory and empirically comparing to standard baselines. This comparison shows the competitive advantage of our approach over these baselines.


title: Versatile Millikelvin Hybrid Cooling Platform for Superconductivity Research

abstract: Closed cycle $He^{3}-He^{4}$ dilution cryostats became the platform of choice in quantum sciences in the era of helium shortage. However, in many experiments, the mechanical vibrations induced by the pulsed cryocoolers present a significant drawback reflected both in electronic and mechanical noises. Here, we present a hybrid dilution cryostat platform; we have automated a commercial closed-cycle system to operate on a cryocooler or on a liquid helium battery. We implemented a scanning SQUID microscope in the hybrid dilution refrigerator. In this work we show the design of the hybrid setup and how its operation eliminates vibration artefacts in magnetic imaging.


title: Union Bounds on the Symbol Error Probability of LoRa Modulation for Flat Rician Block Fading Channels

abstract: In this paper the symbol error performance of LoRa modulation is addressed for flat Rician block fading channels. First the exact symbol error probability of the LoRa modulation on Rician fading is derived. Then the upper and lower union bounds are employed on the derived symbol error probability. The proposed bounds are compared against the exact symbol error probability, the numerical evaluation of the symbol error probability and the state-of-art approximation of the LoRa symbol error probability. Numerical results show that while the proposed upper bound is very tight to the exact symbol error probability, there is approximately a 2.5 dB gap for the lower bound.


title: Hilbert expansion based fluid models for kinetic equations describing neutral particles in the plasma edge of a fusion device

abstract: Neutral particles in the plasma edge of fusion devices based on magnetic confinement are described by a transient kinetic equation incorporating ionization, recombination, and charge-exchange collisions. In charge-exchange dominated regimes, the neutral particle velocity distribution approaches the drifting Maxwellian defined by the mean velocity and temperature of the plasma. This enables model order reduction from the kinetic equation to approximate fluid models. We derive transient fluid models consistent with the kinetic equation by exploring a splitting based approach. We split the kinetic equation in sources and sinks on the one hand, and transport combined with charge-exchange on the other hand. Combining transport with charge-exchange collisions allows for deriving Hilbert expansion based fluid models. The retrieved fluid models depend on the assumed importance (scaling) of the different terms in the split equation describing transport and charge-exchange. We explore two scalings: the hydrodynamic scaling and the diffusive scaling. The performance of the fluid models with respect to a discrete velocity model and a Monte Carlo reference solver is assessed in numerical experiments. The code used to perform the numerical experiments is openly available.


title: Gravitational wave production after inflation for a hybrid inflationary model

abstract: We discuss a cosmological scenario with a stochastic background of gravitational waves sourced by the tensor perturbation due to a hybrid inflationary model with cubic potential. The tensor-to-scalar ratio for the present hybrid inflationary model is obtained as $r \approx 0.0006$. Gravitational wave spectrum of this stochastic background, for large-scale CMB modes, $10^{-4}Mpc^{-1}$ to $1Mpc^{-1}$ is studied. The present-day energy spectrum of gravitational waves $\Omega_0^{gw}(f)$ is sensitively related to the tensor power spectrum and r which is, in turn, dependent on the unknown physics of the early cosmos. This uncertainty is characterized by two parameters: $\hat{n_t}(f)$ logarithmic average over the primordial tensor spectral index and $\hat{w}(f)$ logarithmic average over the effective equation of state parameter. Thus, exact constraints in the $\hat{w}(f)$, $\hat{n_t}(f)$ plane can be obtained by comparing theoretical constraints of our model on r and $\Omega_0^{gw}(f)$. We obtain a limit on $\hat{w}(10^{-15}Hz)$<$0.33$ around the modes probed by CMB scales.


title: Compactness of singular solutions to the sixth order GJMS equation

abstract: We study compactness properties of the set of conformally flat singular metrics with constant, positive sixth order Q-curvature on a finitely punctured sphere. Based on a recent classification of the local asymptotic behavior near isolated singularities, we introduce a notion of necksize for these metrics in our moduli space, which we use to characterize compactness. More precisely, we prove that if the punctures remain separated and the necksize at each puncture is bounded away from zero along a sequence of metrics, then a subsequence converges with respect to the Gromov--Hausdorff metric. Our proof relies on an upper bound estimate which is proved using moving planes and a blow-up argument. This is combined with a lower bound estimate which is a consequence of a removable singularity theorem. We also introduce a homological invariant which may be of independent interest for upcoming research.


title: Analyzing DCTCP and Cubic Buffer Sharing under Diverse Router Configurations

abstract: In this work, we look at the impact of router configurations on DCTCP and Cubic traffic when both algorithms share router buffers in the data center. Modern data centers host traffic with mixed congestion controls, including DCTCP and Cubic traffic. Both DCTCP and Cubic in the data center can compete with each other and potentially starve and/or be unfair to each other when sharing buffer space in the data center. This happens since both algorithms are at odds with each other in terms of buffer utilization paradigms where DCTCP attempts to limit buffer utilization while Cubic generally fills buffers to obtain high throughput. As a result, we propose methods for a measurement-driven analysis of DCTCP and Cubic performance when sharing buffers in data center routers via simulation. We run around 10000 simulation experiments with unique router configurations and network conditions. Afterwards, we present a generalizable ML model to capture the effect that different buffer settings have on DCTCP and Cubic streaming traffic in the data center. Finally, we suggest that this model can be used to tune buffer settings in the data center.


title: Set-Asides in USDA Food Procurement Auctions

abstract: We study the partial and full set-asides and their implication for changes in bidding behavior in first-price sealed-bid auctions in the context of United States Department of Agriculture (USDA) food procurement auctions. Using five years of bid data on different beef products, we implement weighted least squares regression models to show that partial set-aside predicts decreases in both offer prices and winning prices among large and small business bidders. Full set-aside predicts a small increase in offer prices and winning prices among small businesses. With these predictions, we infer that net profit of small businesses is unlikely to increase when set-asides are present.


title: Spatial magnetization profile in spherical nanomagnets with surface anisotropy: Green's function approach

abstract: We consider a single spherical nanomagnet and investigate the spatial magnetization profile $\mathbf{m}\left(\mathbf{r}\right)$ in the continuum approach, using the Green's function formalism. The energy of the (many-spin) nanomagnet comprises an isotropic exchange interaction, a uniaxial anisotropy in the core and N\'eel's surface anisotropy, and an external magnetic field. We derive a semi-analytical expression for the magnetization vector field $\mathbf{m}\left(\mathbf{r}\right)$ for an arbitrary position $\mathbf{r}$ within and on the boundary of the nanomagnet, as a solution of a homogeneous Helmholtz equation with inhomogeneous Neumann boundary conditions. ... For a more plausible comparison with experiments, e.g. using the technique of small-angle magnetic neutron scattering, we have averaged over the direction solid angle and derived the spatial profile in terms of the distance $r$. We believe that the predictions of the present study could help to characterize and understand the effects of size and surface anisotropy on the magnetization configurations in nanomagnet assemblies such as arrays of well-spaced platelets.


title: A Study of the Properties and Dynamics of the Disk of Satellites in a Milky-Way-like Galaxy System

abstract: The dynamics of the satellite systems of Milky-Way-like galaxies offer a useful means by which to study the galaxy formation process in the cosmological context. It has been suggested that the currently observed anisotropic distribution of the satellites in such galaxy systems is inconsistent with the concordance $\Lambda CDM$ cosmology model on the galactic scale if the observed satellites are random samples of the dark matter (DM) sub-halos that are nearly isotropically distributed around the central galaxy. In this study, we present original high-resolution zoom-in studies of central galaxies and satellite systems based upon initial conditions for the DM distribution from the Aquarius simulations but with substantial high-resolution baryon physics added. We find that the galaxy most like the Milky Way in this study does indeed contain a disk of satellites (DOS). Although one galaxy DOS system does not answer the question of how common such disks are, it does allow the opportunity to explore the properties and dynamics of the DOS system. Our investigation centers on the spatial arrangement (distances, angles, etc.) of satellites in this Milky-Way-like galaxy system with a specific emphasis on identifying and analyzing the disk-like structure along with its dynamical and morphological properties. Among the conclusions from this study, we find that the satellites and DM sub-halos in the galaxy simulations are anisotropically distributed. The dynamical properties of the satellites, however, indicate that the direction of the angular momentum vector of the whole satellite system is different from the normal direction of the fitted DOS and from the normal direction of the velocity dispersion of the system. Hence, the fitted DOS appears to be comprised of infalling sub-halos and is not a rotationally supported system.


title: Implementation and Performance Analysis of a Low Resolution OFDM System Prototype With Low Cost Hardware

abstract: The present work focus on the implementation and analyze of performance of a low-resolution OFDM system prototype with low-cost hardware. A software defined radio (SDR) system was chosen in this implementation due to its various advantages over a traditional radio system. Among the options of SDR devices available, the use of universal software radio peripherals (USRP) was avoided due to its high cost, despite its popularity in this field of research. Alternatively, a combination of two low-cost SDRs, "Hackrf One" and "RTL- SDR Blog V3" with the GNU Radio, a popular, free and open source radio software, were used. Thus, it was possible to emulate the behavior of a low resolution ADC in the receiver, characterize its performance and estimate its energy savings. This allowed us to determine the feasibility of building a component with the analog-to-digital conversion function with few bits of resolution. We conclude that the performance of an ADC with at least 5 bits of resolution is pretty reasonable and that this reduction in the number of bits, in comparison to 8-bit ADC, represents a fairly expressive energy saving.


title: Stochastic Surprisal: An inferential measurement of Free Energy in Neural Networks

abstract: This paper conjectures and validates a framework that allows for action during inference in supervised neural networks. Supervised neural networks are constructed with the objective to maximize their performance metric in any given task. This is done by reducing free energy and its associated surprisal during training. However, the bottom-up inference nature of supervised networks is a passive process that renders them fallible to noise. In this paper, we provide a thorough background of supervised neural networks, both generative and discriminative, and discuss their functionality from the perspective of free energy principle. We then provide a framework for introducing action during inference. We introduce a new measurement called stochastic surprisal that is a function of the network, the input, and any possible action. This action can be any one of the outputs that the neural network has learnt, thereby lending stochasticity to the measurement. Stochastic surprisal is validated on two applications: Image Quality Assessment and Recognition under noisy conditions. We show that, while noise characteristics are ignored to make robust recognition, they are analyzed to estimate image quality scores. We apply stochastic surprisal on two applications, three datasets, and as a plug-in on twelve networks. In all, it provides a statistically significant increase among all measures. We conclude by discussing the implications of the proposed stochastic surprisal in other areas of cognitive psychology including expectancy-mismatch and abductive reasoning.


title: Magic Moments: A Collaboration with John Bell

abstract: I want to give an impression of the time I spent together with John S. Bell, of the atmosphere of our collaboration and friendship. I briefly review our work, the methods of nonrelativistic approximations to quantum field theory for calculating the properties of heavy quark-antiquark bound states.


title: Candidate triple-star system with ellipsoidal components detected in Vulpecula through TESS photometry

abstract: MaGiV-1 is a candidate triple ellipsoidal star system in Vulpecula at coordinates RA(J2000) 19:52:19.13 DEC(J2000) +23:29:59.7 classified as ELL+ELL, number 2344411 in AAVSO VSX database. Through photometry from the TESS Space Telescope, two significant periods describing the orbital times of the components were identified using the Fourier transform. The analysis led to determining P(A-BC) = 4.269d the orbital period of A-BC pair, the primary component with the secondary component described by another pair, and P(BC) = 0.610d the orbital period of B-C pair, the inner ellipsoidal system. However, it cannot be completely ruled out that the shorter period can be explained by pulsations of one of the two components (e.g. by the GDOR type).


title: How to prepare your task head for finetuning

abstract: In deep learning, transferring information from a pretrained network to a downstream task by finetuning has many benefits. The choice of task head plays an important role in fine-tuning, as the pretrained and downstream tasks are usually different. Although there exist many different designs for finetuning, a full understanding of when and why these algorithms work has been elusive. We analyze how the choice of task head controls feature adaptation and hence influences the downstream performance. By decomposing the learning dynamics of adaptation, we find that the key aspect is the training accuracy and loss at the beginning of finetuning, which determines the "energy" available for the feature's adaptation. We identify a significant trend in the effect of changes in this initial energy on the resulting features after fine-tuning. Specifically, as the energy increases, the Euclidean and cosine distances between the resulting and original features increase, while their dot products (and the resulting features' norm) first increase and then decrease. Inspired by this, we give several practical principles that lead to better downstream performance. We analytically prove this trend in an overparamterized linear setting and verify its applicability to different experimental settings.


title: Predicting municipalities in financial distress: a machine learning approach enhanced by domain expertise

abstract: Financial distress of municipalities, although comparable to bankruptcy of private companies, has a far more serious impact on the well-being of communities. For this reason, it is essential to detect deficits as soon as possible. Predicting financial distress in municipalities can be a complex task, as it involves understanding a wide range of factors that can affect a municipality's financial health. In this paper, we evaluate machine learning models to predict financial distress in Italian municipalities. Accounting judiciary experts have specialized knowledge and experience in evaluating the financial performance, and they use a range of indicators to make their assessments. By incorporating these indicators in the feature extraction process, we can ensure that the model is taking into account a wide range of information that is relevant to the financial health of municipalities. The results of this study indicate that using machine learning models in combination with the knowledge of accounting judiciary experts can aid in the early detection of financial distress, leading to better outcomes for the communities.


title: Transport Signatures of Fractional Quantum Hall Binding Transitions

abstract: Certain fractional quantum Hall edges have been predicted to undergo quantum phase transitions which reduce the number of edge channels and at the same time bind electrons together. However, detailed studies of experimental signatures of such a ``binding transition'' remain lacking. Here, we propose quantum transport signatures with focus on the edge at filling $\nu=9/5$. We demonstrate theoretically that in the regime of non-equilibrated edge transport, the bound and unbound edge phases have distinct conductance and noise characteristics. We also show that for a quantum point contact in the strong back-scattering regime, the bound phase produces a minimum Fano-factor $F_{SBS}=3$ corresponding to three-electron tunneling, whereas single electron tunneling is strongly suppressed at low energies. Together with recent experimental developments, our results will be useful for detecting binding transitions in the fractional quantum Hall regime.


title: On modular Soergel bimodules, Harish-Chandra bimodules, and category O

abstract: In this paper we continue the study of the category of modular Harish-Chandra bimodules initiated by Bezrukavnikov and Riche and also study the modular version of the BGG category $\mathcal{O}$. We prove a version of the Bezrukavnikov-Mirkovic-Rumynin localization theorem for the Harish-Chandra bimodules and for the category $\mathcal{O}$. We also relate the category of Harish-Chandra bimodules to the affine Hecke category building on the prior work of Bezrukavnikov and Riche.


title: ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction

abstract: Deep neural networks (DNN) have shown great capacity of modeling a dynamical system; nevertheless, they usually do not obey physics constraints such as conservation laws. This paper proposes a new learning framework named ConCerNet to improve the trustworthiness of the DNN based dynamics modeling to endow the invariant properties. ConCerNet consists of two steps: (i) a contrastive learning method to automatically capture the system invariants (i.e. conservation properties) along the trajectory observations; (ii) a neural projection layer to guarantee that the learned dynamics models preserve the learned invariants. We theoretically prove the functional relationship between the learned latent representation and the unknown system invariant function. Experiments show that our method consistently outperforms the baseline neural networks in both coordinate error and conservation metrics by a large margin. With neural network based parameterization and no dependence on prior knowledge, our method can be extended to complex and large-scale dynamics by leveraging an autoencoder.


title: On the number of edges of cyclic subgroup graphs of finite groups

abstract: In this note, we show that among finite nilpotent groups of a given order or finite groups of a given odd order, the cyclic group of that order has the minimum number of edges in its cyclic subgroup graph. We also conjecture that this holds for arbitrary finite groups.


title: Configurational entropy and shape complexity of strange vector kaons in AdS/QCD

abstract: The mass spectrum of strange vector kaons resonances is scrutinized, using AdS/QCD with a deformed dilaton that arises from the constituent quark masses. Both the differential configurational entropy and the differential configurational complexity are computed and used to achieve the mass spectrum of strange vector kaons resonances with higher radial quantum numbers. This approach amalgamates AdS/QCD and the experimental mass spectrum of already detected strange vector kaons in the Particle Data Group, providing a hybrid technique to study the next generation of strange vector kaons resonances.


title: Calculation of Electron Swarm Parameters in Ar/H2 and Kr/H2 Mixtures

abstract: To develop the modeling of electrical discharges in a gaseous medium, it must to know a numbers of basic data such as: the electron collision cross sections for particle modeling or the electron swarm parameters in the case of the fluid modeling. The knowledge of all the electron swarm parameters as a function of the reduced electric field is indispensable for the fluid models based on the first and the second moment of the electron energy distribution function. The objective of this work is to calculate the mobility, the characteristic energy, and the ionization coefficient in the case of the mixtures Ar/H2 and Kr/H2. The results obtained will be compared with measurements already made by different authors.


title: Differentially Private Normalizing Flows for Density Estimation, Data Synthesis, and Variational Inference with Application to Electronic Health Records

abstract: Electronic health records (EHR) often contain sensitive medical information about individual patients, posing significant limitations to sharing or releasing EHR data for downstream learning and inferential tasks. We use normalizing flows (NF), a family of deep generative models, to estimate the probability density of a dataset with differential privacy (DP) guarantees, from which privacy-preserving synthetic data are generated. We apply the technique to an EHR dataset containing patients with pulmonary hypertension. We assess the learning and inferential utility of the synthetic data by comparing the accuracy in the prediction of the hypertension status and variational posterior distribution of the parameters of a physics-based model. In addition, we use a simulated dataset from a nonlinear model to compare the results from variational inference (VI) based on privacy-preserving synthetic data, and privacy-preserving VI obtained from directly privatizing NFs for VI with DP guarantees given the original non-private dataset. The results suggest that synthetic data generated through differentially private density estimation with NF can yield good utility at a reasonable privacy cost. We also show that VI obtained from differentially private NF based on the free energy bound loss may produce variational approximations with significantly altered correlation structure, and loss formulations based on alternative dissimilarity metrics between two distributions might provide improved results.


title: Fairness-aware Multi-view Clustering

abstract: In the era of big data, we are often facing the challenge of data heterogeneity and the lack of label information simultaneously. In the financial domain (e.g., fraud detection), the heterogeneous data may include not only numerical data (e.g., total debt and yearly income), but also text and images (e.g., financial statement and invoice images). At the same time, the label information (e.g., fraud transactions) may be missing for building predictive models. To address these challenges, many state-of-the-art multi-view clustering methods have been proposed and achieved outstanding performance. However, these methods typically do not take into consideration the fairness aspect and are likely to generate biased results using sensitive information such as race and gender. Therefore, in this paper, we propose a fairness-aware multi-view clustering method named FairMVC. It incorporates the group fairness constraint into the soft membership assignment for each cluster to ensure that the fraction of different groups in each cluster is approximately identical to the entire data set. Meanwhile, we adopt the idea of both contrastive learning and non-contrastive learning and propose novel regularizers to handle heterogeneous data in complex scenarios with missing data or noisy features. Experimental results on real-world data sets demonstrate the effectiveness and efficiency of the proposed framework. We also derive insights regarding the relative performance of the proposed regularizers in various scenarios.


title: Physics of collective cell migration

abstract: Movement of cell clusters along extracellular matrices (ECM) during tissue development, wound healing, and early stage of cancer invasion involve various inter-connected migration modes such as: (1) cell movement within clusters, (2) cluster extension (wetting) and compression (de-wetting), and (3) directional cluster movement. It has become increasingly evident that dilational and volumetric viscoelasticity of cell clusters and their surrounding substrate significantly influence these migration modes through physical parameters such as: cell and matrix surface tensions, interfacial tension between cells and substrate, gradients of surface and interfacial tensions, as well as, the accumulation of cell and matrix residual stresses. Inhomogeneous distribution of cell surface tension along migrating cell cluster can appear as a consequence of different strength of cell-cell adhesion contacts and cell contractility between leader and follower cells. While the directional cell migration caused by the matrix stiffness gradient (i.e. durotaxis) has been widely elaborated, the structural changes of matrix surface caused by cell tractions which lead to the generation of the matrix surface tension gradient has not been considered yet. The main goal of this theoretical consideration is to clarify the roles of various physical parameters in collective cell migration based on the formulating biophysical model. This complex phenomenon is discussed on the model systems such as the movement of cell clusters on the collagen I gel matrix by simultaneously reviewing various experimental data with and without cells.


title: Dimension Reduction and MARS

abstract: The multivariate adaptive regression spline (MARS) is one of the popular estimation methods for nonparametric multivariate regressions. However, as MARS is based on marginal splines, to incorporate interactions of covariates, products of the marginal splines must be used, which leads to an unmanageable number of basis functions when the order of interaction is high and results in low estimation efficiency. In this paper, we improve the performance of MARS by using linear combinations of the covariates which achieve sufficient dimension reduction. The special basis functions of MARS facilitate calculation of gradients of the regression function, and estimation of the linear combinations is obtained via eigen-analysis of the outer-product of the gradients. Under some technical conditions, the asymptotic theory is established for the proposed estimation method. Numerical studies including both simulation and empirical applications show its effectiveness in dimension reduction and improvement over MARS and other commonly-used nonparametric methods in regression estimation and prediction.


title: The BAR approach for multiclass queueing networks with SBP service policies

abstract: The basic adjoint relationship (BAR) approach is an analysis technique based on the stationary equation of a Markov process. This approach was introduced to study heavy-traffic, steady-state convergence of generalized Jackson networks in which each service station has a single job class. We extend it to multiclass queueing networks operating under static-buffer-priority (SBP) service disciplines. Our extension makes a connection with Palm distributions that allows one to attack a difficulty arising from queue-length truncation, which appears to be unavoidable in the multiclass setting. For multiclass queueing networks operating under SBP service disciplines, our BAR approach provides an alternative to the "interchange of limits" approach that has dominated the literature in the last twenty years. The BAR approach can produce sharp results and allows one to establish steady-state convergence under three additional conditions: stability, state space collapse (SSC) and a certain matrix being "tight." These three conditions do not appear to depend on the interarrival and service-time distributions beyond their means, and their verification can be studied as three separate modules. In particular, they can be studied in a simpler, continuous-time Markov chain setting when all distributions are exponential. As an example, these three conditions are shown to hold in reentrant lines operating under last-buffer-first-serve discipline. In a two-station, five-class reentrant line, under the heavy-traffic condition, the tight-matrix condition implies both the stability condition and the SSC condition. Whether such a relationship holds generally is an open problem.


title: An Evaluation Algorithm for Datalog with Equality

abstract: We describe an evaluation algorithm for relational Horn logic (RHL). RHL extends Datalog with quantification over sorts, existential quantification in conclusions and, crucially, the ability to infer equalities. These capabilities allow RHL evaluation to subsume and expand applications of Datalog and congruence closure algorithms. We explain how aspects of a fast congruence closure algorithm can be incorporated into Datalog evaluation to obtain an efficient RHL evaluation algorithm. We then sketch how Steensgaard's points-to analysis and type inference can be implemented using RHL evaluation. RHL and the evaluation algorithm described here are the foundation of the Eqlog Datalog engine.


title: Distributional GFlowNets with Quantile Flows

abstract: Generative Flow Networks (GFlowNets) are a new family of probabilistic samplers where an agent learns a stochastic policy for generating complex combinatorial structure through a series of decision-making steps. Despite being inspired from reinforcement learning, the current GFlowNet framework is relatively limited in its applicability and cannot handle stochasticity in the reward function. In this work, we adopt a distributional paradigm for GFlowNets, turning each flow function into a distribution, thus providing more informative learning signals during training. By parameterizing each edge flow through their quantile functions, our proposed \textit{quantile matching} GFlowNet learning algorithm is able to learn a risk-sensitive policy, an essential component for handling scenarios with risk uncertainty. Moreover, we find that the distributional approach can achieve substantial improvement on existing benchmarks compared to prior methods due to our enhanced training algorithm, even in settings with deterministic rewards.


title: Mutation-Based Adversarial Attacks on Neural Text Detectors

abstract: Neural text detectors aim to decide the characteristics that distinguish neural (machine-generated) from human texts. To challenge such detectors, adversarial attacks can alter the statistical characteristics of the generated text, making the detection task more and more difficult. Inspired by the advances of mutation analysis in software development and testing, in this paper, we propose character- and word-based mutation operators for generating adversarial samples to attack state-of-the-art natural text detectors. This falls under white-box adversarial attacks. In such attacks, attackers have access to the original text and create mutation instances based on this original text. The ultimate goal is to confuse machine learning models and classifiers and decrease their prediction accuracy.


title: Assessment HTN (A-HTN) for Automated Task Performance Assessment in 3D Serious Games

abstract: In the recent years, various 3D mixed reality serious games have been developed for different applications such as physical training, rehabilitation, and education. Task performance in a serious game is a measurement of how efficiently and accurately users accomplish the game's objectives. Prior research includes a graph-based representation of tasks, e.g. Hierarchical Task Network (HTN), which only models a game's tasks but does not perform assessment. In this paper, we propose Assessment HTN (A-HTN), which both models the task efficiently and incorporates assessment logic for game objectives. Based on how the task performance is evaluated, A-HTN automatically performs: (a) Task-level Assessment by comparing object manipulations and (b) Action-level Assessment by comparing motion trajectories. The system can also categorize the task performance assessment into single user or multi-user based on who is being assessed. We showcase the effectiveness of the A-HTN using two 3D VR serious games: a hydrometer experiment and a multi-user chemistry experiment. The A-HTN experiments show a high correlation between instructor scores and the system generated scores indicating that the proposed A-HTN generalizes automatic assessment at par with Subject Matter Experts.


title: Grey Wolf Optimizer and Whale Optimization Algorithm for Stochastic Inventory Management of Reusable Products in a two-level Supply Chain

abstract: Product reuse and recovery is an efficient tool that helps companies to simultaneously address economic and environmental dimensions of sustainability. This paper presents a novel problem for stock management of reusable products in a single-vendor, multi-product, multi-retailer network. Several constraints, such as the maximum budget, storage capacity, number of orders, etc., are considered in their stochastic form to provide a more realistic framework. The presented problem is formulated as a constrained nonlinear mathematical model. The chance-constrained programming method is suggested to deal with the constraints' uncertainty. Regarding the nonlinearity of the model, grey wolf optimizer (GWO) and whale optimization algorithm (WOA) as two novel metaheuristics are presented as solution approaches, and the sequential quadratic programming (SQP) exact algorithm validates their performance. The parameters of algorithms are calibrated using the Taguchi method for the design of experiments. Extensive analysis is established by solving several numerical results in different sizes and utilizing several comparison measures. Also, the results are compared statistically using proper parametric and non-parametric tests. The analysis of the results shows a significant difference between the algorithms, and GWO has a better performance for solving the presented problem. In addition, both algorithms perform well in searching the solution space, where the GWO and WOA differences with the optimal solution of the SQP algorithm are negligible.


title: Global Convergence Rate of Deep Equilibrium Models with General Activations

abstract: In a recent paper, Ling et al. investigated the over-parametrized Deep Equilibrium Model (DEQ) with ReLU activation. They proved that the gradient descent converges to a globally optimal solution for the quadratic loss function at a linear convergence rate. This paper shows that this fact still holds for DEQs with any generally bounded activation with bounded first and second derivatives. Since the new activation function is generally non-homogeneous, bounding the least eigenvalue of the Gram matrix of the equilibrium point is particularly challenging. To accomplish this task, we must create a novel population Gram matrix and develop a new form of dual activation with Hermite polynomial expansion.


title: Optimizing Orthogonalized Tensor Deflation via Random Tensor Theory

abstract: This paper tackles the problem of recovering a low-rank signal tensor with possibly correlated components from a random noisy tensor, or so-called spiked tensor model. When the underlying components are orthogonal, they can be recovered efficiently using tensor deflation which consists of successive rank-one approximations, while non-orthogonal components may alter the tensor deflation mechanism, thereby preventing efficient recovery. Relying on recently developed random tensor tools, this paper deals precisely with the non-orthogonal case by deriving an asymptotic analysis of a parameterized deflation procedure performed on an order-three and rank-two spiked tensor. Based on this analysis, an efficient tensor deflation algorithm is proposed by optimizing the parameter introduced in the deflation mechanism, which in turn is proven to be optimal by construction for the studied tensor model. The same ideas could be extended to more general low-rank tensor models, e.g., higher ranks and orders, leading to more efficient tensor methods with a broader impact on machine learning and beyond.


title: Rearrangements during slow compression of a jammed two-dimensional emulsion

abstract: As amorphous materials get jammed, both geometric and dynamic heterogeneity are observed. We investigate the correlation between the local geometric heterogeneity and local rearrangements in a slowly compressed bidisperse quasi-two-dimensional emulsion system. The compression is driven by evaporation of the continuous phase, and causes the area packing fraction to increase from 0.88 to 0.99. We quantify the structural heterogeneity of the system using the radical Voronoi tessellation following the method of [Rieser et al., Phys. Rev. Lett. 116, 088001 (2016)]. We define two structural quantities characterizing local structure, the first which considers nearest neighbors and the second of which includes information from second nearest neighbors. We find that droplets in heterogeneous local regions are more likely to have local rearrangements. These rearrangements are generally T1 events where two droplets converge toward a void, and two droplets move away from the void to make room for the converging droplets. Thus the presence of the voids tends to orient the T1 events. The presence of a correlation between the structural quantities and the rearrangement dynamics remains qualitatively unchanged over the entire range of packing fractions observed.


title: Continuous manipulation and characterization of colloidal beads and liposomes via diffusiophoresis in single- and double-junction microchannels

abstract: We reveal an unreported physical mechanism that enables the pre-concentration, sorting and characterization of charged polystyrene nanobeads and liposomes dispersed in a continuous flow within a straight micron-sized channel. Initially, a single $\Psi$-junction microfluidic chip is used to generate a steady-state salt concentration gradient in the direction perpendicular to the flow. As a result, fluorescent nanobeas dispersed in the electrolyte solutions accumulate into symmetric regions of the channel, appearing as two distinct symmetric stripes when the channel is observed from top via epi-fluorescence microscopy. Depending on the electrolyte flow configuration and, thus, the direction of the salt gradient field, the fluorescent stripes get closer to or apart from each other as the distance from the inlet increases. Our numerical and experimental analysis shows that, although diffusiophoresis and hydrodynamic effects are involved in the accumulation process, diffusioosmosis plays a crucial role in the observed particles dynamics. In addition, we developed a proof-of-concept double $\Psi$-junction microfluidic device which exploits this new accumulation mechanism for the size-based separation and size detection of nanobeads as well as for the measurement of zeta potential and charged lipid composition of liposomes under continuous flow settings. This device is also used to investigate the effect of fluid-like or gel-like states of the lipid membranes on the liposome diffusiophoretic response. The proposed strategies for solute-driven manipulation of colloids have great potential for microfluidic bio-analytical testing applications, including bioparticle pre-concentration, sorting, sensing and analysis.


title: BLAST: A Wafer-scale Transfer Process for Heterogeneous Integration of Optics and Electronics

abstract: We present a general transfer method for the heterogeneous integration of different photonic and electronic materials systems and devices onto a single substrate. Called BLAST, for Bond, Lift, Align, and Slide Transfer, the process works at wafer scale and offers precision alignment, high yield, varying topographies, and suitability for subsequent lithographic processing. We demonstrate BLAST's capabilities by integrating both GaAs and GaN microLEDs with silicon photovoltaics to fabricate optical wireless integrated circuits that up-convert photons from the red to the blue. We also show that BLAST can be applied to a variety of other devices and substrates, including CMOS electronics, vertical cavity surface emitting lasers (VCSELs), and 2D materials. BLAST further enables the modularization of optoelectronic microsystems, where optical devices fabricated on one material substrate can be lithographically integrated with electronic devices on a different substrate in a scalable process.


title: Universal preference for low energy core-shifted grain boundaries at the surfaces of fcc metals

abstract: Grain boundaries with [111] tilt axes are common in polycrystalline face centered cubic metals. For copper (111) films, emergent grain boundaries close to surface have tilt axes that are shifted away from [111] that are lower in energy than the corresponding truncated bulk boundaries. Geometrical analysis and atomic calculations were used to study the driving force for this same relaxation phenomenon in representative fcc elemental metals. We show that the reduction in boundary energy scales with the elimination of energetically costly boundary core facets. We find that for a wide range of misorientation angles low energy core-shifted boundaries are also favored in Al, Ni, Au and Pt and discuss the significance for electromigration and other metal properties.


title: TPE-Net: Track Point Extraction and Association Network for Rail Path Proposal Generation

abstract: One essential feature of an autonomous train is minimizing collision risks with third-party objects. To estimate the risk, the control system must identify topological information of all the rail routes ahead on which the train can possibly move, especially within merging or diverging rails. This way, the train can figure out the status of potential obstacles with respect to its route and hence, make a timely decision. Numerous studies have successfully extracted all rail tracks as a whole within forward-looking images without considering element instances. Still, some image-based methods have employed hard-coded prior knowledge of railway geometry on 3D data to associate left-right rails and generate rail route instances. However, we propose a rail path extraction pipeline in which left-right rail pixels of each rail route instance are extracted and associated through a fully convolutional encoder-decoder architecture called TPE-Net. Two different regression branches for TPE-Net are proposed to regress the locations of center points of each rail route, along with their corresponding left-right pixels. Extracted rail pixels are then spatially clustered to generate topological information of all the possible train routes (ego-paths), discarding non-ego-path ones. Experimental results on a challenging, publicly released benchmark show true-positive-pixel level average precision and recall of 0.9207 and 0.8721, respectively, at about 12 frames per second. Even though our evaluation results are not higher than the SOTA, the proposed regression pipeline performs remarkably in extracting the correspondences by looking once at the image. It generates strong rail route hypotheses without reliance on camera parameters, 3D data, and geometrical constraints.


title: A flexible Expectation-Maximization framework for fast, scalable and high-fidelity multi-frame astronomical image deconvolution

abstract: We present a computationally efficient expectation-maximization framework for multi-frame image deconvolution and super-resolution. Our method is well adapted for processing large scale imaging data from modern astronomical surveys. Our Tensorflow implementation is flexible, benefits from advanced algorithmic solutions, and allows users to seamlessly leverage Graphical Processing Unit (GPU) acceleration, thus making it viable for use in modern astronomical software pipelines. The testbed for our method is a set of $4$K by $4$K Hyper Suprime-Cam exposures, which are closest in terms of quality to imaging data from the upcoming Rubin Observatory. The preliminary results are extremely promising: our method produces a high-fidelity non-parametric reconstruction of the night sky, from which we recover unprecedented details such as the shape of the spiral arms of galaxies, while also managing to deconvolve stars perfectly into essentially single pixels.


title: Core shift controls grain boundary energy scaling in Cu and Al

abstract: Grain boundary energies in different elements are correlated. The proportional scaling constants relating the energies of crystallography-equivalent boundaries in any two f.c.c. elements are nearly constant, with the notable exception of aluminum where these constants are known to vary significantly. However, the origins of the exceptional behavior of aluminum are not understood. Previously, we reported that for fcc metals there is a preference for boundaries to shift their tilt axis across the (1-10) plane towards [112] and to ultimately form low energy [112] core shifted boundaries (CSBs). By comparing grain boundary energies in copper and aluminum with different tilt axis in (1-10) plane, we now report the existence of a well-defined scaling behavior for the case of low angle boundaries. In contrast, the scaling constant for high angle boundaries is essentially fixed regardless of their tilt axis shift. This results in a gradual change in the scaling constants from low angle to high angle boundaries, which is responsible for the apparent exceptional scaling behavior found in aluminum. An analysis of structure evolution during core shifting points to the significance of boundary-core dissociation, a form of correlated relaxation of individual atoms at boundaries, in controlling the scaling of the boundary energies.


title: Random Utility, Repeated Choice, and Consumption Dependence

abstract: We study consumption dependence in the context of random utility and repeated choice. We show that, in the presence of consumption dependence, the random utility model is a misspecified model of repeated rational choice. This misspecification leads to biased estimators and failures of standard random utility axioms. We characterize exactly when and by how much the random utility model is misspecified when utilities are consumption dependent. As one possible solution to this problem, we consider time disaggregated data. We offer a characterization of consumption dependent random utility when we observe time disaggregated data. Using this characterization, we develop a hypothesis test for consumption dependent random utility that offers computational improvements over the natural extension of Kitamura and Stoye (2018) to our setting.


title: Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play

abstract: Standard empirical risk minimization (ERM) training can produce deep neural network (DNN) models that are accurate on average but under-perform in under-represented population subgroups, especially when there are imbalanced group distributions in the long-tailed training data. Therefore, approaches that improve the accuracy-group robustness trade-off frontier of a DNN model (i.e. improving worst-group accuracy without sacrificing average accuracy, or vice versa) is of crucial importance. Uncertainty-based active learning (AL) can potentially improve the frontier by preferentially sampling underrepresented subgroups to create a more balanced training dataset. However, the quality of uncertainty estimates from modern DNNs tend to degrade in the presence of spurious correlations and dataset bias, compromising the effectiveness of AL for sampling tail groups. In this work, we propose Introspective Self-play (ISP), a simple approach to improve the uncertainty estimation of a deep neural network under dataset bias, by adding an auxiliary introspection task requiring a model to predict the bias for each data point in addition to the label. We show that ISP provably improves the bias-awareness of the model representation and the resulting uncertainty estimates. On two real-world tabular and language tasks, ISP serves as a simple "plug-in" for AL model training, consistently improving both the tail-group sampling rate and the final accuracy-fairness trade-off frontier of popular AL methods.


title: Long-term option pricing with a lower reflecting barrier

abstract: This paper considers the pricing of long-term options on assets such as housing, where either government intervention or the economic nature of the asset is assumed to limit large falls in prices. The observed asset price is modelled by a geometric Brownian motion (the 'notional price') reflected at a lower barrier. The resulting observed price has standard dynamics but with localised intervention at the barrier, which allows arbitrage with interim losses; this is funded by the government's unlimited powers of intervention, and its exploitation is subject to credit constraints. Despite the lack of an equivalent martingale measure for the observed price, options on this price can be expressed as compound options on the arbitrage-free notional price, to which standard risk-neutral arguments can be applied. Because option deltas tend to zero when the observed price approaches the barrier, hedging with the observed price gives the same results as hedging with the notional price, and so exactly replicates option payoffs. Hedging schemes are not unique, with the cheapest scheme for any derivative being the one which best exploits the interventions at the barrier. The price of a put is clear: direct replication has a lower initial cost than synthetic replication, and the replication portfolio always has positive value. The price of a call is ambiguous: synthetic replication has a lower initial cost than direct replication, but the replication portfolio may give interim losses, and so the preferred replication strategy (and hence price) of a call may depend on what margin payments need to be made on these losses.


title: Airfoil trailing-edge noise source identification using large-eddy simulation and wavelet transform

abstract: Airfoil noise is predicted and analyzed using wall-resolved large-eddy simulations and wavelet transforms for a NACA 0012 airfoil at a Mach number of 0.06 and a Reynolds number of 400,000 using a stair-strip forced transition and a natural transition. At a high angle of attack, vortex shedding and a laminar separation bubble (LSB) occur on the suction side. The LSB triggers the flow transition for both the forced and natural transition cases. The wavelet thresholding and denoising algorithm is used to decompose the pressure fields into the coherent or denoised pressure and the incoherent or background noise pressure. This denoising technique provides a clear picture of true noise generation and propagation. It also reveals the dominant noise source at specific frequencies when multiple noise sources are present. In another usage, the wavelet thresholding algorithm with down-sampling separates noise on the basis of flow structures. For example, the wavelet method separates noise between low-frequency vortex shedding noise and high-frequency LSB noise as well as trailing-edge noise. Finally, the wavelet transform is used to decompose the hydrodynamic and acoustic pressure components near the surface using the coherence between near-field pressure and far-field pressure. Overall, the wavelet-based decomposition is a valuable tool to study and reveal the mechanisms of airfoil noise generation.


title: Low Sample Complexity Participatory Budgeting

abstract: We study low sample complexity mechanisms in participatory budgeting (PB), where each voter votes for a preferred allocation of funds to various projects, subject to project costs and total spending constraints. We analyze the distortion that PB mechanisms introduce relative to the minimum-social-cost outcome in expectation. The Random Dictator mechanism for this problem obtains a distortion of 2. In a special case where every voter votes for exactly one project, [Fain et al '17] obtain a distortion of 4/3 We show that when PB outcomes are determined as any convex combination of the votes of two voters, the distortion is 2. When three uniformly randomly sampled votes are used, we give a PB mechanism that obtains a distortion of at most 1.66, thus breaking the barrier of 2 with the smallest possible sample complexity. We give a randomized Nash bargaining scheme where two uniformly randomly chosen voters bargain with the disagreement point as the vote of a voter chosen uniformly at random. This mechanism has a distortion of at most 1.66. We provide a lower bound of 1.38 for the distortion of this scheme. Further, we show that PB mechanisms that output a median of the votes of three voters chosen uniformly at random have a distortion of at most 1.80.


title: Hierarchical control and learning of a foraging CyberOctopus

abstract: Inspired by the unique neurophysiology of the octopus, we propose a hierarchical framework that simplifies the coordination of multiple soft arms by decomposing control into high-level decision making, low-level motor activation, and local reflexive behaviors via sensory feedback. When evaluated in the illustrative problem of a model octopus foraging for food, this hierarchical decomposition results in significant improvements relative to end-to-end methods. Performance is achieved through a mixed-modes approach, whereby qualitatively different tasks are addressed via complementary control schemes. Here, model-free reinforcement learning is employed for high-level decision-making, while model-based energy shaping takes care of arm-level motor execution. To render the pairing computationally tenable, a novel neural-network energy shaping (NN-ES) controller is developed, achieving accurate motions with time-to-solutions 200 times faster than previous attempts. Our hierarchical framework is then successfully deployed in increasingly challenging foraging scenarios, including an arena littered with obstacles in 3D space, demonstrating the viability of our approach.


title: Software-Defined MIMO OFDM Joint Radar-Communication Platform with Fully Digital mmWave Architecture

abstract: Large-scale deployment of connected vehicles with cooperative sensing and maneuvering technologies increases the demand for vehicle-to-everything communication (V2X) band in 5.9 GHz. Besides the V2X spectrum, the under-utilized millimeter-wave (mmWave) bands at 24 and 77 GHz can be leveraged to supplement V2X communication and support high data rates for emerging broadband applications. For this purpose, joint radar-communication (JRC) systems have been proposed in the literature to perform both functions using the same waveform and hardware. In this work, we present a software-defined multiple-input and multiple-output (MIMO) JRC with orthogonal frequency division multiplexing (OFDM) for the 24 GHz mmWave band. We implement a real-time operating full-duplex JRC platform using commercially available software-defined radios and custom-built mmWave front-ends. With fully digital MIMO architecture, we demonstrate simultaneous data transmission and high-resolution radar imaging capabilities of MIMO OFDM JRC in the mmWave band.


title: Lazard-style CAD and Equational Constraints

abstract: McCallum-style Cylindrical Algebra Decomposition (CAD) is a major improvement on the original Collins version, and has had many subsequent advances, notably for total or partial equational constraints. But it suffers from a problem with nullification. The recently-justified Lazard-style CAD does not have this problem. However, transporting the equational constraints work to Lazard-style does reintroduce nullification issues. This paper explains the problem, and the solutions to it, based on the second author's Ph.D. thesis and the Brown--McCallum improvement to Lazard. With a single equational constraint, we can gain the same improvements in Lazard-style as in McCallum-style CAD . Moreover, our approach does not fail where McCallum would due to nullification. Unsurprisingly, it does not achieve the same level of improvement as it does in the non-nullified cases. We also consider the case of multiple equational constraints.


title: Quantum emitter formation dynamics and probing of radiation induced atomic disorder in silicon

abstract: Near infrared color centers in silicon are emerging candidates for on-chip integrated quantum emitters, optical access quantum memories and sensing. We access ensemble G color center formation dynamics and radiation-induced atomic disorder in silicon for a series of MeV proton flux conditions. Photoluminescence results reveal that the G-centers are formed more efficiently by pulsed proton irradiation than continuous wave proton irradiation. The enhanced transient excitations and dynamic annealing within nanoseconds allows optimizing the ratio of G-center formation to nonradiative defect accumulation. The G-centers preserve narrow linewidths of about 0.1 nm when they are generated by moderate pulsed proton fluences, while the linewidth broadens significantly as the pulsed proton fluence increases. This implies vacancy/interstitial clustering by overlapping collision cascades. Tracking G-center properties for a series of irradiation conditions enables sensitive probing of atomic disorder, serving as a complimentary analytical method for sensing damage accumulation. Aided by ${\it ab}$ ${\it initio}$ electronic structure calculations, we provide insight into the atomic disorder-induced inhomogeneous broadening by introducing vacancies and silicon interstitials in the vicinity of a G-center. A vacancy leads to a tensile strain and can result in either a redshift or blueshift of the G-center emission, depending on its position relative to the G-center. Meanwhile, Si interstitials lead to compressive strain, which results in a monotonic redshift. High flux and tunable ion pulses enable the exploration of fundamental dynamics of radiation-induced defects as well as methods for defect engineering and qubit synthesis for quantum information processing.


title: E-unification for Second-Order Abstract Syntax

abstract: Higher-order unification (HOU) concerns unification of (extensions of) $\lambda$-calculus and can be seen as an instance of equational unification ($E$-unification) modulo $\beta\eta$-equivalence of $\lambda$-terms. We study equational unification of terms in languages with arbitrary variable binding constructions modulo arbitrary second-order equational theories. Abstract syntax with general variable binding and parametrised metavariables allows us to work with arbitrary binders without committing to $\lambda$-calculus or use inconvenient and error-prone term encodings, leading to a more flexible framework. In this paper, we introduce $E$-unification for second-order abstract syntax and describe a unification procedure for such problems, merging ideas from both full HOU and general $E$-unification. We prove that the procedure is sound and complete.


title: A Policy Gradient Framework for Stochastic Optimal Control Problems with Global Convergence Guarantee

abstract: We consider policy gradient methods for stochastic optimal control problem in continuous time. In particular, we analyze the gradient flow for the control, viewed as a continuous time limit of the policy gradient method. We prove the global convergence of the gradient flow and establish a convergence rate under some regularity assumptions. The main novelty in the analysis is the notion of local optimal control function, which is introduced to characterize the local optimality of the iterate.


title: Level Generation Through Large Language Models

abstract: Large Language Models (LLMs) are powerful tools, capable of leveraging their training on natural language to write stories, generate code, and answer questions. But can they generate functional video game levels? Game levels, with their complex functional constraints and spatial relationships in more than one dimension, are very different from the kinds of data an LLM typically sees during training. Datasets of game levels are also hard to come by, potentially taxing the abilities of these data-hungry models. We investigate the use of LLMs to generate levels for the game Sokoban, finding that LLMs are indeed capable of doing so, and that their performance scales dramatically with dataset size. We also perform preliminary experiments on controlling LLM level generators and discuss promising areas for future work.


title: Synaptic Stripping: How Pruning Can Bring Dead Neurons Back To Life

abstract: Rectified Linear Units (ReLU) are the default choice for activation functions in deep neural networks. While they demonstrate excellent empirical performance, ReLU activations can fall victim to the dead neuron problem. In these cases, the weights feeding into a neuron end up being pushed into a state where the neuron outputs zero for all inputs. Consequently, the gradient is also zero for all inputs, which means that the weights which feed into the neuron cannot update. The neuron is not able to recover from direct back propagation and model capacity is reduced as those parameters can no longer be further optimized. Inspired by a neurological process of the same name, we introduce Synaptic Stripping as a means to combat this dead neuron problem. By automatically removing problematic connections during training, we can regenerate dead neurons and significantly improve model capacity and parametric utilization. Synaptic Stripping is easy to implement and results in sparse networks that are more efficient than the dense networks they are derived from. We conduct several ablation studies to investigate these dynamics as a function of network width and depth and we conduct an exploration of Synaptic Stripping with Vision Transformers on a variety of benchmark datasets.


title: New Clebsch-Gordan-type integrals involving threefold products of complete elliptic integrals

abstract: Multiple elliptic integrals related to the generalized Clebsch-Gordan (CG) integral are of importance in many areas in physics and special functions theory. Zhou has introduced and applied Legendre function-based techniques to prove symbolic evaluations for integrals of CG form involving twofold and threefold products of complete elliptic integral expressions, and this includes Zhou's remarkable proof of an open problem due to Wan. The foregoing considerations motivate the results introduced in this article, in which we prove closed-form evaluations for new CG-type integrals that involve threefold products of the complete elliptic integrals $K$ and $E$. Our methods are based on the use of fractional derivative operators, via a variant of a technique we had previously referred to as semi-integration by parts.


title: Improved noise performance from the next-generation buried-channel p-Mosfet SiSeROs

abstract: The Single electron Sensitive Read Out (SiSeRO) is a novel on-chip charge detector output stage for charge-coupled device (CCD) image sensors. Developed at MIT Lincoln Laboratory, this technology uses a p-MOSFET transistor with a depleted internal gate beneath the transistor channel. The transistor source-drain current is modulated by the transfer of charge into the internal gate. At Stanford, we have developed a readout module based on the drain current of the on-chip transistor to characterize the device. In our earlier work, we characterized a number of first prototype SiSeROs with the MOSFET transistor channels at the surface layer. An equivalent noise charge (ENC) of around 15 electrons root mean square (RMS) was obtained. In this work, we examine the first buried-channel SiSeRO. We have achieved substantially improved noise performance of around 4.5 electrons root mean square (RMS) and a full width half maximum (FWHM) energy resolution of 132 eV at 5.9 keV, for a readout speed of 625 kpixel/s. We also discuss how digital filtering techniques can be used to further improve the SiSeRO noise performance. Additional measurements and device simulations will be essential to further mature the SiSeRO technology. This new device class presents an exciting new technology for the next-generation astronomical X-ray telescopes requiring fast, low-noise, radiation-hard megapixel imagers with moderate spectroscopic resolution.


title: On discontinuous differential equations and the method of solution-regions

abstract: We adapt the method of solution regions to prove new existence and localization results for systems of discontinuous differential equations. Some assumptions concerning the definition of a solution region are relaxed and thus our results enlarge the applicability of this method even in the case of Carath\'eodory nonlinearities. Several examples are provided to illustrate the theory.


title: Interpretable Diversity Analysis: Visualizing Feature Representations In Low-Cost Ensembles

abstract: Diversity is an important consideration in the construction of robust neural network ensembles. A collection of well trained models will generalize better if they are diverse in the patterns they respond to and the predictions they make. Diversity is especially important for low-cost ensemble methods because members often share network structure in order to avoid training several independent models from scratch. Diversity is traditionally analyzed by measuring differences between the outputs of models. However, this gives little insight into how knowledge representations differ between ensemble members. This paper introduces several interpretability methods that can be used to qualitatively analyze diversity. We demonstrate these techniques by comparing the diversity of feature representations between child networks using two low-cost ensemble algorithms, Snapshot Ensembles and Prune and Tune Ensembles. We use the same pre-trained parent network as a starting point for both methods which allows us to explore how feature representations evolve over time. This approach to diversity analysis can lead to valuable insights and new perspectives for how we measure and promote diversity in ensemble methods.