title: Robust feedback stabilization of interacting multi-agent systems under uncertainty

abstract: We consider control strategies for large-scale interacting agent systems under uncertainty. The particular focus is on the design of robust controls that allow to bound the variance of the controlled system over time. To this end we consider $\mathcal{H}_\infty$ control strategies on the agent and mean field description of the system. We show a bound on the $\mathcal{H}_\infty$ norm for a stabilizing controller independent on the number of agents. Furthermore, we compare the new control with existing approaches to treat uncertainty by generalized polynomial chaos expansion. Numerical results are presented for one-dimensional and two-dimensional agent systems.


title: Spin-orbit enhancement in Si/SiGe heterostructures with oscillating Ge concentration

abstract: We show that Ge concentration oscillations within the quantum well region of a Si/SiGe heterostructure can significantly enhance the spin-orbit coupling of the low-energy conduction-band valleys. Specifically, we find that for Ge oscillation wavelengths near $\lambda = 1.57~\text{nm}$ with an average Ge concentration of $\bar{n}_{\text{Ge}} = 5\%$ in the quantum well region, a Dresselhaus spin-orbit coupling is induced, at all physically relevant electric field strengths, which is over an order of magnitude larger than what is found in conventional Si/SiGe heterostructures without Ge concentration oscillations. This enhancement is caused by the Ge concentration oscillations producing wave-function satellite peaks a distance $2 \pi/\lambda$ away in momentum space from each valley, which then couple to the opposite valley through Dresselhaus spin-orbit coupling. Our results indicate that the enhanced spin-orbit coupling can enable fast spin manipulation within Si quantum dots using electric dipole spin resonance in the absence of micromagnets. Indeed, our calculations yield a Rabi frequency $\Omega_{\text{Rabi}}/B > 500~\text{MHz/T}$ near the optimal Ge oscillation wavelength $\lambda = 1.57~\text{nm}$.


title: Knowledge Distillation based Contextual Relevance Matching for E-commerce Product Search

abstract: Online relevance matching is an essential task of e-commerce product search to boost the utility of search engines and ensure a smooth user experience. Previous work adopts either classical relevance matching models or Transformer-style models to address it. However, they ignore the inherent bipartite graph structures that are ubiquitous in e-commerce product search logs and are too inefficient to deploy online. In this paper, we design an efficient knowledge distillation framework for e-commerce relevance matching to integrate the respective advantages of Transformer-style models and classical relevance matching models. Especially for the core student model of the framework, we propose a novel method using $k$-order relevance modeling. The experimental results on large-scale real-world data (the size is 6$\sim$174 million) show that the proposed method significantly improves the prediction accuracy in terms of human relevance judgment. We deploy our method to the anonymous online search platform. The A/B testing results show that our method significantly improves 5.7% of UV-value under price sort mode.


title: Enhanced skyrmion metastability under applied strain in FeGe

abstract: Mechanical straining of skyrmion hosting materials has previously demonstrated increased phase stability through the expansion of the skyrmion equilibrium pocket. Additionally, metastable skyrmions can be generated via rapid field-cooling to form significant skyrmion populations at low temperatures. Using small-angle x-ray scattering and x-ray holographic imaging on a thermally strained 200 nm thick FeGe lamella, we observe temperature-induced strain effects on the structure and metastability of the skyrmion lattice. We find that in this sample orientation (H || [1 1 0]) with no strain, metastable skyrmions produced by field cooling through the equilibrium skyrmion pocket vanish from the sample upon dropping below the well known helical reorientation temperature. However, when strain is applied along [110] axis, and this procedure is repeated, a substantial volume fraction of metastable skyrmions persist upon cooling below this temperature down to 100 K. Additionally, we observe a large number of skyrmions retained after a complete magnetic field polarity reversal, implying that the metastable energy barrier protecting skyrmions from decay is enhanced.


title: Improving Label-Deficient Keyword Spotting Through Self-Supervised Pretraining

abstract: Keyword Spotting (KWS) models are becoming increasingly integrated into various systems, e.g. voice assistants. To achieve satisfactory performance, these models typically rely on a large amount of labelled data, limiting their applications only to situations where such data is available. Self-supervised Learning (SSL) methods can mitigate such a reliance by leveraging readily-available unlabelled data. Most SSL methods for speech have primarily been studied for large models, whereas this is not ideal, as compact KWS models are generally required. This paper explores the effectiveness of SSL on small models for KWS and establishes that SSL can enhance the performance of small KWS models when labelled data is scarce. We pretrain three compact transformer-based KWS models using Data2Vec, and fine-tune them on a label-deficient setup of the Google Speech Commands data set. It is found that Data2Vec pretraining leads to a significant increase in accuracy, with label-deficient scenarios showing an improvement of 8.22% 11.18% absolute accuracy.


title: $L_p$-Sampling recovery for non-compact subclasses of $L_\infty$

abstract: In this paper we study the sampling recovery problem for certain relevant multivariate function classes which are not compactly embedded into $L_\infty$. Recent tools relating the sampling numbers to the Kolmogorov widths in the uniform norm are therefore not applicable. In a sense, we continue the research on the small smoothness problem by considering "very" small smoothness in the context of Besov and Triebel-Lizorkin spaces with dominating mixed regularity. There is not much known on the recovery of such functions except of an old result by Oswald in the univariate situation. As a first step we prove the uniform boundedness of the $\ell_p$-norm of the Faber-Schauder coefficients in a fixed level. Using this we are able to control the error made by a (Smolyak) truncated Faber-Schauder series in $L_q$ with $q<\infty$. It turns out that the main rate of convergence is sharp. As a consequence we obtain results also for $S^1_{1,\infty}F([0,1]^d)$, a space which is ``close'' to the space $S^1_1W([0,1]^d)$ which is important in numerical analysis, especially numerical integration, but has rather bad Fourier analytic properties.


title: Tensor-reduced atomic density representations

abstract: Density based representations of atomic environments that are invariant under Euclidean symmetries have become a widely used tool in the machine learning of interatomic potentials, broader data-driven atomistic modelling and the visualisation and analysis of materials datasets.The standard mechanism used to incorporate chemical element information is to create separate densities for each element and form tensor products between them. This leads to a steep scaling in the size of the representation as the number of elements increases. Graph neural networks, which do not explicitly use density representations, escape this scaling by mapping the chemical element information into a fixed dimensional space in a learnable way. We recast this approach as tensor factorisation by exploiting the tensor structure of standard neighbour density based descriptors. In doing so, we form compact tensor-reduced representations whose size does not depend on the number of chemical elements, but remain systematically convergeable and are therefore applicable to a wide range of data analysis and regression tasks.


title: A Fuzzy Logic-based Cascade Control without Actuator Saturation for the Unmanned Underwater Vehicle Trajectory Tracking

abstract: An intelligent control strategy is proposed to eliminate the actuator saturation problem that exists in the trajectory tracking process of unmanned underwater vehicles (UUV). The control strategy consists of two parts: for the kinematic modeling part, a fuzzy logic-refined backstepping control is developed to achieve control velocities within acceptable ranges and errors of small fluctuations; on the basis of the velocities deducted by the improved kinematic control, the sliding mode control (SMC) is introduced in the dynamic modeling to obtain corresponding torques and forces that should be applied to the vehicle body. With the control velocities computed by the kinematic model and applied forces derived by the dynamic model, the robustness and accuracy of the UUV trajectory without actuator saturation can be achieved.


title: Multiple Instance Learning for Detecting Anomalies over Sequential Real-World Datasets

abstract: Detecting anomalies over real-world datasets remains a challenging task. Data annotation is an intensive human labor problem, particularly in sequential datasets, where the start and end time of anomalies are not known. As a result, data collected from sequential real-world processes can be largely unlabeled or contain inaccurate labels. These characteristics challenge the application of anomaly detection techniques based on supervised learning. In contrast, Multiple Instance Learning (MIL) has been shown effective on problems with incomplete knowledge of labels in the training dataset, mainly due to the notion of bags. While largely under-leveraged for anomaly detection, MIL provides an appealing formulation for anomaly detection over real-world datasets, and it is the primary contribution of this paper. In this paper, we propose an MIL-based formulation and various algorithmic instantiations of this framework based on different design decisions for key components of the framework. We evaluate the resulting algorithms over four datasets that capture different physical processes along different modalities. The experimental evaluation draws out several observations. The MIL-based formulation performs no worse than single instance learning on easy to moderate datasets and outperforms single-instance learning on more challenging datasets. Altogether, the results show that the framework generalizes well over diverse datasets resulting from different real-world application domains.


title: Conquering the Communication Constraints to Enable Large Pre-Trained Models in Federated Learning

abstract: Federated learning (FL) has emerged as a promising paradigm for enabling the collaborative training of models without centralized access to the raw data on local devices. In the typical FL paradigm (e.g., FedAvg), model weights are sent to and from the server each round to participating clients. Recently, the use of small pre-trained models has been shown effective in federated learning optimization and improving convergence. However, recent state-of-the-art pre-trained models are getting more capable but also have more parameters. In conventional FL, sharing the enormous model weights can quickly put a massive communication burden on the system, especially if more capable models are employed. Can we find a solution to enable those strong and readily-available pre-trained models in FL to achieve excellent performance while simultaneously reducing the communication burden? To this end, we investigate the use of parameter-efficient fine-tuning in federated learning and thus introduce a new framework: FedPEFT. Specifically, we systemically evaluate the performance of FedPEFT across a variety of client stability, data distribution, and differential privacy settings. By only locally tuning and globally sharing a small portion of the model weights, significant reductions in the total communication overhead can be achieved while maintaining competitive or even better performance in a wide range of federated learning scenarios, providing insight into a new paradigm for practical and effective federated systems.


title: Algebraic logic for the negation fragment of classical logic

abstract: The general aim of this article is to study the negation fragment of classical logic within the framework of contemporary (Abstract) Algebraic Logic. More precisely, we shall find the three classes of algebras that are canonically associated with a logic in Algebraic Logic, that is, we find the classes $\mathrm{Alg}^*$, $\mathrm{Alg}$ and the intrinsic variety of the negation fragment of classical logic. In order to achieve this, firstly we propose a Hilbert-style axiomatization for this fragment. Then, we characterize the reduced matrix models and the full generalized matrix models of this logic. Also, we classify the negation fragment in the Leibniz and Frege hierarchies.


title: Deterministic and stochastic cooperation transitions in evolutionary games on networks

abstract: Although the cooperative dynamics emerging from a network of interacting players has been exhaustively investigated, it is not yet fully understood when and how network reciprocity drives cooperation transitions. In this work, we investigate the critical behavior of evolutionary social dilemmas on structured populations by using the framework of master equations and Monte Carlo simulations. The developed theory describes the existence of absorbing, quasi-absorbing, and mixed strategy states and the transition nature, continuous or discontinuous, between the states as the parameters of the system change. In particular, when the decision-making process is deterministic, in the limit of zero effective temperature of the Fermi function, we find that the copying probabilities are discontinuous functions of the system's parameters and of the network degrees sequence. This may induce abrupt changes in the final state for any system size, in excellent agreement with the Monte Carlo simulation results. Our analysis also reveals the existence of continuous and discontinuous phase transitions for large systems as the temperature increases, which is explained in the mean-field approximation. Interestingly, for some game parameters, we find optimal "social temperatures" maximizing/minimizing the cooperation frequency/density.


title: The Kuramoto-Sivashinsky Equation

abstract: The Kuramoto-Sivashinsky equation was introduced as a simple 1-dimensional model of instabilities in flames, but it turned out to mathematically fascinating in its own right. One reason is that this equation is a simple model of Galilean-invariant chaos with an arrow of time. Starting from random initial conditions, manifestly time-asymmetric stripe-like patterns emerge. As we move forward in time, it appears that these stripes are born and merge, but do not die or split. We pose a precise conjecture to this effect, which requires a precise definition of 'stripes'.


title: Autoionizing polaritons with the Jaynes-Cummings model

abstract: Intense laser pulses have the capability to couple resonances in the continuum, leading to the formation of a split pair of autoionizing polaritons. These polaritons can exhibit extended lifetimes due to interference between radiative and Auger decay channels. In this work, we introduce an extension of the Jaynes-Cummings model to autoionizing states. This model quantitatively reproduces the observed phenomenology and it allows us to study how the dressing laser parameters can be tuned to control the ionization rate of the polariton multiplet. We illustrate the potential of this model by estimating the splitting and widths of the polaritonic features in the attosecond transient-absorption spectrum of the argon atom in the energy region close to the $3s^{-1}n\ell$ autoionizing states. The model's predictions are compared with \emph{ab initio} simulations of the spectrum under realistic experimental conditions.


title: Anatomically constrained CT image translation for heterogeneous blood vessel segmentation

abstract: Anatomical structures such as blood vessels in contrast-enhanced CT (ceCT) images can be challenging to segment due to the variability in contrast medium diffusion. The combined use of ceCT and contrast-free (CT) CT images can improve the segmentation performances, but at the cost of a double radiation exposure. To limit the radiation dose, generative models could be used to synthesize one modality, instead of acquiring it. The CycleGAN approach has recently attracted particular attention because it alleviates the need for paired data that are difficult to obtain. Despite the great performances demonstrated in the literature, limitations still remain when dealing with 3D volumes generated slice by slice from unpaired datasets with different fields of view. We present an extension of CycleGAN to generate high fidelity images, with good structural consistency, in this context. We leverage anatomical constraints and automatic region of interest selection by adapting the Self-Supervised Body Regressor. These constraints enforce anatomical consistency and allow feeding anatomically-paired input images to the algorithm. Results show qualitative and quantitative improvements, compared to stateof-the-art methods, on the translation task between ceCT and CT images (and vice versa).


title: Probing flux and charge noise with macroscopic resonant tunneling

abstract: We report on measurements of flux and charge noise in an rf-SQUID flux qubit using macroscopic resonant tunneling (MRT). We measure rates of incoherent tunneling from the lowest energy state in the initial well to the ground and first excited states in the target well. The result of the measurement consists of two peaks. The first peak corresponds to tunneling to the ground state of the target well, and is dominated by flux noise. The second peak is due to tunneling to the excited state and is wider due to an intrawell relaxation process dominated by charge noise. We develop a theoretical model that allows us to extract information about flux and charge noise within one experimental setup. The model agrees very well with experimental data over a wide dynamic range and provides parameters that characterize charge and flux noise.


title: Modelling of the scandium abundance evolution in AmFm stars

abstract: Scandium is a key element of the Am star phenomenon since its surface under-abundance is one of the criteria that characterise such stars. Thanks to the availability of a sufficiently complete set of theoretical atomic data for this element, reliable radiative accelerations for Sc can now be computed, which allows its behaviour under the action of atomic diffusion to be modelled. We explore the required conditions, in terms of mixing processes or mass loss, for our models to reproduce the observed surface abundances of Sc in Am stars. The models are computed with the Toulouse-Geneva evolution code, which uses the parametric single-valued parameter method for the calculation of radiative accelerations. Fingering mixing is included, using a prescription that comes from 3D hydrodynamical simulations. Other parameter-dependent turbulent mixing processes are also considered. A global mass loss is also implemented. When no mass loss is considered, the observed abundances of Sc are rather in favour of the models whose superficial layers are fully mixed down to the iron accumulation zone, although other mixing prescriptions are also able to reproduce the observations for the most massive model presented here ($2.0 M_\odot$). The models including mass loss with rates in the range of $[10^{-13};10^{-14}] M_\odot$/yr are compatible with some of the observations, while other observations suggest that the mass-loss rate could be lower. The constraints brought by the modelling of Sc are consistent with those derived using other chemical elements.


title: Beam deflection and negative drag in a moving nonlinear medium

abstract: Light propagating in a moving medium with refractive index other than unity is subject to light drag. While the light drag effect due to the linear refractive index is often negligibly small, it can be enhanced in materials with a large group index. Here we show that the nonlinear refractive index can also play a crucial role in propagation of light in moving media and results in a beam deflection that might be confused with the transverse drag effect. We perform an experiment with a rotating ruby crystal which exhibits a very large negative group index and a positive nonlinear refractive index. The negative group index drags the light opposite to the motion of the medium. However, the positive nonlinear refractive index deflects the beam towards the motion of the medium and hinders the observation of the negative drag effect. Hence, we show that it is necessary to measure not only the transverse shift of the beam, but also its output angle to discriminate the light-drag effect from beam deflection -- a crucial step missing in earlier experiments.


title: High-field Studies on Layered Magnetic and Polar Dirac Metals

abstract: Recently, the interplay between the Dirac/Weyl fermion and various bulk properties, such as magnetism, has attracted considerable attention, since unconventional transport and optical phenomena were discovered. However, the design principles for such materials have not been established well. Here, we propose that the layered material $A$Mn$X_2$ ($A$: alkaline and rare-earth ions, $X$: Sb, Bi) is a promising platform for systematically exploring strongly correlated Dirac metals, which consists of the alternative stack of the $X^-$ square net layer hosting a 2D Dirac fermion and the $A^{2+}$-Mn$^{2+}$-$X^{3-}$ magnetic block layer. In this article, we shall review recent high-field studies on this series of materials to demonstrate that various types of Dirac fermions are realized by designing the block layer. First, we give an overview of the Dirac fermion coupled with the magnetic order in EuMnBi$_2$ ($A$=Eu). This material exhibits large magnetoresistance by the field-induced change in the magnetic order of Eu layers, which is associated with the strong exchange interaction between the Dirac fermion and the local Eu moment. Second, we review the Dirac fermion coupled with the lattice polarization in BaMn$X_2$ ($A$=Ba). There, spin-valley coupling manifests itself owing to the Zeeman-type spin-orbit interaction, which is experimentally evidenced by the bulk quantum Hall effect observed at high fields.


title: Detecting and Denoising Gravitational Wave Signals from Binary Black Holes using Deep Learning

abstract: We present a convolutional neural network, designed in the auto-encoder configuration that can detect and denoise astrophysical gravitational waves from merging black hole binaries, orders of magnitude faster than the conventional matched-filtering based detection that is currently employed at advanced LIGO (aLIGO). The Neural-Net architecture is such that it learns from the sparse representation of data in the time-frequency domain and constructs a non-linear mapping function that maps this representation into two separate masks for signal and noise, facilitating the separation of the two, from raw data. This approach is the first of its kind to apply machine learning based gravitational wave detection/denoising in the 2D representation of gravitational wave data. We applied our formalism to the first gravitational wave event detected, GW150914, successfully recovering the signal at all three phases of coalescence at both detectors. This method is further tested on the gravitational wave data from the second observing run ($O2$) of aLIGO, reproducing all binary black hole mergers detected in $O2$ at both the aLIGO detectors. The Neural-Net seems to have uncovered a pattern of 'ringing' after the ringdown phase of the coalescence, which is not a feature that is present in the conventional binary merger templates. This method can also interpolate and extrapolate between modeled templates and explore gravitational waves that are unmodeled and hence not present in the template bank of signals used in the matched-filtering detection pipelines. Faster and efficient detection schemes, such as this method, will be instrumental as ground based detectors reach their design sensitivity, likely to result in several hundreds of potential detections in a few months of observing runs.


title: Learning Temporal Resolution in Spectrogram for Audio Classification

abstract: The audio spectrogram is a time-frequency representation that has been widely used for audio classification. One of the key attributes of the audio spectrogram is the temporal resolution, which depends on the hop size used in the Short-Time Fourier Transform (STFT). Previous works generally assume the hop size should be a constant value (e.g., 10 ms). However, a fixed temporal resolution is not always optimal for different types of sound. The temporal resolution affects not only classification accuracy but also computational cost. This paper proposes a novel method, DiffRes, that enables differentiable temporal resolution modeling for audio classification. Given a spectrogram calculated with a fixed hop size, DiffRes merges non-essential time frames while preserving important frames. DiffRes acts as a "drop-in" module between an audio spectrogram and a classifier and can be jointly optimized with the classification task. We evaluate DiffRes on five audio classification tasks, using mel-spectrograms as the acoustic features, followed by off-the-shelf classifier backbones. Compared with previous methods using the fixed temporal resolution, the DiffRes-based method can achieve the equivalent or better classification accuracy with at least 25% computational cost reduction. We further show that DiffRes can improve classification accuracy by increasing the temporal resolution of input acoustic features, without adding to the computational cost.


title: A categorical proof of the Carath\'eodory extension theorem

abstract: The Carath\'eodory extension theorem is a fundamental result in measure theory. Often we do not know what a general measurable subset looks like. The Carath\'eodory extension theorem states that to define a measure we only need to assign values to subsets in a generating Boolean algebra. To prove this result categorically, we represent (pre)measures and outer measures by certain (co)lax and strict transformations. The Carath\'eodory extension then corresponds to a Kan extension of strict transformations. We develop a general framework for extensions of transformations between poset-valued functors and give several results on the existence and construction of extensions of these transformations. We proceed by showing that transformations and functors corresponding to measures satisfy these results, which proves the Carath\'eodory extension theorem.


title: MBW: Multi-view Bootstrapping in the Wild

abstract: Labeling articulated objects in unconstrained settings have a wide variety of applications including entertainment, neuroscience, psychology, ethology, and many fields of medicine. Large offline labeled datasets do not exist for all but the most common articulated object categories (e.g., humans). Hand labeling these landmarks within a video sequence is a laborious task. Learned landmark detectors can help, but can be error-prone when trained from only a few examples. Multi-camera systems that train fine-grained detectors have shown significant promise in detecting such errors, allowing for self-supervised solutions that only need a small percentage of the video sequence to be hand-labeled. The approach, however, is based on calibrated cameras and rigid geometry, making it expensive, difficult to manage, and impractical in real-world scenarios. In this paper, we address these bottlenecks by combining a non-rigid 3D neural prior with deep flow to obtain high-fidelity landmark estimates from videos with only two or three uncalibrated, handheld cameras. With just a few annotations (representing 1-2% of the frames), we are able to produce 2D results comparable to state-of-the-art fully supervised methods, along with 3D reconstructions that are impossible with other existing approaches. Our Multi-view Bootstrapping in the Wild (MBW) approach demonstrates impressive results on standard human datasets, as well as tigers, cheetahs, fish, colobus monkeys, chimpanzees, and flamingos from videos captured casually in a zoo. We release the codebase for MBW as well as this challenging zoo dataset consisting image frames of tail-end distribution categories with their corresponding 2D, 3D labels generated from minimal human intervention.


title: Aggregations of quadratic inequalities and hidden hyperplane convexity

abstract: We study properties of the convex hull of a set $S$ described by quadratic inequalities. A simple way of generating inequalities valid on $S$ is to take a nonnegative linear combinations of the defining inequalities of $S$. We call such inequalities aggregations. Special aggregations naturally contain the convex hull of $S$, and we give sufficient conditions for such aggregations to define the convex hull. We introduce the notion of hidden hyperplane convexity (HHC), which is related to the classical notion of hidden convexity of quadratic maps. We show that if the quadratic map associated with $S$ satisfies HHC, then the convex hull of $S$ is defined by special aggregations. To the best of our knowledge, this result generalizes all known results regarding aggregations defining convex hulls. Using this sufficient condition, we are able to recognize previously unknown classes of sets where aggregations lead to convex hull. We show that the condition known as positive definite linear combination together with hidden hyerplane convexity is a sufficient condition for finitely many aggregations to define the convex hull. All the above results are for sets defined using open quadratic inequalities. For closed quadratic inequalities, we prove a new result regarding aggregations giving the convex hull, without topological assumptions on $S$.


title: Dense Prediction Transformer for Scale Estimation in Monocular Visual Odometry

abstract: Monocular visual odometry consists of the estimation of the position of an agent through images of a single camera, and it is applied in autonomous vehicles, medical robots, and augmented reality. However, monocular systems suffer from the scale ambiguity problem due to the lack of depth information in 2D frames. This paper contributes by showing an application of the dense prediction transformer model for scale estimation in monocular visual odometry systems. Experimental results show that the scale drift problem of monocular systems can be reduced through the accurate estimation of the depth map by this model, achieving competitive state-of-the-art performance on a visual odometry benchmark.


title: Spectral stability of near-extremal spacetimes

abstract: It has been suggested that the spectrum of quasinormal modes of rotating black holes is unstable against additional potential terms in the perturbation equation, as the operator associated with the equation is non-self-adjoint. We point out that a bi-linear form has been constructed previously to allow a perturbation analysis of the spectrum, which was applied to study the quasinormal modes of weakly charged Kerr-Newman black holes [Phys. Rev. D 91, 044025 (2015)]. The proposed spectral instability should be restated as instability against potential terms that have infinitesimal ``energy" norm that is specifically defined by the type of inner products introduced by Jaramillo et al. and preserving the physical meaning of energy. We argue that it is necessary to address the stability of all previous mode analysis results to reveal their susceptibility to energetically infinitesimal perturbations. In particular, for near extremal Kerr spacetime, we show that the spectrum of zero-damping modes, which have slow decay rates, is unstable (with order unity fractional change in decay rates) with fine-tuned modification of the potential. The decay rates are, however, always positive with energetically infinitesimal perturbations. If finite potential modifications are allowed near the black hole, it is possible to find superradiantly unstable modes, i.e., a ``black hole bomb" without an explicit outer shell. For the zero-damping modes in near-extremal Reissner-Nordstr\"{o}m-de Sitter black holes, which are relevant for the breakdown of strong cosmic censorship, we find that the corresponding spectrum is stable under energetically infinitesimal perturbations.


title: MEDFAIR: Benchmarking Fairness for Medical Imaging

abstract: A multitude of work has shown that machine learning-based medical diagnosis systems can be biased against certain subgroups of people. This has motivated a growing number of bias mitigation algorithms that aim to address fairness issues in machine learning. However, it is difficult to compare their effectiveness in medical imaging for two reasons. First, there is little consensus on the criteria to assess fairness. Second, existing bias mitigation algorithms are developed under different settings, e.g., datasets, model selection strategies, backbones, and fairness metrics, making a direct comparison and evaluation based on existing results impossible. In this work, we introduce MEDFAIR, a framework to benchmark the fairness of machine learning models for medical imaging. MEDFAIR covers eleven algorithms from various categories, nine datasets from different imaging modalities, and three model selection criteria. Through extensive experiments, we find that the under-studied issue of model selection criterion can have a significant impact on fairness outcomes; while in contrast, state-of-the-art bias mitigation algorithms do not significantly improve fairness outcomes over empirical risk minimization (ERM) in both in-distribution and out-of-distribution settings. We evaluate fairness from various perspectives and make recommendations for different medical application scenarios that require different ethical principles. Our framework provides a reproducible and easy-to-use entry point for the development and evaluation of future bias mitigation algorithms in deep learning. Code is available at https://github.com/ys-zong/MEDFAIR.


title: Detecting asset price bubbles using deep learning

abstract: In this paper we employ deep learning techniques to detect financial asset bubbles by using observed call option prices. The proposed algorithm is widely applicable and model-independent. We test the accuracy of our methodology in numerical experiments within a wide range of models and apply it to market data of tech stocks in order to assess if asset price bubbles are present. Under a given condition on the pricing of call options under asset price bubbles, we are able to provide a theoretical foundation of our approach for positive and continuous stochastic asset price processes. When such a condition is not satisfied, we focus on local volatility models. To this purpose, we give a new necessary and sufficient condition for a process with time-dependent local volatility function to be a strict local martingale.


title: Enhanced CNN with Global Features for Fault Diagnosis of Complex Chemical Processes

abstract: Convolutional neural network (CNN) models have been widely used for fault diagnosis of complex systems. However, traditional CNN models rely on small kernel filters to obtain local features from images. Thus, an excessively deep CNN is required to capture global features, which are critical for fault diagnosis of dynamical systems. In this work, we present an improved CNN that embeds global features (GF-CNN). Our method uses a multi-layer perceptron (MLP) for dimension reduction to directly extract global features and integrate them into the CNN. The advantage of this method is that both local and global patterns in images can be captured by a simple model architecture instead of establishing deep CNN models. The proposed method is applied to the fault diagnosis of the Tennessee Eastman process. Simulation results show that the GF-CNN can significantly improve the fault diagnosis performance compared to traditional CNN. The proposed method can also be applied to other areas such as computer vision and image processing.


title: Kinematic analysis of the Large Magellanic Cloud using Gaia DR3

abstract: Context: The high quality of the Gaia mission data is allowing to study the internal kinematics of the Large Magellanic Cloud (LMC) in unprecedented detail, providing insights on the non-axisymmetric structure of its disc. Aims: To define and validate an improved selection strategy to distinguish the LMC stars from the Milky Way foreground. To check the possible biases that assumed parameters or sample contamination from the Milky Way can introduce in the analysis of the internal kinematics of the LMC using Gaia data. Methods: Our selection is based on a supervised Neural Network classifier using as much as of the Gaia DR3 data as possible. We select three samples of candidate LMC stars with different degrees of completeness and purity; we validate them using different test samples and we compare them with the Gaia Collaboration paper sample. We analyse the resulting velocity profiles and maps, and we check how these results change when using also the line-of-sight velocities, available for a subset of stars. Results: The contamination in the samples from Milky Way stars affects basically the results for the outskirts of the LMC, and the absence of line-of-sight velocities does not bias the results for the kinematics in the inner disc. For the first time, we perform a kinematic analysis of the LMC using samples with the full three dimensional velocity information from Gaia DR3. Conclusions: The dynamics in the inner disc is mainly bar dominated; the kinematics on the spiral arm over-density seem to be dominated by an inward motion and a rotation faster than that of the disc in the piece of the arm attached to the bar; contamination of MW stars seem to dominate the outer parts of the disc and mainly affects old evolutionary phases; uncertainties in the assumed disc morphological parameters and line-of-sight velocity of the LMC can in some cases have significant effects. [ABRIDGED]


title: On parity-odd sector in metric-affine theories

abstract: We undertake the construction of quadratic parity-violating terms involving the curvature in the four-dimensional metric-affine gravity. We demonstrate that there are only 12 linearly independent scalars, plus an additional one that can be removed by using the Pontryagin invariant. Several convenient bases for this sector are provided in both components and differential form notation. We also particularize our general findings to some constrained geometries like Weyl-Cartan and metric-compatible connections.


title: Mergers as a Probe of Particle Dark Matter

abstract: Black holes below Chandrasekhar mass limit (1.4 $M_{\odot}$) can not be produced via any standard stellar evolution. Recently, gravitational wave experiments have also discovered unusually low mass black holes whose origin is yet to be known. We propose a simple yet novel formation mechanism of such low mass black holes. Non-annihilating particle dark matter, owing to their interaction with stellar nuclei, can gradually accumulate inside compact stars, and eventually swallows them to low mass black holes, ordinarily impermissible by the Chandrasekhar limit. We point out several avenues to test this proposal, concentrating on the cosmic evolution of the binary merger rates.


title: Semi-ampleness of NQC generalized log canonical pairs

abstract: We establish a Koll\'ar-type gluing theory for NQC generalized log canonical pairs and use it to prove semi-ampleness results of NQC generalized pairs. As consequences, we prove the existence of flips for any NQC generalized log canonical pair, and show that NQC generalized log canonical singularities are Du Bois.


title: Robust Multi-Agent Coordination from CaTL+ Specifications

abstract: We consider the problem of controlling a heterogeneous multi-agent system required to satisfy temporal logic requirements. Capability Temporal Logic (CaTL) was recently proposed to formalize such specifications for deploying a team of autonomous agents with different capabilities and cooperation requirements. In this paper, we extend CaTL to a new logic CaTL+, which is more expressive than CaTL and has semantics over a continuous workspace shared by all agents. We define two novel robustness metrics for CaTL+: the traditional robustness and the exponential robustness. The latter is sound, differentiable almost everywhere and eliminates masking, which is one of the main limitations of the traditional robustness metric. We formulate a control synthesis problem to maximize CaTL+ robustness and propose a two-step optimization method to solve this problem. Simulation results are included to illustrate the increased expressivity of CaTL+ and the efficacy of the proposed control synthesis approach.


title: Extraction of Pion Transverse Momentum Distributions from Drell-Yan data

abstract: We map the distribution of unpolarized quarks inside a unpolarized pion as a function of the quark's transverse momentum, encoded in unpolarized Transverse Momentum Distributions (TMDs). We extract the pion TMDs from available data of unpolarized pion-nucleus Drell-Yan processes, where the cross section is differential in the lepton-pair transverse momentum. In the cross section, pion TMDs are convoluted with nucleon TMDs that we consistently take from our previous studies. We obtain a fairly good agreement with data. We present also predictions for pion-nucleus scattering that is being measured by the COMPASS Collaboration.


title: Text Characterization Toolkit

abstract: In NLP, models are usually evaluated by reporting single-number performance scores on a number of readily available benchmarks, without much deeper analysis. Here, we argue that - especially given the well-known fact that benchmarks often contain biases, artefacts, and spurious correlations - deeper results analysis should become the de-facto standard when presenting new models or benchmarks. We present a tool that researchers can use to study properties of the dataset and the influence of those properties on their models' behaviour. Our Text Characterization Toolkit includes both an easy-to-use annotation tool, as well as off-the-shelf scripts that can be used for specific analyses. We also present use-cases from three different domains: we use the tool to predict what are difficult examples for given well-known trained models and identify (potentially harmful) biases and heuristics that are present in a dataset.


title: Entanglement of Bipartite Gaussian States: a Simple Criterion and its Geometric Interpretation

abstract: Werner and Wolf have proven in Phys. Rev. Lett. 86(16) (2001) a very elegant necessary and sufficient condition for a bosonic continuous variable bipartite Gaussian mixed quantum state to be separable. This condition is, however, difficult to implement in practice. In the present Letter, we propose a simpler condition which only involves the calculation of the symplectic matrix in the Williamson diagonalization of the covariance matrix of the state under consideration. The main tool in our construction is the observation, proved in previous work, that the Wigner transform is covariant only under symplectic or antisymplectic linear transformations. We also give a geometric interpretation of our condition in terms of the orthogonal projections of "quantum blobs"..


title: Using Entropy Measures for Monitoring the Evolution of Activity Patterns

abstract: In this work, we apply information theory inspired methods to quantify changes in daily activity patterns. We use in-home movement monitoring data and show how they can help indicate the occurrence of healthcare-related events. Three different types of entropy measures namely Shannon's entropy, entropy rates for Markov chains, and entropy production rate have been utilised. The measures are evaluated on a large-scale in-home monitoring dataset that has been collected within our dementia care clinical study. The study uses Internet of Things (IoT) enabled solutions for continuous monitoring of in-home activity, sleep, and physiology to develop care and early intervention solutions to support people living with dementia (PLWD) in their own homes. Our main goal is to show the applicability of the entropy measures to time-series activity data analysis and to use the extracted measures as new engineered features that can be fed into inference and analysis models. The results of our experiments show that in most cases the combination of these measures can indicate the occurrence of healthcare-related events. We also find that different participants with the same events may have different measures based on one entropy measure. So using a combination of these measures in an inference model will be more effective than any of the single measures.


title: TF1 Snowmass Report: Quantum gravity, string theory, and black holes

abstract: We give an overview of the field of quantum gravity, string theory and black holes summarizing various white papers in this subject that were submitted as part of the Snowmass process.


title: ASIF: Coupled Data Turns Unimodal Models to Multimodal Without Training

abstract: CLIP proved that aligning visual and language spaces is key to solving many vision tasks without explicit training, but required to train image and text encoders from scratch on a huge dataset. LiT improved this by only training the text encoder and using a pre-trained vision network. In this paper, we show that a common space can be created without any training at all, using single-domain encoders (trained with or without supervision) and a much smaller amount of image-text pairs. Furthermore, our model has unique properties. Most notably, deploying a new version with updated training samples can be done in a matter of seconds. Additionally, the representations in the common space are easily interpretable as every dimension corresponds to the similarity of the input to a unique image-text pair in the multimodal dataset. Experiments on standard zero-shot visual benchmarks demonstrate the typical transfer ability of image-text models. Overall, our method represents a simple yet surprisingly strong baseline for foundation multimodal models, raising important questions on their data efficiency and on the role of retrieval in machine learning.


title: Enabling a Zero Trust Architecture in a 5G-enabled Smart Grid

abstract: One of the most promising applications of the IoT is the Smart Grid (SG). Integrating SG's data communications network into the power grid allows gathering and analyzing information from power lines, distribution power stations, and end users. A smart grid (SG) requires a prompt and dependable connection to provide real-time monitoring through the IoT. Hence 5G could be considered a catalyst for upgrading the existing power grid systems. Nonetheless, the additional attack surface of information infrastructure has been brought about by the widespread adoption of ubiquitous connectivity in 5G, to which the typical information security system in the smart grid cannot respond promptly. Therefore, guaranteeing the Privacy and Security of a network in a threatening, ever-changing environment requires groundbreaking architectures that go well beyond the limitations of traditional, static security measures. With "Continuous Identity Authentication and Dynamic Access Control" as its foundation, this article analyzes the Zero Trust (ZT) architecture specific to the power system of IoT and uses that knowledge to develop a security protection architecture.


title: Periodic oscillations in the restricted Hip-Hop 2N+1 body problem

abstract: We prove the existence of periodic solutions of the restricted $(2N+1)$-body problem when the $2N$-primaries move on a periodic Hip-Hop solution and the massless body moves on the line that contains the center of mass and is perpendicular to the base of the antiprism formed by the $2N$-primaries.


title: Neural Conservation Laws: A Divergence-Free Perspective

abstract: We investigate the parameterization of deep neural networks that by design satisfy the continuity equation, a fundamental conservation law. This is enabled by the observation that any solution of the continuity equation can be represented as a divergence-free vector field. We hence propose building divergence-free neural networks through the concept of differential forms, and with the aid of automatic differentiation, realize two practical constructions. As a result, we can parameterize pairs of densities and vector fields that always exactly satisfy the continuity equation, foregoing the need for extra penalty methods or expensive numerical simulation. Furthermore, we prove these models are universal and so can be used to represent any divergence-free vector field. Finally, we experimentally validate our approaches by computing neural network-based solutions to fluid equations, solving for the Hodge decomposition, and learning dynamical optimal transport maps.


title: CADet: Fully Self-Supervised Out-Of-Distribution Detection With Contrastive Learning

abstract: Handling out-of-distribution (OOD) samples has become a major stake in the real-world deployment of machine learning systems. This work explores the use of self-supervised contrastive learning to the simultaneous detection of two types of OOD samples: unseen classes and adversarial perturbations. First, we pair self-supervised contrastive learning with the maximum mean discrepancy (MMD) two-sample test. This approach enables us to robustly test whether two independent sets of samples originate from the same distribution, and we demonstrate its effectiveness by discriminating between CIFAR-10 and CIFAR-10.1 with higher confidence than previous work. Motivated by this success, we introduce CADet (Contrastive Anomaly Detection), a novel method for OOD detection of single samples. CADet draws inspiration from MMD, but leverages the similarity between contrastive transformations of a same sample. CADet outperforms existing adversarial detection methods in identifying adversarially perturbed samples on ImageNet and achieves comparable performance to unseen label detection methods on two challenging benchmarks: ImageNet-O and iNaturalist. Significantly, CADet is fully self-supervised and requires neither labels for in-distribution samples nor access to OOD examples.


title: Covariance Steering of Discrete-Time Linear Systems with Mixed Multiplicative and Additive Noise

abstract: In this paper, we study the covariance steering (CS) problem for discrete-time linear systems subject to multiplicative and additive noise. Specifically, we consider two variants of the so-called CS problem. The goal of the first problem, which is called the exact CS problem, is to steer the mean and the covariance of the state process to their desired values in finite time. In the second one, which is called the ``relaxed'' CS problem, the covariance assignment constraint is relaxed into a positive semi-definite constraint. We show that the relaxed CS problem can be cast as an equivalent convex semi-definite program (SDP) after applying suitable variable transformations and constraint relaxations. Furthermore, we propose a two-step solution procedure for the exact CS problem based on the relaxed problem formulation which returns a feasible solution, if there exists one. Finally, results from numerical experiments are provided to show the efficacy of the proposed solution methods.


title: Bang-Bang Boosting of RRTs

abstract: This paper presents methods for dramatically improving the performance of sampling-based kinodynamic planners. The key component is the first-known complete, exact steering method that produces a time-optimal trajectory between any states for a vector of synchronized double integrators. This method is applied in three ways: 1) to generate RRT edges that quickly solve the two-point boundary-value problems, 2) to produce a (quasi)metric for more accurate Voronoi bias in RRTs, and 3) to iteratively time-optimize a given collision-free trajectory. Experiments are performed for state spaces with up to 2000 dimensions, resulting in improved computed trajectories and orders of magnitude computation time improvements over using ordinary metrics and constant controls.


title: Making Decisions under Outcome Performativity

abstract: Decision-makers often act in response to data-driven predictions, with the goal of achieving favorable outcomes. In such settings, predictions don't passively forecast the future; instead, predictions actively shape the distribution of outcomes they are meant to predict. This performative prediction setting raises new challenges for learning "optimal" decision rules. In particular, existing solution concepts do not address the apparent tension between the goals of forecasting outcomes accurately and steering individuals to achieve desirable outcomes. To contend with this concern, we introduce a new optimality concept -- performative omniprediction -- adapted from the supervised (non-performative) learning setting. A performative omnipredictor is a single predictor that simultaneously encodes the optimal decision rule with respect to many possibly-competing objectives. Our main result demonstrates that efficient performative omnipredictors exist, under a natural restriction of performative prediction, which we call outcome performativity. On a technical level, our results follow by carefully generalizing the notion of outcome indistinguishability to the outcome performative setting. From an appropriate notion of Performative OI, we recover many consequences known to hold in the supervised setting, such as omniprediction and universal adaptability.


title: The temperature-dependence of carrier mobility is not a reliable indicator of the dominant scattering mechanism

abstract: The temperature dependence of experimental charge carrier mobility is commonly used as a predictor of the dominant carrier scattering mechanism in semiconductors, particularly in thermoelectric applications. In this work, we critically evaluate whether this practice is well founded. A review of 47 state-of-the-art mobility calculations reveals no correlation between the major scattering mechanism and the temperature trend of mobility. Instead, we demonstrate that the phonon frequencies are the prevailing driving forces behind the temperature dependence and can cause it to vary between $T^{-1}$ to $T^{-3}$ even for an idealised material. To demonstrate this, we calculate the mobility of 23,000 materials and review their temperature dependence, including separating the contributions from deformation, polar, and impurity scattering mechanisms. We conclusively demonstrate that a temperature dependence of $T^{-1.5}$ is not a reliable indicator of deformation potential scattering. Our work highlights the potential pitfalls of predicting the major scattering type based on the experimental mobility temperature trend alone.


title: Learning from Demonstrations of Critical Driving Behaviours Using Driver's Risk Field

abstract: In recent years, imitation learning (IL) has been widely used in industry as the core of autonomous vehicle (AV) planning modules. However, previous IL works show sample inefficiency and low generalisation in safety-critical scenarios, on which they are rarely tested. As a result, IL planners can reach a performance plateau where adding more training data ceases to improve the learnt policy. First, our work presents an IL model using the spline coefficient parameterisation and offline expert queries to enhance safety and training efficiency. Then, we expose the weakness of the learnt IL policy by synthetically generating critical scenarios through optimisation of parameters of the driver's risk field (DRF), a parametric human driving behaviour model implemented in a multi-agent traffic simulator based on the Lyft Prediction Dataset. To continuously improve the learnt policy, we retrain the IL model with augmented data. Thanks to the expressivity and interpretability of the DRF, the desired driving behaviours can be encoded and aggregated to the original training data. Our work constitutes a full development cycle that can efficiently and continuously improve the learnt IL policies in closed-loop. Finally, we show that our IL planner developed with less training resource still has superior performance compared to the previous state-of-the-art.


title: Sharp Analysis of Stochastic Optimization under Global Kurdyka-{\L}ojasiewicz Inequality

abstract: We study the complexity of finding the global solution to stochastic nonconvex optimization when the objective function satisfies global Kurdyka-Lojasiewicz (KL) inequality and the queries from stochastic gradient oracles satisfy mild expected smoothness assumption. We first introduce a general framework to analyze Stochastic Gradient Descent (SGD) and its associated nonlinear dynamics under the setting. As a byproduct of our analysis, we obtain a sample complexity of $\mathcal{O}(\epsilon^{-(4-\alpha)/\alpha})$ for SGD when the objective satisfies the so called $\alpha$-PL condition, where $\alpha$ is the degree of gradient domination. Furthermore, we show that a modified SGD with variance reduction and restarting (PAGER) achieves an improved sample complexity of $\mathcal{O}(\epsilon^{-2/\alpha})$ when the objective satisfies the average smoothness assumption. This leads to the first optimal algorithm for the important case of $\alpha=1$ which appears in applications such as policy optimization in reinforcement learning.


title: Lightweight Strategy for XOR PUFs as Security Primitives for Resource-constrained IoT device

abstract: Physical Unclonable Functions (PUFs) are promising security primitives for resource-constrained IoT devices. And the XOR Arbiter PUF (XOR-PUF) is one of the most studied PUFs, out of an effort to improve the resistance against machine learning attacks of probably the most lightweight delay-based PUFs - the Arbiter PUFs. However, recent attack studies reveal that even XOR-PUFs with large XOR sizes are still not safe against machine learning attacks. Increasing PUF stages or components and using different challenges for different components are two ways to improve the security of APUF-based PUFs, but more stages or components lead to more hardware cost and higher operation power, and different challenges for different components require the transmission of more bits during operations, which also leads to higher power consumption. In this paper, we present a strategy that combines the choice of XOR Arbiter PUF (XOR-PUF) architecture parameters with the way XOR-PUFs are used to achieve lightweights in hardware cost and energy consumption as well as security against machine learning attacks. Experimental evaluations show that with the proposed strategy, highly lightweight component-differentially challenged XOR-PUFs can withstand the most powerful machine learning attacks developed so far and maintain excellent intra-device and inter-device performance, rendering this strategy a potential blueprint for the fabrication and use of XOR-PUFs for resource-constrained IoT applications.


title: Modular Approach to Machine Reading Comprehension: Mixture of Task-Aware Experts

abstract: In this work we present a Mixture of Task-Aware Experts Network for Machine Reading Comprehension on a relatively small dataset. We particularly focus on the issue of common-sense learning, enforcing the common ground knowledge by specifically training different expert networks to capture different kinds of relationships between each passage, question and choice triplet. Moreover, we take inspi ration on the recent advancements of multitask and transfer learning by training each network a relevant focused task. By making the mixture-of-networks aware of a specific goal by enforcing a task and a relationship, we achieve state-of-the-art results and reduce over-fitting.


title: Proportoids

abstract: Analogical proportions are expressions of the form ``$a$ is to $b$ what $c$ is to $d$'' at the core of analogical reasoning. This paper contributes to the mathematical foundations of analogical proportions in the axiomatic tradition as initiated by Yves Lepage two decades ago. For this we introduce proportoids as sets endowed with a 4-ary analogical proportion relation $a:b::c:d$ satisfying a suitable set of axioms and study different kinds of proportion-preserving mappings and relations and their properties.


title: Probe field ellipticity-induced shift in an atomic clock

abstract: We investigate the probe field induced shift for atomic lattice-based and ion-trap clocks, which can be considered as a near resonant ac-Stark shift, connected to the Zeeman structure of atomic levels and their splitting in a dc magnetic field. This shift arises from possible residual ellipticity in the polarization of the probe field and uncertainty in the magnetic field orientation. Such a shift can have an arbitrary sign and, for some experimental conditions, can reach the fractional value of the order of 10$^{-18}$-10$^{-19}$, i.e., it is not negligible. Thus, it should be taken into account in the uncertainty budgets for the modern ultra-precise atomic clocks. In addition, it is shown that when using hyper-Ramsey spectroscopy, this shift can be reduced to a level much lower than $10^{-19}$.


title: HYPRO: A Hybridly Normalized Probabilistic Model for Long-Horizon Prediction of Event Sequences

abstract: In this paper, we tackle the important yet under-investigated problem of making long-horizon prediction of event sequences. Existing state-of-the-art models do not perform well at this task due to their autoregressive structure. We propose HYPRO, a hybridly normalized probabilistic model that naturally fits this task: its first part is an autoregressive base model that learns to propose predictions; its second part is an energy function that learns to reweight the proposals such that more realistic predictions end up with higher probabilities. We also propose efficient training and inference algorithms for this model. Experiments on multiple real-world datasets demonstrate that our proposed HYPRO model can significantly outperform previous models at making long-horizon predictions of future events. We also conduct a range of ablation studies to investigate the effectiveness of each component of our proposed methods.


title: Search for gravitational-wave bursts in the third Advanced LIGO-Virgo run with coherent WaveBurst enhanced by Machine Learning

abstract: This paper presents a search for generic short-duration gravitational-wave (GW) transients (or GW bursts) in the data from the third observing run of Advanced LIGO and Advanced Virgo. We use coherent WaveBurst (cWB) pipeline enhanced with a decision-tree classification algorithm for more efficient separation of GW signals from noise transients. The machine-learning (ML) algorithm is trained on a representative set of noise events and a set of simulated stochastic signals that are not correlated with any known signal model. This training procedure preserves the model-independent nature of the search. We demonstrate that the ML-enhanced cWB pipeline can detect GW signals at a larger distance than previous model-independent searches. The sensitivity improvements are achieved across the broad spectrum of simulated signals, with the goal of testing the robustness of this model-agnostic search. At a false-alarm rate of one event per century, the detectable signal amplitudes are reduced up to almost an order of magnitude, most notably for the single-cycle signal morphologies. This ML-enhanced pipeline also improves the detection efficiency of compact binary mergers in a wide range of masses, from stellar mass to intermediate-mass black holes, both with circular and elliptical orbits. After excluding previously detected compact binaries, no new gravitational-wave signals are observed for the two-fold Hanford-Livingston and the three-fold Hanford-Livingston-Virgo detector networks. With the improved sensitivity of the all-sky search, we obtain the most stringent constraints on the isotropic emission of gravitational-wave energy from short-duration burst sources.


title: Late-time H/He-poor circumstellar interaction in the type-Ic supernova SN 2021ocs: an exposed oxygen-magnesium layer and extreme stripping of the progenitor

abstract: Supernova (SN) 2021ocs was discovered in the galaxy NGC 7828 ($z = 0.01911$) within the interacting system Arp 144, and subsequently classified as a normal type-Ic SN around peak brightness. VLT/FORS2 observations in the nebular phase at 148 d reveal that the spectrum is dominated by oxygen and magnesium emission lines of different transitions and ionization states: O I, [O I], [O II], [O III], Mg I, and Mg II. Such a spectrum has no counterpart in the literature, though it bears a few features similar to those of some interacting type Ibn and Icn SNe. Additionally, SN 2021ocs showed a blue color, $(g-r) \lesssim -0.5$ mag, after the peak and up to late phases, atypical for a type-Ic SN. Together with the nebular spectrum, this suggests that SN 2021ocs underwent late-time interaction with an H/He-poor circumstellar medium (CSM), resulting from the pre-SN progenitor mass loss during its final $\sim$1000 days. The strong O and Mg lines and the absence of strong C and He lines suggest that the progenitor star's O-Mg layer is exposed, which places SN 2021ocs as the most extreme case of massive progenitor star's envelope stripping in interacting SNe, followed by type-Icn (stripped C-O layer) and Ibn (stripped He-rich layer) SNe. This is the first time such a case is reported in the literature. SN 2021ocs emphasizes the importance of late-time spectroscopy of SNe, even for those classified as normal events, to reveal the inner ejecta and progenitor star's CSM and mass loss.


title: Fundamental Investigation of Reactive-Convective Transport: Implications for Long-Term Carbon dioxide (CO2) Sequestration

abstract: The density-driven convection coupled with chemical reaction is the preferred mechanism for permanently storing CO2 in saline aquifers. This study uses a 2D visual Hele-Shaw cell to evaluate and visualize the density-driven convection formed due to gravitational instabilities, also known as Rayleigh-Taylor instability. The primary goal of the experiments is to understand the various mechanisms for the mass transfer of gaseous CO2 into brine with different initial ionic concentrations and flow permeability. Moreover, the impact of CO2 flow rates, injection locations, reservoir dipping angle, and permeability heterogeneity is also investigated. We observed that the presence of salts resulted in earlier onset of convection and a larger convective finger wavelength than the case with no dissolved salts. In addition, experimental data showed a higher lateral mixing between CO2 fingers when dipping is involved. The visual investigation also revealed that the CO2 dissolution rate, measured by the rate of the convective fingers advance, depends on the type and concentration of the ions present in the brine. The CO2 dissolution for solutions with varying salt dissolved, indicated by the area of the pH-depressed region, is observed to be 0.38-0.77 times compared to when no salt is present. Although convective flow is slowed down in the presence of salts, the diffusive flux is enhanced, as observed from both qualitative and quantitative results. Moreover, the reduced formation permeability, introduced by using a flow barrier, resulted in numerous regions not being swept by the dissolved CO2, indicating an inefficient dissolution. We also investigated the effect of discrete high conductivity fractures within the flow barriers, which showed an uneven vertical sweep and enhanced flow channeling. Lastly, the parameters regarding CO2 leakage risk during storage are identified and discussed.


title: Transportability of model-based estimands in evidence synthesis

abstract: In evidence synthesis, effect modifiers are typically described as variables that induce treatment effect heterogeneity at the individual level, through treatment-covariate interactions in an outcome model parametrized at such level. As such, effect modification is defined with respect to a conditional measure, but marginal effect estimates are required for population-level decisions in health technology assessment. For non-collapsible measures, purely prognostic variables that are not determinants of treatment response at the individual level may modify marginal effects, even where there is individual-level treatment effect homogeneity. With heterogeneity, marginal effects for measures that are not directly collapsible cannot be expressed in terms of marginal covariate moments, and generally depend on the joint distribution of conditional effect measure modifiers and purely prognostic variables. There are implications for recommended practices in evidence synthesis. Unadjusted anchored indirect comparisons can be biased in the absence of individual-level treatment effect heterogeneity, or when marginal covariate moments are balanced across studies. Covariate adjustment may be necessary to account for cross-study imbalances in joint covariate distributions involving purely prognostic variables. In the absence of individual patient data for the target, covariate adjustment approaches are inherently limited in their ability to remove bias for measures that are not directly collapsible. Directly collapsible measures would facilitate the transportability of marginal effects between studies by: (1) reducing dependence on model-based covariate adjustment where there is individual-level treatment effect homogeneity or marginal covariate moments are balanced; and (2) facilitating the selection of baseline covariates for adjustment where there is individual-level treatment effect heterogeneity.


title: Isolating Chemical Reaction Mechanism as a Variable with Reactive Coarse-Grained Molecular Dynamics: Step-Growth versus Chain-Growth Polymerization

abstract: We present a general approach to isolate chemical reaction mechanism as an independently controllable variable across chemically distinct systems. Modern approaches to reduce the computational expense of molecular dynamics simulations often group multiple atoms into a single "coarse-grained" interaction site, which leads to a loss of chemical resolution. In this work we convert this shortcoming into a feature and use identical coarse-grained models to represent molecules that share non-reactive characteristics but react by different mechanisms. As a proof of concept we use this approach to simulate and investigate distinct, yet similar, trifunctional isocyanurate resin formulations that polymerize by either chain- or step-growth. Since the underlying molecular mechanics of these models are identical, all emergent differences are a function of the reaction mechanism only. We find that the microscopic morphologies resemble related all-atom simulations and that simulated mechanical testing reasonably agrees with experiment.


title: Distributed Differentially Private Control Synthesis for Multi-Agent Systems with Metric Temporal Logic Specifications

abstract: In this paper, we propose a distributed differentially private receding horizon control (RHC) approach for multi-agent systems (MAS) with metric temporal logic (MTL) specifications. In the MAS considered in this paper, each agent privatizes its sensitive information from other agents using a differential privacy mechanism. In other words, each agent adds privacy noise (e.g., Gaussian noise) to its output to maintain its privacy and communicates its noisy output with its neighboring agents. We define two types of MTL specifications for the MAS: agent-level specifications and system-level specifications. Agents should collaborate to satisfy the system-level MTL specifications with a minimum probability while each agent must satisfy its own agent-level MTL specifications at the same time. In the proposed distributed RHC approach, each agent communicates with its neighboring agents to acquire their noisy outputs and calculates an estimate of the system-level trajectory. Then each agent synthesizes its own control inputs such that the system-level specifications are satisfied with a minimum probability while the agent-level specifications are also satisfied. In the proposed optimization formulation of RHC, we directly incorporate Kalman filter equations to calculate the estimates of the system-level trajectory, and we use mixed-integer linear programming (MILP) to encode the MTL specifications as optimization constraints. Finally, we implement the proposed distributed RHC approach in a case study.


title: Evaluating Disentanglement in Generative Models Without Knowledge of Latent Factors

abstract: Probabilistic generative models provide a flexible and systematic framework for learning the underlying geometry of data. However, model selection in this setting is challenging, particularly when selecting for ill-defined qualities such as disentanglement or interpretability. In this work, we address this gap by introducing a method for ranking generative models based on the training dynamics exhibited during learning. Inspired by recent theoretical characterizations of disentanglement, our method does not require supervision of the underlying latent factors. We evaluate our approach by demonstrating the need for disentanglement metrics which do not require labels\textemdash the underlying generative factors. We additionally demonstrate that our approach correlates with baseline supervised methods for evaluating disentanglement. Finally, we show that our method can be used as an unsupervised indicator for downstream performance on reinforcement learning and fairness-classification problems.


title: A Framework for Web Services Retrieval Using Bio Inspired Clustering

abstract: Efficiently discovering relevant Web services with respect to a specific user query has become a growing challenge owing to the incredible growth in the field of web technologies. In previous works, different clustering models have been used to address these issues. But, most of the traditional clustering techniques are computationally intensive and fail to address all the problems involved. Also, the current standards fail to incorporate the semantic relatedness of Web services during clustering and retrieval resulting in decreased performance. In this paper, we propose a framework for web services retrieval that uses a bottom-up, decentralized and self organising approach to cluster available services. It also provides online, dynamic computation of clusters thus overcoming the drawbacks of traditional clustering methods. We also use the semantic similarity between Web services for the clustering process to enhance the precision and lower the recall.


title: Electronic states of a disordered 2D quasiperiodic tiling: from critical states to Anderson localization

abstract: We consider critical eigenstates in a two dimensional quasicrystal and their evolution as a function of disorder. By exact diagonalization of finite size systems we show that the evolution of properties of a typical wave-function is non-monotonic. That is, disorder leads to states delocalizing, until a certain crossover disorder strength is attained, after which they start to localize. Although this non-monotonic behavior is only present in finite-size systems and vanishes in the thermodynamic limit, the crossover disorder strength decreases logarithmically slowly with system size, and is quite large even for very large approximants. The non-monotonic evolution of spatial properties of eigenstates can be observed in the anomalous dimensions of the wave-function amplitudes, in their multifractal spectra, and in their dynamical properties. We compute the two-point correlation functions of wave-function amplitudes and show that these follow power laws in distance and energy, consistent with the idea that wave-functions retain their multifractal structure on a scale which depends on disorder strength. Dynamical properties are studied as a function of disorder. We find that the diffusion exponents do not reflect the non-monotonic wave-function evolution. Instead, they are essentially independent of disorder until disorder increases beyond the crossover value, after which they decrease rapidly, until the strong localization regime is reached. The differences between our results and earlier studies on geometrically disordered ``phason-flip'' models lead us to propose that the two models are in different universality classes. We conclude by discussing some implications of our results for transport and a proposal for a Mott hopping mechanism between power law localized wave-functions, in moderately disordered quasicrystals.


title: Using Balancing Weights to Target the Treatment Effect on the Treated when Overlap is Poor

abstract: Inverse probability weights are commonly used in epidemiology to estimate causal effects in observational studies. Researchers can typically focus on either the average treatment effect or the average treatment effect on the treated with inverse probability weighting estimators. However, when overlap between the treated and control groups is poor, this can produce extreme weights that can result in biased estimates and large variances. One alternative to inverse probability weights are overlap weights, which target the population with the most overlap on observed characteristics. While estimates based on overlap weights produce less bias in such contexts, the causal estimand can be difficult to interpret. One alternative to inverse probability weights are balancing weights, which directly target imbalances during the estimation process. Here, we explore whether balancing weights allow analysts to target the average treatment effect on the treated in cases where inverse probability weights are biased due to poor overlap. We conduct three simulation studies and an empirical application. We find that in many cases, balancing weights allow the analyst to still target the average treatment effect on the treated even when overlap is poor. We show that while overlap weights remain a key tool for estimating causal effects, more familiar estimands can be targeted by using balancing weights instead of inverse probability weights.


title: Atomic Level Strain Induced by Static and Dynamic Oxygen Vacancies on Reducible Oxide Surfaces

abstract: Surface strain often controls properties of the material including charge transport and chemical reactivity. Localized surface strain is measured with atomic resolution on (111) ceria nanoparticle surfaces using environmental transmission electron microscopy under different redox conditions. Density Functional Theory (DFT) coupled with TEM image simulations have been used for aid in interpreting the experimental data. Oxygen vacancy creation/annihilation introduces strain at surface and near surface regions on cation sublattice. Static and fluxional strainmaps are generated from images at these different conditions and compared. While fluxional strain is highest at locations associated with unstable vacancies at active sites, highly inhomogeneous static strain fields comprising of alternating tensile/compressing strain is seen at surface and subsurfaces linked to the presence of stable oxygen vacancies. Interestingly, both stable and unstable oxygen vacancies are found within a few atomic spacing of each other on the same surface. The static strain pattern depends on the ambient inside TEM. Oxidizing environments tend to lower vacancy concentrations at the surface whereas a highly reducing environment created using high electron dose creates oxygen vacancies everywhere (bulk and surfaces) in the nanoparticle.


title: One Transformer Can Understand Both 2D & 3D Molecular Data

abstract: Unlike vision and language data which usually has a unique format, molecules can naturally be characterized using different chemical formulations. One can view a molecule as a 2D graph or define it as a collection of atoms located in a 3D space. For molecular representation learning, most previous works designed neural networks only for a particular data format, making the learned models likely to fail for other data formats. We believe a general-purpose neural network model for chemistry should be able to handle molecular tasks across data modalities. To achieve this goal, in this work, we develop a novel Transformer-based Molecular model called Transformer-M, which can take molecular data of 2D or 3D formats as input and generate meaningful semantic representations. Using the standard Transformer as the backbone architecture, Transformer-M develops two separated channels to encode 2D and 3D structural information and incorporate them with the atom features in the network modules. When the input data is in a particular format, the corresponding channel will be activated, and the other will be disabled. By training on 2D and 3D molecular data with properly designed supervised signals, Transformer-M automatically learns to leverage knowledge from different data modalities and correctly capture the representations. We conducted extensive experiments for Transformer-M. All empirical results show that Transformer-M can simultaneously achieve strong performance on 2D and 3D tasks, suggesting its broad applicability. The code and models will be made publicly available at https://github.com/lsj2408/Transformer-M.


title: Multi-Agent Chance-Constrained Stochastic Shortest Path with Application to Risk-Aware Intelligent Intersection

abstract: In transportation networks, where traffic lights have traditionally been used for vehicle coordination, intersections act as natural bottlenecks. A formidable challenge for existing automated intersections lies in detecting and reasoning about uncertainty from the operating environment and human-driven vehicles. In this paper, we propose a risk-aware intelligent intersection system for autonomous vehicles (AVs) as well as human-driven vehicles (HVs). We cast the problem as a novel class of Multi-agent Chance-Constrained Stochastic Shortest Path (MCC-SSP) problems and devise an exact Integer Linear Programming (ILP) formulation that is scalable in the number of agents' interaction points (e.g., potential collision points at the intersection). In particular, when the number of agents within an interaction point is small, which is often the case in intersections, the ILP has a polynomial number of variables and constraints. To further improve the running time performance, we show that the collision risk computation can be performed offline. Additionally, a trajectory optimization workflow is provided to generate risk-aware trajectories for any given intersection. The proposed framework is implemented in CARLA simulator and evaluated under a fully autonomous intersection with AVs only as well as in a hybrid setup with a signalized intersection for HVs and an intelligent scheme for AVs. As verified via simulations, the featured approach improves intersection's efficiency by up to $200\%$ while also conforming to the specified tunable risk threshold.


title: Supervised Parameter Estimation of Neuron Populations from Multiple Firing Events

abstract: The firing dynamics of biological neurons in mathematical models is often determined by the model's parameters, representing the neurons' underlying properties. The parameter estimation problem seeks to recover those parameters of a single neuron or a neuron population from their responses to external stimuli and interactions between themselves. Most common methods for tackling this problem in the literature use some mechanistic models in conjunction with either a simulation-based or solution-based optimization scheme. In this paper, we study an automatic approach of learning the parameters of neuron populations from a training set consisting of pairs of spiking series and parameter labels via supervised learning. Unlike previous work, this automatic learning does not require additional simulations at inference time nor expert knowledge in deriving an analytical solution or in constructing some approximate models. We simulate many neuronal populations with different parameter settings using a stochastic neuron model. Using that data, we train a variety of supervised machine learning models, including convolutional and deep neural networks, random forest, and support vector regression. We then compare their performance against classical approaches including a genetic search, Bayesian sequential estimation, and a random walk approximate model. The supervised models almost always outperform the classical methods in parameter estimation and spike reconstruction errors, and computation expense. Convolutional neural network, in particular, is the best among all models across all metrics. The supervised models can also generalize to out-of-distribution data to a certain extent.


title: Disentanglement with Biological Constraints: A Theory of Functional Cell Types

abstract: Neurons in the brain are often finely tuned for specific task variables. Moreover, such disentangled representations are highly sought after in machine learning. Here we mathematically prove that simple biological constraints on neurons, namely nonnegativity and energy efficiency in both activity and weights, promote such sought after disentangled representations by enforcing neurons to become selective for single factors of task variation. We demonstrate these constraints lead to disentanglement in a variety of tasks and architectures, including variational autoencoders. We also use this theory to explain why the brain partitions its cells into distinct cell types such as grid and object-vector cells, and also explain when the brain instead entangles representations in response to entangled task factors. Overall, this work provides a mathematical understanding of why single neurons in the brain often represent single human-interpretable factors, and steps towards an understanding task structure shapes the structure of brain representation.


title: Mind Reader: Reconstructing complex images from brain activities

abstract: Understanding how the brain encodes external stimuli and how these stimuli can be decoded from the measured brain activities are long-standing and challenging questions in neuroscience. In this paper, we focus on reconstructing the complex image stimuli from fMRI (functional magnetic resonance imaging) signals. Unlike previous works that reconstruct images with single objects or simple shapes, our work aims to reconstruct image stimuli that are rich in semantics, closer to everyday scenes, and can reveal more perspectives. However, data scarcity of fMRI datasets is the main obstacle to applying state-of-the-art deep learning models to this problem. We find that incorporating an additional text modality is beneficial for the reconstruction problem compared to directly translating brain signals to images. Therefore, the modalities involved in our method are: (i) voxel-level fMRI signals, (ii) observed images that trigger the brain signals, and (iii) textual description of the images. To further address data scarcity, we leverage an aligned vision-language latent space pre-trained on massive datasets. Instead of training models from scratch to find a latent space shared by the three modalities, we encode fMRI signals into this pre-aligned latent space. Then, conditioned on embeddings in this space, we reconstruct images with a generative model. The reconstructed images from our pipeline balance both naturalness and fidelity: they are photo-realistic and capture the ground truth image contents well.


title: Snowmass 2021 Cross Frontier Report: Dark Matter Complementarity (Extended Version)

abstract: The fundamental nature of Dark Matter is a central theme of the Snowmass 2021 process, extending across all frontiers. In the last decade, advances in detector technology, analysis techniques and theoretical modeling have enabled a new generation of experiments and searches while broadening the types of candidates we can pursue. Over the next decade, there is great potential for discoveries that would transform our understanding of dark matter. In the following, we outline a road map for discovery developed in collaboration among the frontiers. A strong portfolio of experiments that delves deep, searches wide, and harnesses the complementarity between techniques is key to tackling this complicated problem, requiring expertise, results, and planning from all Frontiers of the Snowmass 2021 process.


title: AnoML-IoT: An End to End Re-configurable Multi-protocol Anomaly Detection Pipeline for Internet of Things

abstract: The rapid development in ubiquitous computing has enabled the use of microcontrollers as edge devices. These devices are used to develop truly distributed IoT-based mechanisms where machine learning (ML) models are utilized. However, integrating ML models to edge devices requires an understanding of various software tools such as programming languages and domain-specific knowledge. Anomaly detection is one of the domains where a high level of expertise is required to achieve promising results. In this work, we present AnoML which is an end-to-end data science pipeline that allows the integration of multiple wireless communication protocols, anomaly detection algorithms, deployment to the edge, fog, and cloud platforms with minimal user interaction. We facilitate the development of IoT anomaly detection mechanisms by reducing the barriers that are formed due to the heterogeneity of an IoT environment. The proposed pipeline supports four main phases: (i) data ingestion, (ii) model training, (iii) model deployment, (iv) inference and maintaining. We evaluate the pipeline with two anomaly detection datasets while comparing the efficiency of several machine learning algorithms within different nodes. We also provide the source code (https://gitlab.com/IOTGarage/anoml-iot-analytics) of the developed tools which are the main components of the pipeline.


title: Joint Reconstruction and Parcellation of Cortical Surfaces

abstract: The reconstruction of cerebral cortex surfaces from brain MRI scans is instrumental for the analysis of brain morphology and the detection of cortical thinning in neurodegenerative diseases like Alzheimer's disease (AD). Moreover, for a fine-grained analysis of atrophy patterns, the parcellation of the cortical surfaces into individual brain regions is required. For the former task, powerful deep learning approaches, which provide highly accurate brain surfaces of tissue boundaries from input MRI scans in seconds, have recently been proposed. However, these methods do not come with the ability to provide a parcellation of the reconstructed surfaces. Instead, separate brain-parcellation methods have been developed, which typically consider the cortical surfaces as given, often computed beforehand with FreeSurfer. In this work, we propose two options, one based on a graph classification branch and another based on a novel generic 3D reconstruction loss, to augment template-deformation algorithms such that the surface meshes directly come with an atlas-based brain parcellation. By combining both options with two of the latest cortical surface reconstruction algorithms, we attain highly accurate parcellations with a Dice score of 90.2 (graph classification branch) and 90.4 (novel reconstruction loss) together with state-of-the-art surfaces.


title: $\zeta$-Pade SRWS theory with lowest order approximation

abstract: In my previous paper about SRWS-$\zeta$ theory[Y.Ueoka,viXra:2205.014,2022], I proposed an approximation of rough averaged summation of typical critical Green function for the Anderson transition in the Orthogonal class. In this paper, I remove a rough approximate summation for the series of the typical critical Green function by replacing summation with integral. Pade approximant is used to take a summation. The perturbation series of the critical exponent $\nu$ of localization length from upper critical dimension is obtained. The dimensional dependence of the critical exponent is again directly related with Riemann $\zeta$ function. Degree of freedom about lower critical exponent improve estimate compared with previous studies. When I fix lower critical dimension equal to two, I obtained similar estimate of the critical exponent compared with fitting curve estimate of the critical exponent[E.Tarquini et al.,PhysRevB.95(2017)094204]. 1


title: MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization

abstract: Portfolio management is a fundamental problem in finance. It involves periodic reallocations of assets to maximize the expected returns within an appropriate level of risk exposure. Deep reinforcement learning (RL) has been considered a promising approach to solving this problem owing to its strong capability in sequential decision making. However, due to the non-stationary nature of financial markets, applying RL techniques to portfolio optimization remains a challenging problem. Extracting trading knowledge from various expert strategies could be helpful for agents to accommodate the changing markets. In this paper, we propose MetaTrader, a novel two-stage RL-based approach for portfolio management, which learns to integrate diverse trading policies to adapt to various market conditions. In the first stage, MetaTrader incorporates an imitation learning objective into the reinforcement learning framework. Through imitating different expert demonstrations, MetaTrader acquires a set of trading policies with great diversity. In the second stage, MetaTrader learns a meta-policy to recognize the market conditions and decide on the most proper learned policy to follow. We evaluate the proposed approach on three real-world index datasets and compare it to state-of-the-art baselines. The empirical results demonstrate that MetaTrader significantly outperforms those baselines in balancing profits and risks. Furthermore, thorough ablation studies validate the effectiveness of the components in the proposed approach.


title: Electrical frequency discrimination by fungi Pleurotus ostreatus

abstract: We stimulate mycelian networks of oyster fungi Pleurotus ostreatus with low frequency sinusoidal electrical signals. We demonstrate that the fungal networks can discriminate between frequencies in a fuzzy or threshold based manner. Details about the mixing of frequencies by the mycelium networks are provided. The results advance the novel field of fungal electronics and pave ground for the design of living, fully recyclable, electron devices.


title: DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking

abstract: Predicting the binding structure of a small molecule ligand to a protein -- a task known as molecular docking -- is critical to drug design. Recent deep learning methods that treat docking as a regression problem have decreased runtime compared to traditional search-based methods but have yet to offer substantial improvements in accuracy. We instead frame molecular docking as a generative modeling problem and develop DiffDock, a diffusion generative model over the non-Euclidean manifold of ligand poses. To do so, we map this manifold to the product space of the degrees of freedom (translational, rotational, and torsional) involved in docking and develop an efficient diffusion process on this space. Empirically, DiffDock obtains a 38% top-1 success rate (RMSD<2A) on PDBBind, significantly outperforming the previous state-of-the-art of traditional docking (23%) and deep learning (20%) methods. Moreover, while previous methods are not able to dock on computationally folded structures (maximum accuracy 10.4%), DiffDock maintains significantly higher precision (21.7%). Finally, DiffDock has fast inference times and provides confidence estimates with high selective accuracy.


title: High-Redshift Galaxy Candidates at $z = 9-10$ as Revealed by JWST Observations of WHL0137-08

abstract: We report the discovery of four galaxy candidates observed 450 - 600 Myr after the Big Bang with photometric redshifts between z ~ 8.3 - 10.2 measured using the JWST NIRCam imaging of the galaxy cluster WHL0137-08 observed in 8 filters spanning 0.8-5.0 micron, plus 9 Hubble filters spanning 0.4-1.7 micron. One candidate is gravitationally lensed with a magnification of ~8, while the other three are located in a nearby NIRCam module with expected magnifications of <~ 1.1. Using SED fitting, we estimate the stellar masses of these galaxies are typically in the range log M*/Msun = 8.3 - 8.7. All appear young with mass-weighted ages < 240 Myr, low dust content A_V < 0.15 mag, and specific star formation rates sSFR ~0.25 - 10 /Gyr for most. One z ~ 9 candidate is consistent with an age < 5 Myr and a sSFR ~10 /Gyr, as inferred from a strong F444W excess, implying [OIII]+H-beta rest-frame equivalent width ~2000 Angstrom, although an older z ~ 10 object is also allowed. Another z ~ 9 candidate is lensed into an arc 2.4 arcsec long with a magnification of ~8. This arc is the most spatially-resolved galaxy at z ~ 9 known to date, revealing structures ~30 pc across. Follow-up spectroscopy of WHL0137-08 with JWST/NIRSpec will be useful to spectroscopically confirm these high-redshift galaxy candidates and to study their physical properties in more detail.