title: Early Accretion of Large Amount of Solids for Directly-Imaged Exoplanets

abstract: As the number of planetary mass objects (PMOs, $\lessapprox$13 M$_{\rm{Jupitr}}$) at wider separation ($\gtrapprox$10 AU) grows, there is emerging evidence that they form differently from their higher-mass brown-dwarf (BD) counterparts. Namely, PMOs' atmospheres are enriched by metals which is usually interpreted as a sign of solid accretion. This points to the formation channel through core accretion. However, there has hitherto been no quantitative analysis at population level to investigate the amount and timing of solid accretion. Here, we analyze a sample of five directly-imaged exoplanets with measured stellar and planetary chemical abundances (51 Eri b, $\beta$ Pic b, HIP 65426 b, and HR 8799 c and e). We show that these PMOs accrete large amount of solids regardless of formation channels. On average $\gtrapprox$100 M$_\oplus$ solids (ranging from 98.6 to 845.2 M$_\oplus$ for individual systems) are accreted to enrich planet atmospheres if forming via core accretion whereas the solid accretion needs to be $\gtrapprox$20 M$_\oplus$ (ranging from 22.4 to 782.3 M$_\oplus$) if forming via gravitational instability. The result implies that the solid accretion process has to happen at an early stage ($<$1 Myr) when large amount of solids are available in young protoplanetary disks.


title: Spectral Retrieval with JWST Photometric data: a Case Study for HIP 65426 b

abstract: Half of the JWST high-contrast imaging objects will only have photometric data {{as of Cycle 2}}. However, to better understand their atmospheric chemistry which informs formation origin, spectroscopic data are preferred. Using HIP 65426 b, we investigate to what extent planet properties and atmospheric chemical abundance can be retrieved with only JWST photometric data points (2.5-15.5 $\mu$m) in conjunction with ground-based archival low-resolution spectral data (1.0-2.3 $\mu$m). We find that the data is consistent with an atmosphere with solar metallicity and C/O ratios at 0.40 and 0.55. We rule out 10x solar metallicity and an atmosphere with C/O = 1.0. We also find strong evidence of silicate clouds but no sign of an enshrouding featureless {{dust}} extinction. This work offers guidance and cautionary tales on analyzing data in the absence of medium-to-high resolution spectral data.


title: On the Counting of Involutory MDS Matrices

abstract: The optimal branch number of MDS matrices has established their prominence in the design of diffusion layers for various block ciphers and hash functions. Consequently, several matrix structures have been proposed for designing MDS matrices, including Hadamard and circulant matrices. In this paper, we first provide the count of Hadamard MDS matrices of order $4$ over the field $\mathbb{F}_{2^r}$. Subsequently, we present the counts of order $2$ MDS matrices and order $2$ involutory MDS matrices over the field $\mathbb{F}_{2^r}$. Finally, leveraging these counts of order $2$ matrices, we derive an upper bound for the number of all involutory MDS matrices of order $4$ over $\mathbb{F}_{2^r}$.


title: Towards Automated Accessibility Report Generation for Mobile Apps

abstract: Many apps have basic accessibility issues, like missing labels or low contrast. Automated tools can help app developers catch basic issues, but can be laborious or require writing dedicated tests. We propose a system, motivated by a collaborative process with accessibility stakeholders at a large technology company, to generate whole app accessibility reports by combining varied data collection methods (e.g., app crawling, manual recording) with an existing accessibility scanner. Many such scanners are based on single-screen scanning, and a key problem in whole app accessibility reporting is to effectively de-duplicate and summarize issues collected across an app. To this end, we developed a screen grouping model with 96.9% accuracy (88.8% F1-score) and UI element matching heuristics with 97% accuracy (98.2% F1-score). We combine these technologies in a system to report and summarize unique issues across an app, and enable a unique pixel-based ignore feature to help engineers and testers better manage reported issues across their app's lifetime. We conducted a qualitative evaluation with 18 accessibility-focused engineers and testers which showed this system can enhance their existing accessibility testing toolkit and address key limitations in current accessibility scanning tools.


title: Voice2Action: Language Models as Agent for Efficient Real-Time Interaction in Virtual Reality

abstract: Large Language Models (LLMs) are trained and aligned to follow natural language instructions with only a handful of examples, and they are prompted as task-driven autonomous agents to adapt to various sources of execution environments. However, deploying agent LLMs in virtual reality (VR) has been challenging due to the lack of efficiency in online interactions and the complex manipulation categories in 3D environments. In this work, we propose Voice2Action, a framework that hierarchically analyzes customized voice signals and textual commands through action and entity extraction and divides the execution tasks into canonical interaction subsets in real-time with error prevention from environment feedback. Experiment results in an urban engineering VR environment with synthetic instruction data show that Voice2Action can perform more efficiently and accurately than approaches without optimizations.


title: DataDAM: Efficient Dataset Distillation with Attention Matching

abstract: Researchers have long tried to minimize training costs in deep learning while maintaining strong generalization across diverse datasets. Emerging research on dataset distillation aims to reduce training costs by creating a small synthetic set that contains the information of a larger real dataset and ultimately achieves test accuracy equivalent to a model trained on the whole dataset. Unfortunately, the synthetic data generated by previous methods are not guaranteed to distribute and discriminate as well as the original training data, and they incur significant computational costs. Despite promising results, there still exists a significant performance gap between models trained on condensed synthetic sets and those trained on the whole dataset. In this paper, we address these challenges using efficient Dataset Distillation with Attention Matching (DataDAM), achieving state-of-the-art performance while reducing training costs. Specifically, we learn synthetic images by matching the spatial attention maps of real and synthetic data generated by different layers within a family of randomly initialized neural networks. Our method outperforms the prior methods on several datasets, including CIFAR10/100, TinyImageNet, ImageNet-1K, and subsets of ImageNet-1K across most of the settings, and achieves improvements of up to 6.5% and 4.1% on CIFAR100 and ImageNet-1K, respectively. We also show that our high-quality distilled images have practical benefits for downstream applications, such as continual learning and neural architecture search.


title: What are the Radial Distributions of Density, Outflow Rates, and Cloud Structures in the M 82 Wind?

abstract: Galactic winds play essential roles in the evolution of galaxies through the feedback they provide. Despite intensive studies of winds, the radial distributions of their properties and feedback are rarely observable. Here we present such measurements for the prototypical starburst galaxy, M 82, based on observations by Subaru telescope. We determine the radial distribution of outflow densities ($n_e$) from the spatially-resolved [S II] $\lambda\lambda$ 6717, 6731 emission-lines. We find $n_e$ drops from 200 to 40 cm$^{-3}$ with radius ($r$) between 0.5 and 2.2 kpc with a best-fit power-law index of $r^{-1.2}$. Combined with resolved H$\alpha$ lines, we derive mass, momentum, and energy outflow rates, which drop quite slowly (almost unchanged within error bars) over this range of $r$. This suggests that the galactic wind in M 82 can carry mass, momentum, and energy from the central regions to a few kpc with minimal losses. We further derive outflow cloud properties, including size and column densities. The clouds we measure have pressures and densities that are too high to match those from recent theoretical models and numerical simulations of winds. By comparing with a sample of outflows in local star-forming galaxies studied with UV absorption-lines, the above-derived properties for M 82 outflows match well with the published scaling relationships. These matches suggest that the ionized gas clouds traced in emission and absorption are strongly related. Our measurements motivate future spatially resolved studies of galactic winds, which is the only way to map the structure of their feedback effects.


title: Theory of angular momentum transfer from light to molecules

abstract: We present a theory describing interaction of structured light, such as light carrying orbital angular momentum, with molecules. The light-matter interaction Hamiltonian we derive is expressed through couplings between spherical gradients of the electric field and the (transition) multipole moments of a particle of any non-trivial rotation point group. Our model can therefore accommodate for an arbitrary complexity of the molecular and electric field structure, and can be straightforwardly extended to atoms or nanostructures. Applying this framework to ro-vibrational spectroscopy of molecules, we uncover the general mechanism of angular momentum exchange between the spin and orbital angular momenta of light, molecular rotation and its center-of-mass motion. We show that the non-zero vorticity of Laguerre-Gaussian beams can strongly enhance certain ro-vibrational transitions that are considered forbidden in the case of non-helical light. We discuss the experimental requirements for the observation of these forbidden transitions in state-of-the-art spatially-resolved spectroscopy measurements.

categories: physics.atom-ph physics.atm-clus physics.chem-ph physics.optics


title: Towards Few-Call Model Stealing via Active Self-Paced Knowledge Distillation and Diffusion-Based Image Generation

abstract: Diffusion models showcased strong capabilities in image synthesis, being used in many computer vision tasks with great success. To this end, we propose to explore a new use case, namely to copy black-box classification models without having access to the original training data, the architecture, and the weights of the model, \ie~the model is only exposed through an inference API. More specifically, we can only observe the (soft or hard) labels for some image samples passed as input to the model. Furthermore, we consider an additional constraint limiting the number of model calls, mostly focusing our research on few-call model stealing. In order to solve the model extraction task given the applied restrictions, we propose the following framework. As training data, we create a synthetic data set (called proxy data set) by leveraging the ability of diffusion models to generate realistic and diverse images. Given a maximum number of allowed API calls, we pass the respective number of samples through the black-box model to collect labels. Finally, we distill the knowledge of the black-box teacher (attacked model) into a student model (copy of the attacked model), harnessing both labeled and unlabeled data generated by the diffusion model. We employ a novel active self-paced learning framework to make the most of the proxy data during distillation. Our empirical results on two data sets confirm the superiority of our framework over two state-of-the-art methods in the few-call model extraction scenario.


title: Pointwise uncertainty quantification for sparse variational Gaussian process regression with a Brownian motion prior

abstract: We study pointwise estimation and uncertainty quantification for a sparse variational Gaussian process method with eigenvector inducing variables. For a rescaled Brownian motion prior, we derive theoretical guarantees and limitations for the frequentist size and coverage of pointwise credible sets. For sufficiently many inducing variables, we precisely characterize the asymptotic frequentist coverage, deducing when credible sets from this variational method are conservative and when overconfident/misleading. We numerically illustrate the applicability of our results and discuss connections with other common Gaussian process priors.


title: Federated Learning with Differential Privacy for End-to-End Speech Recognition

abstract: While federated learning (FL) has recently emerged as a promising approach to train machine learning models, it is limited to only preliminary explorations in the domain of automatic speech recognition (ASR). Moreover, FL does not inherently guarantee user privacy and requires the use of differential privacy (DP) for robust privacy guarantees. However, we are not aware of prior work on applying DP to FL for ASR. In this paper, we aim to bridge this research gap by formulating an ASR benchmark for FL with DP and establishing the first baselines. First, we extend the existing research on FL for ASR by exploring different aspects of recent $\textit{large end-to-end transformer models}$: architecture design, seed models, data heterogeneity, domain shift, and impact of cohort size. With a $\textit{practical}$ number of central aggregations we are able to train $\textbf{FL models}$ that are \textbf{nearly optimal} even with heterogeneous data, a seed model from another domain, or no pre-trained seed model. Second, we apply DP to FL for ASR, which is non-trivial since DP noise severely affects model training, especially for large transformer models, due to highly imbalanced gradients in the attention block. We counteract the adverse effect of DP noise by reviving per-layer clipping and explaining why its effect is more apparent in our case than in the prior work. Remarkably, we achieve user-level ($7.2$, $10^{-9}$)-$\textbf{DP}$ (resp. ($4.5$, $10^{-9}$)-$\textbf{DP}$) with a 1.3% (resp. 4.6%) absolute drop in the word error rate for extrapolation to high (resp. low) population scale for $\textbf{FL with DP in ASR}$.


title: Denoising and Selecting Pseudo-Heatmaps for Semi-Supervised Human Pose Estimation

abstract: We propose a new semi-supervised learning design for human pose estimation that revisits the popular dual-student framework and enhances it two ways. First, we introduce a denoising scheme to generate reliable pseudo-heatmaps as targets for learning from unlabeled data. This uses multi-view augmentations and a threshold-and-refine procedure to produce a pool of pseudo-heatmaps. Second, we select the learning targets from these pseudo-heatmaps guided by the estimated cross-student uncertainty. We evaluate our proposed method on multiple evaluation setups on the COCO benchmark. Our results show that our model outperforms previous state-of-the-art semi-supervised pose estimators, especially in extreme low-data regime. For example with only 0.5K labeled images our method is capable of surpassing the best competitor by 7.22 mAP (+25% absolute improvement). We also demonstrate that our model can learn effectively from unlabeled data in the wild to further boost its generalization and performance.


title: Multilingual Natural Language Processing Model for Radiology Reports -- The Summary is all you need!

abstract: The impression section of a radiology report summarizes important radiology findings and plays a critical role in communicating these findings to physicians. However, the preparation of these summaries is time-consuming and error-prone for radiologists. Recently, numerous models for radiology report summarization have been developed. Nevertheless, there is currently no model that can summarize these reports in multiple languages. Such a model could greatly improve future research and the development of Deep Learning models that incorporate data from patients with different ethnic backgrounds. In this study, the generation of radiology impressions in different languages was automated by fine-tuning a model, publicly available, based on a multilingual text-to-text Transformer to summarize findings available in English, Portuguese, and German radiology reports. In a blind test, two board-certified radiologists indicated that for at least 70% of the system-generated summaries, the quality matched or exceeded the corresponding human-written summaries, suggesting substantial clinical reliability. Furthermore, this study showed that the multilingual model outperformed other models that specialized in summarizing radiology reports in only one language, as well as models that were not specifically designed for summarizing radiology reports, such as ChatGPT.


title: Overgroups of exterior powers of an elementary group. Normalizers

abstract: We establish two characterizations of an algebraic group scheme $\bigwedge^m GL_n$ over $\mathbb{Z}$. Geometrically, the scheme $\bigwedge^m GL_n$ is a stabilizer of an explicitly given invariant form or, generally, an invariant ideal of forms. Algebraically, $\bigwedge^m GL_n$ is isomorphic (as a scheme over $\mathbb{Z}$) to a normalizer of the elementary subgroup functor $\bigwedge^m E_n$ and a normalizer of the subscheme $\bigwedge^m SL_n$. Our immediate goal is to apply both descriptions in the "sandwich classification" of overgroups of the elementary subgroup. Additionally, the results can be seen as a solution of the linear preserver problem for algebraic group schemes over $\mathbb{Z}$, providing a more functorial description that goes beyond geometry of the classical case over fields.


title: Geometry of the Minimum Distance

abstract: Let \({\mathbb K}\) be any field, let \(X\subset {\mathbb P}^{k-1}\) be a set of \(n\) distinct \({\mathbb K}\)-rational points, and let \(a\geq 1\) be an integer. In this paper we find lower bounds for the minimum distance \(d(X)_a\) of the evaluation code of order \(a\) associated to \(X\). The first results use \(\alpha(X)\), the initial degree of the defining ideal of \(X\), and the bounds are true for any set \(X\). In another result we use \(s(X)\), the minimum socle degree, to find a lower bound for the case when \(X\) is in general linear position. In both situations we improve and generalize known results.


title: Linkage principle for small quantum groups

abstract: We consider small quantum groups with root systems of Cartan, super and modular type, among others. These are constructed as Drinfeld doubles of finite-dimensional Nichols algebras of diagonal type. We prove a linkage principle for them by adapting techniques from the work of Andersen, Jantzen and Soergel in the context of small quantum groups at roots of unity. Consequently we characterize the blocks of the category of modules. We also find a notion of (a)typicality similar to the one in the representation theory of Lie superalgebras. The typical simple modules turn out to be the simple and projective Verma modules. Moreover, we deduce a character formula for 1-atypical simple modules.


title: Galaxy Distribution Systems as Fractals

abstract: This work tests if the large-scale galaxy distribution can be characterized as a fractal system. The $\Lambda$CDM cosmology with $H_0=(70\pm 5)$ km/s/Mpc is adopted to study the UltraVISTA DR1, COSMOS2015 and SPLASH surveys, alongside the number density equations of these galaxy distribution systems as fractals with dimension D. The relativistic distance definitions $d_L$, $d_Z$ and $d_G$ are used to estimate the galaxy number densities in the redshift interval $0.1 \leq z \leq 4$ at volume limited subsamples. Applying the appropriate relations for the description of galaxy fractal structures with single dimension $D$ in the relativistic settings to these surveys datasets it is possible to state that for $z<1$ the UltraVISTA DR1 galaxies presented an average of $D=(1.58\pm 0.20)$, the COSMOS2015 galaxies produced $D=(1.39\pm 0.19)$ and the SPLASH galaxies generated $D=(1.00\pm 0.12)$. For $1 \leq z \leq 4$ the dimensions respectively decreased to $D=(0.59\pm 0.28)$, $D=0.54^{+0.27}_{-0.26}$ and $D=0.83^{+0.36}_{-0.37}$. These results are robust under the Hubble constant uncertainty assumed here. Analysis of blue and red galaxies subsamples in the COSMOS2015 and SPLASH surveys show that the fractal dimensions of blue galaxies present essentially no alteration from the values above, although the ones for the red galaxies changed mostly to smaller values, meaning that D may be assumed as a more intrinsic property of the distribution of objects in the Universe, thus allowing for the fractal dimension to be used as a tool to study different populations of galaxies. All results confirm the decades old theoretical prediction of a decrease in the fractal dimension for $z>1$ suggesting that either there are yet unclear observational biases causing such decrease in the fractal dimension, or the galaxy clustering was possibly more sparse and the universe void dominated in a not too distant past.


title: Latent Space Symmetry Discovery

abstract: Equivariant neural networks require explicit knowledge of the symmetry group. Automatic symmetry discovery methods aim to relax this constraint and learn invariance and equivariance from data. However, existing symmetry discovery methods are limited to simple linear symmetries and cannot handle the complexity of real-world data. We propose a novel generative model, Latent LieGAN (LaLiGAN), which can discover symmetries of nonlinear group actions. It learns a mapping from the data space to a latent space where the symmetries become linear and simultaneously discovers symmetries in the latent space. Theoretically, we show that our method can express any nonlinear symmetry under some conditions about the group action. Experimentally, we demonstrate that our method can accurately discover the intrinsic symmetry in high-dimensional dynamical systems. LaLiGAN also results in a well-structured latent space that is useful for downstream tasks including equation discovery and long-term forecasting.


title: FashionFlow: Leveraging Diffusion Models for Dynamic Fashion Video Synthesis from Static Imagery

abstract: Our study introduces a new image-to-video generator called FashionFlow to generate fashion videos. By utilising a diffusion model, we are able to create short videos from still fashion images. Our approach involves developing and connecting relevant components with the diffusion model, which results in the creation of high-fidelity videos that are aligned with the conditional image. The components include the use of pseudo-3D convolutional layers to generate videos efficiently. VAE and CLIP encoders capture vital characteristics from still images to condition the diffusion model at a global level. Our research demonstrates a successful synthesis of fashion videos featuring models posing from various angles, showcasing the fit and appearance of the garment. Our findings hold great promise for improving and enhancing the shopping experience for the online fashion industry.


title: Linear classification methods for multivariate repeated measures data -- a simulation study

abstract: Researchers in the behavioral and social sciences often use linear discriminant analysis (LDA) for predictions of group membership (classification) and for identifying the variables most relevant to group separation among a set of continuous correlated variables (description). In this paper, we compare existing linear classification algorithms for nonnormally distributed multivariate repeated measures data in a simulation study based on Likert-type data. It is widely accepted that, as a multivariate technique, LDA provides more accurate results by examining not only the relationship between the independent and dependent variables but also the relationships within the independent variables themselves. In educational and psychological research and other disciplines, longitudinal data are often collected which provide additional temporal information. However, linear classification methods for repeated measures data are rarely discussed in the literature despite these potential applications. These methods are more sensitive to actual group differences by taking the complex correlations between time points and variables into account, when compared to analyzing the data at each time point separately. Moreover, data in the behavioral and social sciences rarely fulfill the multivariate normality assumption, so we consider techniques that additionally do not require multivariate normality. The results show that methods which include multivariate outlier removal before parameter estimation as well as robust parameter estimation using generalized estimating equations (GEE) perform better than the standard repeated measures LDA which assumes multivariate normality. The results of the longitudinal support vector machine (SVM) were not competitive.


title: Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Study

abstract: Membership inference attacks (MIAs) aim to infer whether a data point has been used to train a machine learning model. These attacks can be employed to identify potential privacy vulnerabilities and detect unauthorized use of personal data. While MIAs have been traditionally studied for simple classification models, recent advancements in multi-modal pre-training, such as CLIP, have demonstrated remarkable zero-shot performance across a range of computer vision tasks. However, the sheer scale of data and models presents significant computational challenges for performing the attacks. This paper takes a first step towards developing practical MIAs against large-scale multi-modal models. We introduce a simple baseline strategy by thresholding the cosine similarity between text and image features of a target point and propose further enhancing the baseline by aggregating cosine similarity across transformations of the target. We also present a new weakly supervised attack method that leverages ground-truth non-members (e.g., obtained by using the publication date of a target model and the timestamps of the open data) to further enhance the attack. Our evaluation shows that CLIP models are susceptible to our attack strategies, with our simple baseline achieving over $75\%$ membership identification accuracy. Furthermore, our enhanced attacks outperform the baseline across multiple models and datasets, with the weakly supervised attack demonstrating an average-case performance improvement of $17\%$ and being at least $7$X more effective at low false-positive rates. These findings highlight the importance of protecting the privacy of multi-modal foundational models, which were previously assumed to be less susceptible to MIAs due to less overfitting. Our code is available at https://github.com/ruoxi-jia-group/CLIP-MIA.


title: FedAIoT: A Federated Learning Benchmark for Artificial Intelligence of Things

abstract: There is a significant relevance of federated learning (FL) in the realm of Artificial Intelligence of Things (AIoT). However, most existing FL works do not use datasets collected from authentic IoT devices and thus do not capture unique modalities and inherent challenges of IoT data. To fill this critical gap, in this work, we introduce FedAIoT, an FL benchmark for AIoT. FedAIoT includes eight datasets collected from a wide range of IoT devices. These datasets cover unique IoT modalities and target representative applications of AIoT. FedAIoT also includes a unified end-to-end FL framework for AIoT that simplifies benchmarking the performance of the datasets. Our benchmark results shed light on the opportunities and challenges of FL for AIoT. We hope FedAIoT could serve as an invaluable resource to foster advancements in the important field of FL for AIoT. The repository of FedAIoT is maintained at https://github.com/AIoT-MLSys-Lab/FedAIoT.


title: Gradient and Uncertainty Enhanced Sequential Sampling for Global Fit

abstract: Surrogate models based on machine learning methods have become an important part of modern engineering to replace costly computer simulations. The data used for creating a surrogate model are essential for the model accuracy and often restricted due to cost and time constraints. Adaptive sampling strategies have been shown to reduce the number of samples needed to create an accurate model. This paper proposes a new sampling strategy for global fit called Gradient and Uncertainty Enhanced Sequential Sampling (GUESS). The acquisition function uses two terms: the predictive posterior uncertainty of the surrogate model for exploration of unseen regions and a weighted approximation of the second and higher-order Taylor expansion values for exploitation. Although various sampling strategies have been proposed so far, the selection of a suitable method is not trivial. Therefore, we compared our proposed strategy to 9 adaptive sampling strategies for global surrogate modeling, based on 26 different 1 to 8-dimensional deterministic benchmarks functions. Results show that GUESS achieved on average the highest sample efficiency compared to other surrogate-based strategies on the tested examples. An ablation study considering the behavior of GUESS in higher dimensions and the importance of surrogate choice is also presented.


title: Memory-efficient compression of $\mathcal{DH}^2$-matrices for high-frequency problems

abstract: Directional interpolation is a fast and efficient compression technique for high-frequency Helmholtz boundary integral equations, but it requires a very large amount of storage in its original form. Algebraic recompression can significantly reduce the storage requirements and speed up the solution process accordingly. During the recompression process, weight matrices are required to correctly measure the influence of different basis vectors on the final result, and for highly accurate approximations, these weight matrices require more storage than the final compressed matrix. We present a compression method for the weight matrices and demonstrate that it introduces only a controllable error to the overall approximation. Numerical experiments show that the new method leads to a significant reduction in storage requirements.


title: Reinforcement Learning for Node Selection in Branch-and-Bound

abstract: A big challenge in branch and bound lies in identifying the optimal node within the search tree from which to proceed. Current state-of-the-art selectors utilize either hand-crafted ensembles that automatically switch between naive sub-node selectors, or learned node selectors that rely on individual node data. We propose a novel simulation technique that uses reinforcement learning (RL) while considering the entire tree state, rather than just isolated nodes. To achieve this, we train a graph neural network that produces a probability distribution based on the path from the model's root to its "to-be-selected" leaves. Modelling node-selection as a probability distribution allows us to train the model using state-of-the-art RL techniques that capture both intrinsic node-quality and node-evaluation costs. Our method induces a high quality node selection policy on a set of varied and complex problem sets, despite only being trained on specially designed, synthetic travelling salesmen problem (TSP) instances. Using such a fixed pretrained policy shows significant improvements on several benchmarks in optimality gap reductions and per-node efficiency under strict time constraints.


title: HyperMask: Adaptive Hypernetwork-based Masks for Continual Learning

abstract: Artificial neural networks suffer from catastrophic forgetting when they are sequentially trained on multiple tasks. Many continual learning (CL) strategies are trying to overcome this problem. One of the most effective is the hypernetwork-based approach. The hypernetwork generates the weights of a target model based on the task's identity. The model's main limitation is that, in practice, the hypernetwork can produce completely different architectures for subsequent tasks. To solve such a problem, we use the lottery ticket hypothesis, which postulates the existence of sparse subnetworks, named winning tickets, that preserve the performance of a whole network. In the paper, we propose a method called HyperMask, which dynamically filters a target network depending on the CL task. The hypernetwork produces semi-binary masks to obtain dedicated target subnetworks. Moreover, due to the lottery ticket hypothesis, we can use a single network with weighted subnets. Depending on the task, the importance of some weights may be dynamically enhanced while others may be weakened. HyperMask achieves competitive results in several CL datasets and, in some scenarios, goes beyond the state-of-the-art scores, both with derived and unknown task identities.


title: Realistic non-collinear ground states of solids with source-free exchange correlation functional

abstract: In this work, we extend the source-free (SF) exchange correlation (XC) functional developed by Sangeeta Sharma and co-workers to plane-wave density functional theory (DFT) based on the projector augmented wave (PAW) method. This constraint is implemented by the current authors within the VASP source code, using a fast Poisson solver that capitalizes on the parallel three-dimensional fast Fourier transforms (FFTs) implemented in VASP. Using this modified XC functional, we explore the improved convergence behavior that results from applying this constraint to the GGA-PBE+$U$+$J$ functional. In the process, we compare the non-collinear magnetic ground state computed by each functional and their SF counterpart for a select number of magnetic materials in order to provide a metric for comparing with experimentally determined magnetic orderings. We observe significantly improved agreement with experimentally measured magnetic ground state structures after applying the source-free constraint. Furthermore, we explore the importance of considering probability current densities in spin polarized systems, even under no applied field. We analyze the XC torque as well, in order to provide theoretical and computational analyses of the net XC magnetic torque induced by the source-free constraint. Along these lines, we highlight the importance of properly considering the real-space integral of the source-free local magnetic XC field. Our analysis on probability currents, net torque, and constant terms draws additional links to the rich body of previous research on spin-current density functional theory (SCDFT), and paves the way for future extensions and corrections to the SF corrected XC functional.


title: Learning Over Molecular Conformer Ensembles: Datasets and Benchmarks

abstract: Molecular Representation Learning (MRL) has proven impactful in numerous biochemical applications such as drug discovery and enzyme design. While Graph Neural Networks (GNNs) are effective at learning molecular representations from a 2D molecular graph or a single 3D structure, existing works often overlook the flexible nature of molecules, which continuously interconvert across conformations via chemical bond rotations and minor vibrational perturbations. To better account for molecular flexibility, some recent works formulate MRL as an ensemble learning problem, focusing on explicitly learning from a set of conformer structures. However, most of these studies have limited datasets, tasks, and models. In this work, we introduce the first MoleculAR Conformer Ensemble Learning (MARCEL) benchmark to thoroughly evaluate the potential of learning on conformer ensembles and suggest promising research directions. MARCEL includes four datasets covering diverse molecule- and reaction-level properties of chemically diverse molecules including organocatalysts and transition-metal catalysts, extending beyond the scope of common GNN benchmarks that are confined to drug-like molecules. In addition, we conduct a comprehensive empirical study, which benchmarks representative 1D, 2D, and 3D molecular representation learning models, along with two strategies that explicitly incorporate conformer ensembles into 3D MRL models. Our findings reveal that direct learning from an accessible conformer space can improve performance on a variety of tasks and models.


title: Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz Regularization

abstract: To improve the robustness of deep classifiers against adversarial perturbations, many approaches have been proposed, such as designing new architectures with better robustness properties (e.g., Lipschitz-capped networks), or modifying the training process itself (e.g., min-max optimization, constrained learning, or regularization). These approaches, however, might not be effective at increasing the margin in the input (feature) space. As a result, there has been an increasing interest in developing training procedures that can directly manipulate the decision boundary in the input space. In this paper, we build upon recent developments in this category by developing a robust training algorithm whose objective is to increase the margin in the output (logit) space while regularizing the Lipschitz constant of the model along vulnerable directions. We show that these two objectives can directly promote larger margins in the input space. To this end, we develop a scalable method for calculating guaranteed differentiable upper bounds on the Lipschitz constant of neural networks accurately and efficiently. The relative accuracy of the bounds prevents excessive regularization and allows for more direct manipulation of the decision boundary. Furthermore, our Lipschitz bounding algorithm exploits the monotonicity and Lipschitz continuity of the activation layers, and the resulting bounds can be used to design new layers with controllable bounds on their Lipschitz constant. Experiments on the MNIST, CIFAR-10, and Tiny-ImageNet data sets verify that our proposed algorithm obtains competitively improved results compared to the state-of-the-art.


title: ABScribe: Rapid Exploration & Organization of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large Language Models

abstract: Exploring alternative ideas by rewriting text is integral to the writing process. State-of-the-art Large Language Models (LLMs) can simplify writing variation generation. However, current interfaces pose challenges for simultaneous consideration of multiple variations: creating new variations without overwriting text can be difficult, and pasting them sequentially can clutter documents, increasing workload and disrupting writers' flow. To tackle this, we present ABScribe, an interface that supports rapid, yet visually structured, exploration and organization of writing variations in human-AI co-writing tasks. With ABScribe, users can swiftly modify variations using LLM prompts, which are auto-converted into reusable buttons. Variations are stored adjacently within text fields for rapid in-place comparisons using mouse-over interactions on a popup toolbar. Our user study with 12 writers shows that ABScribe significantly reduces task workload (d = 1.20, p < 0.001), enhances user perceptions of the revision process (d = 2.41, p < 0.001) compared to a popular baseline workflow, and provides insights into how writers explore variations using LLMs.


title: Transforming Materials Discovery for Artificial Photosynthesis: High-Throughput Screening of Earth-Abundant Semiconductors

abstract: We present a highly efficient workflow for designing semiconductor structures with specific physical properties, which can be utilized for a range of applications, including photocatalytic water splitting. Our algorithm generates candidate structures composed of earth-abundant elements that exhibit optimal light-trapping, high efficiency in \ce{H2} and/or \ce{O2} production, and resistance to reduction and oxidation in aqueous media. To achieve this, we use an ionic translation model trained on the Inorganic Crystal Structure Database (ICSD) to predict over thirty thousand undiscovered semiconductor compositions. These predictions are then screened for redox stability under Hydrogen Evolution Reaction (HER) or Oxygen Evolution Reaction (OER) conditions before generating thermodynamically stable crystal structures and calculating accurate band gap values for the compounds. Our approach results in the identification of dozens of promising semiconductor candidates with ideal properties for artificial photosynthesis, offering a significant advancement toward the conversion of sunlight into chemical fuels.


title: Fewshot learning on global multimodal embeddings for earth observation tasks

abstract: In this work we pretrain a CLIP/ViT based model using three different modalities of satellite imagery across five AOIs covering over ~10\% of Earth's total landmass, namely Sentinel 2 RGB optical imagery, Sentinel 1 SAR radar amplitude and interferometric coherence. This model uses $\sim 250$ M parameters. Then, we use the embeddings produced for each modality with a classical machine learning method to attempt different downstream tasks for earth observation related to vegetation, built up surface, croplands and permanent water. We consistently show how we reduce the need for labeled data by 99\%, so that with ~200-500 randomly selected labeled examples (around 4K-10K km$^2$) we reach performance levels analogous to those achieved with the full labeled datasets (about 150K image chips or 3M km$^2$ in each area of interest - AOI) on all modalities, AOIs and downstream tasks. This leads us to think that the model has captured significant earth features useful in a wide variety of scenarios. To enhance our model's usability in practice, its architecture allows inference in contexts with missing modalities and even missing channels within each modality. Additionally, we visually show that this embedding space, obtained with no labels, is sensible to the different earth features represented by the labelled datasets we selected.


title: Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs

abstract: Memory complexity and data scarcity have so far prohibited learning solution operators of partial differential equations (PDEs) at high resolutions. We address these limitations by introducing a new data efficient and highly parallelizable operator learning approach with reduced memory requirement and better generalization, called multi-grid tensorized neural operator (MG-TFNO). MG-TFNO scales to large resolutions by leveraging local and global structures of full-scale, real-world phenomena, through a decomposition of both the input domain and the operator's parameter space. Our contributions are threefold: i) we enable parallelization over input samples with a novel multi-grid-based domain decomposition, ii) we represent the parameters of the model in a high-order latent subspace of the Fourier domain, through a global tensor factorization, resulting in an extreme reduction in the number of parameters and improved generalization, and iii) we propose architectural improvements to the backbone FNO. Our approach can be used in any operator learning setting. We demonstrate superior performance on the turbulent Navier-Stokes equations where we achieve less than half the error with over 150x compression. The tensorization combined with the domain decomposition, yields over 150x reduction in the number of parameters and 7x reduction in the domain size without losses in accuracy, while slightly enabling parallelism.


title: Tridiagonal matrix decomposition for Hamiltonian simulation on a quantum computer

abstract: The construction of quantum circuits to simulate Hamiltonian evolution is central to many quantum algorithms. State-of-the-art circuits are based on oracles whose implementation is often omitted, and the complexity of the algorithm is estimated by counting oracle queries. However, in practical applications, an oracle implementation contributes a large constant factor to the overall complexity of the algorithm. The key finding of this work is the efficient procedure for representation of a tridiagonal matrix in the Pauli basis, which allows one to construct a Hamiltonian evolution circuit without the use of oracles. The procedure represents a general tridiagonal matrix $2^n \times 2^n$ by systematically determining all Pauli strings present in the decomposition, dividing them into commuting subsets. The efficiency is in the number of commuting subsets $O(n)$. The method is demonstrated using the one-dimensional wave equation, verifying numerically that the gate complexity as function of the number of qubits is lower than the oracle based approach for $n < 15$ and requires half the number of qubits. This method is applicable to other Hamiltonians based on the tridiagonal matrices.


title: Dimension bounds for escape on average in homogeneous spaces

abstract: Let $X = G/\Gamma$, where $G$ is a Lie group and $\Gamma$ is a uniform lattice in $G$, and let $O$ be an open subset of $X$. We give an upper estimate for the Hausdorff dimension of the set of points whose trajectories escape $O$ on average with frequency $\delta$, where $0 < \delta \le 1$.


title: Introducing the Condor Array Telescope: III. The expansion and age of the shell of the dwarf nova Z Camelopardalis, and detection of a second, larger shell

abstract: The existence of a vast nova shell surrounding the prototypical dwarf nova Z Camelopardalis (Z Cam) proves that some old novae undergo metamorphosis to appear as dwarf novae thousands of years after a nova eruption. The expansion rates of ancient nova shells offer a way to constrain both the time between nova eruptions and the time for post-nova mass transfer rates to decrease significantly, simultaneously testing nova thermonuclear runaway models and hibernation theory. Previous limits on the expansion rate of part of the Z Cam shell constrain the inter-eruption time between Z Cam nova events to be $>$ 1300 years. Deeper narrow-band imaging of the ejecta of Z Cam with the Condor Array Telescope now reveals very low surface brightness areas of the remainder of the shell. A second, even fainter shell is also detected, concentric with and nearly three times the size of the "inner" shell. This is the first observational support of the prediction that concentric shells must surround the frequently-erupting novae of relatively massive white dwarfs. The Condor images extend our Z Cam imaging baseline to 15 years, yielding the inner shell's expansion rate as $v = 83 \pm 37$ km s$^{-1}$ at 23 degrees South of West, in excellent agreement with our 2012 prediction. This velocity corresponds to an approximate age of $t = 2672^{-817}_{+2102}$ yr. While consistent with the suggestion that the most recent nova eruption of Z Cam was the transient recorded by Chinese Imperial astrologers in the year 77 BCE, the age uncertainty is still too large to support or disprove a connection with Z Cam.


title: Bidirectional multi-photon communication between remote superconducting nodes

abstract: Quantum communication testbeds provide a useful resource for experimentally investigating a variety of communication protocols. Here we demonstrate a superconducting circuit testbed with bidirectional multi-photon state transfer capability using time-domain shaped wavepackets. The system we use to achieve this comprises two remote nodes, each including a tunable superconducting transmon qubit and a tunable microwave-frequency resonator, linked by a 2 m-long superconducting coplanar waveguide, which serves as a transmission line. We transfer both individual and superposition Fock states between the two remote nodes, and additionally show that this bidirectional state transfer can be done simultaneously, as well as used to entangle elements in the two nodes.


title: Covariance Expressions for Multi-Fidelity Sampling with Multi-Output, Multi-Statistic Estimators: Application to Approximate Control Variates

abstract: We provide a collection of results on covariance expressions between Monte Carlo based multi-output mean, variance, and Sobol main effect variance estimators from an ensemble of models. These covariances can be used within multi-fidelity uncertainty quantification strategies that seek to reduce the estimator variance of high-fidelity Monte Carlo estimators with an ensemble of low-fidelity models. Such covariance expressions are required within approaches like the approximate control variate and multi-level best linear unbiased estimator. While the literature provides these expressions for some single-output cases such as mean and variance, our results are relevant to both multiple function outputs and multiple statistics across any sampling strategy. Following the description of these results, we use them within an approximate control variate scheme to show that leveraging multiple outputs can dramatically reduce estimator variance compared to single-output approaches. Synthetic examples are used to highlight the effects of optimal sample allocation and pilot sample estimation. A flight-trajectory simulation of entry, descent, and landing is used to demonstrate multi-output estimation in practical applications.


title: Simulations for Meta-analysis of Magnitude Measures

abstract: Meta-analysis aims to combine effect measures from several studies. For continuous outcomes, the most popular effect measures use simple or standardized differences in sample means. However, a number of applications focus on the absolute values of these effect measures (i.e., unsigned magnitude effects). We provide statistical methods for meta-analysis of magnitude effects based on standardized mean differences. We propose a suitable statistical model for random-effects meta-analysis of absolute standardized mean differences (ASMD), investigate a number of statistical methods for point and interval estimation, and provide practical recommendations for choosing among them.


title: Sensor Placement for Flapping Wing Model Using Stochastic Observability Gramians

abstract: Systems in nature are stochastic as well as nonlinear. In traditional applications, engineered filters aim to minimize the stochastic effects caused by process and measurement noise. Conversely, a previous study showed that the process noise can reveal the observability of a system that was initially categorized as unobservable when deterministic tools were used. In this paper, we develop a stochastic framework to explore observability analysis and sensor placement. This framework allows for direct studies of the effects of stochasticity on optimal sensor placement and selection to improve filter error covariance. Numerical results are presented for sensor selection that optimizes stochastic empirical observability in a bioinspired setting.


title: Ultrafast Raman thermometry in driven YBa$_2$Cu$_3$O$_{6.48}$

abstract: Signatures of photo-induced superconductivity have been reported in cuprate materials subjected to a coherent phonon drive. A 'cold' superfluid was extracted from the transient Terahertz conductivity and was seen to coexist with 'hot' uncondensed quasi-particles, a hallmark of a driven-dissipative system of which the interplay between coherent and incoherent responses are not well understood. Here, time resolved spontaneous Raman scattering was used to probe the lattice temperature in the photo-induced superconducting state of YBa2Cu3O6.48. An increase in lattice temperature of approximately 80 K was observed by measuring the time dependent Raman scattering intensity of an undriven 'spectator' phonon mode. This is to be compared with an estimated increase in quasi-particle temperatures of nearly 200 K. These temperature changes provide quantitative information on the nature of the driven state and its decay, and may provide a strategy to optimize this effect.


title: ILB: Graph Neural Network Enabled Emergency Demand Response Program For Electricity

abstract: Demand Response (DR) programs have become a crucial component of smart electricity grids as they shift the flexibility of electricity consumption from supply to demand in response to the ever-growing demand for electricity. In particular, in times of crisis, an emergency DR program is required to manage unexpected spikes in energy demand. In this paper, we propose the Incentive-Driven Load Balancer (ILB), a program designed to efficiently manage demand and response during crisis situations. By offering incentives to flexible households likely to reduce demand, the ILB facilitates effective demand reduction and prepares them for unexpected events. To enable ILB, we introduce a two-step machine learning-based framework for participant selection, which employs a graph-based approach to identify households capable of easily adjusting their electricity consumption. This framework utilizes two Graph Neural Networks (GNNs): one for pattern recognition and another for household selection. Through extensive experiments on household-level electricity consumption in California, Michigan, and Texas, we demonstrate the ILB program's significant effectiveness in supporting communities during emergencies.


title: Lower bound for KVol on the minimal stratum of translation surfaces

abstract: We are interested in the algebraic intersection of closed curves of a given length on translation surfaces. Namely, we study the quantity KVol which measures how many times can two closed curves of a given length intersect. In this paper, we construct families of translation surfaces in each connected component of the minimal stratum $\mathcal{H}(2g-2)$ of the moduli space of translation surfaces of genus $g \geq 2$ such that KVol is arbitrarily close to the genus of the surface, which is conjectured to be the infimum of KVol on $\mathcal{H}(2g-2)$.


title: Event-Triggered Control of Neuron Growth with Actuation at Soma

abstract: We introduce a dynamic event-triggering mechanism for regulating the axonal growth of a neuron. We apply boundary actuation at the soma (the part of a neuron that contains the nucleus) and regulate the dynamics of tubulin concentration and axon length. The control law is formulated by applying a Zero-Order Hold (ZOH) to a continuous-time controller which guides the axon to reach the desired length. The proposed dynamic event-triggering mechanism determines the specific time instants at which control inputs are sampled from the continuous-time control law. We establish the existence of a minimum dwell-time between two triggering times that ensures avoidance of Zeno behavior. Through employing the Lyapunov analysis with PDE backstepping, we prove the local stability of the closed-loop system in $L_2$-norm, initially for the target system, and subsequently for the original system. The effectiveness of the proposed method is showcased through numerical simulations.


title: QDFormer: Towards Robust Audiovisual Segmentation in Complex Environments with Quantization-based Semantic Decomposition

abstract: Audiovisual segmentation (AVS) is a challenging task that aims to segment visual objects in videos according to their associated acoustic cues. With multiple sound sources and background disturbances involved, establishing robust correspondences between audio and visual contents poses unique challenges due to (1) complex entanglement across sound sources and (2) frequent changes in the occurrence of distinct sound events. Assuming sound events occur independently, the multi-source semantic space can be represented as the Cartesian product of single-source sub-spaces. We are motivated to decompose the multi-source audio semantics into single-source semantics for more effective interactions with visual content. We propose a semantic decomposition method based on product quantization, where the multi-source semantics can be decomposed and represented by several disentangled and noise-suppressed single-source semantics. Furthermore, we introduce a global-to-local quantization mechanism, which distills knowledge from stable global (clip-level) features into local (frame-level) ones, to handle frequent changes in audio semantics. Extensive experiments demonstrate that our semantically decomposed audio representation significantly improves AVS performance, e.g., +21.2% mIoU on the challenging AVS-Semantic benchmark with ResNet50 backbone. https://github.com/lxa9867/QSD.


title: Prior Mismatch and Adaptation in PnP-ADMM with a Nonconvex Convergence Analysis

abstract: Plug-and-Play (PnP) priors is a widely-used family of methods for solving imaging inverse problems by integrating physical measurement models with image priors specified using image denoisers. PnP methods have been shown to achieve state-of-the-art performance when the prior is obtained using powerful deep denoisers. Despite extensive work on PnP, the topic of distribution mismatch between the training and testing data has often been overlooked in the PnP literature. This paper presents a set of new theoretical and numerical results on the topic of prior distribution mismatch and domain adaptation for alternating direction method of multipliers (ADMM) variant of PnP. Our theoretical result provides an explicit error bound for PnP-ADMM due to the mismatch between the desired denoiser and the one used for inference. Our analysis contributes to the work in the area by considering the mismatch under nonconvex data-fidelity terms and expansive denoisers. Our first set of numerical results quantifies the impact of the prior distribution mismatch on the performance of PnP-ADMM on the problem of image super-resolution. Our second set of numerical results considers a simple and effective domain adaption strategy that closes the performance gap due to the use of mismatched denoisers. Our results suggest the relative robustness of PnP-ADMM to prior distribution mismatch, while also showing that the performance gap can be significantly reduced with few training samples from the desired distribution.


title: Brauer's problem 21 for principal blocks

abstract: Problem 21 of Brauer's list of problems from 1963 asks whether for any positive integer k there are finitely many isomorphism classes of groups that occur as the defect group of a block with k irreducible characters. We solve this problem for principal blocks. Another long-standing open problem (from 1982) in this area asks whether the defect group of a block with 3 irreducible characters is necessarily the cyclic group of order 3. In most cases we reduce this problem to a question on simple groups that is closely related to the recent solution of Brauer's height zero conjecture.


title: Alpha-Fair Routing in Urban Air Mobility with Risk-Aware Constraints

abstract: In the vision of urban air mobility, air transport systems serve the demands of urban communities by routing flight traffic in networks formed by vertiports and flight corridors. We develop a routing algorithm to ensure that the air traffic flow fairly serves the demand of multiple communities subject to stochastic network capacity constraints. This algorithm guarantees that the flight traffic volume allocated to different communities satisfies the \emph{alpha-fairness conditions}, a commonly used family of fairness conditions in resource allocation. It further ensures robust satisfaction of stochastic network capacity constraints by bounding the coherent risk measures of capacity violation. We prove that implementing the proposed algorithm is equivalent to solving a convex optimization problem. We demonstrate the proposed algorithm using a case study based on the city of Austin. Compared with one that maximizes the total served demands, the proposed algorithm promotes even distributions of served demands for different communities.


title: Optimizing performance in Basketball: A Game-Theoretic Approach to Shot Percentage Distribution in a team

abstract: In this paper, we propose a shot percentage distribution strategy among the players of a basketball team to maximize the score that can be achieved by them. The approach is based on the concepts of game theory related to network flow.


title: On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspective

abstract: The neural tangent kernel (NTK) has garnered significant attention as a theoretical framework for describing the behavior of large-scale neural networks. Kernel methods are theoretically well-understood and as a result enjoy algorithmic benefits, which can be demonstrated to hold in wide synthetic neural network architectures. These advantages include faster optimization, reliable uncertainty quantification and improved continual learning. However, current results quantifying the rate of convergence to the kernel regime suggest that exploiting these benefits requires architectures that are orders of magnitude wider than they are deep. This assumption raises concerns that architectures used in practice do not exhibit behaviors as predicted by the NTK. Here, we supplement previous work on the NTK by empirically investigating whether the limiting regime predicts practically relevant behavior of large-width architectures. Our results demonstrate that this is not the case across multiple domains. This observed disconnect between theory and practice further calls into question to what degree NTK theory should inform architectural and algorithmic choices.


title: Highly Coherent Supercontinuum Generation in Circular Lattice Photonic Crystal Fibers Using Low-power Pulses

abstract: Two structures of circular lattice Photonic Crystal Fibers (PCFs) based on $Ge_{20}Sb_{15}Se_{65}$ (GSS) material have been proposed for a highly coherent broadband supercontinuum generation (SCG) in the mid-infrared region. Numerical studies on both structures show that the fundamental modes are well confined in the core while the confinement losses are very low. Also, the high nonlinear coefficient of 22.01 $W^{-1}m^{-1}$ and 17.99202 $W^{-1}m^{-1}$ for the two structures ensure that these structures can accomplish a high nonlinear activity. It has been found that broadband supercontinuum (SCs) spanning from 0.45 $\mu$m to 5.3 $\mu$m and 0.48 ${\mu}$m to 6.5 ${\mu}$m can be generated using a hyperbolic secant pulse of 0.5 kW. The proposed structures also show very good structural tolerance to optical properties that prevent any radical shift in SC spectra owing to potential fabrication mismatch.


title: Coarse-graining and criticality in the human connectome

abstract: In the face of the stupefying complexity of the human brain, network analysis is a most useful tool that allows one to greatly simplify the problem, typically by approximating the billions of neurons comprising the brain by means of a coarse-grained picture with a practicable number of nodes. But even such relatively small and coarse networks, such as the human connectome with its 100-1000 nodes, may present challenges for some computationally demanding analyses that are incapable of handling networks with more than a handful of nodes. With such applications in mind, we set out to further coarse-grain the human connectome by taking a modularity-based approach, the goal being to produce a network of a relatively small number of modules. We applied this approach to study critical phenomena in the brain; we formulated a hypothesis based on the coarse-grained networks in the context of criticality in the Wilson-Cowan and Ising models, and we verified the hypothesis, which connected a transition value of the former with the critical temperature of the latter, using the original networks. We found that the qualitative behavior of the coarse-grained networks reflected that of the original networks, albeit to a less pronounced extent. This, in principle, allows for the drawing of similar qualitative conclusions by analysing the smaller networks, which opens the door for studying the human connectome in contexts typically regarded as computationally intractable, such Integrated Information Theory and quantum models of the human brain.


title: GASS: Generalizing Audio Source Separation with Large-scale Data

abstract: Universal source separation targets at separating the audio sources of an arbitrary mix, removing the constraint to operate on a specific domain like speech or music. Yet, the potential of universal source separation is limited because most existing works focus on mixes with predominantly sound events, and small training datasets also limit its potential for supervised learning. Here, we study a single general audio source separation (GASS) model trained to separate speech, music, and sound events in a supervised fashion with a large-scale dataset. We assess GASS models on a diverse set of tasks. Our strong in-distribution results show the feasibility of GASS models, and the competitive out-of-distribution performance in sound event and speech separation shows its generalization abilities. Yet, it is challenging for GASS models to generalize for separating out-of-distribution cinematic and music content. We also fine-tune GASS models on each dataset and consistently outperform the ones without pre-training. All fine-tuned models (except the music separation one) obtain state-of-the-art results in their respective benchmarks.


title: The Gift of Feedback: Improving ASR Model Quality by Learning from User Corrections through Federated Learning

abstract: Automatic speech recognition (ASR) models are typically trained on large datasets of transcribed speech. As language evolves and new terms come into use, these models can become outdated and stale. In the context of models trained on the server but deployed on edge devices, errors may result from the mismatch between server training data and actual on-device usage. In this work, we seek to continually learn from on-device user corrections through Federated Learning (FL) to address this issue. We explore techniques to target fresh terms that the model has not previously encountered, learn long-tail words, and mitigate catastrophic forgetting. In experimental evaluations, we find that the proposed techniques improve model recognition of fresh terms, while preserving quality on the overall language distribution.


title: Aerial Interaction with Tactile Sensing

abstract: While autonomous Uncrewed Aerial Vehicles (UAVs) have grown rapidly, most applications only focus on passive visual tasks. Aerial interaction aims to execute tasks involving physical interactions, which offers a way to assist humans in high-risk, high-altitude operations, thereby reducing cost, time, and potential hazards. The coupled dynamics between the aerial vehicle and manipulator, however, pose challenges for precision control. Previous research has typically employed either position control, which often fails to meet mission accuracy, or force control using expensive, heavy, and cumbersome force/torque sensors that also lack local semantic information. Conversely, tactile sensors, being both cost-effective and lightweight, are capable of sensing contact information including force distribution, as well as recognizing local textures. Existing work on tactile sensing mainly focuses on tabletop manipulation tasks within a quasi-static process. In this paper, we pioneer the use of vision-based tactile sensors on a fully-actuated UAV to improve the accuracy of the more dynamic aerial manipulation tasks. We introduce a pipeline utilizing tactile feedback for real-time force tracking via a hybrid motion-force controller and a method for wall texture detection during aerial interactions. Our experiments demonstrate that our system can effectively replace or complement traditional force/torque sensors, improving flight performance by approximately 16% in position tracking error when using the fused force estimate compared to relying on a single sensor. Our tactile sensor achieves 93.4% accuracy in real-time texture recognition and 100% post-contact. To the best of our knowledge, this is the first work to incorporate a vision-based tactile sensor into aerial interaction tasks.


title: Static solutions to symplectic curvature flow in dimension four

abstract: This article studies special solutions to symplectic curvature flow in dimension four. Firstly, we derive a local normal form for static solutions in terms of holomorphic data and use this normal form to show that every complete static solution to symplectic curvature flow in dimension four is Kahler-Einstein. Secondly, we perform an exterior differential systems analysis of the soliton equation for symplectic curvature flow and use the Cartan-Kahler theorem to prove a local existence and generality theorem for solitons.


title: Probabilistic Sampling-Enhanced Temporal-Spatial GCN: A Scalable Framework for Transaction Anomaly Detection in Ethereum Networks

abstract: The rapid evolution of the Ethereum network necessitates sophisticated techniques to ensure its robustness against potential threats and to maintain transparency. While Graph Neural Networks (GNNs) have pioneered anomaly detection in such platforms, capturing the intricacies of both spatial and temporal transactional patterns has remained a challenge. This study presents a fusion of Graph Convolutional Networks (GCNs) with Temporal Random Walks (TRW) enhanced by probabilistic sampling to bridge this gap. Our approach, unlike traditional GCNs, leverages the strengths of TRW to discern complex temporal sequences in Ethereum transactions, thereby providing a more nuanced transaction anomaly detection mechanism. Preliminary evaluations demonstrate that our TRW-GCN framework substantially advances the performance metrics over conventional GCNs in detecting anomalies and transaction bursts. This research not only underscores the potential of temporal cues in Ethereum transactional data but also offers a scalable and effective methodology for ensuring the security and transparency of decentralized platforms. By harnessing both spatial relationships and time-based transactional sequences as node features, our model introduces an additional layer of granularity, making the detection process more robust and less prone to false positives. This work lays the foundation for future research aimed at optimizing and enhancing the transparency of blockchain technologies, and serves as a testament to the significance of considering both time and space dimensions in the ever-evolving landscape of the decentralized platforms.


title: 3D Reconstruction in Noisy Agricultural Environments: A Bayesian Optimization Perspective for View Planning

abstract: 3D reconstruction is a fundamental task in robotics that gained attention due to its major impact in a wide variety of practical settings, including agriculture, underwater, and urban environments. This task can be carried out via view planning (VP), which aims to optimally place a certain number of cameras in positions that maximize the visual information, improving the resulting 3D reconstruction. Nonetheless, in most real-world settings, existing environmental noise can significantly affect the performance of 3D reconstruction. To that end, this work advocates a novel geometric-based reconstruction quality function for VP, that accounts for the existing noise of the environment, without requiring its closed-form expression. With no analytic expression of the objective function, this work puts forth an adaptive Bayesian optimization algorithm for accurate 3D reconstruction in the presence of noise. Numerical tests on noisy agricultural environments showcase the merits of the proposed approach for 3D reconstruction with even a small number of available cameras.


title: Diver Identification Using Anthropometric Data Ratios for Underwater Multi-Human-Robot Collaboration

abstract: Recent advances in efficient design, perception algorithms, and computing hardware have made it possible to create improved human-robot interaction (HRI) capabilities for autonomous underwater vehicles (AUVs). To conduct secure missions as underwater human-robot teams, AUVs require the ability to accurately identify divers. However, this remains an open problem due to divers' challenging visual features, mainly caused by similar-looking scuba gear. In this paper, we present a novel algorithm that can perform diver identification using either pre-trained models or models trained during deployment. We exploit anthropometric data obtained from diver pose estimates to generate robust features that are invariant to changes in distance and photometric conditions. We also propose an embedding network that maximizes inter-class distances in the feature space and minimizes those for the intra-class features, which significantly improves classification performance. Furthermore, we present an end-to-end diver identification framework that operates on an AUV and evaluate the accuracy of the proposed algorithm. Quantitative results in controlled-water experiments show that our algorithm achieves a high level of accuracy in diver identification.


title: Beyond-DFT $\textit{ab initio}$ Calculations for Accurate Prediction of Sub-GeV Dark Matter Experimental Reach

abstract: As the search space for light dark matter (DM) has shifted to sub-GeV DM candidate particles, increasing attention has turned to solid state detectors built from quantum materials. While traditional solid state detector targets (e.g. Si or Ge) have been utilized in searches for dark matter (DM) for decades, more complex, anisotropic materials with narrow band gaps are desirable for detecting sub-MeV dark matter through DM-electron scattering and absorption channels. In order to determine if a novel target material can expand the search space for light DM it is necessary to determine the projected reach of a dark matter search conducted with that material in the DM mass - DM-electron scattering cross-section parameter space. The DM-electron scattering rate can be calculated from first-principles with knowledge of the loss function, however the accuracy of these predictions is limited by the first-principles level of theory used to calculate the dielectric function. Here we perform a case study on silicon, a well-studied semiconducting material, to demonstrate that traditional Kohn-Sham density functional theory (DFT) calculations erroneously overestimate projected experimental reach. We show that for silicon this can be remedied by the incorporation of self-energy corrections as implemented in the GW approximation. Moreover, we emphasize the care that must taken in selecting the appropriate level of theory for predicting experimental reach of next-generation complex DM detector materials.


title: First VLTI/GRAVITY Observations of HIP 65426 b: Evidence for a Low or Moderate Orbital Eccentricity

abstract: Giant exoplanets have been directly imaged over orders of magnitude of orbital separations, prompting theoretical and observational investigations of their formation pathways. In this paper, we present new VLTI/GRAVITY astrometric data of HIP 65426 b, a cold, giant exoplanet which is a particular challenge for most formation theories at a projected separation of 92 au from its primary. Leveraging GRAVITY's astrometric precision, we present an updated eccentricity posterior that disfavors large eccentricities. The eccentricity posterior is still prior-dependent, and we extensively interpret and discuss the limits of the posterior constraints presented here. We also perform updated spectral comparisons with self-consistent forward-modeled spectra, finding a best fit ExoREM model with solar metallicity and C/O=0.6. An important caveat is that it is difficult to estimate robust errors on these values, which are subject to interpolation errors as well as potentially missing model physics. Taken together, the orbital and atmospheric constraints paint a preliminary picture of formation inconsistent with scattering after disk dispersal. Further work is needed to validate this interpretation. Analysis code used to perform this work is available at https://github.com/sblunt/hip65426.


title: One for All: Towards Training One Graph Model for All Classification Tasks

abstract: Designing a single model to address multiple tasks has been a long-standing objective in artificial intelligence. Recently, large language models have demonstrated exceptional capability in solving different tasks within the language domain. However, a unified model for various graph tasks remains underexplored, primarily due to the challenges unique to the graph learning domain. First, graph data from different areas carry distinct attributes and follow different distributions. Such discrepancy makes it hard to represent graphs in a single representation space. Second, tasks on graphs diversify into node, link, and graph tasks, requiring distinct embedding strategies. Finally, an appropriate graph prompting paradigm for in-context learning is unclear. We propose \textbf{One for All (OFA)}, the first general framework that can use a single graph model to address the above challenges. Specifically, OFA proposes text-attributed graphs to unify different graph data by describing nodes and edges with natural language and uses language models to encode the diverse and possibly cross-domain text attributes to feature vectors in the same embedding space. Furthermore, OFA introduces the concept of nodes-of-interest to standardize different tasks with a single task representation. For in-context learning on graphs, OFA introduces a novel graph prompting paradigm that appends prompting substructures to the input graph, which enables it to address varied tasks without fine-tuning. We train the OFA model using graph data from multiple domains (including citation networks, molecular graphs, knowledge graphs, etc.) simultaneously and evaluate its ability in supervised, few-shot, and zero-shot learning scenarios. OFA performs well across different tasks, making it the first general-purpose across-domains classification model on graphs.


title: Zero temperature phase transitions and their anomalous influence on thermodynamic behavior in the q-state Potts model on a diamond chain

abstract: The q-state Potts model on a diamond chain has mathematical significance in analyzing phase transitions and critical behaviors in diverse fields, including statistical physics, condensed matter physics, and materials science. By focusing on the 3-state Potts model on a diamond chain, we reveal rich and analytically solvable behaviors without phase transitions at finite temperatures. Upon investigating thermodynamic properties such as internal energy, entropy, specific heat, and correlation length, we observe sharp changes near zero temperature. Magnetic properties, including magnetization and magnetic susceptibility, display distinct behaviors that provide insights into spin configurations in different phases. However, the Potts model lacks genuine phase transitions at finite temperatures, in line with the Peierls argument for one-dimensional systems. Nonetheless, in the general case of an arbitrary $q$-state, magnetic properties such as correlation length, magnetization, and magnetic susceptibility exhibit intriguing remnants of a zero-temperature phase transition at finite temperatures. Furthermore, residual entropy uncovers unusual frustrated regions at zero-temperature phase transitions. This feature leads to the peculiar thermodynamic properties of phase boundaries, including a sharp entropy change resembling a first-order discontinuity without an entropy jump, and pronounced peaks in second-order derivatives of free energy, suggestive of a second-order phase transition divergence, but without singularities. This unusual behavior is also observed in the correlation length at the pseudo-critical temperature, which could potentially be misleading as a divergence.


title: Overview of Use Cases in Single Channel Full Duplex Techniques for Satellite Communication

abstract: This paper provides an overview of the diverse range of applications and use cases for Single-Channel Full-Duplex (SCFD) techniques within the field of satellite communication. SCFD, allowing simultaneous transmission and reception on a single frequency channel, presents a transformative approach to enhancing satellite communication systems. We select eight potential use cases with the objective of highlighting the substantial potential of SCFD techniques in revolutionizing SatCom across a multitude of critical domains. In addition, preliminary results from the qualitative assessment are shown. This work is carried out within the European Space Agency (ESA) ongoing activity FDSAT: Single Channel Full Duplex Techniques for Satellite Communications.


title: Learning to Rewrite Prompts for Personalized Text Generation

abstract: Facilitated by large language models (LLMs), personalized text generation has become a rapidly growing research direction. Most existing studies focus on designing specialized models for a particular domain, or they require fine-tuning the LLMs to generate personalized text. We consider a typical scenario in which the large language model, which generates personalized output, is frozen and can only be accessed through APIs. Under this constraint, all one can do is to improve the input text (i.e., text prompts) sent to the LLM, a procedure that is usually done manually. In this paper, we propose a novel method to automatically revise prompts for personalized text generation. The proposed method takes the initial prompts generated by a state-of-the-art, multistage framework for personalized generation and rewrites a few critical components that summarize and synthesize the personal context. The prompt rewriter employs a training paradigm that chains together supervised learning (SL) and reinforcement learning (RL), where SL reduces the search space of RL and RL facilitates end-to-end training of the rewriter. Using datasets from three representative domains, we demonstrate that the rewritten prompts outperform both the original prompts and the prompts optimized via supervised learning or reinforcement learning alone. In-depth analysis of the rewritten prompts shows that they are not only human readable, but also able to guide manual revision of prompts when there is limited resource to employ reinforcement learning to train the prompt rewriter, or when it is costly to deploy an automatic prompt rewriter for inference.


title: Conformal Metamaterials with Active Tunability and Self-adaptivity for Magnetic Resonance Imaging

abstract: Ongoing effort has been devoted to applying metamaterials to boost the imaging performance of magnetic resonance imaging owing to their unique capacity for electromagnetic field confinement and enhancement. However, there are still major obstacles to widespread clinical adoption of conventional metamaterials due to several notable restrictions, namely: their typically bulky and rigid structures, deviations in their optimal resonance frequency, and their inevitable interference with the transmission RF field in MRI. Herein, we address these restrictions and report a conformal, smart metamaterial, which may not only be readily tuned to achieve the desired, precise frequency match with MRI by a controlling circuit, but is also capable of selectively amplifying the magnetic field during the RF reception phase by sensing the excitation signal strength passively, thereby remaining off during the RF transmission phase and thereby ensuring its optimal performance when applied to MRI as an additive technology. By addressing a host of current technological challenges, the metamaterial presented herein paves the way toward the wide-ranging utilization of metamaterials in clinical MRI, thereby translating this promising technology to the MRI bedside.


title: Primal Dual Continual Learning: Balancing Stability and Plasticity through Adaptive Memory Allocation

abstract: Continual learning is inherently a constrained learning problem. The goal is to learn a predictor under a no-forgetting requirement. Although several prior studies formulate it as such, they do not solve the constrained problem explicitly. In this work, we show that it is both possible and beneficial to undertake the constrained optimization problem directly. To do this, we leverage recent results in constrained learning through Lagrangian duality. We focus on memory-based methods, where a small subset of samples from previous tasks can be stored in a replay buffer. In this setting, we analyze two versions of the continual learning problem: a coarse approach with constraints at the task level and a fine approach with constraints at the sample level. We show that dual variables indicate the sensitivity of the optimal value of the continual learning problem with respect to constraint perturbations. We then leverage this result to partition the buffer in the coarse approach, allocating more resources to harder tasks, and to populate the buffer in the fine approach, including only impactful samples. We derive a deviation bound on dual variables as sensitivity indicators, and empirically corroborate this result in diverse continual learning benchmarks. We also discuss the limitations of these methods with respect to the amount of memory available and the expressiveness of the parametrization.


title: The High-Energy Spectrum of the Young Planet Host V1298 Tau

abstract: V1298 Tau is a young pre-main sequence star hosting four known exoplanets that are prime targets for transmission spectroscopy with current-generation instruments. This work pieces together observations from the NICER X-ray telescope, the Space Telescope Imaging Spectrograph and Cosmic Origins Spectrograph instruments aboard Hubble Space Telescope, and empirically informed models to create a panchromatic spectral energy distribution for V1298 Tau spanning 1 to 100000 Angstroms. We describe the methods and assumptions used to assemble the panchromatic spectrum and show that despite this star's brightness, its high-energy spectrum is near the limit of present X-ray and ultraviolet observatories' abilities to characterize. We conclude by using the V1298 Tau spectrum as a benchmark for the activity saturation stage of high-energy radiation from solar-mass stars to compare the lifetime cumulative high-energy irradiation of the V1298 Tau planets to other planets orbiting similarly massive stars.


title: Learning Generalizable Tool-use Skills through Trajectory Generation

abstract: Autonomous systems that efficiently utilize tools can assist humans in completing many common tasks such as cooking and cleaning. However, current systems fall short of matching human-level of intelligence in terms of adapting to novel tools. Prior works based on affordance often make strong assumptions about the environments and cannot scale to more complex, contact-rich tasks. In this work, we tackle this challenge and explore how agents can learn to use previously unseen tools to manipulate deformable objects. We propose to learn a generative model of the tool-use trajectories as a sequence of tool point clouds, which generalizes to different tool shapes. Given any novel tool, we first generate a tool-use trajectory and then optimize the sequence of tool poses to align with the generated trajectory. We train a single model on four different challenging deformable object manipulation tasks, using demonstration data from only one tool per task. The model generalizes to various novel tools, significantly outperforming baselines. We further test our trained policy in the real world with unseen tools, where it achieves the performance comparable to human. Additional materials can be found on our project website: https://sites.google.com/view/toolgen.


title: The poset associahedron $f$-vector is a comparability invariant

abstract: We show that the $f$-vector of Galashin's poset associahedron $\mathscr A(P)$ only depends on the comparability graph of $P$. In particular, this allows us to produce a family of polytopes with the same $f$-vectors as permutohedra, but that are not combinatorially equivalent to permutohedra.


title: Feedback-guided Data Synthesis for Imbalanced Classification

abstract: Current status quo in machine learning is to use static datasets of real images for training, which often come from long-tailed distributions. With the recent advances in generative models, researchers have started augmenting these static datasets with synthetic data, reporting moderate performance improvements on classification tasks. We hypothesize that these performance gains are limited by the lack of feedback from the classifier to the generative model, which would promote the usefulness of the generated samples to improve the classifier's performance. In this work, we introduce a framework for augmenting static datasets with useful synthetic samples, which leverages one-shot feedback from the classifier to drive the sampling of the generative model. In order for the framework to be effective, we find that the samples must be close to the support of the real data of the task at hand, and be sufficiently diverse. We validate three feedback criteria on a long-tailed dataset (ImageNet-LT) as well as a group-imbalanced dataset (NICO++). On ImageNet-LT, we achieve state-of-the-art results, with over 4 percent improvement on underrepresented classes while being twice efficient in terms of the number of generated synthetic samples. NICO++ also enjoys marked boosts of over 5 percent in worst group accuracy. With these results, our framework paves the path towards effectively leveraging state-of-the-art text-to-image models as data sources that can be queried to improve downstream applications.


title: P\'olya urns on hypergraphs

abstract: We study P\'olya urns on hypergraphs and prove that, when the incidence matrix of the hypergraph is injective, there exists a point $v=v(H)$ such that the random process converges to $v$ almost surely. We also provide a partial result when the incidence matrix is not injective.


title: Self-Specialization: Uncovering Latent Expertise within Large Language Models

abstract: Recent works have demonstrated the effectiveness of self-alignment in which a large language model is aligned to follow general instructions using instructional data generated from the model itself starting from a handful of human-written seeds. Instead of general alignment, in this work, we focus on self-alignment for expert domain specialization (e.g., biomedicine, finance). As a preliminary, we quantitively show the marginal effect that generic instruction-following training has on downstream expert domains' performance. To remedy this, we propose self-specialization - allowing for effective model specialization while achieving cross-task generalization by leveraging only a few labeled seeds. Self-specialization offers a data- and parameter-efficient way of "carving out" an expert model out of a generalist pre-trained LLM. Exploring a variety of popular open large models as a base for specialization, our experimental results in both biomedical and financial domains show that our self-specialized models outperform their base models by a large margin, and even larger models that are generally instruction-tuned or that have been adapted to the target domain by other means.


title: Detection-Oriented Image-Text Pretraining for Open-Vocabulary Detection

abstract: We present a new open-vocabulary detection approach based on detection-oriented image-text pretraining to bridge the gap between image-level pretraining and open-vocabulary object detection. At the pretraining phase, we replace the commonly used classification architecture with the detector architecture, which better serves the region-level recognition needs of detection by enabling the detector heads to learn from noisy image-text pairs. Using only standard contrastive loss and no pseudo-labeling, our approach is a simple yet effective extension of the contrastive learning method to learn emergent object-semantic cues. In addition, we propose a shifted-window learning approach upon window attention to make the backbone representation more robust, translation-invariant, and less biased by the window pattern. On the popular LVIS open-vocabulary detection benchmark, our approach sets a new state of the art of 40.4 mask AP$_r$ using the common ViT-L backbone, significantly outperforming the best existing approach by +6.5 mask AP$_r$ at system level. On the COCO benchmark, we achieve very competitive 40.8 novel AP without pseudo labeling or weak supervision. In addition, we evaluate our approach on the transfer detection setup, where ours outperforms the baseline significantly. Visualization reveals emerging object locality from the pretraining recipes compared to the baseline. Code and models will be publicly released.


title: Evidence of weak circumstellar medium interaction in the Type II SN 2023axu

abstract: We present high-cadence photometric and spectroscopic observations of SN~2023axu, a classical Type II supernova with an absolute $V$-band peak magnitude of $-16.5 \pm 0.1$ mag. SN~2023axu was discovered by the Distance Less Than 40 Mpc (DLT40) survey within 1 day of the last non-detection in the nearby galaxy NGC 2283 at 13.7 Mpc. We modeled the early light curve using a recently updated shock cooling model that includes the effects of line blanketing and found the explosion epoch to be MJD 59971.48 $\pm$ 0.03 and the probable progenitor to be a red supergiant with a radius of 417 $\pm$ 28 $R_\odot$. The shock cooling model cannot match the rise of observed data in the $r$ and $i$ bands and underpredicts the overall UV data which points to possible interaction with circumstellar material. This interpretation is further supported by spectral behavior. We see a ledge feature around 4600 \AA\ in the very early spectra (+1.1 and +1.5 days after the explosion) which can be a sign of circumstellar interaction. The signs of circumstellar material are further bolstered by the presence of absorption features blueward of H$\alpha$ and H$\beta$ at day $>$40 which is also generally attributed to circumstellar interaction. Our analysis shows the need for high-cadence early photometric and spectroscopic data to decipher the mass-loss history of the progenitor.


title: Cook2LTL: Translating Cooking Recipes to LTL Formulae using Large Language Models

abstract: Cooking recipes are challenging to translate to robot plans as they feature rich linguistic complexity, temporally-extended interconnected tasks, and an almost infinite space of possible actions. Our key insight is that combining a source of cooking domain knowledge with a formalism that captures the temporal richness of cooking recipes could enable the extraction of unambiguous, robot-executable plans. In this work, we use Linear Temporal Logic (LTL) as a formal language expressive enough to model the temporal nature of cooking recipes. Leveraging a pretrained Large Language Model (LLM), we present Cook2LTL, a system that translates instruction steps from an arbitrary cooking recipe found on the internet to a set of LTL formulae, grounding high-level cooking actions to a set of primitive actions that are executable by a manipulator in a kitchen environment. Cook2LTL makes use of a caching scheme that dynamically builds a queryable action library at runtime. We instantiate Cook2LTL in a realistic simulation environment (AI2-THOR), and evaluate its performance across a series of cooking recipes. We demonstrate that our system significantly decreases LLM API calls (-51%), latency (-59%), and cost (-42%) compared to a baseline that queries the LLM for every newly encountered action at runtime.


title: PRIME: Prioritizing Interpretability in Failure Mode Extraction

abstract: In this work, we study the challenge of providing human-understandable descriptions for failure modes in trained image classification models. Existing works address this problem by first identifying clusters (or directions) of incorrectly classified samples in a latent space and then aiming to provide human-understandable text descriptions for them. We observe that in some cases, describing text does not match well with identified failure modes, partially owing to the fact that shared interpretable attributes of failure modes may not be captured using clustering in the feature space. To improve on these shortcomings, we propose a novel approach that prioritizes interpretability in this problem: we start by obtaining human-understandable concepts (tags) of images in the dataset and then analyze the model's behavior based on the presence or absence of combinations of these tags. Our method also ensures that the tags describing a failure mode form a minimal set, avoiding redundant and noisy descriptions. Through several experiments on different datasets, we show that our method successfully identifies failure modes and generates high-quality text descriptions associated with them. These results highlight the importance of prioritizing interpretability in understanding model failures.


title: SCoRe: Submodular Combinatorial Representation Learning

abstract: In this paper we introduce the SCoRe (Submodular Combinatorial Representation Learning) framework, a novel approach in representation learning that addresses inter-class bias and intra-class variance. SCoRe provides a new combinatorial viewpoint to representation learning, by introducing a family of loss functions based on set-based submodular information measures. We develop two novel combinatorial formulations for loss functions, using the Total Information and Total Correlation, that naturally minimize intra-class variance and inter-class bias. Several commonly used metric/contrastive learning loss functions like supervised contrastive loss, orthogonal projection loss, and N-pairs loss, are all instances of SCoRe, thereby underlining the versatility and applicability of SCoRe in a broad spectrum of learning scenarios. Novel objectives in SCoRe naturally model class-imbalance with up to 7.6\% improvement in classification on CIFAR-10-LT, CIFAR-100-LT, MedMNIST, 2.1% on ImageNet-LT, and 19.4% in object detection on IDD and LVIS (v1.0), demonstrating its effectiveness over existing approaches.