title: Understanding Server-Assisted Federated Learning in the Presence of Incomplete Client Participation

abstract: Existing works in federated learning (FL) often assume an ideal system with either full client or uniformly distributed client participation. However, in practice, it has been observed that some clients may never participate in FL training (aka incomplete client participation) due to a myriad of system heterogeneity factors. A popular approach to mitigate impacts of incomplete client participation is the server-assisted federated learning (SA-FL) framework, where the server is equipped with an auxiliary dataset. However, despite SA-FL has been empirically shown to be effective in addressing the incomplete client participation problem, there remains a lack of theoretical understanding for SA-FL. Meanwhile, the ramifications of incomplete client participation in conventional FL are also poorly understood. These theoretical gaps motivate us to rigorously investigate SA-FL. Toward this end, we first show that conventional FL is {\em not} PAC-learnable under incomplete client participation in the worst case. Then, we show that the PAC-learnability of FL with incomplete client participation can indeed be revived by SA-FL, which theoretically justifies the use of SA-FL for the first time. Lastly, to provide practical guidance for SA-FL training under {\em incomplete client participation}, we propose the $\mathsf{SAFARI}$ (server-assisted federated averaging) algorithm that enjoys the same linear convergence speedup guarantees as classic FL with ideal client participation assumptions, offering the first SA-FL algorithm with convergence guarantee. Extensive experiments on different datasets show $\mathsf{SAFARI}$ significantly improves the performance under incomplete client participation.


title: Sharp Spectral Projection Estimates for the Torus at $p_c=\frac{2(n+1)}{n-1}$

abstract: We prove sharp spectral projection estimates for tori in all dimensions at the exponent $p_c=\frac{2(n+1)}{n-1}$ for shrinking windows of width $1$ down to windows of length $\lambda^{-1+\kappa}$ for fixed $\kappa>0$. This improves and slightly generalizes the work of Blair-Huang-Sogge who proved sharp results for windows of width $\lambda^{-\frac{1}{n+3}}$, and the work of Hickman, Germain-Myerson, and Demeter-Germain who proved results for windows of all widths but incurred a sub-polynomial loss. Our work uses the approaches of these two groups of authors, combining the bilinear decomposition and microlocal techniques of Blair-Huang-Sogge with the decoupling theory and explicit lattice point lemmas used by Hickman, Germain-Myerson, and Demeter-Germain to remove these losses.


title: Wobbling motion in triaxial nuclei

abstract: The experimental evidence for the collective wobbling motion of triaxial nuclei is reviewed. The classification into transverse and longitudinal in the presence of quasiparticle excitations is discussed. The description by means of the quasiparticle+triaxial rotor model is discussed in detail. The structure of the states is analyzed using the spin-coherent-state and spin-squeezed-state representations of the reduced density matrices of the total and particle angular momenta, which distill the corresponding classical precessional motions. Various approximate solutions of the quasiparticle+triaxial rotor model are evaluated. The microscopic studies of wobbling in the small-amplitude random phase approximation are discussed. Selected studies of wobbling by means of the triaxial projected shell model are presented, which focus on how this microscopic approach removes certain deficiencies of the semi-microscopic quasiparticle+triaxial rotor model.


title: A Bayesian mixture model approach to quantifying the empirical nuclear saturation point

abstract: The equation of state (EOS) in the limit of infinite symmetric nuclear matter exhibits an equilibrium density, $n_0 \approx 0.16 \, \mathrm{fm}^{-3}$, at which the pressure vanishes and the energy per particle attains its minimum, $E_0 \approx -16 \, \mathrm{MeV}$. Although not directly measurable, the nuclear saturation point $(n_0,E_0)$ can be extrapolated by density functional theory (DFT), providing tight constraints for microscopic interactions derived from chiral effective field theory (EFT). However, when considering several DFT predictions for $(n_0,E_0)$ together, a discrepancy between model classes emerges at high confidence levels that each model prediction's uncertainty cannot explain. How can we leverage these DFT constraints to rigorously benchmark nuclear saturation properties of chiral interactions? To address this question, we present a Bayesian mixture model that combines multiple DFT predictions for $(n_0,E_0)$ using an efficient conjugate prior approach. The inferred posterior distribution for the saturation point's mean and covariance matrix follows a Normal-inverse-Wishart class, resulting in posterior predictives in the form of correlated, bivariate $t$-distributions. The DFT uncertainty reports are then used to mix these posteriors using an ordinary Monte Carlo approach. At the 95% credibility level, we estimate $n_0 \approx 0.157 \pm 0.010 \, \mathrm{fm}^{-3}$ and $E_0 \approx -15.97 \pm 0.40 \, \mathrm{MeV}$ for the marginal (univariate) $t$-distributions. Combined with chiral EFT calculations of the pure neutron matter EOS, we obtain bivariate normal distributions for the nuclear symmetry energy and its slope parameter evaluated at $n_0$: $S_v \approx 32.0 \pm 1.1 \, \mathrm{MeV}$ and $L\approx 52.6\pm 8.1 \, \mathrm{MeV}$ (95%), respectively. Our Bayesian framework is publicly available, so practitioners can readily use and extend our results.


title: Sub-goal Distillation: A Method to Improve Small Language Agents

abstract: While Large Language Models (LLMs) have demonstrated significant promise as agents in interactive tasks, their substantial computational requirements and restricted number of calls constrain their practical utility, especially in long-horizon interactive tasks such as decision-making or in scenarios involving continuous ongoing tasks. To address these constraints, we propose a method for transferring the performance of an LLM with billions of parameters to a much smaller language model (770M parameters). Our approach involves constructing a hierarchical agent comprising a planning module, which learns through Knowledge Distillation from an LLM to generate sub-goals, and an execution module, which learns to accomplish these sub-goals using elementary actions. In detail, we leverage an LLM to annotate an oracle path with a sequence of sub-goals towards completing a goal. Subsequently, we utilize this annotated data to fine-tune both the planning and execution modules. Importantly, neither module relies on real-time access to an LLM during inference, significantly reducing the overall cost associated with LLM interactions to a fixed cost. In ScienceWorld, a challenging and multi-task interactive text environment, our method surpasses standard imitation learning based solely on elementary actions by 16.7% (absolute). Our analysis highlights the efficiency of our approach compared to other LLM-based methods. Our code and annotated data for distillation can be found on GitHub.


title: Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding

abstract: Large language models (LLMs) tend to inadequately integrate input context during text generation, relying excessively on encoded prior knowledge in model parameters, potentially resulting in generated text with factual inconsistencies or contextually unfaithful content. LLMs utilize two primary knowledge sources: 1) prior (parametric) knowledge from pretraining, and 2) contextual (non-parametric) knowledge from input prompts. The study addresses the open question of how LLMs effectively balance these knowledge sources during the generation process, specifically in the context of open-domain question answering. To address this issue, we introduce a novel approach integrating contrastive decoding with adversarial irrelevant passages as negative samples to enhance robust context grounding during generation. Notably, our method operates at inference time without requiring further training. We conduct comprehensive experiments to demonstrate its applicability and effectiveness, providing empirical evidence showcasing its superiority over existing methodologies. Our code is publicly available at: https://github.com/amazon-science/ContextualUnderstanding-ContrastiveDecoding.


title: Deep Image Restoration For Image Anti-Forensics

abstract: While image forensics is concerned with whether an image has been tampered with, image anti-forensics attempts to prevent image forensics methods from detecting tampered images. The competition between these two fields started long before the advancement of deep learning. JPEG compression, blurring and noising, which are simple methods by today's standards, have long been used for anti-forensics and have been the subject of much research in both forensics and anti-forensics. Although these traditional methods are old, they make it difficult to detect fake images and are used for data augmentation in training deep image forgery detection models. In addition to making the image difficult to detect, these methods leave traces on the image and consequently degrade the image quality. Separate image forensics methods have also been developed to detect these traces. In this study, we go one step further and improve the image quality after these methods with deep image restoration models and make it harder to detect the forged image. We evaluate the impact of these methods on image quality. We then test both our proposed methods with deep learning and methods without deep learning on the two best existing image manipulation detection models. In the obtained results, we show how existing image forgery detection models fail against the proposed methods. Code implementation will be publicly available at https://github.com/99eren99/DIRFIAF .


title: Weak Supercyclicity - An Expository Survey

abstract: This is an exposition on supercyclicity and weak supercyclicity, especially designed to advance further developments in weakly supercyclicity, which is a recent research field showing significant momentum during the past two decades. For operators on a normed space, the present paper explores the relationship among supercyclicity and three versions of weak supercyclicity, namely, weak l-sequential, weak sequential, and weak supercyclicities, and their connection with strong stability, weak stability, and weak quasistability. A survey of the literature on weak supercyclicity is followed by an analysis of the interplay between supercyclicity and strong stability, as well as between weak supercyclicity and weak stability. A description of the spectrum of weakly l-sequentially supercyclic operators is also given.


title: Unscented Trajectory Optimization

abstract: In a nutshell, unscented trajectory optimization is the generation of optimal trajectories through the use of an unscented transform. Although unscented trajectory optimization was introduced by the authors about a decade ago, it is reintroduced in this paper as a special instantiation of tychastic optimal control theory. Tychastic optimal control theory (from \textit{Tyche}, the Greek goddess of chance) avoids the use of a Brownian motion and the resulting It\^{o} calculus even though it uses random variables across the entire spectrum of a problem formulation. This approach circumvents the enormous technical and numerical challenges associated with stochastic trajectory optimization. Furthermore, it is shown how a tychastic optimal control problem that involves nonlinear transformations of the expectation operator can be quickly instantiated using an unscented transform. These nonlinear transformations are particularly useful in managing trajectory dispersions be it associated with path constraints or targeted values of final-time conditions. This paper also presents a systematic and rapid process for formulating and computing the most desirable tychastic trajectory using an unscented transform. Numerical examples are used to illustrate how unscented trajectory optimization may be used for risk reduction and mission recovery caused by uncertainties and failures.

categories: math.OC cs.SY eess.SY math.ST stat.CO stat.TH


title: Implicit Safe Set Algorithm for Provably Safe Reinforcement Learning

abstract: Deep reinforcement learning (DRL) has demonstrated remarkable performance in many continuous control tasks. However, a significant obstacle to the real-world application of DRL is the lack of safety guarantees. Although DRL agents can satisfy system safety in expectation through reward shaping, designing agents to consistently meet hard constraints (e.g., safety specifications) at every time step remains a formidable challenge. In contrast, existing work in the field of safe control provides guarantees on persistent satisfaction of hard safety constraints. However, these methods require explicit analytical system dynamics models to synthesize safe control, which are typically inaccessible in DRL settings. In this paper, we present a model-free safe control algorithm, the implicit safe set algorithm, for synthesizing safeguards for DRL agents that ensure provable safety throughout training. The proposed algorithm synthesizes a safety index (barrier certificate) and a subsequent safe control law solely by querying a black-box dynamic function (e.g., a digital twin simulator). Moreover, we theoretically prove that the implicit safe set algorithm guarantees finite time convergence to the safe set and forward invariance for both continuous-time and discrete-time systems. We validate the proposed algorithm on the state-of-the-art Safety Gym benchmark, where it achieves zero safety violations while gaining $95\% \pm 9\%$ cumulative reward compared to state-of-the-art safe DRL methods. Furthermore, the resulting algorithm scales well to high-dimensional systems with parallel computing.


title: On the Impact of Dark Matter Scattering on the Trajectory of High-Energy Cosmic Rays

abstract: We study the impact on the trajectory of high-energy cosmic-ray protons of scattering off the cosmic dark matter. We compute the scattering angle as a function of the cosmic-ray energy, of the dark matter mass, and of the interaction strength for a few representative choices for the relevant interaction cross section. We find that the typical deflection angle over the cosmic ray path is largely independent of the dark matter mass. Given existing limits on the interaction strength, we compute the average deflection angle. We find that for large interaction cross sections and low cosmic ray energies, the predicted deflection angle is much larger than the angular resolution of very high-energy cosmic-ray observatories such as Pierre Auger.


title: Efficient Open Modification Spectral Library Searching in High-Dimensional Space with Multi-Level-Cell Memory

abstract: Open Modification Search (OMS) is a promising algorithm for mass spectrometry analysis that enables the discovery of modified peptides. However, OMS encounters challenges as it exponentially extends the search scope. Existing OMS accelerators either have limited parallelism or struggle to scale effectively with growing data volumes. In this work, we introduce an OMS accelerator utilizing multi-level-cell (MLC) RRAM memory to enhance storage capacity by 3x. Through in-memory computing, we achieve up to 77x faster data processing with two to three orders of magnitude better energy efficiency. Testing was done on a fabricated MLC RRAM chip. We leverage hyperdimensional computing to tolerate up to 10% memory errors while delivering massive parallelism in hardware.


title: A Kinetic Model of Solar Wind Generation and Heating by Kinetic Alfv\'en Wave Turbulence

abstract: We present results from a kinetic model of collisionless gyrotropic coronal hole protons heated by cyclotron and Landau resonant dissipation of critically balanced kinetic Alfv\'en waves. The model incorporates the kinetic effects of gravity, ambipolar electric field, ponderomotive force of the large-scale Alfv\'en waves and the mirror force in a super-radially expanding flux tube. The flow speed is self- consistently obtained as the bulk flow of the proton distribution. Two cases, taking the intensities of the turbulent spectra to be balanced in the parallel propagation direction or imbalanced at a ratio of 9:1 show almost no difference. The distributions develop parallel extensions outward due to the Landau resonance, but exhibit very little perpendicular heating under the parameter choices used here. The speeds and temperatures fall well short of the requirements for a fast solar wind. Rather than continuing to explore the parameter space of this system, we propose that a modified model using turbulent spectra based on the recent helicity barrier scenario would be an appropriate next step.


title: Size of isogeny classes of abelian varieties in the extended Lubin-Tate case

abstract: We prove a lower bound for the size of the isogeny class of a simple abelian variety over a finite field with commutative endomorphism ring in the extended Lubin-Tate case. Moreover, based on the expected size of the isogeny classes in the Newton stratum, we conjecture that this lower bound is sharp.


title: Region-Aware Color Smudging

abstract: Color smudge operations from digital painting software enable users to create natural shading effects in high-fidelity paintings by interactively mixing colors. To precisely control results in traditional painting software, users tend to organize flat-filled color regions in multiple layers and smudge them to generate different color gradients. However, the requirement to carefully deal with regions makes the smudging process time-consuming and laborious, especially for non-professional users. This motivates us to investigate how to infer user-desired smudging effects when users smudge over regions in a single layer. To investigate improving color smudge performance, we first conduct a formative study. Following the findings of this study, we design SmartSmudge, a novel smudge tool that offers users dynamical smudge brushes and real-time region selection for easily generating natural and efficient shading effects. We demonstrate the efficiency and effectiveness of the proposed tool via a user study and quantitative analysis


title: GTFS2STN: Analyzing GTFS Transit Data by Generating Spatiotemporal Transit Network

abstract: GTFS, the General Transit Feed Specialization, is an open standard format to record transit information used by thousands of transit agencies across the world. By converting a static GTFS transit network to a spatiotemporal network connecting bus stops over space and time, a preliminary tool named GTFS2STN is implemented to analyze the accessibility of the transit system. Furthermore, a simple application is built for users to generate spatiotemporal network online. The online tool also supports some basic analysis including generate isochrone maps given origin, generate travel time variability over time given a pair of origin and destination, etc. Results show that the tool has a similar result compared with Mapnificent, another open source endeavour to generate isochrone maps given GTFS inputs. Compared with Mapnificent, the proposed GTFS2STN tool is suited for research and evaluation purposes because the users can upload any historical GTFS dataset by any transit agencies to evaluate the accessibility and travel time variability of transit networks over time.


title: Mid-latitude interactions expand the Hadley circulation

abstract: The Hadley circulation describes a planetary-scale tropical atmospheric flow, which has a major influence on climate. Contemporary theoretical understanding is based upon angular momentum conservation, the basic dynamical constraint governing the state of the flow pattern. However, despite the degree of success in representing the Hadley circulation, the canonical theoretical model does not treat interactions with other regions, particularly the mid-latitudes. Here, we extend the original model of Held and Hou (1980) to include the influence of mid-latitude large-scale atmospheric dynamics, which we treat using the planetary-scale heat equation with a parameterized poleward heat flux driven by synoptic eddies. The energy flux balance within the Hadley cell includes the poleward heat flux at the poleward edge of the cell, which is controlled by the baroclinic instability of the sub-tropical jet. We find that an increase (decrease) in the poleward heat flux leads to a strengthening (weakening) of tropical convection, driving an equatorward (poleward) shift of the edge of the Hadley cell. Thus, our theoretical solutions suggest that global warming, which can reduce the baroclinicity of the subtropical jet, can lead to the poleward expansion of the Hadley cell due to the change in energy flux balance within it.


title: TK-Planes: Tiered K-Planes with High Dimensional Feature Vectors for Dynamic UAV-based Scenes

abstract: In this paper, we present a new approach to bridge the domain gap between synthetic and real-world data for un- manned aerial vehicle (UAV)-based perception. Our formu- lation is designed for dynamic scenes, consisting of moving objects or human actions, where the goal is to recognize the pose or actions. We propose an extension of K-Planes Neural Radiance Field (NeRF), wherein our algorithm stores a set of tiered feature vectors. The tiered feature vectors are generated to effectively model conceptual information about a scene as well as an image decoder that transforms output feature maps into RGB images. Our technique leverages the information amongst both static and dynamic objects within a scene and is able to capture salient scene attributes of high altitude videos. We evaluate its performance on challenging datasets, including Okutama Action and UG2, and observe considerable improvement in accuracy over state of the art aerial perception algorithms.


title: Can Nuanced Language Lead to More Actionable Insights? Exploring the Role of Generative AI in Analytical Narrative Structure

abstract: Relevant language describing trends in data can be useful for generating summaries to help with readers' takeaways. However, the language employed in these often template-generated summaries tends to be simple, ranging from describing simple statistical information (e.g., extrema and trends) without additional context and richer language to provide actionable insights. Recent advances in Large Language Models (LLMs) have shown promising capabilities in capturing subtle nuances in language when describing information. This workshop paper specifically explores how LLMs can provide more actionable insights when describing trends by focusing on three dimensions of analytical narrative structure: semantic, rhetorical, and pragmatic. Building on prior research that examines visual and linguistic signatures for univariate line charts, we examine how LLMs can further leverage the semantic dimension of analytical narratives using quantified semantics to describe shapes in trends as people intuitively view them. These semantic descriptions help convey insights in a way that leads to a pragmatic outcome, i.e., a call to action, persuasion, warning vs. alert, and situational awareness. Finally, we identify rhetorical implications for how well these generated narratives align with the perceived shape of the data, thereby empowering users to make informed decisions and take meaningful actions based on these data insights.


title: Assessing Adversarial Robustness of Large Language Models: An Empirical Study

abstract: Large Language Models (LLMs) have revolutionized natural language processing, but their robustness against adversarial attacks remains a critical concern. We presents a novel white-box style attack approach that exposes vulnerabilities in leading open-source LLMs, including Llama, OPT, and T5. We assess the impact of model size, structure, and fine-tuning strategies on their resistance to adversarial perturbations. Our comprehensive evaluation across five diverse text classification tasks establishes a new benchmark for LLM robustness. The findings of this study have far-reaching implications for the reliable deployment of LLMs in real-world applications and contribute to the advancement of trustworthy AI systems.


title: Detecting Edited Knowledge in Language Models

abstract: Knowledge editing techniques (KEs) can update language models' obsolete or inaccurate knowledge learned from pre-training. However, KE also faces potential malicious applications, e.g. inserting misinformation and toxic content. Moreover, in the context of responsible AI, it is instructive for end-users to know whether a generated output is driven by edited knowledge or first-hand knowledge from pre-training. To this end, we study detecting edited knowledge in language models by introducing a novel task: given an edited model and a specific piece of knowledge the model generates, our objective is to classify the knowledge as either "non-edited" (based on the pre-training), or ``edited'' (based on subsequent editing). We initiate the task with two state-of-the-art KEs, two language models, and two datasets. We further propose a simple classifier, RepReg, a logistic regression model that takes hidden state representations as input features. Our results reveal that RepReg establishes a strong baseline, achieving a peak accuracy of 99.81%, and 97.79% in out-of-domain settings. Second, RepReg achieves near-optimal performance with a limited training set (200 training samples), and it maintains its performance even in out-of-domain settings. Last, we find it more challenging to separate edited and non-edited knowledge when they contain the same subject or object.


title: Beyond Unimodal Learning: The Importance of Integrating Multiple Modalities for Lifelong Learning

abstract: While humans excel at continual learning (CL), deep neural networks (DNNs) exhibit catastrophic forgetting. A salient feature of the brain that allows effective CL is that it utilizes multiple modalities for learning and inference, which is underexplored in DNNs. Therefore, we study the role and interactions of multiple modalities in mitigating forgetting and introduce a benchmark for multimodal continual learning. Our findings demonstrate that leveraging multiple views and complementary information from multiple modalities enables the model to learn more accurate and robust representations. This makes the model less vulnerable to modality-specific regularities and considerably mitigates forgetting. Furthermore, we observe that individual modalities exhibit varying degrees of robustness to distribution shift. Finally, we propose a method for integrating and aligning the information from different modalities by utilizing the relational structural similarities between the data points in each modality. Our method sets a strong baseline that enables both single- and multimodal inference. Our study provides a promising case for further exploring the role of multiple modalities in enabling CL and provides a standard benchmark for future research.


title: A Decade in a Systematic Review: The Evolution and Impact of Cell Painting

abstract: High-content image-based assays have fueled significant discoveries in the life sciences in the past decade (2013-2023), including novel insights into disease etiology, mechanism of action, new therapeutics, and toxicology predictions. Here, we systematically review the substantial methodological advancements and applications of Cell Painting. Advancements include improvements in the Cell Painting protocol, assay adaptations for different types of perturbations and applications, and improved methodologies for feature extraction, quality control, and batch effect correction. Moreover, machine learning methods recently surpassed classical approaches in their ability to extract biologically useful information from Cell Painting images. Cell Painting data have been used alone or in combination with other -omics data to decipher the mechanism of action of a compound, its toxicity profile, and many other biological effects. Overall, key methodological advances have expanded the ability of Cell Painting to capture cellular responses to various perturbations. Future advances will likely lie in advancing computational and experimental techniques, developing new publicly available datasets, and integrating them with other high-content data types.


title: A short way to directed J\'onsson terms

abstract: We show that a variety with J\'onsson terms $t_1, \dots, t_{n-1}$ has directed J\'onsson terms $d_1, \dots, d_{n-1}$, for the same value of the indices. Refined results are obtained for locally finite varieties.


title: Linear Convergence of Independent Natural Policy Gradient in Games with Entropy Regularization

abstract: This work focuses on the entropy-regularized independent natural policy gradient (NPG) algorithm in multi-agent reinforcement learning. In this work, agents are assumed to have access to an oracle with exact policy evaluation and seek to maximize their respective independent rewards. Each individual's reward is assumed to depend on the actions of all the agents in the multi-agent system, leading to a game between agents. We assume all agents make decisions under a policy with bounded rationality, which is enforced by the introduction of entropy regularization. In practice, a smaller regularization implies the agents are more rational and behave closer to Nash policies. On the other hand, agents with larger regularization acts more randomly, which ensures more exploration. We show that, under sufficient entropy regularization, the dynamics of this system converge at a linear rate to the quantal response equilibrium (QRE). Although regularization assumptions prevent the QRE from approximating a Nash equilibrium, our findings apply to a wide range of games, including cooperative, potential, and two-player matrix games. We also provide extensive empirical results on multiple games (including Markov games) as a verification of our theoretical analysis.


title: PhilHumans: Benchmarking Machine Learning for Personal Health

abstract: The use of machine learning in Healthcare has the potential to improve patient outcomes as well as broaden the reach and affordability of Healthcare. The history of other application areas indicates that strong benchmarks are essential for the development of intelligent systems. We present Personal Health Interfaces Leveraging HUman-MAchine Natural interactions (PhilHumans), a holistic suite of benchmarks for machine learning across different Healthcare settings - talk therapy, diet coaching, emergency care, intensive care, obstetric sonography - as well as different learning settings, such as action anticipation, timeseries modeling, insight mining, language modeling, computer vision, reinforcement learning and program synthesis


title: MMEarth: Exploring Multi-Modal Pretext Tasks For Geospatial Representation Learning

abstract: The volume of unlabelled Earth observation (EO) data is huge, but many important applications lack labelled training data. However, EO data offers the unique opportunity to pair data from different modalities and sensors automatically based on geographic location and time, at virtually no human labor cost. We seize this opportunity to create a diverse multi-modal pretraining dataset at global scale. Using this new corpus of 1.2 million locations, we propose a Multi-Pretext Masked Autoencoder (MP-MAE) approach to learn general-purpose representations for optical satellite images. Our approach builds on the ConvNeXt V2 architecture, a fully convolutional masked autoencoder (MAE). Drawing upon a suite of multi-modal pretext tasks, we demonstrate that our MP-MAE approach outperforms both MAEs pretrained on ImageNet and MAEs pretrained on domain-specific satellite images. This is shown on several downstream tasks including image classification and semantic segmentation. We find that multi-modal pretraining notably improves the linear probing performance, e.g. 4pp on BigEarthNet and 16pp on So2Sat, compared to pretraining on optical satellite images only. We show that this also leads to better label and parameter efficiency which are crucial aspects in global scale applications.


title: SkinGrip: An Adaptive Soft Robotic Manipulator with Capacitive Sensing for Whole-Limb Bathing Assistance

abstract: Robotics presents a promising opportunity for enhancing bathing assistance, potentially to alleviate labor shortages and reduce care costs, while offering consistent and gentle care for individuals with physical disabilities. However, ensuring flexible and efficient cleaning of the human body poses challenges as it involves direct physical contact between the human and the robot, and necessitates simple, safe, and effective control. In this paper, we introduce a soft, expandable robotic manipulator with embedded capacitive proximity sensing arrays, designed for safe and efficient bathing assistance. We conduct a thorough evaluation of our soft manipulator, comparing it with a baseline rigid end effector in a human study involving 12 participants across $96$ bathing trails. Our soft manipulator achieves an an average cleaning effectiveness of 88.8% on arms and 81.4% on legs, far exceeding the performance of the baseline. Participant feedback further validates the manipulator's ability to maintain safety, comfort, and thorough cleaning.


title: Magnetar Eruptions and Electromagnetic Fireworks

abstract: Highly magnetized neutron stars are a source of extreme transients observed in different bands, like the fast radio burst (FRB) and associated hard X-ray burst from the Galactic magnetar SGR 1935+2154. The origin of such outbursts, hard X-rays on the one hand and millisecond duration FRBs on the other hand, is still unknown. We present a global model for various kinds of such magnetar outbursting activities. Crustal surface motions are expected to twist the inner magnetar magnetosphere by shifting the frozen-in footpoints of magnetic field lines. We discuss criteria for the development of instabilities of 3D twisted flux bundles in the force-free dipolar magnetospheres and compare their energetic properties to observations of magnetar X-ray flares. We then review a recently developed FRB generation mechanism in the outer magnetosphere of a magnetar. The strong magnetic pulse induced by a magnetar flare collides with the current sheet of the magnetar wind, compresses and fragments it into a self-similar chain of magnetic islands. Time-dependent plasma currents created during their collisions produce relatively narrow-band GHz emission with luminosities sufficient to explain bright extragalactic FRBs.


title: Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs

abstract: This work focuses on leveraging and selecting from vast, unlabeled, open data to pre-fine-tune a pre-trained language model. The goal is to minimize the need for costly domain-specific data for subsequent fine-tuning while achieving desired performance levels. While many data selection algorithms have been designed for small-scale applications, rendering them unsuitable for our context, some emerging methods do cater to language data scales. However, they often prioritize data that aligns with the target distribution. While this strategy may be effective when training a model from scratch, it can yield limited results when the model has already been pre-trained on a different distribution. Differing from prior work, our key idea is to select data that nudges the pre-training distribution closer to the target distribution. We show the optimality of this approach for fine-tuning tasks under certain conditions. We demonstrate the efficacy of our methodology across a diverse array of tasks (NLU, NLG, zero-shot) with models up to 2.7B, showing that it consistently surpasses other selection methods. Moreover, our proposed method is significantly faster than existing techniques, scaling to millions of samples within a single GPU hour. Our code is open-sourced (Code repository: https://anonymous.4open.science/r/DV4LLM-D761/ ). While fine-tuning offers significant potential for enhancing performance across diverse tasks, its associated costs often limit its widespread adoption; with this work, we hope to lay the groundwork for cost-effective fine-tuning, making its benefits more accessible.


title: U-duality in quantum M2-branes and gauged supergravities

abstract: In this paper, we study the relation of the M2-brane with fluxes and monodromy in $SL(2,\mathbb{Z})$, which has a quantum discrete supersymmetric spectrum with finite multiplicity, and type IIB gauged supergravities in nine dimensions. $SL(2,\mathbb{Z})$ is the group of isotopy classes of the area preserving diffeomorphisms. The global description of these M2-branes we are considering is formulated on twisted torus bundles, and they are classified in terms of $H^2(\Sigma,\mathcal{Z}_{\mathcal{\rho}})$, or equivalently, by their coinvariants for a given monodromy subgroup. We find the 'gauge' symmetries between equivalent M2-branes, on torus bundles with monodromy, that leads to $\mathbb{R}$, $SO(2)$ or $SO(1,1)$, the symmetry groups of type IIB gauged supergravities in 9d. We obtain an explicit relation between the equivalent classes of M2-brane bundles and the mass parameters that classify the gaugings of type IIB supergravities in 9d. We also find that the symmetries, between inequivalent M2-branes on twisted torus bundles for a given monodromy, are related with $\mathbb{Z}$, $\mathcal{Z}_3$, $\mathcal{Z}_5$, $\mathcal{Z}_9$ or $\mathcal{Z}_{2n-7}$ for $n\geq 5$, the U-duality symmetry group, a subgroup of $SL(2,\mathbb{Z})$. In distinction, in the case without monodromy, related to type II maximal supergravity at low energies, its U-duality group corresponds to the full $SL(2,\mathbb{Z})$.


title: Hypergeometric accelerations with shifted indices

abstract: Chu and Zhang, in 2014, introduced hypergeometric transforms derived through the application of an Abel-type summation lemma to Dougall's ${}_{5}H_{5}$-series. These transforms were applied by Chu and Zhang to obtain accelerated rates of convergence, yielding rational series related to the work of Ramanujan and Guillera. We apply a variant of an acceleration method due to Wilf using what we refer to as shifted indices for Pochhammer symbols involved in our first-order, inhomogeneous recurrences derived via Zeilberger's algorithm, to build upon Chu and Zhang's accelerations, recovering many of their accelerated series and introducing many inequivalent series for universal constants, including series of Ramanujan type involving linear polynomials as summand factors, as in Ramanujan's series for $\frac{1}{\pi}$.


title: Normed modules and the categorization of Lebesgue integration

abstract: We explore the assignment of norms to $\Lambda$-modules over a finite-dimensional algebra $\Lambda$, resulting in the establishment of normed $\Lambda$-modules. Our primary contribution lies in constructing a new category $\mathscr{N}\!\!or^p$ related to normed modules along with its full subcategory $\mathscr{A}^p$. By examining the objects and morphisms in these categories, we establish a framework for understanding the categorization of Lebesgue integration.


title: Improve Temporal Awareness of LLMs for Sequential Recommendation

abstract: Large language models (LLMs) have demonstrated impressive zero-shot abilities in solving a wide range of general-purpose tasks. However, it is empirically found that LLMs fall short in recognizing and utilizing temporal information, rendering poor performance in tasks that require an understanding of sequential data, such as sequential recommendation. In this paper, we aim to improve temporal awareness of LLMs by designing a principled prompting framework inspired by human cognitive processes. Specifically, we propose three prompting strategies to exploit temporal information within historical interactions for LLM-based sequential recommendation. Besides, we emulate divergent thinking by aggregating LLM ranking results derived from these strategies. Evaluations on MovieLens-1M and Amazon Review datasets indicate that our proposed method significantly enhances the zero-shot capabilities of LLMs in sequential recommendation tasks.


title: Estimating Complier Average Causal Effects with Mixtures of Experts

abstract: Understanding the causal impact of medical interventions is essential in healthcare research, especially through randomized controlled trials (RCTs). Despite their prominence, challenges arise due to discrepancies between treatment allocation and actual intake, influenced by various factors like patient non-adherence or procedural errors. This paper focuses on the Complier Average Causal Effect (CACE), crucial for evaluating treatment efficacy among compliant patients. Existing methodologies often rely on assumptions such as exclusion restriction and monotonicity, which can be problematic in practice. We propose a novel approach, leveraging supervised learning architectures, to estimate CACE without depending on these assumptions. Our method involves a two-step process: first estimating compliance probabilities for patients, then using these probabilities to estimate two nuisance components relevant to CACE calculation. Building upon the principal ignorability assumption, we introduce four root-n consistent, asymptotically normal, CACE estimators, and prove that the underlying mixtures of experts' nuisance components are identifiable. Our causal framework allows our estimation procedures to enjoy reduced mean squared errors when exclusion restriction or monotonicity assumptions hold. Through simulations and application to a breastfeeding promotion RCT, we demonstrate the method's performance and applicability.


title: A hydrodynamic model of capillary flow in an axially symmetric tube with a non-slowly-varying cross section and a boundary slip

abstract: The capillary flow of a Newtonian and incompressible fluid in an axially symmetric horizontal tube with a non-slowly-varying cross section and a boundary slip is considered theoretically under the assumption that the Reynolds number is small enough for the Stokes approximation to be valid. Combining the Stokes equation with the hydrodynamic model assuming the Hagen-Poiseulle flow, a general formula for the capillary flow in a non-slowly-varying tube is derived. Using the newly derived formula, the capillary imbibition and the time evolution of meniscus in tubes with non-uniform cross sections such as a conical tube, a power-law-shaped diverging tube, and a power-law-shaped converging tube are reconsidered. The perturbation parameters and the corrections due to the non-slowly-varying effects are elucidated and the new scaling formulas for the time evolution of the meniscus of these specific examples are derived. Our study could be useful for understanding various natural fluidic systems and for designing functional fluidic devices such as a diode and a switch.


title: Instantaneous Perception of Moving Objects in 3D

abstract: The perception of 3D motion of surrounding traffic participants is crucial for driving safety. While existing works primarily focus on general large motions, we contend that the instantaneous detection and quantification of subtle motions is equally important as they indicate the nuances in driving behavior that may be safety critical, such as behaviors near a stop sign of parking positions. We delve into this under-explored task, examining its unique challenges and developing our solution, accompanied by a carefully designed benchmark. Specifically, due to the lack of correspondences between consecutive frames of sparse Lidar point clouds, static objects might appear to be moving - the so-called swimming effect. This intertwines with the true object motion, thereby posing ambiguity in accurate estimation, especially for subtle motions. To address this, we propose to leverage local occupancy completion of object point clouds to densify the shape cue, and mitigate the impact of swimming artifacts. The occupancy completion is learned in an end-to-end fashion together with the detection of moving objects and the estimation of their motion, instantaneously as soon as objects start to move. Extensive experiments demonstrate superior performance compared to standard 3D motion estimation approaches, particularly highlighting our method's specialized treatment of subtle motions.


title: A self-supervised text-vision framework for automated brain abnormality detection

abstract: Artificial neural networks trained on large, expert-labelled datasets are considered state-of-the-art for a range of medical image recognition tasks. However, categorically labelled datasets are time-consuming to generate and constrain classification to a pre-defined, fixed set of classes. For neuroradiological applications in particular, this represents a barrier to clinical adoption. To address these challenges, we present a self-supervised text-vision framework that learns to detect clinically relevant abnormalities in brain MRI scans by directly leveraging the rich information contained in accompanying free-text neuroradiology reports. Our training approach consisted of two-steps. First, a dedicated neuroradiological language model - NeuroBERT - was trained to generate fixed-dimensional vector representations of neuroradiology reports (N = 50,523) via domain-specific self-supervised learning tasks. Next, convolutional neural networks (one per MRI sequence) learnt to map individual brain scans to their corresponding text vector representations by optimising a mean square error loss. Once trained, our text-vision framework can be used to detect abnormalities in unreported brain MRI examinations by scoring scans against suitable query sentences (e.g., 'there is an acute stroke', 'there is hydrocephalus' etc.), enabling a range of classification-based applications including automated triage. Potentially, our framework could also serve as a clinical decision support tool, not only by suggesting findings to radiologists and detecting errors in provisional reports, but also by retrieving and displaying examples of pathologies from historical examinations that could be relevant to the current case based on textual descriptors.


title: Linear Noise Approximation Assisted Bayesian Inference on Mechanistic Model of Partially Observed Stochastic Reaction Network

abstract: To support mechanism online learning and facilitate digital twin development for biomanufacturing processes, this paper develops an efficient Bayesian inference approach for partially observed enzymatic stochastic reaction network (SRN), a fundamental building block of multi-scale bioprocess mechanistic model. To tackle the critical challenges brought by the nonlinear stochastic differential equations (SDEs)-based mechanistic model with partially observed state and having measurement error, an interpretable Bayesian updating linear noise approximation (LNA) metamodel, incorporating the structure information of the mechanistic model, is proposed to approximate the likelihood of observations. Then, an efficient posterior sampling approach is developed by utilizing the gradients of the derived likelihood to speed up the convergence of MCMC. The empirical study demonstrates that the proposed approach has a promising performance.


title: MR-Transformer: Vision Transformer for Total Knee Replacement Prediction Using Magnetic Resonance Imaging

abstract: A transformer-based deep learning model, MR-Transformer, was developed for total knee replacement (TKR) prediction using magnetic resonance imaging (MRI). The model incorporates the ImageNet pre-training and captures three-dimensional (3D) spatial correlation from the MR images. The performance of the proposed model was compared to existing state-of-the-art deep learning models for knee injury diagnosis using MRI. Knee MR scans of four different tissue contrasts from the Osteoarthritis Initiative and Multicenter Osteoarthritis Study databases were utilized in the study. Experimental results demonstrated the state-of-the-art performance of the proposed model on TKR prediction using MRI.


title: Fused attention mechanism-based ore sorting network

abstract: Deep learning has had a significant impact on the identification and classification of mineral resources, especially playing a key role in efficiently and accurately identifying different minerals, which is important for improving the efficiency and accuracy of mining. However, traditional ore sorting meth- ods often suffer from inefficiency and lack of accuracy, especially in complex mineral environments. To address these challenges, this study proposes a method called OreYOLO, which incorporates an attentional mechanism and a multi-scale feature fusion strategy, based on ore data from gold and sul- fide ores. By introducing the progressive feature pyramid structure into YOLOv5 and embedding the attention mechanism in the feature extraction module, the detection performance and accuracy of the model are greatly improved. In order to adapt to the diverse ore sorting scenarios and the deployment requirements of edge devices, the network structure is designed to be lightweight, which achieves a low number of parameters (3.458M) and computational complexity (6.3GFLOPs) while maintaining high accuracy (99.3% and 99.2%, respectively). In the experimental part, a target detection dataset containing 6000 images of gold and sulfuric iron ore is constructed for gold and sulfuric iron ore classification training, and several sets of comparison experiments are set up, including the YOLO series, EfficientDet, Faster-RCNN, and CenterNet, etc., and the experiments prove that OreYOLO outperforms the commonly used high-performance object detection of these architectures


title: A causal inference approach of monosynapses from spike trains

abstract: Neuroscientists have worked on the problem of estimating synaptic properties, such as connectivity and strength, from simultaneously recorded spike trains since the 1960s. Recent years have seen renewed interest in the problem, coinciding with rapid advances in the technology of high-density neural recordings and optogenetics, which can be used to calibrate causal hypotheses about functional connectivity. Here, a rigorous causal inference framework for pairwise excitatory and inhibitory monosynaptic effects between spike trains is developed. Causal interactions are identified by separating spike interactions in pairwise spike trains by their timescales. Fast algorithms for computing accurate estimates of associated quantities are also developed. Through the lens of this framework, the link between biophysical parameters and statistical definitions of causality between spike trains is examined across a spectrum of dynamical systems simulations. In an idealized setting, we demonstrate a correspondence between the synaptic causal metric developed here and the probabilities of causation developed by Tian and Pearl. Since the probabilities of causation are derived under distinct assumptions and include data from experimental randomization, this opens up the possibility of testing the synaptic inference framework's assumptions with juxtacellular or optogenetic stimulation. We simulate such an experiment with a biophysically detailed channelrhodopsin model and show that randomization is not achieved; strong confounding persists even with strong stimulations. A principal goal is to ask how carefully articulated causal assumptions might better inform the design of neural stimulation experiments and, in turn, support experimental tests of those assumptions.


title: Light Field Spatial Resolution Enhancement Framework

abstract: Light field (LF) imaging captures both angular and spatial light distributions, enabling advanced photographic techniques. However, micro-lens array (MLA)- based cameras face a spatial-angular resolution tradeoff due to a single shared sensor. We propose a novel light field framework for resolution enhancement, employing a modular approach. The first module generates a high-resolution, all-in-focus image. The second module, a texture transformer network, enhances the resolution of each light field perspective independently using the output of the first module as a reference image. The final module leverages light field regularity to jointly improve resolution across all LF image perspectives. Our approach demonstrates superior performance to existing methods in both qualitative and quantitative evaluations.


title: Antenna Failure Resilience: Deep Learning-Enabled Robust DOA Estimation with Single Snapshot Sparse Arrays

abstract: Recent advancements in Deep Learning (DL) for Direction of Arrival (DOA) estimation have highlighted its superiority over traditional methods, offering faster inference, enhanced super-resolution, and robust performance in low Signal-to-Noise Ratio (SNR) environments. Despite these advancements, existing research predominantly focuses on multi-snapshot scenarios, a limitation in the context of automotive radar systems which demand high angular resolution and often rely on limited snapshots, sometimes as scarce as a single snapshot. Furthermore, the increasing interest in sparse arrays for automotive radar, owing to their cost-effectiveness and reduced antenna element coupling, presents additional challenges including susceptibility to random sensor failures. This paper introduces a pioneering DL framework featuring a sparse signal augmentation layer, meticulously crafted to bolster single snapshot DOA estimation across diverse sparse array setups and amidst antenna failures. To our best knowledge, this is the first work to tackle this issue. Our approach improves the adaptability of deep learning techniques to overcome the unique difficulties posed by sparse arrays with single snapshot. We conduct thorough evaluations of our network's performance using simulated and real-world data, showcasing the efficacy and real-world viability of our proposed solution. The code and real-world dataset employed in this study are available at https://github.com/ruxinzh/Deep_RSA_DOA.


title: Heyde theorem for locally compact Abelian groups containing no subgroups topologically isomorphic to the 2-dimensional torus

abstract: We prove the following group analogue of the well-known Heyde theorem on a characterization of the Gaussian distribution on the real line. Let $X$ be a second countable locally compact Abelian group containing no subgroups topologically isomorphic to the 2-dimensional torus. Let $G$ be the subgroup of $X$ generated by all elements of $X$ of order $2$ and let $\alpha$ be a topological automorphism of the group $X$ such that ${\rm Ker}(I+\alpha)=\{0\}$. Let $\xi_1$ and $\xi_2$ be independent random variables with values in $X$ and distributions $\mu_1$ and $\mu_2$ with nonvanishing characteristic functions. If the conditional distribution of the linear form $L_2 = \xi_1 + \alpha\xi_2$ given $L_1 = \xi_1 +\xi_2$ is symmetric, then $\mu_j$ are convolutions of Gaussian distributions on $X$ and distributions supported in $G$. We also prove that this theorem is false if $X$ is the 2-dimensional torus.


title: Confidential and Protected Disease Classifier using Fully Homomorphic Encryption

abstract: With the rapid surge in the prevalence of Large Language Models (LLMs), individuals are increasingly turning to conversational AI for initial insights across various domains, including health-related inquiries such as disease diagnosis. Many users seek potential causes on platforms like ChatGPT or Bard before consulting a medical professional for their ailment. These platforms offer valuable benefits by streamlining the diagnosis process, alleviating the significant workload of healthcare practitioners, and saving users both time and money by avoiding unnecessary doctor visits. However, Despite the convenience of such platforms, sharing personal medical data online poses risks, including the presence of malicious platforms or potential eavesdropping by attackers. To address privacy concerns, we propose a novel framework combining FHE and Deep Learning for a secure and private diagnosis system. Operating on a question-and-answer-based model akin to an interaction with a medical practitioner, this end-to-end secure system employs Fully Homomorphic Encryption (FHE) to handle encrypted input data. Given FHE's computational constraints, we adapt deep neural networks and activation functions to the encryted domain. Further, we also propose a faster algorithm to compute summation of ciphertext elements. Through rigorous experiments, we demonstrate the efficacy of our approach. The proposed framework achieves strict security and privacy with minimal loss in performance.


title: Efficient Text-driven Motion Generation via Latent Consistency Training

abstract: Motion diffusion models excel at text-driven motion generation but struggle with real-time inference since motion sequences are time-axis redundant and solving reverse diffusion trajectory involves tens or hundreds of sequential iterations. In this paper, we propose a Motion Latent Consistency Training (MLCT) framework, which allows for large-scale skip sampling of compact motion latent representation by constraining the consistency of the outputs of adjacent perturbed states on the precomputed trajectory. In particular, we design a flexible motion autoencoder with quantization constraints to guarantee the low-dimensionality, succinctness, and boundednes of the motion embedding space. We further present a conditionally guided consistency training framework based on conditional trajectory simulation without additional pre-training diffusion model, which significantly improves the conditional generation performance with minimal training cost. Experiments on two benchmarks demonstrate our model's state-of-the-art performance with an 80\% inference cost saving and around 14 ms on a single RTX 4090 GPU.


title: Jointly Learning Spatial, Angular, and Temporal Information for Enhanced Lane Detection

abstract: This paper introduces a novel approach for enhanced lane detection by integrating spatial, angular, and temporal information through light field imaging and novel deep learning models. Utilizing lenslet-inspired 2D light field representations and LSTM networks, our method significantly improves lane detection in challenging conditions. We demonstrate the efficacy of this approach with modified CNN architectures, showing superior per- formance over traditional methods. Our findings suggest this integrated data approach could advance lane detection technologies and inspire new models that leverage these multidimensional insights for autonomous vehicle percep- tion.


title: ImageInWords: Unlocking Hyper-Detailed Image Descriptions

abstract: Despite the longstanding adage "an image is worth a thousand words," creating accurate and hyper-detailed image descriptions for training Vision-Language models remains challenging. Current datasets typically have web-scraped descriptions that are short, low-granularity, and often contain details unrelated to the visual content. As a result, models trained on such data generate descriptions replete with missing information, visual inconsistencies, and hallucinations. To address these issues, we introduce ImageInWords (IIW), a carefully designed human-in-the-loop annotation framework for curating hyper-detailed image descriptions and a new dataset resulting from this process. We validate the framework through evaluations focused on the quality of the dataset and its utility for fine-tuning with considerations for readability, comprehensiveness, specificity, hallucinations, and human-likeness. Our dataset significantly improves across these dimensions compared to recently released datasets (+66%) and GPT-4V outputs (+48%). Furthermore, models fine-tuned with IIW data excel by +31% against prior work along the same human evaluation dimensions. Given our fine-tuned models, we also evaluate text-to-image generation and vision-language reasoning. Our model's descriptions can generate images closest to the original, as judged by both automated and human metrics. We also find our model produces more compositionally rich descriptions, outperforming the best baseline by up to 6% on ARO, SVO-Probes, and Winoground datasets.


title: Octopi: Object Property Reasoning with Large Tactile-Language Models

abstract: Physical reasoning is important for effective robot manipulation. Recent work has investigated both vision and language modalities for physical reasoning; vision can reveal information about objects in the environment and language serves as an abstraction and communication medium for additional context. Although these works have demonstrated success on a variety of physical reasoning tasks, they are limited to physical properties that can be inferred from visual or language inputs. In this work, we investigate combining tactile perception with language, which enables embodied systems to obtain physical properties through interaction and apply commonsense reasoning. We contribute a new dataset PhysiCLeAR, which comprises both physical/property reasoning tasks and annotated tactile videos obtained using a GelSight tactile sensor. We then introduce Octopi, a system that leverages both tactile representation learning and large vision-language models to predict and reason about tactile inputs with minimal language fine-tuning. Our evaluations on PhysiCLeAR show that Octopi is able to effectively use intermediate physical property predictions to improve its performance on various tactile-related tasks. PhysiCLeAR and Octopi are available at https://github.com/clear-nus/octopi.


title: Graph as Point Set

abstract: Graph is a fundamental data structure to model interconnections between entities. Set, on the contrary, stores independent elements. To learn graph representations, current Graph Neural Networks (GNNs) primarily use message passing to encode the interconnections. In contrast, this paper introduces a novel graph-to-set conversion method that bijectively transforms interconnected nodes into a set of independent points and then uses a set encoder to learn the graph representation. This conversion method holds dual significance. Firstly, it enables using set encoders to learn from graphs, thereby significantly expanding the design space of GNNs. Secondly, for Transformer, a specific set encoder, we provide a novel and principled approach to inject graph information losslessly, different from all the heuristic structural/positional encoding methods adopted in previous graph transformers. To demonstrate the effectiveness of our approach, we introduce Point Set Transformer (PST), a transformer architecture that accepts a point set converted from a graph as input. Theoretically, PST exhibits superior expressivity for both short-range substructure counting and long-range shortest path distance tasks compared to existing GNNs. Extensive experiments further validate PST's outstanding real-world performance. Besides Transformer, we also devise a Deepset-based set encoder, which achieves performance comparable to representative GNNs, affirming the versatility of our graph-to-set method.


title: Canonical data-reconstructions via kernels, Hilbert space-valued Gaussian processes, and quantum states

abstract: We offer new results and new directions in the study of operator-valued kernels and their factorizations. Our approach provides both more explicit realizations and new results, as well as new applications. These include: (i) an explicit covariance analysis for Hilbert space-valued Gaussian processes, (ii) optimization results for quantum gates (from quantum information), (iii) new results for positive operator-valued measures (POVMs), and (iv) a new approach/result in inverse problems for quantum measurements.


title: Adapting to Distribution Shift by Visual Domain Prompt Generation

abstract: In this paper, we aim to adapt a model at test-time using a few unlabeled data to address distribution shifts. To tackle the challenges of extracting domain knowledge from a limited amount of data, it is crucial to utilize correlated information from pre-trained backbones and source domains. Previous studies fail to utilize recent foundation models with strong out-of-distribution generalization. Additionally, domain-centric designs are not flavored in their works. Furthermore, they employ the process of modelling source domains and the process of learning to adapt independently into disjoint training stages. In this work, we propose an approach on top of the pre-computed features of the foundation model. Specifically, we build a knowledge bank to learn the transferable knowledge from source domains. Conditioned on few-shot target data, we introduce a domain prompt generator to condense the knowledge bank into a domain-specific prompt. The domain prompt then directs the visual features towards a particular domain via a guidance module. Moreover, we propose a domain-aware contrastive loss and employ meta-learning to facilitate domain knowledge extraction. Extensive experiments are conducted to validate the domain knowledge extraction. The proposed method outperforms previous work on 5 large-scale benchmarks including WILDS and DomainNet.


title: Structural Balance in Real-World Social Networks: Incorporating Direction and Transitivity in Measuring Partial Balance

abstract: Structural balance theory predicts that triads in networks gravitate towards stable configurations. The theory has been verified for undirected graphs. Since real-world networks are often directed, we introduce a novel method for considering both transitivity and sign consistency for evaluating partial balance in signed digraphs. We test our approach on graphs constructed by using different methods for identifying edge signs: natural language processing to infer signs from underlying text data, and self-reported survey data. Our results show that for various social contexts and edge sign detection methods, partial balance of these digraphs are moderately high, ranging from 61% to 96%. Our approach not only enhances the theoretical framework of structural balance but also provides practical insights into the stability of social networks, enabling a deeper understanding of interpersonal and group dynamics across different communication platforms.


title: Solar Flare Ribbon Fronts. II. Evolution of heating rates in individual flare footpoints

abstract: Solar flare ribbon fronts appear ahead of the bright structures that normally characterise solar flares, and can persist for an extended period of time in spatially localised patches before transitioning to `regular' bright ribbons. They likely represent the initial onset of flare energy deposition into the chromosphere. Chromospheric spectra (e.g. He I 10830A and the Mg II near-UV lines) from ribbon fronts exhibit properties rather different to typical flare behaviour. In prior numerical modelling efforts we were unable to reproduce the long lifetime of ribbon fronts. Here we present a series of numerical experiments that are rather simple but which have important implications. We inject a very low flux of nonthermal electrons ($F = 5\times10^{8}$ erg s$^{-1}$ cm$^{-2}$) into the chromosphere for 100 s before ramping up to standard flare energy fluxes $(F = 10^{10-11}$ erg s$^{-1}$ cm$^{-2}$). Synthetic spectra not only sustained their ribbon front-like properties for significantly longer, in the case of harder nonthermal electron spectra the ribbon front behaviour persisted for the entirety of this weak-heating phase. Lengthening or shortening the duration of the weak-heating phase commensurately lengthened or shortened the ribbon front lifetimes. Ribbon fronts transitioned to regular bright ribbons when the upper chromosphere became sufficiently hot and dense, which happened faster for softer nonthermal electron spectra. Thus, the lifetime of flare ribbon fronts are a direct measure of the duration over which a relatively low flux of high energy electrons precipitates to the chromosphere prior to the bombardment of a much larger energy flux.


title: Designing Distinguishable Mid-Air Ultrasound Tactons with Temporal Parameters

abstract: Mid-air ultrasound technology offers new design opportunities for contactless tactile patterns (i.e., Tactons) in user applications. Yet, few guidelines exist for making ultrasound Tactons easy to distinguish for users. In this paper, we investigated the distinguishability of temporal parameters of ultrasound Tactons in five studies (n=72 participants). Study 1 established the discrimination thresholds for amplitude-modulated (AM) frequencies. In Studies 2-5, we investigated distinguishable ultrasound Tactons by creating four Tacton sets based on mechanical vibrations in the literature and collected similarity ratings for the ultrasound Tactons. We identified a subset of temporal parameters, such as rhythm and low envelope frequency, that could create distinguishable ultrasound Tactons. Also, a strong correlation (mean Spearman's $\rho$=0.75) existed between similarity ratings for ultrasound Tactons and similarities of mechanical Tactons from the literature, suggesting vibrotactile designers can transfer their knowledge to ultrasound design. We present design guidelines and future directions for creating distinguishable mid-air ultrasound Tactons.


title: Mozart's Touch: A Lightweight Multi-modal Music Generation Framework Based on Pre-Trained Large Models

abstract: In recent years, AI-Generated Content (AIGC) has witnessed rapid advancements, facilitating the generation of music, images, and other forms of artistic expression across various industries. However, researches on general multi-modal music generation model remain scarce. To fill this gap, we propose a multi-modal music generation framework Mozart's Touch. It could generate aligned music with the cross-modality inputs, such as images, videos and text. Mozart's Touch is composed of three main components: Multi-modal Captioning Module, Large Language Model (LLM) Understanding & Bridging Module, and Music Generation Module. Unlike traditional approaches, Mozart's Touch requires no training or fine-tuning pre-trained models, offering efficiency and transparency through clear, interpretable prompts. We also introduce "LLM-Bridge" method to resolve the heterogeneous representation problems between descriptive texts of different modalities. We conduct a series of objective and subjective evaluations on the proposed model, and results indicate that our model surpasses the performance of current state-of-the-art models. Our codes and examples is availble at: https://github.com/WangTooNaive/MozartsTouch


title: Permutation time irreversibility in sleep electroencephalograms: Dependence on sleep stage and the effect of equal values

abstract: Time irreversibility (TIR) refers to the manifestation of nonequilibrium brain activity influenced by various physiological conditions; however, the influence of sleep on electroencephalogram (EEG) TIR has not been sufficiently investigated. In this paper, a comprehensive study on permutation TIR (pTIR) of EEG data under different sleep stages is conducted. Two basic ordinal patterns (i.e., the original and amplitude permutations) are distinguished to simplify sleep EEGs, and then the influences of equal values and forbidden permutation on pTIR are elucidated. To detect pTIR of brain electric signals, 5 groups of EEGs in the awake, stages I, II, III, and rapid eye movement (REM) stages are collected from the public Polysomnographic Database in PhysioNet. Test results suggested that the pTIR of sleep EEGs significantly decreases as the sleep stage increases (p<0.001), with the awake and REM EEGs, demonstrating greater differences than others. Comparative analysis and numerical simulations support the importance of equal values. Distribution of equal states, a simple quantification of amplitude fluctuations, significantly increases with the sleep stage (p<0.001). If these equalities are ignored, incorrect probabilistic differences may arise in the forward-backward and symmetric permutations of TIR, leading to contradictory results; moreover, the ascending and descending orders for symmetric permutations also lead different outcomes in sleep EEGs. Overall, pTIR in sleep EEGs contributes to our understanding of quantitative TIR and classification of sleep EEGs.


title: Is Flash Attention Stable?

abstract: Training large-scale machine learning models poses distinct system challenges, given both the size and complexity of today's workloads. Recently, many organizations training state-of-the-art Generative AI models have reported cases of instability during training, often taking the form of loss spikes. Numeric deviation has emerged as a potential cause of this training instability, although quantifying this is especially challenging given the costly nature of training runs. In this work, we develop a principled approach to understanding the effects of numeric deviation, and construct proxies to put observations into context when downstream effects are difficult to quantify. As a case study, we apply this framework to analyze the widely-adopted Flash Attention optimization. We find that Flash Attention sees roughly an order of magnitude more numeric deviation as compared to Baseline Attention at BF16 when measured during an isolated forward pass. We then use a data-driven analysis based on the Wasserstein Distance to provide upper bounds on how this numeric deviation impacts model weights during training, finding that the numerical deviation present in Flash Attention is 2-5 times less significant than low-precision training.


title: Consequences of the $\mu$-$\tau$ reflection symmetry for leptogenesis in a seesaw model with diagonal Dirac neutrino mass matrix

abstract: In this paper, we have studied the consequences of the $\mu$-$\tau$ reflection symmetry for leptogenesis in the type-I seesaw model with diagonal Dirac neutrino mass matrix. We have first obtained the phenomenologically allowed values of the model parameters, which show that there may exist zero or equal entries in the Majorana mass matrix for the right-handed neutrinos, and then studied their predictions for three right-handed neutrino masses, which show that there may exist two nearly degenerate right-handed neutrinos. Then, we have studied the consequences of the model for leptogenesis. Due to the $\mu$-$\tau$ reflection symmetry, leptogenesis can only work in the two-flavor regime. Furthermore, leptogenesis cannot work for the particular case of $r=1$. Accordingly, for some benchmark values of $r \neq 1$, we have given the constraints of leptogenesis for relevant parameters. Furthermore, we have investigated the possibilities of leptogenesis being induced by the renormalization group evolution effects for two particular scenarios. For the particular case of $r=1$, the renormalization group evolution effects will break the orthogonality relations among different columns of $M^{}_{\rm D}$ and consequently induce leptogenesis to work. For the low-scale resonant leptogenesis scenario which is realized for nearly degenerate right-handed neutrinos, the renormalization group evolution effects can break the $\mu$-$\tau$ reflection symmetry and consequently induce leptogenesis to work.


title: Verlet Flows: Exact-Likelihood Integrators for Flow-Based Generative Models

abstract: Approximations in computing model likelihoods with continuous normalizing flows (CNFs) hinder the use of these models for importance sampling of Boltzmann distributions, where exact likelihoods are required. In this work, we present Verlet flows, a class of CNFs on an augmented state-space inspired by symplectic integrators from Hamiltonian dynamics. When used with carefully constructed Taylor-Verlet integrators, Verlet flows provide exact-likelihood generative models which generalize coupled flow architectures from a non-continuous setting while imposing minimal expressivity constraints. On experiments over toy densities, we demonstrate that the variance of the commonly used Hutchinson trace estimator is unsuitable for importance sampling, whereas Verlet flows perform comparably to full autograd trace computations while being significantly faster.


title: Foreground removal and angular power spectrum estimation of 21 cm signal using harmonic space ILC method

abstract: Mapping the distribution of neutral atomic hydrogen (HI) in the Universe through its 21 cm emission line provides a powerful cosmological probe to map the large-scale structures and shed light on various cosmological phenomena. The Baryon Acoustic Oscillations at low redshifts can potentially be probed by sensitive HI intensity mapping experiments and constrain the properties of dark energy. However, the 21 cm signal detection faces formidable challenges due to the dominance of various astrophysical foregrounds, which can be several orders of magnitude stronger. Our current work introduces a novel and model-independent Internal Linear Combination (ILC) method in harmonic space using the principal components of the 21 cm signal for accurate foreground removal and power spectrum estimation. We estimate the principal components by incorporating prior knowledge of the theoretical 21 cm covariance matrix. We test our methodology by detailed simulations of radio observations, incorporating synchrotron emission, free-free radiation, extragalactic point sources, and thermal noise. We estimate the full sky 21 cm angular power spectrum after application of a mask on the full sky cleaned 21 cm signal by using the mode-mode coupling matrix. These full sky estimates of angular spectra can be directly used to measure the cosmological parameters. For the first time, we demonstrate the effectiveness of a foreground model-independent ILC method in harmonic space to reconstruct the 21 cm signal.


title: Kinematic analysis of structural mechanics based on convolutional neural network

abstract: Attempt to use convolutional neural network to achieve kinematic analysis of plane bar structure. Through 3dsMax animation software and OpenCV module, self-build image dataset of geometrically stable system and geometrically unstable system. we construct and train convolutional neural network model based on the TensorFlow and Keras deep learning platform framework. The model achieves 100% accuracy on the training set, validation set, and test set. The accuracy on the additional test set is 93.7%, indicating that convolutional neural network can learn and master the relevant knowledge of kinematic analysis of structural mechanics. In the future, the generalization ability of the model can be improved through the diversity of dataset, which has the potential to surpass human experts for complex structures. Convolutional neural network has certain practical value in the field of kinematic analysis of structural mechanics. Using visualization technology, we reveal how convolutional neural network learns and recognizes structural features. Using pre-trained VGG16 model for feature extraction and fine-tuning, we found that the generalization ability is inferior to the self-built model.


title: An Interactive Tool for Simulating Mid-Air Ultrasound Tactons on the Skin

abstract: Mid-air ultrasound haptic technology offers a myriad of temporal and spatial parameters for contactless haptic design. Yet, predicting how these parameters interact to render an ultrasound signal is difficult before testing them on a mid-air ultrasound haptic device. Thus, haptic designers often use a trial-and-error process with different parameter combinations to obtain desired tactile patterns (i.e., Tactons) for user applications. We propose an interactive tool with five temporal and three spatiotemporal design parameters that can simulate the temporal and spectral properties of stimulation at specific skin points. As a preliminary verification, we measured vibrations induced from the ultrasound Tactons varying on one temporal and two spatiotemporal parameters. The measurements and simulation showed similar results for three different ultrasound rendering techniques, suggesting the efficacy of the simulation tool. We present key insights from the simulation and discuss future directions for enhancing the capabilities of simulations.


title: Does Optimal Control Always Benefit from Better Prediction? An Analysis Framework for Predictive Optimal Control

abstract: The ``prediction + optimal control'' scheme has shown good performance in many applications of automotive, traffic, robot, and building control. In practice, the prediction results are simply considered correct in the optimal control design process. However, in reality, these predictions may never be perfect. Under a conventional stochastic optimal control formulation, it is difficult to answer questions like ``what if the predictions are wrong''. This paper presents an analysis framework for predictive optimal control where the subjective belief about the future is no longer considered perfect. A novel concept called the hidden prediction state is proposed to establish connections among the predictors, the subjective beliefs, the control policies and the objective control performance. Based on this framework, the predictor evaluation problem is analyzed. Three commonly-used predictor evaluation measures, including the mean squared error, the regret and the log-likelihood, are considered. It is shown that neither using the mean square error nor using the likelihood can guarantee a monotonic relationship between the predictor error and the optimal control cost. To guarantee control cost improvement, it is suggested the predictor should be evaluated with the control performance, e.g., using the optimal control cost or the regret to evaluate predictors. Numerical examples and examples from automotive applications with real-world driving data are provided to illustrate the ideas and the results.


title: Adaptive deep density approximation for stochastic dynamical systems

abstract: In this paper we consider adaptive deep neural network approximation for stochastic dynamical systems. Based on the Liouville equation associated with the stochastic dynamical systems, a new temporal KRnet (tKRnet) is proposed to approximate the probability density functions (PDFs) of the state variables. The tKRnet gives an explicit density model for the solution of the Liouville equation, which alleviates the curse of dimensionality issue that limits the application of traditional grid based numerical methods. To efficiently train the tKRnet, an adaptive procedure is developed to generate collocation points for the corresponding residual loss function, where samples are generated iteratively using the approximate density function at each iteration. A temporal decomposition technique is also employed to improve the long-time integration. Theoretical analysis of our proposed method is provided, and numerical examples are presented to demonstrate its performance.


title: PVTransformer: Point-to-Voxel Transformer for Scalable 3D Object Detection

abstract: 3D object detectors for point clouds often rely on a pooling-based PointNet to encode sparse points into grid-like voxels or pillars. In this paper, we identify that the common PointNet design introduces an information bottleneck that limits 3D object detection accuracy and scalability. To address this limitation, we propose PVTransformer: a transformer-based point-to-voxel architecture for 3D detection. Our key idea is to replace the PointNet pooling operation with an attention module, leading to a better point-to-voxel aggregation function. Our design respects the permutation invariance of sparse 3D points while being more expressive than the pooling-based PointNet. Experimental results show our PVTransformer achieves much better performance compared to the latest 3D object detectors. On the widely used Waymo Open Dataset, our PVTransformer achieves state-of-the-art 76.5 mAPH L2, outperforming the prior art of SWFormer by +1.7 mAPH L2.


title: Neural Network Enhanced Single-Photon Fock State Tomography

abstract: Even though heralded single-photon sources have been generated routinely through the spontaneous parametric down conversion, vacuum and multiple photon states are unavoidably involved. With machine-learning, we report the experimental implementation of single-photon quantum state tomography by directly estimating target parameters. Compared to the Hanbury Brown and Twiss (HBT) measurements only with clicked events recorded, our neural network enhanced quantum state tomography characterizes the photon number distribution for all possible photon number states from the balanced homodyne detectors. By using the histogram-based architecture, a direct parameter estimation on the negativity in Wigner's quasi-probability phase space is demonstrated. Such a fast, robust, and precise quantum state tomography provides us a crucial diagnostic toolbox for the applications with single-photon Fock states and other non-Gaussisan quantum states.


title: Model Predictive Control for Joint Ramping and Regulation-Type Service from Distributed Energy Resource Aggregations

abstract: Distributed energy resources (DERs) such as grid-responsive loads and batteries can be harnessed to provide ramping and regulation services across the grid. This paper concerns the problem of optimal allocation of different classes of DERs, where each class is an aggregation of similar DERs, to balance net-demand forecasts. The resulting resource allocation problem is solved using model-predictive control (MPC) that utilizes a rolling sequence of finite time-horizon constrained optimizations. This is based on the concept that we have more accurate estimates of the load forecast in the short term, so each optimization in the rolling sequence of optimization problems uses more accurate short term load forecasts while ensuring satisfaction of capacity and dynamical constraints. Simulations demonstrate that the MPC solution can indeed reduce the ramping required from bulk generation, while mitigating near-real time grid disturbances.


title: NegativePrompt: Leveraging Psychology for Large Language Models Enhancement via Negative Emotional Stimuli

abstract: Large Language Models (LLMs) have become integral to a wide spectrum of applications, ranging from traditional computing tasks to advanced artificial intelligence (AI) applications. This widespread adoption has spurred extensive research into LLMs across various disciplines, including the social sciences. Notably, studies have revealed that LLMs possess emotional intelligence, which can be further developed through positive emotional stimuli. This discovery raises an intriguing question: can negative emotions similarly influence LLMs, potentially enhancing their performance? In response to this question, we introduce NegativePrompt, a novel approach underpinned by psychological principles, involving ten specifically designed negative emotional stimuli. We embark on rigorous experimental evaluations of five LLMs including Flan-T5-Large, Vicuna, Llama 2, ChatGPT, and GPT-4, across a set of 45 tasks. The results are revealing: NegativePrompt markedly enhances the performance of LLMs, evidenced by relative improvements of 12.89% in Instruction Induction tasks and 46.25% in BIG-Bench tasks. Moreover, we conduct attention visualization experiments to decipher the underlying mechanisms of NegativePrompt's influence. Our research contributes significantly to the understanding of LLMs and emotion interaction, demonstrating the practical efficacy of NegativePrompt as an emotion-driven method and offering novel insights for the enhancement of LLMs in real-world applications. The code is available at https://github.com/wangxu0820/NegativePrompt.


title: Region-specific Risk Quantification for Interpretable Prognosis of COVID-19

abstract: The COVID-19 pandemic has strained global public health, necessitating accurate diagnosis and intervention to control disease spread and reduce mortality rates. This paper introduces an interpretable deep survival prediction model designed specifically for improved understanding and trust in COVID-19 prognosis using chest X-ray (CXR) images. By integrating a large-scale pretrained image encoder, Risk-specific Grad-CAM, and anatomical region detection techniques, our approach produces regional interpretable outcomes that effectively capture essential disease features while focusing on rare but critical abnormal regions. Our model's predictive results provide enhanced clarity and transparency through risk area localization, enabling clinicians to make informed decisions regarding COVID-19 diagnosis with better understanding of prognostic insights. We evaluate the proposed method on a multi-center survival dataset and demonstrate its effectiveness via quantitative and qualitative assessments, achieving superior C-indexes (0.764 and 0.727) and time-dependent AUCs (0.799 and 0.691). These results suggest that our explainable deep survival prediction model surpasses traditional survival analysis methods in risk prediction, improving interpretability for clinical decision making and enhancing AI system trustworthiness.


title: Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximization

abstract: This paper introduces Stochastic RAG--a novel approach for end-to-end optimization of retrieval-augmented generation (RAG) models that relaxes the simplifying assumptions of marginalization and document independence, made in most prior work. Stochastic RAG casts the retrieval process in RAG as a stochastic sampling without replacement process. Through this formulation, we employ straight-through Gumbel-top-k that provides a differentiable approximation for sampling without replacement and enables effective end-to-end optimization for RAG. We conduct extensive experiments on seven diverse datasets on a wide range of tasks, from open-domain question answering to fact verification to slot-filling for relation extraction and to dialogue systems. By applying this optimization method to a recent and effective RAG model, we advance state-of-the-art results on six out of seven datasets.


title: HuixiangDou-CR: Coreference Resolution in Group Chats

abstract: How to eliminate pronominal reference in group chats? In this work, we have preprocessed 58k authentic chat data and manually annotated 2.3k questions. The reliability of this annotation was confirmed by the scaling law. After this, we conducted fine-tuning on Qwen models, ranging from 0.5B to 32B parameters. The optimal version improved 29.07 in F1 score. This confirms the viability of fine-tuning Large Language Model (LLM) for downstream Natural Language Processing (NLP) tasks. Our contributions are: 1) Created Supervised Fine-Tuning (SFT) training data in alpaca format, along with a set of Low-Rank Adaptation (LoRA) weights, and 2) Developed a method for acquiring high-quality data leveraging scaling law principle. The script, raw data with alpaca format and experiments track are open-sourced on Github https://github.com/InternLM/HuixiangDou/tree/main/web/tools, HuggingFace https://huggingface.co/tpoisonooo and WandB https://wandb.ai/tpoisonooo/huixiangdou-cr/table?nw=nwusertpoisonooo . The privacy of the data involved has been authorized by users.


title: Site-Specific Deployment Optimization of Intelligent Reflecting Surface for Coverage Enhancement

abstract: Intelligent Reflecting Surface (IRS) is a promising technology for next generation wireless networks. Despite substantial research in IRS-aided communications, the assumed antenna and channel models are typically simplified without considering site-specific characteristics, which in turn critically affect the IRS deployment and performance in a given environment. In this paper, we first investigate the link-level performance of active or passive IRS taking into account the IRS element radiation pattern (ERP) as well as the antenna radiation pattern of the access point (AP). Then the network-level coverage performance is evaluated/optimized in site-specific multi-building scenarios, by properly deploying multiple IRSs on candidate building facets to serve a given set of users or Points of Interests (PoIs). The problem is reduced to an integer linear programming (ILP) based on given link-level metrics, which is then solved efficiently under moderate network sizes. Numerical results confirm the impact of AP antenna/IRS element pattern on the link-level performance. In addition, it is found that active IRSs, though associated with higher hardware complexity and cost, significantly improve the site-specific network coverage performance in terms of average ergodic rate and fairness among the PoIs as well as the range of serving area, compared with passive IRSs that have a much larger number of elements.


title: Authentic Majorana versus singlet Dirac neutrino contributions to $\mu^-\mu^-\to \ell^-\ell^-$ $(\ell=e,\tau)$ transitions

abstract: We revisited the computation of the cross section for the $\mu^-\mu^-\to \ell^-\ell^{-}$ $(\ell^-=\tau^-,\, e^-)$ transitions in the presence of new heavy Majorana neutrinos. We focus on scenarios where the masses of the new states are around some few TeV, the so-called low-scale seesaw models. Our analysis is set in a simple model with only two new heavy Majorana neutrinos considering the current limits on the heavy-light mixings. Our results illustrate the difference between the genuine effects of two nondegenerate heavy Majorana states and the degenerate case where they form a Dirac singlet field.


title: Coat stiffening explains the consensus pathway of clathrin-mediated endocytosis

abstract: Clathrin-mediated endocytosis is the main pathway used by eukaryotic cells to take up extracellular material, but the dominant physical mechanisms driving this process are still elusive. Recently several high-resolution imaging techniques have been used on different cell lines to measure the geometrical properties of clathrin-coated pits over their whole lifetime. Here we first show that all datasets follow the same consensus pathway, which is well described by the recently introduced cooperative curvature model, which predicts a flat-to curved transition at finite area, followed by linear growth and subsequent saturation of curvature. We then apply an energetic model for the composite of plasma membrane and clathrin coat to the consensus pathway to show that the dominant mechanism for invagination is coat stiffening, which results from cooperative interactions between the different clathrin molecules and progressively drives the system towards its intrinsic curvature. Our theory predicts that two length scales determine the time course of invagination, namely the patch size at which the flat-to-curved transition occurs and the final pit radius.


title: Sim2Real Transfer for Audio-Visual Navigation with Frequency-Adaptive Acoustic Field Prediction

abstract: Sim2real transfer has received increasing attention lately due to the success of learning robotic tasks in simulation end-to-end. While there has been a lot of progress in transferring vision-based navigation policies, the existing sim2real strategy for audio-visual navigation performs data augmentation empirically without measuring the acoustic gap. The sound differs from light in that it spans across much wider frequencies and thus requires a different solution for sim2real. We propose the first treatment of sim2real for audio-visual navigation by disentangling it into acoustic field prediction (AFP) and waypoint navigation. We first validate our design choice in the SoundSpaces simulator and show improvement on the Continuous AudioGoal navigation benchmark. We then collect real-world data to measure the spectral difference between the simulation and the real world by training AFP models that only take a specific frequency subband as input. We further propose a frequency-adaptive strategy that intelligently selects the best frequency band for prediction based on both the measured spectral difference and the energy distribution of the received audio, which improves the performance on the real data. Lastly, we build a real robot platform and show that the transferred policy can successfully navigate to sounding objects. This work demonstrates the potential of building intelligent agents that can see, hear, and act entirely from simulation, and transferring them to the real world.


title: A central limit theorem for partial transposes of multipartite Wishart matrices

abstract: The partial transposition from quantum information theory provides a new source to distill the so-called asymptotic freeness without the assumption of classical independence between random matrices. Indeed, a recent paper [MP19] established asymptotic freeness between partial transposes in the bipartite situation. In this paper, we prove almost sure asymptotic freeness in the general multipartite situation and establish a central limit theorem for the partial transposes.