title: Multistep Inverse Is Not All You Need

abstract: In real-world control settings, the observation space is often unnecessarily high-dimensional and subject to time-correlated noise. However, the controllable dynamics of the system are often far simpler than the dynamics of the raw observations. It is therefore desirable to learn an encoder to map the observation space to a simpler space of control-relevant variables. In this work, we consider the Ex-BMDP model, first proposed by Efroni et al. (2022), which formalizes control problems where observations can be factorized into an action-dependent latent state which evolves deterministically, and action-independent time-correlated noise. Lamb et al. (2022) proposes the "AC-State" method for learning an encoder to extract a complete action-dependent latent state representation from the observations in such problems. AC-State is a multistep-inverse method, in that it uses the encoding of the the first and last state in a path to predict the first action in the path. However, we identify cases where AC-State will fail to learn a correct latent representation of the agent-controllable factor of the state. We therefore propose a new algorithm, ACDF, which combines multistep-inverse prediction with a latent forward model. ACDF is guaranteed to correctly infer an action-dependent latent state encoder for a large class of Ex-BMDP models. We demonstrate the effectiveness of ACDF on tabular Ex-BMDPs through numerical simulations; as well as high-dimensional environments using neural-network-based encoders. Code is available at https://github.com/midi-lab/acdf.


title: Perfect Zero-Knowledge PCPs for #P

abstract: We construct perfect zero-knowledge probabilistically checkable proofs (PZK-PCPs) for every language in #P. This is the first construction of a PZK-PCP for any language outside BPP. Furthermore, unlike previous constructions of (statistical) zero-knowledge PCPs, our construction simultaneously achieves non-adaptivity and zero knowledge against arbitrary (adaptive) polynomial-time malicious verifiers. Our construction consists of a novel masked sumcheck PCP, which uses the combinatorial nullstellensatz to obtain antisymmetric structure within the hypercube and randomness outside of it. To prove zero knowledge, we introduce the notion of locally simulatable encodings: randomised encodings in which every local view of the encoding can be efficiently sampled given a local view of the message. We show that the code arising from the sumcheck protocol (the Reed-Muller code augmented with subcube sums) admits a locally simulatable encoding. This reduces the algebraic problem of simulating our masked sumcheck to a combinatorial property of antisymmetric functions.


title: Exploring Facial Expression Recognition through Semi-Supervised Pretraining and Temporal Modeling

abstract: Facial Expression Recognition (FER) plays a crucial role in computer vision and finds extensive applications across various fields. This paper aims to present our approach for the upcoming 6th Affective Behavior Analysis in-the-Wild (ABAW) competition, scheduled to be held at CVPR2024. In the facial expression recognition task, The limited size of the FER dataset poses a challenge to the expression recognition model's generalization ability, resulting in subpar recognition performance. To address this problem, we employ a semi-supervised learning technique to generate expression category pseudo-labels for unlabeled face data. At the same time, we uniformly sampled the labeled facial expression samples and implemented a debiased feedback learning strategy to address the problem of category imbalance in the dataset and the possible data bias in semi-supervised learning. Moreover, to further compensate for the limitation and bias of features obtained only from static images, we introduced a Temporal Encoder to learn and capture temporal relationships between neighbouring expression image features. In the 6th ABAW competition, our method achieved outstanding results on the official validation set, a result that fully confirms the effectiveness and competitiveness of our proposed method.


title: Galois groups of random polynomials over the rational function field

abstract: For a fixed prime power $q$ and natural number $d$ we consider a random polynomial $$f=x^n+a_{n-1}(t)x^{n-1}+\ldots+a_1(t)x+a_0(t)\in\mathbb F_q[t][x]$$ with $a_i$ drawn uniformly and independently at random from the set of all polynomials in $\mathbb F_q[t]$ of degree $\le d$. We show that with probability tending to 1 as $n\to\infty$ the Galois group $G_f$ of $f$ over $\mathbb F_q(t)$ is isomorphic to $S_{n-k}\times C$, where $C$ is cyclic, $k$ and $|C|$ are small quantities with a simple explicit dependence on $f$. As a corollary we deduce that $\mathbb P(G_f=S_n\,|\,f\mbox{ irreducible})\to 1$ as $n\to\infty$. Thus we are able to overcome the $S_n$ versus $A_n$ ambiguity in the most natural small box random polynomial model over $\mathbb F_q[t]$, which has not been achieved over $\mathbb Z$ so far.


title: The Share-a-Ride Problem with mixed ride-hailing and logistic vehicles

abstract: This study explores the potential of using ride-hailing vehicles (RVs) for integrated passenger and freight transport based on shared mobility. In this crowd-sourced mode, ride-hailing platforms can profit from parcel delivery services, and logistics companies can reduce operational costs by utilizing the capacities of RVs. The Share-a-Ride problem with ride-hailing and logistic vehicles (SARP-RL) determines the number of logistic vehicles (LVs) and the assignment of passenger/parcel requests to RVs and LVs, aiming at maximizing the total RV profits and minimizing logistic costs. An exact solution framework is proposed by (1) generating a feasible trip that serves a given set of requests at maximal profits; (2) generating all feasible trips for the entire set of passenger and parcel requests via an efficient enumeration method; and (3) finding all Pareto-optimal solutions of the bi-objective problem via an $\varepsilon$-constraint method. Not only is the proposed method exact, it also converts the NP-hard problem to a simple vehicle-trip matching problem. More importantly, the total computational time can be compressed to an arbitrary degree via straightforward parallelization. A case study of the Manhattan network demonstrates the solution characteristics of SARP-RL. The results indicate that: (i) Coordinating RV and LV operations to serve passenger and parcel requests (SARP-RL) can simultaneously reduce logistic costs and increase RV profits. (ii) Key factors influencing the performance of SARP-RL include the RV fleet size, spatial distribution of parcel requests, passenger/parcel request ratio, and unit price of transport service, which are quantitatively analyzed to offer managerial insights for real-world implementation.


title: Kernel Modelling of Fading Memory Systems

abstract: The paper introduces a kernel-based framework to model and identify time-invariant systems with the fading memory property. The key departure from the previous literature is to bypass the state-space representation of the model. Instead, a kernel representation is used to directly model the memory functional that maps past inputs to the present output. We explore the versatility of this approach to encode important system properties in the hyperparameters of the kernel. The approach is illustrated on the Hodgkin and Huxley model of neuronal excitability.


title: Atmospheric muons measured with the KM3NeT detectors in comparison with updated numeric predictions

abstract: The measurement of the flux of muons produced in cosmic ray air showers is essential for the study of primary cosmic rays. Such measurements are important in extensive air shower detectors to assess the energy spectrum and the chemical composition of the cosmic ray flux, complementary to the information provided by fluorescence detectors. Detailed simulations of the cosmic ray air showers are carried out, using codes such as CORSIKA, to estimate the muon flux at sea level. These simulations are based on the choice of hadronic interaction models, for which improvements have been implemented in the post-LHC era. In this work, a deficit in simulations that use state-of-the-art QCD models with respect to the measurement deep underwater with the KM3NeT neutrino detectors is reported. The KM3NeT/ARCA and KM3NeT/ORCA neutrino telescopes are sensitive to TeV muons originating mostly from primary cosmic rays with energies around 10 TeV. The predictions of state-of-the-art QCD models show that the deficit with respect to the data is constant in zenith angle; no dependency on the water overburden is observed. The observed deficit at a depth of several kilometres is compatible with the deficit seen in the comparison of the simulations and measurements at sea level.


title: Explainable Reinforcement Learning-based Home Energy Management Systems using Differentiable Decision Trees

abstract: With the ongoing energy transition, demand-side flexibility has become an important aspect of the modern power grid for providing grid support and allowing further integration of sustainable energy sources. Besides traditional sources, the residential sector is another major and largely untapped source of flexibility, driven by the increased adoption of solar PV, home batteries, and EVs. However, unlocking this residential flexibility is challenging as it requires a control framework that can effectively manage household energy consumption, and maintain user comfort while being readily scalable across different, diverse houses. We aim to address this challenging problem and introduce a reinforcement learning-based approach using differentiable decision trees. This approach integrates the scalability of data-driven reinforcement learning with the explainability of (differentiable) decision trees. This leads to a controller that can be easily adapted across different houses and provides a simple control policy that can be explained to end-users, further improving user acceptance. As a proof-of-concept, we analyze our method using a home energy management problem, comparing its performance with commercially available rule-based baseline and standard neural network-based RL controllers. Through this preliminary study, we show that the performance of our proposed method is comparable to standard RL-based controllers, outperforming baseline controllers by ~20% in terms of daily cost savings while being straightforward to explain.


title: Learning Dynamical Systems Encoding Non-Linearity within Space Curvature

abstract: Dynamical Systems (DS) are an effective and powerful means of shaping high-level policies for robotics control. They provide robust and reactive control while ensuring the stability of the driving vector field. The increasing complexity of real-world scenarios necessitates DS with a higher degree of non-linearity, along with the ability to adapt to potential changes in environmental conditions, such as obstacles. Current learning strategies for DSs often involve a trade-off, sacrificing either stability guarantees or offline computational efficiency in order to enhance the capabilities of the learned DS. Online local adaptation to environmental changes is either not taken into consideration or treated as a separate problem. In this paper, our objective is to introduce a method that enhances the complexity of the learned DS without compromising efficiency during training or stability guarantees. Furthermore, we aim to provide a unified approach for seamlessly integrating the initially learned DS's non-linearity with any local non-linearities that may arise due to changes in the environment. We propose a geometrical approach to learn asymptotically stable non-linear DS for robotics control. Each DS is modeled as a harmonic damped oscillator on a latent manifold. By learning the manifold's Euclidean embedded representation, our approach encodes the non-linearity of the DS within the curvature of the space. Having an explicit embedded representation of the manifold allows us to showcase obstacle avoidance by directly inducing local deformations of the space. We demonstrate the effectiveness of our methodology through two scenarios: first, the 2D learning of synthetic vector fields, and second, the learning of 3D robotic end-effector motions in real-world settings.


title: Higher time-derivative theories from space-time interchanged integrable field theories

abstract: We compare a relativistic and a nonrelativistic version of Ostrogradsky's method for higher-time derivative theories extended to scalar field theories and consider as an alternative a multi-field variant. We apply the schemes to space-time rotated modified Korteweg-de Vries systems and, exploiting their integrability, to Hamiltonian systems built from space-time rotated inverse Legendre transformed higher-order charges of these systems. We derive the equal-time Poisson bracket structures of these theories, establish the integrability of the latter theories by means of the Painlev\'e test and construct exact analytical period benign solutions in terms of Jacobi elliptic functions to the classical equations of motion. The classical energies of these partially complex solutions are real when they respect a certain modified CPT-symmetry and complex when this symmetry is broken. The higher order Cauchy and initial-boundary value problem are addressed analytically and numerically. Finally, we provide the explicit quantization of the simplest mKdV system, exhibiting the usual conundrum of having the choice between either having to deal with a theory that includes non-normalizable states or spectra that are unbounded from below. In our non-Hermitian system the choice is dictated by the correct sign in the decay width.


title: Fusion of deterministically generated photonic graph states

abstract: Entanglement has evolved from an enigmatic concept of quantum physics to a key ingredient of quantum technology. It explains correlations between measurement outcomes that contradict classical physics, and has been widely explored with small sets of individual qubits. Multi-partite entangled states build up in gate-based quantum-computing protocols, and $\unicode{x2013}$ from a broader perspective $\unicode{x2013}$ were proposed as the main resource for measurement-based quantum-information processing. The latter requires the ex-ante generation of a multi-qubit entangled state described by a graph. Small graph states such as Bell or linear cluster states have been produced with photons, but the proposed quantum computing and quantum networking applications require fusion of such states into larger and more powerful states in a programmable fashion. Here we achieve this goal by employing an optical resonator containing two individually addressable atoms. Ring and tree graph states with up to eight qubits, with the names reflecting the entanglement topology, are efficiently fused from the photonic states emitted by the individual atoms. The fusion process itself employs a cavity-assisted gate between the two atoms. Our technique is in principle scalable to even larger numbers of qubits, and is the decisive step towards, for instance, a memory-less quantum repeater in a future quantum internet.


title: Displacement Field Analysis via Optimal Transport: Multi-Tracer Approach to Cosmological Reconstruction

abstract: We demonstrate the effectiveness of one of the many multi-tracer analyses enabled by Optimal Transport (OT) reconstruction. Leveraging a semi-discrete OT algorithm, we determine the displacements between initial and observed positions of biased tracers and the remaining matter field. With only redshift-space distorted final positions of biased tracers and a simple premise for the remaining mass distribution as input, OT solves the displacement field. This extracted field, assuming asymptotically uniform density and a gradient flow displacement, enables reconstruction of the initial overdensity fluctuation field. We show that the divergence of the OT displacement field is a good proxy of the linear density field, even though the method never assumes the linear theory growth. Additionally, this divergence field can be combined with the reconstructed protohalos to provide a higher signal-to-noise measurement of the BAO standard ruler than was possible with either measurement individually.


title: Exploring Estonia's Open Government Data Development as a Journey towards Excellence: Unveiling the Progress of Local Governments in Open Data Provision

abstract: Estonia has a global reputation of a digital state or e-country. However, despite the success in digital governance, the country has faced challenges in the realm of Open Government Data (OGD) area, with significant advancements in its OGD ecosystem, as reflected in various open data rankings from 2020 and onwards, in the recent years being recognized among trend-setters. This paper aims to explore the evolution and positioning of Estonia's OGD development, encompassing national and local levels, through an integrated analysis of various indices, primary data from the Estonian OGD portal, and a thorough literature review. The research shows that Estonia has made progress in the national level open data ecosystem, primarily due to improvements in the OGD portal usability and legislation amendments. However, the local level is not as developed, with local governments lagging behind in OGD provision. The literature review highlights the lack of previous research focusing on Estonian and European local open data, emphasizing the need for future studies to explore the barriers and enablers of municipal OGD. This study contributes to a nuanced understanding of Estonia's dynamic journey in the OGD landscape, shedding light on both achievements and areas warranting further attention for establishing a sustainable open data ecosystem.


title: Advancing COVID-19 Detection in 3D CT Scans

abstract: To make a more accurate diagnosis of COVID-19, we propose a straightforward yet effective model. Firstly, we analyse the characteristics of 3D CT scans and remove the non-lung parts, facilitating the model to focus on lesion-related areas and reducing computational cost. We use ResNeSt50 as the strong feature extractor, initializing it with pretrained weights which have COVID-19-specific prior knowledge. Our model achieves a Macro F1 Score of 0.94 on the validation set of the 4th COV19D Competition Challenge $\mathrm{I}$, surpassing the baseline by 16%. This indicates its effectiveness in distinguishing between COVID-19 and non-COVID-19 cases, making it a robust method for COVID-19 detection.


title: Robust Estimation and Inference in Categorical Data

abstract: In empirical science, many variables of interest are categorical. Like any model, models for categorical responses can be misspecified, leading to possibly large biases in estimation. One particularly troublesome source of misspecification is inattentive responding in questionnaires, which is well-known to jeopardize the validity of structural equation models (SEMs) and other survey-based analyses. I propose a general estimator that is designed to be robust to misspecification of models for categorical responses. Unlike hitherto approaches, the estimator makes no assumption whatsoever on the degree, magnitude, or type of misspecification. The proposed estimator generalizes maximum likelihood estimation, is strongly consistent, asymptotically Gaussian, has the same time complexity as maximum likelihood, and can be applied to any model for categorical responses. In addition, I develop a novel test that tests whether a given response can be fitted well by the assumed model, which allows one to trace back possible sources of misspecification. I verify the attractive theoretical properties of the proposed methodology in Monte Carlo experiments, and demonstrate its practical usefulness in an empirical application on a SEM of personality traits, where I find compelling evidence for the presence of inattentive responding whose adverse effects the proposed estimator can withstand, unlike maximum likelihood.


title: Inferring Belief States in Partially-Observable Human-Robot Teams

abstract: We investigate the real-time estimation of human situation awareness using observations from a robot teammate with limited visibility. In human factors and human-autonomy teaming, it is recognized that individuals navigate their environments using an internal mental simulation, or mental model. The mental model informs cognitive processes including situation awareness, contextual reasoning, and task planning. In teaming domains, the mental model includes a team model of each teammate's beliefs and capabilities, enabling fluent teamwork without the need for explicit communication. However, little work has applied team models to human-robot teaming. We compare the performance of two current methods at estimating user situation awareness over varying visibility conditions. Our results indicate that the methods are largely resilient to low-visibility conditions in our domain, however opportunities exist to improve their overall performance.


title: Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment

abstract: With the rapid development of generative models, Artificial Intelligence-Generated Contents (AIGC) have exponentially increased in daily lives. Among them, Text-to-Video (T2V) generation has received widespread attention. Though many T2V models have been released for generating high perceptual quality videos, there is still lack of a method to evaluate the quality of these videos quantitatively. To solve this issue, we establish the largest-scale Text-to-Video Quality Assessment DataBase (T2VQA-DB) to date. The dataset is composed of 10,000 videos generated by 9 different T2V models. We also conduct a subjective study to obtain each video's corresponding mean opinion score. Based on T2VQA-DB, we propose a novel transformer-based model for subjective-aligned Text-to-Video Quality Assessment (T2VQA). The model extracts features from text-video alignment and video fidelity perspectives, then it leverages the ability of a large language model to give the prediction score. Experimental results show that T2VQA outperforms existing T2V metrics and SOTA video quality assessment models. Quantitative analysis indicates that T2VQA is capable of giving subjective-align predictions, validating its effectiveness. The dataset and code will be released at https://github.com/QMME/T2VQA.


title: Relative aspherical conjecture and higher codimensional obstruction to positive scalar curvature

abstract: Motivated by the solution of the aspherical conjecture up to dimension 5 [CL20][Gro20], we want to study a relative version of the aspherical conjecture. We present a natural condition generalizing the model $X\times\mathbb{T}^k$ to the relative aspherical setting. Such model is closely related to submanifold obstruction of positive scalar curvature (PSC), and would be in similar spirit as [HPS15][CRZ23] in codim 2 case. In codim 3 and 4, we prove results on how 3-manifold obstructs the existence of PSC under our relative aspherical condition, the proof of which relies on a newly introduced geometric quantity called the it spherical width. This could be regarded as a relative version extension of the aspherical conjecture up to dim 5.


title: Language Evolution with Deep Learning

abstract: Computational modeling plays an essential role in the study of language emergence. It aims to simulate the conditions and learning processes that could trigger the emergence of a structured language within a simulated controlled environment. Several methods have been used to investigate the origin of our language, including agent-based systems, Bayesian agents, genetic algorithms, and rule-based systems. This chapter explores another class of computational models that have recently revolutionized the field of machine learning: deep learning models. The chapter introduces the basic concepts of deep and reinforcement learning methods and summarizes their helpfulness for simulating language emergence. It also discusses the key findings, limitations, and recent attempts to build realistic simulations. This chapter targets linguists and cognitive scientists seeking an introduction to deep learning as a tool to investigate language evolution.


title: IVAC-P2L: Leveraging Irregular Repetition Priors for Improving Video Action Counting

abstract: Video Action Counting (VAC) is crucial in analyzing sports, fitness, and everyday activities by quantifying repetitive actions in videos. However, traditional VAC methods have overlooked the complexity of action repetitions, such as interruptions and the variability in cycle duration. Our research addresses the shortfall by introducing a novel approach to VAC, called Irregular Video Action Counting (IVAC). IVAC prioritizes modeling irregular repetition patterns in videos, which we define through two primary aspects: Inter-cycle Consistency and Cycle-interval Inconsistency. Inter-cycle Consistency ensures homogeneity in the spatial-temporal representations of cycle segments, signifying action uniformity within cycles. Cycle-interval inconsistency highlights the importance of distinguishing between cycle segments and intervals based on their inherent content differences. To encapsulate these principles, we propose a new methodology that includes consistency and inconsistency modules, supported by a unique pull-push loss (P2L) mechanism. The IVAC-P2L model applies a pull loss to promote coherence among cycle segment features and a push loss to clearly distinguish features of cycle segments from interval segments. Empirical evaluations conducted on the RepCount dataset demonstrate that the IVAC-P2L model sets a new benchmark in VAC task performance. Furthermore, the model demonstrates exceptional adaptability and generalization across various video contents, outperforming existing models on two additional datasets, UCFRep and Countix, without the need for dataset-specific optimization. These results confirm the efficacy of our approach in addressing irregular repetitions in videos and pave the way for further advancements in video analysis and understanding.


title: CASPER: Causality-Aware Spatiotemporal Graph Neural Networks for Spatiotemporal Time Series Imputation

abstract: Spatiotemporal time series is the foundation of understanding human activities and their impacts, which is usually collected via monitoring sensors placed at different locations. The collected data usually contains missing values due to various failures, which have significant impact on data analysis. To impute the missing values, a lot of methods have been introduced. When recovering a specific data point, most existing methods tend to take into consideration all the information relevant to that point regardless of whether they have a cause-and-effect relationship. During data collection, it is inevitable that some unknown confounders are included, e.g., background noise in time series and non-causal shortcut edges in the constructed sensor network. These confounders could open backdoor paths between the input and output, in other words, they establish non-causal correlations between the input and output. Over-exploiting these non-causal correlations could result in overfitting and make the model vulnerable to noises. In this paper, we first revisit spatiotemporal time series imputation from a causal perspective, which shows the causal relationships among the input, output, embeddings and confounders. Next, we show how to block the confounders via the frontdoor adjustment. Based on the results of the frontdoor adjustment, we introduce a novel Causality-Aware SPatiotEmpoRal graph neural network (CASPER), which contains a novel Spatiotemporal Causal Attention (SCA) and a Prompt Based Decoder (PBD). PBD could reduce the impact of confounders and SCA could discover the sparse causal relationships among embeddings. Theoretical analysis reveals that SCA discovers causal relationships based on the values of gradients. We evaluate Casper on three real-world datasets, and the experimental results show that Casper outperforms the baselines and effectively discovers causal relationships.


title: Enhanced Event-Based Video Reconstruction with Motion Compensation

abstract: Deep neural networks for event-based video reconstruction often suffer from a lack of interpretability and have high memory demands. A lightweight network called CISTA-LSTC has recently been introduced showing that high-quality reconstruction can be achieved through the systematic design of its architecture. However, its modelling assumption that input signals and output reconstructed frame share the same sparse representation neglects the displacement caused by motion. To address this, we propose warping the input intensity frames and sparse codes to enhance reconstruction quality. A CISTA-Flow network is constructed by integrating a flow network with CISTA-LSTC for motion compensation. The system relies solely on events, in which predicted flow aids in reconstruction and then reconstructed frames are used to facilitate flow estimation. We also introduce an iterative training framework for this combined system. Results demonstrate that our approach achieves state-of-the-art reconstruction accuracy and simultaneously provides reliable dense flow estimation. Furthermore, our model exhibits flexibility in that it can integrate different flow networks, suggesting its potential for further performance enhancement.


title: Extrinsically homogeneous Lagrangian submanifolds of the pseudo-nearly K\"ahler $\mathrm{SL}(2,\mathbb{R})\times\mathrm{SL}(2,\mathbb{R})$

abstract: We consider the pseudo-nearly K\"ahler $\mathrm{SL}(2,\mathbb{R})\times\mathrm{SL}(2,\mathbb{R})$ and we study its Lagrangian submanifolds. We provide examples of Lagrangian submanifolds which do not have an analogue in $\mathbb{S}^3\times\mathbb{S}^3$. We also provide an expression for the isometry group of $\mathrm{SL}(2,\mathbb{R})\times\mathrm{SL}(2,\mathbb{R})$ with the pseudo-Riemannian nearly K\"ahler metric. The main result is a complete classification of extrinsically homogeneous Lagrangian submanifolds in this space.


title: Transfer Learning Beyond Bounded Density Ratios

abstract: We study the fundamental problem of transfer learning where a learning algorithm collects data from some source distribution $P$ but needs to perform well with respect to a different target distribution $Q$. A standard change of measure argument implies that transfer learning happens when the density ratio $dQ/dP$ is bounded. Yet, prior thought-provoking works by Kpotufe and Martinet (COLT, 2018) and Hanneke and Kpotufe (NeurIPS, 2019) demonstrate cases where the ratio $dQ/dP$ is unbounded, but transfer learning is possible. In this work, we focus on transfer learning over the class of low-degree polynomial estimators. Our main result is a general transfer inequality over the domain $\mathbb{R}^n$, proving that non-trivial transfer learning for low-degree polynomials is possible under very mild assumptions, going well beyond the classical assumption that $dQ/dP$ is bounded. For instance, it always applies if $Q$ is a log-concave measure and the inverse ratio $dP/dQ$ is bounded. To demonstrate the applicability of our inequality, we obtain new results in the settings of: (1) the classical truncated regression setting, where $dQ/dP$ equals infinity, and (2) the more recent out-of-distribution generalization setting for in-context learning linear functions with transformers. We also provide a discrete analogue of our transfer inequality on the Boolean Hypercube $\{-1,1\}^n$, and study its connections with the recent problem of Generalization on the Unseen of Abbe, Bengio, Lotfi and Rizk (ICML, 2023). Our main conceptual contribution is that the maximum influence of the error of the estimator $\widehat{f}-f^*$ under $Q$, $\mathrm{I}_{\max}(\widehat{f}-f^*)$, acts as a sufficient condition for transferability; when $\mathrm{I}_{\max}(\widehat{f}-f^*)$ is appropriately bounded, transfer is possible over the Boolean domain.


title: Probabilistic Calibration by Design for Neural Network Regression

abstract: Generating calibrated and sharp neural network predictive distributions for regression problems is essential for optimal decision-making in many real-world applications. To address the miscalibration issue of neural networks, various methods have been proposed to improve calibration, including post-hoc methods that adjust predictions after training and regularization methods that act during training. While post-hoc methods have shown better improvement in calibration compared to regularization methods, the post-hoc step is completely independent of model training. We introduce a novel end-to-end model training procedure called Quantile Recalibration Training, integrating post-hoc calibration directly into the training process without additional parameters. We also present a unified algorithm that includes our method and other post-hoc and regularization methods, as particular cases. We demonstrate the performance of our method in a large-scale experiment involving 57 tabular regression datasets, showcasing improved predictive accuracy while maintaining calibration. We also conduct an ablation study to evaluate the significance of different components within our proposed method, as well as an in-depth analysis of the impact of the base model and different hyperparameters on predictive accuracy.


title: Effects of strangeness on the chiral pseudocritical line

abstract: Within a 2+1 flavor Nambu\textendash Jona-Lasinio model, we calculate the curvature coefficients and check them against available lattice QCD estimations. With the observation that the flavor mixing due to the `t Hooft determinant term significantly affects the $\kappa_{2}^{S}$, we explore the effect of $\mu_{S}$ on the $T-\mu_{B}$ crossover lines. With the novel determination of negative $\kappa_{2}^{B}$ at large $\mu_{S}$, we advocate the importance of studying the same in lattice QCD.


title: Informed Spectral Normalized Gaussian Processes for Trajectory Prediction

abstract: Prior parameter distributions provide an elegant way to represent prior expert and world knowledge for informed learning. Previous work has shown that using such informative priors to regularize probabilistic deep learning (DL) models increases their performance and data-efficiency. However, commonly used sampling-based approximations for probabilistic DL models can be computationally expensive, requiring multiple inference passes and longer training times. Promising alternatives are compute-efficient last layer kernel approximations like spectral normalized Gaussian processes (SNGPs). We propose a novel regularization-based continual learning method for SNGPs, which enables the use of informative priors that represent prior knowledge learned from previous tasks. Our proposal builds upon well-established methods and requires no rehearsal memory or parameter expansion. We apply our informed SNGP model to the trajectory prediction problem in autonomous driving by integrating prior drivability knowledge. On two public datasets, we investigate its performance under diminishing training data and across locations, and thereby demonstrate an increase in data-efficiency and robustness to location-transfers over non-informed and informed baselines.


title: Probing Site-Resolved Current in Strongly Interacting Superconducting Circuit Lattices

abstract: Transport measurements are fundamental for understanding condensed matter phenomena, from superconductivity to the fractional quantum Hall effect. Analogously, they can be powerful tools for probing synthetic quantum matter in quantum simulators. Here we demonstrate the measurement of in-situ particle current in a superconducting circuit lattice and apply it to study transport in both coherent and bath-coupled lattices. Our method utilizes controlled tunneling in a double-well potential to map current to on-site density, revealing site-resolved current and current statistics. We prepare a strongly interacting Bose-Hubbard lattice at different lattice fillings, and observe the change in current statistics as the many-body states transition from superfluid to Mott insulator. Furthermore, we explore non-equilibrium current dynamics by coupling the lattice to engineered driven-dissipative baths that serve as tunable particle source and drain. We observe steady-state current in discrete conduction channels and interaction-assisted transport. These results establish a versatile platform to investigate microscopic quantum transport in superconducting circuits.


title: Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory

abstract: Conditional diffusion models serve as the foundation of modern image synthesis and find extensive application in fields like computational biology and reinforcement learning. In these applications, conditional diffusion models incorporate various conditional information, such as prompt input, to guide the sample generation towards desired properties. Despite the empirical success, theory of conditional diffusion models is largely missing. This paper bridges this gap by presenting a sharp statistical theory of distribution estimation using conditional diffusion models. Our analysis yields a sample complexity bound that adapts to the smoothness of the data distribution and matches the minimax lower bound. The key to our theoretical development lies in an approximation result for the conditional score function, which relies on a novel diffused Taylor approximation technique. Moreover, we demonstrate the utility of our statistical theory in elucidating the performance of conditional diffusion models across diverse applications, including model-based transition kernel estimation in reinforcement learning, solving inverse problems, and reward conditioned sample generation.


title: Mechanisms enabling efficient fountain-flow supracellular migration

abstract: In a prototypical mode of single-cell migration, retrograde cytoskeletal flow is mechanically coupled to the environment, propels the cell, and is sustained by an anterograde cytosolic flow of disassembled cytoskeletal components. Supracellular collectives also develop fountain-flows to migrate, but the opposing cellular streams interact with the environment producing conflicting forces. To understand the biophysical constraints of fountain-flow supracellular migration, we develop an active gel model of a cell cluster driven by a polarized peripheral contractile cable. While the model develops fountain-flows and directed migration, efficiency and cluster velocity are extremely small compared to observations. We find that patterned friction or cluster-polarized single-cell directed migration, both suggested by contact inhibition of locomotion, rescue robust and efficient supracellular migration.


title: A Maximum Entropy Principle in Deep Thermalization and in Hilbert-Space Ergodicity

abstract: We report universal statistical properties displayed by ensembles of pure states that naturally emerge in quantum many-body systems. Specifically, two classes of state ensembles are considered: those formed by i) the temporal trajectory of a quantum state under unitary evolution or ii) the quantum states of small subsystems obtained by partial, local projective measurements performed on their complements. These cases respectively exemplify the phenomena of "Hilbert-space ergodicity" and "deep thermalization." In both cases, the resultant ensembles are defined by a simple principle: the distributions of pure states have maximum entropy, subject to constraints such as energy conservation, and effective constraints imposed by thermalization. We present and numerically verify quantifiable signatures of this principle by deriving explicit formulae for all statistical moments of the ensembles; proving the necessary and sufficient conditions for such universality under widely-accepted assumptions; and describing their measurable consequences in experiments. We further discuss information-theoretic implications of the universality: our ensembles have maximal information content while being maximally difficult to interrogate, establishing that generic quantum state ensembles that occur in nature hide (scramble) information as strongly as possible. Our results generalize the notions of Hilbert-space ergodicity to time-independent Hamiltonian dynamics and deep thermalization from infinite to finite effective temperature. Our work presents new perspectives to characterize and understand universal behaviors of quantum dynamics using statistical and information theoretic tools.


title: Universal fluctuations and noise learning from Hilbert-space ergodicity

abstract: Systems reaching thermal equilibrium are ubiquitous. For classical systems, this phenomenon is typically understood statistically through ergodicity in phase space, but translating this to quantum systems is a long-standing problem of interest. Recently a quantum notion of ergodicity has been proposed, namely that isolated, global quantum states uniformly explore their available state space, dubbed Hilbert-space ergodicity. Here we observe signatures of this process with an experimental Rydberg quantum simulator and various numerical models, before generalizing to the case of a local quantum system interacting with its environment. For a closed system, where the environment is a complementary subsystem, we predict and observe a smooth quantum-to-classical transition in that observables progress from large, quantum fluctuations to small, Gaussian fluctuations as the bath size grows. This transition is universal on a quantitative level amongst a wide range of systems, including those at finite temperature, those with itinerant particles, and random circuits. Then, we consider the case of an open system interacting noisily with an external environment. We predict the statistics of observables under largely arbitrary noise channels including those with correlated errors, allowing us to discriminate candidate error models both for continuous Hamiltonian time evolution and for digital random circuits. Ultimately our results clarify the role of ergodicity in quantum dynamics, with fundamental and practical consequences.


title: Typing supernova remnant G352.7$-$0.1 using XMM-Newton X-ray observations

abstract: G352.7$-$0.1 is a mixed-morphology (MM) supernova remnant (SNR) with multiple radio arcs and has a disputed supernova origin. We conducted a spatially resolved spectroscopic study of the remnant with XMM-Newton X-ray data to investigate its explosion mechanism and explain its morphology. The global X-ray spectra of the SNR can be adequately reproduced using a metal-rich thermal plasma model with a temperature of $\sim 2$ keV and ionization timescale of $\sim 3\times 10^{10}~{\rm cm^{-3}~s}$. Through a comparison with various supernova nucleosynthesis models, we found that observed metal properties from Mg to Fe can be better described using core-collapse supernova models, while thermonuclear models fail to explain the observed high Mg/Si ratio. The best-fit supernova model suggests a $\sim 13$ $M_\odot$ progenitor star, consistent with previous estimates using the wind bubble size. We also discussed the possible mechanisms that may lead to SNR G352.7$-$0.1 being an MMSNR. By dividing the SNR into several regions, we found that the temperature and abundance do not significantly vary with regions, except for a decreased temperature and abundance in a region interacting with molecular clouds. The brightest X-ray emission of the SNR spatially matches with the inner radio structure, suggesting that the centrally filled X-ray morphology results from a projection effect.


title: Quantum reference frames, measurement schemes and the type of local algebras in quantum field theory

abstract: We develop an operational framework, combining relativistic quantum measurement theory with quantum reference frames (QRFs), in which local measurements of a quantum field on a background with symmetries are performed relative to a QRF. This yields a joint algebra of quantum-field and reference-frame observables that is invariant under the natural action of the group of spacetime isometries. For the appropriate class of quantum reference frames, this algebra is parameterised in terms of crossed products. Provided that the quantum field has good thermal properties (expressed by the existence of a KMS state at some nonzero temperature), one can use modular theory to show that the invariant algebra admits a semifinite trace. If furthermore the quantum reference frame has good thermal behaviour (expressed by the existence of a KMS weight) at the same temperature, this trace is finite. We give precise conditions for the invariant algebra of physical observables to be a type $\textnormal{II}_1$ factor. Our results build upon recent work of Chandrasekaran, Longo, Penington and Witten [JHEP 2023, 82 (2023)], providing both a significant mathematical generalisation of these findings and a refined operational understanding of their model.


title: OUCopula: Bi-Channel Multi-Label Copula-Enhanced Adapter-Based CNN for Myopia Screening Based on OU-UWF Images

abstract: Myopia screening using cutting-edge ultra-widefield (UWF) fundus imaging is potentially significant for ophthalmic outcomes. Current multidisciplinary research between ophthalmology and deep learning (DL) concentrates primarily on disease classification and diagnosis using single-eye images, largely ignoring joint modeling and prediction for Oculus Uterque (OU, both eyes). Inspired by the complex relationships between OU and the high correlation between the (continuous) outcome labels (Spherical Equivalent and Axial Length), we propose a framework of copula-enhanced adapter convolutional neural network (CNN) learning with OU UWF fundus images (OUCopula) for joint prediction of multiple clinical scores. We design a novel bi-channel multi-label CNN that can (1) take bi-channel image inputs subject to both high correlation and heterogeneity (by sharing the same backbone network and employing adapters to parameterize the channel-wise discrepancy), and (2) incorporate correlation information between continuous output labels (using a copula). Solid experiments show that OUCopula achieves satisfactory performance in myopia score prediction compared to backbone models. Moreover, OUCopula can far exceed the performance of models constructed for single-eye inputs. Importantly, our study also hints at the potential extension of the bi-channel model to a multi-channel paradigm and the generalizability of OUCopula across various backbone CNNs.


title: Strongly-ordered infrared counterterms from factorisation

abstract: In the context of infrared subtraction algorithms beyond next-to-leading order, it becomes necessary to consider multiple infrared limits of scattering amplitudes, in which several particles become soft or collinear in a strongly-ordered sequence. We study these limits from the point of view of infrared factorisation, and we provide general definitions of strongly-ordered soft and collinear kernels in terms of gauge-invariant operator matrix elements. With these definitions in hand, it is possible to construct local subtraction counterterms for strongly-ordered configurations. Because of their factorised structure, these counterterms cancel infrared poles of real-virtual contributions by construction. We test these ideas at tree level for multiple emissions, and at one loop for single and double emissions, contributing to NNLO and N3LO distributions, respectively.


title: On the upper bound of wavefront sets of representations of p-adic groups

abstract: In this paper we study the upper bound of wavefront sets of irreducible admissible representations of connected reductive groups defined over non-Archimedean local fields of characteristic zero. We formulate a new conjecture on the upper bound and show that it can be reduced to that of anti-discrete series representations, namely, those whose Aubert-Zelevinsky duals are discrete series. Then, we show that this conjecture is equivalent to the Jiang conjecture on the upper bound of wavefront sets of representations in local Arthur packets and also equivalent to an analogous conjecture on the upper bound of wavefront sets of representations in local ABV packets.


title: Uncovering the lowest thickness limit for room-temperature ferromagnetism of Cr$_{1.6}$Te$_{2}$

abstract: Metallic ferromagnetic transition metal dichalcogenides have emerged as important building blocks for scalable magnonics and memory applications. Downscaling such systems to the ultra-thin limit is critical to integrate them into technology. Here, we achieved layer-by-layer control over the transition metal dichalcogenide Cr$_{1.6}$Te$_{2}$ by using pulsed laser deposition, and we uncovered the minimum critical thickness above which room temperature magnetic order is maintained. The electronic and magnetic structure is explored experimentally and theoretically and it is shown that the films exhibit strong in-plane magnetic anisotropy as a consequence of large spin-orbit effects. Our study elucidates both magnetic and electronic properties of Cr$_{1.6}$Te$_{2}$, and corroborates the importance of intercalation to tune the magnetic properties of nanoscale materials architectures.


title: Pedestrian Tracking with Monocular Camera using Unconstrained 3D Motion Model

abstract: A first-principle single-object model is proposed for pedestrian tracking. It is assumed that the extent of the moving object can be described via known statistics in 3D, such as pedestrian height. The proposed model thus need not constrain the object motion in 3D to a common ground plane, which is usual in 3D visual tracking applications. A nonlinear filter for this model is implemented using the unscented Kalman filter (UKF) and tested using the publicly available MOT-17 dataset. The proposed solution yields promising results in 3D while maintaining perfect results when projected into the 2D image. Moreover, the estimation error covariance matches the true one. Unlike conventional methods, the introduced model parameters have convenient meaning and can readily be adjusted for a problem.


title: Enlightening the blind spot of the Michaelis-Menten rate law: The role of relaxation dynamics in molecular complex formation

abstract: The century-long Michaelis-Menten rate law and its modifications in the modeling of biochemical rate processes stand on the assumption that the concentration of the complex of interacting molecules, at each moment, rapidly approaches an equilibrium (quasi-steady state) compared to the pace of molecular concentration changes. Yet, in the case of actively time-varying molecular concentrations with transient or oscillatory dynamics, the deviation of the complex profile from the quasi-steady state becomes relevant. A recent theoretical approach, known as the effective time-delay scheme (ETS), suggests that the delay by the relaxation time of molecular complex formation contributes to the substantial breakdown of the quasi-steady state assumption. Here, we systematically expand this ETS and inquire into the comprehensive roles of relaxation dynamics in complex formation. Through the modeling of rhythmic protein-protein and protein-DNA interactions and the mammalian circadian clock, our analysis reveals the effect of the relaxation dynamics beyond the time delay, which extends to the dampening of changes in the complex concentration with a reduction in the oscillation amplitude against the quasi-steady state. Interestingly, the combined effect of the time delay and amplitude reduction shapes both qualitative and quantitative oscillatory patterns such as the emergence and variability of the mammalian circadian rhythms. These findings highlight the drawback of the routine assumption of quasi-steady states and enhance the mechanistic understanding of rich time-varying biomolecular activities.

categories: q-bio.MN physics.bio-ph q-bio.BM q-bio.SC


title: Comparative Raman Scattering Study of Crystal Field Excitations in Co-based Quantum Magnets

abstract: Co-based materials have recently been explored due to potential to realise complex bond-dependent anisotropic magnetism. Prominent examples include Na$_2$Co$_2$TeO$_6$, BaCo$_2$(AsO$_4$)$_2$, Na$_2$BaCo(PO$_4$)$_2$, and CoX$_2$ (X = Cl, Br, I). In order to provide insight into the magnetic interactions in these compounds, we make a comparative analysis of their local crystal electric field excitations spectra via Raman scattering measurements. Combining these measurements with theoretical analysis confirms the validity of $j_{\rm eff} = 1/2$ single-ion ground states for all compounds, and provides accurate experimental estimates of the local crystal distortions, which play a prominent role in the magnetic couplings between spin-orbital coupled Co moments.


title: Diffusion Denoising as a Certified Defense against Clean-label Poisoning

abstract: We present a certified defense to clean-label poisoning attacks. These attacks work by injecting a small number of poisoning samples (e.g., 1%) that contain $p$-norm bounded adversarial perturbations into the training data to induce a targeted misclassification of a test-time input. Inspired by the adversarial robustness achieved by $denoised$ $smoothing$, we show how an off-the-shelf diffusion model can sanitize the tampered training data. We extensively test our defense against seven clean-label poisoning attacks and reduce their attack success to 0-16% with only a negligible drop in the test time accuracy. We compare our defense with existing countermeasures against clean-label poisoning, showing that the defense reduces the attack success the most and offers the best model utility. Our results highlight the need for future work on developing stronger clean-label attacks and using our certified yet practical defense as a strong baseline to evaluate these attacks.


title: Enhanced Electron Extraction in Co-Doped TiO2 Quantified by Drift-Diffusion Simulation for Stable CsPbI3 Solar Cells

abstract: Solar cells based on inorganic perovskite CsPbI3 are promising candidates to resolve the challenge of operational stability in the field of perovskite photovoltaics. For stable operation, however, it is crucial to thoroughly understand the extractive and recombinative processes occurring at the interfaces of perovskite and the charge-selective layers. In this study, we focus on the electronic properties of (doped) TiO2 as an electron-selective contact. We show via KPFM that co-doping of TiO2 with Nb(V) and Sn(IV) reduces the materials work function by 270 meV, giving it stronger n-type characteristics compared to Nb(V) mono-doped TiO2. The altered electronic alignment with CsPbI3 translates to enhanced electron extraction, as demonstrated with ssPL, trPL and trSPV in triad. Importantly, we extract crucial parameters, such as the concentration of extracted electrons and the interface hole recombination velocity, from the SPV transients via 2D drift-diffusion simulations. When implementing the co-doped TiO2 into full n-i-p solar cells, the operational stability is enhanced to 32000 h of projected TS80 lifetime. This study provides fundamental understanding of interfacial charge extraction and its correlation with operational stability of perovskite solar cells, which can be transferred to other charge-selective contacts.


title: Proposal of a general framework to categorize continuous predictor variables

abstract: The use of discretized variables in the development of prediction models is a common practice, in part because the decision-making process is more natural when it is based on rules created from segmented models. Although this practice is perhaps more common in medicine, it is extensible to any area of knowledge where a predictive model helps in decision-making. Therefore, providing researchers with a useful and valid categorization method could be a relevant issue when developing prediction models. In this paper, we propose a new general methodology that can be applied to categorize a predictor variable in any regression model where the response variable belongs to the exponential family distribution. Furthermore, it can be applied in any multivariate context, allowing to categorize more than one continuous covariate simultaneously. In addition, a computationally very efficient method is proposed to obtain the optimal number of categories, based on a pseudo-BIC proposal. Several simulation studies have been conducted in which the efficiency of the method with respect to both the location and the number of estimated cut-off points is shown. Finally, the categorization proposal has been applied to a real data set of 543 patients with chronic obstructive pulmonary disease from Galdakao Hospital's five outpatient respiratory clinics, who were followed up for 10 years. We applied the proposed methodology to jointly categorize the continuous variables six-minute walking test and forced expiratory volume in one second in a multiple Poisson generalized additive model for the response variable rate of the number of hospital admissions by years of follow-up. The location and number of cut-off points obtained were clinically validated as being in line with the categorizations used in the literature.


title: Using Generative Text Models to Create Qualitative Codebooks for Student Evaluations of Teaching

abstract: Feedback is a critical aspect of improvement. Unfortunately, when there is a lot of feedback from multiple sources, it can be difficult to distill the information into actionable insights. Consider student evaluations of teaching (SETs), which are important sources of feedback for educators. They can give instructors insights into what worked during a semester. A collection of SETs can also be useful to administrators as signals for courses or entire programs. However, on a large scale as in high-enrollment courses or administrative records over several years, the volume of SETs can render them difficult to analyze. In this paper, we discuss a novel method for analyzing SETs using natural language processing (NLP) and large language models (LLMs). We demonstrate the method by applying it to a corpus of 5,000 SETs from a large public university. We show that the method can be used to extract, embed, cluster, and summarize the SETs to identify the themes they express. More generally, this work illustrates how to use the combination of NLP techniques and LLMs to generate a codebook for SETs. We conclude by discussing the implications of this method for analyzing SETs and other types of student writing in teaching and research settings.


title: SceneSense: Diffusion Models for 3D Occupancy Synthesis from Partial Observation

abstract: When exploring new areas, robotic systems generally exclusively plan and execute controls over geometry that has been directly measured. When entering space that was previously obstructed from view such as turning corners in hallways or entering new rooms, robots often pause to plan over the newly observed space. To address this we present SceneScene, a real-time 3D diffusion model for synthesizing 3D occupancy information from partial observations that effectively predicts these occluded or out of view geometries for use in future planning and control frameworks. SceneSense uses a running occupancy map and a single RGB-D camera to generate predicted geometry around the platform at runtime, even when the geometry is occluded or out of view. Our architecture ensures that SceneSense never overwrites observed free or occupied space. By preserving the integrity of the observed map, SceneSense mitigates the risk of corrupting the observed space with generative predictions. While SceneSense is shown to operate well using a single RGB-D camera, the framework is flexible enough to extend to additional modalities. SceneSense operates as part of any system that generates a running occupancy map `out of the box', removing conditioning from the framework. Alternatively, for maximum performance in new modalities, the perception backbone can be replaced and the model retrained for inference in new applications. Unlike existing models that necessitate multiple views and offline scene synthesis, or are focused on filling gaps in observed data, our findings demonstrate that SceneSense is an effective approach to estimating unobserved local occupancy information at runtime. Local occupancy predictions from SceneSense are shown to better represent the ground truth occupancy distribution during the test exploration trajectories than the running occupancy map.


title: Noncompact surfaces, triangulations and rigidity

abstract: Every noncompact surface is shown to have a (3,6)-tight triangulation and a characterisation of such triangulations is given in terms of length constraints on the boundary walks of superfaces of a given genus. Applications are given to the generic rigidity of associated countable bar-joint frameworks in R^3. In particular, every noncompact surface has a (3,6)-tight triangulation that is minimally 3-rigid. A proof of Ker\'ekj\'art\'o's classification of noncompact surfaces is also given.


title: Suppression of the superfluid Kelvin-Helmholtz instability due to massive vortex cores, friction and confinement

abstract: We characterize the dynamical instability responsible for the breakdown of regular rows and necklaces of quantized vortices that appear at the interface between two superfluids in relative motion. Making use of a generalized point-vortex model, we identify several mechanisms leading to the suppression of this instability. They include a non-zero mass of the vortex cores, dissipative processes resulting from the interaction between the vortices and the excitations of the superfluid, and the proximity of the vortex array to the sample boundaries. We show that massive vortex cores not only have a mitigating effect on the dynamical instability, but also change the associated scaling law and affect the direction along which it develops. The predictions of our massive and dissipative point-vortex model are eventually compared against recent experimental measurements of the maximum instability growth rate relevant to vortex necklaces in a cold-atom platform.


title: Measurement-driven Langevin modeling of superparamagnetic tunnel junctions

abstract: Superparamagnetic tunnel junctions are important devices for a range of emerging technologies, but most existing compact models capture only their mean switching rates. Capturing qualitatively accurate analog dynamics of these devices will be important as the technology scales up. Here we present results using a one-dimensional overdamped Langevin equation that captures statistical properties of measured time traces, including voltage histograms, drift and diffusion characteristics as measured with Kramers-Moyal coefficients, and dwell times distributions. While common macrospin models are more physically-motivated magnetic models than the Langevin model, we show that for the device measured here, they capture even fewer of the measured experimental behaviors.


title: Encoding of linear kinetic plasma problems in quantum circuits via data compression

abstract: We propose an algorithm for encoding of linear kinetic plasma problems in quantum circuits. The focus is made on modeling electrostatic linear waves in one-dimensional Maxwellian electron plasma. The waves are described by the linearized Vlasov-Amp\`ere system with a spatially localized external current that drives plasma oscillations. This system is formulated as a boundary-value problem and cast in the form of a linear vector equation $A\psi = b$ to be solved by using the quantum signal processing algorithm. The latter requires encoding of the matrix $A$ in a quantum circuit as a subblock of a unitary matrix. We propose how to encode $A$ in a circuit in a compressed form and discuss how the resulting circuit scales with the problem size and the desired precision.


title: GetMesh: A Controllable Model for High-quality Mesh Generation and Manipulation

abstract: Mesh is a fundamental representation of 3D assets in various industrial applications, and is widely supported by professional softwares. However, due to its irregular structure, mesh creation and manipulation is often time-consuming and labor-intensive. In this paper, we propose a highly controllable generative model, GetMesh, for mesh generation and manipulation across different categories. By taking a varying number of points as the latent representation, and re-organizing them as triplane representation, GetMesh generates meshes with rich and sharp details, outperforming both single-category and multi-category counterparts. Moreover, it also enables fine-grained control over the generation process that previous mesh generative models cannot achieve, where changing global/local mesh topologies, adding/removing mesh parts, and combining mesh parts across categories can be intuitively, efficiently, and robustly accomplished by adjusting the number, positions or features of latent points. Project page is https://getmesh.github.io.


title: Elastic and breaking properties of epitaxial face-centered crystals in neutron star crusts and white dwarf cores

abstract: Crystallization of dense matter in neutron star crusts and white dwarf cores may be similar to epitaxial crystal growth in terrestrial laboratories. However in stellar crystals, the spacing between horizontal planes has to gradually increase with the outward movement of the crystallization front, tracing decrease of the electron density. This process produces Coulomb crystals with stretched rather than cubic elementary cells. We extend the analysis of the elastic and breaking properties of such crystals to the face-centered (fc) lattice. Shear deformations orthogonal to the stretch direction have been studied for 22 crystallographic shear planes. A common property for all these planes is a reduction and eventual nulling of the breaking shear strain with deviation from the unstretched configuration. The effective shear moduli for deformations orthogonal to the stretch direction have been calculated. It is possible that the epitaxial crystallization in compact stars results in a formation of large-scale crystallites or, at least, in growth of the whole crystallization front perpendicular to particular crystallographic planes. For fc structure growth orthogonal to the $\{111\}$ planes, we expect that, at any density, $\sim 5\%$ ($\sim 0.5\%$) of crystallite height is occupied by layers one (two) orders of magnitude weaker than the bulk of the crystallite. This may be important for realistic modeling of crustquakes on neutron stars.


title: Sub-photon accuracy noise reduction of single shot coherent diffraction pattern with atomic model trained autoencoder

abstract: Single-shot imaging with femtosecond X-ray lasers is a powerful measurement technique that can achieve both high spatial and temporal resolution. However, its accuracy has been severely limited by the difficulty of applying conventional noise-reduction processing. This study uses deep learning to validate noise reduction techniques, with autoencoders serving as the learning model. Focusing on the diffraction patterns of nanoparticles, we simulated a large dataset treating the nanoparticles as composed of many independent atoms. Three neural network architectures are investigated: neural network, convolutional neural network and U-net, with U-net showing superior performance in noise reduction and subphoton reproduction. We also extended our models to apply to diffraction patterns of particle shapes different from those in the simulated data. We then applied the U-net model to a coherent diffractive imaging study, wherein a nanoparticle in a microfluidic device is exposed to a single X-ray free-electron laser pulse. After noise reduction, the reconstructed nanoparticle image improved significantly even though the nanoparticle shape was different from the training data, highlighting the importance of transfer learning.


title: Adaptive stepsize algorithms for Langevin dynamics

abstract: We discuss the design of an invariant measure-preserving transformed dynamics for the numerical treatment of Langevin dynamics based on rescaling of time, with the goal of sampling from an invariant measure. Given an appropriate monitor function which characterizes the numerical difficulty of the problem as a function of the state of the system, this method allows the stepsizes to be reduced only when necessary, facilitating efficient recovery of long-time behavior. We study both the overdamped and underdamped Langevin dynamics. We investigate how an appropriate correction term that ensures preservation of the invariant measure should be incorporated into a numerical splitting scheme. Finally, we demonstrate the use of the technique in several model systems, including a Bayesian sampling problem with a steep prior.


title: Stability of simplex slicing

abstract: We establish dimension-free stability of Webb's sharp simplex slicing (1996). Incidentally, we investigate Lipschitzness of volume of hyperplane central sections of arbitrary convex bodies.


title: Hamiltonian-reconstruction distance as a success metric for the Variational Quantum Eigensolver

abstract: The Variational Quantum Eigensolver (VQE) is a hybrid quantum-classical algorithm for quantum simulation that can be run on near-term quantum hardware. A challenge in VQE -- as well as any other heuristic algorithm for finding ground states of Hamiltonians -- is to know how close the algorithm's output solution is to the true ground state, when the true ground state and ground-state energy are unknown. This is especially important in iterative algorithms, such as VQE, where one wants to avoid erroneous early termination. Recent developments in Hamiltonian reconstruction -- the inference of a Hamiltonian given an eigenstate -- give a metric can be used to assess the quality of a variational solution to a Hamiltonian-eigensolving problem. This metric can assess the proximity of the variational solution to the ground state without any knowledge of the true ground state or ground-state energy. In numerical simulations and in demonstrations on a cloud-based trapped-ion quantum computer, we show that for examples of both one-dimensional transverse-field-Ising (11 qubits) and two-dimensional J1-J2 transverse-field-Ising (6 qubits) spin problems, the Hamiltonian-reconstruction distance gives a helpful indication of whether VQE has yet found the ground state or not. Our experiments included cases where the energy plateaus as a function of the VQE iteration, which could have resulted in erroneous early stopping of the VQE algorithm, but where the Hamiltonian-reconstruction distance correctly suggests to continue iterating. We find that the Hamiltonian-reconstruction distance has a useful correlation with the fidelity between the VQE solution and the true ground state. Our work suggests that the Hamiltonian-reconstruction distance may be a useful tool for assessing success in VQE, including on noisy quantum processors in practice.


title: Accelerating Scientific Discovery with Generative Knowledge Extraction, Graph-Based Representation, and Multimodal Intelligent Graph Reasoning

abstract: Leveraging generative Artificial Intelligence (AI), we have transformed a dataset comprising 1,000 scientific papers into an ontological knowledge graph. Through an in-depth structural analysis, we have calculated node degrees, identified communities and connectivities, and evaluated clustering coefficients and betweenness centrality of pivotal nodes, uncovering fascinating knowledge architectures. The graph has an inherently scale-free nature, is highly connected, and can be used for graph reasoning by taking advantage of transitive and isomorphic properties that reveal unprecedented interdisciplinary relationships that can be used to answer queries, identify gaps in knowledge, propose never-before-seen material designs, and predict material behaviors. We compute deep node embeddings for combinatorial node similarity ranking for use in a path sampling strategy links dissimilar concepts that have previously not been related. One comparison revealed structural parallels between biological materials and Beethoven's 9th Symphony, highlighting shared patterns of complexity through isomorphic mapping. In another example, the algorithm proposed a hierarchical mycelium-based composite based on integrating path sampling with principles extracted from Kandinsky's 'Composition VII' painting. The resulting material integrates an innovative set of concepts that include a balance of chaos/order, adjustable porosity, mechanical strength, and complex patterned chemical functionalization. We uncover other isomorphisms across science, technology and art, revealing a nuanced ontology of immanence that reveal a context-dependent heterarchical interplay of constituents. Graph-based generative AI achieves a far higher degree of novelty, explorative capacity, and technical detail, than conventional approaches and establishes a widely useful framework for innovation by revealing hidden connections.

categories: cs.LG cond-mat.mes-hall cond-mat.mtrl-sci cond-mat.soft cs.AI cs.CL


title: Revisiting the effect of lens mass models in cosmological applications of strong gravitational lensing

abstract: Strong gravitational lens system catalogues are typically used to constrain a combination of cosmological and empirical lens mass model parameters, even though the simplest singular isothermal sphere (SIS) models yield a $\chi^2$ per degree of freedom $\simeq 2$. To date, this problem has been alleviated by introducing additional empirical parameters in the extended power law (EPL) models and constraints from high resolution imagery. The EPL parameters are taken to vary from lens to lens, rather than defining universal lens profiles. We investigate these lens models using Bayesian methods through a novel alternative that treats spatial curvature via the non-FLRW Timescape cosmology. We apply Markov Chain Monte Carlo methods using the catalogue of 161 lens systems of Chen et al (arXiv:1809.09845) to simulate large mock catalogues for: (i) the standard $\Lambda$CDM model with zero spatial curvature; and (ii) the Timescape model. Furthermore, this methodology can be applied to any cosmological model. In agreement with previous results we find that in combination with SIS parameters, models with zero FLRW spatial curvature fit better as the free parameter approaches an unphysical empty universe, $\Omega_{\rm M0}\to0$. By contrast, the Timescape cosmology is found to prefer parameter values in which its cosmological parameter, the present void fraction, is driven to $f_{\rm v0}\to0.73$ matches, close to values found to best fit independent cosmological data sets: supernovae Ia distances and cosmic microwave background. This conclusion holds for a large range of seed values $f_{\rm v0}\to0.73\in\{0.1,0.9\}$, and for Timescape fits to both Timescape and FLRW mocks. Regardless of cosmology, unphysical estimates of the distance ratios given from power-law lens models result in poor goodness of fit. Nonetheless, the results are consistent with non-FLRW spatial curvature evolution.


title: Learning Useful Representations of Recurrent Neural Network Weight Matrices

abstract: Recurrent Neural Networks (RNNs) are general-purpose parallel-sequential computers. The program of an RNN is its weight matrix. How to learn useful representations of RNN weights that facilitate RNN analysis as well as downstream tasks? While the mechanistic approach directly looks at some RNN's weights to predict its behavior, the functionalist approach analyzes its overall functionality-specifically, its input-output mapping. We consider several mechanistic approaches for RNN weights and adapt the permutation equivariant Deep Weight Space layer for RNNs. Our two novel functionalist approaches extract information from RNN weights by 'interrogating' the RNN through probing inputs. We develop a theoretical framework that demonstrates conditions under which the functionalist approach can generate rich representations that help determine RNN behavior. We release the first two 'model zoo' datasets for RNN weight representation learning. One consists of generative models of a class of formal languages, and the other one of classifiers of sequentially processed MNIST digits.With the help of an emulation-based self-supervised learning technique we compare and evaluate the different RNN weight encoding techniques on multiple downstream applications. On the most challenging one, namely predicting which exact task the RNN was trained on, functionalist approaches show clear superiority.


title: HIRI-ViT: Scaling Vision Transformer with High Resolution Inputs

abstract: The hybrid deep models of Vision Transformer (ViT) and Convolution Neural Network (CNN) have emerged as a powerful class of backbones for vision tasks. Scaling up the input resolution of such hybrid backbones naturally strengthes model capacity, but inevitably suffers from heavy computational cost that scales quadratically. Instead, we present a new hybrid backbone with HIgh-Resolution Inputs (namely HIRI-ViT), that upgrades prevalent four-stage ViT to five-stage ViT tailored for high-resolution inputs. HIRI-ViT is built upon the seminal idea of decomposing the typical CNN operations into two parallel CNN branches in a cost-efficient manner. One high-resolution branch directly takes primary high-resolution features as inputs, but uses less convolution operations. The other low-resolution branch first performs down-sampling and then utilizes more convolution operations over such low-resolution features. Experiments on both recognition task (ImageNet-1K dataset) and dense prediction tasks (COCO and ADE20K datasets) demonstrate the superiority of HIRI-ViT. More remarkably, under comparable computational cost ($\sim$5.0 GFLOPs), HIRI-ViT achieves to-date the best published Top-1 accuracy of 84.3% on ImageNet with 448$\times$448 inputs, which absolutely improves 83.4% of iFormer-S by 0.9% with 224$\times$224 inputs.


title: Notochord: a Flexible Probabilistic Model for Real-Time MIDI Performance

abstract: Deep learning-based probabilistic models of musical data are producing increasingly realistic results and promise to enter creative workflows of many kinds. Yet they have been little-studied in a performance setting, where the results of user actions typically ought to feel instantaneous. To enable such study, we designed Notochord, a deep probabilistic model for sequences of structured events, and trained an instance of it on the Lakh MIDI dataset. Our probabilistic formulation allows interpretable interventions at a sub-event level, which enables one model to act as a backbone for diverse interactive musical functions including steerable generation, harmonization, machine improvisation, and likelihood-based interfaces. Notochord can generate polyphonic and multi-track MIDI, and respond to inputs with latency below ten milliseconds. Training code, model checkpoints and interactive examples are provided as open source software.


title: No-gap second-order conditions for minimization problems in spaces of measures

abstract: Over the last years, minimization problems over spaces of measures have received increased interest due to their relevance in the context of inverse problems, optimal control and machine learning. A fundamental role in their numerical analysis is played by the assumption that the optimal dual state admits finitely many global extrema and satisfies a second-order sufficient optimality condition in each one of them. In this work, we show the full equivalence of these structural assumptions to a no-gap second-order condition involving the second subderivative of the Radon norm as well as to a local quadratic growth property of the objective functional with respect to the bounded Lipschitz norm.


title: DreamMotion: Space-Time Self-Similarity Score Distillation for Zero-Shot Video Editing

abstract: Text-driven diffusion-based video editing presents a unique challenge not encountered in image editing literature: establishing real-world motion. Unlike existing video editing approaches, here we focus on score distillation sampling to circumvent the standard reverse diffusion process and initiate optimization from videos that already exhibit natural motion. Our analysis reveals that while video score distillation can effectively introduce new content indicated by target text, it can also cause significant structure and motion deviation. To counteract this, we propose to match space-time self-similarities of the original video and the edited video during the score distillation. Thanks to the use of score distillation, our approach is model-agnostic, which can be applied for both cascaded and non-cascaded video diffusion frameworks. Through extensive comparisons with leading methods, our approach demonstrates its superiority in altering appearances while accurately preserving the original structure and motion.


title: GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learning

abstract: Self-supervised learning has achieved remarkable success in acquiring high-quality representations from unlabeled data. The widely adopted contrastive learning framework aims to learn invariant representations by minimizing the distance between positive views originating from the same image. However, existing techniques to construct positive views highly rely on manual transformations, resulting in limited diversity and potentially false positive pairs. To tackle these challenges, we present GenView, a controllable framework that augments the diversity of positive views leveraging the power of pretrained generative models while preserving semantics. We develop an adaptive view generation method that dynamically adjusts the noise level in sampling to ensure the preservation of essential semantic meaning while introducing variability. Additionally, we introduce a quality-driven contrastive loss, which assesses the quality of positive pairs by considering both foreground similarity and background diversity. This loss prioritizes the high-quality positive pairs we construct while reducing the influence of low-quality pairs, thereby mitigating potential semantic inconsistencies introduced by generative models and aggressive data augmentation. Thanks to the improved positive view quality and the quality-driven contrastive loss, GenView significantly improves self-supervised learning across various tasks. For instance, GenView improves MoCov2 performance by 2.5%/2.2% on ImageNet linear/semi-supervised classification. Moreover, GenView even performs much better than naively augmenting the ImageNet dataset with Laion400M or ImageNet21K. Code is available at https://github.com/xiaojieli0903/genview.


title: The Value, Benefits, and Concerns of Generative AI-Powered Assistance in Writing

abstract: Recent advances in generative AI technologies like large language models raise both excitement and concerns about the future of human-AI co-creation in writing. To unpack people's attitude towards and experience with generative AI-powered writing assistants, in this paper, we conduct an experiment to understand whether and how much value people attach to AI assistance, and how the incorporation of AI assistance in writing workflows changes people's writing perceptions and performance. Our results suggest that people are willing to forgo financial payments to receive writing assistance from AI, especially if AI can provide direct content generation assistance and the writing task is highly creative. Generative AI-powered assistance is found to offer benefits in increasing people's productivity and confidence in writing. However, direct content generation assistance offered by AI also comes with risks, including decreasing people's sense of accountability and diversity in writing. We conclude by discussing the implications of our findings.


title: Visualization for Trust in Machine Learning Revisited: The State of the Field in 2023

abstract: Visualization for explainable and trustworthy machine learning remains one of the most important and heavily researched fields within information visualization and visual analytics with various application domains, such as medicine, finance, and bioinformatics. After our 2020 state-of-the-art report comprising 200 techniques, we have persistently collected peer-reviewed articles describing visualization techniques, categorized them based on the previously established categorization schema consisting of 119 categories, and provided the resulting collection of 542 techniques in an online survey browser. In this survey article, we present the updated findings of new analyses of this dataset as of fall 2023 and discuss trends, insights, and eight open challenges for using visualizations in machine learning. Our results corroborate the rapidly growing trend of visualization techniques for increasing trust in machine learning models in the past three years, with visualization found to help improve popular model explainability methods and check new deep learning architectures, for instance.


title: Approximate Stability Radius Analysis and Design in Linear Systems

abstract: The robustness of the stability properties of dynamical systems in the presence of unknown/adversarial perturbations to system parameters is a desirable property. In this paper, we present methods to efficiently compute and improve the approximate stability radius of linear time-invariant systems. We propose two methods to derive closed-form expressions of approximate stability radius, and use these to re-design the system matrix to increase the stability radius. Our numerical studies show that the approximations work well and are able to improve the robustness of the stability of the system.


title: Defining Effective Engagement For Enhancing Cancer Patients' Well-being with Mobile Digital Behavior Change Interventions

abstract: Digital Behavior Change Interventions (DBCIs) are supporting development of new health behaviors. Evaluating their effectiveness is crucial for their improvement and understanding of success factors. However, comprehensive guidance for developers, particularly in small-scale studies with ethical constraints, is limited. Building on the CAPABLE project, this study aims to define effective engagement with DBCIs for supporting cancer patients in enhancing their quality of life. We identify metrics for measuring engagement, explore the interest of both patients and clinicians in DBCIs, and propose hypotheses for assessing the impact of DBCIs in such contexts. Our findings suggest that clinician prescriptions significantly increase sustained engagement with mobile DBCIs. In addition, while one weekly engagement with a DBCI is sufficient to maintain well-being, transitioning from extrinsic to intrinsic motivation may require a higher level of engagement.


title: SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion

abstract: We present Stable Video 3D (SV3D) -- a latent video diffusion model for high-resolution, image-to-multi-view generation of orbital videos around a 3D object. Recent work on 3D generation propose techniques to adapt 2D generative models for novel view synthesis (NVS) and 3D optimization. However, these methods have several disadvantages due to either limited views or inconsistent NVS, thereby affecting the performance of 3D object generation. In this work, we propose SV3D that adapts image-to-video diffusion model for novel multi-view synthesis and 3D generation, thereby leveraging the generalization and multi-view consistency of the video models, while further adding explicit camera control for NVS. We also propose improved 3D optimization techniques to use SV3D and its NVS outputs for image-to-3D generation. Extensive experimental results on multiple datasets with 2D and 3D metrics as well as user study demonstrate SV3D's state-of-the-art performance on NVS as well as 3D reconstruction compared to prior works.


title: Leveraging Spatial and Semantic Feature Extraction for Skin Cancer Diagnosis with Capsule Networks and Graph Neural Networks

abstract: In the realm of skin lesion image classification, the intricate spatial and semantic features pose significant challenges for conventional Convolutional Neural Network (CNN)-based methodologies. These challenges are compounded by the imbalanced nature of skin lesion datasets, which hampers the ability of models to learn minority class features effectively. Despite augmentation strategies, such as those using Generative Adversarial Networks (GANs), previous attempts have not fully addressed these complexities. This study introduces an innovative approach by integrating Graph Neural Networks (GNNs) with Capsule Networks to enhance classification performance. GNNs, known for their proficiency in handling graph-structured data, offer an advanced mechanism for capturing complex patterns and relationships beyond the capabilities of traditional CNNs. Capsule Networks further contribute by providing superior recognition of spatial hierarchies within images. Our research focuses on evaluating and enhancing the Tiny Pyramid Vision GNN (Tiny Pyramid ViG) architecture by incorporating it with a Capsule Network. This hybrid model was applied to the MNIST:HAM10000 dataset, a comprehensive skin lesion dataset designed for benchmarking classification models. After 75 epochs of training, our model achieved a significant accuracy improvement, reaching 89.23% and 95.52%, surpassing established benchmarks such as GoogLeNet (83.94%), InceptionV3 (86.82%), MobileNet V3 (89.87%), EfficientNet-B7 (92.07%), ResNet18 (92.22%), ResNet34 (91.90%), ViT-Base (73.70%), and IRv2-SA (93.47%) on the same dataset. This outcome underscores the potential of our approach in overcoming the inherent challenges of skin lesion classification, contributing to the advancement of image-based diagnosis in dermatology.


title: VideoMV: Consistent Multi-View Generation Based on Large Video Generative Model

abstract: Generating multi-view images based on text or single-image prompts is a critical capability for the creation of 3D content. Two fundamental questions on this topic are what data we use for training and how to ensure multi-view consistency. This paper introduces a novel framework that makes fundamental contributions to both questions. Unlike leveraging images from 2D diffusion models for training, we propose a dense consistent multi-view generation model that is fine-tuned from off-the-shelf video generative models. Images from video generative models are more suitable for multi-view generation because the underlying network architecture that generates them employs a temporal module to enforce frame consistency. Moreover, the video data sets used to train these models are abundant and diverse, leading to a reduced train-finetuning domain gap. To enhance multi-view consistency, we introduce a 3D-Aware Denoising Sampling, which first employs a feed-forward reconstruction module to get an explicit global 3D model, and then adopts a sampling strategy that effectively involves images rendered from the global 3D model into the denoising sampling loop to improve the multi-view consistency of the final images. As a by-product, this module also provides a fast way to create 3D assets represented by 3D Gaussians within a few seconds. Our approach can generate 24 dense views and converges much faster in training than state-of-the-art approaches (4 GPU hours versus many thousand GPU hours) with comparable visual quality and consistency. By further fine-tuning, our approach outperforms existing state-of-the-art methods in both quantitative metrics and visual effects. Our project page is aigc3d.github.io/VideoMV.


title: HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data

abstract: 3D hand-object interaction data is scarce due to the hardware constraints in scaling up the data collection process. In this paper, we propose HOIDiffusion for generating realistic and diverse 3D hand-object interaction data. Our model is a conditional diffusion model that takes both the 3D hand-object geometric structure and text description as inputs for image synthesis. This offers a more controllable and realistic synthesis as we can specify the structure and style inputs in a disentangled manner. HOIDiffusion is trained by leveraging a diffusion model pre-trained on large-scale natural images and a few 3D human demonstrations. Beyond controllable image synthesis, we adopt the generated 3D data for learning 6D object pose estimation and show its effectiveness in improving perception systems. Project page: https://mq-zhang1.github.io/HOIDiffusion


title: Convergence of Kinetic Langevin Monte Carlo on Lie groups

abstract: Explicit, momentum-based dynamics for optimizing functions defined on Lie groups was recently constructed, based on techniques such as variational optimization and left trivialization. We appropriately add tractable noise to the optimization dynamics to turn it into a sampling dynamics, leveraging the advantageous feature that the trivialized momentum variable is Euclidean despite that the potential function lives on a manifold. We then propose a Lie-group MCMC sampler, by delicately discretizing the resulting kinetic-Langevin-type sampling dynamics. The Lie group structure is exactly preserved by this discretization. Exponential convergence with explicit convergence rate for both the continuous dynamics and the discrete sampler are then proved under $W_2$ distance. Only compactness of the Lie group and geodesically $L$-smoothness of the potential function are needed. To the best of our knowledge, this is the first convergence result for kinetic Langevin on curved spaces, and also the first quantitative result that requires no convexity or, at least not explicitly, any common relaxation such as isoperimetry.

categories: math.ST cs.LG cs.NA math.NA math.PR stat.ML stat.TH


title: GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image

abstract: We introduce GeoWizard, a new generative foundation model designed for estimating geometric attributes, e.g., depth and normals, from single images. While significant research has already been conducted in this area, the progress has been substantially limited by the low diversity and poor quality of publicly available datasets. As a result, the prior works either are constrained to limited scenarios or suffer from the inability to capture geometric details. In this paper, we demonstrate that generative models, as opposed to traditional discriminative models (e.g., CNNs and Transformers), can effectively address the inherently ill-posed problem. We further show that leveraging diffusion priors can markedly improve generalization, detail preservation, and efficiency in resource usage. Specifically, we extend the original stable diffusion model to jointly predict depth and normal, allowing mutual information exchange and high consistency between the two representations. More importantly, we propose a simple yet effective strategy to segregate the complex data distribution of various scenes into distinct sub-distributions. This strategy enables our model to recognize different scene layouts, capturing 3D geometry with remarkable fidelity. GeoWizard sets new benchmarks for zero-shot depth and normal prediction, significantly enhancing many downstream applications such as 3D reconstruction, 2D content creation, and novel viewpoint synthesis.


title: EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents

abstract: Recent SOTA approaches for embodied learning via interaction directly employ large language models (LLMs) as agents to determine the next steps in an environment. Due to their world knowledge and reasoning capabilities, LLM agents achieve stronger performance than previous smaller agents based on reinforcement learning (RL); however, frequently calling LLMs is slow and expensive. Instead of directly employing LLMs as agents, can we use LLMs' reasoning capabilities to adaptively create training environments to help smaller embodied RL agents learn useful skills that they are weak at? We propose EnvGen, a novel framework to address this question. First, we prompt an LLM to generate training environments that allow agents to quickly learn different tasks in parallel. Concretely, the LLM is given the task description and simulator objectives that the agents should learn and is then asked to generate a set of environment configurations (e.g., different terrains, items given to agents, etc.). Next, we train a small RL agent in a mixture of the original and LLM-generated environments. Then, we enable the LLM to continuously adapt the generated environments to progressively improve the skills that the agent is weak at, by providing feedback to the LLM in the form of the agent's performance. We demonstrate the usefulness of EnvGen with comprehensive experiments in Crafter and Heist environments. We find that a small RL agent trained with EnvGen can outperform SOTA methods, including a GPT-4 agent, and learns long-horizon tasks significantly faster. We show qualitatively how the LLM adapts training environments to help improve RL agents' weaker skills over time. Additionally, EnvGen is substantially more efficient as it only uses a small number of LLM calls (e.g., 4 in total), whereas LLM agents require thousands of LLM calls. Lastly, we present detailed ablation studies for our design choices.


title: Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation

abstract: Diffusion models are the main driver of progress in image and video synthesis, but suffer from slow inference speed. Distillation methods, like the recently introduced adversarial diffusion distillation (ADD) aim to shift the model from many-shot to single-step inference, albeit at the cost of expensive and difficult optimization due to its reliance on a fixed pretrained DINOv2 discriminator. We introduce Latent Adversarial Diffusion Distillation (LADD), a novel distillation approach overcoming the limitations of ADD. In contrast to pixel-based ADD, LADD utilizes generative features from pretrained latent diffusion models. This approach simplifies training and enhances performance, enabling high-resolution multi-aspect ratio image synthesis. We apply LADD to Stable Diffusion 3 (8B) to obtain SD3-Turbo, a fast model that matches the performance of state-of-the-art text-to-image generators using only four unguided sampling steps. Moreover, we systematically investigate its scaling behavior and demonstrate LADD's effectiveness in various applications such as image editing and inpainting.