title: HYDRA: A Hyper Agent for Dynamic Compositional Visual Reasoning

abstract: Recent advances in visual reasoning (VR), particularly with the aid of Large Vision-Language Models (VLMs), show promise but require access to large-scale datasets and face challenges such as high computational costs and limited generalization capabilities. Compositional visual reasoning approaches have emerged as effective strategies; however, they heavily rely on the commonsense knowledge encoded in Large Language Models (LLMs) to perform planning, reasoning, or both, without considering the effect of their decisions on the visual reasoning process, which can lead to errors or failed procedures. To address these challenges, we introduce HYDRA, a multi-stage dynamic compositional visual reasoning framework designed for reliable and incrementally progressive general reasoning. HYDRA integrates three essential modules: a planner, a Reinforcement Learning (RL) agent serving as a cognitive controller, and a reasoner. The planner and reasoner modules utilize an LLM to generate instruction samples and executable code from the selected instruction, respectively, while the RL agent dynamically interacts with these modules, making high-level decisions on selection of the best instruction sample given information from the historical state stored through a feedback loop. This adaptable design enables HYDRA to adjust its actions based on previous feedback received during the reasoning process, leading to more reliable reasoning outputs and ultimately enhancing its overall effectiveness. Our framework demonstrates state-of-the-art performance in various VR tasks on four different widely-used datasets.


title: Improved decay results for micropolar flows with nonlinear damping

abstract: We examine the long-time behavior of solutions (and their derivatives) to the micropolar equations with nonlinear velocity damping. Additionally, we get a speed-up gain of $ t^{1/2} $ for the angular velocity, consistent with established findings for classic micropolar flows lacking nonlinear damping. Consequently, we also obtain a sharper result regarding the asymptotic stability of the micro-rotational velocity $\ww(\cdot,t)$. Related results of independent interest are also included.


title: EmoVOCA: Speech-Driven Emotional 3D Talking Heads

abstract: The domain of 3D talking head generation has witnessed significant progress in recent years. A notable challenge in this field consists in blending speech-related motions with expression dynamics, which is primarily caused by the lack of comprehensive 3D datasets that combine diversity in spoken sentences with a variety of facial expressions. Whereas literature works attempted to exploit 2D video data and parametric 3D models as a workaround, these still show limitations when jointly modeling the two motions. In this work, we address this problem from a different perspective, and propose an innovative data-driven technique that we used for creating a synthetic dataset, called EmoVOCA, obtained by combining a collection of inexpressive 3D talking heads and a set of 3D expressive sequences. To demonstrate the advantages of this approach, and the quality of the dataset, we then designed and trained an emotional 3D talking head generator that accepts a 3D face, an audio file, an emotion label, and an intensity value as inputs, and learns to animate the audio-synchronized lip movements with expressive traits of the face. Comprehensive experiments, both quantitative and qualitative, using our data and generator evidence superior ability in synthesizing convincing animations, when compared with the best performing methods in the literature. Our code and pre-trained model will be made available.


title: Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transport

abstract: We study the convergence of gradient flow for the training of deep neural networks. If Residual Neural Networks are a popular example of very deep architectures, their training constitutes a challenging optimization problem due notably to the non-convexity and the non-coercivity of the objective. Yet, in applications, those tasks are successfully solved by simple optimization algorithms such as gradient descent. To better understand this phenomenon, we focus here on a ``mean-field'' model of infinitely deep and arbitrarily wide ResNet, parameterized by probability measures over the product set of layers and parameters and with constant marginal on the set of layers. Indeed, in the case of shallow neural networks, mean field models have proven to benefit from simplified loss-landscapes and good theoretical guarantees when trained with gradient flow for the Wasserstein metric on the set of probability measures. Motivated by this approach, we propose to train our model with gradient flow w.r.t. the conditional Optimal Transport distance: a restriction of the classical Wasserstein distance which enforces our marginal condition. Relying on the theory of gradient flows in metric spaces we first show the well-posedness of the gradient flow equation and its consistency with the training of ResNets at finite width. Performing a local Polyak-\L{}ojasiewicz analysis, we then show convergence of the gradient flow for well-chosen initializations: if the number of features is finite but sufficiently large and the risk is sufficiently small at initialization, the gradient flow converges towards a global minimizer. This is the first result of this type for infinitely deep and arbitrarily wide ResNets.


title: Electrical readout of spins in the absence of spin blockade

abstract: In semiconductor nanostructures, spin blockade (SB) is the most scalable mechanism for electrical spin readout requiring only two bound spins for its implementation which, in conjunction with charge sensing techniques, has led to high-fidelity readout of spins in semiconductor-based quantum processors. However, various mechanisms may lift SB, such as strong spin-orbit coupling (SOC) or low-lying excited states, hence posing challenges to perform spin readout at scale and with high fidelity in such systems. Here, we present a method, based on the dependence of the two-spin system polarizability on energy detuning, to perform spin state readout even when SB lifting mechanisms are dominant. It leverages SB lifting as a resource to detect different spin measurement outcomes selectively and positively. We demonstrate the method using a hybrid system formed by a quantum dot (QD) and a Boron acceptor in a silicon p-type transistor and show spin selective and positive readout of different spin states under SB lifting conditions due to (i) SOC and (ii) low-lying orbital states in the QD. We further use the method to determine the detuning-dependent spin relaxation time of 0.1-8~$\mu$s. Our method should help perform high-fidelity projective spin measurements in systems subject to strong SOC and may facilitate quantum tomography and state leakage studies.


title: P-wave pairing near a spin-split Josephson junction

abstract: Superconductivity and magnetism are competing effects that can coexist in certain regimes. Their co-existence leads to unexpected new behaviors that include the onset of exotic electron pair mechanisms and topological phases. In this work, we study the properties of a Josephson junction between two spin-split superconductors. The spin-splitting in the superconductors can arise from either the coupling to a ferromagnetic material or an external magnetic field. The properties of the junction are dominated by the Andreev bound states that are also split. One of these states can cross the superconductor's Fermi level, leading to a ground state transition characterized by a suppressed supercurrent. We interpret the supercurrent blockade as coming from a dominance of p-wave pairing close to the junction, where electrons at both sides of the junction pair. To support this interpretation, we analyze the different pairing channels and show that p-wave pairing is favored in the case where the magnetization of the two superconductors is parallel and suppressed in the anti-parallel case. We also analyze the noise spectrum that shows signatures of the ground state transition in the form of an elevated zero-frequency noise.


title: $\operatorname{SL}(n)$ contravariant function-valued valuations on polytopes

abstract: We present a complete classification of $\operatorname{SL}(n)$ contravariant, $C(\mathbb{R}^n\setminus\{o\})$-valued valuations on polytopes, without any additional assumptions.It extends the previous results of the second author [Int. Math. Res. Not. 2020] which have a good connection with the $L_p$ and Orlicz Brunn-Minkowski theory. Additionally, our results deduce a complete classification of $\operatorname{SL}(n)$ contravariant symmetric-tensor-valued valuations on polytopes.


title: Adaptive Visual Imitation Learning for Robotic Assisted Feeding Across Varied Bowl Configurations and Food Types

abstract: In this study, we introduce a novel visual imitation network with a spatial attention module for robotic assisted feeding (RAF). The goal is to acquire (i.e., scoop) food items from a bowl. However, achieving robust and adaptive food manipulation is particularly challenging. To deal with this, we propose a framework that integrates visual perception with imitation learning to enable the robot to handle diverse scenarios during scooping. Our approach, named AVIL (adaptive visual imitation learning), exhibits adaptability and robustness across different bowl configurations in terms of material, size, and position, as well as diverse food types including granular, semi-solid, and liquid, even in the presence of distractors. We validate the effectiveness of our approach by conducting experiments on a real robot. We also compare its performance with a baseline. The results demonstrate improvement over the baseline across all scenarios, with an enhancement of up to 2.5 times in terms of a success metric. Notably, our model, trained solely on data from a transparent glass bowl containing granular cereals, showcases generalization ability when tested zero-shot on other bowl configurations with different types of food.


title: Uoc luong kenh truyen trong he thong da robot su dung SDR

abstract: This study focuses on developing an experimental system for estimating communication channels in a multi-robot mobile system using software-defined radio (SDR) devices. The system consists of two mobile robots programmed for two scenarios: one where the robot remains stationary and another where it follows a predefined trajectory. Communication within the system is conducted through orthogonal frequency-division multiplexing (OFDM) to mitigate the effects of multipath propagation in indoor environments. The system's performance is evaluated using the bit error rate (BER). Connections related to robot motion and communication are implemented using Raspberry Pi 3 and BladeRF x115, respectively. The least squares (LS) technique is employed to estimate the channel with a bit error rate of approximately 10^(-2).


title: Wideband Modeling and Beamforming for Beyond Diagonal Reconfigurable Intelligent Surfaces

abstract: This work studies the wideband modeling and beamforming design of beyond diagonal reconfigurable intelligent surface (BD-RIS), which generalizes and goes beyond conventional RIS with diagonal phase shift matrices to achieve enhanced channel gain. Specifically, we investigate the response of BD-RIS in wideband systems by going back to its hardware circuit realizations. We propose a novel wideband model which has simple expressions while capturing the response variations of BD-RIS for signals with different frequencies. With this wideband model, we propose a BD-RIS design algorithm for an orthogonal frequency division multiplexing system to maximize the average rate over all subcarriers. Finally, we provide simulation results to evaluate the performance of the proposed design and show the importance of wideband modeling for BD-RIS.


title: MEDBind: Unifying Language and Multimodal Medical Data Embeddings

abstract: Medical vision-language pretraining models (VLPM) have achieved remarkable progress in fusing chest X-rays (CXR) with clinical texts, introducing image-text data binding approaches that enable zero-shot learning and downstream clinical tasks. However, the current landscape lacks the holistic integration of additional medical modalities, such as electrocardiograms (ECG). We present MEDBind (Medical Electronic patient recorD), which learns joint embeddings across CXR, ECG, and medical text. Using text data as the central anchor, MEDBind features tri-modality binding, delivering competitive performance in top-K retrieval, zero-shot, and few-shot benchmarks against established VLPM, and the ability for CXR-to-ECG zero-shot classification and retrieval. This seamless integration is achieved through combination of contrastive loss on modality-text pairs with our proposed contrastive loss function, Edge-Modality Contrastive Loss, fostering a cohesive embedding space for CXR, ECG, and text. Finally, we demonstrate that MEDBind can improve downstream tasks by directly integrating CXR and ECG embeddings into a large-language model for multimodal prompt tuning.


title: mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding

abstract: Structure information is critical for understanding the semantics of text-rich images, such as documents, tables, and charts. Existing Multimodal Large Language Models (MLLMs) for Visual Document Understanding are equipped with text recognition ability but lack general structure understanding abilities for text-rich document images. In this work, we emphasize the importance of structure information in Visual Document Understanding and propose the Unified Structure Learning to boost the performance of MLLMs. Our Unified Structure Learning comprises structure-aware parsing tasks and multi-grained text localization tasks across 5 domains: document, webpage, table, chart, and natural image. To better encode structure information, we design a simple and effective vision-to-text module H-Reducer, which can not only maintain the layout information but also reduce the length of visual features by merging horizontal adjacent patches through convolution, enabling the LLM to understand high-resolution images more efficiently. Furthermore, by constructing structure-aware text sequences and multi-grained pairs of texts and bounding boxes for publicly available text-rich images, we build a comprehensive training set DocStruct4M to support structure learning. Finally, we construct a small but high-quality reasoning tuning dataset DocReason25K to trigger the detailed explanation ability in the document domain. Our model DocOwl 1.5 achieves state-of-the-art performance on 10 visual document understanding benchmarks, improving the SOTA performance of MLLMs with a 7B LLM by more than 10 points in 5/10 benchmarks. Our codes, models, and datasets are publicly available at https://github.com/X-PLUG/mPLUG-DocOwl/tree/main/DocOwl1.5.


title: Quantum Onsager relations

abstract: Using quantum information geometry, I derive quantum generalizations of the Onsager rate equations, which model the dynamics of an open system near a steady state. The generalized equations hold for a flexible definition of the forces as well as a large class of statistical divergence measures and quantum-Fisher-information metrics beyond the conventional definition of entropy production. I also derive quantum Onsager-Casimir relations for the transport tensors by proposing a general concept of time reversal and detailed balance for open quantum systems. The results establish a remarkable connection between statistical mechanics and parameter estimation theory.


title: On black bounce space-times in non-linear electrodynamics

abstract: One of the main issues in gravitation is the presence of singularities in the most common space-time solutions of General Relativity, as the case of black holes. A way of constructing regular solutions that remove spacelike singularities consists in implement a bounce on such space-time, leading to what is usually known as black bounce space-times. Such space-times are known to describe regular black holes or traversable wormholes. However, one of the main issues lies on reconstructing the appropriate source that leads to such a solution. In this paper, a reconstruction method is implemented to show that such types of metrics can be well accommodated in non-linear electrodynamics with the presence of a scalar field. Some of the most important black bounces solutions are reconstructed in this framework, both in 3+1 as in 2+1 dimensions. For the first time in the literature, these solutions have an electrically charged source of matter from non-linear electrodynamics. Specific features are indicated that distinguish electric sources from magnetic ones, previously found for the same space-times.


title: Plane Hamiltonian Cycles in Convex Drawings

abstract: A conjecture by Rafla from 1988 asserts that every simple drawing of the complete graph $K_n$ admits a plane Hamiltonian cycle. It turned out that already the existence of much simpler non-crossing substructures in such drawings is hard to prove. Recent progress was made by Aichholzer et al. and by Suk and Zeng who proved the existence of a plane path of length $\Omega(\log n / \log \log n)$ and of a plane matching of size $\Omega(n^{1/2})$ in every simple drawing of $K_{n}$. Instead of studying simpler substructures, we prove Rafla's conjecture for the subclass of convex drawings, the most general class in the convexity hierarchy introduced by Arroyo et al. Moreover, we show that every convex drawing of $K_n$ contains a plane Hamiltonian path between each pair of vertices (Hamiltonian connectivity) and a plane $k$-cycle for each $3 \leq k \leq n$ (pancyclicity), and present further results on maximal plane subdrawings.


title: Comments Concerning a Hypothetical Mesoscopic Dark Dimension

abstract: Motivated by string-theoretic swampland conjectures, the existence of a dark fifth dimension, whose size is roughly 1 -- 10 microns, has been proposed. A great deal of supporting evidence has been presented, and definitive experimental tests are likely to be carried out. The basic idea is that the four-dimensional spacetime that we observe lives on a brane that is localized in the dark dimension. This short note points out that there are two distinct ways to realize such a scenario in string theory/M-theory. In the one considered previously the dark dimension is topologically a circle and our observable 4d spacetime is confined to a brane that is localized in a GUT-scale region of the circle. An alternative possibility is that the dark dimension is a line interval with branes attached at each end. This option would imply the existence of a parallel 4d spacetime microns away from us!


title: Toward Sustainable GenAI using Generation Directives for Carbon-Friendly Large Language Model Inference

abstract: The rapid advancement of Generative Artificial Intelligence (GenAI) across diverse sectors raises significant environmental concerns, notably the carbon emissions from their cloud and high performance computing (HPC) infrastructure. This paper presents Sprout, an innovative framework designed to address these concerns by reducing the carbon footprint of generative Large Language Model (LLM) inference services. Sprout leverages the innovative concept of "generation directives" to guide the autoregressive generation process, thereby enhancing carbon efficiency. Our proposed method meticulously balances the need for ecological sustainability with the demand for high-quality generation outcomes. Employing a directive optimizer for the strategic assignment of generation directives to user prompts and an original offline quality evaluator, Sprout demonstrates a significant reduction in carbon emissions by over 40% in real-world evaluations using the Llama2 LLM and global electricity grid data. This research marks a critical step toward aligning AI technology with sustainable practices, highlighting the potential for mitigating environmental impacts in the rapidly expanding domain of generative artificial intelligence.


title: Automated Tabletop Exfoliation and Identification of Monolayer Graphene Flakes

abstract: The discovery of graphene, one of the most-studied materials in condensed matter physics due to its singular mechanical, optical, and electronic properties, was enabled by manual ``Scotch Tape'' exfoliation. Nearly two decades later, this method is still widely used to obtain chemically-pristine flakes of graphene and other 2D van der Waals materials. Unfortunately, the yield of large, pristine flakes with uniform thickness is inconsistent. Thus, significant time and effort are required to exfoliate and locate flakes suitable for fabricating multilayer van der Waals heterostructures. Here, we describe a relatively affordable tabletop device (the ''eXfoliator'') that can reproducibly control key parameters and largely automate the exfoliation process. In a typical exfoliation run, the eXfoliator produces 3 or more large ($\ge400\ \mu$m$^2$) high-quality monolayers, allowing new users to produce large pristine graphene monolayers at a rate comparable to manual exfoliation by an experienced user. We use an automated mapping system and computer vision algorithm to locate candidate flakes.


title: Measurement of vector boson production cross sections and their ratios using $pp$ collisions at $\sqrt{s}=13.6$ TeV with the ATLAS detector

abstract: Fiducial and total $W^\pm$ and $Z$ boson cross sections, their ratios and the ratio of top-antitop-quark pair and $W$-boson fiducial cross sections are measured in proton-proton collisions at a centre-of-mass energy of $\sqrt{s}=13.6$ TeV, corresponding to an integrated luminosity of 29 fb$^{-1}$ of data collected in 2022 by the ATLAS experiment at the Large Hadron Collider. The measured fiducial cross-section values for $W^+\to \ell^+\nu$, $W^-\to \ell^-\bar{\nu}$, and $Z\to \ell^+\ell^-$ ($\ell=e$ or $\mu$) boson productions are $4250\pm 150$ pb, $3310\pm 120$ pb, and $744\pm 20$ pb, respectively, where the uncertainty is the total uncertainty, including that arising from the luminosity of about 2.2%. The measurements are in agreement with Standard-Model predictions calculated at next-to-next-to-leading-order in $\alpha_s$, next-to-next-to-leading logarithmic accuracy and next-to-leading-order electroweak accuracy.


title: Geodesic vector fields, induced contact structures and tightness in dimension three

abstract: In this paper, we provide new and simpler proofs of two theorems of Gluck and Harrison on contact structures induced by great circle or line fibrations. Furthermore, we prove that a geodesic vector field whose Jacobi tensor is parallel along flow lines (e.g. if the underlying manifold is locally symmetric) induces a contact structure if the 'mixed' sectional curvatures are nonnegative, and if a certain nondegeneracy condition holds. Additionally, we prove that in dimension three, contact structures admitting a Reeb flow which is either periodic, isometric, or free and proper, must be universally tight. In particular, we generalise an earlier result of Geiges and the author, by showing that every contact form on $\mathbb{R}^3$ whose Reeb vector field spans a line fibration is necessarily tight. Furthermore, we provide a characterisation of isometric Reeb vector fields. As an application, we recover a result of Kegel and Lange on Seifert fibrations spanned by Reeb vector fields, and we classify closed contact $3$-manifolds with isometric Reeb flows (also known as $R$-contact manifolds) up to diffeomorphism.


title: Social bots sour activist sentiment without eroding engagement

abstract: Social media platforms have witnessed a substantial increase in social bot activity, significantly affecting online discourse. Our study explores the dynamic nature of bot engagement related to Extinction Rebellion climate change protests from 18 November 2019 to 10 December 2019. We find that bots exert a greater influence on human behavior than vice versa during heated online periods. To assess the causal impact of human-bot communication, we compared communication histories between human users who directly interacted with bots and matched human users who did not. Our findings demonstrate a consistent negative impact of bot interactions on subsequent human sentiment, with exposed users displaying significantly more negative sentiment than their counterparts. Furthermore, the nature of bot interaction influences human tweeting activity and the sentiment towards protests. Political astroturfing bots increase activity, whereas other bots decrease it. Sentiment changes towards protests depend on the user's original support level, indicating targeted manipulation. However, bot interactions do not change activists' engagement towards protests. Despite the seemingly minor impact of individual bot encounters, the cumulative effect is profound due to the large volume of bot communication. Our findings underscore the importance of unrestricted access to social media data for studying the prevalence and influence of social bots, as with new technological advancements distinguishing between bots and humans becomes nearly impossible.


title: Ziv-Zakai-Optimal OFDM Resource Allocation for Time-of-Arrival Estimation

abstract: This paper presents methods of optimizing the placement and power allocations of pilots in an orthogonal frequency-division multiplexing (OFDM) signal to minimize time-of-arrival (TOA) estimation errors under power and resource allocation constraints. TOA errors in this optimization are quantified through the Ziv-Zakai bound (ZZB), which captures error thresholding effects caused by sidelobes in the signal's autocorrelation function (ACF) which are not captured by the Cramer-Rao lower bound. This paper is the first to solve for these ZZB-optimal allocations in the context of OFDM signals, under integer resource allocation constraints, and under both coherent and noncoherent reception. Under convex constraints, the optimization of the ZZB is proven to be convex; under integer constraints, the optimization is lower bounded by a convex relaxation and a branch-and-bound algorithm is proposed for efficiently allocating pilot resources. These allocations are evaluated by their ZZBs and ACFs, compared against a typical uniform allocation, and deployed on a software-defined radio TOA measurement platform to demonstrate their applicability in real-world systems.


title: TexDreamer: Towards Zero-Shot High-Fidelity 3D Human Texture Generation

abstract: Texturing 3D humans with semantic UV maps remains a challenge due to the difficulty of acquiring reasonably unfolded UV. Despite recent text-to-3D advancements in supervising multi-view renderings using large text-to-image (T2I) models, issues persist with generation speed, text consistency, and texture quality, resulting in data scarcity among existing datasets. We present TexDreamer, the first zero-shot multimodal high-fidelity 3D human texture generation model. Utilizing an efficient texture adaptation finetuning strategy, we adapt large T2I model to a semantic UV structure while preserving its original generalization capability. Leveraging a novel feature translator module, the trained model is capable of generating high-fidelity 3D human textures from either text or image within seconds. Furthermore, we introduce ArTicuLated humAn textureS (ATLAS), the largest high-resolution (1024 X 1024) 3D human texture dataset which contains 50k high-fidelity textures with text descriptions.


title: Discovery of a hot post-AGB star in Galactic globular cluster E3

abstract: We report a new hot post-asymptotic giant branch (PAGB) star in the Galactic globular cluster (GC) E3, which is one of the first of the identified PAGB stars in a GC to show a binary signature. The star stands out as the brightest source in E3 in the \mbox{{\em Astrosat}}/UVIT images. We confirmed its membership with the cluster E3 using Gaia DR3 kinematics and parallax measurements. We supplemented the photometric observations with radial velocities (RVs) from high-resolution spectroscopic observations at two epochs and with ground- and space-based photometric observations from 0.13 $\mu$m to 22 $\mu$m. We find that the RVs vary over $\sim$6 \kms\ between the two epochs. This is an indication of the star being in a binary orbit. A simulation of possible binary systems with the observed RVs suggests a binary period of either 39.12 days or 17.83 days with mass ratio q$\geq$1.0. The [Fe/H] derived using the high-resolution spectra is $\sim -$0.7 dex, which closely matches the cluster metallicity. The spectroscopic and photometric measurements suggest \Teff\ and $\log g$ of the star as 17\,500$\pm$1\,000~K and 2.37$\pm$0.20~dex, respectively. Various PAGB evolutionary tracks on the Hertzsprung--Russell (H-R) diagram suggest a current mass of the star in the range 0.51$-$0.55 \Msun. The star is enriched with C and O abundances, showing similar CNO abundances compared to the other PAGB stars in GCs with the evidence of the third dredge-up on the AGB phase.


title: Regularised Spectral Estimation for High-Dimensional Point Processes

abstract: Advances in modern technology have enabled the simultaneous recording of neural spiking activity, which statistically can be represented by a multivariate point process. We characterise the second order structure of this process via the spectral density matrix, a frequency domain equivalent of the covariance matrix. In the context of neuronal analysis, statistics based on the spectral density matrix can be used to infer connectivity in the brain network between individual neurons. However, the high-dimensional nature of spike train data mean that it is often difficult, or at times impossible, to compute these statistics. In this work, we discuss the importance of regularisation-based methods for spectral estimation, and propose novel methodology for use in the point process setting. We establish asymptotic properties for our proposed estimators and evaluate their performance on synthetic data simulated from multivariate Hawkes processes. Finally, we apply our methodology to neuroscience spike train data in order to illustrate its ability to infer brain connectivity.


title: Local reconstruction analysis of inverting the Radon transform in the plane from noisy discrete data

abstract: In this paper, we investigate the reconstruction error, $N_\e^{\text{rec}}(x)$, when a linear, filtered back-projection (FBP) algorithm is applied to noisy, discrete Radon transform data with sampling step size $\epsilon$ in two-dimensions. Specifically, we analyze $N_\e^{\text{rec}}(x)$ for $x$ in small, $O(\e)$-sized neighborhoods around a generic fixed point, $x_0$, in the plane, where the measurement noise values, $\eta_{k,j}$ (i.e., the errors in the sinogram space), are random variables. The latter are independent, but not necessarily identically distributed. We show, under suitable assumptions on the first three moments of the $\eta_{k,j}$, that the following limit exists: $N^{\text{rec}}(\chx;x_0) = \lim_{\e\to0}N_\e^{\text{rec}}(x_0+\e\chx)$, for $\check x$ in a bounded domain. Here, $N_\e^{\text{rec}}$ and $ N^{\text{rec}}$ are viewed as continuous random variables, and the limit is understood in the sense of distributions. Once the limit is established, we prove that $N^{\text{rec}}$ is a zero mean Gaussian random field and compute explicitly its covariance. In addition, we validate our theory using numerical simulations and pseudo random noise.


title: Yell At Your Robot: Improving On-the-Fly from Language Corrections

abstract: Hierarchical policies that combine language and low-level control have been shown to perform impressively long-horizon robotic tasks, by leveraging either zero-shot high-level planners like pretrained language and vision-language models (LLMs/VLMs) or models trained on annotated robotic demonstrations. However, for complex and dexterous skills, attaining high success rates on long-horizon tasks still represents a major challenge -- the longer the task is, the more likely it is that some stage will fail. Can humans help the robot to continuously improve its long-horizon task performance through intuitive and natural feedback? In this paper, we make the following observation: high-level policies that index into sufficiently rich and expressive low-level language-conditioned skills can be readily supervised with human feedback in the form of language corrections. We show that even fine-grained corrections, such as small movements ("move a bit to the left"), can be effectively incorporated into high-level policies, and that such corrections can be readily obtained from humans observing the robot and making occasional suggestions. This framework enables robots not only to rapidly adapt to real-time language feedback, but also incorporate this feedback into an iterative training scheme that improves the high-level policy's ability to correct errors in both low-level execution and high-level decision-making purely from verbal feedback. Our evaluation on real hardware shows that this leads to significant performance improvement in long-horizon, dexterous manipulation tasks without the need for any additional teleoperation. Videos and code are available at https://yay-robot.github.io/.


title: Boundary Layer Estimates in Stochastic Homogenization

abstract: We prove quantitative decay estimates for the boundary layer corrector in stochastic homogenization in the case of a half-space boundary. Our estimates are of optimal order and show that the gradient of the boundary layer corrector features nearly fluctuation-order decay; its expected value decays even one order faster. As a corollary, we deduce estimates on the accuracy of the representative volume element method for the computation of effective coefficients: our understanding of the decay of boundary layers enables us to improve the order of convergence of the RVE method for $d\geq 3$.


title: Strangers in a foreign land: 'Yeastizing' plant enzymes

abstract: Expressing plant metabolic pathways in microbial platforms is an efficient, cost-effective solution for producing many desired plant compounds. As eukaryotic organisms, yeasts are often the preferred platform. However, expression of plant enzymes in a yeast frequently leads to failure because the enzymes are poorly adapted to the foreign yeast cellular environment. Here we first summarize current engineering approaches for optimizing performance of plant enzymes in yeast. A critical limitation of these approaches is that they are labor-intensive and must be customized for each individual enzyme, which significantly hinders the establishment of plant pathways in cellular factories. In response to this challenge, we propose the development of a cost-effective computational pipeline to redesign plant enzymes for better adaptation to the yeast cellular milieu. This proposition is underpinned by compelling evidence that plant and yeast enzymes exhibit distinct sequence features that are generalizable across enzyme families. Consequently, we introduce a data-driven machine learning framework designed to extract 'yeastizing' rules from natural protein sequence variations, which can be broadly applied to all enzymes. Additionally, we discuss the potential to integrate the machine learning model into a full design-build-test-cycle.


title: Fact Checking Chatbot: A Misinformation Intervention for Instant Messaging Apps and an Analysis of Trust in the Fact Checkers

abstract: In Singapore, there has been a rise in misinformation on mobile instant messaging services (MIMS). MIMS support both small peer-to-peer networks and large groups. Misinformation in the former may spread due to recipients' trust in the sender while in the latter, misinformation can directly reach a wide audience. The encryption of MIMS makes it difficult to address misinformation directly. As such, chatbots have become an alternative solution where users can disclose their chat content directly to fact checking services. To understand how effective fact checking chatbots are as an intervention and how trust in three different fact checkers (i.e., Government, News Outlets, and Artificial Intelligence) may affect this trust, we conducted a within-subjects experiment with 527 Singapore residents. We found mixed results for the fact checkers but support for the chatbot intervention overall. We also found a striking contradiction between participants' trust in the fact checkers and their behaviour towards them. Specifically, those who reported a high level of trust in the government performed worse and tended to follow the fact checking tool less when it was endorsed by the government.


title: Unitarity constraints on large multiplets of arbitrary gauge groups

abstract: We impose partial-wave unitarity on $2 \to 2$ tree-level scattering processes to derive constraints on the dimensions of large scalar and fermionic multiplets of arbitrary gauge groups. We apply our results to scalar and fermionic extensions of the Standard Model, and also to the Grand Unified Theories (GUTs) based on the groups $SU(5)$, $SO(10)$, and $E_6$. We find scenarios within the latter two GUTs that violate the unitarity condition; this may require a reevaluation of the validity of perturbation theory in those scenarios.


title: Ultra-High-Resolution Image Synthesis with Pyramid Diffusion Model

abstract: We introduce the Pyramid Diffusion Model (PDM), a novel architecture designed for ultra-high-resolution image synthesis. PDM utilizes a pyramid latent representation, providing a broader design space that enables more flexible, structured, and efficient perceptual compression which enable AutoEncoder and Network of Diffusion to equip branches and deeper layers. To enhance PDM's capabilities for generative tasks, we propose the integration of Spatial-Channel Attention and Res-Skip Connection, along with the utilization of Spectral Norm and Decreasing Dropout Strategy for the Diffusion Network and AutoEncoder. In summary, PDM achieves the synthesis of images with a 2K resolution for the first time, demonstrated on two new datasets comprising images of sizes 2048x2048 pixels and 2048x1024 pixels respectively. We believe that this work offers an alternative approach to designing scalable image generative models, while also providing incremental reinforcement for existing frameworks.


title: Effects of Automated Misinformation Warning Labels on the Intents to Like, Comment and Share Posts

abstract: With fact-checking by professionals being difficult to scale on social media, algorithmic techniques have been considered. However, it is uncertain how the public may react to labels by automated fact-checkers. In this study, we investigate the use of automated warning labels derived from misinformation detection literature and investigate their effects on three forms of post engagement. Focusing on political posts, we also consider how partisanship affects engagement. In a two-phases within-subjects experiment with 200 participants, we found that the generic warnings suppressed intents to comment on and share posts, but not on the intent to like them. Furthermore, when different reasons for the labels were provided, their effects on post engagement were inconsistent, suggesting that the reasons could have undesirably motivated engagement instead. Partisanship effects were observed across the labels with higher engagement for politically congruent posts. We discuss the implications on the design and use of automated warning labels.


title: When is Trust Robust?

abstract: We examine an economy in which interactions are more productive if agents can trust others to refrain from cheating. Some agents are scoundrels, who always cheat, while others cheat only if the cost of cheating, a decreasing function of the proportion of cheaters, is sufficiently low. The economy exhibits multiple equilibria. As the proportion of scoundrels in the economy declines, the high-trust equilibrium can be disrupted by arbitrarily small perturbations or infusions of low-trust agents, while the low-trust equilibrium becomes impervious to perturbations and infusions of high-trust agents. The resilience of trust may thus hinge upon the prevalence of scoundrels.


title: Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts

abstract: Pretrained Language Models (PLMs) have advanced Natural Language Processing (NLP) tasks significantly, but finetuning PLMs on low-resource datasets poses significant challenges such as instability and overfitting. Previous methods tackle these issues by finetuning a strategically chosen subnetwork on a downstream task, while keeping the remaining weights fixed to the pretrained weights. However, they rely on a suboptimal criteria for sub-network selection, leading to suboptimal solutions. To address these limitations, we propose a regularization method based on attention-guided weight mixup for finetuning PLMs. Our approach represents each network weight as a mixup of task-specific weight and pretrained weight, controlled by a learnable attention parameter, providing finer control over sub-network selection. Furthermore, we employ a bi-level optimization (BLO) based framework on two separate splits of the training dataset, improving generalization and combating overfitting. We validate the efficacy of our proposed method through extensive experiments, demonstrating its superiority over previous methods, particularly in the context of finetuning PLMs on low-resource datasets.


title: Generalized Ramsey--Tur\'an density for cliques

abstract: We study the generalized Ramsey--Tur\'an function $\mathrm{RT}(n,K_s,K_t,o(n))$, which is the maximum possible number of copies of $K_s$ in an $n$-vertex $K_t$-free graph with independence number $o(n)$. The case when $s=2$ was settled by Erd{\H{o}}s, S{\'o}s, Bollob{\'a}s, Hajnal, and Szemer\'{e}di in the 1980s. We combinatorially resolve the general case for all $s\ge 3$, showing that the (asymptotic) extremal graphs for this problem have simple (bounded) structures. In particular, it implies that the extremal structures follow a periodic pattern when $t$ is much larger than $s$. Our results disprove a conjecture of Balogh, Liu, and Sharifzadeh and show that a relaxed version does hold.


abstract: In this paper, we introduce Semantic Layering in Room Segmentation via LLMs (SeLRoS), an advanced method for semantic room segmentation by integrating Large Language Models (LLMs) with traditional 2D map-based segmentation. Unlike previous approaches that solely focus on the geometric segmentation of indoor environments, our work enriches segmented maps with semantic data, including object identification and spatial relationships, to enhance robotic navigation. By leveraging LLMs, we provide a novel framework that interprets and organizes complex information about each segmented area, thereby improving the accuracy and contextual relevance of room segmentation. Furthermore, SeLRoS overcomes the limitations of existing algorithms by using a semantic evaluation method to accurately distinguish true room divisions from those erroneously generated by furniture and segmentation inaccuracies. The effectiveness of SeLRoS is verified through its application across 30 different 3D environments. Source code and experiment videos for this work are available at: https://sites.google.com/view/selros.


title: Probing the Fermi surface with Quantum Oscillation Measurements in the Dirac semimetal TaNiTe$_5$

abstract: We report a detailed investigation of the Fermi surface in the layered Dirac semimetal TaNiTe$_5$, which hosts quasi-one-dimensional (quasi-1D) topological electronic properties. We have investigated the magnetoresistance and magnetic behaviour in high-quality TaNiTe$_5$ single crystals. Pronounced Shubnikov-de Haas (SdH) and de Haas-van Alphen (dHvA) oscillations are observed in magnetic fields above 3T and at temperatures of up to 22K. Five fundamental frequencies and light effective quasiparticle masses are obtained by fast Fourier transformation (FFT) and Lifshitz-Kosevich (LK) formula fits. The high resolution of the low-temperature FFT spectra allows us to investigate individual peaks in detail for the magnetic fields applied along all three crystallographic axes and in the a-c plane. The extracted values for the Berry phase of the obtained Fermi pockets approach $\pi$, demonstrating the nontrivial topological electronic properties of layered TaNiTe$_5$. The observed quantum oscillations in the magnetoresistance and magnetization data are consistent with ultra-high mobility Dirac carriers in the conduction band.


title: Contextual AD Narration with Interleaved Multimodal Sequence

abstract: The Audio Description (AD) task aims to generate descriptions of visual elements for visually impaired individuals to help them access long-form video contents, like movie. With video feature, text, character bank and context information as inputs, the generated ADs are able to correspond to the characters by name and provide reasonable, contextual descriptions to help audience understand the storyline of movie. To achieve this goal, we propose to leverage pre-trained foundation models through a simple and unified framework to generate ADs with interleaved multimodal sequence as input, termed as Uni-AD. To enhance the alignment of features across various modalities with finer granularity, we introduce a simple and lightweight module that maps video features into the textual feature space. Moreover, we also propose a character-refinement module to provide more precise information by identifying the main characters who play more significant role in the video context. With these unique designs, we further incorporate contextual information and a contrastive loss into our architecture to generate more smooth and contextual ADs. Experiments on the MAD-eval dataset show that Uni-AD can achieve state-of-the-art performance on AD generation, which demonstrates the effectiveness of our approach. Code will be available at https://github.com/MCG-NJU/Uni-AD.


title: Solving Combinatorial Pricing Problems using Embedded Dynamic Programming Models

abstract: The combinatorial pricing problem (CPP) is a bilevel problem in which the leader maximizes their revenue by imposing tolls on certain items that they can control. Based on the tolls set by the leader, the follower selects a subset of items corresponding to an optimal solution of a combinatorial optimization problem. To accomplish the leader's goal, the tolls need to be sufficiently low to discourage the follower from choosing the items offered by the competitors. In this paper, we derive a single-level reformulation for the CPP by rewriting the follower's problem as a longest path problem using a dynamic programming model, and then taking its dual and applying strong duality. We proceed to solve the reformulation in a dynamic fashion with a cutting plane method. We apply this methodology to 2 distinct dynamic programming models, namely, a novel formulation designated as selection diagram and the well-known decision diagram. We also produce numerical results to evaluate their performances across 3 different specializations of the CPP and a closely related problem that is the knapsack interdiction problem. Our results showcase the potential of the 2 proposed reformulations over the natural value function approach, expanding the set of tools to solve combinatorial bilevel programs.


title: Supporting Energy Policy Research with Large Language Models

abstract: The recent growth in renewable energy development in the United States has been accompanied by a simultaneous surge in renewable energy siting ordinances. These zoning laws play a critical role in dictating the placement of wind and solar resources that are critical for achieving low-carbon energy futures. In this context, efficient access to and management of siting ordinance data becomes imperative. The National Renewable Energy Laboratory (NREL) recently introduced a public wind and solar siting database to fill this need. This paper presents a method for harnessing Large Language Models (LLMs) to automate the extraction of these siting ordinances from legal documents, enabling this database to maintain accurate up-to-date information in the rapidly changing energy policy landscape. A novel contribution of this research is the integration of a decision tree framework with LLMs. Our results show that this approach is 85 to 90% accurate with outputs that can be used directly in downstream quantitative modeling. We discuss opportunities to use this work to support similar large-scale policy research in the energy sector. By unlocking new efficiencies in the extraction and analysis of legal documents using LLMs, this study enables a path forward for automated large-scale energy policy research.


title: On the $N$-waves hierarchy with constant boundary conditions. Spectral properties

abstract: The paper is devoted to $N$-wave equations with constant boundary conditions related to symplectic Lie algebras. We study the spectral properties of a class of Lax operators $L$, whose potentials $Q(x,t)$ tend to constants $Q_\pm$ for $x\to \pm \infty$. For special choices of $Q_\pm$ we outline the spectral properties of $L$, the direct scattering transform and construct its fundamental analytic solutions. We generalise Wronskian relations for the case of CBC -- this allows us to analyse the mapping between the scattering data and the $x$-derivative of the potential $Q_x$. Next, using the Wronskian relations we derive the dispersion laws for the $N$-wave hierarchy and describe the NLEE related to the given Lax operator.


title: Decoherence-free algebras in quantum dynamics

abstract: In this Article we analyze the algebraic properties of the asymptotic dynamics of finite-dimensional open quantum systems in the Heisenberg picture. In particular, a natural product (Choi-Effros product) can be defined in the asymptotic regime. Motivated by this structure, we introduce a new space called the Choi-Effros decoherence-free algebra. Interestingly, this space is both a C* -algebra with respect to the composition product, and a B* -algebra with respect to the Choi-Effros product. Moreover, such space admits a direct-sum decomposition revealing a clear relationship with the attractor subspace of the dynamics. In particular, the equality between the attractor subspace and the Choi-Effros decoherence-free algebra is a necessary and sufficient condition for a faithful dynamics. Finally, we show how all the findings do not rely on complete positivity but on the much weaker Schwarz property.


title: Feedback and galaxy dynamics: A study of turbulence and star formation in 34 galaxies using the PHANGS survey

abstract: The correlation between interstellar turbulent speed and local star formation rate surface density, Sigma_SFR, is studied using CO observations in the PHANGS survey. The local velocity dispersion of molecular gas, sigma, increases with Sigma_SFR, but the virial parameter, alpha_vir, is about constant, suggesting the molecular gas remains self-gravitating. The correlation arises because sigma depends on the molecular surface density, Sigma_mol, and object cloud mass, M_mol, with the usual molecular cloud correlations, while Sigma_SFR increases with both of these quantities because of a nearly constant star formation efficiency for CO. Pressure fluctuations with Delta Sigma_SFR are also examined. Azimuthal variations of molecular pressure, Delta P_mol, have a weaker correlation with Delta Sigma_SFR than expected from the power-law correlation between the total quantities, suggesting slightly enhanced SFR efficiency per molecule in spiral arms. Dynamical equilibrium pressure and star formation rate correlate well for the whole sample, as P_DE~Sigma_SFR^1.3, which is steeper than in other studies. The azimuthal fluctuations, Delta P_DE(Delta Sigma_SFR), follow the total correlation P_DE(Sigma_SFR) closely, hinting that some of this correlation may be a precursor to star formation, rather than a reaction. Galactic dynamical processes correlate linearly such that Sigma_SFR~(Sigma_gas R)^(1.0\pm0.3) for total gas surface density Sigma_gas and galactic dynamical rates, R, equal to kappa, A, or Omega, representing epicyclic frequency, shear rate A, and orbit rate Omega. These results suggest important roles for both feedback and galactic dynamics.


title: Rapid AIdeation: Generating Ideas With the Self and in Collaboration With Large Language Models

abstract: Generative artificial intelligence (GenAI) can rapidly produce large and diverse volumes of content. This lends to it a quality of creativity which can be empowering in the early stages of design. In seeking to understand how creative ways to address practical issues can be conceived between humans and GenAI, we conducted a rapid ideation workshop with 21 participants where they used a large language model (LLM) to brainstorm potential solutions and evaluate them. We found that the LLM produced a greater variety of ideas that were of high quality, though not necessarily of higher quality than human-generated ideas. Participants typically prompted in a straightforward manner with concise instructions. We also observed two collaborative dynamics with the LLM fulfilling a consulting role or an assisting role depending on the goals of the users. Notably, we observed an atypical anti-collaboration dynamic where participants used an antagonistic approach to prompt the LLM.


title: Observation of the anomalous Nernst effect in altermagnetic candidate Mn5Si3

abstract: The anomalous Nernst effect generates transverse voltage to the applied thermal gradient in magnetically ordered systems. The effect was previously considered excluded in compensated magnetic materials with collinear ordering. However, in the recently identified class of compensated magnetic materials, dubbed altermagnets, time-reversal symmetry breaking in the electronic band structure makes the presence of the anomalous Nernst effect possible despite the collinear spin arrangement. In this work, we investigate epitaxial Mn5Si3 thin films known to be an altermagnetic candidate. We show that the material manifests a sizable anomalous Nernst coefficient despite the small net magnetization of the films. The measured magnitudes of the anomalous Nernst coefficient reach a scale of microVolts per Kelvin. We support our magneto-thermoelectric measurements by density-functional theory calculations of the material's spin-split electronic structure, which allows for the finite Berry curvature in the reciprocal space. Furthermore, we present our calculations of the intrinsic Berry-curvature Nernst conductivity, which agree with our experimental observations.


title: Identifiability and Observability of Nonsmooth Systems via Taylor-like Approximations

abstract: New sensitivity-based methods are developed for determining identifiability and observability of nonsmooth input-output systems. More specifically, lexicographic calculus is used to construct nonsmooth sensitivity rank condition (SERC) tests, which we call lexicographic SERC (L-SERC) tests. The introduced L-SERC tests are: (i) practically implementable and amenable to large-scale problems; (ii) accurate since they directly treat the nonsmoothness while avoiding, e.g., smoothing approximations; and (iii) analogous to (and indeed recover) their smooth counterparts. To accomplish this, a first-order Taylor-like approximation theory is developed using lexicographic differentiation to directly treat nonsmooth functions. A practically implementable algorithm is proposed that determines partial structural identifiability or observability, a useful characterization in the nonsmooth setting. Lastly, the theory is illustrated through an application in climate modeling.


title: You Only Sample Once: Taming One-Step Text-to-Image Synthesis by Self-Cooperative Diffusion GANs

abstract: We introduce YOSO, a novel generative model designed for rapid, scalable, and high-fidelity one-step image synthesis. YOSO integrates the diffusion process with GANs to achieve the best of two worlds. Specifically, we smooth the distribution by the denoising generator itself, performing self-cooperative learning. We show that our method can serve as a one-step generation model training from scratch with competitive performance. Moreover, we show that our method can be extended to finetune pre-trained text-to-image diffusion for high-quality one-step text-to-image synthesis even with LoRA fine-tuning. In particular, we provide the first diffusion transformer that can generate images in one step trained on 512 resolution, with the capability of adapting to 1024 resolution without extra explicit training. Our code is provided at https://github.com/Luo-Yihong/YOSO


title: Revisiting the abundance pattern and charge-exchange emission in the M82 centre

abstract: The interstellar medium (ISM) in starburst galaxies contains plenty of chemical elements synthesised by core-collapse supernova explosions. By measuring the abundances of these metals, we can study the chemical enrichment within galaxies and the transportation of metals into circumgalactic environments through powerful outflows. We perform the spectral analysis of the X-ray emissions from the M82 core using the Reflection Grating Spectrometer (RGS) onboard XMM-Newton to accurately estimate the metal abundances in the ISM. We analyse over 300 ks of RGS data observed with fourteen position angles, covering an 80 arcsec cross-dispersion width. We employ multi-temperature thermal plasma components in collisional ionisation equilibrium (CIE) to reproduce the observed spectra, each exhibiting different spatial broadenings. The O vii band CCD image shows a broader distribution compared to those for O viii and Fe-L bands. The O viii line profiles have a prominent double-peaked structure, corresponding to the northward and southward outflows. The O vii triplet feature exhibits marginal peaks, and a single CIE component, convolved with the O vii band image, approximately reproduces the spectral shape. Combining a CIE model with a charge-exchange emission model also successfully reproduces the O vii line profiles. However, the ratio of these two components varies significantly with the observed position angles, which is physically implausible. Spectral fitting of the broadband spectra suggests a multi-temperature phase in the ISM, approximated by three components at 0.1, 0.4, and 0.7 keV. Notably, the 0.1 keV component exhibits a broader distribution than the 0.4 and 0.7 keV plasmas. The derived abundance pattern shows super-solar N/O, solar Ne/O and Mg/O, and half-solar Fe/O ratios. These results indicate the chemical enrichments by core-collapse supernovae in starburst galaxies.


title: Zero-Reference Low-Light Enhancement via Physical Quadruple Priors

abstract: Understanding illumination and reducing the need for supervision pose a significant challenge in low-light enhancement. Current approaches are highly sensitive to data usage during training and illumination-specific hyper-parameters, limiting their ability to handle unseen scenarios. In this paper, we propose a new zero-reference low-light enhancement framework trainable solely with normal light images. To accomplish this, we devise an illumination-invariant prior inspired by the theory of physical light transfer. This prior serves as the bridge between normal and low-light images. Then, we develop a prior-to-image framework trained without low-light data. During testing, this framework is able to restore our illumination-invariant prior back to images, automatically achieving low-light enhancement. Within this framework, we leverage a pretrained generative diffusion model for model ability, introduce a bypass decoder to handle detail distortion, as well as offer a lightweight version for practicality. Extensive experiments demonstrate our framework's superiority in various scenarios as well as good interpretability, robustness, and efficiency. Code is available on our project homepage: http://daooshee.github.io/QuadPrior-Website/


title: Chemical differentiation and gas kinematics around massive young stellar objects in RCW 120

abstract: We present results of a spectral survey towards a dense molecular condensation and young stellar objects (YSOs) projected on the border of the HII region RCW 120 and discuss emission of 20 molecules which produce the brightest lines. The survey was performed with the APEX telescope in the frequency range 200 -- 260 GHz. We provide evidences for two outflows in the dense gas. The first one is powered by the RCW 120 S2 YSO and oriented along the line of sight. The second outflow around RCW 120 S1 is aligned almost perpendicular to the line of sight. We show that area with bright emission of CH$_3$OH, CH$_3$CCH and CH$_3$CN are organised into an onion-like structure where CH$_3$CN traces warmer regions around the YSOs than the other molecules. Methanol seems to be released to the gas phase by shock waves in the vicinity of the outflows while thermal evaporation still does not work towards the YSOs. We find only a single manifestation of the UV radiation to the molecules, namely, enhanced abundances of small hydrocarbons CCH and c-C$_3$H$_2$ in the photo-dissociation region.


title: Segment Anything for comprehensive analysis of grapevine cluster architecture and berry properties

abstract: Grape cluster architecture and compactness are complex traits influencing disease susceptibility, fruit quality, and yield. Evaluation methods for these traits include visual scoring, manual methodologies, and computer vision, with the latter being the most scalable approach. Most of the existing computer vision approaches for processing cluster images often rely on conventional segmentation or machine learning with extensive training and limited generalization. The Segment Anything Model (SAM), a novel foundation model trained on a massive image dataset, enables automated object segmentation without additional training. This study demonstrates out-of-the-box SAM's high accuracy in identifying individual berries in 2D cluster images. Using this model, we managed to segment approximately 3,500 cluster images, generating over 150,000 berry masks, each linked with spatial coordinates within their clusters. The correlation between human-identified berries and SAM predictions was very strong (Pearson r2=0.96). Although the visible berry count in images typically underestimates the actual cluster berry count due to visibility issues, we demonstrated that this discrepancy could be adjusted using a linear regression model (adjusted R2=0.87). We emphasized the critical importance of the angle at which the cluster is imaged, noting its substantial effect on berry counts and architecture. We proposed different approaches in which berry location information facilitated the calculation of complex features related to cluster architecture and compactness. Finally, we discussed SAM's potential integration into currently available pipelines for image generation and processing in vineyard conditions.


title: Automatic Information Extraction From Employment Tribunal Judgements Using Large Language Models

abstract: Court transcripts and judgments are rich repositories of legal knowledge, detailing the intricacies of cases and the rationale behind judicial decisions. The extraction of key information from these documents provides a concise overview of a case, crucial for both legal experts and the public. With the advent of large language models (LLMs), automatic information extraction has become increasingly feasible and efficient. This paper presents a comprehensive study on the application of GPT-4, a large language model, for automatic information extraction from UK Employment Tribunal (UKET) cases. We meticulously evaluated GPT-4's performance in extracting critical information with a manual verification process to ensure the accuracy and relevance of the extracted data. Our research is structured around two primary extraction tasks: the first involves a general extraction of eight key aspects that hold significance for both legal specialists and the general public, including the facts of the case, the claims made, references to legal statutes, references to precedents, general case outcomes and corresponding labels, detailed order and remedies and reasons for the decision. The second task is more focused, aimed at analysing three of those extracted features, namely facts, claims and outcomes, in order to facilitate the development of a tool capable of predicting the outcome of employment law disputes. Through our analysis, we demonstrate that LLMs like GPT-4 can obtain high accuracy in legal information extraction, highlighting the potential of LLMs in revolutionising the way legal information is processed and utilised, offering significant implications for legal research and practice.


title: Elucidating the Nature of $\pi$-hydrogen Bonding in Liquid Water and Ammonia

abstract: Aromatic compounds form an unusual kind of hydrogen bond with water and ammonia molecules, known as the $\pi$-hydrogen bond. In this work, we report ab initio path integral molecular dynamics simulations enhanced by machine-learning potentials to study the structural, dynamical, and spectroscopic properties of solutions of benzene in liquid water and ammonia. Specifically, we model the spatial distribution functions of the solvents around the benzene molecule, establish the $\pi$-hydrogen bonding interaction as a prominent structural motive, and set up existence criteria to distinguish the $\pi$-hydrogen bonded configurations. These serve as a structural basis to calculate binding affinities of the solvent molecules in $\pi$hydrogen bonds, identify an anticooperativity effect across the aromatic ring in water (but not ammonia), and estimate $\pi$-hydrogen bond lifetimes in both solvents. Finally, we model hydration-shell-resolved vibrational spectra to clearly identify the vibrational signature of this structural motif in our simulations. These decomposed spectra corroborate previous experimental findings for benzene in water, offer additional insights, and further emphasize the contrast between $\pi$-hydrogen bonds in water and in ammonia. Our simulations provide a comprehensive picture of the studied phenomenon and, at the same time, serve as a meaningful \textit{ab initio} reference for an accurate description of $\pi$-hydrogen bonding using empirical force fields in more complex situations, such as the hydration of biological interfaces.


title: Neural Differential Algebraic Equations

abstract: Differential-Algebraic Equations (DAEs) describe the temporal evolution of systems that obey both differential and algebraic constraints. Of particular interest are systems that contain implicit relationships between their components, such as conservation relationships. Here, we present Neural Differential-Algebraic Equations (NDAEs) suitable for data-driven modeling of DAEs. This methodology is built upon the concept of the Universal Differential Equation; that is, a model constructed as a system of Neural Ordinary Differential Equations informed by theory from particular science domains. In this work, we show that the proposed NDAEs abstraction is suitable for relevant system-theoretic data-driven modeling tasks. Presented examples include (i) the inverse problem of tank-manifold dynamics and (ii) discrepancy modeling of a network of pumps, tanks, and pipes. Our experiments demonstrate the proposed method's robustness to noise and extrapolation ability to (i) learn the behaviors of the system components and their interaction physics and (ii) disambiguate between data trends and mechanistic relationships contained in the system.


title: Hidden Zeros in Scaffolded General Relativity and Exceptional Field Theories

abstract: It was recently discovered by Arkani-Hamed et al that the colour-ordered scattering amplitudes of Tr$(\Phi^3)$, the NLSM and scaffolded YM vanish at specific loci. We build on this observation and demonstrate that, beyond colour ordering, scattering amplitudes can display multiple instances of these hidden zeros. A first example are the flavour-ordered amplitudes of scaffolded General Relativity, corresponding to graviton exchange. The other two cases are multi-flavour DBI and the special Galileon. These are the natural generalisations of the NLSM with multiple instances of Adler's zero. The zero loci naturally define factorisation limits; in all three cases these include a metric-coupled cubic scalar theory. We comment on the general picture of hidden zeros and the relation to the double copy.


title: Tournaments and random walks

abstract: We study the relationship between tournaments and random walks. This connection was first observed by Erd\H{o}s and Moser. Winston and Kleitman came close to showing that $S_n=\Theta(4^n/n^{5/2})$. Building on this, and works by Tak\'acs, these asymptotic bounds were confirmed by Kim and Pittel. In this work, we verify Moser's conjecture that $S_n\sim C4^n/n^{5/2}$, using limit theory for integrated random walk bridges. Moreover, we show that $C$ can be described in terms of random walks. Combining this with a recent proof and number-theoretic description of $C$ by the second author, we obtain an analogue of Louchard's formula, for the Laplace transform of the squared Brownian excursion/Airy area measure. Finally, we describe the scaling limit of random score sequences, in terms of the Kolmogorov excursions, studied recently by B\"{a}r, Duraj and Wachtel. Our results can also be interpreted as answering questions related to a class of random polymers, which began with influential work of Sina\u{i}. From this point of view, our methods yield the precise asymptotics of a persistence probability, related to the pinning/wetting models from statistical physics, that was estimated up to constants by Aurzada, Dereich and Lifshits, as conjectured by Caravenna and Deuschel.


title: Asymptotics for Sina\u{i} excursions

abstract: We study a class of random polymers, introduced by Sina\u{i}, which are related to persistence probabilities in integrated simple random walk bridges. We find the precise asymptotics of these probabilities, and describe their combinatorics, using limit theory for infinitely divisible distributions, and the number-theoretic subset counting formulas of von Sterneck from the early 1900s. Our results sharpen estimates by Aurzada, Dereich and Lifshits, and respond to a conjecture of Caravenna and Deuschel, which arose in their study of the pinning/wetting models, for random linear chains exhibiting entropic repulsion. Our key combinatorial result is an analogue of Sparre Andersen's classical formula.


title: The physics of Core-Collapse Supernovae: explosion mechanism and explosive nucleosynthesis

abstract: Recent developments in multi-dimensional simulations of core-collapse supernovae have considerably improved our understanding of this complex phenomenon. In addition to that, one-dimensional (1D) studies have been employed to study the explosion mechanism and its causal connection to the pre-collapse structure of the star, as well as to explore the vast parameter space of supernovae. Nonetheless, many uncertainties still affect the late stages of the evolution of massive stars, their collapse, and the subsequent shock propagation. In this review, we will briefly summarize the state-of-the-art of both 1D and 3D simulations and how they can be employed to study the evolution of massive stars, supernova explosions, and shock propagation, focusing on the uncertainties that affect each of these phases. Finally, we will illustrate the typical nucleosynthesis products that emerge from the explosion.


title: Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers

abstract: While large-scale robotic systems typically rely on textual instructions for tasks, this work explores a different approach: can robots infer the task directly from observing humans? This shift necessitates the robot's ability to decode human intent and translate it into executable actions within its physical constraints and environment. We introduce Vid2Robot, a novel end-to-end video-based learning framework for robots. Given a video demonstration of a manipulation task and current visual observations, Vid2Robot directly produces robot actions. This is achieved through a unified representation model trained on a large dataset of human video and robot trajectory. The model leverages cross-attention mechanisms to fuse prompt video features to the robot's current state and generate appropriate actions that mimic the observed task. To further improve policy performance, we propose auxiliary contrastive losses that enhance the alignment between human and robot video representations. We evaluate Vid2Robot on real-world robots, demonstrating a 20% improvement in performance compared to other video-conditioned policies when using human demonstration videos. Additionally, our model exhibits emergent capabilities, such as successfully transferring observed motions from one object to another, and long-horizon composition, thus showcasing its potential for real-world applications. Project website: vid2robot.github.io


title: Statistical analysis of IUVB values in Mexico City from 2000 to 2022

abstract: The solar radiation are electromagnetic waves, composed of infrared, visible spectrum and ultraviolet. The infrared component is the cause of thermal energy, the visible spectrum allows to see and the ultraviolet component is the most energetic part and dangerous for the human body (skin and eyes). The ultraviolet rays are divided in a wavelength range, in three parts; called UVA (100-280 nm), UVB (280-315 nm) and UVC (315-400 nm). The UVC are the most energetic (followed by the UVB rays ), however are stopped in the ozone layer. In this work it shown a statistical analysis of the UVB index measured by five station in Mexico City, from the years 2000 to 2022. Through a Gaussian fit distribution, it was found that the range when the IUVB value exceeds the value of seven, which is from 11:00 to 16:00, having an average mean value of 13:09 hrs. $\pm$ 4 min. i.e. the time at which the UVB index reaches its maximum value. This occurs in the months from February to October. More than 80% of radiation is for UVB values less than 7 and less than the remaining 20% is for IUVB values greater than 7. Finally, it was possible to observe that the mean value per year, reaches its maximum when the solar cycles occurs, which was in the years 2003 and 2013.


title: DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset

abstract: The creation of large, diverse, high-quality robot manipulation datasets is an important stepping stone on the path toward more capable and robust robotic manipulation policies. However, creating such datasets is challenging: collecting robot manipulation data in diverse environments poses logistical and safety challenges and requires substantial investments in hardware and human labour. As a result, even the most general robot manipulation policies today are mostly trained on data collected in a small number of environments with limited scene and task diversity. In this work, we introduce DROID (Distributed Robot Interaction Dataset), a diverse robot manipulation dataset with 76k demonstration trajectories or 350 hours of interaction data, collected across 564 scenes and 84 tasks by 50 data collectors in North America, Asia, and Europe over the course of 12 months. We demonstrate that training with DROID leads to policies with higher performance and improved generalization ability. We open source the full dataset, policy learning code, and a detailed guide for reproducing our robot hardware setup.


title: Sample Complexity of Offline Distributionally Robust Linear Markov Decision Processes

abstract: In offline reinforcement learning (RL), the absence of active exploration calls for attention on the model robustness to tackle the sim-to-real gap, where the discrepancy between the simulated and deployed environments can significantly undermine the performance of the learned policy. To endow the learned policy with robustness in a sample-efficient manner in the presence of high-dimensional state-action space, this paper considers the sample complexity of distributionally robust linear Markov decision processes (MDPs) with an uncertainty set characterized by the total variation distance using offline data. We develop a pessimistic model-based algorithm and establish its sample complexity bound under minimal data coverage assumptions, which outperforms prior art by at least $\widetilde{O}(d)$, where $d$ is the feature dimension. We further improve the performance guarantee of the proposed algorithm by incorporating a carefully-designed variance estimator.


title: Fundamental limitations on the recoverability of quantum processes

abstract: Quantum information processing and computing tasks can be understood as quantum networks, comprising quantum states and channels and possible physical transformations on them. It is hence pertinent to estimate the change in informational content of quantum processes due to physical transformations they undergo. The physical transformations of quantum states are described by quantum channels, while the transformations of quantum channels are described by quantum superchannels. In this work, we determine fundamental limitations on how well can the physical transformation on quantum channels be undone or reversed, which are of crucial interest to design and benchmark quantum information and computation devices. In particular, we refine the quantum data processing inequality for quantum channels by strengthening the monotonicity inequality of quantum relative entropy of quantum channels under the action of an arbitrary quantum superchannel. We identify a class of quantum superchannels, which appears to be quantum superchannel analogue of quantum subunital channels, under the action of which the entropy of an arbitrary quantum channel is nondecreasing.


title: On Safety in Safe Bayesian Optimization

abstract: Optimizing an unknown function under safety constraints is a central task in robotics, biomedical engineering, and many other disciplines, and increasingly safe Bayesian Optimization (BO) is used for this. Due to the safety critical nature of these applications, it is of utmost importance that theoretical safety guarantees for these algorithms translate into the real world. In this work, we investigate three safety-related issues of the popular class of SafeOpt-type algorithms. First, these algorithms critically rely on frequentist uncertainty bounds for Gaussian Process (GP) regression, but concrete implementations typically utilize heuristics that invalidate all safety guarantees. We provide a detailed analysis of this problem and introduce Real-\b{eta}-SafeOpt, a variant of the SafeOpt algorithm that leverages recent GP bounds and thus retains all theoretical guarantees. Second, we identify assuming an upper bound on the reproducing kernel Hilbert space (RKHS) norm of the target function, a key technical assumption in SafeOpt-like algorithms, as a central obstacle to real-world usage. To overcome this challenge, we introduce the Lipschitz-only Safe Bayesian Optimization (LoSBO) algorithm, which guarantees safety without an assumption on the RKHS bound, and empirically show that this algorithm is not only safe, but also exhibits superior performance compared to the state-of-the-art on several function classes. Third, SafeOpt and derived algorithms rely on a discrete search space, making them difficult to apply to higher-dimensional problems. To widen the applicability of these algorithms, we introduce Lipschitz-only GP-UCB (LoS-GP-UCB), a variant of LoSBO applicable to moderately high-dimensional problems, while retaining safety.


title: A Novel Energy-Efficient Cross-Layer Design for Scheduling and Routing in 6TiSCH Networks

abstract: The 6TiSCH protocol stack was proposed to ensure high-performance communications in the Industrial Internet of Things (IIoT). However, the lack of sufficient time slots for nodes outside the 6TiSCH's Destination Oriented Directed Acyclic Graph (DODAG) to transmit their Destination Advertisement Object (DAO) messages and cell reservation requests significantly hinders their integration into the DODAG. This oversight not only prolongs the device's join time but also increases energy consumption during the network formation phase. Moreover, challenges emerge due to the substantial number of control packets employed by both the 6TiSCH Scheduling Function (SF) and routing protocol (RPL), thus draining more energy resources, increasing medium contention, and decreasing spatial reuse. Furthermore, an SF that overlooks previously allocated slots when assigning new ones to the same node may increase jitter, and more complications ensue when it neglects the state of the TSCH queue, thus leading to packet dropping due to queue saturation. Additional complexity arises when the RPL disregards the new parent's schedule saturation during parent switching, which results in inefficient energy and time usage. To address these issues, we introduce in this paper novel mechanisms, strategically situated at the intersection of SF and RPL that are designed to balance the control packet distribution and adaptively manage parent switching. Our proposal, implemented within the 6TiSCH simulator, demonstrates significant improvements across vital performance metrics, such as node's joining time, jitter, latency, energy consumption, and amount of traffic, in comparison to the conventional 6TiSCH benchmark.


title: Optimal and Adaptive Non-Stationary Dueling Bandits Under a Generalized Borda Criterion

abstract: In dueling bandits, the learner receives preference feedback between arms, and the regret of an arm is defined in terms of its suboptimality to a winner arm. The more challenging and practically motivated non-stationary variant of dueling bandits, where preferences change over time, has been the focus of several recent works (Saha and Gupta, 2022; Buening and Saha, 2023; Suk and Agarwal, 2023). The goal is to design algorithms without foreknowledge of the amount of change. The bulk of known results here studies the Condorcet winner setting, where an arm preferred over any other exists at all times. Yet, such a winner may not exist and, to contrast, the Borda version of this problem (which is always well-defined) has received little attention. In this work, we establish the first optimal and adaptive Borda dynamic regret upper bound, which highlights fundamental differences in the learnability of severe non-stationarity between Condorcet vs. Borda regret objectives in dueling bandits. Surprisingly, our techniques for non-stationary Borda dueling bandits also yield improved rates within the Condorcet winner setting, and reveal new preference models where tighter notions of non-stationarity are adaptively learnable. This is accomplished through a novel generalized Borda score framework which unites the Borda and Condorcet problems, thus allowing reduction of Condorcet regret to a Borda-like task. Such a generalization was not previously known and is likely to be of independent interest.


title: Legendrian loops and cluster modular groups

abstract: This work studies Legendrian loop actions on exact Lagrangian fillings of Legendrian links in $(\R^3, \xi_{\st})$. By identifying the induced action of Legendrian loops as generators of cluster modular groups, we establish the existence of faithful group actions on the exact Lagrangian fillings of several families of Legendrian positive braid closures, including all positive torus links. In addition, we leverage a Nielsen-Thurston-like classification of cluster automorphisms to provide new combinatorial and algebraic tools for proving that a Legendrian loop action has infinite order.


title: Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models

abstract: Advancements in vision-language models (VLMs) have propelled the field of computer vision, particularly in the zero-shot learning setting. Despite their promise, the effectiveness of these models often diminishes due to domain shifts in test environments. To address this, we introduce the Test-Time Prototype Shifting (TPS) framework, a pioneering approach designed to adapt VLMs to test datasets using unlabeled test inputs. Our method is based on the notion of modulating per-class prototypes in the shared embedding space. By pre-computing and caching prototypes generated with the pre-trained text encoder, TPS not only facilitates optimization-free prototype reuse for subsequent predictions but also enables seamless integration with current advancements in prompt engineering. At test-time, TPS dynamically learns shift vectors for each prototype based solely on the given test sample, effectively bridging the domain gap and enhancing classification accuracy. A notable aspect of our framework is its significantly reduced memory and computational demands when compared to conventional text-prompt tuning methods. Extensive evaluations across 15 datasets involving natural distribution shifts and cross-dataset generalization demonstrate TPS's superior performance, achieving state-of-the-art results while reducing resource requirements.


title: FutureDepth: Learning to Predict the Future Improves Video Depth Estimation

abstract: In this paper, we propose a novel video depth estimation approach, FutureDepth, which enables the model to implicitly leverage multi-frame and motion cues to improve depth estimation by making it learn to predict the future at training. More specifically, we propose a future prediction network, F-Net, which takes the features of multiple consecutive frames and is trained to predict multi-frame features one time step ahead iteratively. In this way, F-Net learns the underlying motion and correspondence information, and we incorporate its features into the depth decoding process. Additionally, to enrich the learning of multiframe correspondence cues, we further leverage a reconstruction network, R-Net, which is trained via adaptively masked auto-encoding of multiframe feature volumes. At inference time, both F-Net and R-Net are used to produce queries to work with the depth decoder, as well as a final refinement network. Through extensive experiments on several benchmarks, i.e., NYUDv2, KITTI, DDAD, and Sintel, which cover indoor, driving, and open-domain scenarios, we show that FutureDepth significantly improves upon baseline models, outperforms existing video depth estimation methods, and sets new state-of-the-art (SOTA) accuracy. Furthermore, FutureDepth is more efficient than existing SOTA video depth estimation models and has similar latencies when comparing to monocular models


title: Damped energy-norm a posteriori error estimates for fully discrete approximations of the wave equation using C2-reconstructions

abstract: We derive a posteriori error estimates for the the scalar wave equation discretized in space by continuous finite elements and in time by the explicit leapfrog scheme. Our analysis combines the idea of invoking extra time-regularity for the right-hand side, as previously introduced in the space semi-discrete setting, with a novel, piecewise quartic, globally twice-differentiable time-reconstruction of the fully discrete solution. Our main results show that the proposed estimator is reliable and efficient in a damped energy norm. These properties are illustrated in a series of numerical examples.


title: Abiogenesis: a possible quantum interpretation of the telepoietic conjecture

abstract: In the research on the origin of life, topics that can be considered reasonably shared by the generality of researchers are initially identified. It is then shown that the application of these principles to the results obtained with the IdLE-IdLA mathematical model for the simulation of aggregative processes, leads to the conclusion that the primordial formation of self-replicating structures is difficult to reconcile with deterministic aggregative dynamics in the classical sense. Regardless of the extent to which the process itself is governed by chance or by aggregative codes written in the laws of chemistry, no conventional causality is likely. The model itself suggests only one possible way out, consistent with thermodynamics: the existence of information sets rushing into the system in a different way from the perceived time stream. The possibilities offered by quantum mechanics and its most recent interpretations are consequently investigated to try to interpret, at the level of particle physics, the suggestion of IdLE-IdLA model. The attempt leads to a mutual accreditation of macroscopic telepoiesis and a kind of quantum retrocausality. The result is a vision of the natural world in which the coexistence of causal and retrocausal dynamics is presented as a possible interpretative key of the whole complex of vital manifestations.


title: Representations of currents taking values in $PGL(2, Q_q)$

abstract: Let $G=PGL(2,Q_q)$. In this paper we shall investigate the group of measurable currents taking values in $G$. The key observation is that $G$ is acting by automorphisms on a homogeneous tree, which will play the role of the upper half plane in the case of $PSL(2,R)$.


title: GVGEN: Text-to-3D Generation with Volumetric Representation

abstract: In recent years, 3D Gaussian splatting has emerged as a powerful technique for 3D reconstruction and generation, known for its fast and high-quality rendering capabilities. To address these shortcomings, this paper introduces a novel diffusion-based framework, GVGEN, designed to efficiently generate 3D Gaussian representations from text input. We propose two innovative techniques:(1) Structured Volumetric Representation. We first arrange disorganized 3D Gaussian points as a structured form GaussianVolume. This transformation allows the capture of intricate texture details within a volume composed of a fixed number of Gaussians. To better optimize the representation of these details, we propose a unique pruning and densifying method named the Candidate Pool Strategy, enhancing detail fidelity through selective optimization. (2) Coarse-to-fine Generation Pipeline. To simplify the generation of GaussianVolume and empower the model to generate instances with detailed 3D geometry, we propose a coarse-to-fine pipeline. It initially constructs a basic geometric structure, followed by the prediction of complete Gaussian attributes. Our framework, GVGEN, demonstrates superior performance in qualitative and quantitative assessments compared to existing 3D generation methods. Simultaneously, it maintains a fast generation speed ($\sim$7 seconds), effectively striking a balance between quality and efficiency.


title: Dated Data: Tracing Knowledge Cutoffs in Large Language Models

abstract: Released Large Language Models (LLMs) are often paired with a claimed knowledge cutoff date, or the dates at which training data was gathered. Such information is crucial for applications where the LLM must provide up to date information. However, this statement only scratches the surface: do all resources in the training data share the same knowledge cutoff date? Does the model's demonstrated knowledge for these subsets closely align to their cutoff dates? In this work, we define the notion of an effective cutoff. This is distinct from the LLM designer reported cutoff and applies separately to sub-resources and topics. We propose a simple approach to estimate effective cutoffs on the resource-level temporal alignment of an LLM by probing across versions of the data. Using this analysis, we find that effective cutoffs often differ from reported cutoffs. To understand the root cause of this observation, we conduct a direct large-scale analysis on open pre-training datasets. Our analysis reveals two reasons for these inconsistencies: (1) temporal biases of CommonCrawl data due to non-trivial amounts of old data in new dumps and (2) complications in LLM deduplication schemes involving semantic duplicates and lexical near-duplicates. Overall, our results show that knowledge cutoffs are not as simple as they have seemed and that care must be taken both by LLM dataset curators as well as practitioners who seek to use information from these models.


title: WHAC: World-grounded Humans and Cameras

abstract: Estimating human and camera trajectories with accurate scale in the world coordinate system from a monocular video is a highly desirable yet challenging and ill-posed problem. In this study, we aim to recover expressive parametric human models (i.e., SMPL-X) and corresponding camera poses jointly, by leveraging the synergy between three critical players: the world, the human, and the camera. Our approach is founded on two key observations. Firstly, camera-frame SMPL-X estimation methods readily recover absolute human depth. Secondly, human motions inherently provide absolute spatial cues. By integrating these insights, we introduce a novel framework, referred to as WHAC, to facilitate world-grounded expressive human pose and shape estimation (EHPS) alongside camera pose estimation, without relying on traditional optimization techniques. Additionally, we present a new synthetic dataset, WHAC-A-Mole, which includes accurately annotated humans and cameras, and features diverse interactive human motions as well as realistic camera trajectories. Extensive experiments on both standard and newly established benchmarks highlight the superiority and efficacy of our framework. We will make the code and dataset publicly available.


title: FaceXFormer: A Unified Transformer for Facial Analysis

abstract: In this work, we introduce FaceXformer, an end-to-end unified transformer model for a comprehensive range of facial analysis tasks such as face parsing, landmark detection, head pose estimation, attributes recognition, and estimation of age, gender, race, and landmarks visibility. Conventional methods in face analysis have often relied on task-specific designs and preprocessing techniques, which limit their approach to a unified architecture. Unlike these conventional methods, our FaceXformer leverages a transformer-based encoder-decoder architecture where each task is treated as a learnable token, enabling the integration of multiple tasks within a single framework. Moreover, we propose a parameter-efficient decoder, FaceX, which jointly processes face and task tokens, thereby learning generalized and robust face representations across different tasks. To the best of our knowledge, this is the first work to propose a single model capable of handling all these facial analysis tasks using transformers. We conducted a comprehensive analysis of effective backbones for unified face task processing and evaluated different task queries and the synergy between them. We conduct experiments against state-of-the-art specialized models and previous multi-task models in both intra-dataset and cross-dataset evaluations across multiple benchmarks. Additionally, our model effectively handles images "in-the-wild," demonstrating its robustness and generalizability across eight different tasks, all while maintaining the real-time performance of 37 FPS.


title: TexTile: A Differentiable Metric for Texture Tileability

abstract: We introduce TexTile, a novel differentiable metric to quantify the degree upon which a texture image can be concatenated with itself without introducing repeating artifacts (i.e., the tileability). Existing methods for tileable texture synthesis focus on general texture quality, but lack explicit analysis of the intrinsic repeatability properties of a texture. In contrast, our TexTile metric effectively evaluates the tileable properties of a texture, opening the door to more informed synthesis and analysis of tileable textures. Under the hood, TexTile is formulated as a binary classifier carefully built from a large dataset of textures of different styles, semantics, regularities, and human annotations.Key to our method is a set of architectural modifications to baseline pre-train image classifiers to overcome their shortcomings at measuring tileability, along with a custom data augmentation and training regime aimed at increasing robustness and accuracy. We demonstrate that TexTile can be plugged into different state-of-the-art texture synthesis methods, including diffusion-based strategies, and generate tileable textures while keeping or even improving the overall texture quality. Furthermore, we show that TexTile can objectively evaluate any tileable texture synthesis method, whereas the current mix of existing metrics produces uncorrelated scores which heavily hinders progress in the field.


title: FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation

abstract: The remarkable efficacy of text-to-image diffusion models has motivated extensive exploration of their potential application in video domains. Zero-shot methods seek to extend image diffusion models to videos without necessitating model training. Recent methods mainly focus on incorporating inter-frame correspondence into attention mechanisms. However, the soft constraint imposed on determining where to attend to valid features can sometimes be insufficient, resulting in temporal inconsistency. In this paper, we introduce FRESCO, intra-frame correspondence alongside inter-frame correspondence to establish a more robust spatial-temporal constraint. This enhancement ensures a more consistent transformation of semantically similar content across frames. Beyond mere attention guidance, our approach involves an explicit update of features to achieve high spatial-temporal consistency with the input video, significantly improving the visual coherence of the resulting translated videos. Extensive experiments demonstrate the effectiveness of our proposed framework in producing high-quality, coherent videos, marking a notable improvement over existing zero-shot methods.