title: Concurrent Stochastic Lossy Channel Games

abstract: Concurrent stochastic games are an important formalism for the rational verification of probabilistic multi-agent systems, which involves verifying whether a temporal logic property is satisfied in some or all game-theoretic equilibria of such systems. In this work, we study the rational verification of probabilistic multi-agent systems where agents can cooperate by communicating over unbounded lossy channels. To model such systems, we present concurrent stochastic lossy channel games (CSLCG) and employ an equilibrium concept from cooperative game theory known as the core, which is the most fundamental and widely studied cooperative equilibrium concept. Our main contribution is twofold. First, we show that the rational verification problem is undecidable for systems whose agents have almost-sure LTL objectives. Second, we provide a decidable fragment of such a class of objectives that subsumes almost-sure reachability and safety. Our techniques involve reductions to solving infinite-state zero-sum games with conjunctions of qualitative objectives. To the best of our knowledge, our result represents the first decidability result on the rational verification of stochastic multi-agent systems on infinite arenas.


title: Extension of Minimax for Algorithmic Lower Bounds

abstract: This paper considers the use of Yao's Minimax Theorem in robust algorithm design, e.g., for online algorithms, where the algorithm designer aims to minimize the ratio of the algorithm's performance to the optimal performance. When applying Minimax, the objective becomes the expectation of these ratios. The main result of the paper is that it is equally valid for the objective (after applying Minimax) to be the ratio of the expectations.


title: Optimal control of interacting active particles on complex landscapes

abstract: Active many-body systems composed of many interacting degrees of freedom often operate out of equilibrium, giving rise to non-trivial emergent behaviors which can be functional in both evolved and engineered contexts. This naturally suggests the question of control to optimize function. Using navigation as a paradigm for function, we deploy the language of stochastic optimal control theory to formulate the inverse problem of shepherding a system of interacting active particles across a complex landscape. We implement a solution to this high-dimensional problem using an Adjoint-based Path Integral Control (APIC) algorithm that combines the power of recently introduced continuous-time back-propagation methods and automatic differentiation with the classical Feynman-Kac path integral formulation in statistical mechanics. Numerical experiments for controlling individual and interacting particles in complex landscapes show different classes of successful navigation strategies as a function of landscape complexity, as well as the intrinsic noise and drive of the active particles. However, in all cases, we see the emergence of paths that correspond to traversal along the edges of ridges and ravines, which we can understand using a variational analysis. We also show that the work associated with optimal strategies is inversely proportional to the length of the time horizon of optimal control, a result that follows from scaling considerations. All together, our approach serves as a foundational framework to control active non-equilibrium systems optimally to achieve functionality, embodied as a path on a high-dimensional manifold.

categories: cond-mat.soft cond-mat.dis-nn cond-mat.other cond-mat.stat-mech


title: Rumors with Changing Credibility

abstract: Randomized rumor spreading processes diffuse information on an undirected graph and have been widely studied. In this work, we present a generic framework for analyzing a broad class of such processes on regular graphs. Our analysis is protocol-agnostic, as it only requires the expected proportion of newly informed vertices in each round to be bounded, and a natural negative correlation property. This framework allows us to analyze various protocols, including PUSH, PULL, and PUSH-PULL, thereby extending prior research. Unlike previous work, our framework accommodates message failures at any time $t\geq 0$ with a probability of $1-q(t)$, where the credibility $q(t)$ is any function of time. This enables us to model real-world scenarios in which the transmissibility of rumors may fluctuate, as seen in the spread of ``fake news'' and viruses. Additionally, our framework is sufficiently broad to cover dynamic graphs.


title: Efficient In-Context Learning in Vision-Language Models for Egocentric Videos

abstract: Recent advancements in text-only large language models (LLMs) have highlighted the benefit of in-context learning for adapting to new tasks with a few demonstrations. However, extending in-context learning to large vision-language models (VLMs) using a huge amount of naturalistic vision-language data has shown limited success, particularly for egocentric videos, due to high data collection costs. We propose a novel training method $\mathbb{E}$fficient $\mathbb{I}$n-context $\mathbb{L}$earning on $\mathbb{E}$gocentric $\mathbb{V}$ideos ($\mathbb{EILEV}$), which elicits in-context learning in VLMs for egocentric videos without requiring massive, naturalistic egocentric video datasets. $\mathbb{EILEV}$ involves architectural and training data adaptations to allow the model to process contexts interleaved with video clips and narrations, sampling of in-context examples with clusters of similar verbs and nouns, use of data with skewed marginal distributions with a long tail of infrequent verbs and nouns, as well as homonyms and synonyms. Our evaluations show that $\mathbb{EILEV}$-trained models outperform larger VLMs trained on a huge amount of naturalistic data in in-context learning. Furthermore, they can generalize to not only out-of-distribution, but also novel, rare egocentric videos and texts via in-context learning, demonstrating potential for applications requiring cost-effective training, and rapid post-deployment adaptability. Our code and demo are available at \url{https://github.com/yukw777/EILEV}.


title: Adversarial Diffusion Distillation

abstract: We introduce Adversarial Diffusion Distillation (ADD), a novel training approach that efficiently samples large-scale foundational image diffusion models in just 1-4 steps while maintaining high image quality. We use score distillation to leverage large-scale off-the-shelf image diffusion models as a teacher signal in combination with an adversarial loss to ensure high image fidelity even in the low-step regime of one or two sampling steps. Our analyses show that our model clearly outperforms existing few-step methods (GANs, Latent Consistency Models) in a single step and reaches the performance of state-of-the-art diffusion models (SDXL) in only four steps. ADD is the first method to unlock single-step, real-time image synthesis with foundation models. Code and weights available under https://github.com/Stability-AI/generative-models and https://huggingface.co/stabilityai/ .


title: LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models

abstract: In this work, we present a novel method to tackle the token generation challenge in Vision Language Models (VLMs) for video and image understanding, called LLaMA-VID. Current VLMs, while proficient in tasks like image captioning and visual question answering, face computational burdens when processing long videos due to the excessive visual tokens. LLaMA-VID addresses this issue by representing each frame with two distinct tokens, namely context token and content token. The context token encodes the overall image context based on user input, whereas the content token encapsulates visual cues in each frame. This dual-token strategy significantly reduces the overload of long videos while preserving critical information. Generally, LLaMA-VID empowers existing frameworks to support hour-long videos and pushes their upper limit with an extra context token. It is proved to surpass previous methods on most of video- or image-based benchmarks. Code is available https://github.com/dvlab-research/LLaMA-VID}{https://github.com/dvlab-research/LLaMA-VID


title: Gauging Non-Invertible Symmetries: Topological Interfaces and Generalized Orbifold Groupoid in 2d QFT

abstract: Gauging is a powerful operation on symmetries in quantum field theory (QFT), as it connects distinct theories and also reveals hidden structures in a given theory. We initiate a systematic investigation of gauging discrete generalized symmetries in two-dimensional QFT. Such symmetries are described by topological defect lines (TDLs) which obey fusion rules that are non-invertible in general. Despite this seemingly exotic feature, all well-known properties in gauging invertible symmetries carry over to this general setting, which greatly enhances both the scope and the power of gauging. This is established by formulating generalized gauging in terms of topological interfaces between QFTs, which explains the physical picture for the mathematical concept of algebra objects and associated module categories over fusion categories that encapsulate the algebraic properties of generalized symmetries and their gaugings. This perspective also provides simple physical derivations of well-known mathematical theorems in category theory from basic axiomatic properties of QFT in the presence of such interfaces. We discuss a bootstrap-type analysis to classify such topological interfaces and thus the possible generalized gaugings and demonstrate the procedure in concrete examples of fusion categories. Moreover we present a number of examples to illustrate generalized gauging and its properties in concrete conformal field theories (CFTs). In particular, we identify the generalized orbifold groupoid that captures the structure of fusion between topological interfaces (equivalently sequential gaugings) as well as a plethora of new self-dualities in CFTs under generalized gaugings.

categories: hep-th cond-mat.str-el math-ph math.MP math.QA


title: Jets of foliations and $b^k$-algebroids

abstract: In this article, we introduce and study singular foliations of $b^k$-type. These singular foliations formalize the properties of vector fields that are tangent to order $k$ along a submanifold $W \subset M$. Our first result is a classification of these foliations, relating them to geometric structures defined in a formal neighborhood of the submanifold, such as jets of distributions that are involutive up to order $k-1$. When $W$ is a hypersurface, singular foliations of $b^k$-type are Lie algebroids. In this particular case, they are generalizations of the $b^k$-tangent bundles introduced by Scott. Indeed, they are always locally isomorphic to $b^k$-tangent bundles, but globally such an isomorphism is obstructed by a holonomy invariant. Our second main result is a Riemann-Hilbert-style classification of singular foliations of $b^k$-type in terms of holonomy representations. In this paper, we study singular foliations of $b^k$-type from several different perspectives. In particular: (1) We study the problem of extending a $k$-th-order foliation to a $(k+1)$-th order foliation and prove that this is obstructed by a characteristic class. (2) When $W$ is a hypersurface, we give a detailed study of algebroid differential forms and extend Scott's calculation of the cohomology. (3) We study algebroid symplectic forms in terms of the geometric structures induced on $W$. In particular, we find that there is a close relationship between the above obstruction class for extensions and the symplectic variation of the symplectic foliation induced on $W$.


title: Refined BPS structures and topological recursion - the Weber and Whittaker curves

abstract: We study properties of the recently established refined topological recursion for some simple spectral curves associated to quadratic differentials. We prove explicit formulas for the free energy and Voros coefficients of the corresponding quantum curves, and conjecture expressions for all other (smooth) genus zero degree two curves. The results can be written in terms of Bridgeland's notion of refined BPS structure associated to the same initial data, together with a quantum correction to the central charge. The corresponding "invariants" appear to be new, but their interpretation in terms of Donaldson-Thomas theory or QFT is not entirely clear.


title: Tight bounds for antidistinguishability and circulant sets of pure quantum states

abstract: A set of pure quantum states is said to be antidistinguishable if upon sampling one at random, there exists a measurement to perfectly determine some state that was not sampled. We show that antidistinguishability of a set of $n$ pure states is equivalent to a property of its Gram matrix called $(n-1)$-incoherence, thus establishing a connection with quantum resource theories that lets us apply a wide variety of new tools to antidistinguishability. As a particular application of our result, we present an explicit formula (not involving any semidefinite programming) that determines whether or not a set with a circulant Gram matrix is antidistinguishable. We also show that if all inner products are smaller than $\sqrt{(n-2)/(2n-2)}$ then the set must be antidistinguishable, and we show that this bound is tight when $n \leq 4$. We also give a simpler proof that if all the inner products are strictly larger than $(n-2)/(n-1)$, then the set cannot be antidistinguishable, and we show that this bound is tight for all $n$.


title: Zero-shot Referring Expression Comprehension via Structural Similarity Between Images and Captions

abstract: Zero-shot referring expression comprehension aims at localizing bounding boxes in an image corresponding to provided textual prompts, which requires: (i) a fine-grained disentanglement of complex visual scene and textual context, and (ii) a capacity to understand relationships among disentangled entities. Unfortunately, existing large vision-language alignment (VLA) models, e.g., CLIP, struggle with both aspects so cannot be directly used for this task. To mitigate this gap, we leverage large foundation models to disentangle both images and texts into triplets in the format of (subject, predicate, object). After that, grounding is accomplished by calculating the structural similarity matrix between visual and textual triplets with a VLA model, and subsequently propagate it to an instance-level similarity matrix. Furthermore, to equip VLA models with the ability of relationship understanding, we design a triplet-matching objective to fine-tune the VLA models on a collection of curated dataset containing abundant entity relationships. Experiments demonstrate that our visual grounding performance increase of up to 19.5% over the SOTA zero-shot model on RefCOCO/+/g. On the more challenging Who's Waldo dataset, our zero-shot approach achieves comparable accuracy to the fully supervised model. Code is available at https://github.com/Show-han/Zeroshot_REC.


title: MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training

abstract: Contrastive pretraining of image-text foundation models, such as CLIP, demonstrated excellent zero-shot performance and improved robustness on a wide range of downstream tasks. However, these models utilize large transformer-based encoders with significant memory and latency overhead which pose challenges for deployment on mobile devices. In this work, we introduce MobileCLIP -- a new family of efficient image-text models optimized for runtime performance along with a novel and efficient training approach, namely multi-modal reinforced training. The proposed training approach leverages knowledge transfer from an image captioning model and an ensemble of strong CLIP encoders to improve the accuracy of efficient models. Our approach avoids train-time compute overhead by storing the additional knowledge in a reinforced dataset. MobileCLIP sets a new state-of-the-art latency-accuracy tradeoff for zero-shot classification and retrieval tasks on several datasets. Our MobileCLIP-S2 variant is 2.3$\times$ faster while more accurate compared to previous best CLIP model based on ViT-B/16. We further demonstrate the effectiveness of our multi-modal reinforced training by training a CLIP model based on ViT-B/16 image backbone and achieving +2.9% average performance improvement on 38 evaluation benchmarks compared to the previous best. Moreover, we show that the proposed approach achieves 10$\times$-1000$\times$ improved learning efficiency when compared with non-reinforced CLIP training. Code and models are available at https://github.com/apple/ml-mobileclip .


title: Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Models

abstract: We present Surf-D, a novel method for generating high-quality 3D shapes as Surfaces with arbitrary topologies using Diffusion models. Previous methods explored shape generation with different representations and they suffer from limited topologies and poor geometry details. To generate high-quality surfaces of arbitrary topologies, we use the Unsigned Distance Field (UDF) as our surface representation to accommodate arbitrary topologies. Furthermore, we propose a new pipeline that employs a point-based AutoEncoder to learn a compact and continuous latent space for accurately encoding UDF and support high-resolution mesh extraction. We further show that our new pipeline significantly outperforms the prior approaches to learning the distance fields, such as the grid-based AutoEncoder, which is not scalable and incapable of learning accurate UDF. In addition, we adopt a curriculum learning strategy to efficiently embed various surfaces. With the pretrained shape latent space, we employ a latent diffusion model to acquire the distribution of various shapes. Extensive experiments are presented on using Surf-D for unconditional generation, category conditional generation, image conditional generation, and text-to-shape tasks. The experiments demonstrate the superior performance of Surf-D in shape generation across multiple modalities as conditions. Visit our project page at https://yzmblog.github.io/projects/SurfD/.


title: Long-time asymptotic series for the Painleve II equation: Riemann-Hilbert approach

abstract: We elaborate a systematic way to obtain higher order contributions in the nonlinear steepest descent method for Riemann-Hilbert problem associated with homogeneous Painleve II equation. The problem is reformulated as a matrix factorization problem on two circles and can be solved perturbatively reducing it to finite systems of algebraic linear equations. The method is applied to find explicitly long-time asymptotic behaviour for tau function of Painleve II equation.


title: A large-scale particle system with independent jumps and distributed synchronization

abstract: We study a system consisting of $n$ particles, moving forward in jumps on the real line. Each particle can make both independent jumps, whose sizes have some distribution, or ``synchronization'' jumps, which allow it to join a randomly chosen other particle if the latter happens to be ahead of it. System state is the empirical distribution of particle locations. The mean-field asymptotic regime, where $n\to\infty$, is considered. We prove that $v_n$, the steady-state speed of the particle system advance, converges, as $n\to\infty$, to a limit $v_{**}$ which can be easily found from a {\em minimum speed selection principle.} Also, as $n\to\infty$, we prove the convergence of the system dynamics to that of a deterministic mean-field limit (MFL). We show that the average speed of advance of any MFL is lower bounded by $v_{**}$, and the speed of a ``benchmark'' MFL, resulting from all particles initially co-located, is equal to $v_{**}$. In the special case of exponentially distributed independent jump sizes, we prove that a traveling wave MFL with speed $v$ exists if and only if $v\ge v_{**}$, with $v_{**}$ having simple explicit form; we also show the existence of traveling waves for the modified systems, with a left or right boundary moving at a constant speed $v$. Using these traveling wave existence results, we provide bounds on an MFL average speed of advance, depending on the right tail exponent of its initial state. We conjecture that these results for exponential jump sizes generalize to general jump sizes.


title: DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models

abstract: Nature evolves creatures with a high complexity of morphological and behavioral intelligence, meanwhile computational methods lag in approaching that diversity and efficacy. Co-optimization of artificial creatures' morphology and control in silico shows promise for applications in physical soft robotics and virtual character creation; such approaches, however, require developing new learning algorithms that can reason about function atop pure structure. In this paper, we present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. DiffuseBot bridges the gap between virtually generated content and physical utility by (i) augmenting the diffusion process with a physical dynamical simulation which provides a certificate of performance, and (ii) introducing a co-design procedure that jointly optimizes physical design and control by leveraging information about physical sensitivities from differentiable simulation. We showcase a range of simulated and fabricated robots along with their capabilities. Check our website at https://diffusebot.github.io/


title: Using the Energy and Momentum Conceptual Survey to investigate progression in student understanding from introductory to advanced levels

abstract: The Energy and Momentum Conceptual Survey (EMCS) is a multiple-choice survey that contains conceptual problems involving a variety of energy and momentum concepts covered in a typical introductory physics course for science and engineering majors. Prior studies suggest that many concepts on the survey are challenging for introductory physics students and the average student scores after traditional lecture-based instruction are low. The research presented here investigates the progression in student understanding of the EMCS concepts including their evolution from the beginning to the end of their courses in introductory and advanced level undergraduate physics after traditional lecture-based instruction. We find that on all EMCS questions on which less than 50% of the introductory physics students answered a question correctly after traditional instruction, less than two-thirds of the upper-level undergraduate students provided the correct response after traditional lecture-based instruction. We discuss the EMCS questions that remain challenging and the common alternate conceptions among upper-level students. The findings presented here are consistent with prior research showing that traditional instruction in upper-level courses, which typically focuses primarily on quantitative problem solving and often incentivizes use of algorithmic approaches, is not effective for helping many students develop a functional understanding of underlying concepts.


title: No Representation Rules Them All in Category Discovery

abstract: In this paper we tackle the problem of Generalized Category Discovery (GCD). Specifically, given a dataset with labelled and unlabelled images, the task is to cluster all images in the unlabelled subset, whether or not they belong to the labelled categories. Our first contribution is to recognize that most existing GCD benchmarks only contain labels for a single clustering of the data, making it difficult to ascertain whether models are using the available labels to solve the GCD task, or simply solving an unsupervised clustering problem. As such, we present a synthetic dataset, named 'Clevr-4', for category discovery. Clevr-4 contains four equally valid partitions of the data, i.e based on object shape, texture, color or count. To solve the task, models are required to extrapolate the taxonomy specified by the labelled set, rather than simply latching onto a single natural grouping of the data. We use this dataset to demonstrate the limitations of unsupervised clustering in the GCD setting, showing that even very strong unsupervised models fail on Clevr-4. We further use Clevr-4 to examine the weaknesses of existing GCD algorithms, and propose a new method which addresses these shortcomings, leveraging consistent findings from the representation learning literature to do so. Our simple solution, which is based on 'mean teachers' and termed $\mu$GCD, substantially outperforms implemented baselines on Clevr-4. Finally, when we transfer these findings to real data on the challenging Semantic Shift Benchmark (SSB), we find that $\mu$GCD outperforms all prior work, setting a new state-of-the-art. For the project webpage, see https://www.robots.ox.ac.uk/~vgg/data/clevr4/


title: Self-Supervised Motion Magnification by Backpropagating Through Optical Flow

abstract: This paper presents a simple, self-supervised method for magnifying subtle motions in video: given an input video and a magnification factor, we manipulate the video such that its new optical flow is scaled by the desired amount. To train our model, we propose a loss function that estimates the optical flow of the generated video and penalizes how far if deviates from the given magnification factor. Thus, training involves differentiating through a pretrained optical flow network. Since our model is self-supervised, we can further improve its performance through test-time adaptation, by finetuning it on the input video. It can also be easily extended to magnify the motions of only user-selected objects. Our approach avoids the need for synthetic magnification datasets that have been used to train prior learning-based approaches. Instead, it leverages the existing capabilities of off-the-shelf motion estimators. We demonstrate the effectiveness of our method through evaluations of both visual quality and quantitative metrics on a range of real-world and synthetic videos, and we show our method works for both supervised and unsupervised optical flow methods.


title: ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions

abstract: Current approaches for 3D human motion synthesis generate high-quality animations of digital humans performing a wide variety of actions and gestures. However, a notable technological gap exists in addressing the complex dynamics of multi-human interactions within this paradigm. In this work, we present ReMoS, a denoising diffusion-based model that synthesizes full-body reactive motion of a person in a two-person interaction scenario. Assuming the motion of one person is given, we employ a combined spatio-temporal cross-attention mechanism to synthesize the reactive body and hand motion of the second person, thereby completing the interactions between the two. We demonstrate ReMoS across challenging two-person scenarios such as pair-dancing, Ninjutsu, kickboxing, and acrobatics, where one person's movements have complex and diverse influences on the other. We also contribute the ReMoCap dataset for two-person interactions containing full-body and finger motions. We evaluate ReMoS through multiple quantitative metrics, qualitative visualizations, and a user study, and also indicate usability in interactive motion editing applications.


title: Panoptic Video Scene Graph Generation

abstract: Towards building comprehensive real-world visual perception systems, we propose and study a new problem called panoptic scene graph generation (PVSG). PVSG relates to the existing video scene graph generation (VidSGG) problem, which focuses on temporal interactions between humans and objects grounded with bounding boxes in videos. However, the limitation of bounding boxes in detecting non-rigid objects and backgrounds often causes VidSGG to miss key details crucial for comprehensive video understanding. In contrast, PVSG requires nodes in scene graphs to be grounded by more precise, pixel-level segmentation masks, which facilitate holistic scene understanding. To advance research in this new area, we contribute the PVSG dataset, which consists of 400 videos (289 third-person + 111 egocentric videos) with a total of 150K frames labeled with panoptic segmentation masks as well as fine, temporal scene graphs. We also provide a variety of baseline methods and share useful design practices for future work.


title: Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications

abstract: This paper addresses the problem of designing optimal control policies for mobile robots with mission and safety requirements specified using Linear Temporal Logic (LTL). We consider robots with unknown stochastic dynamics operating in environments with unknown geometric structure. The robots are equipped with sensors allowing them to detect obstacles. Our goal is to synthesize a control policy that maximizes the probability of satisfying an LTL-encoded task in the presence of motion and environmental uncertainty. Several deep reinforcement learning (DRL) algorithms have been proposed recently to address similar problems. A common limitation in related works is that of slow learning performance. In order to address this issue, we propose a novel DRL algorithm, which has the capability to learn control policies at a notably faster rate compared to similar methods. Its sample efficiency is due to a mission-driven exploration strategy that prioritizes exploration towards directions that may contribute to mission accomplishment. Identifying these directions relies on an automaton representation of the LTL task as well as a learned neural network that (partially) models the unknown system dynamics. We provide comparative experiments demonstrating the efficiency of our algorithm on robot navigation tasks in unknown environments.


title: Material Palette: Extraction of Materials from a Single Image

abstract: In this paper, we propose a method to extract physically-based rendering (PBR) materials from a single real-world image. We do so in two steps: first, we map regions of the image to material concepts using a diffusion model, which allows the sampling of texture images resembling each material in the scene. Second, we benefit from a separate network to decompose the generated textures into Spatially Varying BRDFs (SVBRDFs), providing us with materials ready to be used in rendering applications. Our approach builds on existing synthetic material libraries with SVBRDF ground truth, but also exploits a diffusion-generated RGB texture dataset to allow generalization to new samples using unsupervised domain adaptation (UDA). Our contributions are thoroughly evaluated on synthetic and real-world datasets. We further demonstrate the applicability of our method for editing 3D scenes with materials estimated from real photographs. The code and models will be made open-source. Project page: https://astra-vision.github.io/MaterialPalette/


title: HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting

abstract: Realistic 3D human generation from text prompts is a desirable yet challenging task. Existing methods optimize 3D representations like mesh or neural fields via score distillation sampling (SDS), which suffers from inadequate fine details or excessive training time. In this paper, we propose an efficient yet effective framework, HumanGaussian, that generates high-quality 3D humans with fine-grained geometry and realistic appearance. Our key insight is that 3D Gaussian Splatting is an efficient renderer with periodic Gaussian shrinkage or growing, where such adaptive density control can be naturally guided by intrinsic human structures. Specifically, 1) we first propose a Structure-Aware SDS that simultaneously optimizes human appearance and geometry. The multi-modal score function from both RGB and depth space is leveraged to distill the Gaussian densification and pruning process. 2) Moreover, we devise an Annealed Negative Prompt Guidance by decomposing SDS into a noisier generative score and a cleaner classifier score, which well addresses the over-saturation issue. The floating artifacts are further eliminated based on Gaussian size in a prune-only phase to enhance generation smoothness. Extensive experiments demonstrate the superior efficiency and competitive quality of our framework, rendering vivid 3D humans under diverse scenarios. Project Page: https://alvinliu0.github.io/projects/HumanGaussian


title: Impact of astrophysical scatter on the Epoch of Reionization [H I]$_{\rm 21cm}$ bispectrum

abstract: It is believed that the first star-forming galaxies are the main drivers of cosmic reionization. It is usually assumed that there is a one-to-one relationship between the star formation rate (SFR) inside a galaxy and the host halo mass in semi-analytical/numerical modeling of large-scale ionization maps during the epoch of reionization. However, more accurate simulations and observations suggest that the SFR and ionizing luminosity in galaxies may vary considerably even if the host halo mass is the same. This astrophysical scatter can introduce an additional non-Gaussianity in the [H I]$_{21\rm cm}$ signal, which might not be captured adequately in the power spectrum. In this work, we have studied the impact of the scatter on the [H I]$_{21\rm cm}$ bispectrum using semi-numerical simulations. We find that the scatter primarily affects small ionized regions, whereas the large ionized bubbles remain largely unaffected. Although, the fractional change in the [H I]$_{21\rm cm}$ bispectra due to the scatter is found to be more than a factor of $10$ at large scales ($\lesssim 1\, {\rm Mpc}^{-1}$), it is found to be statistically insignificant. However, at small scales ($k\sim2.55$ Mpc$^{-1}$), we have found the impact due to the scatter to be high in magnitude ($|\langle \Delta B \rangle/B_{\text{no-scatter}}| \sim 1$) and statistically significant ($|\langle\Delta B\rangle/\sigma_{\Delta B}| \gtrsim 5$) at neutral fraction, $\overline{x}_{\rm HI}\sim 0.8$. We have also found that in the most optimistic scenario, SKA1-Low might be able to detect these signatures of astrophysical scatter, at $\sim 3\sigma$ and $\sim 5\sigma$ detection significance for $\overline{x}_{\rm HI} \sim$ 0.8 and 0.9 respectively, for the equilateral [H I]$_{21\rm cm}$ bispectrum.


title: Multistability and chaos in SEIRS epidemic model with a periodic time-dependent transmission rate

abstract: In this work, we study the dynamics of a SEIRS epidemic model with a periodic time-dependent transmission rate. Emphasizing the influence of the seasonality frequency on the system dynamics, we analyze the largest Lyapunov exponent along parameter planes finding large chaotic regions. Furthermore, in some ranges there are shrimp-like periodic strutures. We highlight the system multistability, identifying the coexistence of periodic orbits for the same parameter values, with the infections maximum distinguishing by up one order of magnitude, depending only on the initial conditions. In this case, the basins of attraction has self-similarity. Parametric configurations, for which both periodic and non-periodic orbits occur, cover $13.20\%$ of the evaluated range. We also identified the coexistence of periodic and chaotic attractors with different maxima of infectious cases, where the periodic scenario peak reaching approximately $50\%$ higher than the chaotic one.


title: A Contradiction to the Law of Energy Conservation by Waves Interference in Symmetric/Asymmetric mode

abstract: It can be agreed that the linear superposition and energy conservation are two independent physics laws in general. The former allows the energy to be re-distributed over space and the latter restricts the energy in the total amount. However, Levine shows the contradiction of the two laws mentioned above by creating a cleaver model that demonstrates the energy "doubling"- and "missing" phenomenon with the constrictive- and destructive interference at every point of whole space, respectively. While, he presented a wrong explanation by using one of the radiating sources to compare with an isolated source by the compensation of the impedance, where the mistake is simply analyzed in this paper. By setting up a spatial symmetric- and asymmetric-mode, we work upon Poynting theorem from the sources to the waves with the considerations of the superposition. The theoretical results reveal the invalidity of the energy conservation. Moreover, the experiments performed in the microwave anechoic chamber confirm the theoretical conclusion.


title: Efficient Deep Speech Understanding at the Edge

abstract: In contemporary speech understanding (SU), a sophisticated pipeline is employed, encompassing the ingestion of streaming voice input. The pipeline executes beam search iteratively, invoking a deep neural network to generate tentative outputs (referred to as hypotheses) in an autoregressive manner. Periodically, the pipeline assesses attention and Connectionist Temporal Classification (CTC) scores. This paper aims to enhance SU performance on edge devices with limited resources. Adopting a hybrid strategy, our approach focuses on accelerating on-device execution and offloading inputs surpassing the device's capacity. While this approach is established, we tackle SU's distinctive challenges through innovative techniques: (1) Late Contextualization: This involves the parallel execution of a model's attentive encoder during input ingestion. (2) Pilot Inference: Addressing temporal load imbalances in the SU pipeline, this technique aims to mitigate them effectively. (3) Autoregression Offramps: Decisions regarding offloading are made solely based on hypotheses, presenting a novel approach. These techniques are designed to seamlessly integrate with existing speech models, pipelines, and frameworks, offering flexibility for independent or combined application. Collectively, they form a hybrid solution for edge SU. Our prototype, named XYZ, has undergone testing on Arm platforms featuring 6 to 8 cores, demonstrating state-of-the-art accuracy. Notably, it achieves a 2x reduction in end-to-end latency and a corresponding 2x decrease in offloading requirements.


title: Cluster trajectory of SOFA score in predicting mortality in sepsis

abstract: Objective: Sepsis is a life-threatening condition. Sequential Organ Failure Assessment (SOFA) score is commonly used to assess organ dysfunction and predict ICU mortality, but it is taken as a static measurement and fails to capture dynamic changes. This study aims to investigate the relationship between dynamic changes in SOFA scores over the first 72 hours of ICU admission and patient outcomes. Design, setting, and participants: 3,253 patients in the Medical Information Mart for Intensive Care IV database who met the sepsis-3 criteria and were admitted from the emergency department with at least 72 hours of ICU admission and full-active resuscitation status were analysed. Group-based trajectory modelling with dynamic time warping and k-means clustering identified distinct trajectory patterns in dynamic SOFA scores. They were subsequently compared using Python. Main outcome measures: Outcomes including hospital and ICU mortality, length of stay in hospital and ICU, and readmission during hospital stay, were collected. Discharge time from ICU to wards and cut-offs at 7-day and 14-day were taken. Results: Four clusters were identified: A (consistently low SOFA scores), B (rapid increase followed by a decline in SOFA scores), C (higher baseline scores with gradual improvement), and D (persistently elevated scores). Cluster D had the longest ICU and hospital stays, highest ICU and hospital mortality. Discharge rates from ICU were similar for Clusters A and B, while Cluster C had initially comparable rates but a slower transition to ward. Conclusion: Monitoring dynamic changes in SOFA score is valuable for assessing sepsis severity and treatment responsiveness.


title: Hypergraphs Demonstrate Anastomoses During Divergent Integration

abstract: Complex networks can be used to analyze structures and systems in the embryo. Not only can we characterize growth and the emergence of form, but also differentiation. The process of differentiation from precursor cell populations to distinct functional tissues is of particular interest. These phenomena can be captured using a hypergraph consisting of nodes represented by cell type categories and arranged as a directed cyclic graph (lineage hypergraph) and a complex network (spatial hypergraph). The lineage hypergraph models the developmental process as an n-ary tree, which can model two or more descendent categories per division event. A lineage tree based on the mosaic development of the nematode C. elegans (2-ary tree), is used to capture this process. Each round of divisions produces a new set of categories that allow for exchange of cells between types. An example from single-cell morphogenesis based on the cyanobacterial species Nostoc punctiforme (multiple discontinuous 2-ary tree) is also used to demonstrate the flexibility of this method. This model allows for new structures to emerge (such as a connectome) while also demonstrating how precursor categories are maintained for purposes such as dedifferentiation or other forms of cell fate plasticity. To understand this process of divergent integration, we analyze the directed hypergraph and categorical models, in addition to considering the role of network fistulas (spaces that conjoin two functional modules) and spatial restriction.


title: Deep convolutional encoder-decoder hierarchical neural networks for conjugate heat transfer surrogate modeling

abstract: Conjugate heat transfer (CHT) models are vital for the design of many engineering systems. However, high-fidelity CHT models are computationally intensive, which limits their use in applications such as design optimization, where hundreds to thousands of model evaluations are required. In this work, we develop a modular deep convolutional encoder-decoder hierarchical (DeepEDH) neural network, a novel deep-learning-based surrogate modeling methodology for computationally intensive CHT models. Leveraging convective temperature dependencies, we propose a two-stage temperature prediction architecture that couples velocity and temperature models. The proposed DeepEDH methodology is demonstrated by modeling the pressure, velocity, and temperature fields for a liquid-cooled cold-plate-based battery thermal management system with variable channel geometry. A computational model of the cold plate is developed and solved using the finite element method (FEM), generating a dataset of 1,500 simulations. The FEM results are transformed and scaled from unstructured to structured, image-like meshes to create training and test datasets. The DeepEDH methodology's performance is examined in relation to data scaling, training dataset size, and network depth. Our performance analysis covers the impact of the novel architecture, separate field models, output geometry masks, multi-stage temperature models, and optimizations of the hyperparameters and architecture. Furthermore, we quantify the influence of the CHT thermal boundary condition on surrogate model performance, highlighting improved temperature model performance with higher heat fluxes. Compared to other deep learning neural network surrogate models, such as U-Net and DenseED, the proposed DeepEDH methodology for CHT models exhibits up to a 65% enhancement in the coefficient of determination ($R^{2}$).


title: Broad class of nonlinear Langevin equations with drift and diffusion cofficients separable in time and space: Generalized n-moment, ergodicity, Einstein relation and fluctuations of the system

abstract: A wide class of nonlinear Langevin equations with drift and diffusion coefficients separable in time and space driven by the Gaussian white noise is analyzed in terms of a generalized n-moment. We show the system may present ergodic property, a key property in statistical mechanics, for space-time-dependent drift and diffusion coefficients. A generalized Einstein relation is also obtained. Besides, we show that the first two generalized moments and variance are useful to describe the drift and fluctuations of the system.


title: Quantum Entanglement without nonlocal causation in (3,2)-dimensional spacetime

abstract: This work aims at exploring whether the nonlocal correlations due to quantum entanglement could exist without nonlocal causation. This is done with the aid of a toy model to investigate whether the ability of two quantum entangled particles to "correlate" their behaviors even at very large distances and in the absence of any physical connection can be seen as due to an exchange of information through an extra-temporal dimension. Since superluminal information exchange is forbidden in our (3,1) spacetime, an extra-temporal dimension is needed to recover the physical picture of finite velocity information exchange between entangled entities. Assuming that the geometry of space-time of dimension (3,2) is described by a metric containing a warping factor, the confinement of the massive particles in the extra time dimension follows. Therefore, why we do not experience an infinitely large extra time dimension can be explained. The toy model proposed here is defined by borrowing Bohm-Bub's proposal to describe the wavefunction collapse using nonlinear (non-unitary) dynamical equations and then elaborating this approach for an entangled system. The model so obtained is just speculative without any claim of being robust against any criticism, nevertheless, it satisfies the purpose of giving the possibility to the hypotheses formulated above to be verified experimentally; in fact, it proposes an experiment potentially interesting which would otherwise be immediately dismissed as manifestly trivial. The proposed experiment would consist of checking the possible violation of Bell's inequality between two identical but independent systems under appropriate conditions. Beyond its theoretical interest, entanglement is a key topic in quantum computing and quantum technologies, so any attempt to gain a deeper understanding of it could be useful.


title: Globular Clusters Contribute to the Nuclear Star Cluster and Galaxy Center Gamma-Ray Excess, Moderated by Galaxy Assembly History

abstract: Two unresolved questions at galaxy centers, namely the formation of the nuclear star cluster (NSC) and the origin of the gamma-ray excess in the Milky Way (MW) and Andromeda (M31), are both related to the formation and evolution of globular clusters (GCs). They migrate towards the galaxy center due to dynamical friction, and get tidally disrupted to release the stellar mass content including millisecond pulsars (MSPs), which contribute to the NSC and gamma-ray excess. In this study, we propose a semi-analytical model of GC formation and evolution that utilizes the Illustris cosmological simulation to accurately capture the formation epochs of GCs and simulate their subsequent evolution. Our analysis confirms that our GC properties at z=0 are consistent with observations, and our model naturally explains the formation of a massive NSC in a galaxy similar to the MW and M31. We also find a remarkable similarity in our model prediction with the gamma-ray excess signal in the MW. However, our predictions fall short by approximately an order of magnitude in M31, indicating distinct origins for the two gamma-ray excesses. Meanwhile, we utilize the catalog of Illustris halos to investigate the influence of galaxy assembly history. We find that the earlier a galaxy is assembled, the heavier and spatially more concentrated its GC system behaves at z=0. This results in a larger NSC mass and brighter gamma-ray emission from deposited MSPs


title: IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers

abstract: Generative training has been demonstrated to be powerful for building visual-language models. However, on zero-shot discriminative benchmarks, there is still a performance gap between models trained with generative and discriminative objectives. In this paper, we aim to narrow this gap by improving the efficacy of generative training on classification tasks, without any finetuning processes or additional modules. Specifically, we focus on narrowing the gap between the generative captioner and the CLIP classifier. We begin by analysing the predictions made by the captioner and classifier and observe that the caption generation inherits the distribution bias from the language model trained with pure text modality, making it less grounded on the visual signal. To tackle this problem, we redesign the scoring objective for the captioner to alleviate the distributional bias and focus on measuring the gain of information brought by the visual inputs. We further design a generative training objective to match the evaluation objective. We name our model trained and evaluated from the novel procedures as Information Gain (IG) captioner. We pretrain the models on the public Laion-5B dataset and perform a series of discriminative evaluations. For the zero-shot classification on ImageNet, IG captioner achieves $> 18\%$ improvements over the standard captioner, achieving comparable performances with the CLIP classifier. IG captioner also demonstrated strong performance on zero-shot image-text retrieval tasks on MSCOCO and Flickr30K. We hope this paper inspires further research towards unifying generative and discriminative training procedures for visual-language models.


title: Practical Layout-Aware Analog/Mixed-Signal Design Automation with Bayesian Neural Networks

abstract: The high simulation cost has been a bottleneck of practical analog/mixed-signal design automation. Many learning-based algorithms require thousands of simulated data points, which is impractical for expensive to simulate circuits. We propose a learning-based algorithm that can be trained using a small amount of data and, therefore, scalable to tasks with expensive simulations. Our efficient algorithm solves the post-layout performance optimization problem where simulations are known to be expensive. Our comprehensive study also solves the schematic-level sizing problem. For efficient optimization, we utilize Bayesian Neural Networks as a regression model to approximate circuit performance. For layout-aware optimization, we handle the problem as a multi-fidelity optimization problem and improve efficiency by exploiting the correlations from cheaper evaluations. We present three test cases to demonstrate the efficiency of our algorithms. Our tests prove that the proposed approach is more efficient than conventional baselines and state-of-the-art algorithms.


title: Self-Supervised Learning of Whole and Component-Based Semantic Representations for Person Re-Identification

abstract: Person Re-Identification (ReID) is a challenging problem, focusing on identifying individuals across diverse settings. However, previous ReID methods primarily concentrated on a single domain or modality, such as Clothes-Changing ReID (CC-ReID) and video ReID. Real-world ReID is not constrained by factors like clothes or input types. Recent approaches emphasize on learning semantics through pre-training to enhance ReID performance but are hindered by coarse granularity, on-clothes focus and pre-defined areas. To address these limitations, we propose a Local Semantic Extraction (LSE) module inspired by Interactive Segmentation Models. The LSE module captures fine-grained, biometric, and flexible local semantics, enhancing ReID accuracy. Additionally, we introduce Semantic ReID (SemReID), a pre-training method that leverages LSE to learn effective semantics for seamless transfer across various ReID domains and modalities. Extensive evaluations across nine ReID datasets demonstrates SemReID's robust performance across multiple domains, including clothes-changing ReID, video ReID, unconstrained ReID, and short-term ReID. Our findings highlight the importance of effective semantics in ReID, as SemReID can achieve great performances without domain-specific designs.


title: Ground-breaking Exoplanet Science with the ANDES spectrograph at the ELT

abstract: In the past decade the study of exoplanet atmospheres at high-spectral resolution, via transmission/emission spectroscopy and cross-correlation techniques for atomic/molecular mapping, has become a powerful and consolidated methodology. The current limitation is the signal-to-noise ratio during a planetary transit. This limitation will be overcome by ANDES, an optical and near-infrared high-resolution spectrograph for the ELT. ANDES will be a powerful transformational instrument for exoplanet science. It will enable the study of giant planet atmospheres, allowing not only an exquisite determination of atmospheric composition, but also the study of isotopic compositions, dynamics and weather patterns, mapping the planetary atmospheres and probing atmospheric formation and evolution models. The unprecedented angular resolution of ANDES, will also allow us to explore the initial conditions in which planets form in proto-planetary disks. The main science case of ANDES, however, is the study of small, rocky exoplanet atmospheres, including the potential for biomarker detections, and the ability to reach this science case is driving its instrumental design. Here we discuss our simulations and the observing strategies to achieve this specific science goal. Since ANDES will be operational at the same time as NASA's JWST and ESA's ARIEL missions, it will provide enormous synergies in the characterization of planetary atmospheres at high and low spectral resolution. Moreover, ANDES will be able to probe for the first time the atmospheres of several giant and small planets in reflected light. In particular, we show how ANDES will be able to unlock the reflected light atmospheric signal of a golden sample of nearby non-transiting habitable zone earth-sized planets within a few tenths of nights, a scientific objective that no other currently approved astronomical facility will be able to reach.


title: Compositional Chain-of-Thought Prompting for Large Multimodal Models

abstract: The combination of strong visual backbones and Large Language Model (LLM) reasoning has led to Large Multimodal Models (LMMs) becoming the current standard for a wide range of vision and language (VL) tasks. However, recent research has shown that even the most advanced LMMs still struggle to capture aspects of compositional visual reasoning, such as attributes and relationships between objects. One solution is to utilize scene graphs (SGs)--a formalization of objects and their relations and attributes that has been extensively used as a bridge between the visual and textual domains. Yet, scene graph data requires scene graph annotations, which are expensive to collect and thus not easily scalable. Moreover, finetuning an LMM based on SG data can lead to catastrophic forgetting of the pretraining objective. To overcome this, inspired by chain-of-thought methods, we propose Compositional Chain-of-Thought (CCoT), a novel zero-shot Chain-of-Thought prompting method that utilizes SG representations in order to extract compositional knowledge from an LMM. Specifically, we first generate an SG using the LMM, and then use that SG in the prompt to produce a response. Through extensive experiments, we find that the proposed CCoT approach not only improves LMM performance on several vision and language VL compositional benchmarks but also improves the performance of several popular LMMs on general multimodal benchmarks, without the need for fine-tuning or annotated ground-truth SGs. Code: https://github.com/chancharikmitra/CCoT


title: Game-Theoretic Analysis of Adversarial Decision Making in a Complex Sociophysical System

abstract: We apply Game Theory to a mathematical representation of two competing teams of agents connected within a complex network, where the ability of each side to manoeuvre their resource and degrade that of the other depends on their ability to internally synchronise decision-making while out-pacing the other. Such a representation of an adversarial socio-physical system has application in a range of business, sporting, and military contexts. Specifically, we unite here two physics-based models, that of Kuramoto to represent decision-making cycles, and an adaptation of a multi-species Lotka-Volterra system for the resource competition. For complex networks we employ variations of the Barab\'asi-Alberts scale-free graph, varying how resources are initially distributed between graph hub and periphery. We adapt as equilibrium solution Nash Dominant Game Pruning as a means of efficiently exploring the dynamical decision tree. Across various scenarios we find Nash solutions where the side initially concentrating resources in the periphery can sustain competition to achieve victory except when asymmetries exist between the two. When structural advantage is limited we find that agility in how the victor stays ahead of decision-state of the other becomes critical.

categories: physics.soc-ph math-ph math.MP nlin.AO nlin.CD


title: Data Imbalance, Uncertainty Quantification, and Generalization via Transfer Learning in Data-driven Parameterizations: Lessons from the Emulation of Gravity Wave Momentum Transport in WACCM

abstract: Neural networks (NNs) are increasingly used for data-driven subgrid-scale parameterization in weather and climate models. While NNs are powerful tools for learning complex nonlinear relationships from data, there are several challenges in using them for parameterizations. Three of these challenges are 1) data imbalance related to learning rare (often large-amplitude) samples; 2) uncertainty quantification (UQ) of the predictions to provide an accuracy indicator; and 3) generalization to other climates, e.g., those with higher radiative forcing. Here, we examine the performance of methods for addressing these challenges using NN-based emulators of the Whole Atmosphere Community Climate Model (WACCM) physics-based gravity wave (GW) parameterizations as the test case. WACCM has complex, state-of-the-art parameterizations for orography-, convection- and frontal-driven GWs. Convection- and orography-driven GWs have significant data imbalance due to the absence of convection or orography in many grid points. We address data imbalance using resampling and/or weighted loss functions, enabling the successful emulation of parameterizations for all three sources. We demonstrate that three UQ methods (Bayesian NNs, variational auto-encoders, and dropouts) provide ensemble spreads that correspond to accuracy during testing, offering criteria on when a NN gives inaccurate predictions. Finally, we show that the accuracy of these NNs decreases for a warmer climate (4XCO2). However, the generalization accuracy is significantly improved by applying transfer learning, e.g., re-training only one layer using ~1% new data from the warmer climate. The findings of this study offer insights for developing reliable and generalizable data-driven parameterizations for various processes, including (but not limited) to GWs.


title: A statistical mechanical analysis on the possibility of achieving fair cylindrical dice

abstract: Many have dedicated their time trying to determine the ideal conditions for a cylinder to have equal probabilities of falling with one of its faces facing upwards or on its side. However, to this day, there is no concrete analysis of what these conditions should be. In order to determine such circumstances, a theoretical analysis was conducted, considering approaches from Rigid Body Dynamics and Statistical Mechanics. An experimental system was also built to improve control over the launches, and a comparative analysis was performed between the results obtained experimentally and the theory. It was concluded that the environment and other launching conditions have a significant influence; nevertheless, it is possible, under controlled conditions, to determine, within certain limits, the expected probabilities.


title: Combating the "Sameness" in AI Art: Reflections on the Interactive AI Installation Fencing Hallucination

abstract: The article summarizes three types of "sameness" issues in Artificial Intelligence(AI) art, each occurring at different stages of development in AI image creation tools. Through the Fencing Hallucination project, the article reflects on the design of AI art production in alleviating the sense of uniformity, maintaining the uniqueness of images from an AI image synthesizer, and enhancing the connection between the artworks and the audience. This paper endeavors to stimulate the creation of distinctive AI art by recounting the efforts and insights derived from the Fencing Hallucination project, all dedicated to addressing the issue of "sameness".


title: I-MedSAM: Implicit Medical Image Segmentation with Segment Anything

abstract: With the development of Deep Neural Networks (DNNs), many efforts have been made to handle medical image segmentation. Traditional methods such as nnUNet train specific segmentation models on the individual datasets. Plenty of recent methods have been proposed to adapt the foundational Segment Anything Model (SAM) to medical image segmentation. However, they still focus on discrete representations to generate pixel-wise predictions, which are spatially inflexible and scale poorly to higher resolution. In contrast, implicit methods learn continuous representations for segmentation, which is crucial for medical image segmentation. In this paper, we propose I-MedSAM, which leverages the benefits of both continuous representations and SAM, to obtain better cross-domain ability and accurate boundary delineation. Since medical image segmentation needs to predict detailed segmentation boundaries, we designed a novel adapter to enhance the SAM features with high-frequency information during Parameter Efficient Fine Tuning (PEFT). To convert the SAM features and coordinates into continuous segmentation output, we utilize Implicit Neural Representation (INR) to learn an implicit segmentation decoder. We also propose an uncertainty-guided sampling strategy for efficient learning of INR. Extensive evaluations on 2D medical image segmentation tasks have shown that our proposed method with only 1.6M trainable parameters outperforms existing methods including discrete and continuous methods. The code will be released.


title: DreamPropeller: Supercharge Text-to-3D Generation with Parallel Sampling

abstract: Recent methods such as Score Distillation Sampling (SDS) and Variational Score Distillation (VSD) using 2D diffusion models for text-to-3D generation have demonstrated impressive generation quality. However, the long generation time of such algorithms significantly degrades the user experience. To tackle this problem, we propose DreamPropeller, a drop-in acceleration algorithm that can be wrapped around any existing text-to-3D generation pipeline based on score distillation. Our framework generalizes Picard iterations, a classical algorithm for parallel sampling an ODE path, and can account for non-ODE paths such as momentum-based gradient updates and changes in dimensions during the optimization process as in many cases of 3D generation. We show that our algorithm trades parallel compute for wallclock time and empirically achieves up to 4.7x speedup with a negligible drop in generation quality for all tested frameworks.


title: CLiC: Concept Learning in Context

abstract: This paper addresses the challenge of learning a local visual pattern of an object from one image, and generating images depicting objects with that pattern. Learning a localized concept and placing it on an object in a target image is a nontrivial task, as the objects may have different orientations and shapes. Our approach builds upon recent advancements in visual concept learning. It involves acquiring a visual concept (e.g., an ornament) from a source image and subsequently applying it to an object (e.g., a chair) in a target image. Our key idea is to perform in-context concept learning, acquiring the local visual concept within the broader context of the objects they belong to. To localize the concept learning, we employ soft masks that contain both the concept within the mask and the surrounding image area. We demonstrate our approach through object generation within an image, showcasing plausible embedding of in-context learned concepts. We also introduce methods for directing acquired concepts to specific locations within target images, employing cross-attention mechanisms, and establishing correspondences between source and target objects. The effectiveness of our method is demonstrated through quantitative and qualitative experiments, along with comparisons against baseline techniques.


title: DepthSSC: Depth-Spatial Alignment and Dynamic Voxel Resolution for Monocular 3D Semantic Scene Completion

abstract: The task of 3D semantic scene completion with monocular cameras is gaining increasing attention in the field of autonomous driving. Its objective is to predict the occupancy status of each voxel in the 3D scene from partial image inputs. Despite the existence of numerous methods, many of them overlook the issue of accurate alignment between spatial and depth information. To address this, we propose DepthSSC, an advanced method for semantic scene completion solely based on monocular cameras. DepthSSC combines the ST-GF (Spatial Transformation Graph Fusion) module with geometric-aware voxelization, enabling dynamic adjustment of voxel resolution and considering the geometric complexity of 3D space to ensure precise alignment between spatial and depth information. This approach successfully mitigates spatial misalignment and distortion issues observed in prior methods. Through evaluation on the SemanticKITTI dataset, DepthSSC not only demonstrates its effectiveness in capturing intricate 3D structural details but also achieves state-of-the-art performance. We believe DepthSSC provides a fresh perspective on monocular camera-based 3D semantic scene completion research and anticipate it will inspire further related studies.


title: Beyond Visual Cues: Synchronously Exploring Target-Centric Semantics for Vision-Language Tracking

abstract: Single object tracking aims to locate one specific target in video sequences, given its initial state. Classical trackers rely solely on visual cues, restricting their ability to handle challenges such as appearance variations, ambiguity, and distractions. Hence, Vision-Language (VL) tracking has emerged as a promising approach, incorporating language descriptions to directly provide high-level semantics and enhance tracking performance. However, current VL trackers have not fully exploited the power of VL learning, as they suffer from limitations such as heavily relying on off-the-shelf backbones for feature extraction, ineffective VL fusion designs, and the absence of VL-related loss functions. Consequently, we present a novel tracker that progressively explores target-centric semantics for VL tracking. Specifically, we propose the first Synchronous Learning Backbone (SLB) for VL tracking, which consists of two novel modules: the Target Enhance Module (TEM) and the Semantic Aware Module (SAM). These modules enable the tracker to perceive target-related semantics and comprehend the context of both visual and textual modalities at the same pace, facilitating VL feature extraction and fusion at different semantic levels. Moreover, we devise the dense matching loss to further strengthen multi-modal representation learning. Extensive experiments on VL tracking datasets demonstrate the superiority and effectiveness of our methods.


title: PEA-Diffusion: Parameter-Efficient Adapter with Knowledge Distillation in non-English Text-to-Image Generation

abstract: Text-to-image diffusion models are well-known for their ability to generate realistic images based on textual prompts. However, the existing works have predominantly focused on English, lacking support for non-English text-to-image models. The most commonly used translation methods cannot solve the generation problem related to language culture, while training from scratch on a specific language dataset is prohibitively expensive. In this paper, we are inspired to propose a simple plug-and-play language transfer method based on knowledge distillation. All we need to do is train a lightweight MLP-like parameter-efficient adapter (PEA) with only 6M parameters under teacher knowledge distillation along with a small parallel data corpus. We are surprised to find that freezing the parameters of UNet can still achieve remarkable performance on the language-specific prompt evaluation set, demonstrating that PEA can stimulate the potential generation ability of the original UNet. Additionally, it closely approaches the performance of the English text-to-image model on a general prompt evaluation set. Furthermore, our adapter can be used as a plugin to achieve significant results in downstream tasks in cross-lingual text-to-image generation. Code will be available at: https://github.com/OPPO-Mente-Lab/PEA-Diffusion


title: Rethinking Mixup for Improving the Adversarial Transferability

abstract: Mixup augmentation has been widely integrated to generate adversarial examples with superior adversarial transferability when immigrating from a surrogate model to other models. However, the underlying mechanism influencing the mixup's effect on transferability remains unexplored. In this work, we posit that the adversarial examples located at the convergence of decision boundaries across various categories exhibit better transferability and identify that Admix tends to steer the adversarial examples towards such regions. However, we find the constraint on the added image in Admix decays its capability, resulting in limited transferability. To address such an issue, we propose a new input transformation-based attack called Mixing the Image but Separating the gradienT (MIST). Specifically, MIST randomly mixes the input image with a randomly shifted image and separates the gradient of each loss item for each mixed image. To counteract the imprecise gradient, MIST calculates the gradient on several mixed images for each input sample. Extensive experimental results on the ImageNet dataset demonstrate that MIST outperforms existing SOTA input transformation-based attacks with a clear margin on both Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) w/wo defense mechanisms, supporting MIST's high effectiveness and generality.


title: Unsupervised Multimodal Deepfake Detection Using Intra- and Cross-Modal Inconsistencies

abstract: Deepfake videos present an increasing threat to society with potentially negative impact on criminal justice, democracy, and personal safety and privacy. Meanwhile, detecting deepfakes, at scale, remains a very challenging task that often requires labeled training data from existing deepfake generation methods. Further, even the most accurate supervised deepfake detection methods do not generalize to deepfakes generated using new generation methods. In this paper, we propose a novel unsupervised method for detecting deepfake videos by directly identifying intra-modal and cross-modal inconsistency between video segments. The fundamental hypothesis behind the proposed detection method is that motion or identity inconsistencies are inevitable in deepfake videos. We will mathematically and empirically support this hypothesis, and then proceed to constructing our method grounded in our theoretical analysis. Our proposed method outperforms prior state-of-the-art unsupervised deepfake detection methods on the challenging FakeAVCeleb dataset, and also has several additional advantages: it is scalable because it does not require pristine (real) samples for each identity during inference and therefore can apply to arbitrarily many identities, generalizable because it is trained only on real videos and therefore does not rely on a particular deepfake method, reliable because it does not rely on any likelihood estimation in high dimensions, and explainable because it can pinpoint the exact location of modality inconsistencies which are then verifiable by a human expert.


title: Multi-Scale 3D Gaussian Splatting for Anti-Aliased Rendering

abstract: 3D Gaussians have recently emerged as a highly efficient representation for 3D reconstruction and rendering. Despite its high rendering quality and speed at high resolutions, they both deteriorate drastically when rendered at lower resolutions or from far away camera position. During low resolution or far away rendering, the pixel size of the image can fall below the Nyquist frequency compared to the screen size of each splatted 3D Gaussian and leads to aliasing effect. The rendering is also drastically slowed down by the sequential alpha blending of more splatted Gaussians per pixel. To address these issues, we propose a multi-scale 3D Gaussian splatting algorithm, which maintains Gaussians at different scales to represent the same scene. Higher-resolution images are rendered with more small Gaussians, and lower-resolution images are rendered with fewer larger Gaussians. With similar training time, our algorithm can achieve 13\%-66\% PSNR and 160\%-2400\% rendering speed improvement at 4$\times$-128$\times$ scale rendering on Mip-NeRF360 dataset compared to the single scale 3D Gaussian splitting. Our code and more results are available on our project website https://jokeryan.github.io/projects/ms-gs/


title: Dynamic Change of Amplitude for OCT Functional Imaging

abstract: Optical coherence tomography (OCT) is capable of non-destructively obtaining cross-sectional information of samples with micrometer spatial resolution, which plays an important role in ophthalmology and endovascular medicine. Measuring OCT amplitude can obtain three-dimensional structural information of the sample, such as the layered structure of the retina, but is of limited use for functional information such as tissue specificity, blood flow, and mechanical properties. OCT functional imaging techniques based on other optical field properties including phase, polarization state, and wavelength have emerged, such as Doppler OCT, optical coherence elastography, polarization-sensitive OCT, and visible-light OCT. Among them, functional imaging techniques based on dynamic changes of amplitude have significant robustness and complexity advantages, and achieved significant clinical success in label-free blood flow imaging. In addition, dynamic light scattering OCT for 3D blood flow velocity measurement, dynamic OCT with the ability to display label-free tissue/cell specificity, and OCT thermometry for monitoring the temperature field of thermophysical treatments are the frontiers in OCT functional imaging. In this paper, the principles and applications of the above technologies are summarized, the remaining technical challenges are analyzed, and the future development is envisioned.


title: Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models

abstract: Fine-tuning pre-trained vision-language models (VLMs), e.g., CLIP, for the open-world generalization has gained increasing popularity due to its practical value. However, performance advancements are limited when relying solely on intricate algorithmic designs for a single model, even one exhibiting strong performance, e.g., CLIP-ViT-B/16. This paper, for the first time, explores the collaborative potential of leveraging much weaker VLMs to enhance the generalization of a robust single model. The affirmative findings motivate us to address the generalization problem from a novel perspective, i.e., ensemble of pre-trained VLMs. We introduce three customized ensemble strategies, each tailored to one specific scenario. Firstly, we introduce the zero-shot ensemble, automatically adjusting the logits of different models based on their confidence when only pre-trained VLMs are available. Furthermore, for scenarios with extra few-shot samples, we propose the training-free and tuning ensemble, offering flexibility based on the availability of computing resources. The proposed ensemble strategies are evaluated on zero-shot, base-to-new, and cross-dataset generalization, achieving new state-of-the-art performance. Notably, this work represents an initial stride toward enhancing the generalization performance of VLMs via ensemble. The code is available at https://github.com/zhiheLu/Ensemble_VLM.git.


title: SEED-Bench-2: Benchmarking Multimodal Large Language Models

abstract: Multimodal large language models (MLLMs), building upon the foundation of powerful large language models (LLMs), have recently demonstrated exceptional capabilities in generating not only texts but also images given interleaved multimodal inputs (acting like a combination of GPT-4V and DALL-E 3). However, existing MLLM benchmarks remain limited to assessing only models' comprehension ability of single image-text inputs, failing to keep up with the strides made in MLLMs. A comprehensive benchmark is imperative for investigating the progress and uncovering the limitations of current MLLMs. In this work, we categorize the capabilities of MLLMs into hierarchical levels from $L_0$ to $L_4$ based on the modalities they can accept and generate, and propose SEED-Bench-2, a comprehensive benchmark that evaluates the \textbf{hierarchical} capabilities of MLLMs. Specifically, SEED-Bench-2 comprises 24K multiple-choice questions with accurate human annotations, which spans 27 dimensions, including the evaluation of both text and image generation. Multiple-choice questions with groundtruth options derived from human annotation enables an objective and efficient assessment of model performance, eliminating the need for human or GPT intervention during evaluation. We further evaluate the performance of 23 prominent open-source MLLMs and summarize valuable observations. By revealing the limitations of existing MLLMs through extensive evaluations, we aim for SEED-Bench-2 to provide insights that will motivate future research towards the goal of General Artificial Intelligence. Dataset and evaluation code are available at \href{https://github.com/AILab-CVC/SEED-Bench}


title: PAWS-VMK: A Unified Approach To Semi-Supervised Learning And Out-of-Distribution Detection

abstract: This paper describes PAWS-VMK, a prototypical deep learning approach that obtains state-of-the-art results for image classification tasks in both a semi-supervised learning (SSL) and out-of-distribution (OOD) detection context. We consider developments in the fields of SSL, OOD detection, and computer vision foundation models to introduce a number of innovations that connect the key ideas within these works to create PAWS-VMK. These innovations include (1) parametric von Mises-Fisher Stochastic Neighbour Embedding (vMF-SNE) to initialise a projection head for SSL using the high-quality embeddings of the foundation model; (2) the PAWS-MixMatch loss, that creates more compact embeddings and obtains higher accuracy in comparison to the consistency loss used in PAWS and (3) simple $k$-Means prototype selection (SKMPS), a simple technique that obtains competitive performance with more complex unsupervised label selection approaches. PAWS-VMK sets new benchmarks in semi-supervised learning for CIFAR-10 (99.2%) and CIFAR-100 (89.8%) with four labelled instances per class, and Food-101 (90.1%) with two labelled instances per class. We also observe that PAWS-VMK can efficiently detect OOD samples in a manner that is competitive with specialised methods specifically designed for this purpose, achieving 93.1/98.0 and 95.2/96.3 on the CIFAR-10 and CIFAR-100 OpenOOD benchmarks.


title: In Search of a Data Transformation That Accelerates Neural Field Training

abstract: Neural field is an emerging paradigm in data representation that trains a neural network to approximate the given signal. A key obstacle that prevents its widespread adoption is the encoding speed-generating neural fields requires an overfitting of a neural network, which can take a significant number of SGD steps to reach the desired fidelity level. In this paper, we delve into the impacts of data transformations on the speed of neural field training, specifically focusing on how permuting pixel locations affect the convergence speed of SGD. Counterintuitively, we find that randomly permuting the pixel locations can considerably accelerate the training. To explain this phenomenon, we examine the neural field training through the lens of PSNR curves, loss landscapes, and error patterns. Our analyses suggest that the random pixel permutations remove the easy-to-fit patterns, which facilitate easy optimization in the early stage but hinder capturing fine details of the signal.


title: Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models

abstract: From image-text pairs, large-scale vision-language models (VLMs) learn to implicitly associate image regions with words, which prove effective for tasks like visual question answering. However, leveraging the learned association for open-vocabulary semantic segmentation remains a challenge. In this paper, we propose a simple, yet extremely effective, training-free technique, Plug-and-Play Open-Vocabulary Semantic Segmentation (PnP-OVSS) for this task. PnP-OVSS leverages a VLM with direct text-to-image cross-attention and an image-text matching loss. To balance between over-segmentation and under-segmentation, we introduce Salience Dropout; by iteratively dropping patches that the model is most attentive to, we are able to better resolve the entire extent of the segmentation mask. PnP-OVSS does not require any neural network training and performs hyperparameter tuning without the need for any segmentation annotations, even for a validation set. PnP-OVSS demonstrates substantial improvements over comparable baselines (+26.2% mIoU on Pascal VOC, +20.5% mIoU on MS COCO, +3.1% mIoU on COCO Stuff and +3.0% mIoU on ADE20K). Our codebase is at https://github.com/letitiabanana/PnP-OVSS.


title: Robust Transductive Few-shot Learning via Joint Message Passing and Prototype-based Soft-label Propagation

abstract: Few-shot learning (FSL) aims to develop a learning model with the ability to generalize to new classes using a few support samples. For transductive FSL tasks, prototype learning and label propagation methods are commonly employed. Prototype methods generally first learn the representative prototypes from the support set and then determine the labels of queries based on the metric between query samples and prototypes. Label propagation methods try to propagate the labels of support samples on the constructed graph encoding the relationships between both support and query samples. This paper aims to integrate these two principles together and develop an efficient and robust transductive FSL approach, termed Prototype-based Soft-label Propagation (PSLP). Specifically, we first estimate the soft-label presentation for each query sample by leveraging prototypes. Then, we conduct soft-label propagation on our learned query-support graph. Both steps are conducted progressively to boost their respective performance. Moreover, to learn effective prototypes for soft-label estimation as well as the desirable query-support graph for soft-label propagation, we design a new joint message passing scheme to learn sample presentation and relational graph jointly. Our PSLP method is parameter-free and can be implemented very efficiently. On four popular datasets, our method achieves competitive results on both balanced and imbalanced settings compared to the state-of-the-art methods. The code will be released upon acceptance.


title: Anonymous Jamming Detection in 5G with Bayesian Network Model Based Inference Analysis

abstract: Jamming and intrusion detection are critical in 5G research, aiming to maintain reliability, prevent user experience degradation, and avoid infrastructure failure. This paper introduces an anonymous jamming detection model for 5G based on signal parameters from the protocol stacks. The system uses supervised and unsupervised learning for real-time, high-accuracy detection of jamming, including unknown types. Supervised models reach an AUC of 0.964 to 1, compared to LSTM models with an AUC of 0.923 to 1. However, the need for data annotation limits the supervised approach. To address this, an unsupervised auto-encoder-based anomaly detection is presented with an AUC of 0.987. The approach is resistant to adversarial training samples. For transparency and domain knowledge injection, a Bayesian network-based causation analysis is introduced.


title: DyRA: Portable Dynamic Resolution Adjustment Network for Existing Detectors

abstract: Achieving constant accuracy in object detection is challenging due to the inherent variability of object sizes. One effective approach to this problem involves optimizing input resolution, referred to as a multi-resolution strategy. Previous approaches to resolution optimization have often been based on pre-defined resolutions with manual selection. However, there is a lack of study on run-time resolution optimization for existing architectures. This paper introduces DyRA, a dynamic resolution adjustment network providing an image-specific scale factor for existing detectors. This network is co-trained with detectors utilizing specially designed loss functions, namely ParetoScaleLoss and BalanceLoss. ParetoScaleLoss determines an adaptive scale factor for robustness, while BalanceLoss optimizes overall scale factors according to the localization performance of the detector. The loss function is devised to minimize the accuracy drop across contrasting objectives of different-sized objects for scaling. Our proposed network can improve accuracy across various models, including RetinaNet, Faster-RCNN, FCOS, DINO, and H-Deformable-DETR. The code is available at https://github.com/DaEunFullGrace/DyRA.git.


title: StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences

abstract: Occlusions between consecutive frames have long posed a significant challenge in optical flow estimation. The inherent ambiguity introduced by occlusions directly violates the brightness constancy constraint and considerably hinders pixel-to-pixel matching. To address this issue, multi-frame optical flow methods leverage adjacent frames to mitigate the local ambiguity. Nevertheless, prior multi-frame methods predominantly adopt recursive flow estimation, resulting in a considerable computational overlap. In contrast, we propose a streamlined in-batch framework that eliminates the need for extensive redundant recursive computations while concurrently developing effective spatio-temporal modeling approaches under in-batch estimation constraints. Specifically, we present a Streamlined In-batch Multi-frame (SIM) pipeline tailored to video input, attaining a similar level of time efficiency to two-frame networks. Furthermore, we introduce an efficient Integrative Spatio-temporal Coherence (ISC) modeling method for effective spatio-temporal modeling during the encoding phase, which introduces no additional parameter overhead. Additionally, we devise a Global Temporal Regressor (GTR) that effectively explores temporal relations during decoding. Benefiting from the efficient SIM pipeline and effective modules, StreamFlow not only excels in terms of performance on the challenging KITTI and Sintel datasets, with particular improvement in occluded areas but also attains a remarkable $63.82\%$ enhancement in speed compared with previous multi-frame methods. The code will be available soon at https://github.com/littlespray/StreamFlow.


title: Automatic cross-validation in structured models: Is it time to leave out leave-one-out?

abstract: Standard techniques such as leave-one-out cross-validation (LOOCV) might not be suitable for evaluating the predictive performance of models incorporating structured random effects. In such cases, the correlation between the training and test sets could have a notable impact on the model's prediction error. To overcome this issue, an automatic group construction procedure for leave-group-out cross validation (LGOCV) has recently emerged as a valuable tool for enhancing predictive performance measurement in structured models. The purpose of this paper is (i) to compare LOOCV and LGOCV within structured models, emphasizing model selection and predictive performance, and (ii) to provide real data applications in spatial statistics using complex structured models fitted with INLA, showcasing the utility of the automatic LGOCV method. First, we briefly review the key aspects of the recently proposed LGOCV method for automatic group construction in latent Gaussian models. We also demonstrate the effectiveness of this method for selecting the model with the highest predictive performance by simulating extrapolation tasks in both temporal and spatial data analyses. Finally, we provide insights into the effectiveness of the LGOCV method in modelling complex structured data, encompassing spatio-temporal multivariate count data, spatial compositional data, and spatio-temporal geospatial data.


title: Robust Diffusion GAN using Semi-Unbalanced Optimal Transport

abstract: Diffusion models, a type of generative model, have demonstrated great potential for synthesizing highly detailed images. By integrating with GAN, advanced diffusion models like DDGAN \citep{xiao2022DDGAN} could approach real-time performance for expansive practical applications. While DDGAN has effectively addressed the challenges of generative modeling, namely producing high-quality samples, covering different data modes, and achieving faster sampling, it remains susceptible to performance drops caused by datasets that are corrupted with outlier samples. This work introduces a robust training technique based on semi-unbalanced optimal transport to mitigate the impact of outliers effectively. Through comprehensive evaluations, we demonstrate that our robust diffusion GAN (RDGAN) outperforms vanilla DDGAN in terms of the aforementioned generative modeling criteria, i.e., image quality, mode coverage of distribution, and inference speed, and exhibits improved robustness when dealing with both clean and corrupted datasets.


title: Splinets -- Orthogonal Splines and FDA for the Classification Problem

abstract: This study introduces an efficient workflow for functional data analysis in classification problems, utilizing advanced orthogonal spline bases. The methodology is based on the flexible Splinets package, featuring a novel spline representation designed for enhanced data efficiency. Several innovative features contribute to this efficiency: 1)Utilization of Orthonormal Spline Bases 2)Consideration of Spline Support Sets 3)Data-Driven Knot Selection. Illustrating this approach, we applied the workflow to the Fashion MINST dataset. We demonstrate the classification process and highlight significant efficiency gains. Particularly noteworthy are the improvements that can be achieved through the 2D generalization of our methodology, especially in scenarios where data sparsity and dimension reduction are critical factors. A key advantage of our workflow is the projection operation into the space of splines with arbitrarily chosen knots, allowing for versatile functional data analysis associated with classification problems. Moreover, the study explores Splinets package features suited for functional data analysis. The algebra and calculus of splines use Taylor expansions at the knots within the support sets. Various orthonormalization techniques for B-splines are implemented, including the highly recommended dyadic method, which leads to the creation of splinets. Importantly, the locality of B-splines concerning support sets is preserved in the corresponding splinet. Using this locality, along with implemented algorithms, provides a powerful computational tool for functional data analysis.


title: Single-cell Multi-view Clustering via Community Detection with Unknown Number of Clusters

abstract: Single-cell multi-view clustering enables the exploration of cellular heterogeneity within the same cell from different views. Despite the development of several multi-view clustering methods, two primary challenges persist. Firstly, most existing methods treat the information from both single-cell RNA (scRNA) and single-cell Assay of Transposase Accessible Chromatin (scATAC) views as equally significant, overlooking the substantial disparity in data richness between the two views. This oversight frequently leads to a degradation in overall performance. Additionally, the majority of clustering methods necessitate manual specification of the number of clusters by users. However, for biologists dealing with cell data, precisely determining the number of distinct cell types poses a formidable challenge. To this end, we introduce scUNC, an innovative multi-view clustering approach tailored for single-cell data, which seamlessly integrates information from different views without the need for a predefined number of clusters. The scUNC method comprises several steps: initially, it employs a cross-view fusion network to create an effective embedding, which is then utilized to generate initial clusters via community detection. Subsequently, the clusters are automatically merged and optimized until no further clusters can be merged. We conducted a comprehensive evaluation of scUNC using three distinct single-cell datasets. The results underscored that scUNC outperforms the other baseline methods.


title: Single-Cell Deep Clustering Method Assisted by Exogenous Gene Information: A Novel Approach to Identifying Cell Types

abstract: In recent years, the field of single-cell data analysis has seen a marked advancement in the development of clustering methods. Despite advancements, most of these algorithms still concentrate on analyzing the provided single-cell matrix data. However, in medical applications, single-cell data often involves a wealth of exogenous information, including gene networks. Overlooking this aspect could lead to information loss and clustering results devoid of significant clinical relevance. An innovative single-cell deep clustering method, incorporating exogenous gene information, has been proposed to overcome this limitation. This model leverages exogenous gene network information to facilitate the clustering process, generating discriminative representations. Specifically, we have developed an attention-enhanced graph autoencoder, which is designed to efficiently capture the topological features between cells. Concurrently, we conducted a random walk on an exogenous Protein-Protein Interaction (PPI) network, thereby acquiring the gene's topological features. Ultimately, during the clustering process, we integrated both sets of information and reconstructed the features of both cells and genes to generate a discriminative representation. Extensive experiments have validated the effectiveness of our proposed method. This research offers enhanced insights into the characteristics and distribution of cells, thereby laying the groundwork for early diagnosis and treatment of diseases.


title: On the Calibration of Human Pose Estimation

abstract: Most 2D human pose estimation frameworks estimate keypoint confidence in an ad-hoc manner, using heuristics such as the maximum value of heatmaps. The confidence is part of the evaluation scheme, e.g., AP for the MSCOCO dataset, yet has been largely overlooked in the development of state-of-the-art methods. This paper takes the first steps in addressing miscalibration in pose estimation. From a calibration point of view, the confidence should be aligned with the pose accuracy. In practice, existing methods are poorly calibrated. We show, through theoretical analysis, why a miscalibration gap exists and how to narrow the gap. Simply predicting the instance size and adjusting the confidence function gives considerable AP improvements. Given the black-box nature of deep neural networks, however, it is not possible to fully close this gap with only closed-form adjustments. As such, we go one step further and learn network-specific adjustments by enforcing consistency between confidence and pose accuracy. Our proposed Calibrated ConfidenceNet (CCNet) is a light-weight post-hoc addition that improves AP by up to 1.4% on off-the-shelf pose estimation frameworks. Applied to the downstream task of mesh recovery, CCNet facilitates an additional 1.0mm decrease in 3D keypoint error.


title: Level-Rank Dualities from $\Phi$-Harish-Chandra Series and Affine Springer Fibers

abstract: For any generic finite reductive group $\mathbb{G}$, integer $e > 0$, and $\Phi_e$-cuspidal pair $(\mathbb{L}, \lambda)$, Brou\'{e}-Malle-Michel conjectured that the endomorphism rings of the Deligne-Lusztig representations attached to $\mathbb{G}, (\mathbb{L}, \lambda)$ all come from the same generic cyclotomic Hecke algebra. We propose a new conjecture about the Harish-Chandra theory of such pairs, involving two integers $e$ and $m$: namely, that the intersection of an $\Phi_e$-Harish-Chandra series and a $\Phi_m$-Harish-Chandra series is parametrized by both a union of $\Phi_m$-blocks of the $\Phi_e$-Hecke algebra and a union of $\Phi_e$-blocks of the $\Phi_m$-Hecke algebra, in a way that matches blocks. We also conjecture that when blocks match, there is an equivalence of categories between their highest-weight covers. When $e = 1$, we provide evidence that our bijections are essentially realized by bimodules that Oblomkov-Yun construct from the cohomology of affine Springer fibers. This suggests a strange analogy: Roughly, homogeneous affine Springer fibers are to roots of unity as tensor products of Deligne-Lusztig representations are to prime powers. We predict the generic Hecke parameters for arbitrary $\Phi$-cuspidal pairs of the groups $\mathbb{G}\mathbb{L}_n$ and $\mathbb{G}\mathbb{U}_n$, unifying the known cases. We prove that they would imply our conjectural bijections for these groups and coprime $e, m$. Then we show that the bijections for $\mathbb{G}\mathbb{L}_n$ are related by affine permutations to Uglov's bijections between bases of higher-level Fock spaces. This relates our conjectural equivalences of categories to those conjectured by Chuang-Miyachi, and proved by several authors, under the name of level-rank duality. Finally, for many cases in exceptional types, we verify that the parameters predicted by Brou\'e-Malle are compatible with our conjectures.