title: TAGA: Text-Attributed Graph Self-Supervised Learning by Synergizing Graph and Text Mutual Transformations

abstract: Text-Attributed Graphs (TAGs) enhance graph structures with natural language descriptions, enabling detailed representation of data and their relationships across a broad spectrum of real-world scenarios. Despite the potential for deeper insights, existing TAG representation learning primarily relies on supervised methods, necessitating extensive labeled data and limiting applicability across diverse contexts. This paper introduces a new self-supervised learning framework, Text-And-Graph Multi-View Alignment (TAGA), which overcomes these constraints by integrating TAGs' structural and semantic dimensions. TAGA constructs two complementary views: Text-of-Graph view, which organizes node texts into structured documents based on graph topology, and the Graph-of-Text view, which converts textual nodes and connections into graph data. By aligning representations from both views, TAGA captures joint textual and structural information. In addition, a novel structure-preserving random walk algorithm is proposed for efficient training on large-sized TAGs. Our framework demonstrates strong performance in zero-shot and few-shot scenarios across eight real-world datasets.


title: The neutrino force in neutrino backgrounds: Spin dependence and parity-violating effects

abstract: The neutrino force results from the exchange of a pair of neutrinos. A neutrino background can significantly influence this force. In this work, we present a comprehensive calculation of the neutrino force in various neutrino backgrounds with spin dependence taken into account. In particular, we calculate the spin-independent and spin-dependent parity-conserving neutrino forces, in addition to the spin-dependent parity-violating neutrino forces with and without the presence of a neutrino background for both isotropic and anisotropic backgrounds. Compared with the vacuum case, the neutrino background can effectively violate Lorentz invariance and lead to additional parity-violating terms that are not suppressed by the velocity of external particles. We estimate the magnitude of the effect of atomic parity-violation experiments, and it turns out to be well below the current experimental sensitivity.


title: AutoCV: Empowering Reasoning with Automated Process Labeling via Confidence Variation

abstract: In this work, we propose a novel method named \textbf{Auto}mated Process Labeling via \textbf{C}onfidence \textbf{V}ariation (\textbf{\textsc{AutoCV}}) to enhance the reasoning capabilities of large language models (LLMs) by automatically annotating the reasoning steps. Our approach begins by training a verification model on the correctness of final answers, enabling it to generate automatic process annotations. This verification model assigns a confidence score to each reasoning step, indicating the probability of arriving at the correct final answer from that point onward. We detect relative changes in the verification's confidence scores across reasoning steps to automatically annotate the reasoning process. This alleviates the need for numerous manual annotations or the high computational costs associated with model-induced annotation approaches. We experimentally validate that the confidence variations learned by the verification model trained on the final answer correctness can effectively identify errors in the reasoning steps. Subsequently, we demonstrate that the process annotations generated by \textsc{AutoCV} can improve the accuracy of the verification model in selecting the correct answer from multiple outputs generated by LLMs. Notably, we achieve substantial improvements across five datasets in mathematics and commonsense reasoning. The source code of \textsc{AutoCV} is available at \url{https://github.com/rookie-joe/AUTOCV}.


title: TIE: Revolutionizing Text-based Image Editing for Complex-Prompt Following and High-Fidelity Editing

abstract: As the field of image generation rapidly advances, traditional diffusion models and those integrated with multimodal large language models (LLMs) still encounter limitations in interpreting complex prompts and preserving image consistency pre and post-editing. To tackle these challenges, we present an innovative image editing framework that employs the robust Chain-of-Thought (CoT) reasoning and localizing capabilities of multimodal LLMs to aid diffusion models in generating more refined images. We first meticulously design a CoT process comprising instruction decomposition, region localization, and detailed description. Subsequently, we fine-tune the LISA model, a lightweight multimodal LLM, using the CoT process of Multimodal LLMs and the mask of the edited image. By providing the diffusion models with knowledge of the generated prompt and image mask, our models generate images with a superior understanding of instructions. Through extensive experiments, our model has demonstrated superior performance in image generation, surpassing existing state-of-the-art models. Notably, our model exhibits an enhanced ability to understand complex prompts and generate corresponding images, while maintaining high fidelity and consistency in images before and after generation.


title: Gas-phase formation of fullerene/9-hydroxyfluorene cluster cations

abstract: In interstellar environment, fullerene species readily react with large molecules (e.g., PAHs and their derivatives) in the gas phase, which may be the formation route of carbon dust grains in space. In this work, the gas-phase ion-molecule collision reaction between fullerene cations (Cn+, n=32, 34, ..., 60) and functionalized PAH molecules (9-hydroxyfluorene, C13H10O) are investigated both experimentally and theoretically. The experimental results show that fullerene/9-hydroxyfluorene cluster cations are efficiently formed, leading to a series of large fullerene/9-hydroxyfluorene cluster cations (e.g., [(C13H10O)C60]+, [(C13H10O)3C58+, and [(C26H18O)(C13H10O)2C48]+). The binding energies and optimized structures of typical fullerene/9-hydroxyfluorene cluster cations were calculated. The bonding ability plays a decisive role in the cluster formation processes. The reaction surfaces, modes and combination reaction sites can result in different binding energies, which represent the relative chemical reactivity. Therefore, the geometry and composition of fullerene/9-hydroxyfluorene cluster cations are complicated. In addition, there is an enhanced chemical reactivity for smaller fullerene cations, which is mainly attributed to the newly formed deformed carbon rings (e.g., 7 C-ring). As part of the coevolution network of interstellar fullerene chemistry, our results suggest that ion-molecule collision reactions contribute to the formation of various fullerene/9-hydroxyfluorene cluster cations in the ISM, providing insights into different chemical reactivity caused by oxygenated functional groups (e.g., hydroxyl, OH, or ether, C-O-C) on the cluster formations.


title: Gradient Compressed Sensing: A Query-Efficient Gradient Estimator for High-Dimensional Zeroth-Order Optimization

abstract: We study nonconvex zeroth-order optimization (ZOO) in a high-dimensional space $\mathbb R^d$ for functions with approximately $s$-sparse gradients. To reduce the dependence on the dimensionality $d$ in the query complexity, high-dimensional ZOO methods seek to leverage gradient sparsity to design gradient estimators. The previous best method needs $O\big(s\log\frac ds\big)$ queries per step to achieve $O\big(\frac1T\big)$ rate of convergence w.r.t. the number T of steps. In this paper, we propose *Gradient Compressed Sensing* (GraCe), a query-efficient and accurate estimator for sparse gradients that uses only $O\big(s\log\log\frac ds\big)$ queries per step and still achieves $O\big(\frac1T\big)$ rate of convergence. To our best knowledge, we are the first to achieve a *double-logarithmic* dependence on $d$ in the query complexity under weaker assumptions. Our proposed GraCe generalizes the Indyk--Price--Woodruff (IPW) algorithm in compressed sensing from linear measurements to nonlinear functions. Furthermore, since the IPW algorithm is purely theoretical due to its impractically large constant, we improve the IPW algorithm via our *dependent random partition* technique together with our corresponding novel analysis and successfully reduce the constant by a factor of nearly 4300. Our GraCe is not only theoretically query-efficient but also achieves strong empirical performance. We benchmark our GraCe against 12 existing ZOO methods with 10000-dimensional functions and demonstrate that GraCe significantly outperforms existing methods.


title: Entity Alignment with Noisy Annotations from Large Language Models

abstract: Entity alignment (EA) aims to merge two knowledge graphs (KGs) by identifying equivalent entity pairs. While existing methods heavily rely on human-generated labels, it is prohibitively expensive to incorporate cross-domain experts for annotation in real-world scenarios. The advent of Large Language Models (LLMs) presents new avenues for automating EA with annotations, inspired by their comprehensive capability to process semantic information. However, it is nontrivial to directly apply LLMs for EA since the annotation space in real-world KGs is large. LLMs could also generate noisy labels that may mislead the alignment. To this end, we propose a unified framework, LLM4EA, to effectively leverage LLMs for EA. Specifically, we design a novel active learning policy to significantly reduce the annotation space by prioritizing the most valuable entities based on the entire inter-KG and intra-KG structure. Moreover, we introduce an unsupervised label refiner to continuously enhance label accuracy through in-depth probabilistic reasoning. We iteratively optimize the policy based on the feedback from a base EA model. Extensive experiments demonstrate the advantages of LLM4EA on four benchmark datasets in terms of effectiveness, robustness, and efficiency. Codes are available via https://github.com/chensyCN/llm4ea_official.


title: Extreme Compression of Adaptive Neural Images

abstract: Implicit Neural Representations (INRs) and Neural Fields are a novel paradigm for signal representation, from images and audio to 3D scenes and videos. The fundamental idea is to represent a signal as a continuous and differentiable neural network. This idea offers unprecedented benefits such as continuous resolution and memory efficiency, enabling new compression techniques. However, representing data as neural networks poses new challenges. For instance, given a 2D image as a neural network, how can we further compress such a neural image?. In this work, we present a novel analysis on compressing neural fields, with the focus on images. We also introduce Adaptive Neural Images (ANI), an efficient neural representation that enables adaptation to different inference or transmission requirements. Our proposed method allows to reduce the bits-per-pixel (bpp) of the neural image by 4x, without losing sensitive details or harming fidelity. We achieve this thanks to our successful implementation of 4-bit neural representations. Our work offers a new framework for developing compressed neural fields.


title: Geometric Phase in Kitaev Quantum Spin Liquid

abstract: Quantum spin liquid has massive many spin entanglement in the ground state, we can evaluate it by the entanglement entropy, but the latter can not be observed directly by experiment. In this manuscript, we try to characterize its topological properties by the geometric phase. However the usual adiabatic or non-adiabatic geometric phase can not appear in the density matrix of entanglement entropy, so we extend it to the sub-geometric phase which can exist in the density matrix and have influence on the entanglement entropy, spin correlation function as well as other physical observable. We will demonstrate that the imaginary part of sub-geometric phase will deviate the resonance peak by an amount concerning with this phase and affect the energy level crossing, while the real part of sub-geometric phase will determine the stability of initial state, it may provide a complement on the selection rule of quantum transition.


title: Trajectory Data Suffices for Statistically Efficient Learning in Offline RL with Linear $q^\pi$-Realizability and Concentrability

abstract: We consider offline reinforcement learning (RL) in $H$-horizon Markov decision processes (MDPs) under the linear $q^\pi$-realizability assumption, where the action-value function of every policy is linear with respect to a given $d$-dimensional feature function. The hope in this setting is that learning a good policy will be possible without requiring a sample size that scales with the number of states in the MDP. Foster et al. [2021] have shown this to be impossible even under $\textit{concentrability}$, a data coverage assumption where a coefficient $C_\text{conc}$ bounds the extent to which the state-action distribution of any policy can veer off the data distribution. However, the data in this previous work was in the form of a sequence of individual transitions. This leaves open the question of whether the negative result mentioned could be overcome if the data was composed of sequences of full trajectories. In this work we answer this question positively by proving that with trajectory data, a dataset of size $\text{poly}(d,H,C_\text{conc})/\epsilon^2$ is sufficient for deriving an $\epsilon$-optimal policy, regardless of the size of the state space. The main tool that makes this result possible is due to Weisz et al. [2023], who demonstrate that linear MDPs can be used to approximate linearly $q^\pi$-realizable MDPs. The connection to trajectory data is that the linear MDP approximation relies on "skipping" over certain states. The associated estimation problems are thus easy when working with trajectory data, while they remain nontrivial when working with individual transitions. The question of computational efficiency under our assumptions remains open.


title: Performance evaluation of Reddit Comments using Machine Learning and Natural Language Processing methods in Sentiment Analysis

abstract: Sentiment analysis, an increasingly vital field in both academia and industry, plays a pivotal role in machine learning applications, particularly on social media platforms like Reddit. However, the efficacy of sentiment analysis models is hindered by the lack of expansive and fine-grained emotion datasets. To address this gap, our study leverages the GoEmotions dataset, comprising a diverse range of emotions, to evaluate sentiment analysis methods across a substantial corpus of 58,000 comments. Distinguished from prior studies by the Google team, which limited their analysis to only two models, our research expands the scope by evaluating a diverse array of models. We investigate the performance of traditional classifiers such as Naive Bayes and Support Vector Machines (SVM), as well as state-of-the-art transformer-based models including BERT, RoBERTa, and GPT. Furthermore, our evaluation criteria extend beyond accuracy to encompass nuanced assessments, including hierarchical classification based on varying levels of granularity in emotion categorization. Additionally, considerations such as computational efficiency are incorporated to provide a comprehensive evaluation framework. Our findings reveal that the RoBERTa model consistently outperforms the baseline models, demonstrating superior accuracy in fine-grained sentiment classification tasks. This underscores the substantial potential and significance of the RoBERTa model in advancing sentiment analysis capabilities.


title: Gas-phase hydrogenation of large, astronomically relevant PAH cations

abstract: To investigate the gas-phase hydrogenation processes of large, astronomically relevant cationic polycyclic aromatic hydrocarbon (PAH) molecules under the interstellar environments, the ion-molecule collision reaction between six PAH cations and H-atoms is studied. The experimental results show that the hydrogenated PAH cations are efficiently formed, and no even-odd hydrogenated mass patterns are observed in the hydrogenation processes. The structure of newly formed hydrogenated PAH cations and the bonding energy for the hydrogenation reaction pathways are investigated with quantum theoretical calculations. The exothermic energy for each reaction pathway is relatively high, and the competition between hydrogenation and dehydrogenation is confirmed. From the theoretical calculation, the bonding ability plays an important role in the gas-phase hydrogenation processes. The factors that affect the hydrogenation chemical reactivity are discussed, including the effect of carbon skeleton structure, the side-edged structure, the molecular size, the five- and six-membered C-ring structure, the bay region structure, and the neighboring hydrogenation. The IR spectra of hydrogenated PAH cations are also calculated. These results we obtain once again validate the complexity of hydrogenated PAH molecules, and provide the direction for the simulations and observations under the coevolution interstellar chemistry network. We infer that if we do not consider other chemical evolution processes (e.g., photo-evolution), then the hydrogenation states and forms of PAH compounds are intricate and complex in the interstellar medium (ISM).


title: Enhanced Geological Prediction for Tunnel Excavation Using Full Waveform Inversion Integrating Sobolev Space Regularization with a Quadratic Penalty Method

abstract: In the process of tunnel excavation, advance geological prediction technology has become indispensable for safe, economical, and efficient tunnel construction. Although traditional methods such as drilling and geological analysis are effective, they typically involve destructive processes, carry high risks, and incur significant costs. In contrast, non-destructive geophysical exploration offers a more convenient and economical alternative. However, the accuracy and precision of these non-destructive methods can be severely compromised by complex geological structures and environmental noise. To address these challenges effectively, a novel approach using frequency domain full waveform inversion, based on a penalty method and Sobolev space regularization, has been proposed to enhance the performance of non-destructive predictions. The proposed method constructs a soft-constrained optimization problem by restructuring the misfit function into a combination of data misfit and wave equation drive terms to enhance convexity. Additionally, it semi-extends the search space to both the wavefield and the model parameters to mitigate the strong nonlinearity of the optimization, facilitating high-resolution inversion. Furthermore, a Sobolev space regularization algorithm is introduced to flexibly adjust the regularization path, addressing issues related to noise and artefacts to enhance the robustness of the algorithm. We evaluated the modified full waveform inversion with several tunnel fault models by comparing the results of the enhanced method with those of traditional least squares-based Tikhonov regularization and total variation regularization full waveform inversion methods. The verification results confirm the superior capabilities of the proposed method as expected.


title: SiNGR: Brain Tumor Segmentation via Signed Normalized Geodesic Transform Regression

abstract: One of the primary challenges in brain tumor segmentation arises from the uncertainty of voxels close to tumor boundaries. However, the conventional process of generating ground truth segmentation masks fails to treat such uncertainties properly. Those ``hard labels'' with 0s and 1s conceptually influenced the majority of prior studies on brain image segmentation. As a result, tumor segmentation is often solved through voxel classification. In this work, we instead view this problem as a voxel-level regression, where the ground truth represents a certainty mapping from any pixel based on the distance to tumor border. We propose a novel ground truth label transformation, which is based on a signed geodesic transform, to capture the uncertainty in brain tumors' vicinity, while maintaining a margin between positive and negative samples. We combine this idea with a Focal-like regression L1-loss that enables effective regression learning in high-dimensional output space by appropriately weighting voxels according to their difficulty. We thoroughly conduct an experimental evaluation to validate the components of our proposed method, compare it to a diverse array of state-of-the-art segmentation models, and show that it is architecture-agnostic. The code of our method is made publicly available (\url{https://github.com/Oulu-IMEDS/SiNGR/}).


title: Series Associated with Harmonic Numbers, Fibonacci Numbers and Central Binomial Coefficients $\binom{2n}{n}$

abstract: We find various series that involves the central binomial coefficients $\binom{2n}{n}$, harmonic numbers and Fibonacci Numbers.\\ Contrary to the traditional hypergeometric function $_pF_q$ approach, our method utilizes a straightforward transformation to obtain new evaluations linked to Fibonacci numbers and the golden ratio. Before the end of this paper, we also gave a new series representation for $\zeta(2)$.


title: Image-level Regression for Uncertainty-aware Retinal Image Segmentation

abstract: Accurate retinal vessel segmentation is a crucial step in the quantitative assessment of retinal vasculature, which is needed for the early detection of retinal diseases and other conditions. Numerous studies have been conducted to tackle the problem of segmenting vessels automatically using a pixel-wise classification approach. The common practice of creating ground truth labels is to categorize pixels as foreground and background. This approach is, however, biased, and it ignores the uncertainty of a human annotator when it comes to annotating e.g. thin vessels. In this work, we propose a simple and effective method that casts the retinal image segmentation task as an image-level regression. For this purpose, we first introduce a novel Segmentation Annotation Uncertainty-Aware (SAUNA) transform, which adds pixel uncertainty to the ground truth using the pixel's closeness to the annotation boundary and vessel thickness. To train our model with soft labels, we generalize the earlier proposed Jaccard metric loss to arbitrary hypercubes, which is a second contribution of this work. The proposed SAUNA transform and the new theoretical results allow us to directly train a standard U-Net-like architecture at the image level, outperforming all recently published methods. We conduct thorough experiments and compare our method to a diverse set of baselines across 5 retinal image datasets. Our implementation is available at \url{https://github.com/Oulu-IMEDS/SAUNA}.


title: Characterization and Novel Application of Power Over Fiber for Electronics in a Harsh Environment

abstract: Power-over-Fiber (PoF) technology has been used extensively in settings where high voltages require isolation from ground. In a novel application of PoF, power is provided to photon detector modules located on a surface at $\sim$ 300 kV with respect to ground in the planned DUNE experiment. In cryogenic environments, PoF offers a reliable means of power transmission, leveraging optical fibers to transfer power with minimal system degradation. PoF technology excels in maintaining low noise levels when delivering power to sensitive electronic systems operating in extreme temperatures and high voltage environments. This paper presents the R$\&$D effort of PoF in extreme conditions and underscores its capacity to revolutionize power delivery and management in critical applications, offering a dependable solution with low noise, optimal efficiency, and superior isolation.


title: Controlling Rate, Distortion, and Realism: Towards a Single Comprehensive Neural Image Compression Model

abstract: In recent years, neural network-driven image compression (NIC) has gained significant attention. Some works adopt deep generative models such as GANs and diffusion models to enhance perceptual quality (realism). A critical obstacle of these generative NIC methods is that each model is optimized for a single bit rate. Consequently, multiple models are required to compress images to different bit rates, which is impractical for real-world applications. To tackle this issue, we propose a variable-rate generative NIC model. Specifically, we explore several discriminator designs tailored for the variable-rate approach and introduce a novel adversarial loss. Moreover, by incorporating the newly proposed multi-realism technique, our method allows the users to adjust the bit rate, distortion, and realism with a single model, achieving ultra-controllability. Unlike existing variable-rate generative NIC models, our method matches or surpasses the performance of state-of-the-art single-rate generative NIC models while covering a wide range of bit rates using just one model. Code will be available at https://github.com/iwa-shi/CRDR


title: Advancing Behavior Generation in Mobile Robotics through High-Fidelity Procedural Simulations

abstract: This paper introduces YamaS, a simulator integrating Unity3D Engine with Robotic Operating System for robot navigation research and aims to facilitate the development of both Deep Reinforcement Learning (Deep-RL) and Natural Language Processing (NLP). It supports single and multi-agent configurations with features like procedural environment generation, RGB vision, and dynamic obstacle navigation. Unique to YamaS is its ability to construct single and multi-agent environments, as well as generating agent's behaviour through textual descriptions. The simulator's fidelity is underscored by comparisons with the real-world Yamabiko Beego robot, demonstrating high accuracy in sensor simulations and spatial reasoning. Moreover, YamaS integrates Virtual Reality (VR) to augment Human-Robot Interaction (HRI) studies, providing an immersive platform for developers and researchers. This fusion establishes YamaS as a versatile and valuable tool for the development and testing of autonomous systems, contributing to the fields of robot simulation and AI-driven training methodologies.


title: Automatic Domain Adaptation by Transformers in In-Context Learning

abstract: Selecting or designing an appropriate domain adaptation algorithm for a given problem remains challenging. This paper presents a Transformer model that can provably approximate and opt for domain adaptation methods for a given dataset in the in-context learning framework, where a foundation model performs new tasks without updating its parameters at test time. Specifically, we prove that Transformers can approximate instance-based and feature-based unsupervised domain adaptation algorithms and automatically select an algorithm suited for a given dataset. Numerical results indicate that in-context learning demonstrates an adaptive domain adaptation surpassing existing methods.


title: Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings

abstract: The rapid proliferation of generative AI has raised questions about the competitiveness of lower-parameter, locally tunable, open-weight models relative to high-parameter, API-guarded, closed-weight models in terms of performance, domain adaptation, cost, and generalization. Centering under-resourced yet risk-intolerant settings in government, research, and healthcare, we see for-profit closed-weight models as incompatible with requirements for transparency, privacy, adaptability, and standards of evidence. Yet the performance penalty in using open-weight models, especially in low-data and low-resource settings, is unclear. We assess the feasibility of using smaller, open-weight models to replace GPT-4-Turbo in zero-shot, few-shot, and fine-tuned regimes, assuming access to only a single, low-cost GPU. We assess value-sensitive issues around bias, privacy, and abstention on three additional tasks relevant to those topics. We find that with relatively low effort, very low absolute monetary cost, and relatively little data for fine-tuning, small open-weight models can achieve competitive performance in domain-adapted tasks without sacrificing generality. We then run experiments considering practical issues in bias, privacy, and hallucination risk, finding that open models offer several benefits over closed models. We intend this work as a case study in understanding the opportunity cost of reproducibility and transparency over for-profit state-of-the-art zero shot performance, finding this cost to be marginal under realistic settings.


title: Perturbation-Restrained Sequential Model Editing

abstract: Model editing is an emerging field that focuses on updating the knowledge embedded within large language models (LLMs) without extensive retraining. However, current model editing methods significantly compromise the general abilities of LLMs as the number of edits increases, and this trade-off poses a substantial challenge to the continual learning of LLMs. In this paper, we first theoretically analyze that the factor affecting the general abilities in sequential model editing lies in the condition number of the edited matrix. The condition number of a matrix represents its numerical sensitivity, and therefore can be used to indicate the extent to which the original knowledge associations stored in LLMs are perturbed after editing. Subsequently, statistical findings demonstrate that the value of this factor becomes larger as the number of edits increases, thereby exacerbating the deterioration of general abilities. To this end, a framework termed Perturbation Restraint on Upper bouNd for Editing (PRUNE) is proposed, which applies the condition number restraints in sequential editing. These restraints can lower the upper bound on perturbation to edited models, thus preserving the general abilities. Systematically, we conduct experiments employing three popular editing methods on three LLMs across four representative downstream tasks. Evaluation results show that PRUNE can preserve considerable general abilities while maintaining the editing performance effectively in sequential model editing. The code and data are available at https://github.com/mjy1111/PRUNE.


title: Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfels

abstract: Video generative models are receiving particular attention given their ability to generate realistic and imaginative frames. Besides, these models are also observed to exhibit strong 3D consistency, significantly enhancing their potential to act as world simulators. In this work, we present Vidu4D, a novel reconstruction model that excels in accurately reconstructing 4D (i.e., sequential 3D) representations from single generated videos, addressing challenges associated with non-rigidity and frame distortion. This capability is pivotal for creating high-fidelity virtual contents that maintain both spatial and temporal coherence. At the core of Vidu4D is our proposed Dynamic Gaussian Surfels (DGS) technique. DGS optimizes time-varying warping functions to transform Gaussian surfels (surface elements) from a static state to a dynamically warped state. This transformation enables a precise depiction of motion and deformation over time. To preserve the structural integrity of surface-aligned Gaussian surfels, we design the warped-state geometric regularization based on continuous warping fields for estimating normals. Additionally, we learn refinements on rotation and scaling parameters of Gaussian surfels, which greatly alleviates texture flickering during the warping process and enhances the capture of fine-grained appearance details. Vidu4D also contains a novel initialization state that provides a proper start for the warping fields in DGS. Equipping Vidu4D with an existing video generative model, the overall framework demonstrates high-fidelity text-to-4D generation in both appearance and geometry.


title: Unified Editing of Panorama, 3D Scenes, and Videos Through Disentangled Self-Attention Injection

abstract: While text-to-image models have achieved impressive capabilities in image generation and editing, their application across various modalities often necessitates training separate models. Inspired by existing method of single image editing with self attention injection and video editing with shared attention, we propose a novel unified editing framework that combines the strengths of both approaches by utilizing only a basic 2D image text-to-image (T2I) diffusion model. Specifically, we design a sampling method that facilitates editing consecutive images while maintaining semantic consistency utilizing shared self-attention features during both reference and consecutive image sampling processes. Experimental results confirm that our method enables editing across diverse modalities including 3D scenes, videos, and panorama images.


title: Phase transition in binary compressed sensing based on $L_{1}$-norm minimization

abstract: Compressed sensing is a signal processing scheme that reconstructs high-dimensional sparse signals from a limited number of observations. In recent years, various problems involving signals with a finite number of discrete values have been attracting attention in the field of compressed sensing. In particular, binary compressed sensing, which restricts signal elements to binary values $\{0, 1\}$, is the most fundamental and straightforward analysis subject in such problem settings. We evaluate the typical performance of noiseless binary compressed sensing based on $L_{1}$-norm minimization using the replica method, a statistical mechanical approach. We analyze a general setting where the elements of the observation matrix follow a Gaussian distribution, including a non-zero mean. We demonstrate that the biased observation matrix indicates more reconstruction success conditions in binary compressed sensing. Our results are consistent with the outcomes of several prior studies.


title: From limit theorems to mixing limit theorems

abstract: Motivated by work of Dolgopyat and N\'andori, we establish a general method for upgrading limit theorems for Birkhoff sums and cocycles over dynamical systems to mixing limit theorems under mild ergodicity and hyperbolicity assumptions. Building on previous work of Al-Saqban and Forni, we apply this method to obtain mixing limit theorems for particular subbundles of the Kontsevich-Zorich cocycle. In forthcoming work of Arana-Herrera and Honaryar these results are applied to study the arithmetic/homological complexity of long simple closed geodesics on negatively curved surfaces.


title: Image formation near hyperbolic umbilic in strong gravitational lensing

abstract: Hyperbolic umbilic (HU) is a point singularity of the gravitational lens equation, giving rise to a ring-shaped image formation made of four highly magnified images, off-centred from the lens centre. Recent observations have revealed new strongly lensed image formations near HU singularities, and many more are expected in ongoing and future observations. Like fold/cusp, image formations near HU also satisfy magnification relation ($R_{\rm hu}$), i.e., the signed magnification sum of the four images equals zero. Here, we study how $R_{\rm hu}$ deviates from zero as a function of area ($A_{\rm hu}$) covered by the image formation near HU and the distance ($d$) of the central maxima image (which is part of the HU image formation) from the lens centre for ideal single- and double-component cluster-scale lenses. For lens ellipticity values $\geq0.3$, the central maxima image will form sufficiently far from the lens centre ($d\gtrsim5''$), similar to the observed HU image formations. We also find that, in some cases, double-component and actual cluster-scale lenses can lead to large cross-sections for HU(-like) image formations for sources at $z\gtrsim5$ effectively increasing the chances to observe HU(-like) image formation at high redshift. Finally, we study the time delay distribution in the observed HU image formation, finding that not only are these images highly magnified, but the relative time delay between various pairs of HU characteristic image formation has a typical value of $\sim100$ days, an order of magnitude smaller than generic five-image formations in cluster lenses, making such image formations a promising target for time delay studies.


title: Structure-preserving finite element methods for computing dynamics of rotating Bose-Einstein condensate

abstract: This work is concerned with the construction and analysis of structure-preserving Galerkin methods for computing the dynamics of rotating Bose-Einstein condensate (BEC) based on the Gross-Pitaevskii equation with angular momentum rotation. Due to the presence of the rotation term, constructing finite element methods (FEMs) that preserve both mass and energy remains an unresolved issue, particularly in the context of nonconforming FEMs. Furthermore, in comparison to existing works, we provide a comprehensive convergence analysis, offering a thorough demonstration of the methods' optimal and high-order convergence properties. Finally, extensive numerical results are presented to check the theoretical analysis of the structure-preserving numerical method for rotating BEC, and the quantized vortex lattice's behavior is scrutinized through a series of numerical tests.


title: Kernel-based optimally weighted conformal prediction intervals

abstract: Conformal prediction has been a popular distribution-free framework for uncertainty quantification. In this paper, we present a novel conformal prediction method for time-series, which we call Kernel-based Optimally Weighted Conformal Prediction Intervals (KOWCPI). Specifically, KOWCPI adapts the classic Reweighted Nadaraya-Watson (RNW) estimator for quantile regression on dependent data and learns optimal data-adaptive weights. Theoretically, we tackle the challenge of establishing a conditional coverage guarantee for non-exchangeable data under strong mixing conditions on the non-conformity scores. We demonstrate the superior performance of KOWCPI on real time-series against state-of-the-art methods, where KOWCPI achieves narrower confidence intervals without losing coverage.


title: PyGS: Large-scale Scene Representation with Pyramidal 3D Gaussian Splatting

abstract: Neural Radiance Fields (NeRFs) have demonstrated remarkable proficiency in synthesizing photorealistic images of large-scale scenes. However, they are often plagued by a loss of fine details and long rendering durations. 3D Gaussian Splatting has recently been introduced as a potent alternative, achieving both high-fidelity visual results and accelerated rendering performance. Nonetheless, scaling 3D Gaussian Splatting is fraught with challenges. Specifically, large-scale scenes grapples with the integration of objects across multiple scales and disparate viewpoints, which often leads to compromised efficacy as the Gaussians need to balance between detail levels. Furthermore, the generation of initialization points via COLMAP from large-scale dataset is both computationally demanding and prone to incomplete reconstructions. To address these challenges, we present Pyramidal 3D Gaussian Splatting (PyGS) with NeRF Initialization. Our approach represent the scene with a hierarchical assembly of Gaussians arranged in a pyramidal fashion. The top level of the pyramid is composed of a few large Gaussians, while each subsequent layer accommodates a denser collection of smaller Gaussians. We effectively initialize these pyramidal Gaussians through sampling a rapidly trained grid-based NeRF at various frequencies. We group these pyramidal Gaussians into clusters and use a compact weighting network to dynamically determine the influence of each pyramid level of each cluster considering camera viewpoint during rendering. Our method achieves a significant performance leap across multiple large-scale datasets and attains a rendering time that is over 400 times faster than current state-of-the-art approaches.


title: Structured Graph Network for Constrained Robot Crowd Navigation with Low Fidelity Simulation

abstract: We investigate the feasibility of deploying reinforcement learning (RL) policies for constrained crowd navigation using a low-fidelity simulator. We introduce a representation of the dynamic environment, separating human and obstacle representations. Humans are represented through detected states, while obstacles are represented as computed point clouds based on maps and robot localization. This representation enables RL policies trained in a low-fidelity simulator to deploy in real world with a reduced sim2real gap. Additionally, we propose a spatio-temporal graph to model the interactions between agents and obstacles. Based on the graph, we use attention mechanisms to capture the robot-human, human-human, and human-obstacle interactions. Our method significantly improves navigation performance in both simulated and real-world environments. Video demonstrations can be found at https://sites.google.com/view/constrained-crowdnav/home.


title: Giant anomalous Hall effect and band folding in a Kagome metal with mixed dimensionality

abstract: Magnetic metals with geometric frustration offer a fertile ground for studying novel states of matter with strong quantum fluctuations and unique electromagnetic responses from conduction electrons coupled to spin textures. Recently, TbTi$_3$Bi$_4$ has emerged as such an intriguing platform as it behaves as a quasi-one-dimension (quasi-1D) Ising magnet with antiferromagnetic orderings at 20.4 K and 3 K, respectively. Magnetic fields along the Tb zigzag-chain direction reveal plateaus at 1/3 and 2/3 of saturated magnetization, respectively. At metamagnetic transition boundaries, a record-high anomalous Hall conductivity of 6.2 $\times$ 10$^5$ $\Omega^{-1}$ cm$^{-1}$ is observed. Within the plateau, noncollinear magnetic texture is suggested. In addition to the characteristic Kagome 2D electronic structure, ARPES unequivocally demonstrates quasi-1D electronic structure from the Tb 5$d$ bands and a quasi-1D hybridization gap in the magnetic state due to band folding with $q$ = (1/3, 0, 0) possibly from the spin-density-wave order along the Tb chain. These findings emphasize the crucial role of mixed dimensionality and the strong coupling between magnetic texture and electronic band structure in regulating physical properties of materials, offering new strategies for designing materials for future spintronics applications.


title: Bulk Reconstruction in De Sitter Spacetime

abstract: The bulk reconstruction program involves expressing local bulk fields as non-local operators on the boundary. It was initiated in the context of AdS/CFT correspondence. Attempts to extend it to de Sitter have been successful for heavy(principal series) scalar fields. For other fields, the construction ran into issues. In particular, divergences were found to appear for higher spin fields. In this paper, we resolve these issues and obtain boundary representations for scalars of all masses as well as higher spin fields. We trace the origin of the previously discovered divergences and show that the smearing function becomes distributional for certain values of mass, spin and dimension. We also extend the construction from Bunch-Davies vacuum to all $\alpha$-vacua.


title: Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models

abstract: While large language models (LLMs) such as Llama-2 or GPT-4 have shown impressive zero-shot performance, fine-tuning is still necessary to enhance their performance for customized datasets, domain-specific tasks, or other private needs. However, fine-tuning all parameters of LLMs requires significant hardware resources, which can be impractical for typical users. Therefore, parameter-efficient fine-tuning such as LoRA have emerged, allowing users to fine-tune LLMs without the need for considerable computing resources, with little performance degradation compared to fine-tuning all parameters. Unfortunately, recent studies indicate that fine-tuning can increase the risk to the safety of LLMs, even when data does not contain malicious content. To address this challenge, we propose Safe LoRA, a simple one-liner patch to the original LoRA implementation by introducing the projection of LoRA weights from selected layers to the safety-aligned subspace, effectively reducing the safety risks in LLM fine-tuning while maintaining utility. It is worth noting that Safe LoRA is a training-free and data-free approach, as it only requires the knowledge of the weights from the base and aligned LLMs. Our extensive experiments demonstrate that when fine-tuning on purely malicious data, Safe LoRA retains similar safety performance as the original aligned model. Moreover, when the fine-tuning dataset contains a mixture of both benign and malicious data, Safe LoRA mitigates the negative effect made by malicious data while preserving performance on downstream tasks.


title: Speech enhancement deep-learning architecture for efficient edge processing

abstract: Deep learning has become a de facto method of choice for speech enhancement tasks with significant improvements in speech quality. However, real-time processing with reduced size and computations for low-power edge devices drastically degrades speech quality. Recently, transformer-based architectures have greatly reduced the memory requirements and provided ways to improve the model performance through local and global contexts. However, the transformer operations remain computationally heavy. In this work, we introduce WaveUNet squeeze-excitation Res2 (WSR)-based metric generative adversarial network (WSR-MGAN) architecture that can be efficiently implemented on low-power edge devices for noise suppression tasks while maintaining speech quality. We utilize multi-scale features using Res2Net blocks that can be related to spectral content used in speech-processing tasks. In the generator, we integrate squeeze-excitation blocks (SEB) with multi-scale features for maintaining local and global contexts along with gated recurrent units (GRUs). The proposed approach is optimized through a combined loss function calculated over raw waveform, multi-resolution magnitude spectrogram, and objective metrics using a metric discriminator. Experimental results in terms of various objective metrics on VoiceBank+DEMAND and DNS-2020 challenge datasets demonstrate that the proposed speech enhancement (SE) approach outperforms the baselines and achieves state-of-the-art (SOTA) performance in the time domain.


title: Superionic surface Li-ion transport in carbonaceous materials

abstract: Unlike Li-ion transport in the bulk of carbonaceous materials, little is known about Li-ion diffusion on their surface. In this study, we have discovered an ultra-fast Li-ion transport phenomenon on the surface of carbonaceous materials, particularly when they have limited Li insertion capacity along with a high surface area. This is exemplified by a carbon black, Ketjen Black (KB). An ionic conductivity of 18.1 mS cm-1 at room temperature is observed, far exceeding most solid-state ion conductors. Theoretical calculations reveal a low diffusion barrier for the surface Li species. The species is also identified as Li*, which features a partial positive charge. As a result, lithiated KB functions effectively as an interlayer between Li and solid-state electrolytes (SSE) to mitigate dendrite growth and cell shorting. This function is found to be electrolyte agnostic, effective for both sulfide and halide SSEs. Further, lithiated KB can act as a high-performance mixed ion/electron conductor that is thermodynamically stable at potentials near Li metal. A graphite anode mixed with KB instead of a solid electrolyte demonstrates full utilization with a capacity retention of ~85% over 300 cycles. The discovery of this surface-mediated ultra-fast Li-ion transport mechanism provides new directions for the design of solid-state ion conductors and solid-state batteries.


title: Enhancing Fast Feed Forward Networks with Load Balancing and a Master Leaf Node

abstract: Fast feedforward networks (FFFs) are a class of neural networks that exploit the observation that different regions of the input space activate distinct subsets of neurons in wide networks. FFFs partition the input space into separate sections using a differentiable binary tree of neurons and during inference descend the binary tree in order to improve computational efficiency. Inspired by Mixture of Experts (MoE) research, we propose the incorporation of load balancing and Master Leaf techniques into the FFF architecture to improve performance and simplify the training process. We reproduce experiments found in literature and present results on FFF models enhanced using these techniques. The proposed architecture and training recipe achieves up to 16.3% and 3% absolute classification accuracy increase in training and test accuracy, respectively, compared to the original FFF architecture. Additionally, we observe a smaller variance in the results compared to those reported in prior research. These findings demonstrate the potential of integrating MoE-inspired techniques into FFFs for developing more accurate and efficient models.


title: Enhancing Accuracy in Generative Models via Knowledge Transfer

abstract: This paper investigates the accuracy of generative models and the impact of knowledge transfer on their generation precision. Specifically, we examine a generative model for a target task, fine-tuned using a pre-trained model from a source task. Building on the "Shared Embedding" concept, which bridges the source and target tasks, we introduce a novel framework for transfer learning under distribution metrics such as the Kullback-Leibler divergence. This framework underscores the importance of leveraging inherent similarities between diverse tasks despite their distinct data distributions. Our theory suggests that the shared structures can augment the generation accuracy for a target task, reliant on the capability of a source model to identify shared structures and effective knowledge transfer from source to target learning. To demonstrate the practical utility of this framework, we explore the theoretical implications for two specific generative models: diffusion and normalizing flows. The results show enhanced performance in both models over their non-transfer counterparts, indicating advancements for diffusion models and providing fresh insights into normalizing flows in transfer and non-transfer settings. These results highlight the significant contribution of knowledge transfer in boosting the generation capabilities of these models.


title: Polytopes with low excess degree

abstract: We study the existence and structure of $d$-polytopes for which the number $f_1$ of edges is small compared to the number $f_0$ of vertices. Our results are more elegantly expressed in terms of the excess degree of the polytope, defined as $2f_1-df_0$. We show that the excess degree of a $d$-polytope cannot lie in the range $[d+3,2d-7]$, complementing the known result that values in the range $[1,d-3]$ are impossible. In particular, many pairs $(f_0,f_1)$ are not realised by any polytope. For $d$-polytopes with excess degree $d-2$, strong structural results are known; we establish comparable results for excess degrees $d$, $d+2$, and $2d-6$. Frequently, in polytopes with low excess degree, say at most $2d-6$, the nonsimple vertices all have the same degree and they form either a face or a missing face. We show that excess degree $d+1$ is possible only for $d=3,5$, or $7$, complementing the known result that an excess degree $d-1$ is possible only for $d=3$ or $5$.


title: On the Construction of Singular and Cospectral Hypergraphs

abstract: In this paper, we define two operations, neighbourhood m-splitting hypergraph $NS_m(\mathscr{G}^*)$ and non-neighbourhood splitting hypergraph $NNS(\mathscr{G}^*)$, and obtain several properties of their adjacency spectrum. We also estimate the energies of $NS_m(\mathscr{G}^*)$ and $NNS(\mathscr{G}^*)$. Moreover, we introduce two new join operations on $k$-uniform hypergraphs: the neighbourhood splitting V-vertex join $\mathscr{G}_1^*\veebar \mathscr{G}_2^*$ and the S-vertex join $\mathscr{G}_1^*\barwedge \mathscr{G}_2^*$ of hypergraphs $\mathscr{G}_1^*$ and $\mathscr{G}_2^*$, and determine their adjacency spectrum. As an application, we obtain infinite families of singular hypergraphs and infinite pairs of non-regular non-isomorphic cospectral hypergraphs.


title: Delay Performance Analysis of Delay-Deterministic Wireless Networks with Infinite and Finite Blocklength Transmission

abstract: In order to achieve stable and reliable industrial manufacturing, wireless networks must meet the stringent communication requirements of industrial automation, particularly the need for deterministic low latency communication. The limited wireless resources and time-varying fading channel contribute to the random fluctuations of transmission delay, making it challenging to realize delay-deterministic wireless networks. An open challenge in this context is to model delay determinism, also known as jitter, and analyze delay performance. In this paper, we model jitter as the variance of delay and conduct a comprehensive analysis of delay performance. Specifically, we consider two transmission regimes: infinite blocklength (IBL) and finite blocklength (FBL). In the IBL regime, the distribution of the transmission delay is analyzed, and the closed-form expressions for the average delay, jitter, and delay violation probability are derived. In the FBL regime, an upper bound on the transmission delay is first approximated at a high signalto-noise ratio. Based on this upper bound, the delay distribution, delay violation probability, average delay, and jitter are derived. Finally, simulation results are provided to validate the accuracy of the analysis and derivations. Additionally, the impact of system parameters on jitter is analyzed to gain further insights.


title: Approximation of arbitrarily high-order PDEs by first-order hyperbolic relaxation

abstract: We present a framework for constructing a first-order hyperbolic system whose solution approximates that of a desired higher-order evolution equation. Constructions of this kind have received increasing interest in recent years, and are potentially useful as either analytical or computational tools for understanding the corresponding higher-order equation. We perform a systematic analysis of a family of linear model equations and show that for each member of this family there is a stable hyperbolic approximation whose solution converges to that of the model equation in a certain limit. We then show through several examples that this approach can be applied successfully to a very wide range of nonlinear PDEs of practical interest.


title: Locality, Correlations, Information, and non-Hermitian Quantum Systems

abstract: Local non-Hermitian (NH) quantum systems generically exhibit breakdown of Lieb-Robinson (LR) bounds, motivating study of whether new locality measures might shed light not seen by existing measures. In this paper we discuss extensions of the connected correlation function (CC) as measures of locality and information spreading in both Hermitian and NH systems. We find that in Hermitian systems, $\delta\rho = \rho-\rho_A\otimes\rho_B$ can be written as a linear combination of CCs, allowing placement of a LR bound on $\Vert\delta\rho\Vert_2$, which we show generically extends to a LR bound on mutual information. Additionally, we extend the CC to NH systems in a form that recovers locality, and use the metric formalism to derive a modified CC which recovers not just locality but even LR bounds. We find that even with these CCs, the bound on $\Vert\delta\rho\Vert_2$ breaks down in certain NH cases, which can be used to place a necessary condition on which NH Hamiltonians are capable of nonlocal entanglement generation. Numerical simulations are provided by means of exact diagonalization for the NH Transverse-Field Ising Model, demonstrating both breakdown and recovery of LR bounds.

categories: quant-ph cond-mat.stat-mech math-ph math.MP


title: Non-stochastic Bandits With Evolving Observations

abstract: We introduce a novel online learning framework that unifies and generalizes pre-established models, such as delayed and corrupted feedback, to encompass adversarial environments where action feedback evolves over time. In this setting, the observed loss is arbitrary and may not correlate with the true loss incurred, with each round updating previous observations adversarially. We propose regret minimization algorithms for both the full-information and bandit settings, with regret bounds quantified by the average feedback accuracy relative to the true loss. Our algorithms match the known regret bounds across many special cases, while also introducing previously unknown bounds.


title: Heavy-hole--light-hole mixing at the zone center

abstract: We investigate heavy-hole--light-hole mixing in quasi-two-dimensional semiconducting structures. We propose and analyze several measures that characterize the mixing quantitatively in Ge, Si, and GaAs. We find that a mixing that is `weak' when considering the light-hole content can be `strong' when considering the induced spin-orbit interactions (SOIs). We identify two different types of hole-Hamiltonian terms concerning mixing. The first type changes the direction of the pure spinors, without admixing light-hole components. It is efficient for Rabi driving the heavy-hole spin. The second type induces mixing and changes the eigenvalues of the $g$ tensor. We restrict ourselves to the zone center, what leads to a simple description, analytical results, and physical insights. We show how the model can be used to investigate heavy-hole spin qubit $g$ tensor, dephasing, relaxation, and Rabi frequencies.


title: On Mesa-Optimization in Autoregressively Trained Transformers: Emergence and Capability

abstract: Autoregressively trained transformers have brought a profound revolution to the world, especially with their in-context learning (ICL) ability to address downstream tasks. Recently, several studies suggest that transformers learn a mesa-optimizer during autoregressive (AR) pretraining to implement ICL. Namely, the forward pass of the trained transformer is equivalent to optimizing an inner objective function in-context. However, whether the practical non-convex training dynamics will converge to the ideal mesa-optimizer is still unclear. Towards filling this gap, we investigate the non-convex dynamics of a one-layer linear causal self-attention model autoregressively trained by gradient flow, where the sequences are generated by an AR process $x_{t+1} = W x_t$. First, under a certain condition of data distribution, we prove that an autoregressively trained transformer learns $W$ by implementing one step of gradient descent to minimize an ordinary least squares (OLS) problem in-context. It then applies the learned $\widehat{W}$ for next-token prediction, thereby verifying the mesa-optimization hypothesis. Next, under the same data conditions, we explore the capability limitations of the obtained mesa-optimizer. We show that a stronger assumption related to the moments of data is the sufficient and necessary condition that the learned mesa-optimizer recovers the distribution. Besides, we conduct exploratory analyses beyond the first data condition and prove that generally, the trained transformer will not perform vanilla gradient descent for the OLS problem. Finally, our simulation results verify the theoretical results.


title: A generalised mid summability in Banach spaces

abstract: In this paper, we study the notion of mid summability in a general setting using the duality theory of sequence spaces. We define the vector valued sequence space $\lambda^{mid}(X)$ corresponding to a Banach space $X$ and sequence space $\lambda$. We prove that $\lambda^{mid}(\cdot)$ can be placed in a chain with the vector valued sequence spaces $\lambda^{s}(\cdot)$ and $\lambda^{w}(\cdot)$. Consequently, we define mid $\lambda$-summing operators and obtain the maximality of these operator ideals for a suitably restricted $\lambda$. Furthermore, we define a tensor norm using the vector valued sequence spaces $\lambda^{s}(\cdot)$ and $\lambda^{mid}(\cdot)$, and establish its correspondence with the operator ideal of absolutely mid $\lambda$-summing operators.


title: TokenUnify: Scalable Autoregressive Visual Pre-training with Mixture Token Prediction

abstract: Autoregressive next-token prediction is a standard pretraining method for large-scale language models, but its application to vision tasks is hindered by the non-sequential nature of image data, leading to cumulative errors. Most vision models employ masked autoencoder (MAE) based pretraining, which faces scalability issues. To address these challenges, we introduce \textbf{TokenUnify}, a novel pretraining method that integrates random token prediction, next-token prediction, and next-all token prediction. We provide theoretical evidence demonstrating that TokenUnify mitigates cumulative errors in visual autoregression. Cooperated with TokenUnify, we have assembled a large-scale electron microscopy (EM) image dataset with ultra-high resolution, ideal for creating spatially correlated long sequences. This dataset includes over 120 million annotated voxels, making it the largest neuron segmentation dataset to date and providing a unified benchmark for experimental validation. Leveraging the Mamba network inherently suited for long-sequence modeling on this dataset, TokenUnify not only reduces the computational complexity but also leads to a significant 45\% improvement in segmentation performance on downstream EM neuron segmentation tasks compared to existing methods. Furthermore, TokenUnify demonstrates superior scalability over MAE and traditional autoregressive methods, effectively bridging the gap between pretraining strategies for language and vision models. Code is available at \url{https://github.com/ydchen0806/TokenUnify}.


title: A re-calibration method for object detection with multi-modal alignment bias in autonomous driving

abstract: Multi-modal object detection in autonomous driving has achieved great breakthroughs due to the usage of fusing complementary information from different sensors. The calibration in fusion between sensors such as LiDAR and camera is always supposed to be precise in previous work. However, in reality, calibration matrices are fixed when the vehicles leave the factory, but vibration, bumps, and data lags may cause calibration bias. As the research on the calibration influence on fusion detection performance is relatively few, flexible calibration dependency multi-sensor detection method has always been attractive. In this paper, we conducted experiments on SOTA detection method EPNet++ and proved slight bias on calibration can reduce the performance seriously. We also proposed a re-calibration model based on semantic segmentation which can be combined with a detection algorithm to improve the performance and robustness of multi-modal calibration bias.


title: Sync4D: Video Guided Controllable Dynamics for Physics-Based 4D Generation

abstract: In this work, we introduce a novel approach for creating controllable dynamics in 3D-generated Gaussians using casually captured reference videos. Our method transfers the motion of objects from reference videos to a variety of generated 3D Gaussians across different categories, ensuring precise and customizable motion transfer. We achieve this by employing blend skinning-based non-parametric shape reconstruction to extract the shape and motion of reference objects. This process involves segmenting the reference objects into motion-related parts based on skinning weights and establishing shape correspondences with generated target shapes. To address shape and temporal inconsistencies prevalent in existing methods, we integrate physical simulation, driving the target shapes with matched motion. This integration is optimized through a displacement loss to ensure reliable and genuine dynamics. Our approach supports diverse reference inputs, including humans, quadrupeds, and articulated objects, and can generate dynamics of arbitrary length, providing enhanced fidelity and applicability. Unlike methods heavily reliant on diffusion video generation models, our technique offers specific and high-quality motion transfer, maintaining both shape integrity and temporal consistency.


title: UniCompress: Enhancing Multi-Data Medical Image Compression with Knowledge Distillation

abstract: In the field of medical image compression, Implicit Neural Representation (INR) networks have shown remarkable versatility due to their flexible compression ratios, yet they are constrained by a one-to-one fitting approach that results in lengthy encoding times. Our novel method, ``\textbf{UniCompress}'', innovatively extends the compression capabilities of INR by being the first to compress multiple medical data blocks using a single INR network. By employing wavelet transforms and quantization, we introduce a codebook containing frequency domain information as a prior input to the INR network. This enhances the representational power of INR and provides distinctive conditioning for different image blocks. Furthermore, our research introduces a new technique for the knowledge distillation of implicit representations, simplifying complex model knowledge into more manageable formats to improve compression ratios. Extensive testing on CT and electron microscopy (EM) datasets has demonstrated that UniCompress outperforms traditional INR methods and commercial compression solutions like HEVC, especially in complex and high compression scenarios. Notably, compared to existing INR techniques, UniCompress achieves a 4$\sim$5 times increase in compression speed, marking a significant advancement in the field of medical image compression. Codes will be publicly available.


title: Temporal Spiking Neural Networks with Synaptic Delay for Graph Reasoning

abstract: Spiking neural networks (SNNs) are investigated as biologically inspired models of neural computation, distinguished by their computational capability and energy efficiency due to precise spiking times and sparse spikes with event-driven computation. A significant question is how SNNs can emulate human-like graph-based reasoning of concepts and relations, especially leveraging the temporal domain optimally. This paper reveals that SNNs, when amalgamated with synaptic delay and temporal coding, are proficient in executing (knowledge) graph reasoning. It is elucidated that spiking time can function as an additional dimension to encode relation properties via a neural-generalized path formulation. Empirical results highlight the efficacy of temporal delay in relation processing and showcase exemplary performance in diverse graph reasoning tasks. The spiking model is theoretically estimated to achieve $20\times$ energy savings compared to non-spiking counterparts, deepening insights into the capabilities and potential of biologically inspired SNNs for efficient reasoning. The code is available at https://github.com/pkuxmq/GRSNN.


title: EM Distillation for One-step Diffusion Models

abstract: While diffusion models can learn complex distributions, sampling requires a computationally expensive iterative process. Existing distillation methods enable efficient sampling, but have notable limitations, such as performance degradation with very few sampling steps, reliance on training data access, or mode-seeking optimization that may fail to capture the full distribution. We propose EM Distillation (EMD), a maximum likelihood-based approach that distills a diffusion model to a one-step generator model with minimal loss of perceptual quality. Our approach is derived through the lens of Expectation-Maximization (EM), where the generator parameters are updated using samples from the joint distribution of the diffusion teacher prior and inferred generator latents. We develop a reparametrized sampling scheme and a noise cancellation technique that together stabilizes the distillation process. We further reveal an interesting connection of our method with existing methods that minimize mode-seeking KL. EMD outperforms existing one-step generative methods in terms of FID scores on ImageNet-64 and ImageNet-128, and compares favorably with prior work on distilling text-to-image diffusion models.


title: Color Dependence of Dipole in CatWISE2020 Data

abstract: The signal of dipole anisotropy in quasar number counts is studied using the CatWISE2020 catalog in various color bins. It is found that the dipole signal differs significantly in two color bins, namely, $1.1>W1-W2\ge 0.8$ and $1.4>W1-W2>1.1$. The color bin $1.4>W1-W2>1.1$ appears strongly contaminated, with possibly Galactic contributions and is unreliable for extracting the signal of cosmological dipole. The source of this contamination has not been identified and cannot be attributed to known emissions within the galaxy. Removing this contaminated color bin leads to a strong dipole signal with a direction significantly different from that obtained from full data. If we interpret this dipole as due to our local motion, the extracted velocity turns out to be $900\pm 113$ Kms$^{-1}$, which deviates from the CMB dipole velocity with approximately $4.7$ sigma significance.


title: Knowing What Not to Do: Leverage Language Model Insights for Action Space Pruning in Multi-agent Reinforcement Learning

abstract: Multi-agent reinforcement learning (MARL) is employed to develop autonomous agents that can learn to adopt cooperative or competitive strategies within complex environments. However, the linear increase in the number of agents leads to a combinatorial explosion of the action space, which may result in algorithmic instability, difficulty in convergence, or entrapment in local optima. While researchers have designed a variety of effective algorithms to compress the action space, these methods also introduce new challenges, such as the need for manually designed prior knowledge or reliance on the structure of the problem, which diminishes the applicability of these techniques. In this paper, we introduce Evolutionary action SPAce Reduction with Knowledge (eSpark), an exploration function generation framework driven by large language models (LLMs) to boost exploration and prune unnecessary actions in MARL. Using just a basic prompt that outlines the overall task and setting, eSpark is capable of generating exploration functions in a zero-shot manner, identifying and pruning redundant or irrelevant state-action pairs, and then achieving autonomous improvement from policy feedback. In reinforcement learning tasks involving inventory management and traffic light control encompassing a total of 15 scenarios, eSpark consistently outperforms the combined MARL algorithm in all scenarios, achieving an average performance gain of 34.4% and 9.9% in the two types of tasks respectively. Additionally, eSpark has proven to be capable of managing situations with a large number of agents, securing a 29.7% improvement in scalability challenges that featured over 500 agents. The code can be found in https://github.com/LiuZhihao2022/eSpark.git.


title: Maximal operators given by Fourier multipliers with dilation of fractional dimensions

abstract: In this paper, we investigate $L^p$ bounds of maximal Fourier multiplier operators with dilation of fractional dimensions. For the Fourier multipliers, we suggest a criterion related to dimensions of dilation sets which guarantees $L^p$ bounds of the maximal operators for each $p$. Our criterion covers Mikhlin-type multipliers, multipliers with limited decay, and multipliers with slow decay.


title: Can We Trust LLMs? Mitigate Overconfidence Bias in LLMs through Knowledge Transfer

abstract: The study explores mitigating overconfidence bias in LLMs to improve their reliability. We introduce a knowledge transfer (KT) method utilizing chain of thoughts, where "big" LLMs impart knowledge to "small" LLMs via detailed, sequential reasoning paths. This method uses advanced reasoning of larger models to fine-tune smaller models, enabling them to produce more accurate predictions with calibrated confidence. Experimental evaluation using multiple-choice questions and sentiment analysis across diverse datasets demonstrated the KT method's superiority over the vanilla and question-answer pair (QA) fine-tuning methods. The most significant improvement in three key metrics, where the KT method outperformed the vanilla and QA methods by an average of 55.3% and 43.1%, respectively. These findings underscore the KT method's potential in enhancing model trustworthiness and accuracy, offering precise outputs with well-matched confidence levels across various contexts.


title: KMT-2023-BLG-2669: Ninth Free-floating Planet Candidate with $\theta_{\rm E}$ measurements

abstract: We report a free-floating planet (FFP) candidate identified from the analysis of the microlensing event KMT-2023-BLG-2669. The lensing light curve is characterized by a short duration $(\lesssim 3\,{\rm days})$ and a small amplitude $(\lesssim 0.7\,{\rm mag})$. From the analysis, we find the Einstein timescale of $t_{\rm E} \backsimeq 0.33\,{\rm days}$ and the Einstein radius of $\theta_{\rm E} \backsimeq 4.41\,{\mu}{\rm as}$. These measurements enable us to infer the lens mass as $M = 8\,M_{\oplus} (\pi_{\rm rel} / 0.1\,{\rm mas})^{-1}$, where $\pi_{\rm rel}$ is the relative lens-source parallax. The inference implies that the lens is a sub-Neptune- to Saturn-mass object depending on its unknown distance. This is the ninth isolated planetary-mass microlens with $\theta_{\rm E} < 10\,{\mu}{\rm as}$, which (as shown by \citealt{gould22}) is a useful threshold for a FFP candidate. We conduct extensive searches for possible signals of a host star in the light curve, but find no strong evidence for the host. We discuss the possibility of using late-time high-resolution imaging to probe for possible hosts.


title: Estimating Depth of Monocular Panoramic Image with Teacher-Student Model Fusing Equirectangular and Spherical Representations

abstract: Disconnectivity and distortion are the two problems which must be coped with when processing 360 degrees equirectangular images. In this paper, we propose a method of estimating the depth of monocular panoramic image with a teacher-student model fusing equirectangular and spherical representations. In contrast with the existing methods fusing an equirectangular representation with a cube map representation or tangent representation, a spherical representation is a better choice because a sampling on a sphere is more uniform and can also cope with distortion more effectively. In this processing, a novel spherical convolution kernel computing with sampling points on a sphere is developed to extract features from the spherical representation, and then, a Segmentation Feature Fusion(SFF) methodology is utilized to combine the features with ones extracted from the equirectangular representation. In contrast with the existing methods using a teacher-student model to obtain a lighter model of depth estimation, we use a teacher-student model to learn the latent features of depth images. This results in a trained model which estimates the depth map of an equirectangular image using not only the feature maps extracted from an input equirectangular image but also the distilled knowledge learnt from the ground truth of depth map of a training set. In experiments, the proposed method is tested on several well-known 360 monocular depth estimation benchmark datasets, and outperforms the existing methods for the most evaluation indexes.


title: Gaussian Mixture Model with Rare Events

abstract: We study here a Gaussian Mixture Model (GMM) with rare events data. In this case, the commonly used Expectation-Maximization (EM) algorithm exhibits extremely slow numerical convergence rate. To theoretically understand this phenomenon, we formulate the numerical convergence problem of the EM algorithm with rare events data as a problem about a contraction operator. Theoretical analysis reveals that the spectral radius of the contraction operator in this case could be arbitrarily close to 1 asymptotically. This theoretical finding explains the empirical slow numerical convergence of the EM algorithm with rare events data. To overcome this challenge, a Mixed EM (MEM) algorithm is developed, which utilizes the information provided by partially labeled data. As compared with the standard EM algorithm, the key feature of the MEM algorithm is that it requires additionally labeled data. We find that MEM algorithm significantly improves the numerical convergence rate as compared with the standard EM algorithm. The finite sample performance of the proposed method is illustrated by both simulation studies and a real-world dataset of Swedish traffic signs.


title: Think Before You Act: A Two-Stage Framework for Mitigating Gender Bias Towards Vision-Language Tasks

abstract: Gender bias in vision-language models (VLMs) can reinforce harmful stereotypes and discrimination. In this paper, we focus on mitigating gender bias towards vision-language tasks. We identify object hallucination as the essence of gender bias in VLMs. Existing VLMs tend to focus on salient or familiar attributes in images but ignore contextualized nuances. Moreover, most VLMs rely on the co-occurrence between specific objects and gender attributes to infer the ignored features, ultimately resulting in gender bias. We propose GAMA, a task-agnostic generation framework to mitigate gender bias. GAMA consists of two stages: narrative generation and answer inference. During narrative generation, GAMA yields all-sided but gender-obfuscated narratives, which prevents premature concentration on localized image features, especially gender attributes. During answer inference, GAMA integrates the image, generated narrative, and a task-specific question prompt to infer answers for different vision-language tasks. This approach allows the model to rethink gender attributes and answers. We conduct extensive experiments on GAMA, demonstrating its debiasing and generalization ability.


title: NCIDiff: Non-covalent Interaction-generative Diffusion Model for Improving Reliability of 3D Molecule Generation Inside Protein Pocket

abstract: Advancements in deep generative modeling have changed the paradigm of drug discovery. Among such approaches, target-aware methods that exploit 3D structures of protein pockets were spotlighted for generating ligand molecules with their plausible binding modes. While docking scores superficially assess the quality of generated ligands, closer inspection of the binding structures reveals the inconsistency in local interactions between a pocket and generated ligands. Here, we address the issue by explicitly generating non-covalent interactions (NCIs), which are universal patterns throughout protein-ligand complexes. Our proposed model, NCIDiff, simultaneously denoises NCI types of protein-ligand edges along with a 3D graph of a ligand molecule during the sampling. With the NCI-generating strategy, our model generates ligands with more reliable NCIs, especially outperforming the baseline diffusion-based models. We further adopted inpainting techniques on NCIs to further improve the quality of the generated molecules. Finally, we showcase the applicability of NCIDiff on drug design tasks for real-world settings with specialized objectives by guiding the generation process with desired NCI patterns.


title: Barrow entropy and spacetime foam

abstract: Quantum gravitational effects, on the one hand, lead to a limitation in the accuracy of measuring spatial and time intervals, and, on the other hand, they generate a discrete of spacetime structure (quantum foam). The common source of both measurement limitations and discreteness of space-time are quantum fluctuations, so their characteristics must be related to each other. We study such a relationship using Barrow entropy as a source of fractal space-time structure. The minimum inaccuracy in measuring space-time intervals is expressed through the Barrow entropy parameter. The connection between the level of fractality and the speed of information processing is considered.


title: All-voltage control of Giant Magnetoresistance

abstract: The aim of voltage control of magnetism is to reduce the power consumption of spintronic devices. For a spin valve, the magnetization directions of two ferromagnetic layers determine the giant magnetoresistance magnitude. However, achieving all-voltage manipulation of the magnetization directions between parallel and antiparallel states is a significant challenge. Here, we demonstrate that by utilizing two exchange-biased Co/IrMn bilayers with opposite pinning directions and with ferromagnetic coupling through the Ruderman-Kittel-Kasuya-Yosida interaction between two Co layers, the magnetization directions of the two ferromagnetic layers of a spin valve can be switched between parallel and antiparallel states through allvoltage-induced strain control. The all-voltage controlled giant magnetoresistance is repeatable and nonvolatile. The rotation of magnetizations in the two Co layers under voltages, from antiparallel to parallel states, occurs in opposite directions as revealed through simulations utilizing the Landau-Lifshitz-Gilbert equation. This work can provide valuable reference for the development of low-power all-voltage-controlled spintronic devices.


title: Sparsity comparison of polytopal finite element methods

abstract: In this work we compare crucial parameters for efficiency of different finite element methods for solving partial differential equations (PDEs) on polytopal meshes. We consider the Virtual Element Method (VEM) and different Discontinuous Galerkin (DG) methods, namely the Hybrid DG and Trefftz DG methods. The VEM is a conforming method, that can be seen as a generalization of the classic finite element method to arbitrary polytopal meshes. DG methods are non-conforming methods that offer high flexibility, but also come with high computational costs. Hybridization reduces these costs by introducing additional facet variables, onto which the computational costs can be transfered to. Trefftz DG methods achieve a similar reduction in complexity by selecting a special and smaller set of basis functions on each element. The association of computational costs to different geometrical entities (elements or facets) leads to differences in the performance of these methods on different grid types. This paper aims to compare the dependency of these approaches across different grid configurations.


title: An Investigation of Conformal Isometry Hypothesis for Grid Cells

abstract: This paper investigates the conformal isometry hypothesis as a potential explanation for the emergence of hexagonal periodic patterns in the response maps of grid cells. The hypothesis posits that the activities of the population of grid cells form a high-dimensional vector in the neural space, representing the agent's self-position in 2D physical space. As the agent moves in the 2D physical space, the vector rotates in a 2D manifold in the neural space, driven by a recurrent neural network. The conformal isometry hypothesis proposes that this 2D manifold in the neural space is a conformally isometric embedding of the 2D physical space, in the sense that local displacements of the vector in neural space are proportional to local displacements of the agent in the physical space. Thus the 2D manifold forms an internal map of the 2D physical space, equipped with an internal metric. In this paper, we conduct numerical experiments to show that this hypothesis underlies the hexagon periodic patterns of grid cells. We also conduct theoretical analysis to further support this hypothesis. In addition, we propose a conformal modulation of the input velocity of the agent so that the recurrent neural network of grid cells satisfies the conformal isometry hypothesis automatically. To summarize, our work provides numerical and theoretical evidences for the conformal isometry hypothesis for grid cells and may serve as a foundation for further development of normative models of grid cells and beyond.


title: Hierarchical Rank-One Sequence Convexification for the Relaxation of Variational Problems with Microstructures

abstract: This paper presents an efficient algorithm for the approximation of the rank-one convex hull in the context of nonlinear solid mechanics. It is based on hierarchical rank-one sequences and simultaneously provides first and second derivative information essential for the calculation of mechanical stresses and the computational minimization of discretized energies. For materials, whose microstructure can be well approximated in terms of laminates and where each laminate stage achieves energetic optimality with respect to the current stage, the approximate envelope coincides with the rank-one convex envelope. Although the proposed method provides only an upper bound for the rank-one convex hull, a careful examination of the resulting constraints shows a decent applicability in mechanical problems. Various aspects of the algorithm are discussed, including the restoration of rotational invariance, microstructure reconstruction, comparisons with other semi-convexification algorithms, and mesh independency. Overall, this paper demonstrates the efficiency of the algorithm for both, well-established mathematical benchmark problems as well as nonconvex isotropic finite-strain continuum damage models in two and three dimensions. Thereby, for the first time, a feasible concurrent numerical relaxation is established for an incremental, dissipative large-strain model with relevant applications in engineering problems.


title: Clustering-based Learning for UAV Tracking and Pose Estimation

abstract: UAV tracking and pose estimation plays an imperative role in various UAV-related missions, such as formation control and anti-UAV measures. Accurately detecting and tracking UAVs in a 3D space remains a particularly challenging problem, as it requires extracting sparse features of micro UAVs from different flight environments and continuously matching correspondences, especially during agile flight. Generally, cameras and LiDARs are the two main types of sensors used to capture UAV trajectories in flight. However, both sensors have limitations in UAV classification and pose estimation. This technical report briefly introduces the method proposed by our team "NTU-ICG" for the CVPR 2024 UG2+ Challenge Track 5. This work develops a clustering-based learning detection approach, CL-Det, for UAV tracking and pose estimation using two types of LiDARs, namely Livox Avia and LiDAR 360. We combine the information from the two data sources to locate drones in 3D. We first align the timestamps of Livox Avia data and LiDAR 360 data and then separate the point cloud of objects of interest (OOIs) from the environment. The point cloud of OOIs is clustered using the DBSCAN method, with the midpoint of the largest cluster assumed to be the UAV position. Furthermore, we utilize historical estimations to fill in missing data. The proposed method shows competitive pose estimation performance and ranks 5th on the final leaderboard of the CVPR 2024 UG2+ Challenge.


title: RCDN: Towards Robust Camera-Insensitivity Collaborative Perception via Dynamic Feature-based 3D Neural Modeling

abstract: Collaborative perception is dedicated to tackling the constraints of single-agent perception, such as occlusions, based on the multiple agents' multi-view sensor inputs. However, most existing works assume an ideal condition that all agents' multi-view cameras are continuously available. In reality, cameras may be highly noisy, obscured or even failed during the collaboration. In this work, we introduce a new robust camera-insensitivity problem: how to overcome the issues caused by the failed camera perspectives, while stabilizing high collaborative performance with low calibration cost? To address above problems, we propose RCDN, a Robust Camera-insensitivity collaborative perception with a novel Dynamic feature-based 3D Neural modeling mechanism. The key intuition of RCDN is to construct collaborative neural rendering field representations to recover failed perceptual messages sent by multiple agents. To better model collaborative neural rendering field, RCDN first establishes a geometry BEV feature based time-invariant static field with other agents via fast hash grid modeling. Based on the static background field, the proposed time-varying dynamic field can model corresponding motion vectors for foregrounds with appropriate positions. To validate RCDN, we create OPV2V-N, a new large-scale dataset with manual labelling under different camera failed scenarios. Extensive experiments conducted on OPV2V-N show that RCDN can be ported to other baselines and improve their robustness in extreme camera-insensitivity settings. Our code and datasets will be available soon.


title: Mixture of Modality Knowledge Experts for Robust Multi-modal Knowledge Graph Completion

abstract: Multi-modal knowledge graph completion (MMKGC) aims to automatically discover new knowledge triples in the given multi-modal knowledge graphs (MMKGs), which is achieved by collaborative modeling the structural information concealed in massive triples and the multi-modal features of the entities. Existing methods tend to focus on crafting elegant entity-wise multi-modal fusion strategies, yet they overlook the utilization of multi-perspective features concealed within the modalities under diverse relational contexts. To address this issue, we introduce a novel MMKGC framework with Mixture of Modality Knowledge experts (MoMoK for short) to learn adaptive multi-modal embedding under intricate relational contexts. We design relation-guided modality knowledge experts to acquire relation-aware modality embeddings and integrate the predictions from multi-modalities to achieve comprehensive decisions. Additionally, we disentangle the experts by minimizing their mutual information. Experiments on four public MMKG benchmarks demonstrate the outstanding performance of MoMoK under complex scenarios.


title: Active gel model for one-dimensional cell migration coupling actin flow and adhesion dynamics

abstract: Migration of animal cells is based on the interplay between actin polymerization at the front, adhesion along the cell-substrate interface, and actomyosin contractility at the back. Active gel theory has been used before to demonstrate that actomyosin contractility is sufficient for polarization and self-sustained cell migration in the absence of external cues, but did not consider the dynamics of adhesion. Likewise, migration models based on the mechanosensitive dynamics of adhesion receptors usually do not include the global dynamics of intracellular flow. Here we show that both aspects can be combined in a minimal active gel model for one-dimensional cell migration with dynamic adhesion. This model demonstrates that load sharing between the adhesion receptors leads to symmetry breaking, with stronger adhesion at the front, and that bistability of migration arises for intermediate adhesiveness. Local variations in adhesiveness are sufficient to switch between sessile and motile states, in qualitative agreement with experiments.


title: Multi-Behavior Generative Recommendation

abstract: Multi-behavior sequential recommendation (MBSR) aims to incorporate behavior types of interactions for better recommendations. Existing approaches focus on the next-item prediction objective, neglecting the value of integrating the target behavior type into the learning objective. In this paper, we propose MBGen, a novel Multi-Behavior sequential Generative recommendation framework. We formulate the MBSR task into a consecutive two-step process: (1) given item sequences, MBGen first predicts the next behavior type to frame the user intention, (2) given item sequences and a target behavior type, MBGen then predicts the next items. To model such a two-step process, we tokenize both behaviors and items into tokens and construct one single token sequence with both behaviors and items placed interleaved. Furthermore, MBGen learns to autoregressively generate the next behavior and item tokens in a unified generative recommendation paradigm, naturally enabling a multi-task capability. Additionally, we exploit the heterogeneous nature of token sequences in the generative recommendation and propose a position-routed sparse architecture to efficiently and effectively scale up models. Extensive experiments on public datasets demonstrate that MBGen significantly outperforms existing MBSR models across multiple tasks.


title: On local character expansions for principal series representations of general linear groups

abstract: We obtain two explicit formulas for the full local character expansion of any irreducible representation of a p-adic general linear group in principal blocks. The first, generalizing previous work of the author on the Iwahori-spherical case, expresses the expansion in terms of dimensions of degenerate Whittaker models. The second gives a closed expression in terms of values of Kazhdan-Lusztig polynomials of a suitable permutation group.


title: ContrastAlign: Toward Robust BEV Feature Alignment via Contrastive Learning for Multi-Modal 3D Object Detection

abstract: In the field of 3D object detection tasks, fusing heterogeneous features from LiDAR and camera sensors into a unified Bird's Eye View (BEV) representation is a widely adopted paradigm. However, existing methods are often compromised by imprecise sensor calibration, resulting in feature misalignment in LiDAR-camera BEV fusion. Moreover, such inaccuracies result in errors in depth estimation for the camera branch, ultimately causing misalignment between LiDAR and camera BEV features. In this work, we propose a novel ContrastAlign approach that utilizes contrastive learning to enhance the alignment of heterogeneous modalities, thereby improving the robustness of the fusion process. Specifically, our approach includes the L-Instance module, which directly outputs LiDAR instance features within LiDAR BEV features. Then, we introduce the C-Instance module, which predicts camera instance features through RoI (Region of Interest) pooling on the camera BEV features. We propose the InstanceFusion module, which utilizes contrastive learning to generate similar instance features across heterogeneous modalities. We then use graph matching to calculate the similarity between the neighboring camera instance features and the similarity instance features to complete the alignment of instance features. Our method achieves state-of-the-art performance, with an mAP of 70.3%, surpassing BEVFusion by 1.8% on the nuScenes validation set. Importantly, our method outperforms BEVFusion by 7.3% under conditions with misalignment noise.


title: CoCoGesture: Toward Coherent Co-speech 3D Gesture Generation in the Wild

abstract: Deriving co-speech 3D gestures has seen tremendous progress in virtual avatar animation. Yet, the existing methods often produce stiff and unreasonable gestures with unseen human speech inputs due to the limited 3D speech-gesture data. In this paper, we propose CoCoGesture, a novel framework enabling vivid and diverse gesture synthesis from unseen human speech prompts. Our key insight is built upon the custom-designed pretrain-fintune training paradigm. At the pretraining stage, we aim to formulate a large generalizable gesture diffusion model by learning the abundant postures manifold. Therefore, to alleviate the scarcity of 3D data, we first construct a large-scale co-speech 3D gesture dataset containing more than 40M meshed posture instances across 4.3K speakers, dubbed GES-X. Then, we scale up the large unconditional diffusion model to 1B parameters and pre-train it to be our gesture experts. At the finetune stage, we present the audio ControlNet that incorporates the human voice as condition prompts to guide the gesture generation. Here, we construct the audio ControlNet through a trainable copy of our pre-trained diffusion model. Moreover, we design a novel Mixture-of-Gesture-Experts (MoGE) block to adaptively fuse the audio embedding from the human speech and the gesture features from the pre-trained gesture experts with a routing mechanism. Such an effective manner ensures audio embedding is temporal coordinated with motion features while preserving the vivid and diverse gesture generation. Extensive experiments demonstrate that our proposed CoCoGesture outperforms the state-of-the-art methods on the zero-shot speech-to-gesture generation. The dataset will be publicly available at: https://mattie-e.github.io/GES-X/


title: Digitalization in Infrastructure Construction Projects: A PRISMA-Based Review of Benefits and Obstacles

abstract: The current study presents a comprehensive review of the benefits and barriers associated with the adoption of Building Information Modeling (BIM) in infrastructure projects, focusing on the period from 2013 to 2023. The research explores the manifold advantages offered by BIM, spanning the entire project life cycle, including planning, design, construction, maintenance, and sustainability. Notably, BIM enhances collaboration, facilitates real-time data-driven decision-making, and leads to substantial cost and time savings. In parallel, a systematic literature review was conducted to identify and categorize the barriers hindering BIM adoption within the infrastructure industry. Eleven studies were selected for in-depth analysis, revealing a total of 74 obstacles. Through synthetic analysis and thematic clustering, seven primary impediments to BIM adoption were identified, encompassing challenges related to education/training, resistance to change, business value clarity, perceived cost, lack of standards and guidelines, lack of mandates, and lack of initiatives. This review explores the benefits and barriers in the industry that are facing BIM adoption in infrastructure projects, giving an important perspective toward improving effective BIM adoption strategies, policies, and standards. Future directions for research and industry development are outlined, including efforts to enhance education and training, promote standardization, advocate for policy and mandates, and integrate BIM with emerging technologies.