title: Detect2Interact: Localizing Object Key Field in Visual Question Answering (VQA) with LLMs

abstract: Localization plays a crucial role in enhancing the practicality and precision of VQA systems. By enabling fine-grained identification and interaction with specific parts of an object, it significantly improves the system's ability to provide contextually relevant and spatially accurate responses, crucial for applications in dynamic environments like robotics and augmented reality. However, traditional systems face challenges in accurately mapping objects within images to generate nuanced and spatially aware responses. In this work, we introduce "Detect2Interact", which addresses these challenges by introducing an advanced approach for fine-grained object visual key field detection. First, we use the segment anything model (SAM) to generate detailed spatial maps of objects in images. Next, we use Vision Studio to extract semantic object descriptions. Third, we employ GPT-4's common sense knowledge, bridging the gap between an object's semantics and its spatial map. As a result, Detect2Interact achieves consistent qualitative results on object key field detection across extensive test cases and outperforms the existing VQA system with object detection by providing a more reasonable and finer visual representation.


title: Applications of the Stone-Weierstrass theorem in the Calder\'on problem

abstract: We give examples on the use of the Stone-Weierstrass theorem in inverse problems. We show uniqueness in the linearized Calder\'on problem on holomorphically separable K\"ahler manifolds, and in the Calder\'on problem for nonlinear equations on conformally transversally anisotropic manifolds. We also study the holomorphic separability condition in terms of plurisubharmonic functions. The Stone-Weierstrass theorem allows us to generalize and simplify earlier results. It also makes it possible to circumvent the use of complex geometrical optics solutions and inversion of explicit transforms in certain cases.


title: TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regression

abstract: The main challenge that sets transfer learning apart from traditional supervised learning is the distribution shift, reflected as the shift between the source and target models and that between the marginal covariate distributions. In this work, we tackle model shifts in the presence of covariate shifts in the high-dimensional regression setting. Specifically, we propose a two-step method with a novel fused-regularizer that effectively leverages samples from source tasks to improve the learning performance on a target task with limited samples. Nonasymptotic bound is provided for the estimation error of the target model, showing the robustness of the proposed method to covariate shifts. We further establish conditions under which the estimator is minimax-optimal. Additionally, we extend the method to a distributed setting, allowing for a pretraining-finetuning strategy, requiring just one round of communication while retaining the estimation rate of the centralized version. Numerical tests validate our theory, highlighting the method's robustness to covariate shifts.

categories: stat.ML cs.DC cs.LG math.ST stat.ME stat.TH


title: Uncovering the Text Embedding in Text-to-Image Diffusion Models

abstract: The correspondence between input text and the generated image exhibits opacity, wherein minor textual modifications can induce substantial deviations in the generated image. While, text embedding, as the pivotal intermediary between text and images, remains relatively underexplored. In this paper, we address this research gap by delving into the text embedding space, unleashing its capacity for controllable image editing and explicable semantic direction attributes within a learning-free framework. Specifically, we identify two critical insights regarding the importance of per-word embedding and their contextual correlations within text embedding, providing instructive principles for learning-free image editing. Additionally, we find that text embedding inherently possesses diverse semantic potentials, and further reveal this property through the lens of singular value decomposition (SVD). These uncovered properties offer practical utility for image editing and semantic discovery. More importantly, we expect the in-depth analyses and findings of the text embedding can enhance the understanding of text-to-image diffusion models.


title: Dynamic Modeling and Stability Analysis for Repeated LVRT Process of Wind Turbine Based on Switched System Theory

abstract: The significant electrical distance between wind power collection points and the main grid poses challenges for weak grid-connected wind power systems. A new type of voltage oscillation phenomenon induced by repeated low voltage ride-through (LVRT) of the wind turbine has been observed, threatening the safe and stable operation of such power systems. Therefore, exploring dynamic evolution mechanisms and developing stability analysis approaches for this phenomenon have become pressing imperatives. This paper introduces switched system theory for dynamic modeling, mechanism elucidation, and stability analysis of the repeated LVRT process. Firstly, considering the external connection impedance and internal control dynamics, a novel wind turbine grid-side converter (WT-GSC) switched system model is established to quantitatively characterize the evolution dynamic and mechanism of the voltage oscillation. Subsequently, a sufficient stability criterion and index grounded in the common Lyapunov function are proposed for stability analysis and assessment of the WT-GSC switched system. Moreover, to enhance the system stability, the Sobol' global sensitivity analysis method is adopted to identify dominant parameters, which can be further optimized via the particle swarm optimization (PSO) algorithm. Finally, simulations conducted on a modified IEEE 39-bus test system verify the effectiveness of the proposed dynamic modeling and stability analysis methods.


title: SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining

abstract: Vision-language models (VLMs) have made significant strides in cross-modal understanding through large-scale paired datasets. However, in fashion domain, datasets often exhibit a disparity between the information conveyed in image and text. This issue stems from datasets containing multiple images of a single fashion item all paired with one text, leading to cases where some textual details are not visible in individual images. This mismatch, particularly when non-co-occurring elements are masked, undermines the training of conventional VLM objectives like Masked Language Modeling and Masked Image Modeling, thereby hindering the model's ability to accurately align fine-grained visual and textual features. Addressing this problem, we propose Synchronized attentional Masking (SyncMask), which generate masks that pinpoint the image patches and word tokens where the information co-occur in both image and text. This synchronization is accomplished by harnessing cross-attentional features obtained from a momentum model, ensuring a precise alignment between the two modalities. Additionally, we enhance grouped batch sampling with semi-hard negatives, effectively mitigating false negative issues in Image-Text Matching and Image-Text Contrastive learning objectives within fashion datasets. Our experiments demonstrate the effectiveness of the proposed approach, outperforming existing methods in three downstream tasks.


title: Green AI: Exploring Carbon Footprints, Mitigation Strategies, and Trade Offs in Large Language Model Training

abstract: Prominent works in the field of Natural Language Processing have long attempted to create new innovative models by improving upon previous model training approaches, altering model architecture, and developing more in-depth datasets to better their performance. However, with the quickly advancing field of NLP comes increased greenhouse gas emissions, posing concerns over the environmental damage caused by training LLMs. Gaining a comprehensive understanding of the various costs, particularly those pertaining to environmental aspects, that are associated with artificial intelligence serves as the foundational basis for ensuring safe AI models. Currently, investigations into the CO2 emissions of AI models remain an emerging area of research, and as such, in this paper, we evaluate the CO2 emissions of well-known large language models, which have an especially high carbon footprint due to their significant amount of model parameters. We argue for the training of LLMs in a way that is responsible and sustainable by suggesting measures for reducing carbon emissions. Furthermore, we discuss how the choice of hardware affects CO2 emissions by contrasting the CO2 emissions during model training for two widely used GPUs. Based on our results, we present the benefits and drawbacks of our proposed solutions and make the argument for the possibility of training more environmentally safe AI models without sacrificing their robustness and performance.


title: Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community

abstract: The ability to interact with machines using natural human language is becoming not just commonplace, but expected. The next step is not just text interfaces, but speech interfaces and not just with computers, but with all machines including robots. In this paper, we chronicle the recent history of this growing field of spoken dialogue with robots and offer the community three proposals, the first focused on education, the second on benchmarks, and the third on the modeling of language when it comes to spoken interaction with robots. The three proposals should act as white papers for any researcher to take and build upon.


title: GPU-accelerated Evolutionary Multiobjective Optimization Using Tensorized RVEA

abstract: Evolutionary multiobjective optimization has witnessed remarkable progress during the past decades. However, existing algorithms often encounter computational challenges in large-scale scenarios, primarily attributed to the absence of hardware acceleration. In response, we introduce a Tensorized Reference Vector Guided Evolutionary Algorithm (TensorRVEA) for harnessing the advancements of GPU acceleration. In TensorRVEA, the key data structures and operators are fully transformed into tensor forms for leveraging GPU-based parallel computing. In numerical benchmark tests involving large-scale populations and problem dimensions, TensorRVEA consistently demonstrates high computational performance, achieving up to over 1000$\times$ speedups. Then, we applied TensorRVEA to the domain of multiobjective neuroevolution for addressing complex challenges in robotic control tasks. Furthermore, we assessed TensorRVEA's extensibility by altering several tensorized reproduction operators. Experimental results demonstrate promising scalability and robustness of TensorRVEA. Source codes are available at \url{https://github.com/EMI-Group/tensorrvea}.


title: Diagnosis of Skin Cancer Using VGG16 and VGG19 Based Transfer Learning Models

abstract: Today, skin cancer is considered as one of the most dangerous and common cancers in the world which demands special attention. Skin cancer may be developed in different types; including melanoma, actinic keratosis, basal cell carcinoma, squamous cell carcinoma, and Merkel cell carcinoma. Among them, melanoma is more unpredictable. Melanoma cancer can be diagnosed at early stages increasing the possibility of disease treatment. Automatic classification of skin lesions is a challenging task due to diverse forms and grades of the disease, demanding the requirement of novel methods implementation. Deep convolution neural networks (CNN) have shown an excellent potential for data and image classification. In this article, we inspect skin lesion classification problem using CNN techniques. Remarkably, we present that prominent classification accuracy of lesion detection can be obtained by proper designing and applying of transfer learning framework on pre-trained neural networks, without any requirement for data enlargement procedures i.e. merging VGG16 and VGG19 architectures pre-trained by a generic dataset with modified AlexNet network, and then, fine-tuned by a subject-specific dataset containing dermatology images. The convolution neural network was trained using 2541 images and, in particular, dropout was used to prevent the network from overfitting. Finally, the validity of the model was checked by applying the K-fold cross validation method. The proposed model increased classification accuracy by 3% (from 94.2% to 98.18%) in comparison with other methods.


title: A dynamical systems formulation for inhomogeneous LRS-II spacetimes

abstract: We present a dynamical system formulation for inhomogeneous LRS-II spacetimes using the covariant 1+1+2 decomposition approach. Our approach describes the LRS-II dynamics from the point of view of a comoving observer. Promoting the covariant radial derivatives of the covariant dynamical quantities to new dynamical variables and utilizing the commutation relation between the covariant temporal and radial derivatives, we were able to construct an autonomous system of first-order ordinary differential equations along with some purely algebraic constraints. Using our dynamical system formulation we found several interesting features in the LRS-II phase space with dust, one of them being that the homogeneous solutions constitute an invariant submanifold. For the particular case of LTB, we were also able to recover the previously known result that an expanding LTB tends to Milne in the absence of a cosmological constant, providing a potential validation of our formalism.


title: The 2-character theory for finite 2-groups

abstract: In this work, we generalize the notion of character for 2-representations of finite 2-groups. The properties of 2-characters bear strong similarities to those classical characters of finite groups, including conjugation invariance, additivity, multiplicativity and orthogonality. With a careful analysis using homotopy fixed points and quotients for categories with 2-group actions, we prove that the category of class functors on a 2-group $\mathcal G$ is equivalent to the Drinfeld center of the 2-group algebra $\mathrm{Vec}_{\mathcal G}$, which categorifies the Fourier transform on finite abelian groups. After transferring the canonical nondegenerate braided monoidal structure from $\mathfrak Z_1(\mathrm{Vec}_{\mathcal G})$, we discover that irreducible 2-characters of $\mathcal G$ coincide with full centers of the corresponding 2-representations, which are in a one-to-one correspondence with Lagrangian algebras in the category of class functors on $\mathcal G$. In particular, the fusion rule of $2\mathrm{Rep}(\mathcal G)$ can be calculated from the pointwise product of Lagrangian algebras as class functors. From a topological quantum field theory (TQFT) point of view, the commutative Frobenius algebra structure on a 2-character is induced from a 2D topological sigma-model with target space $\lvert \mathrm{B} \mathcal G \rvert$.


title: Capturing Shock Waves by Relaxation Neural Networks

abstract: In this paper, we put forward a neural network framework to solve the nonlinear hyperbolic systems. This framework, named relaxation neural networks(RelaxNN), is a simple and scalable extension of physics-informed neural networks(PINN). It is shown later that a typical PINN framework struggles to handle shock waves that arise in hyperbolic systems' solutions. This ultimately results in the failure of optimization that is based on gradient descent in the training process. Relaxation systems provide a smooth asymptotic to the discontinuity solution, under the expectation that macroscopic problems can be solved from a microscopic perspective. Based on relaxation systems, the RelaxNN framework alleviates the conflict of losses in the training process of the PINN framework. In addition to the remarkable results demonstrated in numerical simulations, most of the acceleration techniques and improvement strategies aimed at the standard PINN framework can also be applied to the RelaxNN framework.


title: Unified Predefined-time Stability Conditions of Nonlinear Systems with Lyapunov Analysis

abstract: This brief gives a set of unified Lyapunov stability conditions to guarantee the predefined-time/finite-time stability of a dynamical systems. The derived Lyapunov theorem for autonomous systems establishes equivalence with existing theorems on predefined-time/finite-time stability. The findings proposed herein develop a nonsingular sliding mode control framework for an Euler-Lagrange system to analyze its stability, and its upper bound for the settling time can be arbitrarily determined a priori through predefined time constant.


title: LITE: Modeling Environmental Ecosystems with Multimodal Large Language Models

abstract: The modeling of environmental ecosystems plays a pivotal role in the sustainable management of our planet. Accurate prediction of key environmental variables over space and time can aid in informed policy and decision-making, thus improving people's livelihood. Recently, deep learning-based methods have shown promise in modeling the spatial-temporal relationships for predicting environmental variables. However, these approaches often fall short in handling incomplete features and distribution shifts, which are commonly observed in environmental data due to the substantial cost of data collection and malfunctions in measuring instruments. To address these issues, we propose LITE -- a multimodal large language model for environmental ecosystems modeling. Specifically, LITE unifies different environmental variables by transforming them into natural language descriptions and line graph images. Then, LITE utilizes unified encoders to capture spatial-temporal dynamics and correlations in different modalities. During this step, the incomplete features are imputed by a sparse Mixture-of-Experts framework, and the distribution shift is handled by incorporating multi-granularity information from past observations. Finally, guided by domain instructions, a language model is employed to fuse the multimodal representations for the prediction. Our experiments demonstrate that LITE significantly enhances performance in environmental spatial-temporal prediction across different domains compared to the best baseline, with a 41.25% reduction in prediction error. This justifies its effectiveness. Our data and code are available at https://github.com/hrlics/LITE.


title: Scalable Radar-based Roadside Perception: Self-localization and Occupancy Heat Map for Traffic Analysis

abstract: 4D mmWave radar sensors are suitable for roadside perception in city-scale Intelligent Transportation Systems (ITS) due to their long sensing range, weatherproof functionality, simple mechanical design, and low manufacturing cost. In this work, we investigate radar-based ITS for scalable traffic analysis. Localization of these radar sensors at city scale is a fundamental task in ITS. For flexible sensor setups, it requires even more effort. To address this task, we propose a self-localization approach that matches two descriptions of the "road": the one from the geometry of the motion trajectories of cumulatively observed vehicles, and the other one from the aerial laser scan. An Iterative Closest Point (ICP) algorithm is used to register the motion trajectory in the road section of the laser scan. The resulting estimate of the transformation matrix represents the sensor pose in a global reference frame. We evaluate the results and show that it outperforms other map-based radar localization methods, especially for the orientation estimation. Beyond the localization result, we project radar sensor data onto a city-scale laser scan and generate a scalable occupancy heat map as a traffic analysis tool. This is demonstrated using two radar sensors monitoring an urban area in the real world.


title: Multiple Joint Chance Constraints Approximation for Uncertainty Modeling in Dispatch Problems

abstract: Uncertainty modeling has become increasingly important in power system decision-making. The widely-used tractable uncertainty modeling method-chance constraints with Conditional Value at Risk (CVaR) approximation, can be overconservative and even turn an originally feasible problem into an infeasible one. This paper proposes a new approximation method for multiple joint chance constraints (JCCs) to model the uncertainty in dispatch problems, which solves the conservativeness and potential infeasibility concerns of CVaR. The proposed method is also convenient for controlling the risk levels of different JCCs, which is necessary for power system applications since different resources may be affected by varying degrees of uncertainty or have different importance to the system. We then formulate a data-driven distributionally robust chance-constrained programming model for the power system multiperiod dispatch problem and leverage the proposed approximation method to solve it. In the numerical simulations, two small general examples clearly demonstrate the superiority of the proposed method, and the results of the multiperiod dispatch problem on IEEE test cases verify its practicality.


title: Mirror-3DGS: Incorporating Mirror Reflections into 3D Gaussian Splatting

abstract: 3D Gaussian Splatting (3DGS) has marked a significant breakthrough in the realm of 3D scene reconstruction and novel view synthesis. However, 3DGS, much like its predecessor Neural Radiance Fields (NeRF), struggles to accurately model physical reflections, particularly in mirrors that are ubiquitous in real-world scenes. This oversight mistakenly perceives reflections as separate entities that physically exist, resulting in inaccurate reconstructions and inconsistent reflective properties across varied viewpoints. To address this pivotal challenge, we introduce Mirror-3DGS, an innovative rendering framework devised to master the intricacies of mirror geometries and reflections, paving the way for the generation of realistically depicted mirror reflections. By ingeniously incorporating mirror attributes into the 3DGS and leveraging the principle of plane mirror imaging, Mirror-3DGS crafts a mirrored viewpoint to observe from behind the mirror, enriching the realism of scene renderings. Extensive assessments, spanning both synthetic and real-world scenes, showcase our method's ability to render novel views with enhanced fidelity in real-time, surpassing the state-of-the-art Mirror-NeRF specifically within the challenging mirror regions. Our code will be made publicly available for reproducible research.


title: A Comparative Study of Dust Grain Polarisation Efficiencies in the Interstellar and Intracluster Mediums towards Anti-Center Galaxy

abstract: Dust polarisation observations at optical wavelengths help understand the dust grain properties and trace the plane-of-the-sky component of the magnetic field. In this study, we make use of published optical polarisation data acquired with AIMPOL along with distances ($d$) and extinction ($A_{\mathrm{V}}$) data. We study the variation of polarisation efficiency ($P/A_{\mathrm{V}}$) as a function of $A_{\mathrm{V}}$ in the diffuse interstellar medium (ISM) and intracluster mediums (ICM) using the already published polarisation data of six clusters. Among these clusters, NGC 2281, NGC 1664, and NGC 1960 are old; while Stock 8, NGC 1931, and NGC 1893 are young. We categorize stars towards each cluster into foreground, background, and cluster members by employing two clustering algorithms GMM and DBSCAN. Thus, classified field stars and cluster members are used to reveal the polarisation properties of ISM and ICM dust, respectively. We find that the dust grains located in the diffuse ISM show higher polarisation efficiencies when compared to those located in the ICM of younger clusters.


title: Force-EvT: A Closer Look at Robotic Gripper Force Measurement with Event-based Vision Transformer

abstract: Robotic grippers are receiving increasing attention in various industries as essential components of robots for interacting and manipulating objects. While significant progress has been made in the past, conventional rigid grippers still have limitations in handling irregular objects and can damage fragile objects. We have shown that soft grippers offer deformability to adapt to a variety of object shapes and maximize object protection. At the same time, dynamic vision sensors (e.g., event-based cameras) are capable of capturing small changes in brightness and streaming them asynchronously as events, unlike RGB cameras, which do not perform well in low-light and fast-moving environments. In this paper, a dynamic-vision-based algorithm is proposed to measure the force applied to the gripper. In particular, we first set up a DVXplorer Lite series event camera to capture twenty-five sets of event data. Second, motivated by the impressive performance of the Vision Transformer (ViT) algorithm in dense image prediction tasks, we propose a new approach that demonstrates the potential for real-time force estimation and meets the requirements of real-world scenarios. We extensively evaluate the proposed algorithm on a wide range of scenarios and settings, and show that it consistently outperforms recent approaches.


title: Singularity categories via McKay quivers with potential

abstract: In 2018, Kalck and Yang showed that the singularity categories associated with 3-dimensional Gorenstein quotient singularities are triangle equivalent (up to direct summands) to small cluster categories associated with McKay quivers with potential. We introduce graded McKay quivers with potential and generalize Kalck-Yang's theorem to arbitrary dimensions. The singularity categories we consider occur as stable categories of categories of maximal Cohen-Macaulay modules. We refine our description of the singularity categories by showing that these categories of maximal Cohen-Macaulay modules are equivalent to Higgs categories in the sense of Wu. Moreover, we describe the singularity categories in the non-Gorenstein case.


title: Covering convection with thermal blankets: formation of supercontinents

abstract: The continental plates of Earth are known to drift over a geophysical timescale, and their interactions have lead to some of the most spectacular geoformations of our planet while also causing natural disasters such as earthquakes and volcanic activity. Understanding the dynamics of interacting continental plates is thus significant. In this work, we present a fluid mechanical investigation of the plate motion, interaction, and dynamics. Through numerical experiments, we examine the coupling between a convective fluid and plates floating on top of it. With physical modeling, we show the coupling is both mechanical and thermal, leading to the thermal blanket effect: the floating plate is not only transported by the fluid flow beneath, it also prevents the heat from leaving the fluid, leading to a convective flow that further affects the plate motion. By adding several plates to such a coupled fluid-structure interaction, we also investigate how floating plates interact with each other and show that, under proper conditions, small plates can converge into a supercontinent.


title: Strong quantum state transfer on graphs via loop edges

abstract: We quantify the effect of weighted loops at the source and target nodes of a graph on the strength of quantum state transfer between these vertices. We give lower bounds on loop weights that guarantee strong transfer fidelity that works for any graph where this protocol is feasible. By considering local spectral symmetry, we show that the required weight size depends only on the maximum degree of the graph and, in some less favorable cases, the distance between vertices. Additionally, we explore the duration for which transfer strength remains above a specified threshold.


title: SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Grounding

abstract: Temporal video grounding (TVG) is a critical task in video content understanding, requiring precise alignment between video content and natural language instructions. Despite significant advancements, existing methods face challenges in managing confidence bias towards salient objects and capturing long-term dependencies in video sequences. To address these issues, we introduce SpikeMba: a multi-modal spiking saliency mamba for temporal video grounding. Our approach integrates Spiking Neural Networks (SNNs) with state space models (SSMs) to leverage their unique advantages in handling different aspects of the task. Specifically, we use SNNs to develop a spiking saliency detector that generates the proposal set. The detector emits spike signals when the input signal exceeds a predefined threshold, resulting in a dynamic and binary saliency proposal set. To enhance the model's capability to retain and infer contextual information, we introduce relevant slots which learnable tensors that encode prior knowledge. These slots work with the contextual moment reasoner to maintain a balance between preserving contextual information and exploring semantic relevance dynamically. The SSMs facilitate selective information propagation, addressing the challenge of long-term dependency in video content. By combining SNNs for proposal generation and SSMs for effective contextual reasoning, SpikeMba addresses confidence bias and long-term dependencies, thereby significantly enhancing fine-grained multimodal relationship capture. Our experiments demonstrate the effectiveness of SpikeMba, which consistently outperforms state-of-the-art methods across mainstream benchmarks.


title: Reconstructing Robust Background IFU spectra using Machine Learning

abstract: In astronomy, spectroscopy consists of observing an astrophysical source and extracting its spectrum of electromagnetic radiation. Once extracted, a model is fit to the spectra to measure the observables, leading to an understanding of the underlying physics of the emission mechanism. One crucial, and often overlooked, aspect of this model is the background emission, which contains foreground and background astrophysical sources, intervening atmospheric emission, and artifacts related to the instrument such as noise. This paper proposes an algorithmic approach to constructing a background model for SITELLE observations using statistical tools and supervised machine learning algorithms. We apply a segmentation algorithm implemented in photutils to divide the data cube into background and source spaxels. After applying a principal component analysis (PCA) on the background spaxels, we train an artificial neural network to interpolate from the background to the source spaxels in the PCA coefficient space, which allows us to generate a local background model over the entire data cube. We highlight the performance of this methodology by applying it to SITELLE observations obtained of a SIGNALS galaxy, \NGC4449, and the Perseus galaxy cluster of galaxies, NGC 1275. We discuss the physical interpretation of the principal components and noise reduction in the resulting PCA-based reconstructions. Additionally, we compare the fit results using our new background modeling approach to standard methods used in the literature and find that our method better captures the emission from HII regions in NGC 4449 and the faint emission regions in NGC 1275. These methods also demonstrate that the background does change as a function of the position of the datacube.


title: Using Chao's Estimator as a Stopping Criterion for Technology-Assisted Review

abstract: Technology-Assisted Review (TAR) aims to reduce the human effort required for screening processes such as abstract screening for systematic literature reviews. Human reviewers label documents as relevant or irrelevant during this process, while the system incrementally updates a prediction model based on the reviewers' previous decisions. After each model update, the system proposes new documents it deems relevant, to prioritize relevant documentsover irrelevant ones. A stopping criterion is necessary to guide users in stopping the review process to minimize the number of missed relevant documents and the number of read irrelevant documents. In this paper, we propose and evaluate a new ensemble-based Active Learning strategy and a stopping criterion based on Chao's Population Size Estimator that estimates the prevalence of relevant documents in the dataset. Our simulation study demonstrates that this criterion performs well on several datasets and is compared to other methods presented in the literature.


title: Poisoning Decentralized Collaborative Recommender System and Its Countermeasures

abstract: To make room for privacy and efficiency, the deployment of many recommender systems is experiencing a shift from central servers to personal devices, where the federated recommender systems (FedRecs) and decentralized collaborative recommender systems (DecRecs) are arguably the two most representative paradigms. While both leverage knowledge (e.g., gradients) sharing to facilitate learning local models, FedRecs rely on a central server to coordinate the optimization process, yet in DecRecs, the knowledge sharing directly happens between clients. Knowledge sharing also opens a backdoor for model poisoning attacks, where adversaries disguise themselves as benign clients and disseminate polluted knowledge to achieve malicious goals like promoting an item's exposure rate. Although research on such poisoning attacks provides valuable insights into finding security loopholes and corresponding countermeasures, existing attacks mostly focus on FedRecs, and are either inapplicable or ineffective for DecRecs. Compared with FedRecs where the tampered information can be universally distributed to all clients once uploaded to the cloud, each adversary in DecRecs can only communicate with neighbor clients of a small size, confining its impact to a limited range. To fill the gap, we present a novel attack method named Poisoning with Adaptive Malicious Neighbors (PAMN). With item promotion in top-K recommendation as the attack objective, PAMN effectively boosts target items' ranks with several adversaries that emulate benign clients and transfers adaptively crafted gradients conditioned on each adversary's neighbors. Moreover, with the vulnerabilities of DecRecs uncovered, a dedicated defensive mechanism based on user-level gradient clipping with sparsified updating is proposed. Extensive experiments demonstrate the effectiveness of the poisoning attack and the robustness of our defensive mechanism.


title: Projective contact log varieties

abstract: We study contact structures on smooth complex projective varieties with a simple normal crossing divisor, generalizing some well-known results concerning the non-logarithmic case. In particular, we describe the structure of elementary log contractions of such log varieties and we construct the corresponding contact structures.


title: BEM: Balanced and Entropy-based Mix for Long-Tailed Semi-Supervised Learning

abstract: Data mixing methods play a crucial role in semi-supervised learning (SSL), but their application is unexplored in long-tailed semi-supervised learning (LTSSL). The primary reason is that the in-batch mixing manner fails to address class imbalance. Furthermore, existing LTSSL methods mainly focus on re-balancing data quantity but ignore class-wise uncertainty, which is also vital for class balance. For instance, some classes with sufficient samples might still exhibit high uncertainty due to indistinguishable features. To this end, this paper introduces the Balanced and Entropy-based Mix (BEM), a pioneering mixing approach to re-balance the class distribution of both data quantity and uncertainty. Specifically, we first propose a class balanced mix bank to store data of each class for mixing. This bank samples data based on the estimated quantity distribution, thus re-balancing data quantity. Then, we present an entropy-based learning approach to re-balance class-wise uncertainty, including entropy-based sampling strategy, entropy-based selection module, and entropy-based class balanced loss. Our BEM first leverages data mixing for improving LTSSL, and it can also serve as a complement to the existing re-balancing methods. Experimental results show that BEM significantly enhances various LTSSL frameworks and achieves state-of-the-art performances across multiple benchmarks.


title: On the fundamental group of a spherical variety

abstract: Let $X=G/H$ be a spherical variety over an algebraically closed field of characteristic $p\ge0$. We compute the $p'$-parts of $\pi_0(H)$ and $\pi_1(X)$ from the spherical system of $X$.


title: Constraining Dark Matter Annihilation with Fermi-LAT Observations of Ultra-Faint Compact Stellar Systems

abstract: Recent results from numerical simulations and models of galaxy formation suggest that recently discovered ultra-faint compact stellar systems (UFCSs) in the halo of the Milky Way (MW) may be some of the smallest and faintest galaxies. If this is the case, these systems would be attractive targets for indirect searches of weakly interacting massive particle (WIMP) dark matter (DM) annihilation due to their relative proximity and high expected DM content. In this study, we analyze 14.3 years of gamma-ray data collected by the Fermi-LAT coincident with 26 UFCSs. No significant excess gamma-ray emission is detected, and we present gamma-ray flux upper limits for these systems. Assuming that the UFCSs are dark-matter-dominated galaxies consistent with being among the faintest and least massive MW dwarf spheroidal (dSphs) satellite galaxies, we derive the projected sensitivity for a dark matter annihilation signal. We find that observations of UFCSs have the potential to yield some of the most powerful constraints on DM annihilation, with sensitivity comparable to observations of known dSphs and the Galactic center. This result emphasizes the importance of precise kinematic studies of UFCSs to empirically determine their DM content.


title: A Neuro-Symbolic Approach to Monitoring Salt Content in Food

abstract: We propose a dialogue system that enables heart failure patients to inquire about salt content in foods and help them monitor and reduce salt intake. Addressing the lack of specific datasets for food-based salt content inquiries, we develop a template-based conversational dataset. The dataset is structured to ask clarification questions to identify food items and their salt content. Our findings indicate that while fine-tuning transformer-based models on the dataset yields limited performance, the integration of Neuro-Symbolic Rules significantly enhances the system's performance. Our experiments show that by integrating neuro-symbolic rules, our system achieves an improvement in joint goal accuracy of over 20% across different data sizes compared to naively fine-tuning transformer-based models.


title: Positioning is All You Need

abstract: One can drive safely with a GPS without memorizing a world map (not to mention the dark regions that humans have never explored). Such a locality-based attention mechanism has a profound implication on our understanding of how the brain works. This paper refines the existing embodied cognition framework by turning the locality from a constraint to an advantage. Analogous to GPS-based navigation, positioning represents a computationally more efficient solution to flexible behaviors than reconstruction. This simple intuition implies that {\em positioning is all you need} to understand cognitive functions generated by hippocampal-neocortical systems. That is, the neocortex generates thousands of local maps whose indexing is maintained by the hippocampus. Geometrically, we present a simple manifold positioning framework to explain the principle of localized embodied cognition. The positioning operation implemented by the attention mechanism can be interpreted as a nonlinear projection linking the discovery of local subspace structure by the neocortex (a sensorimotor machine interacting with the world locally) for the navigation task in mind without discovering global manifold topology.


title: Efficient Motion Planning for Manipulators with Control Barrier Function-Induced Neural Controller

abstract: Sampling-based motion planning methods for manipulators in crowded environments often suffer from expensive collision checking and high sampling complexity, which make them difficult to use in real time. To address this issue, we propose a new generalizable control barrier function (CBF)-based steering controller to reduce the number of samples needed in a sampling-based motion planner RRT. Our method combines the strength of CBF for real-time collision-avoidance control and RRT for long-horizon motion planning, by using CBF-induced neural controller (CBF-INC) to generate control signals that steer the system towards sampled configurations by RRT. CBF-INC is learned as Neural Networks and has two variants handling different inputs, respectively: state (signed distance) input and point-cloud input from LiDAR. In the latter case, we also study two different settings: fully and partially observed environmental information. Compared to manually crafted CBF which suffers from over-approximating robot geometry, CBF-INC can balance safety and goal-reaching better without being over-conservative. Given state-based input, our neural CBF-induced neural controller-enhanced RRT (CBF-INC-RRT) can increase the success rate by 14% while reducing the number of nodes explored by 30%, compared with vanilla RRT on hard test cases. Given LiDAR input where vanilla RRT is not directly applicable, we demonstrate that our CBF-INC-RRT can improve the success rate by 10%, compared with planning with other steering controllers. Our project page with supplementary material is at https://mit-realm.github.io/CBF-INC-RRT-website/.


title: Symmetry of Convex Solutions to Fully Nonlinear Elliptic Systems: Unbounded Domains

abstract: In this paper, we are concerned with the monotonic and symmetric properties of convex solutions Monge-Amp\`ere systems for instance, considering \begin{equation*} \det(D^2u^i)=f^i(x,{\bf u},\nabla u^i), \ 1\leq i\leq m, \end{equation*} over unbounded domains of various cases, including the whole spaces $\mathbb{R}^n$, the half spaces $\mathbb{R}^n_+$ and the unbounded tube shape domains in $\mathbb{R}^n$. We obtain monotonic and symmetric properties of the solutions to the problem with respect to the geometry of domains and the monotonic and symmetric properties of right-hand side terms. The proof is based on carefully using the moving plane method together with various maximum principles and Hopf's lemmas.


title: Optimised observables and new physics prospects in the penguin-mediated decays $B_{d(s)}\to K^{(*)0}\phi$

abstract: We study the penguin-mediated decays $\bar{B}_{d(s)}\to\bar{K}^{*0}(K^{*0})\phi$ transitions. We propose a new optimised observable $L_{K^*\phi}$ from the ratio of longitudinal branching ratios of these decays, with limited hadronic uncertainties and enhanced sensitivity to New Physics. It exhibits a deviation at the $1.48\, \sigma$ level between its experimental value and its SM determination within QCD factorisation. This result can be accommodated together with the significant deviations found for the $K^{(*)}\bar{K}^{(*)}$ modes in our earlier works, if New Physics affects either the QCD penguin operator $Q_4$ or the chromomagnetic dipole operator $Q_{8g}$ for both $b\to d$ and $b\to s$ transitions. The allowed range for the Wilson coefficients $C_{4s,8gs}$ is narrower compared to $C_{4d,8gd}$ since the $b\to s$ transition $\bar{B}_d\to\bar{K}^{*0}\phi$ is in better agreement with the SM. If we add the measured branching ratio for the $\bar{B}_{d}\to \bar{K}^{0}\phi$ to our analysis, the simultaneous explanation of all the experimental data for the $K^{(*)}\bar{K}^{(*)}$ and the $K^*(\bar{K}^{(*)})\phi$ channels in terms of New Physics in $C_{4d,s}$ or $C_{8gd,s}$ operators only becomes very constrained. This set of observables can be explained more easily if we assume New Physics either in $(C_{4f},C_{6f})$ or $(C_{6f},C_{8gf})$ in both $f=d,s$. This should provide a strong incentive for the LHCb experiment to perform a measurement of the branching ratios of $\bar{B}_{d}\to\bar{K}^{(*)0}\phi$ and an improved measurement for the branching ratio of $\bar{B}_s\to K^{*0}\phi$.


title: Modeling the low-pressure high-voltage branch of the Paschen curve for hydrogen and deuterium

abstract: A physical and numerical model of the Townsend discharge in molecular hydrogen and deuterium has been developed to meet the needs of designing a plasma-based switching device for power grid application. The model allows to predict the low-pressure branch of the Paschen curve for applied voltage in the range of several hundred kiloVolts. In the regime of interest, electrons are in a runaway state and ionization by ions and fast neutrals sustains the discharge. It was essential to correctly account for both gas-phase and surface interactions (electron emission and electron back-scattering), especially in terms of their dependence on particle energy. The model yields results consistent with prior data obtained for lower voltage. The three-species (electrons, ions and fast neutrals) model successfully captures the essential physics of the process.


title: MonoBox: Tightness-free Box-supervised Polyp Segmentation using Monotonicity Constraint

abstract: We propose MonoBox, an innovative box-supervised segmentation method constrained by monotonicity to liberate its training from the user-unfriendly box-tightness assumption. In contrast to conventional box-supervised segmentation, where the box edges must precisely touch the target boundaries, MonoBox leverages imprecisely-annotated boxes to achieve robust pixel-wise segmentation. The 'linchpin' is that, within the noisy zones around box edges, MonoBox discards the traditional misguiding multiple-instance learning loss, and instead optimizes a carefully-designed objective, termed monotonicity constraint. Along directions transitioning from the foreground to background, this new constraint steers responses to adhere to a trend of monotonically decreasing values. Consequently, the originally unreliable learning within the noisy zones is transformed into a correct and effective monotonicity optimization. Moreover, an adaptive label correction is introduced, enabling MonoBox to enhance the tightness of box annotations using predicted masks from the previous epoch and dynamically shrink the noisy zones as training progresses. We verify MonoBox in the box-supervised segmentation task of polyps, where satisfying box-tightness is challenging due to the vague boundaries between the polyp and normal tissues. Experiments on both public synthetic and in-house real noisy datasets demonstrate that MonoBox exceeds other anti-noise state-of-the-arts by improving Dice by at least 5.5% and 3.3%, respectively. Codes are at https://github.com/Huster-Hq/MonoBox.


title: Generating Faithful and Complete Hospital-Course Summaries from the Electronic Health Record

abstract: The rapid adoption of Electronic Health Records (EHRs) has been instrumental in streamlining administrative tasks, increasing transparency, and enabling continuity of care across providers. An unintended consequence of the increased documentation burden, however, has been reduced face-time with patients and, concomitantly, a dramatic rise in clinician burnout. In this thesis, we pinpoint a particularly time-intensive, yet critical, documentation task: generating a summary of a patient's hospital admissions, and propose and evaluate automated solutions. In Chapter 2, we construct a dataset based on 109,000 hospitalizations (2M source notes) and perform exploratory analyses to motivate future work on modeling and evaluation [NAACL 2021]. In Chapter 3, we address faithfulness from a modeling perspective by revising noisy references [EMNLP 2022] and, to reduce the reliance on references, directly calibrating model outputs to metrics [ACL 2023]. These works relied heavily on automatic metrics as human annotations were limited. To fill this gap, in Chapter 4, we conduct a fine-grained expert annotation of system errors in order to meta-evaluate existing metrics and better understand task-specific issues of domain adaptation and source-summary alignments. To learn a metric less correlated to extractiveness (copy-and-paste), we derive noisy faithfulness labels from an ensemble of existing metrics and train a faithfulness classifier on these pseudo labels [MLHC 2023]. Finally, in Chapter 5, we demonstrate that fine-tuned LLMs (Mistral and Zephyr) are highly prone to entity hallucinations and cover fewer salient entities. We improve both coverage and faithfulness by performing sentence-level entity planning based on a set of pre-computed salient entities from the source text, which extends our work on entity-guided news summarization [ACL, 2023], [EMNLP, 2023].


title: Call the Dentist! A (Con-)Cavity in the Value of Information

abstract: A natural way of quantifying the ``amount of information'' in decision problems yields a globally concave value for information. Another (in contrast, adversarial) way almost never does.


title: A Semiparametric Approach for Robust and Efficient Learning with Biobank Data

abstract: With the increasing availability of electronic health records (EHR) linked with biobank data for translational research, a critical step in realizing its potential is to accurately classify phenotypes for patients. Existing approaches to achieve this goal are based on error-prone EHR surrogate outcomes, assisted and validated by a small set of labels obtained via medical chart review, which may also be subject to misclassification. Ignoring the noise in these outcomes can induce severe estimation and validation bias to both EHR phenotyping and risking modeling with biomarkers collected in the biobank. To overcome this challenge, we propose a novel unsupervised and semiparametric approach to jointly model multiple noisy EHR outcomes with their linked biobank features. Our approach primarily aims at disease risk modeling with the baseline biomarkers, and is also able to produce a predictive EHR phenotyping model and validate its performance without observations of the true disease outcome. It consists of composite and nonparametric regression steps free of any parametric model specification, followed by a parametric projection step to reduce the uncertainty and improve the estimation efficiency. We show that our method is robust to violations of the parametric assumptions while attaining the desirable root-$n$ convergence rates on risk modeling. Our developed method outperforms existing methods in extensive simulation studies, as well as a real-world application in phenotyping and genetic risk modeling of type II diabetes.


title: iMD4GC: Incomplete Multimodal Data Integration to Advance Precise Treatment Response Prediction and Survival Analysis for Gastric Cancer

abstract: Gastric cancer (GC) is a prevalent malignancy worldwide, ranking as the fifth most common cancer with over 1 million new cases and 700 thousand deaths in 2020. Locally advanced gastric cancer (LAGC) accounts for approximately two-thirds of GC diagnoses, and neoadjuvant chemotherapy (NACT) has emerged as the standard treatment for LAGC. However, the effectiveness of NACT varies significantly among patients, with a considerable subset displaying treatment resistance. Ineffective NACT not only leads to adverse effects but also misses the optimal therapeutic window, resulting in lower survival rate. However, existing multimodal learning methods assume the availability of all modalities for each patient, which does not align with the reality of clinical practice. The limited availability of modalities for each patient would cause information loss, adversely affecting predictive accuracy. In this study, we propose an incomplete multimodal data integration framework for GC (iMD4GC) to address the challenges posed by incomplete multimodal data, enabling precise response prediction and survival analysis. Specifically, iMD4GC incorporates unimodal attention layers for each modality to capture intra-modal information. Subsequently, the cross-modal interaction layers explore potential inter-modal interactions and capture complementary information across modalities, thereby enabling information compensation for missing modalities. To evaluate iMD4GC, we collected three multimodal datasets for GC study: GastricRes (698 cases) for response prediction, GastricSur (801 cases) for survival analysis, and TCGA-STAD (400 cases) for survival analysis. The scale of our datasets is significantly larger than previous studies. The iMD4GC achieved impressive performance with an 80.2% AUC on GastricRes, 71.4% C-index on GastricSur, and 66.1% C-index on TCGA-STAD, significantly surpassing other compared methods.


title: Enhancing capabilities of particle image/tracking velocimetry with physics-informed neural networks

abstract: Solving inverse problems, which means obtaining model parameters from observed data, using conventional computational fluid dynamics solvers is prohibitively expensive. Here we employ machine learning algorithms to overcome the challenge. As an example, we consider a moderately turbulent fluid flow, excited by a stationary force and described by a two-dimensional Navier-Stokes equation with linear bottom friction. Given sparse and probably noisy data for the velocity and the general form of the model, we reconstruct the dense velocity and pressure fields in the observation domain, infer the driving force, and determine the unknown fluid viscosity and friction coefficient. Our approach involves training a physics-informed neural network by minimizing the loss function, which penalizes deviations from the provided data and violations of the Navier-Stokes equation. The suggested technique extracts additional information from experimental and numerical observations, potentially enhancing the capabilities of particle image/tracking velocimetry.


title: Adaptive Query Prompting for Multi-Domain Landmark Detection

abstract: Medical landmark detection is crucial in various medical imaging modalities and procedures. Although deep learning-based methods have achieve promising performance, they are mostly designed for specific anatomical regions or tasks. In this work, we propose a universal model for multi-domain landmark detection by leveraging transformer architecture and developing a prompting component, named as Adaptive Query Prompting (AQP). Instead of embedding additional modules in the backbone network, we design a separate module to generate prompts that can be effectively extended to any other transformer network. In our proposed AQP, prompts are learnable parameters maintained in a memory space called prompt pool. The central idea is to keep the backbone frozen and then optimize prompts to instruct the model inference process. Furthermore, we employ a lightweight decoder to decode landmarks from the extracted features, namely Light-MLD. Thanks to the lightweight nature of the decoder and AQP, we can handle multiple datasets by sharing the backbone encoder and then only perform partial parameter tuning without incurring much additional cost. It has the potential to be extended to more landmark detection tasks. We conduct experiments on three widely used X-ray datasets for different medical landmark detection tasks. Our proposed Light-MLD coupled with AQP achieves SOTA performance on many metrics even without the use of elaborate structural designs or complex frameworks.


title: Robust Trajectory and Resource Optimization for Communication-assisted UAV SAR Sensing

abstract: In this paper, we investigate joint 3-dimensional (3D) trajectory planning and resource allocation for rotary-wing unmanned aerial vehicle (UAV) synthetic aperture radar (SAR) sensing. To support emerging real-time SAR applications and enable live mission control, we incorporate real-time communication with a ground station (GS). The UAV's main mission is the mapping of large areas of interest (AoIs) using an onboard SAR system and transferring the unprocessed raw radar data to the ground in real time. We propose a robust trajectory and resource allocation design that takes into account random UAV trajectory deviations. To this end, we model the UAV trajectory deviations and study their effect on the radar coverage. Then, we formulate a robust non-convex mixed-integer non-linear program (MINLP) such that the UAV 3D trajectory and resources are jointly optimized for maximization of the radar ground coverage. A low-complexity sub-optimal solution for the formulated problem is presented. Furthermore, to assess the performance of the sub-optimal algorithm, we derive an upper bound on the optimal solution based on monotonic optimization theory. Simulation results show that the proposed sub-optimal algorithm achieves close-to-optimal performance and not only outperforms several benchmark schemes but is also robust with respect to UAV trajectory deviations.


title: Estimating Lexical Complexity from Document-Level Distributions

abstract: Existing methods for complexity estimation are typically developed for entire documents. This limitation in scope makes them inapplicable for shorter pieces of text, such as health assessment tools. These typically consist of lists of independent sentences, all of which are too short for existing methods to apply. The choice of wording in these assessment tools is crucial, as both the cognitive capacity and the linguistic competency of the intended patient groups could vary substantially. As a first step towards creating better tools for supporting health practitioners, we develop a two-step approach for estimating lexical complexity that does not rely on any pre-annotated data. We implement our approach for the Norwegian language and verify its effectiveness using statistical testing and a qualitative evaluation of samples from real assessment tools. We also investigate the relationship between our complexity measure and certain features typically associated with complexity in the literature, such as word length, frequency, and the number of syllables.


title: Getting it Right: Improving Spatial Consistency in Text-to-Image Models

abstract: One of the key shortcomings in current text-to-image (T2I) models is their inability to consistently generate images which faithfully follow the spatial relationships specified in the text prompt. In this paper, we offer a comprehensive investigation of this limitation, while also developing datasets and methods that achieve state-of-the-art performance. First, we find that current vision-language datasets do not represent spatial relationships well enough; to alleviate this bottleneck, we create SPRIGHT, the first spatially-focused, large scale dataset, by re-captioning 6 million images from 4 widely used vision datasets. Through a 3-fold evaluation and analysis pipeline, we find that SPRIGHT largely improves upon existing datasets in capturing spatial relationships. To demonstrate its efficacy, we leverage only ~0.25% of SPRIGHT and achieve a 22% improvement in generating spatially accurate images while also improving the FID and CMMD scores. Secondly, we find that training on images containing a large number of objects results in substantial improvements in spatial consistency. Notably, we attain state-of-the-art on T2I-CompBench with a spatial score of 0.2133, by fine-tuning on <500 images. Finally, through a set of controlled experiments and ablations, we document multiple findings that we believe will enhance the understanding of factors that affect spatial consistency in text-to-image models. We publicly release our dataset and model to foster further research in this area.


title: Nearly-tight Approximation Guarantees for the Improving Multi-Armed Bandits Problem

abstract: We give nearly-tight upper and lower bounds for the improving multi-armed bandits problem. An instance of this problem has $k$ arms, each of whose reward function is a concave and increasing function of the number of times that arm has been pulled so far. We show that for any randomized online algorithm, there exists an instance on which it must suffer at least an $\Omega(\sqrt{k})$ approximation factor relative to the optimal reward. We then provide a randomized online algorithm that guarantees an $O(\sqrt{k})$ approximation factor, if it is told the maximum reward achievable by the optimal arm in advance. We then show how to remove this assumption at the cost of an extra $O(\log k)$ approximation factor, achieving an overall $O(\sqrt{k} \log k)$ approximation relative to optimal.


title: Evaluation of the systematic error induced by quadratic Zeeman effect using hyperfine ground state exchange method in a long-baseline dual-species atom interferometer

abstract: The systematic error induced by the quadratic Zeeman effect is non-negligible in atom interferometers and must be precisely evaluated. We theoretically analyze the phase shift induced by the Zeeman effect, and use a hyperfine ground state exchange (HGSE) method to evaluate the systematic error in the long-baseline $^{85}$Rb-$^{87}$Rb dual-species atom interferometer due to the quadratic Zeeman effect. Compared to the two evaluation methods, mapping the absolute magnetic field in the interference region and performing phase measurements at different bias fields, the HGSE method could obtain the systematic error in real time in case of slow drifts of either the ambient magnetic field or other systematic effects irrelevant to the hyperfine ground states. To validate the effectiveness of the HGSE method, we also employ the mapping magnetic field method and modulating bias field method independently to cross-check and yield consistent results of three methods within an accuracy of $10^{-11}$ level. The HGSE method is helpful in evaluating and suppressing the quadratic Zeeman-effect-induced systematic error in long-baseline atom interferometer-based precision measurements, such as equivalence principle tests.


title: Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimization

abstract: Distributionally robust optimization (DRO) is a powerful framework for training robust models against data distribution shifts. This paper focuses on constrained DRO, which has an explicit characterization of the robustness level. Existing studies on constrained DRO mostly focus on convex loss function, and exclude the practical and challenging case with non-convex loss function, e.g., neural network. This paper develops a stochastic algorithm and its performance analysis for non-convex constrained DRO. The computational complexity of our stochastic algorithm at each iteration is independent of the overall dataset size, and thus is suitable for large-scale applications. We focus on the general Cressie-Read family divergence defined uncertainty set which includes $\chi^2$-divergences as a special case. We prove that our algorithm finds an $\epsilon$-stationary point with a computational complexity of $\mathcal O(\epsilon^{-3k_*-5})$, where $k_*$ is the parameter of the Cressie-Read divergence. The numerical results indicate that our method outperforms existing methods.} Our method also applies to the smoothed conditional value at risk (CVaR) DRO.


title: Search for gravitationally lensed interstellar transmissions

abstract: We consider interstellar light transmission facilitated by gravitational lensing. We study axially-symmetric lensing configurations where the transmitter, lens, and receiver are nearly aligned. With the transmitter positioned in the focal region of the lens, we investigate the caustic formed by a diffraction-limited beam of light emitted by the transmitter. We evaluated the impact of the lens's point spread function (PSF) on the received beam. We estimate the power delivered to a receiver at interstellar distances, evaluate the major noise sources, and determine the detection sensitivity in both the noise- and signal-dominated regimes. Considering realistic assumptions about the transmitter's performance, we explore signal detection strategies enhanced by the spatial broadening of the received beam, a result of the PSF from the transmitting lens. We find that detection of the lensed optical signals from nearby stars may be done relying on established optical engineering technologies. Furthermore, a network of astronomical facilities capable of observations in multiple narrow spectral bands will benefit the search. Our results support the feasibility of interstellar power transmission via gravitational lensing, directly contributing to the ongoing optical SETI efforts.


title: Device-independent quantum key distribution based on routed Bell tests

abstract: Photon losses are the main obstacle to fully photonic implementations of device-independent quantum key distribution (DIQKD). Motivated by recent work showing that routed Bell scenarios offer increased robustness to detection inefficiencies for the certification of long-range quantum correlations, we investigate DIQKD protocols based on a routed setup. In these protocols, in some of the test rounds, photons from the source are routed by an actively controlled switch to a nearby test device instead of the distant one. We show how to analyze the security of these protocols and compute lower bounds on the key rates using non-commutative polynomial optimization and the Brown-Fawzi-Fazwi method. We determine lower bounds on the asymptotic key rates of several simple two-qubit routed DIQKD protocols based on CHSH or BB84 correlations and compare their performance to standard protocols. We find that in an ideal case routed DIQKD protocols can significantly improve detection efficiency requirements, by up to $\sim 30\%$, compared to their non-routed counterparts. Notably, the routed BB84 protocol achieves a positive key rate with a detection efficiency as low as $50\%$ for the distant device, the minimal threshold for any QKD protocol featuring two untrusted measurements. However, the advantages we find are highly sensitive to noise and losses affecting the short-range correlations involving the additional test device.


title: Video Interpolation with Diffusion Models

abstract: We present VIDIM, a generative model for video interpolation, which creates short videos given a start and end frame. In order to achieve high fidelity and generate motions unseen in the input data, VIDIM uses cascaded diffusion models to first generate the target video at low resolution, and then generate the high-resolution video conditioned on the low-resolution generated video. We compare VIDIM to previous state-of-the-art methods on video interpolation, and demonstrate how such works fail in most settings where the underlying motion is complex, nonlinear, or ambiguous while VIDIM can easily handle such cases. We additionally demonstrate how classifier-free guidance on the start and end frame and conditioning the super-resolution model on the original high-resolution frames without additional parameters unlocks high-fidelity results. VIDIM is fast to sample from as it jointly denoises all the frames to be generated, requires less than a billion parameters per diffusion model to produce compelling results, and still enjoys scalability and improved quality at larger parameter counts.


title: The Fine Line: Navigating Large Language Model Pretraining with Down-streaming Capability Analysis

abstract: Uncovering early-stage metrics that reflect final model performance is one core principle for large-scale pretraining. The existing scaling law demonstrates the power-law correlation between pretraining loss and training flops, which serves as an important indicator of the current training state for large language models. However, this principle only focuses on the model's compression properties on the training data, resulting in an inconsistency with the ability improvements on the downstream tasks. Some follow-up works attempted to extend the scaling-law to more complex metrics (such as hyperparameters), but still lacked a comprehensive analysis of the dynamic differences among various capabilities during pretraining. To address the aforementioned limitations, this paper undertakes a comprehensive comparison of model capabilities at various pretraining intermediate checkpoints. Through this analysis, we confirm that specific downstream metrics exhibit similar training dynamics across models of different sizes, up to 67 billion parameters. In addition to our core findings, we've reproduced Amber and OpenLLaMA, releasing their intermediate checkpoints. This initiative offers valuable resources to the research community and facilitates the verification and exploration of LLM pretraining by open-source researchers. Besides, we provide empirical summaries, including performance comparisons of different models and capabilities, and tuition of key metrics for different training phases. Based on these findings, we provide a more user-friendly strategy for evaluating the optimization state, offering guidance for establishing a stable pretraining process.


title: Foundations of Cyber Resilience: The Confluence of Game, Control, and Learning Theories

abstract: Cyber resilience is a complementary concept to cybersecurity, focusing on the preparation, response, and recovery from cyber threats that are challenging to prevent. Organizations increasingly face such threats in an evolving cyber threat landscape. Understanding and establishing foundations for cyber resilience provide a quantitative and systematic approach to cyber risk assessment, mitigation policy evaluation, and risk-informed defense design. A systems-scientific view toward cyber risks provides holistic and system-level solutions. This chapter starts with a systemic view toward cyber risks and presents the confluence of game theory, control theory, and learning theories, which are three major pillars for the design of cyber resilience mechanisms to counteract increasingly sophisticated and evolving threats in our networks and organizations. Game and control theoretic methods provide a set of modeling frameworks to capture the strategic and dynamic interactions between defenders and attackers. Control and learning frameworks together provide a feedback-driven mechanism that enables autonomous and adaptive responses to threats. Game and learning frameworks offer a data-driven approach to proactively reason about adversarial behaviors and resilient strategies. The confluence of the three lays the theoretical foundations for the analysis and design of cyber resilience. This chapter presents various theoretical paradigms, including dynamic asymmetric games, moving horizon control, conjectural learning, and meta-learning, as recent advances at the intersection. This chapter concludes with future directions and discussions of the role of neurosymbolic learning and the synergy between foundation models and game models in cyber resilience.


title: Machine Unlearning for Traditional Models and Large Language Models: A Short Survey

abstract: With the implementation of personal data privacy regulations, the field of machine learning (ML) faces the challenge of the "right to be forgotten". Machine unlearning has emerged to address this issue, aiming to delete data and reduce its impact on models according to user requests. Despite the widespread interest in machine unlearning, comprehensive surveys on its latest advancements, especially in the field of Large Language Models (LLMs) is lacking. This survey aims to fill this gap by providing an in-depth exploration of machine unlearning, including the definition, classification and evaluation criteria, as well as challenges in different environments and their solutions. Specifically, this paper categorizes and investigates unlearning on both traditional models and LLMs, and proposes methods for evaluating the effectiveness and efficiency of unlearning, and standards for performance measurement. This paper reveals the limitations of current unlearning techniques and emphasizes the importance of a comprehensive unlearning evaluation to avoid arbitrary forgetting. This survey not only summarizes the key concepts of unlearning technology but also points out its prominent issues and feasible directions for future research, providing valuable guidance for scholars in the field.


title: Vision-language models for decoding provider attention during neonatal resuscitation

abstract: Neonatal resuscitations demand an exceptional level of attentiveness from providers, who must process multiple streams of information simultaneously. Gaze strongly influences decision making; thus, understanding where a provider is looking during neonatal resuscitations could inform provider training, enhance real-time decision support, and improve the design of delivery rooms and neonatal intensive care units (NICUs). Current approaches to quantifying neonatal providers' gaze rely on manual coding or simulations, which limit scalability and utility. Here, we introduce an automated, real-time, deep learning approach capable of decoding provider gaze into semantic classes directly from first-person point-of-view videos recorded during live resuscitations. Combining state-of-the-art, real-time segmentation with vision-language models (CLIP), our low-shot pipeline attains 91\% classification accuracy in identifying gaze targets without training. Upon fine-tuning, the performance of our gaze-guided vision transformer exceeds 98\% accuracy in gaze classification, approaching human-level precision. This system, capable of real-time inference, enables objective quantification of provider attention dynamics during live neonatal resuscitation. Our approach offers a scalable solution that seamlessly integrates with existing infrastructure for data-scarce gaze analysis, thereby offering new opportunities for understanding and refining clinical decision making.


title: TOFHIR2: The readout ASIC of the CMS Barrel MIP Timing Detector

abstract: The CMS detector will be upgraded for the HL-LHC to include a MIP Timing Detector (MTD). The MTD will consist of barrel and endcap timing layers, BTL and ETL respectively, providing precision timing of charged particles. The BTL sensors are based on LYSO:Ce scintillation crystals coupled to SiPMs with TOFHIR2 ASICs for the front-end readout. A resolution of 30-60 ps for MIP signals at a rate of 2.5 Mhit/s per channel is expected along the HL-LHC lifetime. We present an overview of the TOFHIR2 requirements and design, simulation results and measurements with TOFHIR2 ASICs. The measurements of TOFHIR2 associated to sensor modules were performed in different test setups using internal test pulses or blue and UV laser pulses emulating the signals expected in the experiment. The measurements show a time resolution of 24 ps initially during Beginning of Operation (BoO) and 58 ps at End of Operation (EoO) conditions, matching well the BTL requirements. We also showed that the time resolution is stable up to the highest expected MIP rate. Extensive radiation tests were performed, both with x-rays and heavy ions, showing that TOFHIR2 is not affected by the radiation environment during the experiment lifetime.


title: Hundreds of grocery outlets needed across the United States to achieve walkable cities

abstract: The notion of the $x$-minute city is again popular in urban planning, but the practical implications of developing walkable neighborhoods have not been rigorously explored. What is the scale of the challenge that cities needing to retrofit face? Where should new stores or amenities be located? For 500 cities in the United States, we explored how many additional supermarkets would be required to achieve various levels of $x$-minute access and where new stores should be located so that this access is equally-distributed. Our method is unique because it combines a novel measure of equality with a new model that optimally locates amenities for inequality-minimizing community access. We found that 25% of the studied cities could reach 15-minute access by adding five or fewer stores, while only 10% of the cities could even achieve 5-minute average access when using neighborhood centroids as potential sites; the cities that could, on average, required more than 100 stores each. This work provides a tool for cities to use evidenced-based planning to efficiently retrofit in order to enable active transport, benefiting both the climate and their residents' health. It also highlights the major challenge facing our cities due to the existing and ongoing car-dependent urban design that renders these goals unfeasible.


title: AILS-NTUA at SemEval-2024 Task 6: Efficient model tuning for hallucination detection and analysis

abstract: In this paper, we present our team's submissions for SemEval-2024 Task-6 - SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes. The participants were asked to perform binary classification to identify cases of fluent overgeneration hallucinations. Our experimentation included fine-tuning a pre-trained model on hallucination detection and a Natural Language Inference (NLI) model. The most successful strategy involved creating an ensemble of these models, resulting in accuracy rates of 77.8% and 79.9% on model-agnostic and model-aware datasets respectively, outperforming the organizers' baseline and achieving notable results when contrasted with the top-performing results in the competition, which reported accuracies of 84.7% and 81.3% correspondingly.


title: Non-Hermitian unidirectional routing of photonic qubits

abstract: Efficient and tunable qubit unidirectional routers and spin-wave diodes play an important role in both classical and quantum information processing domains. Here, we reveal that multi-level neutral cold atoms can mediate both dissipative and coherent couplings. Interestingly, we investigate and practically implement this paradigm in experiments, successfully synthesizing a system with dual functionality as both a photonic qubit unidirectional router and a spin-wave diode. By manipulating the helicity of the field, we can effectively balance the coherence coupling and dissipative channel, thereby ensuring the unidirectional transfer of photonic qubits. The qubit fidelity exceeds 97.49%, and the isolation ratio achieves $16.8\pm0.11$ dB while the insertion loss is lower than 0.36 dB. Furthermore, we show that the spin-wave diode can effectively achieve unidirectional information transfer by appropriately setting the coherent coupling parameters. Our work not only provides new ideas for the design of extensive components in quantum networks, but also opens up new possibilities for non-Hermitian quantum physics, complex quantum networks, and unidirectional quantum information transfer.


title: Maximal Secret Reconstruction, Teleportation and Bell's Inequality

abstract: A tripartite state is said to be a potential resource for secret sharing if the state imposes restrictions on the teleportation fidelity of the bipartite dealer--reconstructor and dealer--assistant channels in addition of being useful for the state reconstruction. Given a secret shareable state in a pure three-qubit system, we are able to characterize the set of states with maximum possible reconstruction fidelity (abbreviated as MSR states) for a fixed value of the maximum teleportation fidelity that can be obtained out of both the dealer--receiver channels. Similarly for a value giving the maximum of Bell-CHSH value of both dealer--reconstructor and dealer--assistant channels, we are able to find the maximum achievable reconstruction fidelity. Interestingly, we find that all secret shareable states satisfy Bell's inequality in both dealer--reconstructor and dealer--assistant partitions. This brings out a new mutual exclusivity between secret shareable state and Bell's inequality violations. Our result paves the way in identifying the best candidate among the secret sharing resource states in achieving the maximum reconstruction fidelity thus by setting the practical information transfer limit in a possible resource theoretic extension of secret sharing. It also brings out a new kind of mutual exclusiveness between the bipartite correlation and in the ability of secret sharing in a tripartite setting.


title: Bifurcation on Fully Nonlinear Elliptic Equations and Systems

abstract: In this paper, we study the following fully nonlinear elliptic equations \begin{equation*} \left\{\begin{array}{rl} \left(S_{k}(D^{2}u)\right)^{\frac1k}=\lambda f(-u) & in\quad\Omega \\ u=0 & on\quad \partial\Omega\\ \end{array} \right. \end{equation*} and coupled systems \begin{equation*} \left\{\begin{array}{rl} (S_{k}(D^{2}u))^\frac1k=\lambda g(-u,-v) & in\quad\Omega \\ (S_{k}(D^{2}v))^\frac1k=\lambda h(-u,-v) & in\quad\Omega \\ u=v=0 & on\quad \partial\Omega\\ \end{array} \right. \end{equation*} dominated by $k$-Hessian operators, where $\Omega$ is a $(k$-$1)$-convex bounded domain in $\mathbb{R}^{N}$, $\lambda$ is a non-negative parameter, $f:\left[0,+\infty\right)\rightarrow\left[0,+\infty\right)$ is a continuous function with zeros only at $0$ and $g,h:\left[0,+\infty\right)\times \left[0,+\infty\right)\rightarrow \left[0,+\infty\right)$ are continuous functions with zeros only at $(\cdot,0)$ and $(0,\cdot)$. We determine the interval of $\lambda$ about the existence, non-existence, uniqueness and multiplicity of $k$-convex solutions to the above problems according to various cases of $f,g,h$, which is a complete supplement to the known results in previous literature. In particular, the above results are also new for Laplacian and Monge-Amp\`ere operators. We mainly use bifurcation theory, a-priori estimates, various maximum principles and technical strategies in the proof.


title: High-temperature domain wall current in Mg-doped lithium niobate single crystals up to 400{\deg}C

abstract: Conductive ferroelectric domain walls (DWs) represent a promising topical system for the development of nanoelectronic components and devices. DWs show very different properties as compared to their bulk counterparts. Of central interest here is the domain wall current (DWC) of charged DWs in 5mol\% Mg-doped lithium niobate single crystals; in contrast to former works, we extend the DWC study here to temperatures as high as 400$^\circ$C. Both the temporal stability and the thermal activation energies of 90 - 160 meV are readily deduced from current-voltage sweeps as recorded over multiple heating cycles. Our experimental work is backed up by atomistic modelling of the DWC. The latter suggests that a large band bending renders head-to-head and tail-to-tail DWs semimetallic. These detailed investigations underline the potential to extend DWC-based nanoelectronic applications even into the so-far unexplored high-temperature regime.


title: Miura transformations and large-time behaviors of the Hirota-Satsuma equation

abstract: The good Boussinesq equation has several modified versions such as the modified Boussinesq equation, Mikhailov-Lenells equation and Hirota-Satsuma equation. This work builds the full relations among these equations by Miura transformation and invertible linear transformations and draws a pyramid diagram to demonstrate such relations. The direct and inverse spectral analysis shows that the solution of Riemann-Hilbert problem for Hirota-Satsuma equation has simple pole at origin, the solution of Riemann-Hilbert problem for the good Boussinesq equation has double pole at origin, while the solution of Riemann-Hilbert problem for the modified Boussinesq equation and Mikhailov-Lenells equation doesn't have singularity at origin. Further, the large-time asymptotic behaviors of the Hirota-Satsuma equation with Schwartz class initial value is studied by Deift-Zhou nonlinear steepest descent analysis. In such initial condition, the asymptotic expressions of the Hirota-Satsuma equation and good Boussinesq equation away from the origin are proposed and it is displayed that the leading term of asymptotic formulas match well with direct numerical simulations.


title: Novel Node Category Detection Under Subpopulation Shift

abstract: In real-world graph data, distribution shifts can manifest in various ways, such as the emergence of new categories and changes in the relative proportions of existing categories. It is often important to detect nodes of novel categories under such distribution shifts for safety or insight discovery purposes. We introduce a new approach, Recall-Constrained Optimization with Selective Link Prediction (RECO-SLIP), to detect nodes belonging to novel categories in attributed graphs under subpopulation shifts. By integrating a recall-constrained learning framework with a sample-efficient link prediction mechanism, RECO-SLIP addresses the dual challenges of resilience against subpopulation shifts and the effective exploitation of graph structure. Our extensive empirical evaluation across multiple graph datasets demonstrates the superior performance of RECO-SLIP over existing methods.


title: Incorporating Domain Differential Equations into Graph Convolutional Networks to Lower Generalization Discrepancy

abstract: Ensuring both accuracy and robustness in time series prediction is critical to many applications, ranging from urban planning to pandemic management. With sufficient training data where all spatiotemporal patterns are well-represented, existing deep-learning models can make reasonably accurate predictions. However, existing methods fail when the training data are drawn from different circumstances (e.g., traffic patterns on regular days) compared to test data (e.g., traffic patterns after a natural disaster). Such challenges are usually classified under domain generalization. In this work, we show that one way to address this challenge in the context of spatiotemporal prediction is by incorporating domain differential equations into Graph Convolutional Networks (GCNs). We theoretically derive conditions where GCNs incorporating such domain differential equations are robust to mismatched training and testing data compared to baseline domain agnostic models. To support our theory, we propose two domain-differential-equation-informed networks called Reaction-Diffusion Graph Convolutional Network (RDGCN), which incorporates differential equations for traffic speed evolution, and Susceptible-Infectious-Recovered Graph Convolutional Network (SIRGCN), which incorporates a disease propagation model. Both RDGCN and SIRGCN are based on reliable and interpretable domain differential equations that allow the models to generalize to unseen patterns. We experimentally show that RDGCN and SIRGCN are more robust with mismatched testing data than the state-of-the-art deep learning methods.


title: Towards System Modelling to Support Diseases Data Extraction from the Electronic Health Records for Physicians Research Activities

abstract: The use of Electronic Health Records (EHRs) has increased dramatically in the past 15 years, as, it is considered an important source of managing data od patients. The EHRs are primary sources of disease diagnosis and demographic data of patients worldwide. Therefore, the data can be utilized for secondary tasks such as research. This paper aims to make such data usable for research activities such as monitoring disease statistics for a specific population. As a result, the researchers can detect the disease causes for the behavior and lifestyle of the target group. One of the limitations of EHRs systems is that the data is not available in the standard format but in various forms. Therefore, it is required to first convert the names of the diseases and demographics data into one standardized form to make it usable for research activities. There is a large amount of EHRs available, and solving the standardizing issues requires some optimized techniques. We used a first-hand EHR dataset extracted from EHR systems. Our application uploads the dataset from the EHRs and converts it to the ICD-10 coding system to solve the standardization problem. So, we first apply the steps of pre-processing, annotation, and transforming the data to convert it into the standard form. The data pre-processing is applied to normalize demographic formats. In the annotation step, a machine learning model is used to recognize the diseases from the text. Furthermore, the transforming step converts the disease name to the ICD-10 coding format. The model was evaluated manually by comparing its performance in terms of disease recognition with an available dictionary-based system (MetaMap). The accuracy of the proposed machine learning model is 81%, that outperformed MetaMap accuracy of 67%. This paper contributed to system modelling for EHR data extraction to support research activities.


title: LTL-D*: Incrementally Optimal Replanning for Feasible and Infeasible Tasks in Linear Temporal Logic Specifications

abstract: This paper presents an incremental replanning algorithm, dubbed LTL-D*, for temporal-logic-based task planning in a dynamically changing environment. Unexpected changes in the environment may lead to failures in satisfying a task specification in the form of a Linear Temporal Logic (LTL). In this study, the considered failures are categorized into two classes: (i) the desired LTL specification can be satisfied via replanning, and (ii) the desired LTL specification is infeasible to meet strictly and can only be satisfied in a "relaxed" fashion. To address these failures, the proposed algorithm finds an optimal replanning solution that minimally violates desired task specifications. In particular, our approach leverages the D* Lite algorithm and employs a distance metric within the synthesized automaton to quantify the degree of the task violation and then replan incrementally. This ensures plan optimality and reduces planning time, especially when frequent replanning is required. Our approach is implemented in a robot navigation simulation to demonstrate a significant improvement in the computational efficiency for replanning by two orders of magnitude.


title: Entity-Centric Reinforcement Learning for Object Manipulation from Pixels

abstract: Manipulating objects is a hallmark of human intelligence, and an important task in domains such as robotics. In principle, Reinforcement Learning (RL) offers a general approach to learn object manipulation. In practice, however, domains with more than a few objects are difficult for RL agents due to the curse of dimensionality, especially when learning from raw image observations. In this work we propose a structured approach for visual RL that is suitable for representing multiple objects and their interaction, and use it to learn goal-conditioned manipulation of several objects. Key to our method is the ability to handle goals with dependencies between the objects (e.g., moving objects in a certain order). We further relate our architecture to the generalization capability of the trained agent, based on a theoretical result for compositional generalization, and demonstrate agents that learn with 3 objects but generalize to similar tasks with over 10 objects. Videos and code are available on the project website: https://sites.google.com/view/entity-centric-rl


title: Upper bound for the quantum coupling between free electrons and photons

abstract: The quantum interaction between free electrons and photons is fundamental to free-electron based light sources and free-electron quantum optics applications. A large coupling between free electrons and photons is generally desired. In this manuscript, I study the upper bound for the quantum coupling between free electrons and photons. Our study provides guidance to reach the strong coupling between free electrons and photons.


title: Nonfactorizable charming-loop contribution to FCNC $B_s\to \gamma l^+l^-$ decay

abstract: We present the first theoretical calculation of nonfactorizable charm-quark loop contributions to the $B_s\to \gamma l^+l^-$ amplitude. We calculate the relevant form factors, $H_{A,V}^{\rm NF}(k'^2,k^2)$, and provide convenient parametrizations of our results in the form of fit functions of two variables, $k'^2$ and $k^2$, applicable in the region below hadron resonances, $k'^2 < M_{J/\psi}^2$ and $k^2 < M_{\phi}^2$. We report that factorizable and nonfactorizable charm contributions to the $B_s\to\gamma l^+l^-$ amplitude have opposite signs. To compare the charm and the top contributions, it is convenient to express the NF charming loop contribution as a non-universal (i.e., dependent on the reaction) $q^2$-dependent correction $\Delta^{\rm NF}C_7(q^2)$ to the Wilson coefficient $C_7$. For the $B_s\to\gamma l^+l^-$ amplitude, the correction is found to be positive, $\Delta^{\rm NF} C_7(q^2)/C_7 > 0$.


title: Feature Splatting: Language-Driven Physics-Based Scene Synthesis and Editing

abstract: Scene representations using 3D Gaussian primitives have produced excellent results in modeling the appearance of static and dynamic 3D scenes. Many graphics applications, however, demand the ability to manipulate both the appearance and the physical properties of objects. We introduce Feature Splatting, an approach that unifies physics-based dynamic scene synthesis with rich semantics from vision language foundation models that are grounded by natural language. Our first contribution is a way to distill high-quality, object-centric vision-language features into 3D Gaussians, that enables semi-automatic scene decomposition using text queries. Our second contribution is a way to synthesize physics-based dynamics from an otherwise static scene using a particle-based simulator, in which material properties are assigned automatically via text queries. We ablate key techniques used in this pipeline, to illustrate the challenge and opportunities in using feature-carrying 3D Gaussians as a unified format for appearance, geometry, material properties and semantics grounded on natural language. Project website: https://feature-splatting.github.io/


title: Collaborative Pareto Set Learning in Multiple Multi-Objective Optimization Problems

abstract: Pareto Set Learning (PSL) is an emerging research area in multi-objective optimization, focusing on training neural networks to learn the mapping from preference vectors to Pareto optimal solutions. However, existing PSL methods are limited to addressing a single Multi-objective Optimization Problem (MOP) at a time. When faced with multiple MOPs, this limitation results in significant inefficiencies and hinders the ability to exploit potential synergies across varying MOPs. In this paper, we propose a Collaborative Pareto Set Learning (CoPSL) framework, which learns the Pareto sets of multiple MOPs simultaneously in a collaborative manner. CoPSL particularly employs an architecture consisting of shared and MOP-specific layers. The shared layers are designed to capture commonalities among MOPs collaboratively, while the MOP-specific layers tailor these general insights to generate solution sets for individual MOPs. This collaborative approach enables CoPSL to efficiently learn the Pareto sets of multiple MOPs in a single execution while leveraging the potential relationships among various MOPs. To further understand these relationships, we experimentally demonstrate that shareable representations exist among MOPs. Leveraging these shared representations effectively improves the capability to approximate Pareto sets. Extensive experiments underscore the superior efficiency and robustness of CoPSL in approximating Pareto sets compared to state-of-the-art approaches on a variety of synthetic and real-world MOPs. Code is available at https://github.com/ckshang/CoPSL.


title: SurMo: Surface-based 4D Motion Modeling for Dynamic Human Rendering

abstract: Dynamic human rendering from video sequences has achieved remarkable progress by formulating the rendering as a mapping from static poses to human images. However, existing methods focus on the human appearance reconstruction of every single frame while the temporal motion relations are not fully explored. In this paper, we propose a new 4D motion modeling paradigm, SurMo, that jointly models the temporal dynamics and human appearances in a unified framework with three key designs: 1) Surface-based motion encoding that models 4D human motions with an efficient compact surface-based triplane. It encodes both spatial and temporal motion relations on the dense surface manifold of a statistical body template, which inherits body topology priors for generalizable novel view synthesis with sparse training observations. 2) Physical motion decoding that is designed to encourage physical motion learning by decoding the motion triplane features at timestep t to predict both spatial derivatives and temporal derivatives at the next timestep t+1 in the training stage. 3) 4D appearance decoding that renders the motion triplanes into images by an efficient volumetric surface-conditioned renderer that focuses on the rendering of body surfaces with motion learning conditioning. Extensive experiments validate the state-of-the-art performance of our new paradigm and illustrate the expressiveness of surface-based motion triplanes for rendering high-fidelity view-consistent humans with fast motions and even motion-dependent shadows. Our project page is at: https://taohuumd.github.io/projects/SurMo/