title: Computer Vision for Increased Operative Efficiency via Identification of Instruments in the Neurosurgical Operating Room: A Proof-of-Concept Study

abstract: Objectives Computer vision (CV) is a field of artificial intelligence that enables machines to interpret and understand images and videos. CV has the potential to be of assistance in the operating room (OR) to track surgical instruments. We built a CV algorithm for identifying surgical instruments in the neurosurgical operating room as a potential solution for surgical instrument tracking and management to decrease surgical waste and opening of unnecessary tools. Methods We collected 1660 images of 27 commonly used neurosurgical instruments. Images were labeled using the VGG Image Annotator and split into 80% training and 20% testing sets in order to train a U-Net Convolutional Neural Network using 5-fold cross validation. Results Our U-Net achieved a tool identification accuracy of 80-100% when distinguishing 25 classes of instruments, with 19/25 classes having accuracy over 90%. The model performance was not adequate for sub classifying Adson, Gerald, and Debakey forceps, which had accuracies of 60-80%. Conclusions We demonstrated the viability of using machine learning to accurately identify surgical instruments. Instrument identification could help optimize surgical tray packing, decrease tool usage and waste, decrease incidence of instrument misplacement events, and assist in timing of routine instrument maintenance. More training data will be needed to increase accuracy across all surgical instruments that would appear in a neurosurgical operating room. Such technology has the potential to be used as a method to be used for proving what tools are truly needed in each type of operation allowing surgeons across the world to do more with less.


title: The mechanistic basis of data dependence and abrupt learning in an in-context classification task

abstract: Transformer models exhibit in-context learning: the ability to accurately predict the response to a novel query based on illustrative examples in the input sequence. In-context learning contrasts with traditional in-weights learning of query-output relationships. What aspects of the training data distribution and architecture favor in-context vs in-weights learning? Recent work has shown that specific distributional properties inherent in language, such as burstiness, large dictionaries and skewed rank-frequency distributions, control the trade-off or simultaneous appearance of these two forms of learning. We first show that these results are recapitulated in a minimal attention-only network trained on a simplified dataset. In-context learning (ICL) is driven by the abrupt emergence of an induction head, which subsequently competes with in-weights learning. By identifying progress measures that precede in-context learning and targeted experiments, we construct a two-parameter model of an induction head which emulates the full data distributional dependencies displayed by the attention-based network. A phenomenological model of induction head formation traces its abrupt emergence to the sequential learning of three nested logits enabled by an intrinsic curriculum. We propose that the sharp transitions in attention-based networks arise due to a specific chain of multi-layer operations necessary to achieve ICL, which is implemented by nested nonlinearities sequentially learned during training.


title: Explore, Select, Derive, and Recall: Augmenting LLM with Human-like Memory for Mobile Task Automation

abstract: The advent of large language models (LLMs) has opened up new opportunities in the field of mobile task automation. Their superior language understanding and reasoning capabilities allow users to automate complex and repetitive tasks. However, due to the inherent unreliability and high operational cost of LLMs, their practical applicability is quite limited. To address these issues, this paper introduces MobileGPT, an innovative LLM-based mobile task automator equipped with a human-like app memory. MobileGPT emulates the cognitive process of humans interacting with a mobile app -- explore, select, derive, and recall. This approach allows for a more precise and efficient learning of a task's procedure by breaking it down into smaller, modular sub-tasks that can be re-used, re-arranged, and adapted for various objectives. We implement MobileGPT using online LLMs services (GPT-3.5 and GPT-4) and evaluate its performance on a dataset of 160 user instructions across 8 widely used mobile apps. The results indicate that MobileGPT can automate and learn new tasks with 82.5% accuracy, and is able to adapt them to different contexts with near perfect (98.75%) accuracy while reducing both latency and cost by 62.5% and 68.8%, respectively, compared to the GPT-4 powered baseline.


title: Learning Multi-graph Structure for Temporal Knowledge Graph Reasoning

abstract: Temporal Knowledge Graph (TKG) reasoning that forecasts future events based on historical snapshots distributed over timestamps is denoted as extrapolation and has gained significant attention. Owing to its extreme versatility and variation in spatial and temporal correlations, TKG reasoning presents a challenging task, demanding efficient capture of concurrent structures and evolutional interactions among facts. While existing methods have made strides in this direction, they still fall short of harnessing the diverse forms of intrinsic expressive semantics of TKGs, which encompass entity correlations across multiple timestamps and periodicity of temporal information. This limitation constrains their ability to thoroughly reflect historical dependencies and future trends. In response to these drawbacks, this paper proposes an innovative reasoning approach that focuses on Learning Multi-graph Structure (LMS). Concretely, it comprises three distinct modules concentrating on multiple aspects of graph structure knowledge within TKGs, including concurrent and evolutional patterns along timestamps, query-specific correlations across timestamps, and semantic dependencies of timestamps, which capture TKG features from various perspectives. Besides, LMS incorporates an adaptive gate for merging entity representations both along and across timestamps effectively. Moreover, it integrates timestamp semantics into graph attention calculations and time-aware decoders, in order to impose temporal constraints on events and narrow down prediction scopes with historical statistics. Extensive experimental results on five event-based benchmark datasets demonstrate that LMS outperforms state-of-the-art extrapolation models, indicating the superiority of modeling a multi-graph perspective for TKG reasoning.


title: Few-Shot Anomaly Detection with Adversarial Loss for Robust Feature Representations

abstract: Anomaly detection is a critical and challenging task that aims to identify data points deviating from normal patterns and distributions within a dataset. Various methods have been proposed using a one-class-one-model approach, but these techniques often face practical problems such as memory inefficiency and the requirement of sufficient data for training. In particular, few-shot anomaly detection presents significant challenges in industrial applications, where limited samples are available before mass production. In this paper, we propose a few-shot anomaly detection method that integrates adversarial training loss to obtain more robust and generalized feature representations. We utilize the adversarial loss previously employed in domain adaptation to align feature distributions between source and target domains, to enhance feature robustness and generalization in few-shot anomaly detection tasks. We hypothesize that adversarial loss is effective when applied to features that should have similar characteristics, such as those from the same layer in a Siamese network's parallel branches or input-output pairs of reconstruction-based methods. Experimental results demonstrate that the proposed method generally achieves better performance when utilizing the adversarial loss.


title: Multi-Weight Ranking for Multi-Criteria Decision Making

abstract: Cone distribution functions from statistics are turned into Multi-Criteria Decision Making tools. It is demonstrated that this procedure can be considered as an upgrade of the weighted sum scalarization insofar as it absorbs a whole collection of weighted sum scalarizations at once instead of fixing a particular one in advance. As examples show, this type of scalarization--in contrast to a pure weighted sum scalarization-is also able to detect ``non-convex" parts of the Pareto frontier. Situations are characterized in which different types of rank reversal occur, and it is explained why this might even be useful for analyzing the ranking procedure. The ranking functions are then extended to sets providing unary indicators for set preferences which establishes, for the first time, the link between set optimization methods and set-based multi-objective optimization. A potential application in machine learning is outlined.


title: Modelling JWST mid-infrared counts: excellent consistency with models derived for IRAS, ISO and Spitzer

abstract: Models derived in 2009 to fit mid-infrared (8-24 micron) source counts from the IRAS, ISO and Spitzer missions, provide an excellent fit to deep counts with JWST, demonstrating that the evolution of dusty star-forming galaxies is well understood. The evolution of dust in galaxies at high redshifts is discussed and a simple prescription is proposed to model this. This allows more realistic models for source-counts at submillimetre wavelength. A reasonable fit to 250, 500, 850 and 1100 micron counts is obtained. This paper therefore draws together the IRAS, ISO, Spitzer, Akari, Herschel, submillimetre ground-based, and JWST surveys into a single picture.


title: Deep Reinforcement Learning for Community Battery Scheduling under Uncertainties of Load, PV Generation, and Energy Prices

abstract: In response to the growing uptake of distributed energy resources (DERs), community batteries have emerged as a promising solution to support renewable energy integration, reduce peak load, and enhance grid reliability. This paper presents a deep reinforcement learning (RL) strategy, centered around the soft actor-critic (SAC) algorithm, to schedule a community battery system in the presence of uncertainties, such as solar photovoltaic (PV) generation, local demand, and real-time energy prices. We position the community battery to play a versatile role, in integrating local PV energy, reducing peak load, and exploiting energy price fluctuations for arbitrage, thereby minimizing the system cost. To improve exploration and convergence during RL training, we utilize the noisy network technique. This paper conducts a comparative study of different RL algorithms, including proximal policy optimization (PPO) and deep deterministic policy gradient (DDPG) algorithms, to evaluate their effectiveness in the community battery scheduling problem. The results demonstrate the potential of RL in addressing community battery scheduling challenges and show that the SAC algorithm achieves the best performance compared to RL and optimization benchmarks.


title: I-PHYRE: Interactive Physical Reasoning

abstract: Current evaluation protocols predominantly assess physical reasoning in stationary scenes, creating a gap in evaluating agents' abilities to interact with dynamic events. While contemporary methods allow agents to modify initial scene configurations and observe consequences, they lack the capability to interact with events in real time. To address this, we introduce I-PHYRE, a framework that challenges agents to simultaneously exhibit intuitive physical reasoning, multi-step planning, and in-situ intervention. Here, intuitive physical reasoning refers to a quick, approximate understanding of physics to address complex problems; multi-step denotes the need for extensive sequence planning in I-PHYRE, considering each intervention can significantly alter subsequent choices; and in-situ implies the necessity for timely object manipulation within a scene, where minor timing deviations can result in task failure. We formulate four game splits to scrutinize agents' learning and generalization of essential principles of interactive physical reasoning, fostering learning through interaction with representative scenarios. Our exploration involves three planning strategies and examines several supervised and reinforcement agents' zero-shot generalization proficiency on I-PHYRE. The outcomes highlight a notable gap between existing learning algorithms and human performance, emphasizing the imperative for more research in enhancing agents with interactive physical reasoning capabilities. The environment and baselines will be made publicly available.


title: Mod $p$ Buchstaber invariant

abstract: We investigate the mod $p$ Buchstaber invariant of the skeleta of simplices, for a prime number $p$, and compare them for different values of $p$. For $p=2$, the invariant is the real Buchstaber invariant. Our findings reveal that these values are generally distinct. Additionally, we determine or estimate the mod $p$ Buchstaber invariants of certain universal simplicial complexes $X(\F_p^n)$.


title: InstructBooth: Instruction-following Personalized Text-to-Image Generation

abstract: Personalizing text-to-image models using a limited set of images for a specific object has been explored in subject-specific image generation. However, existing methods often face challenges in aligning with text prompts due to overfitting to the limited training images. In this work, we introduce InstructBooth, a novel method designed to enhance image-text alignment in personalized text-to-image models without sacrificing the personalization ability. Our approach first personalizes text-to-image models with a small number of subject-specific images using a unique identifier. After personalization, we fine-tune personalized text-to-image models using reinforcement learning to maximize a reward that quantifies image-text alignment. Additionally, we propose complementary techniques to increase the synergy between these two processes. Our method demonstrates superior image-text alignment compared to existing baselines, while maintaining high personalization ability. In human evaluations, InstructBooth outperforms them when considering all comprehensive factors. Our project page is at https://sites.google.com/view/instructbooth.


title: A Waddington landscape for prototype learning in generalized Hopfield networks

abstract: Networks in machine learning offer examples of complex high-dimensional dynamical systems reminiscent of biological systems. Here, we study the learning dynamics of Generalized Hopfield networks, which permit a visualization of internal memories. These networks have been shown to proceed through a 'feature-to-prototype' transition, as the strength of network nonlinearity is increased, wherein the learned, or terminal, states of internal memories transition from mixed to pure states. Focusing on the prototype learning dynamics of the internal memories we observe a strong resemblance to the canalized, or low-dimensional, dynamics of cells as they differentiate within a Waddingtonian landscape. Dynamically, we demonstrate that learning in a Generalized Hopfield Network proceeds through sequential 'splits' in memory space. Furthermore, order of splitting is interpretable and reproducible. The dynamics between the splits are canalized in the Waddington sense -- robust to variations in detailed aspects of the system. In attempting to make the analogy a rigorous equivalence, we study smaller subsystems that exhibit similar properties to the full system. We combine analytical calculations with numerical simulations to study the dynamical emergence of the feature-to-prototype transition, and the behaviour of splits in the landscape, saddles points, visited during learning. We exhibit regimes where saddles appear and disappear through saddle-node bifurcations, qualitatively changing the distribution of learned memories as the strength of the nonlinearity is varied -- allowing us to systematically investigate the mechanisms that underlie the emergence of Waddingtonian dynamics. Memories can thus differentiate in a predictive and controlled way, revealing new bridges between experimental biology, dynamical systems theory, and machine learning.


title: Breast Ultrasound Report Generation using LangChain

abstract: Breast ultrasound (BUS) is a critical diagnostic tool in the field of breast imaging, aiding in the early detection and characterization of breast abnormalities. Interpreting breast ultrasound images commonly involves creating comprehensive medical reports, containing vital information to promptly assess the patient's condition. However, the ultrasound imaging system necessitates capturing multiple images of various parts to compile a single report, presenting a time-consuming challenge. To address this problem, we propose the integration of multiple image analysis tools through a LangChain using Large Language Models (LLM), into the breast reporting process. Through a combination of designated tools and text generation through LangChain, our method can accurately extract relevant features from ultrasound images, interpret them in a clinical context, and produce comprehensive and standardized reports. This approach not only reduces the burden on radiologists and healthcare professionals but also enhances the consistency and quality of reports. The extensive experiments shows that each tools involved in the proposed method can offer qualitatively and quantitatively significant results. Furthermore, clinical evaluation on the generated reports demonstrates that the proposed method can make report in clinically meaningful way.


title: Foundation Models for Weather and Climate Data Understanding: A Comprehensive Survey

abstract: As artificial intelligence (AI) continues to rapidly evolve, the realm of Earth and atmospheric sciences is increasingly adopting data-driven models, powered by progressive developments in deep learning (DL). Specifically, DL techniques are extensively utilized to decode the chaotic and nonlinear aspects of Earth systems, and to address climate challenges via understanding weather and climate data. Cutting-edge performance on specific tasks within narrower spatio-temporal scales has been achieved recently through DL. The rise of large models, specifically large language models (LLMs), has enabled fine-tuning processes that yield remarkable outcomes across various downstream tasks, thereby propelling the advancement of general AI. However, we are still navigating the initial stages of crafting general AI for weather and climate. In this survey, we offer an exhaustive, timely overview of state-of-the-art AI methodologies specifically engineered for weather and climate data, with a special focus on time series and text data. Our primary coverage encompasses four critical aspects: types of weather and climate data, principal model architectures, model scopes and applications, and datasets for weather and climate. Furthermore, in relation to the creation and application of foundation models for weather and climate data understanding, we delve into the field's prevailing challenges, offer crucial insights, and propose detailed avenues for future research. This comprehensive approach equips practitioners with the requisite knowledge to make substantial progress in this domain. Our survey encapsulates the most recent breakthroughs in research on large, data-driven models for weather and climate data understanding, emphasizing robust foundations, current advancements, practical applications, crucial resources, and prospective research opportunities.


title: PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimation

abstract: Open-world 3D part segmentation is pivotal in diverse applications such as robotics and AR/VR. Traditional supervised methods often grapple with limited 3D data availability and struggle to generalize to unseen object categories. PartSLIP, a recent advancement, has made significant strides in zero- and few-shot 3D part segmentation. This is achieved by harnessing the capabilities of the 2D open-vocabulary detection module, GLIP, and introducing a heuristic method for converting and lifting multi-view 2D bounding box predictions into 3D segmentation masks. In this paper, we introduce PartSLIP++, an enhanced version designed to overcome the limitations of its predecessor. Our approach incorporates two major improvements. First, we utilize a pre-trained 2D segmentation model, SAM, to produce pixel-wise 2D segmentations, yielding more precise and accurate annotations than the 2D bounding boxes used in PartSLIP. Second, PartSLIP++ replaces the heuristic 3D conversion process with an innovative modified Expectation-Maximization algorithm. This algorithm conceptualizes 3D instance segmentation as unobserved latent variables, and then iteratively refines them through an alternating process of 2D-3D matching and optimization with gradient descent. Through extensive evaluations, we show that PartSLIP++ demonstrates better performance over PartSLIP in both low-shot 3D semantic and instance-based object part segmentation tasks. Code released at https://github.com/zyc00/PartSLIP2.


title: Protein Language Model-Powered 3D Ligand Binding Site Prediction from Protein Sequence

abstract: Prediction of ligand binding sites of proteins is a fundamental and important task for understanding the function of proteins and screening potential drugs. Most existing methods require experimentally determined protein holo-structures as input. However, such structures can be unavailable on novel or less-studied proteins. To tackle this limitation, we propose LaMPSite, which only takes protein sequences and ligand molecular graphs as input for ligand binding site predictions. The protein sequences are used to retrieve residue-level embeddings and contact maps from the pre-trained ESM-2 protein language model. The ligand molecular graphs are fed into a graph neural network to compute atom-level embeddings. Then we compute and update the protein-ligand interaction embedding based on the protein residue-level embeddings and ligand atom-level embeddings, and the geometric constraints in the inferred protein contact map and ligand distance map. A final pooling on protein-ligand interaction embedding would indicate which residues belong to the binding sites. Without any 3D coordinate information of proteins, our proposed model achieves competitive performance compared to baseline methods that require 3D protein structures when predicting binding sites. Given that less than 50% of proteins have reliable structure information in the current stage, LaMPSite will provide new opportunities for drug discovery.


title: AI-driven emergence of frequency information non-uniform distribution via THz metasurface spectrum prediction

abstract: Recently, artificial intelligence has been extensively deployed across various scientific disciplines, optimizing and guiding the progression of experiments through the integration of abundant datasets, whilst continuously probing the vast theoretical space encapsulated within the data. Particularly, deep learning models, due to their end-to-end adaptive learning capabilities, are capable of autonomously learning intrinsic data features, thereby transcending the limitations of traditional experience to a certain extent. Here, we unveil previously unreported information characteristics pertaining to different frequencies emerged during our work on predicting the terahertz spectral modulation effects of metasurfaces based on AI-prediction. Moreover, we have substantiated that our proposed methodology of simply adding supplementary multi-frequency inputs to the existing dataset during the target spectral prediction process can significantly enhance the predictive accuracy of the network. This approach effectively optimizes the utilization of existing datasets and paves the way for interdisciplinary research and applications in artificial intelligence, chemistry, composite material design, biomedicine, and other fields.


title: DreamVideo: High-Fidelity Image-to-Video Generation with Image Retention and Text Guidance

abstract: Image-to-video generation, which aims to generate a video starting from a given reference image, has drawn great attention. Existing methods try to extend pre-trained text-guided image diffusion models to image-guided video generation models. Nevertheless, these methods often result in either low fidelity or flickering over time due to their limitation to shallow image guidance and poor temporal consistency. To tackle these problems, we propose a high-fidelity image-to-video generation method by devising a frame retention branch based on a pre-trained video diffusion model, named DreamVideo. Instead of integrating the reference image into the diffusion process at a semantic level, our DreamVideo perceives the reference image via convolution layers and concatenates the features with the noisy latents as model input. By this means, the details of the reference image can be preserved to the greatest extent. In addition, by incorporating double-condition classifier-free guidance, a single image can be directed to videos of different actions by providing varying prompt texts. This has significant implications for controllable video generation and holds broad application prospects. We conduct comprehensive experiments on the public dataset, and both quantitative and qualitative results indicate that our method outperforms the state-of-the-art method. Especially for fidelity, our model has a powerful image retention ability and delivers the best results in UCF101 compared to other image-to-video models to our best knowledge. Also, precise control can be achieved by giving different text prompts. Further details and comprehensive results of our model will be presented in https://anonymous0769.github.io/DreamVideo/.


title: Towards Optimizations of Quantum Circuit Simulation for Solving Max-Cut Problems with QAOA

abstract: Quantum approximate optimization algorithm (QAOA) is one of the popular quantum algorithms that are used to solve combinatorial optimization problems via approximations. QAOA is able to be evaluated on both physical and virtual quantum computers simulated by classical computers, with virtual ones being favored for their noise-free feature and availability. Nevertheless, performing QAOA on virtual quantum computers suffers from a slow simulation speed for solving combinatorial optimization problems which require large-scale quantum circuit simulation (QCS). In this paper, we propose techniques to accelerate QCS for QAOA using mathematical optimizations to compress quantum operations, incorporating efficient bitwise operations to further lower the computational complexity, and leveraging different levels of parallelisms from modern multi-core processors, with a study case to show the effectiveness on solving max-cut problems.


title: Enhanced Breast Cancer Tumor Classification using MobileNetV2: A Detailed Exploration on Image Intensity, Error Mitigation, and Streamlit-driven Real-time Deployment

abstract: This research introduces a sophisticated transfer learning model based on Google's MobileNetV2 for breast cancer tumor classification into normal, benign, and malignant categories, utilizing a dataset of 1576 ultrasound images (265 normal, 891 benign, 420 malignant). The model achieves an accuracy of 0.82, precision of 0.83, recall of 0.81, ROC-AUC of 0.94, PR-AUC of 0.88, and MCC of 0.74. It examines image intensity distributions and misclassification errors, offering improvements for future applications. Addressing dataset imbalances, the study ensures a generalizable model. This work, using a dataset from Baheya Hospital, Cairo, Egypt, compiled by Walid Al-Dhabyani et al., emphasizes MobileNetV2's potential in medical imaging, aiming to improve diagnostic precision in oncology. Additionally, the paper explores Streamlit-based deployment for real-time tumor classification, demonstrating MobileNetV2's applicability in medical imaging and setting a benchmark for future research in oncology diagnostics.


title: Effective generation of Hecke algebras and explicit estimates of Sato--Tate type

abstract: Assuming the Riemann hypothesis for $L$-functions attached to primitive Dirichlet characters, modular cusp forms, and their tensor products and symmetric squares, we write down explicit finite sets of Hecke operators that span the Hecke algebras acting on the spaces of modular forms of weight greater than two with squarefree level.


title: Beyond Isolation: Multi-Agent Synergy for Improving Knowledge Graph Construction

abstract: Knowledge graph construction (KGC) is a multifaceted undertaking involving the extraction of entities, relations, and events. Traditionally, large language models (LLMs) have been viewed as solitary task-solving agents in this complex landscape. However, this paper challenges this paradigm by introducing a novel framework, CooperKGC. Departing from the conventional approach, CooperKGC establishes a collaborative processing network, assembling a KGC collaboration team capable of concurrently addressing entity, relation, and event extraction tasks. Our experiments unequivocally demonstrate that fostering collaboration and information interaction among diverse agents within CooperKGC yields superior results compared to individual cognitive processes operating in isolation. Importantly, our findings reveal that the collaboration facilitated by CooperKGC enhances knowledge selection, correction, and aggregation capabilities across multiple rounds of interactions.


title: A study of topological quantities of lattice QCD by a modified DCGAN frame

abstract: A modified deep convolutional generative adversarial network (M-DCGAN) frame is proposed to study the N-dimensional (ND) topological quantities in lattice QCD based on the Monte Carlo (MC) simulations. We construct a new scaling structure including fully connected layers to support the generation of high-quality high-dimensional images for the M-DCGAN. Our results show that the M-DCGAN scheme of the Machine learning should be helpful for us to calculate efficiently the 1D distribution of topological charge and the 4D topological charge density compared with the case by the MC simulation alone.


title: Role of Uncertainty in Anticipatory Trajectory Prediction for a Ping-Pong Playing Robot

abstract: Robotic interaction in fast-paced environments presents a substantial challenge, particularly in tasks requiring the prediction of dynamic, non-stationary objects for timely and accurate responses. An example of such a task is ping-pong, where the physical limitations of a robot may prevent it from reaching its goal in the time it takes the ball to cross the table. The scene of a ping-pong match contains rich visual information of a player's movement that can allow future game state prediction, with varying degrees of uncertainty. To this aim, we present a visual modeling, prediction, and control system to inform a ping-pong playing robot utilizing visual model uncertainty to allow earlier motion of the robot throughout the game. We present demonstrations and metrics in simulation to show the benefit of incorporating model uncertainty, the limitations of current standard model uncertainty estimators, and the need for more verifiable model uncertainty estimation. Our code is publicly available.


title: Training on Synthetic Data Beats Real Data in Multimodal Relation Extraction

abstract: The task of multimodal relation extraction has attracted significant research attention, but progress is constrained by the scarcity of available training data. One natural thought is to extend existing datasets with cross-modal generative models. In this paper, we consider a novel problem setting, where only unimodal data, either text or image, are available during training. We aim to train a multimodal classifier from synthetic data that perform well on real multimodal test data. However, training with synthetic data suffers from two obstacles: lack of data diversity and label information loss. To alleviate the issues, we propose Mutual Information-aware Multimodal Iterated Relational dAta GEneration (MI2RAGE), which applies Chained Cross-modal Generation (CCG) to promote diversity in the generated data and exploits a teacher network to select valuable training samples with high mutual information with the ground-truth labels. Comparing our method to direct training on synthetic data, we observed a significant improvement of 24.06% F1 with synthetic text and 26.42% F1 with synthetic images. Notably, our best model trained on completely synthetic images outperforms prior state-of-the-art models trained on real multimodal data by a margin of 3.76% in F1. Our codebase will be made available upon acceptance.


title: Uni3DL: Unified Model for 3D and Language Understanding

abstract: In this work, we present Uni3DL, a unified model for 3D and Language understanding. Distinct from existing unified vision-language models in 3D which are limited in task variety and predominantly dependent on projected multi-view images, Uni3DL operates directly on point clouds. This approach significantly expands the range of supported tasks in 3D, encompassing both vision and vision-language tasks in 3D. At the core of Uni3DL, a query transformer is designed to learn task-agnostic semantic and mask outputs by attending to 3D visual features, and a task router is employed to selectively generate task-specific outputs required for diverse tasks. With a unified architecture, our Uni3DL model enjoys seamless task decomposition and substantial parameter sharing across tasks. Uni3DL has been rigorously evaluated across diverse 3D vision-language understanding tasks, including semantic segmentation, object detection, instance segmentation, visual grounding, 3D captioning, and text-3D cross-modal retrieval. It demonstrates performance on par with or surpassing state-of-the-art (SOTA) task-specific models. We hope our benchmark and Uni3DL model will serve as a solid step to ease future research in unified models in the realm of 3D and language understanding. Project page: https://uni3dl.github.io.


title: Stable Diffusion Exposed: Gender Bias from Prompt to Image

abstract: Recent studies have highlighted biases in generative models, shedding light on their predisposition towards gender-based stereotypes and imbalances. This paper contributes to this growing body of research by introducing an evaluation protocol designed to automatically analyze the impact of gender indicators on Stable Diffusion images. Leveraging insights from prior work, we explore how gender indicators not only affect gender presentation but also the representation of objects and layouts within the generated images. Our findings include the existence of differences in the depiction of objects, such as instruments tailored for specific genders, and shifts in overall layouts. We also reveal that neutral prompts tend to produce images more aligned with masculine prompts than their feminine counterparts, providing valuable insights into the nuanced gender biases inherent in Stable Diffusion.


abstract: Lung cancer is one of the deadliest diseases and the leading cause of illness and death. Since lung cancer cannot predicted at premature stage, it able to only be discovered more broadly once it has spread to other lung parts. The risk grows when radiologists and other specialists determine whether lung cancer is current. Owing to significance of determining type of treatment and its depth based on severity of the illness, critical to develop smart and automatic cancer prediction scheme is precise, at which stage of cancer. In this paper, Double Integral Enhanced Zeroing Neural Network Optimized with ALSOA fostered Lung Cancer Classification using CT Images (LCC-DIEZNN-ALSO-CTI) is proposed. Initially, input CT image is amassed from lung cancer dataset. The input CT image is pre-processing via Unscented Trainable Kalman Filtering (UTKF) technique. In pre-processing stage unwanted noise are removed from CT images. Afterwards, grayscale statistic features and Haralick texture features extracted by Adaptive and Concise Empirical Wavelet Transform (ACEWT). The proposed model is implemented on MATLAB. The performance of the proposed method is analyzed through existing techniques. The proposed method attains 18.32%, 27.20%, and 34.32% higher accuracy analyzed with existing method likes Deep Learning Assisted Predict of Lung Cancer on Computed Tomography Images Utilizing AHHMM (LCC-AHHMM-CT), Convolutional neural networks based pulmonary nodule malignancy assessment in pipeline for classifying lung cancer (LCC-ICNN-CT), Automated Decision Support Scheme for Lung Cancer Identification with Categorization (LCC-RFCN-MLRPN-CT) methods respectively.


title: Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians

abstract: Creating high-fidelity 3D head avatars has always been a research hotspot, but there remains a great challenge under lightweight sparse view setups. In this paper, we propose Gaussian Head Avatar represented by controllable 3D Gaussians for high-fidelity head avatar modeling. We optimize the neutral 3D Gaussians and a fully learned MLP-based deformation field to capture complex expressions. The two parts benefit each other, thereby our method can model fine-grained dynamic details while ensuring expression accuracy. Furthermore, we devise a well-designed geometry-guided initialization strategy based on implicit SDF and Deep Marching Tetrahedra for the stability and convergence of the training procedure. Experiments show our approach outperforms other state-of-the-art sparse-view methods, achieving ultra high-fidelity rendering quality at 2K resolution even under exaggerated expressions.


title: Generating Visually Realistic Adversarial Patch

abstract: Deep neural networks (DNNs) are vulnerable to various types of adversarial examples, bringing huge threats to security-critical applications. Among these, adversarial patches have drawn increasing attention due to their good applicability to fool DNNs in the physical world. However, existing works often generate patches with meaningless noise or patterns, making it conspicuous to humans. To address this issue, we explore how to generate visually realistic adversarial patches to fool DNNs. Firstly, we analyze that a high-quality adversarial patch should be realistic, position irrelevant, and printable to be deployed in the physical world. Based on this analysis, we propose an effective attack called VRAP, to generate visually realistic adversarial patches. Specifically, VRAP constrains the patch in the neighborhood of a real image to ensure the visual reality, optimizes the patch at the poorest position for position irrelevance, and adopts Total Variance loss as well as gamma transformation to make the generated patch printable without losing information. Empirical evaluations on the ImageNet dataset demonstrate that the proposed VRAP exhibits outstanding attack performance in the digital world. Moreover, the generated adversarial patches can be disguised as the scrawl or logo in the physical world to fool the deep models without being detected, bringing significant threats to DNNs-enabled applications.


title: Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

abstract: End-to-end autonomous driving recently emerged as a promising research direction to target autonomy from a full-stack perspective. Along this line, many of the latest works follow an open-loop evaluation setting on nuScenes to study the planning behavior. In this paper, we delve deeper into the problem by conducting thorough analyses and demystifying more devils in the details. We initially observed that the nuScenes dataset, characterized by relatively simple driving scenarios, leads to an under-utilization of perception information in end-to-end models incorporating ego status, such as the ego vehicle's velocity. These models tend to rely predominantly on the ego vehicle's status for future path planning. Beyond the limitations of the dataset, we also note that current metrics do not comprehensively assess the planning quality, leading to potentially biased conclusions drawn from existing benchmarks. To address this issue, we introduce a new metric to evaluate whether the predicted trajectories adhere to the road. We further propose a simple baseline able to achieve competitive results without relying on perception annotations. Given the current limitations on the benchmark and metrics, we suggest the community reassess relevant prevailing research and be cautious whether the continued pursuit of state-of-the-art would yield convincing and universal conclusions. Code and models are available at \url{https://github.com/NVlabs/BEV-Planner}


title: Zero-Shot Point Cloud Registration

abstract: Learning-based point cloud registration approaches have significantly outperformed their traditional counterparts. However, they typically require extensive training on specific datasets. In this paper, we propose , the first zero-shot point cloud registration approach that eliminates the need for training on point cloud datasets. The cornerstone of ZeroReg is the novel transfer of image features from keypoints to the point cloud, enriched by aggregating information from 3D geometric neighborhoods. Specifically, we extract keypoints and features from 2D image pairs using a frozen pretrained 2D backbone. These features are then projected in 3D, and patches are constructed by searching for neighboring points. We integrate the geometric and visual features of each point using our novel parameter-free geometric decoder. Subsequently, the task of determining correspondences between point clouds is formulated as an optimal transport problem. Extensive evaluations of ZeroReg demonstrate its competitive performance against both traditional and learning-based methods. On benchmarks such as 3DMatch, 3DLoMatch, and ScanNet, ZeroReg achieves impressive Recall Ratios (RR) of over 84%, 46%, and 75%, respectively.


title: LiDAR-based Person Re-identification

abstract: Camera-based person re-identification (ReID) systems have been widely applied in the field of public security. However, cameras often lack the perception of 3D morphological information of human and are susceptible to various limitations, such as inadequate illumination, complex background, and personal privacy. In this paper, we propose a LiDAR-based ReID framework, ReID3D, that utilizes pre-training strategy to retrieve features of 3D body shape and introduces Graph-based Complementary Enhancement Encoder for extracting comprehensive features. Due to the lack of LiDAR datasets, we build LReID, the first LiDAR-based person ReID dataset, which is collected in several outdoor scenes with variations in natural conditions. Additionally, we introduce LReID-sync, a simulated pedestrian dataset designed for pre-training encoders with tasks of point cloud completion and shape parameter learning. Extensive experiments on LReID show that ReID3D achieves exceptional performance with a rank-1 accuracy of 94.0, highlighting the significant potential of LiDAR in addressing person ReID tasks. To the best of our knowledge, we are the first to propose a solution for LiDAR-based ReID. The code and datasets will be released soon.


title: Distributed Speech Dereverberation Using Weighted Prediction Error

abstract: Speech dereverberation aims to alleviate the negative impact of late reverberant reflections. The weighted prediction error (WPE) method is a well-established technique known for its superior performance in dereverberation. However, in scenarios where microphone nodes are dispersed, the centralized approach of the WPE method requires aggregating all observations for inverse filtering, resulting in a significant computational burden. This paper introduces a distributed speech dereverberation method that emphasizes low computational complexity at each node. Specifically, we leverage the distributed adaptive node-specific signal estimation (DANSE) algorithm within the multichannel linear prediction (MCLP) process. This approach empowers each node to perform local operations with reduced complexity while achieving the global performance through inter-node cooperation. Experimental results validate the effectiveness of our proposed method, showcasing its ability to achieve efficient speech dereverberation in dispersed microphone node scenarios.


title: SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstraction

abstract: Sketching is a powerful tool for creating abstract images that are sparse but meaningful. Sketch understanding poses fundamental challenges for general-purpose vision algorithms because it requires robustness to the sparsity of sketches relative to natural visual inputs and because it demands tolerance for semantic ambiguity, as sketches can reliably evoke multiple meanings. While current vision algorithms have achieved high performance on a variety of visual tasks, it remains unclear to what extent they understand sketches in a human-like way. Here we introduce SEVA, a new benchmark dataset containing approximately 90K human-generated sketches of 128 object concepts produced under different time constraints, and thus systematically varying in sparsity. We evaluated a suite of state-of-the-art vision algorithms on their ability to correctly identify the target concept depicted in these sketches and to generate responses that are strongly aligned with human response patterns on the same sketch recognition task. We found that vision algorithms that better predicted human sketch recognition performance also better approximated human uncertainty about sketch meaning, but there remains a sizable gap between model and human response patterns. To explore the potential of models that emulate human visual abstraction in generative tasks, we conducted further evaluations of a recently developed sketch generation algorithm (Vinker et al., 2022) capable of generating sketches that vary in sparsity. We hope that public release of this dataset and evaluation protocol will catalyze progress towards algorithms with enhanced capacities for human-like visual abstraction.


title: Loop correction and resummation of vertex functions for a self interacting scalar field in the de Sitter spacetime

abstract: We consider a massless and minimally coupled self interacting quantum scalar field theory in the inflationary de Sitter background of dimension four. The self interaction potential is taken to be either quartic, $\lambda \phi^4/4!$, or quartic plus cubic, $\lambda \phi^4/4!+\beta \phi^3/3!$ ($\lambda \,{\ensuremath >}\,0$). We compute the four and three point vertex functions up to two loop. The purely local or partly local part of these renormalised loop corrected vertex functions grow unboundedly after sufficient number of de Sitter $e$-foldings, due to the appearances of secular logarithms. We focus on the purely local part of the vertex functions and attempt a resummation of them in terms of the dynamically generated mass of the scalar field at late times. It turns out that the resummed, non-perturbative effective vertex functions have values less than that of the tree level. The variation of these vertex functions are investigated with respect to the tree level couplings numerically. Since neither the secular effect, nor the dynamical generation of field mass is possible in the Minkowski spacetime, the above phenomenon has no flat spacetime analogue.


title: Analysis and mining of low-carbon and energy-saving tourism data characteristics based on machine learning algorithm

abstract: In order to study the formation mechanism of residents' low-carbon awareness and provide an important basis for traffic managers to guide urban residents to choose low-carbon travel mode, this paper proposes a low-carbon energy-saving travel data feature analysis and mining based on machine learning algorithm. This paper uses data mining technology to analyze the data of low-carbon travel questionnaire, and regards the 15-dimensional problem under the framework of planned behavior theory as the internal cause variable that characterizes residents' low-carbon travel willingness. The author uses K-means clustering algorithm to classify the intensity of residents' low-carbon travel willingness, and applies the results as the explanatory variables to the random forest model to explore the mechanism of residents' social attribute characteristics, travel characteristics, etc. on their low-carbon travel willingness. The experimental results show that based on the Silhouette index test and t-SNE dimensionality reduction, residents' low-carbon travel willingness can be divided into three categories: strong, neutral, and not strong; Based on the importance index, the four most significant factors are the occupation, residence, family composition and commuting time of residents. Conclusion: This method provides policy recommendations for the development and management of urban traffic low-carbon from multiple perspectives.


title: Sample-based Dynamic Hierarchical Transformer with Layer and Head Flexibility via Contextual Bandit

abstract: Transformer requires a fixed number of layers and heads which makes them inflexible to the complexity of individual samples and expensive in training and inference. To address this, we propose a sample-based Dynamic Hierarchical Transformer (DHT) model whose layers and heads can be dynamically configured with single data samples via solving contextual bandit problems. To determine the number of layers and heads, we use the Uniform Confidence Bound while we deploy combinatorial Thompson Sampling in order to select specific head combinations given their number. Different from previous work that focuses on compressing trained networks for inference only, DHT is not only advantageous for adaptively optimizing the underlying network architecture during training but also has a flexible network for efficient inference. To the best of our knowledge, this is the first comprehensive data-driven dynamic transformer without any additional auxiliary neural networks that implement the dynamic system. According to the experiment results, we achieve up to 74% computational savings for both training and inference with a minimal loss of accuracy.


title: Insights into the first and second hydrostatic core stages from numerical simulations

abstract: The theory of how low mass stars form from the collapse of a dense molecular cloud core has been well-established for decades. Thanks to significant progress in computing and numerical modelling, more physical models have been developed and a wider parameter space explored to understand the early stages of star formation more fully. In this review, I describe the expected physical properties of the first and second core stages and how the inclusion of different physics affects those predicted characteristics. I provide an overview of chemical models and synthetic observations, looking towards the positive identification of the first core in nature, which remains elusive. However, there are a few likely candidate first cores, which are listed, and I briefly discuss the recent progress in characterising the youngest protostellar sources. Chemistry will be instrumental in the firm identification of the first core so we require robust theoretical predictions of the chemical evolution of protostellar cores, especially of the first and second core outflows. Looking ahead, simulations can shed light on how the protostellar collapse phase shapes the evolution of the protostellar disc. Simulations of dust evolution during protostellar core collapse show there is significant enhancement in grain size and abundance towards the centre of the core. Chemical models show that the warm, dense conditions of the first core drive chemical evolution. There is a wide scope for further study of the role that the first and second core stages play in determining the structure and composition of the protostellar disc and envelope and, of course, the eventual influence on the formation of planets.


title: Device-Independent Quantum Secure Direct Communication Under Non-Markovian Quantum Channels

abstract: Device-independent quantum secure direct communication (DI-QSDC) is a promising primitive in quantum cryptography aimed towards addressing the problems of device imperfections and key management. However, significant effort is required to tackle practical challenges such as the distance limitation due to the decohering effects of quantum channels. Here, we explore the constructive effect of non-Markovian noise to improve the performance of DI-QSDC. Considering two different environmental dynamics modelled by the amplitude damping and the dephasing channels, we show that for both cases non-Markovianty leads to a considerable improvement over Markovian dynamics in terms of three benchmark performance criteria of the DI-QSDC task. Specifically, we find that non-Markovian noise (i) enhances the protocol security measured by Bell violation, (ii) leads to a lower quantum bit error rate, and (iii) enables larger communication distances by increasing the capacity of secret communication.


title: Transformer-Based Deep Learning Model for Bored Pile Load-Deformation Prediction in Bangkok Subsoil

abstract: This paper presents a novel deep learning model based on the transformer architecture to predict the load-deformation behavior of large bored piles in Bangkok subsoil. The model encodes the soil profile and pile features as tokenization input, and generates the load-deformation curve as output. The model also incorporates the previous sequential data of load-deformation curve into the decoder to improve the prediction accuracy. The model also incorporates the previous sequential data of load-deformation curve into the decoder. The model shows a satisfactory accuracy and generalization ability for the load-deformation curve prediction, with a mean absolute error of 5.72% for the test data. The model could also be used for parametric analysis and design optimization of piles under different soil and pile conditions, pile cross section, pile length and type of pile.


title: Inherent limitations of LLMs regarding spatial information

abstract: Despite the significant advancements in natural language processing capabilities demonstrated by large language models such as ChatGPT, their proficiency in comprehending and processing spatial information, especially within the domains of 2D and 3D route planning, remains notably underdeveloped. This paper investigates the inherent limitations of ChatGPT and similar models in spatial reasoning and navigation-related tasks, an area critical for applications ranging from autonomous vehicle guidance to assistive technologies for the visually impaired. In this paper, we introduce a novel evaluation framework complemented by a baseline dataset, meticulously crafted for this study. This dataset is structured around three key tasks: plotting spatial points, planning routes in two-dimensional (2D) spaces, and devising pathways in three-dimensional (3D) environments. We specifically developed this dataset to assess the spatial reasoning abilities of ChatGPT. Our evaluation reveals key insights into the model's capabilities and limitations in spatial understanding.


title: Navigating the Synthetic Realm: Harnessing Diffusion-based Models for Laparoscopic Text-to-Image Generation

abstract: Recent advances in synthetic imaging open up opportunities for obtaining additional data in the field of surgical imaging. This data can provide reliable supplements supporting surgical applications and decision-making through computer vision. Particularly the field of image-guided surgery, such as laparoscopic and robotic-assisted surgery, benefits strongly from synthetic image datasets and virtual surgical training methods. Our study presents an intuitive approach for generating synthetic laparoscopic images from short text prompts using diffusion-based generative models. We demonstrate the usage of state-of-the-art text-to-image architectures in the context of laparoscopic imaging with regard to the surgical removal of the gallbladder as an example. Results on fidelity and diversity demonstrate that diffusion-based models can acquire knowledge about the style and semantics in the field of image-guided surgery. A validation study with a human assessment survey underlines the realistic nature of our synthetic data, as medical personnel detects actual images in a pool with generated images causing a false-positive rate of 66%. In addition, the investigation of a state-of-the-art machine learning model to recognize surgical actions indicates enhanced results when trained with additional generated images of up to 5.20%. Overall, the achieved image quality contributes to the usage of computer-generated images in surgical applications and enhances its path to maturity.


title: REST: Enhancing Group Robustness in DNNs through Reweighted Sparse Training

abstract: The deep neural network (DNN) has been proven effective in various domains. However, they often struggle to perform well on certain minority groups during inference, despite showing strong performance on the majority of data groups. This is because over-parameterized models learned \textit{bias attributes} from a large number of \textit{bias-aligned} training samples. These bias attributes are strongly spuriously correlated with the target variable, causing the models to be biased towards spurious correlations (i.e., \textit{bias-conflicting}). To tackle this issue, we propose a novel \textbf{re}weighted \textbf{s}parse \textbf{t}raining framework, dubbed as \textit{\textbf{REST}}, which aims to enhance the performance of biased data while improving computation and memory efficiency. Our proposed REST framework has been experimentally validated on three datasets, demonstrating its effectiveness in exploring unbiased subnetworks. We found that REST reduces the reliance on spuriously correlated features, leading to better performance across a wider range of data groups with fewer training and inference resources. We highlight that the \textit{REST} framework represents a promising approach for improving the performance of DNNs on biased data, while simultaneously improving computation and memory efficiency. By reducing the reliance on spurious correlations, REST has the potential to enhance the robustness of DNNs and improve their generalization capabilities. Code is released at \url{https://github.com/zhao1402072392/REST}


title: Customization Assistant for Text-to-image Generation

abstract: Customizing pre-trained text-to-image generation model has attracted massive research interest recently, due to its huge potential in real-world applications. Although existing methods are able to generate creative content for a novel concept contained in single user-input image, their capability are still far from perfection. Specifically, most existing methods require fine-tuning the generative model on testing images. Some existing methods do not require fine-tuning, while their performance are unsatisfactory. Furthermore, the interaction between users and models are still limited to directive and descriptive prompts such as instructions and captions. In this work, we build a customization assistant based on pre-trained large language model and diffusion model, which can not only perform customized generation in a tuning-free manner, but also enable more user-friendly interactions: users can chat with the assistant and input either ambiguous text or clear instruction. Specifically, we propose a new framework consists of a new model design and a novel training strategy. The resulting assistant can perform customized generation in 2-5 seconds without any test time fine-tuning. Extensive experiments are conducted, competitive results have been obtained across different domains, illustrating the effectiveness of the proposed method.


title: Diversified in-domain synthesis with efficient fine-tuning for few-shot classification

abstract: Few-shot image classification aims to learn an image classifier using only a small set of labeled examples per class. A recent research direction for improving few-shot classifiers involves augmenting the labelled samples with synthetic images created by state-of-the-art text-to-image generation models. Following this trend, we propose Diversified In-domain Synthesis with Efficient Fine-tuning (DISEF), a novel approach which addresses the generalization challenge in few-shot learning using synthetic data. DISEF consists of two main components. First, we propose a novel text-to-image augmentation pipeline that, by leveraging the real samples and their rich semantics coming from an advanced captioning model, promotes in-domain sample diversity for better generalization. Second, we emphasize the importance of effective model fine-tuning in few-shot recognition, proposing to use Low-Rank Adaptation (LoRA) for joint adaptation of the text and image encoders in a Vision Language Model. We validate our method in ten different benchmarks, consistently outperforming baselines and establishing a new state-of-the-art for few-shot classification. Code is available at https://github.com/vturrisi/disef.


title: MagicStick: Controllable Video Editing via Control Handle Transformations

abstract: Text-based video editing has recently attracted considerable interest in changing the style or replacing the objects with a similar structure. Beyond this, we demonstrate that properties such as shape, size, location, motion, etc., can also be edited in videos. Our key insight is that the keyframe transformations of the specific internal feature (e.g., edge maps of objects or human pose), can easily propagate to other frames to provide generation guidance. We thus propose MagicStick, a controllable video editing method that edits the video properties by utilizing the transformation on the extracted internal control signals. In detail, to keep the appearance, we inflate both the pretrained image diffusion model and ControlNet to the temporal dimension and train low-rank adaptions (LORA) layers to fit the specific scenes. Then, in editing, we perform an inversion and editing framework. Differently, finetuned ControlNet is introduced in both inversion and generation for attention guidance with the proposed attention remix between the spatial attention maps of inversion and editing. Yet succinct, our method is the first method to show the ability of video property editing from the pre-trained text-to-image model. We present experiments on numerous examples within our unified framework. We also compare with shape-aware text-based editing and handcrafted motion video generation, demonstrating our superior temporal consistency and editing capability than previous works. The code and models will be made publicly available.


title: DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Control

abstract: Large, pretrained latent diffusion models (LDMs) have demonstrated an extraordinary ability to generate creative content, specialize to user data through few-shot fine-tuning, and condition their output on other modalities, such as semantic maps. However, are they usable as large-scale data generators, e.g., to improve tasks in the perception stack, like semantic segmentation? We investigate this question in the context of autonomous driving, and answer it with a resounding "yes". We propose an efficient data generation pipeline termed DGInStyle. First, we examine the problem of specializing a pretrained LDM to semantically-controlled generation within a narrow domain. Second, we propose a Style Swap technique to endow the rich generative prior with the learned semantic control. Third, we design a Multi-resolution Latent Fusion technique to overcome the bias of LDMs towards dominant objects. Using DGInStyle, we generate a diverse dataset of street scenes, train a domain-agnostic semantic segmentation model on it, and evaluate the model on multiple popular autonomous driving datasets. Our approach consistently increases the performance of several domain generalization methods compared to the previous state-of-the-art methods. Source code and dataset are available at https://dginstyle.github.io.


title: Architectural Approaches to Overcome Challenges in the Development of Data-Intensive Systems

abstract: Orientation of modern software systems towards data-intensive processing raises new difficulties in software engineering on how to build and maintain such systems. Some of the important challenges concern the design of software architecture. In this article, we survey the fundamental challenges when designing data-intensive computing systems and present some of the most popular software architectural styles together with their potential to tackle these challenges.


title: HIG: Hierarchical Interlacement Graph Approach to Scene Graph Generation in Video Understanding

abstract: Visual interactivity understanding within visual scenes presents a significant challenge in computer vision. Existing methods focus on complex interactivities while leveraging a simple relationship model. These methods, however, struggle with a diversity of appearance, situation, position, interaction, and relation in videos. This limitation hinders the ability to fully comprehend the interplay within the complex visual dynamics of subjects. In this paper, we delve into interactivities understanding within visual content by deriving scene graph representations from dense interactivities among humans and objects. To achieve this goal, we first present a new dataset containing Appearance-Situation-Position-Interaction-Relation predicates, named ASPIRe, offering an extensive collection of videos marked by a wide range of interactivities. Then, we propose a new approach named Hierarchical Interlacement Graph (HIG), which leverages a unified layer and graph within a hierarchical structure to provide deep insights into scene changes across five distinct tasks. Our approach demonstrates superior performance to other methods through extensive experiments conducted in various scenarios.


title: Generating Interpretable Networks using Hypernetworks

abstract: An essential goal in mechanistic interpretability to decode a network, i.e., to convert a neural network's raw weights to an interpretable algorithm. Given the difficulty of the decoding problem, progress has been made to understand the easier encoding problem, i.e., to convert an interpretable algorithm into network weights. Previous works focus on encoding existing algorithms into networks, which are interpretable by definition. However, focusing on encoding limits the possibility of discovering new algorithms that humans have never stumbled upon, but that are nevertheless interpretable. In this work, we explore the possibility of using hypernetworks to generate interpretable networks whose underlying algorithms are not yet known. The hypernetwork is carefully designed such that it can control network complexity, leading to a diverse family of interpretable algorithms ranked by their complexity. All of them are interpretable in hindsight, although some of them are less intuitive to humans, hence providing new insights regarding how to "think" like a neural network. For the task of computing L1 norms, hypernetworks find three algorithms: (a) the double-sided algorithm, (b) the convexity algorithm, (c) the pudding algorithm, although only the first algorithm was expected by the authors before experiments. We automatically classify these algorithms and analyze how these algorithmic phases develop during training, as well as how they are affected by complexity control. Furthermore, we show that a trained hypernetwork can correctly construct models for input dimensions not seen in training, demonstrating systematic generalization.


title: Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models

abstract: Solving complex visual tasks such as "Who invented the musical instrument on the right?" involves a composition of skills: understanding space, recognizing instruments, and also retrieving prior knowledge. Recent work shows promise by decomposing such tasks using a large language model (LLM) into an executable program that invokes specialized vision models. However, generated programs are error-prone: they omit necessary steps, include spurious ones, and are unable to recover when the specialized models give incorrect outputs. Moreover, they require loading multiple models, incurring high latency and computation costs. We propose Visual Program Distillation (VPD), an instruction tuning framework that produces a vision-language model (VLM) capable of solving complex visual tasks with a single forward pass. VPD distills the reasoning ability of LLMs by using them to sample multiple candidate programs, which are then executed and verified to identify a correct one. It translates each correct program into a language description of the reasoning steps, which are then distilled into a VLM. Extensive experiments show that VPD improves the VLM's ability to count, understand spatial relations, and reason compositionally. Our VPD-trained PaLI-X outperforms all prior VLMs, achieving state-of-the-art performance across complex vision tasks, including MMBench, OK-VQA, A-OKVQA, TallyQA, POPE, and Hateful Memes. An evaluation with human annotators also confirms that VPD improves model response factuality and consistency. Finally, experiments on content moderation demonstrate that VPD is also helpful for adaptation to real-world applications with limited data.


title: DiffusionPCR: Diffusion Models for Robust Multi-Step Point Cloud Registration

abstract: Point Cloud Registration (PCR) estimates the relative rigid transformation between two point clouds. We propose formulating PCR as a denoising diffusion probabilistic process, mapping noisy transformations to the ground truth. However, using diffusion models for PCR has nontrivial challenges, such as adapting a generative model to a discriminative task and leveraging the estimated nonlinear transformation from the previous step. Instead of training a diffusion model to directly map pure noise to ground truth, we map the predictions of an off-the-shelf PCR model to ground truth. The predictions of off-the-shelf models are often imperfect, especially in challenging cases where the two points clouds have low overlap, and thus could be seen as noisy versions of the real rigid transformation. In addition, we transform the rotation matrix into a spherical linear space for interpolation between samples in the forward process, and convert rigid transformations into auxiliary information to implicitly exploit last-step estimations in the reverse process. As a result, conditioned on time step, the denoising model adapts to the increasing accuracy across steps and refines registrations. Our extensive experiments showcase the effectiveness of our DiffusionPCR, yielding state-of-the-art registration recall rates (95.3%/81.6%) on 3DMatch and 3DLoMatch. The code will be made public upon publication.


title: Braids and Higher-order Exceptional Points from the Interplay Between Lossy Defects and Topological Boundary States

abstract: We show that the perturbation of the Su-Schrieffer-Heeger chain by a localized lossy defect leads to higher-order exceptional points (HOEPs). Depending on the location of the defect, third- and fourth-order exceptional points (EP3s & EP4s) appear in the space of Hamiltonian parameters. On the one hand, they arise due to the non-Abelian braiding properties of exceptional lines (ELs) in parameter space. Namely, the HOEPs lie at intersections of mutually non-commuting ELs. On the other hand, we show that such special intersections happen due to the fact that the delocalization of edge states, induced by the non-Hermitian defect, hybridizes them with defect states. These can then coalesce together into an EP3. When the defect lies at the midpoint of the chain, a special symmetry of the full spectrum can lead to an EP4. In this way, our model illustrates the emergence of interesting non-Abelian topological properties in the multiband structure of non-Hermitian perturbations of topological phases.

categories: cond-mat.mes-hall math-ph math.MP physics.optics quant-ph


title: Front-row seat of the recent R Aqr periastron passage: X-ray multi-epoch spectral and spatial analysis

abstract: We report on the X-ray spectral and spatial evolution of the Symbiotic star R Aqr. Through a multi-epoch observational campaign performed with Chandra between 2017 and 2022, we study the X-ray emission of this binary system, composed of an evolved red giant star and a white dwarf (WD). This analysis is particularly timely as the WD approached the periastron in late 2018/early 2019, thus mass transfer, jet emission and outburst phenomena are to be expected. Through detailed spectral analysis, we detect a significant rise in the soft X-ray (0.5-2 keV) emission of R Aqr, likely linked to jet emission, followed by a decay towards the previous quiescent state. The hard X-ray emission (5-8 keV), is not immediately affected by the periastron passage; the hard component, after maintaining the same flux level between 2017 and 2021, rapidly decays after 2022. Possible explanations for this are a change in the reflection properties of the medium surrounding the binary, obscuration of the central region by material ejected during the periastron passage, or even the partial/complete destruction of the inner regions of the accretion disc surrounding the WD. In addition to this activity in the central region, extended emission is also detected, likely linked to a hot spot in a pre-outburst-emitted jet, which can be observed moving away from the system's central region.


title: Linear polarization of the stochastic gravitational-wave background with pulsar timing arrays

abstract: Pulsar-timing collaborations have recently reported evidence for the detection of an isotropic stochastic gravitational-wave background consistent with one sourced by a population of inspiralling supermassive black hole binaries. However, a certain degree of anisotropy and polarization may be present. Thus, the characterization of the energy density and polarization of the background at different angular scales is important. In this paper, we describe the signatures of linear polarization in the stochastic gravitational-wave background on the timing residuals obtained with pulsar-timing arrays. We expand the linear polarization map in terms of spin-weighted spherical harmonics and recast it into the $E$-mode (parity even) and $B$-mode (parity odd) basis. We provide expressions for the minimum-variance estimators for the coefficients of that expansion and evaluate the smallest detectable signal as a function of the signal-to-noise ratio with which the isotropic GW signal is detected and the number of pulsars in the survey. We evaluate the covariance between the estimators for the spherical-harmonic coefficients of the linear polarization $E$-modes and those for the intensity anisotropy. We also show that there is no covariance between the spherical-harmonic coefficients for the $B$-modes of the linear polarization and those for the circular polarization, even though both have the same parity. Our approach results in simple, elegant, and easily evaluated expressions for the overlap reduction functions for linear polarization.


title: Advantage of Quantum Machine Learning from General Computational Advantages

abstract: An overarching milestone of quantum machine learning (QML) is to demonstrate the advantage of QML over all possible classical learning methods in accelerating a common type of learning task as represented by supervised learning with classical data. However, the provable advantages of QML in supervised learning have been known so far only for the learning tasks designed for using the advantage of specific quantum algorithms, i.e., Shor's algorithms. Here we explicitly construct an unprecedentedly broader family of supervised learning tasks with classical data to offer the provable advantage of QML based on general quantum computational advantages, progressing beyond Shor's algorithms. Our learning task is feasibly achievable by executing a general class of functions that can be computed efficiently in polynomial time for a large fraction of inputs by arbitrary quantum algorithms but not by any classical algorithm. We prove the hardness of achieving this learning task for any possible polynomial-time classical learning method. We also clarify protocols for preparing the classical data to demonstrate this learning task in experiments. These results open routes to exploit a variety of quantum advantages in computing functions for the experimental demonstration of the advantage of QML.


title: Modelling molecular clouds and CO excitation in AGN-host galaxies

abstract: We present a new physically-motivated model for estimating the molecular line emission in active galaxies. The model takes into account (i) the internal density structure of giant molecular clouds (GMCs), (ii) the heating associated both to stars and to the active galactic nuclei (AGN), respectively producing photodissociation regions (PDRs) and X-ray dominated regions (XDRs) within the GMCs, and (iii) the mass distribution of GMCs within the galaxy volume. The model needs, as input parameters, the radial profiles of molecular mass, far-UV flux and X-ray flux for a given galaxy, and it has two free parameters: the CO-to-H2 conversion factor $\alpha_{CO}$, and the X-ray attenuation column density $N_H$. We test this model on a sample of 24 local ($z \leq 0.06$) AGN-host galaxies, simulating their carbon monoxide spectral line energy distribution (CO SLED). We compare the results with the available observations and calculate, for each galaxy, the best ($\alpha_{CO}$, $N_H$) with a Markov chain Monte Carlo algorithm, finding values consistent with those present in the literature. We find a median $\alpha_{CO} = 4.8$ M$_{\odot}$ (K km s$^{-1}$ pc$^{2}$)$^{-1}$ for our sample. In all the modelled galaxies, we find the XDR component of the CO SLED to dominate the CO luminosity from $J_{\text{upp}} \geq 4$. We conclude that, once a detailed distribution of molecular gas density is taken into account, PDR emission at mid-/high-$J$ becomes negligible with respect to XDR.


title: A Foray on SCFT$_3$ via Super Spinor-Helicity and Grassmann Twistor Variables

abstract: In this paper, we develop a momentum super space formalism for $\mathcal{N}=1,2$ superconformal field theories in three dimensions. First, we solve for super-correlators in the usual momentum superspace variables. However, we found that expressing quantities in super space spinor helicity variables greatly simplifies the analysis. Further, by performing a "half" Fourier transform of the Grassmann coordinates which is analogous to the Twistor transform, an even more remarkable simplification occurs. Using these formalism, we first compute all three point correlation functions involving conserved super-currents with arbitrary spins in $\mathcal{N}=1,2$ theories. We discover interesting double copy relations in $\mathcal{N}=1$ super-correlators. Also, we discovered super double copy relations that take us from $\mathcal{N}=1$ to $\mathcal{N}=2$ super-correlators. We also comment on the connection of our results with the flat space super amplitudes in one higher dimension.


title: The JWST Resolved Stellar Populations Early Release Science Program IV: The Star Formation History of the Local Group Galaxy WLM

abstract: We present the first star formation history (SFH) and age-metallicity relation (AMR) derived from resolved stellar populations imaged with the JWST NIRCam instrument. The target is the Local Group star-forming galaxy WLM at 970 kpc. The depth of the color-magnitude diagram (CMD) reaches below the oldest main sequence turn-off with a SNR=10 at M_F090W=+4.6 mag; this is the deepest CMD for any galaxy that is not a satellite of the Milky Way. We use Hubble Space Telescope (HST) optical imaging that overlaps with the NIRCam observations to directly evaluate the SFHs derived based on data from the two great observatories. The JWST and HST-based SFHs are in excellent agreement. We use the metallicity distribution function measured from stellar spectra to confirm the trends in the AMRs based on the JWST data. Together, these results confirm the efficacy of recovering a SFH and AMR with the NIRCam F090W-F150W filter combination and provide validation of the sensitivity and accuracy of stellar evolution libraries in the near-infrared relative to the optical for SFH recovery work. From the JWST data, WLM shows an early onset to star formation, followed by an extended pause post-reionization before star formation re-ignites, which is qualitatively similar to what has been observed in the isolated galaxies Leo~A and Aquarius. Quantitatively, 15% of the stellar mass formed in the first Gyr, while only 10% formed over the next ~5 Gyr; the stellar mass then rapidly doubled in ~2.5 Gyr, followed by constant star formation over the last ~5 Gyr.


title: Resonant Spin-Flavor Precession of Sterile Neutrinos

abstract: We analyze the impact of resonant conversions mediated by non-vanishing magnetic moments between active neutrinos and a heavy sterile neutrino on the supernova neutrino flux. We present the level-crossing scheme for such a scenario and derive the neutrino fluxes after conversion, paying special attention to the order in which the resonances occur. We then compute the expected event rates from the neutronization burst of a future supernova at DUNE and Hyper-Kamiokande to derive new constraints on the neutrino magnetic moment. With this, we find a sensitivity down to a few $10^{-15} \mu_B$ for a sterile neutrino in the $O(\rm{eV})$ mass range.


title: Diffeomorphism-invariant action principles for trace-free Einstein gravity

abstract: Trace-free Einstein gravity is a theory of gravity that is an alternative to general relativity, wherein the cosmological constant arises as an integration constant. However, there are no fully diffeomorphism-invariant action principles available that lead to the equations of motion of this theory. Unimodular gravity comes close to this idea, but it relies on action principles that are invariant only under volume-preserving diffeomorphisms. We present a real $BF$-type action principle for trace-free Einstein gravity that is fully diffeomorphism-invariant and does not require any unimodular condition or nondynamical fields. We generalize this action principle by giving another one involving a free parameter.


title: Real-time correlators in 3+1D thermal lattice gauge theory

abstract: Real-time quantities like spectral functions and transport coefficients are crucial for a proper understanding of the quark-gluon plasma created in relativistic heavy-ion collisions. Their numerical computation is plagued by a severe sign problem inherent in the real-time formulation of lattice field theories. In this letter, we present the first direct ab-initio computation of unequal-time correlation functions in non-Abelian lattice gauge theory, which are necessary to extract real-time quantities. We demonstrate non-trivial consistency relations among correlators, time-translation invariance, and agreement with Monte-Carlo results for thermal equilibrium in 3+1 dimensions by employing our stabilized complex Langevin method. Our work sets the stage to extract real-time observables in lattice gauge theory in a first-principles real-time framework.

categories: hep-lat cond-mat.other hep-ph hep-th nucl-th


title: A robust and simple method for filling in masked data in astronomical images

abstract: Astronomical images often have regions with missing or unwanted information, such as bad pixels, bad columns, cosmic rays, masked objects, or residuals from imperfect model subtractions. In certain situations it can be essential, or preferable, to fill in these regions. Most existing methods use low order interpolations for this task. In this paper a method is described that uses the full information that is contained in the pixels just outside masked regions. These edge pixels are extrapolated inwards, using iterative median filtering. This leads to a smoothly varying spatial resolution within the filled-in regions, and ensures seamless transitions between masked pixels and good pixels. Gaps in continuous, narrow features can be reconstructed with high fidelity, even if they are large. The method is implemented in maskfill, an open-source MIT licensed Python package. Its performance is illustrated with several examples, and compared to several alternative interpolation schemes.


title: Deciphering the JWST spectrum of a 'little red dot' at $z \sim 4.53$: An obscured AGN and its star-forming host

abstract: JWST has revealed a class of numerous, extremely compact sources, with rest-frame red optical/near-infrared (NIR) and blue ultraviolet (UV) colours, nicknamed "little red dots". We present one of the highest signal-to-noise ratio JWST NIRSpec/PRISM spectra of a little red dot, J0647_1045 at $z = 4.5321 \pm 0.0001$, and examine its NIRCam morphology, to differentiate the origin of the UV and optical/NIR emission, and elucidate the nature of the little red dot phenomenon. J0647_1045 is unresolved ($r_e < 0.17$ kpc) in the three NIRCam long-wavelength filters, but significantly extended ($r_e = 0.45 \pm 0.06$ kpc) in the three short-wavelength filters, indicating a red compact source in a blue star-forming galaxy. The spectral continuum shows a clear change in slope, from blue in the optical/UV, to red in the restframe optical/NIR, consistent with two distinct components, fit by power-laws with different attenuation: $A_V = 0.54 \pm 0.01$ (UV) and $A_V = 5.7 \pm 0.2$ (optical/NIR). Fitting the H$\alpha$ line requires both broad (full width at half-maximum $\sim 4300 \pm 300 km s^{-1}$) and narrow components, but none of the other emission lines, including H$\beta$, show evidence of broadness. We calculate $A_V = 1.1 \pm 0.2$ from the Balmer decrement using narrow H$\alpha$ and H$\beta$, and $A_V > 4.1 \pm 0.2$ from broad H$\alpha$ and upper limit on broad H$\beta$, consistent with the blue and red continuum attenuation respectively. Based on single-epoch H$\alpha$ linewidth, the mass of the central black hole is $8 \pm 1 \times 10^8 M_\odot$. Our findings are consistent with a multi-component model, where the optical/NIR and broad lines arise from a highly obscured, spatially unresolved region, likely a relatively massive active galactic nucleus, while the less obscured UV continuum and narrow lines arise, at least partly, from a small but spatially resolved star-forming host galaxy.


title: A Triple Scenario for the Formation of Wide Black Hole Binaries Such As Gaia BH1

abstract: Recently, several non-interacting black hole-stellar binaries have been identified in Gaia data. For example, Gaia BH1, where a Sun-like star is in moderate eccentricity (e=0.45), 185-day orbit around a black hole. This orbit is difficult to explain through binary evolution. The present-day separation suggests the progenitor system would have undergone an episode of common envelope evolution, but a common envelope should shrink the period below the observed one. Since the majority of massive stars form in higher multiplicity systems, a triple evolution scenario is more likely for the progenitors of BH binaries. Here we show that such systems can indeed be more easily explained via evolution in hierarchical triple systems. von Zeipel--Lidov--Kozai oscillations or instabilities can delay the onset of the common envelope phase in the inner binary of the triple, so that the black hole progenitor and low-mass star are more widely separated when it begins, leading to the formation of wider binaries. There are also systems with similar periods but larger eccentricities, where the BH progenitor is a merger product of the inner binary in the triple. Such mergers lead to a more top-heavy black hole mass function.


title: Semi-visible jets, energy-based models, and self-supervision

abstract: We present DarkCLR, a novel framework for detecting semi-visible jets at the LHC. DarkCLR uses a self-supervised contrastive-learning approach to create observables that are approximately invariant under relevant transformations. We use background-enhanced data to create a sensitive representation and evaluate the representations using a normalized autoencoder as a density estimator. Our results show a remarkable sensitivity for a wide range of semi-visible jets and are more robust than a supervised classifier trained on a specific signal.


title: Precise Radial Velocities Using Line Bisectors

abstract: We study the properties of line bisectors in the spectrum of the Sun-as-a-star, as observed using the Integrated Sunlight Spectrometer (ISS) of the SOLIS project. Our motivation is to determine whether changes in line shape, due to magnetic modulation of photospheric convection can be separated from the 9 cm/sec Doppler reflex of the Earth's orbit. Measuring bisectors of 21 lines over a full solar cycle, our results overwhelmingly indicate that solar magnetic activity modulates photospheric convection so as to reduce the asymmetries of line profiles in the spectrum of the Sun-as-a-star (having both C-shaped and reversed-C-shaped bisectors). However, some lines are constant or have variations in shape that are too small to measure. We inject a 9 cm/sec radial velocity signal with a 1-year period into the ISS spectra. Informed by a principal component analysis of the line bisectors, we fit the most significant components to the bisectors of each line by linear regression, including a zero-point offset in velocity that is intended to capture the injected radial velocity signal. Averaging over lines, we are able to recover that signal to solid statistical significance in the presence of much larger changes in the line shapes. Although our work has limitations (that we discuss), we establish that changes in absorption line shapes do not in themselves prevent the detection of an Earth-like planet orbiting a Sun-like star using precise radal velocity techniques.


title: Dissecting the Stochastic Gravitational Wave Background with Astrometry

abstract: Astrometry, the precise measurement of star motions, offers an alternative avenue to investigate low-frequency gravitational waves through the spatial deflection of photons, complementing pulsar timing arrays reliant on timing residuals. Upcoming data from Gaia and Roman can not only cross-check pulsar timing array findings but also explore the uncharted frequency range bridging pulsar timing arrays and LISA. We present an analytical framework to evaluate the feasibility of detecting a gravitational wave background, considering measurement noise and the intrinsic variability of the stochastic background. Furthermore, we highlight astrometry's crucial role in uncovering key properties of the gravitational wave background, such as spectral index and chirality, employing information-matrix analysis. Finally, we simulate the emergence of quadrupolar correlations, commonly referred to as the generalized Hellings-Downs curves.

categories: gr-qc astro-ph.CO astro-ph.HE astro-ph.IM hep-ph


title: Rescuing the Unruh Effect in Lorentz Violating Gravity

abstract: While the robustness of Hawking radiation in the presence of UV Lorentz breaking is well-established, the Unruh effect has posed a challenge, with a large literature concluding that even the low-energy restoration of Lorentz invariance may not be sufficient to sustain this phenomenon. Notably, these previous studies have primarily focused on Lorentz-breaking matter on a conventional Rindler wedge. In this work, we demonstrate that considering the complete structure of Lorentz-breaking gravity, specifically the presence of a hypersurface orthogonal aether field, leads to the selection of a new Rindler wedge configuration characterized by a uniformly accelerated aether flow. This uniform acceleration provides a reference scale for comparison with the Lorentz-breaking one, thus ensuring the persistence of the Unruh effect in this context. We establish this by calculating the expected temperature using a Bogolubov approach, and by analyzing the response of a uniformly accelerated detector. We suggest that this resilience of the Unruh effect opens interesting possibilities towards future developments for using it as a tool to constrain Lorentz breaking theories of gravity.


title: Suppression of Shot Noise in a Dirty Marginal Fermi Liquid

abstract: We study shot noise in a two-dimensional, disordered marginal Fermi liquid (MFL) driven out of equilibrium. We consider electrons with Planckian dissipation on the Fermi surface coupled to quantum-critical bosons in the presence of disorder. In the non-interacting and strong fermion-boson drag limits, MFL effects disappear and our theory reproduces known results. Interestingly, in the case where the bosons remain in equilibrium, MFL effects play a role and the shot noise reduces rapidly as the electron-boson coupling strength increases. Our results suggest that shot noise can be strongly suppressed in quantum-critical systems, and this scenario may be relevant to recent measurements in a heavy-fermion strange metal.


title: Phenomenology of Spillway Preheating: Equation of State and Gravitational Waves

abstract: In the canonical tachyonic resonance preheating scenario, only an order one fraction of energy density in the inflaton is transferred to radiation, due to backreaction effects. One possible way to improve the energy transfer efficiency is to allow for the perturbative decays of the resonantly produced daughter particles, which serve as the "spillway" to drain the direct decay products from inflaton and to reduce the backreaction. In this article, we study two observational consequences of spillway preheating. The first is on the inflationary observables: the scalar spectrum tilt $n_s$ and tensor-to-scalar ratio $r$. The spillway scenario modifies the evolution of the equation of state between the end of inflation and the thermal big bang. As a result, it affects the time elapsed from inflation to the Cosmic Microwave Background (CMB), as well as the fits of inflationary models and their corresponding prediction for $n_s$ and $r$. We map out the equation of state by systematically scanning the parameter space of the spillway scenario, and show that the most efficient spillway scenario predicts a bluer spectrum, compared to the tachyonic preheating scenario. Another consequence is the production of high-frequency gravitational waves (GWs). Comparing the simulation results with those of tachyonic preheating, we find that the existence of spillways leads to sharper-peaked GW spectra with a mildly damped amplitude.


title: Universality in driven open quantum matter

abstract: Universality is a powerful concept, which enables making qualitative and quantitative predictions in systems with extensively many degrees of freedom. It finds realizations in almost all branches of physics, including in the realm of nonequilibrium systems. Our focus here is on its manifestations within a specific class of nonequilibrium stationary states: driven open quantum matter. Progress in this field is fueled by a number of uprising platforms ranging from light-driven quantum materials over synthetic quantum systems like cold atomic gases to the functional devices of the noisy intermediate scale quantum era. These systems share in common that, on the microscopic scale, they obey the laws of quantum mechanics, while detailed balance underlying thermodynamic equilibrium is broken due to the simultaneous presence of Hamiltonian unitary dynamics and nonunitary drive and dissipation. The challenge is then to connect this microscopic physics to macroscopic observables, and to identify universal collective phenomena that uniquely witness the breaking of equilibrium conditions, thus having no equilibrium counterparts. In the framework of a Lindblad-Keldysh field theory, we discuss on the one hand the principles delimiting thermodynamic equilibrium from driven open stationary states, and on the other hand show how unifying concepts such as symmetries, the purity of states, and scaling arguments are implemented. We then present instances of universal behavior structured into three classes: new realizations of paradigmatic nonequilibrium phenomena, including a survey of first experimental realizations; novel instances of nonequilibrium universality found in these systems made of quantum ingredients; and genuinely quantum phenomena out of equilibrium, including in fermionic systems. We also discuss perspectives for future research on driven open quantum matter.

categories: cond-mat.stat-mech cond-mat.quant-gas hep-th quant-ph