title: Steering Llama 2 via Contrastive Activation Addition

abstract: We introduce Contrastive Activation Addition (CAA), an innovative method for steering language models by modifying their activations during forward passes. CAA computes "steering vectors" by averaging the difference in residual stream activations between pairs of positive and negative examples of a particular behavior, such as factual versus hallucinatory responses. During inference, these steering vectors are added at all token positions after the user's prompt with either a positive or negative coefficient, allowing precise control over the degree of the targeted behavior. We evaluate CAA's effectiveness on Llama 2 Chat using multiple-choice behavioral question datasets and open-ended generation tasks. We demonstrate that CAA significantly alters model behavior, is effective over and on top of traditional methods like finetuning and system prompt design, and minimally reduces capabilities. Moreover, we gain deeper insights into CAA's mechanisms by employing various activation space interpretation methods. CAA accurately steers model outputs and sheds light on how high-level concepts are represented in Large Language Models (LLMs).


title: Learning to Denoise Unreliable Interactions for Link Prediction on Biomedical Knowledge Graph

abstract: Link prediction in biomedical knowledge graphs (KGs) aims at predicting unknown interactions between entities, including drug-target interaction (DTI) and drug-drug interaction (DDI), which is critical for drug discovery and therapeutics. Previous methods prefer to utilize the rich semantic relations and topological structure of the KG to predict missing links, yielding promising outcomes. However, all these works only focus on improving the predictive performance without considering the inevitable noise and unreliable interactions existing in the KGs, which limits the development of KG-based computational methods. To address these limitations, we propose a Denoised Link Prediction framework, called DenoisedLP. DenoisedLP obtains reliable interactions based on the local subgraph by denoising noisy links in a learnable way, providing a universal module for mining underlying task-relevant relations. To collaborate with the smoothed semantic information, DenoisedLP introduces the semantic subgraph by blurring conflict relations around the predicted link. By maximizing the mutual information between the reliable structure and smoothed semantic relations, DenoisedLP emphasizes the informative interactions for predicting relation-specific links. Experimental results on real-world datasets demonstrate that DenoisedLP outperforms state-of-the-art methods on DTI and DDI prediction tasks, and verify the effectiveness and robustness of denoising unreliable interactions on the contaminated KGs.


title: AT4CTR: Auxiliary Match Tasks for Enhancing Click-Through Rate Prediction

abstract: Click-through rate (CTR) prediction is a vital task in industrial recommendation systems. Most existing methods focus on the network architecture design of the CTR model for better accuracy and suffer from the data sparsity problem. Especially in industrial recommendation systems, the widely applied negative sample down-sampling technique due to resource limitation worsens the problem, resulting in a decline in performance. In this paper, we propose \textbf{A}uxiliary Match \textbf{T}asks for enhancing \textbf{C}lick-\textbf{T}hrough \textbf{R}ate prediction accuracy (AT4CTR) by alleviating the data sparsity problem. Specifically, we design two match tasks inspired by collaborative filtering to enhance the relevance modeling between user and item. As the "click" action is a strong signal which indicates the user's preference towards the item directly, we make the first match task aim at pulling closer the representation between the user and the item regarding the positive samples. Since the user's past click behaviors can also be treated as the user him/herself, we apply the next item prediction as the second match task. For both the match tasks, we choose the InfoNCE as their loss function. The two match tasks can provide meaningful training signals to speed up the model's convergence and alleviate the data sparsity. We conduct extensive experiments on one public dataset and one large-scale industrial recommendation dataset. The result demonstrates the effectiveness of the proposed auxiliary match tasks. AT4CTR has been deployed in the real industrial advertising system and has gained remarkable revenue.


title: Enhanced E-Commerce Attribute Extraction: Innovating with Decorative Relation Correction and LLAMA 2.0-Based Annotation

abstract: The rapid proliferation of e-commerce platforms accentuates the need for advanced search and retrieval systems to foster a superior user experience. Central to this endeavor is the precise extraction of product attributes from customer queries, enabling refined search, comparison, and other crucial e-commerce functionalities. Unlike traditional Named Entity Recognition (NER) tasks, e-commerce queries present a unique challenge owing to the intrinsic decorative relationship between product types and attributes. In this study, we propose a pioneering framework that integrates BERT for classification, a Conditional Random Fields (CRFs) layer for attribute value extraction, and Large Language Models (LLMs) for data annotation, significantly advancing attribute recognition from customer inquiries. Our approach capitalizes on the robust representation learning of BERT, synergized with the sequence decoding prowess of CRFs, to adeptly identify and extract attribute values. We introduce a novel decorative relation correction mechanism to further refine the extraction process based on the nuanced relationships between product types and attributes inherent in e-commerce data. Employing LLMs, we annotate additional data to expand the model's grasp and coverage of diverse attributes. Our methodology is rigorously validated on various datasets, including Walmart, BestBuy's e-commerce NER dataset, and the CoNLL dataset, demonstrating substantial improvements in attribute recognition performance. Particularly, the model showcased promising results during a two-month deployment in Walmart's Sponsor Product Search, underscoring its practical utility and effectiveness.


title: Causal-CoG: A Causal-Effect Look at Context Generation for Boosting Multi-modal Language Models

abstract: While Multi-modal Language Models (MLMs) demonstrate impressive multimodal ability, they still struggle on providing factual and precise responses for tasks like visual question answering (VQA). In this paper, we address this challenge from the perspective of contextual information. We propose Causal Context Generation, Causal-CoG, which is a prompting strategy that engages contextual information to enhance precise VQA during inference. Specifically, we prompt MLMs to generate contexts, i.e, text description of an image, and engage the generated contexts for question answering. Moreover, we investigate the advantage of contexts on VQA from a causality perspective, introducing causality filtering to select samples for which contextual information is helpful. To show the effectiveness of Causal-CoG, we run extensive experiments on 10 multimodal benchmarks and show consistent improvements, e.g., +6.30% on POPE, +13.69% on Vizwiz and +6.43% on VQAv2 compared to direct decoding, surpassing existing methods. We hope Casual-CoG inspires explorations of context knowledge in multimodal models, and serves as a plug-and-play strategy for MLM decoding.


title: Robo360: A 3D Omnispective Multi-Material Robotic Manipulation Dataset

abstract: Building robots that can automate labor-intensive tasks has long been the core motivation behind the advancements in computer vision and the robotics community. Recent interest in leveraging 3D algorithms, particularly neural fields, has led to advancements in robot perception and physical understanding in manipulation scenarios. However, the real world's complexity poses significant challenges. To tackle these challenges, we present Robo360, a dataset that features robotic manipulation with a dense view coverage, which enables high-quality 3D neural representation learning, and a diverse set of objects with various physical and optical properties and facilitates research in various object manipulation and physical world modeling tasks. We confirm the effectiveness of our dataset using existing dynamic NeRF and evaluate its potential in learning multi-view policies. We hope that Robo360 can open new research directions yet to be explored at the intersection of understanding the physical world in 3D and robot control.


title: Prime orbit theorems for expanding Thurston maps: Genericity of strong non-integrability condition

abstract: An analog of the prime number theorem for a class of branched covering maps on the $2$-sphere $S^2$ called expanding Thurston maps, which are topological models of some non-uniformly expanding rational maps without any smoothness or holomorphicity assumption, is obtained in the second paper of this series. More precisely, the number of primitive periodic orbits, ordered by a weight on each point induced by a non-constant (eventually) positive real-valued H\"{o}lder continuous function on $S^2$ satisfying the $\alpha$-strong non-integrability condition, is asymptotically the same as the well-known logarithmic integral, with an exponential error term. In this third and last paper of the series, we show that the $\alpha$-strong non-integrability condition is generic in the class of $\alpha$-H\"{o}lder continuous functions.


title: Prime orbit theorems for expanding Thurston maps: Latt\`es maps and split Ruelle operators

abstract: We obtain an analog of the prime number theorem for a class of branched covering maps on the $2$-sphere $S^2$ called expanding Thurston maps, which are topological models of some non-uniformly expanding rational maps without any smoothness or holomorphicity assumption. More precisely, we show that the number of primitive periodic orbits, ordered by a weight on each point induced by a non-constant (eventually) positive real-valued H\"{o}lder continuous function on $S^2$ satisfying some additional regularity conditions, is asymptotically the same as the well-known logarithmic integral, with an exponential error term. In particular, our results apply to postcritically-finite rational maps for which the Julia set is the whole Riemann sphere. Moreover, a stronger result is obtained for Latt\`{e}s maps.


title: Introduction to IoT

abstract: The Internet of Things has rapidly transformed the 21st century, enhancing decision-making processes and introducing innovative consumer services such as pay-as-you-use models. The integration of smart devices and automation technologies has revolutionized every aspect of our lives, from health services to the manufacturing industry, and from the agriculture sector to mining. Alongside the positive aspects, it is also essential to recognize the significant safety, security, and trust concerns in this technological landscape. This chapter serves as a comprehensive guide for newcomers interested in the IoT domain, providing a foundation for making future contributions. Specifically, it discusses the overview, historical evolution, key characteristics, advantages, architectures, taxonomy of technologies, and existing applications in major IoT domains. In addressing prevalent issues and challenges in designing and deploying IoT applications, the chapter examines security threats across architectural layers, ethical considerations, user privacy concerns, and trust-related issues. This discussion equips researchers with a solid understanding of diverse IoT aspects, providing a comprehensive understanding of IoT technology along with insights into the extensive potential and impact of this transformative field.


title: Backward Stochastic Differential Equations in Financial Mathematics

abstract: A backward stochastic differential equation (BSDE) is an SDE of the form $-dY_t = f(t,Y_t,Z_t)dt - Z_t^*dW_t;\ Y_T = \xi$. The subject of BSDEs has seen extensive attention since their introduction in the linear case by Bismut (1973) and in the general case by Pardoux and Peng (1990). In contrast with deterministic differential equations, it is not enough to simply reverse the direction of time and treat the terminal condition as an initial condition, as we would then run into problems with adaptedness. Intuitively, our "knowledge" at time $t$ consists only of what has happened at all times $s \in [0,t]$, and we cannot reverse the direction of time whilst keeping this true. The layout of this essay is as follows: In Section 1 we introduce BSDEs and go over the basic results of BSDE theory, including two major theorems: the existence and uniqueness of solutions and the comparison theorem. We also introduce linear BSDEs and the notion of supersolutions of a BSDE. In Section 2 we set up the financial framework in which we will price European contingent claims, and prove a result about the fair price of such claims in a dynamically complete market. In Section 3 we extend the theory of linear BSDEs to include concave BSDEs, and apply this to pricing claims in more complicated market models. In Section 4 we take a look at utility maximisation problems, and see how utilising BSDE theory allows for a relatively simple and neat solution in certain cases.


title: Involutive Yang-Baxter groups never act as Frobenius groups

abstract: A conjecture of S. Ram\'{\i}rez states that every indecomposable non-degenerate involutive set-theoretic solution to the Yang-Baxter equation with dihedral permutation group of order $2n$ has cardinality $2n$. The conjecture is verified for odd $n$ and disproved for even $n$. The proof for odd $n$ is obtained from the more general result that the permutation group of a finite solution never acts as a Frobenius group.


title: Extra Attraction Generated by Spacetime Fluctuations

abstract: We show that, due to the nonlinear nature of gravity, fluctuations in spacetime curvature generate additional gravitational attraction. This fluctuation-induced extra attraction was overlooked in the conventional understanding of the cosmological constant problem. If the quantum vacuum of matter fields possesses positive energy and negative pressure, it would produce enormous gravitational repulsion, resulting in a catastrophic explosion of the universe -- the acceleration of the universe's expansion would exceed the observed value by some 120 orders of magnitude. We argue that such an enormous repulsion produced by the violent matter fields vacuum can be completely suppressed by the even more substantial attraction generated by the zero-point fluctuations in the spacetime curvature. As a result, the predicted catastrophic explosion of the universe is averted. Furthermore, at small microscopic scales, the structure of spacetime becomes locally highly inhomogeneous and anisotropic. When averaged over large macroscopic scales, the zero-point fluctuations of spacetime itself could drive the observed slow acceleration of the universe's expansion through a subtle parametric resonance effect.


title: The parastatistics of braided Majorana fermions

abstract: This paper presents the parastatistics of braided Majorana fermions obtained in the framework of a graded Hopf algebra endowed with a braided tensor product. The braiding property is encoded in a $t$-dependent $4\times 4$ braiding matrix $B_t$ related to the Alexander-Conway polynomial. The nonvanishing complex parameter t defines the braided parastatistics. At $t = 1$ ordinary fermions are recovered. The values of $t$ at roots of unity are organized into levels which specify the maximal number of braided Majorana fermions in a multiparticle sector. Generic values of $t$ and the $t =-1$ root of unity mimick the behaviour of ordinary bosons.

categories: hep-th cond-mat.stat-mech math-ph math.MP quant-ph


title: An Inquiry Into The Economic Linkages Between The Swedish Air Transport Sector And The Economy As A Whole In The Context Of The Covid-19 Pandemic

abstract: This thesis aims to assess the importance of the air transport sector for Sweden's economy in the context of the COVID-19 pandemic. Two complementary research goals are formulated. Firstly, investigating economic linkages of the Swedish air transport sector. Secondly, estimating the effects of the pandemic on Swedish air transport, and the spin-off effects on the economy as a whole. Overview of literature in the field reveals that while a fair amount of research exists on the importance of air transport, unsurprisingly, pandemic effects have yet to be investigated. The methodological framework chosen is input-output analysis, thus, the backbone of data materials used is the Swedish input-output table, complemented by additional datasets. For measuring economic linkages, basic input-output analysis is applied. Meanwhile, to capture the pandemic's impacts, a combination of inoperability analysis and the partial hypothetical extraction model is implemented. It is found that while Swedish air transport plays an important role in turning the cogs of the Swedish economy, when compared to all other sectors, it ranks on the lower end. Furthermore, while the COVID-19 pandemic has a detrimental short-term impact on Swedish air transport, the spin-off effects for the economy as a whole are milder. It is concluded, that out of a value-added perspective, the Swedish government, and by extension policy makers globally need not support failing airlines, and other air transport actors. Nonetheless, some aspects could not be captured through the methods used, such as the possible importance of airlines to national security at times of dodgy global cooperation. Lastly, to accurately capture not only short-term, but also long-term effects of the pandemic, future research using alternative, causality-based frameworks and dynamic input-output models will be highly beneficial.


title: Evolving Reservoirs for Meta Reinforcement Learning

abstract: Animals often demonstrate a remarkable ability to adapt to their environments during their lifetime. They do so partly due to the evolution of morphological and neural structures. These structures capture features of environments shared between generations to bias and speed up lifetime learning. In this work, we propose a computational model for studying a mechanism that can enable such a process. We adopt a computational framework based on meta reinforcement learning as a model of the interplay between evolution and development. At the evolutionary scale, we evolve reservoirs, a family of recurrent neural networks that differ from conventional networks in that one optimizes not the synaptic weights, but hyperparameters controlling macro-level properties of the resulting network architecture. At the developmental scale, we employ these evolved reservoirs to facilitate the learning of a behavioral policy through Reinforcement Learning (RL). Within an RL agent, a reservoir encodes the environment state before providing it to an action policy. We evaluate our approach on several 2D and 3D simulated environments. Our results show that the evolution of reservoirs can improve the learning of diverse challenging tasks. We study in particular three hypotheses: the use of an architecture combining reservoirs and reinforcement learning could enable (1) solving tasks with partial observability, (2) generating oscillatory dynamics that facilitate the learning of locomotion tasks, and (3) facilitating the generalization of learned behaviors to new tasks unknown during the evolution phase.


title: Observation of ultra-high-Q resonators in the ultrasound via bound states in the continuum

abstract: The confinement of waves in open systems represents a fundamental phenomenon extensively explored across various branches of wave physics. Recently, significant attention has been directed towards bound states in the continuum (BIC), a class of modes that are trapped but do not decay in an otherwise unbounded continuum. Here, we theoretically investigate and experimentally demonstrate the existence of quasi-BIC (QBIC) for ultrasonic waves by leveraging an elastic Fabry-P\'erot metasurface resonator. We unveil several intriguing properties of the ultrasound QBIC that are robust to parameter scanning, and we present experimental evidence of a remarkable Q-factor of 350 at around 1 MHz frequency, far exceeding the state-of-the-art using a fully acoustic underwater system. Our findings contribute novel insights into the understanding of BIC for acoustic waves, offering a new paradigm for the design of efficient, ultra-high Q-factor ultrasound devices.


title: Performance of externally validated machine learning models based on histopathology images for the diagnosis, classification, prognosis, or treatment outcome prediction in female breast cancer: A systematic review

abstract: Numerous machine learning (ML) models have been developed for breast cancer using various types of data. Successful external validation (EV) of ML models is important evidence of their generalizability. The aim of this systematic review was to assess the performance of externally validated ML models based on histopathology images for diagnosis, classification, prognosis, or treatment outcome prediction in female breast cancer. A systematic search of MEDLINE, EMBASE, CINAHL, IEEE, MICCAI, and SPIE conferences was performed for studies published between January 2010 and February 2022. The Prediction Model Risk of Bias Assessment Tool (PROBAST) was employed, and the results were narratively described. Of the 2011 non-duplicated citations, 8 journal articles and 2 conference proceedings met inclusion criteria. Three studies externally validated ML models for diagnosis, 4 for classification, 2 for prognosis, and 1 for both classification and prognosis. Most studies used Convolutional Neural Networks and one used logistic regression algorithms. For diagnostic/classification models, the most common performance metrics reported in the EV were accuracy and area under the curve, which were greater than 87% and 90%, respectively, using pathologists' annotations as ground truth. The hazard ratios in the EV of prognostic ML models were between 1.7 (95% CI, 1.2-2.6) and 1.8 (95% CI, 1.3-2.7) to predict distant disease-free survival; 1.91 (95% CI, 1.11-3.29) for recurrence, and between 0.09 (95% CI, 0.01-0.70) and 0.65 (95% CI, 0.43-0.98) for overall survival, using clinical data as ground truth. Despite EV being an important step before the clinical application of a ML model, it hasn't been performed routinely. The large variability in the training/validation datasets, methods, performance metrics, and reported information limited the comparison of the models and the analysis of their results (...)


title: Tiling Ferrers Diagrams

abstract: We will show that a necessary and sufficient condition for a Ferrers board (or Young Diagrams) to be fully tileable with 1x2 dominoes requires the board to be 2-colorable such that no color is adjacent to its own color using both induction and a graph theory approach. We will walk through all prerequisite knowledge and go through the failed attempts we tried while also providing supplementary exercises that fit the topics. Plenty of background content is included, so even if you don't know much about the subject, it should still be readable. If you do know most of the background content, feel free to skip around.


title: Leveraging Generative Language Models for Weakly Supervised Sentence Component Analysis in Video-Language Joint Learning

abstract: A thorough comprehension of textual data is a fundamental element in multi-modal video analysis tasks. However, recent works have shown that the current models do not achieve a comprehensive understanding of the textual data during the training for the target downstream tasks. Orthogonal to the previous approaches to this limitation, we postulate that understanding the significance of the sentence components according to the target task can potentially enhance the performance of the models. Hence, we utilize the knowledge of a pre-trained large language model (LLM) to generate text samples from the original ones, targeting specific sentence components. We propose a weakly supervised importance estimation module to compute the relative importance of the components and utilize them to improve different video-language tasks. Through rigorous quantitative analysis, our proposed method exhibits significant improvement across several video-language tasks. In particular, our approach notably enhances video-text retrieval by a relative improvement of 8.3\% in video-to-text and 1.4\% in text-to-video retrieval over the baselines, in terms of R@1. Additionally, in video moment retrieval, average mAP shows a relative improvement ranging from 2.0\% to 13.7 \% across different baselines.


title: Design and Architecture for a Centralized, Extensible, and Configurable Scoring Application

abstract: In modern-day organizations, many software applications require critical input to decide the next steps in the application workflow and approval. One of the most important inputs to decide the subsequent course of action is the key performance indicator-based scoring for the entities used in the application. Computing the right score for the entities in the application is a critical step that will drive the subsequent processing and help to decide the next course of action for the entity accurately. Computing the right score is a critical parameter for application processing; deriving the precise and correct score is crucial and pivotal for the application's intended objective; this mandates a very efficient and optimized scoring application in place and is of paramount importance for the success of such applications. We will discuss in this article how to envision and design a generic, extensible scoring engine and a few use cases for scoring with the associated intricacies and complexities to implement the scoring framework.


title: Dynamic Adversarial Attacks on Autonomous Driving Systems

abstract: This paper introduces an attacking mechanism to challenge the resilience of autonomous driving systems. Specifically, we manipulate the decision-making processes of an autonomous vehicle by dynamically displaying adversarial patches on a screen mounted on another moving vehicle. These patches are optimized to deceive the object detection models into misclassifying targeted objects, e.g., traffic signs. Such manipulation has significant implications for critical multi-vehicle interactions such as intersection crossing and lane changing, which are vital for safe and efficient autonomous driving systems. Particularly, we make four major contributions. First, we introduce a novel adversarial attack approach where the patch is not co-located with its target, enabling more versatile and stealthy attacks. Moreover, our method utilizes dynamic patches displayed on a screen, allowing for adaptive changes and movement, enhancing the flexibility and performance of the attack. To do so, we design a Screen Image Transformation Network (SIT-Net), which simulates environmental effects on the displayed images, narrowing the gap between simulated and real-world scenarios. Further, we integrate a positional loss term into the adversarial training process to increase the success rate of the dynamic attack. Finally, we shift the focus from merely attacking perceptual systems to influencing the decision-making algorithms of self-driving systems. Our experiments demonstrate the first successful implementation of such dynamic adversarial attacks in real-world autonomous driving scenarios, paving the way for advancements in the field of robust and secure autonomous driving.


title: Polar Linear Canonical Wavelet Transform: Theory and Its Application

abstract: The polar wavelet transform (PWT) has been proven to be a powerful mathematical tool for signal and image processing in recent years. Due to the increasing demand for directional representations of signals in engineering, it is impossible to fully exploit the intrinsic directional features of signals to describe high-dimensional signals like images. Focusing on this problem, the polar linear canonical wavelet transform (PLCWT) is proposed in this paper. Firstly, the theory of the PLCWT is investigated in detail, including its definition, basic properties and inversion formula. Secondly, the convolution and correlation theorems of the PLCWT are derived. Thirdly, uncertainty principles related to the PLCWT are obtained. Finally, the potential application of the PLCWT in image edge detection is discussed. Simulation results verify the correctness and effectiveness of the proposed method.


title: OpenSD: Unified Open-Vocabulary Segmentation and Detection

abstract: Recently, a few open-vocabulary methods have been proposed by employing a unified architecture to tackle generic segmentation and detection tasks. However, their performance still lags behind the task-specific models due to the conflict between different tasks, and their open-vocabulary capability is limited due to the inadequate use of CLIP. To address these challenges, we present a universal transformer-based framework, abbreviated as OpenSD, which utilizes the same architecture and network parameters to handle open-vocabulary segmentation and detection tasks. First, we introduce a decoder decoupled learning strategy to alleviate the semantic conflict between thing and staff categories so that each individual task can be learned more effectively under the same framework. Second, to better leverage CLIP for end-to-end segmentation and detection, we propose dual classifiers to handle the in-vocabulary domain and out-of-vocabulary domain, respectively. The text encoder is further trained to be region-aware for both thing and stuff categories through decoupled prompt learning, enabling them to filter out duplicated and low-quality predictions, which is important to end-to-end segmentation and detection. Extensive experiments are conducted on multiple datasets under various circumstances. The results demonstrate that OpenSD outperforms state-of-the-art open-vocabulary segmentation and detection methods in both closed- and open-vocabulary settings. Code is available at https://github.com/strongwolf/OpenSD


title: SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction

abstract: Creating high-quality 3D models of clothed humans from single images for real-world applications is crucial. Despite recent advancements, accurately reconstructing humans in complex poses or with loose clothing from in-the-wild images, along with predicting textures for unseen areas, remains a significant challenge. A key limitation of previous methods is their insufficient prior guidance in transitioning from 2D to 3D and in texture prediction. In response, we introduce SIFU (Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction), a novel approach combining a Side-view Decoupling Transformer with a 3D Consistent Texture Refinement pipeline.SIFU employs a cross-attention mechanism within the transformer, using SMPL-X normals as queries to effectively decouple side-view features in the process of mapping 2D features to 3D. This method not only improves the precision of the 3D models but also their robustness, especially when SMPL-X estimates are not perfect. Our texture refinement process leverages text-to-image diffusion-based prior to generate realistic and consistent textures for invisible views. Through extensive experiments, SIFU surpasses SOTA methods in both geometry and texture reconstruction, showcasing enhanced robustness in complex scenarios and achieving an unprecedented Chamfer and P2S measurement. Our approach extends to practical applications such as 3D printing and scene building, demonstrating its broad utility in real-world scenarios. Project page https://river-zhang.github.io/SIFU-projectpage/ .


title: Perceiving University Student's Opinions from Google App Reviews

abstract: Google app market captures the school of thought of users from every corner of the globe via ratings and text reviews, in a multilinguistic arena. The potential information from the reviews cannot be extracted manually, due to its exponential growth. So, Sentiment analysis, by machine learning and deep learning algorithms employing NLP, explicitly uncovers and interprets the emotions. This study performs the sentiment classification of the app reviews and identifies the university student's behavior towards the app market via exploratory analysis. We applied machine learning algorithms using the TP, TF, and TF IDF text representation scheme and evaluated its performance on Bagging, an ensemble learning method. We used word embedding, Glove, on the deep learning paradigms. Our model was trained on Google app reviews and tested on Student's App Reviews(SAR). The various combinations of these algorithms were compared amongst each other using F score and accuracy and inferences were highlighted graphically. SVM, amongst other classifiers, gave fruitful accuracy(93.41%), F score(89%) on bigram and TF IDF scheme. Bagging enhanced the performance of LR and NB with accuracy of 87.88% and 86.69% and F score of 86% and 78% respectively. Overall, LSTM on Glove embedding recorded the highest accuracy(95.2%) and F score(88%).


title: UNeR3D: Versatile and Scalable 3D RGB Point Cloud Generation from 2D Images in Unsupervised Reconstruction

abstract: In the realm of 3D reconstruction from 2D images, a persisting challenge is to achieve high-precision reconstructions devoid of 3D Ground Truth data reliance. We present UNeR3D, a pioneering unsupervised methodology that sets a new standard for generating detailed 3D reconstructions solely from 2D views. Our model significantly cuts down the training costs tied to supervised approaches and introduces RGB coloration to 3D point clouds, enriching the visual experience. Employing an inverse distance weighting technique for color rendering, UNeR3D ensures seamless color transitions, enhancing visual fidelity. Our model's flexible architecture supports training with any number of views, and uniquely, it is not constrained by the number of views used during training when performing reconstructions. It can infer with an arbitrary count of views during inference, offering unparalleled versatility. Additionally, the model's continuous spatial input domain allows the generation of point clouds at any desired resolution, empowering the creation of high-resolution 3D RGB point clouds. We solidify the reconstruction process with a novel multi-view geometric loss and color loss, demonstrating that our model excels with single-view inputs and beyond, thus reshaping the paradigm of unsupervised learning in 3D vision. Our contributions signal a substantial leap forward in 3D vision, offering new horizons for content creation across diverse applications. Code is available at https://github.com/HongbinLin3589/UNeR3D.


title: Exploring Public's Perception of Safety and Video Surveillance Technology: A Survey Approach

abstract: Addressing public safety effectively requires incorporating diverse stakeholder perspectives, particularly those of the community, which are often underrepresented compared to other stakeholders. This study presents a comprehensive analysis of the community's general public safety concerns, their view of existing surveillance technologies, and their perception of AI-driven solutions for enhancing safety in urban environments, focusing on Charlotte, NC. Through a survey approach, including in-person surveys conducted in August and September 2023 with 410 participants, this research investigates demographic factors such as age, gender, ethnicity, and educational level to gain insights into public perception and concerns toward public safety and possible solutions. Based on the type of dependent variables, we utilized different statistical and significance analyses, such as logit regression and ordinal logistic regression, to explore the effects of demographic factors on the various dependent variables. Our results reveal demographic differences in public safety concerns. Younger females tend to feel less secure yet trust existing video surveillance systems, whereas older, educated individuals are more concerned about violent crimes in malls. Additionally, attitudes towards AI-driven surveillance differ: older Black individuals demonstrate support for it despite having concerns about data privacy, while educated females show a tendency towards skepticism.


title: Neutral Editing Framework for Diffusion-based Video Editing

abstract: Text-conditioned image editing has succeeded in various types of editing based on a diffusion framework. Unfortunately, this success did not carry over to a video, which continues to be challenging. Existing video editing systems are still limited to rigid-type editing such as style transfer and object overlay. To this end, this paper proposes Neutral Editing (NeuEdit) framework to enable complex non-rigid editing by changing the motion of a person/object in a video, which has never been attempted before. NeuEdit introduces a concept of `neutralization' that enhances a tuning-editing process of diffusion-based editing systems in a model-agnostic manner by leveraging input video and text without any other auxiliary aids (e.g., visual masks, video captions). Extensive experiments on numerous videos demonstrate adaptability and effectiveness of the NeuEdit framework. The website of our work is available here: https://neuedit.github.io


title: AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One

abstract: A handful of visual foundation models (VFMs) have recently emerged as the backbones for numerous downstream tasks. VFMs like CLIP, DINOv2, SAM are trained with distinct objectives, exhibiting unique characteristics for various downstream tasks. We find that despite their conceptual differences, these models can be effectively merged into a unified model through multi-teacher distillation. We name this approach AM-RADIO (Agglomerative Model -- Reduce All Domains Into One). This integrative approach not only surpasses the performance of individual teacher models but also amalgamates their distinctive features, such as zero-shot vision-language comprehension, detailed pixel-level understanding, and open vocabulary segmentation capabilities. In pursuit of the most hardware-efficient backbone, we evaluated numerous architectures in our multi-teacher distillation pipeline using the same training recipe. This led to the development of a novel architecture (E-RADIO) that exceeds the performance of its predecessors and is at least 7x faster than the teacher models. Our comprehensive benchmarking process covers downstream tasks including ImageNet classification, ADE20k semantic segmentation, COCO object detection and LLaVa-1.5 framework. Code: https://github.com/NVlabs/RADIO


title: Class-Prototype Conditional Diffusion Model with Gradient Projection for Continual Learning

abstract: Mitigating catastrophic forgetting is a key hurdle in continual learning. Deep Generative Replay (GR) provides techniques focused on generating samples from prior tasks to enhance the model's memory capabilities using generative AI models ranging from Generative Adversarial Networks (GANs) to the more recent Diffusion Models (DMs). A major issue is the deterioration in the quality of generated data compared to the original, as the generator continuously self-learns from its outputs. This degradation can lead to the potential risk of catastrophic forgetting (CF) occurring in the classifier. To address this, we propose the Gradient Projection Class-Prototype Conditional Diffusion Model (GPPDM), a GR-based approach for continual learning that enhances image quality in generators and thus reduces the CF in classifiers. The cornerstone of GPPDM is a learnable class prototype that captures the core characteristics of images in a given class. This prototype, integrated into the diffusion model's denoising process, ensures the generation of high-quality images of the old tasks, hence reducing the risk of CF in classifiers. Moreover, to further mitigate the CF of diffusion models, we propose a gradient projection technique tailored for the cross-attention layer of diffusion models to maximally maintain and preserve the representations of old task data in the current task as close as possible to their representations when they first arrived. Our empirical studies on diverse datasets demonstrate that our proposed method significantly outperforms existing state-of-the-art models, highlighting its satisfactory ability to preserve image quality and enhance the model's memory retention.


title: Physics Informed Neural Network for Option Pricing

abstract: We apply a physics-informed deep-learning approach the PINN approach to the Black-Scholes equation for pricing American and European options. We test our approach on both simulated as well as real market data, compare it to analytical/numerical benchmarks. Our model is able to accurately capture the price behaviour on simulation data, while also exhibiting reasonable performance for market data. We also experiment with the architecture and learning process of our PINN model to provide more understanding of convergence and stability issues that impact performance.


title: Separate-and-Enhance: Compositional Finetuning for Text2Image Diffusion Models

abstract: Despite recent significant strides achieved by diffusion-based Text-to-Image (T2I) models, current systems are still less capable of ensuring decent compositional generation aligned with text prompts, particularly for the multi-object generation. This work illuminates the fundamental reasons for such misalignment, pinpointing issues related to low attention activation scores and mask overlaps. While previous research efforts have individually tackled these issues, we assert that a holistic approach is paramount. Thus, we propose two novel objectives, the Separate loss and the Enhance loss, that reduce object mask overlaps and maximize attention scores, respectively. Our method diverges from conventional test-time-adaptation techniques, focusing on finetuning critical parameters, which enhances scalability and generalizability. Comprehensive evaluations demonstrate the superior performance of our model in terms of image realism, text-image alignment, and adaptability, notably outperforming prominent baselines. Ultimately, this research paves the way for T2I diffusion models with enhanced compositional capacities and broader applicability.


title: TeTriRF: Temporal Tri-Plane Radiance Fields for Efficient Free-Viewpoint Video

abstract: Neural Radiance Fields (NeRF) revolutionize the realm of visual media by providing photorealistic Free-Viewpoint Video (FVV) experiences, offering viewers unparalleled immersion and interactivity. However, the technology's significant storage requirements and the computational complexity involved in generation and rendering currently limit its broader application. To close this gap, this paper presents Temporal Tri-Plane Radiance Fields (TeTriRF), a novel technology that significantly reduces the storage size for Free-Viewpoint Video (FVV) while maintaining low-cost generation and rendering. TeTriRF introduces a hybrid representation with tri-planes and voxel grids to support scaling up to long-duration sequences and scenes with complex motions or rapid changes. We propose a group training scheme tailored to achieving high training efficiency and yielding temporally consistent, low-entropy scene representations. Leveraging these properties of the representations, we introduce a compression pipeline with off-the-shelf video codecs, achieving an order of magnitude less storage size compared to the state-of-the-art. Our experiments demonstrate that TeTriRF can achieve competitive quality with a higher compression rate.


title: Sensitivity analysis for mixed binary quadratic programming

abstract: We consider sensitivity analysis for Mixed Binary Quadratic Programs (MBQPs) with respect to changing right-hand-sides (rhs). We show that even if the optimal solution of a given MBQP is known, it is NP-hard to approximate the change in objective function value with respect to changes in rhs. Next, we study algorithmic approaches to obtaining dual bounds for MBQP with changing rhs. We leverage Burer's completely-positive (CPP) reformulation of MBQPs. Its dual is an instance of co-positive programming (COP), and can be used to obtain sensitivity bounds. We prove that strong duality between the CPP and COP problems holds if the feasible region is bounded or if the objective function is convex, while the duality gap can be strictly positive if neither condition is met. We also show that the COP dual has multiple optimal solutions, and the choice of the dual solution affects the quality of the bounds with rhs changes. We finally provide a method for finding good nearly optimal dual solutions, and we present preliminary computational results on sensitivity analysis for MBQPs.


title: The improved backward compatible physics-informed neural networks for reducing error accumulation and applications in data-driven higher-order rogue waves

abstract: Due to the dynamic characteristics of instantaneity and steepness, employing domain decomposition techniques for simulating rogue wave solutions is highly appropriate. Wherein, the backward compatible PINN (bc-PINN) is a temporally sequential scheme to solve PDEs over successive time segments while satisfying all previously obtained solutions. In this work, we propose improvements to the original bc-PINN algorithm in two aspects based on the characteristics of error propagation. One is to modify the loss term for ensuring backward compatibility by selecting the earliest learned solution for each sub-domain as pseudo reference solution. The other is to adopt the concatenation of solutions obtained from individual subnetworks as the final form of the predicted solution. The improved backward compatible PINN (Ibc-PINN) is applied to study data-driven higher-order rogue waves for the nonlinear Schr\"{o}dinger (NLS) equation and the AB system to demonstrate the effectiveness and advantages. Transfer learning and initial condition guided learning (ICGL) techniques are also utilized to accelerate the training. Moreover, the error analysis is conducted on each sub-domain and it turns out that the slowdown of Ibc-PINN in error accumulation speed can yield greater advantages in accuracy. In short, numerical results fully indicate that Ibc-PINN significantly outperforms bc-PINN in terms of accuracy and stability without sacrificing efficiency.


title: Deciphering 'What' and 'Where' Visual Pathways from Spectral Clustering of Layer-Distributed Neural Representations

abstract: We present an approach for analyzing grouping information contained within a neural network's activations, permitting extraction of spatial layout and semantic segmentation from the behavior of large pre-trained vision models. Unlike prior work, our method conducts a holistic analysis of a network's activation state, leveraging features from all layers and obviating the need to guess which part of the model contains relevant information. Motivated by classic spectral clustering, we formulate this analysis in terms of an optimization objective involving a set of affinity matrices, each formed by comparing features within a different layer. Solving this optimization problem using gradient descent allows our technique to scale from single images to dataset-level analysis, including, in the latter, both intra- and inter-image relationships. Analyzing a pre-trained generative transformer provides insight into the computational strategy learned by such models. Equating affinity with key-query similarity across attention layers yields eigenvectors encoding scene spatial layout, whereas defining affinity by value vector similarity yields eigenvectors encoding object identity. This result suggests that key and query vectors coordinate attentional information flow according to spatial proximity (a `where' pathway), while value vectors refine a semantic category representation (a `what' pathway).


title: Privacy Issues in Large Language Models: A Survey

abstract: This is the first survey of the active area of AI research that focuses on privacy issues in Large Language Models (LLMs). Specifically, we focus on work that red-teams models to highlight privacy risks, attempts to build privacy into the training or inference process, enables efficient data deletion from trained models to comply with existing privacy regulations, and tries to mitigate copyright issues. Our focus is on summarizing technical research that develops algorithms, proves theorems, and runs empirical evaluations. While there is an extensive body of legal and policy work addressing these challenges from a different angle, that is not the focus of our survey. Nevertheless, these works, along with recent legal developments do inform how these technical problems are formalized, and so we discuss them briefly in Section 1. While we have made our best effort to include all the relevant work, due to the fast moving nature of this research we may have missed some recent work. If we have missed some of your work please contact us, as we will attempt to keep this survey relatively up to date. We are maintaining a repository with the list of papers covered in this survey and any relevant code that was publicly available at https://github.com/safr-ml-lab/survey-llm.


title: Large Scale Foundation Models for Intelligent Manufacturing Applications: A Survey

abstract: Although the applications of artificial intelligence especially deep learning had greatly improved various aspects of intelligent manufacturing, they still face challenges for wide employment due to the poor generalization ability, difficulties to establish high-quality training datasets, and unsatisfactory performance of deep learning methods. The emergence of large scale foundational models(LSFMs) had triggered a wave in the field of artificial intelligence, shifting deep learning models from single-task, single-modal, limited data patterns to a paradigm encompassing diverse tasks, multimodal, and pre-training on massive datasets. Although LSFMs had demonstrated powerful generalization capabilities, automatic high-quality training dataset generation and superior performance across various domains, applications of LSFMs on intelligent manufacturing were still in their nascent stage. A systematic overview of this topic was lacking, especially regarding which challenges of deep learning can be addressed by LSFMs and how these challenges can be systematically tackled. To fill this gap, this paper systematically expounded current statue of LSFMs and their advantages in the context of intelligent manufacturing. and compared comprehensively with the challenges faced by current deep learning models in various intelligent manufacturing applications. We also outlined the roadmaps for utilizing LSFMs to address these challenges. Finally, case studies of applications of LSFMs in real-world intelligent manufacturing scenarios were presented to illustrate how LSFMs could help industries, improve their efficiency.


title: SkyScenes: A Synthetic Dataset for Aerial Scene Understanding

abstract: Real-world aerial scene understanding is limited by a lack of datasets that contain densely annotated images curated under a diverse set of conditions. Due to inherent challenges in obtaining such images in controlled real-world settings, we present SkyScenes, a synthetic dataset of densely annotated aerial images captured from Unmanned Aerial Vehicle (UAV) perspectives. We carefully curate SkyScenes images from CARLA to comprehensively capture diversity across layout (urban and rural maps), weather conditions, times of day, pitch angles and altitudes with corresponding semantic, instance and depth annotations. Through our experiments using SkyScenes, we show that (1) Models trained on SkyScenes generalize well to different real-world scenarios, (2) augmenting training on real images with SkyScenes data can improve real-world performance, (3) controlled variations in SkyScenes can offer insights into how models respond to changes in viewpoint conditions, and (4) incorporating additional sensor modalities (depth) can improve aerial scene understanding.


title: Audio-Visual LLM for Video Understanding

abstract: This paper presents Audio-Visual LLM, a Multimodal Large Language Model that takes both visual and auditory inputs for holistic video understanding. A key design is the modality-augmented training, which involves the integration of modality-specific tokens engineered to activate the appropriate visual and/or auditory encoder selectively. This mechanism is pivotal in enabling end-to-end joint training with video data at different modalities, including visual-only, audio-only, and audio-visual formats. Moreover, we introduce a high-quality video instruction dataset, derived from GPT-4. This dataset allows Audio-Visual LLM to adeptly process a variety of task-oriented video instructions, ranging from multi-turn conversations and audio-visual narratives to complex reasoning tasks. Extensive experiments demonstrate that Audio-Visual LLM impressively achieves strong zero-shot results across a range of video understanding tasks. For example, Audio-Visual LLM achieves an accuracy of 53.7% on MSRVTT-QA, outperforming non-LLM-based InterVideo by 6.6% and LLM-based Valley by 4.4%, respectively. Additionally, our Audio-Visual LLM also achieves competitive performance on audio tasks (e.g., AudioCaps).


title: Counterfactual World Modeling for Physical Dynamics Understanding

abstract: The ability to understand physical dynamics is essential to learning agents acting in the world. This paper presents Counterfactual World Modeling (CWM), a candidate pure vision foundational model for physical dynamics understanding. CWM consists of three basic concepts. First, we propose a simple and powerful temporally-factored masking policy for masked prediction of video data, which encourages the model to learn disentangled representations of scene appearance and dynamics. Second, as a result of the factoring, CWM is capable of generating counterfactual next-frame predictions by manipulating a few patch embeddings to exert meaningful control over scene dynamics. Third, the counterfactual modeling capability enables the design of counterfactual queries to extract vision structures similar to keypoints, optical flows, and segmentations, which are useful for dynamics understanding. We show that zero-shot readouts of these structures extracted by the counterfactual queries attain competitive performance to prior methods on real-world datasets. Finally, we demonstrate that CWM achieves state-of-the-art performance on the challenging Physion benchmark for evaluating physical dynamics understanding.


title: EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning

abstract: The pursuit of artificial general intelligence (AGI) has been accelerated by Multimodal Large Language Models (MLLMs), which exhibit superior reasoning, generalization capabilities, and proficiency in processing multimodal inputs. A crucial milestone in the evolution of AGI is the attainment of human-level planning, a fundamental ability for making informed decisions in complex environments, and solving a wide range of real-world problems. Despite the impressive advancements in MLLMs, a question remains: How far are current MLLMs from achieving human-level planning? To shed light on this question, we introduce EgoPlan-Bench, a comprehensive benchmark to evaluate the planning abilities of MLLMs in real-world scenarios from an egocentric perspective, mirroring human perception. EgoPlan-Bench emphasizes the evaluation of planning capabilities of MLLMs, featuring realistic tasks, diverse action plans, and intricate visual observations. Our rigorous evaluation of a wide range of MLLMs reveals that EgoPlan-Bench poses significant challenges, highlighting a substantial scope for improvement in MLLMs to achieve human-level task planning. To facilitate this advancement, we further present EgoPlan-IT, a specialized instruction-tuning dataset that effectively enhances model performance on EgoPlan-Bench. We have made all codes, data, and a maintained benchmark leaderboard available to advance future research.


title: Learning to See Low-Light Images via Feature Domain Adaptation

abstract: Raw low light image enhancement (LLIE) has achieved much better performance than the sRGB domain enhancement methods due to the merits of raw data. However, the ambiguity between noisy to clean and raw to sRGB mappings may mislead the single-stage enhancement networks. The two-stage networks avoid ambiguity by decoupling the two mappings but usually have large computing complexity. To solve this problem, we propose a single-stage network empowered by Feature Domain Adaptation (FDA) to decouple the denoising and color mapping tasks in raw LLIE. The denoising encoder is supervised by the clean raw image, and then the denoised features are adapted for the color mapping task by an FDA module. We propose a Lineformer to serve as the FDA, which can well explore the global and local correlations with fewer line buffers (friendly to the line-based imaging process). During inference, the raw supervision branch is removed. In this way, our network combines the advantage of a two-stage enhancement process with the efficiency of single-stage inference. Experiments on four benchmark datasets demonstrate that our method achieves state-of-the-art performance with fewer computing costs (60% FLOPs of the two-stage method DNF). Our codes will be released after the acceptance of this work.


title: Efficient and Effective Similarity Search over Bipartite Graphs

abstract: Similarity search over a bipartite graph aims to retrieve from the graph the nodes that are similar to each other, which finds applications in various fields such as online advertising, recommender systems etc. Existing similarity measures either (i) overlook the unique properties of bipartite graphs, or (ii) fail to capture high-order information between nodes accurately, leading to suboptimal result quality. Recently, Hidden Personalized PageRank (HPP) is applied to this problem and found to be more effective compared with prior similarity measures. However, existing solutions for HPP computation incur significant computational costs, rendering it inefficient especially on large graphs. In this paper, we first identify an inherent drawback of HPP and overcome it by proposing bidirectional HPP (BHPP). Then, we formulate similarity search over bipartite graphs as the problem of approximate BHPP computation, and present an efficient solution Approx-BHPP. Specifically, Approx-BHPP offers rigorous theoretical accuracy guarantees with optimal computational complexity by combining deterministic graph traversal with matrix operations in an optimized and non-trivial way. Moreover, our solution achieves significant gain in practical efficiency due to several carefully-designed optimizations. Extensive experiments, comparing BHPP against 8 existing similarity measures over 7 real bipartite graphs, demonstrate the effectiveness of BHPP on query rewriting and item recommendation. Moreover, Approx-BHPP outperforms baseline solutions often by up to orders of magnitude in terms of computational time on both small and large datasets.


title: EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusion

abstract: Generating multiview images from a single view facilitates the rapid generation of a 3D mesh conditioned on a single image. Recent methods that introduce 3D global representation into diffusion models have shown the potential to generate consistent multiviews, but they have reduced generation speed and face challenges in maintaining generalizability and quality. To address this issue, we propose EpiDiff, a localized interactive multiview diffusion model. At the core of the proposed approach is to insert a lightweight epipolar attention block into the frozen diffusion model, leveraging epipolar constraints to enable cross-view interaction among feature maps of neighboring views. The newly initialized 3D modeling module preserves the original feature distribution of the diffusion model, exhibiting compatibility with a variety of base diffusion models. Experiments show that EpiDiff generates 16 multiview images in just 12 seconds, and it surpasses previous methods in quality evaluation metrics, including PSNR, SSIM and LPIPS. Additionally, EpiDiff can generate a more diverse distribution of views, improving the reconstruction quality from generated multiviews. Please see our project page at https://huanngzh.github.io/EpiDiff/.


title: Compress & Align: Curating Image-Text Data with Human Knowledge

abstract: The massive growth of image-text data through web crawling inherently presents the challenge of variability in data quality. This paper introduces a novel algorithm, rooted in human knowledge, to compress this vast corpus of web-crawled image-text datasets to a compact and high-quality form. Our method unfolds in three major steps. First, we collect an image-text dataset, wherein each image is associated with multiple captions sourced from diverse origins. Then, to systemically capture human preferences regarding the best caption paired with each image, we establish a comprehensive set of both subjective and objective criteria for critically guiding the alignment assessment from labelers. Lastly, we train a reward model on the annotated dataset to internalize the nuanced human understanding of image-text alignment. The resulting reward model thus can act as a human-like referee to filter misaligned/low-quality image-text pairs. Extensive experiments demonstrate that we are able to secure (or even improve) model performance by compressing the image-text datasets up to ~90%. An impressive example is that, by aggressively reducing the total training sample from 130M to 15.5M (e.g., ~9x smaller), our BLIP-B/16 models still consistently show superior performance compared with the full-size-dataset counterpart on image-text retrieval (Flickr30K, COCO) by ~2.5% in Recall@1, and on image-captioning (Nocaps, COCO) by ~10.0% in CIDEr and ~2.7% in SPICE.


title: A method for recovery of multidimensional time series based on the detection of behavioral patterns and the use of autoencoders

abstract: This article presents a method for recovering missing values in multidimensional time series. The method combines neural network technologies and an algorithm for searching snippets (behavioral patterns of a time series). It includes the stages of data preprocessing, recognition and reconstruction, using convolutional and recurrent neural networks. Experiments have shown high accuracy of recovery and the advantage of the method over SOTA methods.


title: A Multimodal Dataset and Benchmark for Radio Galaxy and Infrared Host Detection

abstract: We present a novel multimodal dataset developed by expert astronomers to automate the detection and localisation of multi-component extended radio galaxies and their corresponding infrared hosts. The dataset comprises 4,155 instances of galaxies in 2,800 images with both radio and infrared modalities. Each instance contains information on the extended radio galaxy class, its corresponding bounding box that encompasses all of its components, pixel-level segmentation mask, and the position of its corresponding infrared host galaxy. Our dataset is the first publicly accessible dataset that includes images from a highly sensitive radio telescope, infrared satellite, and instance-level annotations for their identification. We benchmark several object detection algorithms on the dataset and propose a novel multimodal approach to identify radio galaxies and the positions of infrared hosts simultaneously.

categories: cs.CV astro-ph.CO astro-ph.GA astro-ph.IM


title: RGNet: A Unified Clip Retrieval and Grounding Network for Long Videos

abstract: Locating specific moments within long videos (20-120 minutes) presents a significant challenge, akin to finding a needle in a haystack. Adapting existing short video (5-30 seconds) grounding methods to this problem yields poor performance. Since most real life videos, such as those on YouTube and AR/VR, are lengthy, addressing this issue is crucial. Existing methods typically operate in two stages: clip retrieval and grounding. However, this disjoint process limits the retrieval module's fine-grained event understanding, crucial for specific moment detection. We propose RGNet which deeply integrates clip retrieval and grounding into a single network capable of processing long videos into multiple granular levels, e.g., clips and frames. Its core component is a novel transformer encoder, RG-Encoder, that unifies the two stages through shared features and mutual optimization. The encoder incorporates a sparse attention mechanism and an attention loss to model both granularity jointly. Moreover, we introduce a contrastive clip sampling technique to mimic the long video paradigm closely during training. RGNet surpasses prior methods, showcasing state-of-the-art performance on long video temporal grounding (LVTG) datasets MAD and Ego4D.


title: Learning of the students in a reproduction of figure by folding

abstract: We are interested in the learning of 6 to 7 years old children in implementations of a situation of reproduction of figure by folding presented in the first part of this article. In the second part we expose our problem as well as our hypothesis and our methodology. The third part presents the mathematical and didactical potentialities of the PLIOX situation within the framework of the Theory of Didactic Situations of Brousseau, in support of Berthelot and Salin's works, and according to a cognitive and semiotic point of view of Duval. According to this previous part, we clarify and analyze the results obtained from implementations in two classrooms (part four).


title: Genixer: Empowering Multimodal Large Language Models as a Powerful Data Generator

abstract: Multimodal Large Language Models (MLLMs) demonstrate exceptional problem-solving capabilities, but there is limited research focusing on their ability to generate data by converting unlabeled images into visual instruction tuning data. To this end, this paper is the first to explore the potential of empowering MLLM to generate data rather than prompting GPT-4. We introduce Genixer, a holistic data generation pipeline consisting of four key steps: (i) instruction data collection, (ii) instruction template design, (iii) empowering MLLMs, and (iv) data generation and filtering. Additionally, we outline two modes of data generation: task-agnostic and task-specific, enabling controllable output. We demonstrate that a synthetic VQA-like dataset trained with LLaVA1.5 enhances performance on 10 out of 12 multimodal benchmarks. Additionally, the grounding MLLM Shikra, when trained with a REC-like synthetic dataset, shows improvements on 7 out of 8 REC datasets. Through experiments and synthetic data analysis, our findings are: (1) current MLLMs can serve as robust data generators without assistance from GPT-4V; (2) MLLMs trained with task-specific datasets can surpass GPT-4V in generating complex instruction tuning data; (3) synthetic datasets enhance performance across various multimodal benchmarks and help mitigate model hallucinations. The data, code, and models can be found at https://github.com/zhaohengyuan1/Genixer.


title: Features of synthesis of TbCr$_3$(BO$_3$)$_4$ single crystals by solution-melt method

abstract: The phase formation of terbium chromium borate in melt solutions based on bismuth trimolybdate and lithium tungstate was studied. It was shown that there is no trigonal phase of terbium chromium borate in a system based on bismuth trimolybdate at all component ratios. The ratio of components of a system based on lithium tungstate has been found, at which trigonal TbCr$_3$(BO$_3$)$_4$ crystals are formed at temperatures above 1100$^\circ$C, and below this temperature only a monoclinic phase is formed. The X-ray properties of the grown crystals were studied.


title: TULIP: Transformer for Upsampling of LiDAR Point Clouds

abstract: LiDAR Upsampling is a challenging task for the perception systems of robots and autonomous vehicles, due to the sparse and irregular structure of large-scale scene contexts. Recent works propose to solve this problem by converting LiDAR data from 3D Euclidean space into an image super-resolution problem in 2D image space. Although their methods can generate high-resolution range images with fine-grained details, the resulting 3D point clouds often blur out details and predict invalid points. In this paper, we propose TULIP, a new method to reconstruct high-resolution LiDAR point clouds from low-resolution LiDAR input. We also follow a range image-based approach but specifically modify the patch and window geometries of a Swin-Transformer-based network to better fit the characteristics of range images. We conducted several experiments on three public real-world and simulated datasets. TULIP outperforms state-of-the-art methods in all relevant metrics and generates robust and more realistic point clouds than prior works.


title: DiffCast: A Unified Framework via Residual Diffusion for Precipitation Nowcasting

abstract: Precipitation nowcasting is an important spatio-temporal prediction task to predict the radar echoes sequences based on current observations, which can serve both meteorological science and smart city applications. Due to the chaotic evolution nature of the precipitation systems, it is a very challenging problem. Previous studies address the problem either from the perspectives of deterministic modeling or probabilistic modeling. However, their predictions suffer from the blurry, high-value echoes fading away and position inaccurate issues. The root reason of these issues is that the chaotic evolutionary precipitation systems are not appropriately modeled. Inspired by the nature of the systems, we propose to decompose and model them from the perspective of global deterministic motion and local stochastic variations with residual mechanism. A unified and flexible framework that can equip any type of spatio-temporal models is proposed based on residual diffusion, which effectively tackles the shortcomings of previous methods. Extensive experimental results on four publicly available radar datasets demonstrate the effectiveness and superiority of the proposed framework, compared to state-of-the-art techniques. Our code is publicly available at https://github.com/DeminYu98/DiffCast.


title: On the feasibility and usefulness of applying the `Schr\"odinger c.q. Liouville-von Neumann equation' to quantum measurement

abstract: The present paper is a sequel to papers dealing with recent developments on the issue of `quantum measurement'. In this paper `measurement within the domain of application of quantum mechanics' is treated as a \emph{quantum mechanical} \emph{interaction} of a `(sub)microscopic object $(o)$' and an `equally (sub)microscopic part of the measuring instrument $(a)$ being sensitive to the (sub)microscopic information', that interaction to be described by a Schr\"odinger equation. The Stern-Gerlach experiment is used as a paradigmatic example. An alternative to the Heisenberg inequality is found, exhibiting the \emph{independent} contributions of `preparation of the initial state of object $(o)$' \emph{and} `interaction of object $(o)$ \emph{and} measuring instrument/probe $(a)$'. Applicability of the Liouville-von Neumann equation is stressed.


title: SqueezeSAM: User friendly mobile interactive segmentation

abstract: The Segment Anything Model (SAM) has been a cornerstone in the field of interactive segmentation, propelling significant progress in generative AI, computational photography, and medical imaging. Despite its ability to process arbitrary user input and generate corresponding segmentation masks, SAM's 600 million parameter architecture, based on ViT-H, is not compatible with current mobile hardware due to its high computational demands and large model size. Our research aims to adapt SAM for use in mobile photography applications. To this end, we have developed a fully convolutional SqueezeSAM model architecture, which is 62.5 times faster and 31.6 times smaller than the original SAM, making it a viable solution for mobile applications. Furthermore, our tiny model achieves an mIOU within 1% of the original VIT-H architecture. Automated segmentation holds significant value in the creation flow for photography applications, as evidenced by its adoption by leading industry players like apple and capcut. To facilitate this automation, we employ salient object detection and simulate potential user clicks for foreground object selection, generating an initial segmentation mask that users can subsequently edit interactively. A common user expectation is that a click on a specific part of an object will result in the segmentation of the entire object. For example, a click on a person's t-shirt in a photo should ideally segment the entire person, not just the t-shirt. However, SAM typically only segments the clicked area. We address this limitation through a novel data augmentation scheme. Consequently, if a user clicks on a person holding a basketball, both the person and the basketball are segmented together, aligning with user expectations and enhancing the overall user experience.


title: Geant4 Silver Anniversary: 25 years enabling scientific production

abstract: This paper summarizes Geant4 contribution to scientific research over the past 25 years through a scientometric analysis of the results with which it has been associated. The scientometric data collected from scholarly literature and databases are evaluated with methods pertaining to econometrics and ecology to quantify relevant traits, diversity and disparity in their scientific and geographic distributions, and to identify statistically significant trends. The analysis reviews the contribution of Geant4 to the field - experimental particle physics - that originally motivated its development and highlights its role in other research domains including nuclear physics and engineering, astrophysics and space science, biomedical physics, archaeology and the cultural heritage.


title: InstructAny2Pix: Flexible Visual Editing via Multimodal Instruction Following

abstract: The ability to provide fine-grained control for generating and editing visual imagery has profound implications for computer vision and its applications. Previous works have explored extending controllability in two directions: instruction tuning with text-based prompts and multi-modal conditioning. However, these works make one or more unnatural assumptions on the number and/or type of modality inputs used to express controllability. We propose InstructAny2Pix, a flexible multi-modal instruction-following system that enables users to edit an input image using instructions involving audio, images, and text. InstructAny2Pix consists of three building blocks that facilitate this capability: a multi-modal encoder that encodes different modalities such as images and audio into a unified latent space, a diffusion model that learns to decode representations in this latent space into images, and a multi-modal LLM that can understand instructions involving multiple images and audio pieces and generate a conditional embedding of the desired output, which can be used by the diffusion decoder. Additionally, to facilitate training efficiency and improve generation quality, we include an additional refinement prior module that enhances the visual quality of LLM outputs. These designs are critical to the performance of our system. We demonstrate that our system can perform a series of novel instruction-guided editing tasks. The code is available at https://github.com/jacklishufan/InstructAny2Pix.git


title: SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models

abstract: Current instruction-based editing methods, such as InstructPix2Pix, often fail to produce satisfactory results in complex scenarios due to their dependence on the simple CLIP text encoder in diffusion models. To rectify this, this paper introduces SmartEdit, a novel approach to instruction-based image editing that leverages Multimodal Large Language Models (MLLMs) to enhance their understanding and reasoning capabilities. However, direct integration of these elements still faces challenges in situations requiring complex reasoning. To mitigate this, we propose a Bidirectional Interaction Module that enables comprehensive bidirectional information interactions between the input image and the MLLM output. During training, we initially incorporate perception data to boost the perception and understanding capabilities of diffusion models. Subsequently, we demonstrate that a small amount of complex instruction editing data can effectively stimulate SmartEdit's editing capabilities for more complex instructions. We further construct a new evaluation dataset, Reason-Edit, specifically tailored for complex instruction-based image editing. Both quantitative and qualitative results on this evaluation dataset indicate that our SmartEdit surpasses previous methods, paving the way for the practical application of complex instruction-based image editing.


title: MonoNPHM: Dynamic Head Reconstruction from Monocular Videos

abstract: We present Monocular Neural Parametric Head Models (MonoNPHM) for dynamic 3D head reconstructions from monocular RGB videos. To this end, we propose a latent appearance space that parameterizes a texture field on top of a neural parametric model. We constrain predicted color values to be correlated with the underlying geometry such that gradients from RGB effectively influence latent geometry codes during inverse rendering. To increase the representational capacity of our expression space, we augment our backward deformation field with hyper-dimensions, thus improving color and geometry representation in topologically challenging expressions. Using MonoNPHM as a learned prior, we approach the task of 3D head reconstruction using signed distance field based volumetric rendering. By numerically inverting our backward deformation field, we incorporated a landmark loss using facial anchor points that are closely tied to our canonical geometry representation. To evaluate the task of dynamic face reconstruction from monocular RGB videos we record 20 challenging Kinect sequences under casual conditions. MonoNPHM outperforms all baselines with a significant margin, and makes an important step towards easily accessible neural parametric face models through RGB tracking.


title: Gaussian Splatting SLAM

abstract: We present the first application of 3D Gaussian Splatting in monocular SLAM, the most fundamental but the hardest setup for Visual SLAM. Our method, which runs live at 3fps, utilises Gaussians as the only 3D representation, unifying the required representation for accurate, efficient tracking, mapping, and high-quality rendering. Designed for challenging monocular settings, our approach is seamlessly extendable to RGB-D SLAM when an external depth sensor is available. Several innovations are required to continuously reconstruct 3D scenes with high fidelity from a live camera. First, to move beyond the original 3DGS algorithm, which requires accurate poses from an offline Structure from Motion (SfM) system, we formulate camera tracking for 3DGS using direct optimisation against the 3D Gaussians, and show that this enables fast and robust tracking with a wide basin of convergence. Second, by utilising the explicit nature of the Gaussians, we introduce geometric verification and regularisation to handle the ambiguities occurring in incremental 3D dense reconstruction. Finally, we introduce a full SLAM system which not only achieves state-of-the-art results in novel view synthesis and trajectory estimation but also reconstruction of tiny and even transparent objects.


title: Honeybee: Locality-enhanced Projector for Multimodal LLM

abstract: In Multimodal Large Language Models (MLLMs), a visual projector plays a crucial role in bridging pre-trained vision encoders with LLMs, enabling profound visual understanding while harnessing the LLMs' robust capabilities. Despite the importance of the visual projector, it has been relatively less explored. In this study, we first identify two essential projector properties: (i) flexibility in managing the number of visual tokens, crucial for MLLMs' overall efficiency, and (ii) preservation of local context from visual features, vital for spatial understanding. Based on these findings, we propose a novel projector design that is both flexible and locality-enhanced, effectively satisfying the two desirable properties. Additionally, we present comprehensive strategies to effectively utilize multiple and multifaceted instruction datasets. Through extensive experiments, we examine the impact of individual design choices. Finally, our proposed MLLM, Honeybee, remarkably outperforms previous state-of-the-art methods across various benchmarks, including MME, MMBench, SEED-Bench, and LLaVA-Bench, achieving significantly higher efficiency. Code and models are available at https://github.com/kakaobrain/honeybee.


title: Fermionic Shift Symmetries in (Anti) de Sitter Space

abstract: We study extended shift symmetries that arise for fermionic fields on anti-de Sitter (AdS) space and de Sitter (dS) space for particular values of the mass relative to the curvature scale. We classify these symmetries for general mixed-symmetry fermionic fields in arbitrary dimension and describe how fields with these symmetries arise as the decoupled longitudinal modes of massive fermions as they approach partially massless points. For the particular case of AdS$_4$, we look for non-trivial Lie superalgebras that can underly interacting theories that involve these fields. We study from this perspective the minimal such theory, the Akulov--Volkov theory on AdS$_4$, which is a non-linear theory of a spin-$1/2$ Goldstino field that describes the spontaneous breaking of ${\cal N}=1$ supersymmetry on AdS$_4$ down to the isometries of AdS$_4$. We show how to write the nonlinear supersymmetry transformation for this theory using the fermionic ambient space formalism. We also study the Lie superalgebras of candidate multi-field examples and rule out the existence of a supersymmetric special galileon on AdS$_4$.


title: One-loop partition function of gravity with leaky boundary conditions

abstract: Leaky boundary conditions in asymptotically AdS spacetimes are relevant to discuss black hole evaporation and the evolution of the Page curve via the island formula. We explore the consequences of leaky boundary conditions on the one-loop partition function of gravity. We focus on JT gravity minimally coupled to a scalar field whose normalizable and non-normalizable modes are both turned on, allowing for leakiness through the AdS boundary. Classically, this yields a flux-balance law relating the scalar news to the time derivative of the mass. Semi-classically, we argue that the usual diffeomorphism-invariant measure is ill-defined, suggesting that the area-non-preserving diffeomorphisms are broken at one loop. We calculate the associated anomaly and its implication on the gravitational Gauss law. Finally, we generalize our arguments to higher dimensions and dS.


title: Broadband maps of eROSITA and their comparison with the ROSAT survey

abstract: By June of 2020, the extended ROentgen Survey with an Imaging Telescope Array (eROSITA) on board the Spectrum Roentgen Gamma observatory had completed its first of the planned eight X-ray all-sky survey (eRASS1). The large effective area of the X-ray telescope makes it ideal for a survey of the faint X-ray diffuse emission over half of the sky with an unprecedented energy resolution and position accuracy. In this work, we produce the X-ray diffuse emission maps of the eRASS1 data with a current calibration, covering the energy range from 0.2 to 8.0 keV. We validated these maps by comparison with X-ray background maps derived from the ROSAT All Sky Survey (RASS). We generated X-ray images with a pixel area of 9 arcmin$^2$ using the observations available to the German eROSITA consortium. The contribution of the particle background to the photons was subtracted from the final maps. We also subtracted all the point sources above a flux threshold dependent on the goal of the subtraction, exploiting the eRASS1 catalog that will soon be available. The accuracy of the eRASS1 maps is shown by a flux match to the RASS X-ray maps, obtained by converting the eROSITA rates into equivalent ROSAT count rates in the standard ROSAT energy bands R4, R5, R6, and R7, within 1.25$\sigma$. We find small residual deviations in the R4, R5, and R6 bands, where eROSITA tends to observe lower flux than ROSAT (~11%), while a better agreement is achieved in the R7 band (~1%). The eRASS maps exhibit lower noise levels than RASS maps at the same resolution above 0.3 keV. We report the average surface brightness and total flux of different large sky regions as a reference. The detection of faint emission from diffuse hot gas in the Milky Way is corroborated by the consistency of the eRASS1 and RASS maps shown in this paper and by their comparable flux dynamic range.


title: Constraints on photon mass and dark photon from the Jovian magnetic field

abstract: The Jovian magnetic field, being the strongest and largest planetary one in the solar system, could offer us new insights into possible microscopic scale new physics, such as a non-zero mass of the Standard Model (SM) photon or a light dark photon kinetically mixing with the SM photon. We employ the immense data set from the latest Juno mission, which provides us unprecedented information about the magnetic field of the gas giant, together with a more rigorous statistical approach compared to the literature, to set strong constraints on the dark photon mass and kinetic mixing parameter, as well as the SM photon mass. The constraint on the dark photon parameters is independent of whether dark photon is (part of) dark matter or not, and serves as the most stringent one in a certain regime of the parameter space.


title: Possible Contamination of the Intergalactic Medium Damping Wing in ULAS J1342+0928 by Proximate Damped Ly$\alpha$ Absorption

abstract: The red damping wing from neutral hydrogen in the intergalactic medium is a smoking-gun signal of ongoing reionization. One potential contaminant of the intergalactic damping wing signal is dense gas associated with foreground galaxies, which can give rise to proximate damped Ly$\alpha$ absorbers. The Ly$\alpha$ imprint of such absorbers on background quasars is indistinguishable from the intergalactic medium within the uncertainty of the intrinsic quasar continuum, and their abundance at $z\gtrsim7$ is unknown. Here we show that the complex of low-ionization metal absorption systems recently discovered by deep JWST/NIRSpec observations in the foreground of the $z=7.54$ quasar ULAS~J1342$+$0928 can potentially reproduce the quasar's spectral profile close to rest-frame Ly$\alpha$ without invoking a substantial contribution from the intergalactic medium, but only if the absorbing gas is extremely metal-poor ($[{\rm O}/{\rm H}]\sim-3.5$). Such a low oxygen abundance has never been observed in a damped Ly$\alpha$ absorber at any redshift, but this possibility still complicates the interpretation of the spectrum. Our analysis highlights the need for deep spectroscopy of high-redshift quasars with JWST or ELT to "purify" damping wing quasar samples, an exercise which is impossible for much fainter objects like galaxies.


title: On Amplitudes and Field Redefinitions

abstract: We derive an off-shell recursion relation for correlators that holds at all loop orders. This allows us to prove how generalized amplitudes transform under generic field redefinitions, starting from an assumed behavior of the one-particle-irreducible effective action. The form of the recursion relation resembles the operation of raising the rank of a tensor by acting with a covariant derivative. This inspires a geometric interpretation, whose features and flaws we investigate.


title: Electroweak Phase Transition with a Double Well Done Doubly Well

abstract: We revisit the electroweak phase transition in the scalar singlet extension of the standard model with a $\mathbb{Z}_2$ symmetry. In significant parts of the parameter space the phase transition occurs in two steps - including canonical benchmarks used in experimental projections for gravitational waves. Domain walls produced in the first step of the transition seed the final step to the electroweak vacuum, an effect which is typically neglected but leads to an exponentially enhanced tunnelling rate. We improve previous results obtained for the seeded transition, which made use of the thin-wall or high temperature approximations, by using the mountain pass algorithm that was recently proposed as a useful tool for seeded processes. We then determine the predictions of the seeded transition for the latent heat, bubble size and characteristic time scale of the transition. Differences compared to homogeneous transitions are most pronounced when there are relatively few domain walls per hubble patch, potentially leading to an enhanced gravitational wave signal. We also provide a derivation of the percolation criteria for a generic seeded transition, which applies to the domain wall seeds we consider as well as to strings and monopoles.


title: Holographic Screen Sequestration

abstract: Holographic screens are codimension-one hypersurfaces that extend the notion of apparent horizons to general (non-black hole) spacetimes and that display interesting thermodynamic properties. We show that if a spacetime contains a codimension-two, boundary-homologous, minimal extremal spacelike surface $X$ (known as an HRT surface in AdS/CFT), then any holographic screens are sequestered to the causal wedges of $X$. That is, any single connected component of a holographic screen can be located in at most one of the causal future, causal past, inner wedge, or outer wedge of $X$. We comment on how this result informs possible coarse grained entropic interpretations of generic holographic screens, as well as on connections to semiclassical objects such as quantum extremal surfaces.


title: Localization of overdamped bosonic modes and transport in strange metals

abstract: A recent theory described strange metal behavior in a model of a Fermi surface coupled a two-dimensional quantum critical bosonic field with a spatially random Yukawa coupling. With the assumption of self-averaging randomness, similar to that in the Sachdev-Ye-Kitaev model, numerous observed properties of a strange metal were obtained for wide range of intermediate temperatures, including the linear-in-temperature resistivity. The Harris criterion implies that spatial fluctuations in the local position of the critical point must dominate at lower temperatures. For an $M$-component boson with $M \geq 2$, we use multiple graphics processing units (GPUs) to compute the real frequency spectrum of the boson propagator in a self-consistent mean-field treatment of the boson self-interactions, but an exact treatment of multiple realizations of the spatial randomness from the random boson mass. We find that Landau damping from the fermions leads to the emergence of the physics of the random transverse-field Ising model at low temperatures, as has been proposed by Hoyos, Kotabage, and Vojta. This regime is controlled by localized overdamped eigenmodes of the bosonic scalar field, also has a resistivity which is nearly linear-in-temperature, and extends into a `quantum critical phase' away from the quantum critical point, as observed in several cuprates. For the $M = 1$ Ising scalar, the mean-field treatment is not applicable, and so we use Hybrid Monte Carlo simulations running on multiple GPUs; we find a rounded transition and localization physics, with strange metal behavior in an extended region around the transition.


title: Symmetric derivatives of parametrized quantum circuits

abstract: Symmetries are crucial for tailoring parametrized quantum circuits to applications, due to their capability to capture the essence of physical systems. In this work, we shift the focus away from incorporating symmetries in the circuit design and towards symmetry-aware training of variational quantum algorithms. For this, we introduce the concept of projected derivatives of parametrized quantum circuits, in particular the equivariant and covariant derivatives. We show that the covariant derivative gives rise to the quantum Fisher information and quantum natural gradient. This provides an operational meaning for the covariant derivative, and allows us to extend the quantum natural gradient to all continuous symmetry groups. Connecting to traditional particle physics, we confirm that our covariant derivative is the same as the one introduced in physical gauge theory. This work provides tools for tailoring variational quantum algorithms to symmetries by incorporating them locally in derivatives, rather than into the design of the circuit.


title: Reflections in the Mirror: Studying Infinite Coxeter Groups of GV-Invariants

abstract: We study the problem of computing Gopakumar-Vafa invariants for multiparameter families of symmetric Calabi-Yau threefolds admitting flops to diffeomorphic manifolds. There are infinite Coxeter groups, generated by permutations and flops, that act as symmetries on the GV-invariants of these manifolds. We describe how these groups are related to symmetries in GLSMs and the existence of multiple mirrors. Some representation theory of these Coxeter groups is also discussed. The symmetries provide an infinite number of relations between the GV-invariants for each fixed genus. This remarkable fact is of assistance in obtaining higher-genus invariants via the BCOV recursion. This proceedings article is based on joint work with Philip Candelas and Xenia de la Ossa.


title: Survey of supergravities

abstract: A large class of supergravities in diverse dimensions are surveyed. This includes maximal supergravities, their general gaugings in the framework of embedding tensor formalism, supergravities with less than maximal supersymmetry, their matter couplings and general gaugings. The emphasis is on summarizing their most general form to date, and primarily in their component formulation. A class of exceptional field theories in an extended geometric framework are summarized briefly. For most of the supergravities surveyed, the bosonic part of the Lagrangians and supertransformations up to leading terms in fermions are given.


title: On the effects of photoionization feedback on second-generation star formation in globular clusters of different masses

abstract: We simulate the formation of second-generation stars in young clusters with masses of $10^{5}$ and $10^6\ {\rm M_\odot}$ within $30-100\ {\rm Myr}$ after the formation of clusters. We assume the clusters move through a uniform interstellar medium with gas densities of $10^{-24}$ and $10^{-23}\ {\rm gcm}^{-3} $ and consider the stellar winds from asymptotic giant branch (AGB) stars, gas accretion onto the cluster, ram pressure, star formation, and photoionization feedback of our stellar systems including binary stars. We find that second-generation (SG) stars can be formed only within the $10^6\ {\rm M_\odot}$ cluster in the high-density simulation, where the cluster can accrete sufficient pristine gas from their surrounding medium, leading to efficient cooling required for the ignition of SG formation and sufficient dilution of the AGB ejecta. Hence, our results indicate that a denser environment is another requirement for the AGB scenario to explain the presence of multiple populations in globular clusters. On the other hand, the ionizing feedback becomes effective in heating the gas in our low-density simulations. As a result, the clusters cannot accumulate a considerable amount of pristine gas at their center. The gas mass within the clusters in these simulations is similar to that in young massive clusters (YMCs). Hence, our studies can provide a possible reason for the lack of gas, star formation, and SG stars in YMCs. Our results indicate that the ionizing stellar feedback is not a severe problem for SG formation; rather, it can help the AGB scenario to account for some observables.


title: Where are NANOGrav's big black holes?

abstract: Multiple pulsar timing array (PTA) collaborations have recently reported the first detection of gravitational waves (GWs) of nanohertz frequencies. The signal is expected to be primarily sourced by inspiralling supermassive black hole binaries (SMBHBs) and these first results are broadly consistent with the expected GW spectrum from such a population. Curiously, the measured amplitude of the GW background in all announced results is a bit larger than theoretical predictions. In this work, we show that the amplitude of the stochastic gravitational wave background (SGWB) predicted from the present-day abundance of SMBHs derived from local scaling relations is significantly smaller than that measured by the PTAs. We demonstrate that this difference cannot be accounted for through changes in the merger history of SMBHs and that there is an upper limit to the boost to the characteristic strain from multiple merger events, due to the fact that they involve black holes of decreasing masses. If we require the current estimate of the black hole mass density -- equal to the integrated quasar luminosity function through the classic Soltan argument -- to be preserved, then the currently measured PTA result would imply that the typical total mass of SMBHs contributing to the background should be at least $\sim 3 \times 10^{10} M_\odot$, a factor of $\sim 10$ larger than previously predicted. The required space density of such massive black holes corresponds to order $10$ $3 \times 10^{10} M_\odot$ SMBHs within the volume accessible by stellar and gas dynamical SMBH measurements. By virtue of the GW signal being dominated by the massive end of the SMBH distribution, PTA measurements offer a unique window into such rare objects and complement existing electromagnetic observations.