title: Stable Code Technical Report

abstract: We introduce Stable Code, the first in our new-generation of code language models series, which serves as a general-purpose base code language model targeting code completion, reasoning, math, and other software engineering-based tasks. Additionally, we introduce an instruction variant named Stable Code Instruct that allows conversing with the model in a natural chat interface for performing question-answering and instruction-based tasks. In this technical report, we detail the data and training procedure leading to both models. Their weights are available via Hugging Face for anyone to download and use at https://huggingface.co/stabilityai/stable-code-3b and https://huggingface.co/stabilityai/stable-code-instruct-3b. This report contains thorough evaluations of the models, including multilingual programming benchmarks, and the MT benchmark focusing on multi-turn dialogues. At the time of its release, Stable Code is the state-of-the-art open model under 3B parameters and even performs comparably to larger models of sizes 7 billion and 15 billion parameters on the popular Multi-PL benchmark. Stable Code Instruct also exhibits state-of-the-art performance on the MT-Bench coding tasks and on Multi-PL completion compared to other instruction tuned models. Given its appealing small size, we also provide throughput measurements on a number of edge devices. In addition, we open source several quantized checkpoints and provide their performance metrics compared to the original model.


title: Method of similar operators in harmonious Banach spaces

abstract: We consider similarity transformations of a perturbed linear operator $A-B$ in a complex Banach space $\mathcal{X}$, where the unperturbed operator $A$ is a generator of a Banach $L_1(\mathbb{R})$-module and the perturbation operator $B$ is a bounded linear operator. The result of the transformation is a simpler operator $A-B_0$. For example, if $A$ is a differentiation operator and $B$ is an operator of multiplication by an operator-valued function, then $B_0$ is an operator of multiplication by a function that is a restriction of an entire function of exponential type and could be $0$ in some cases. As a consequence, we derive the spectral invariance of the operator $A-B$ in a large class of spaces. The study is based on a widely applicable modification of the method of similar operators that is also presented in the paper. This non-traditional modification is rooted in the spectral theory of Banach $L_1(\mathbb{R})$-modules.


title: Adaptive hybrid high-order method for guaranteed lower eigenvalue bounds

abstract: The higher-order guaranteed lower eigenvalue bounds of the Laplacian in the recent work by Carstensen, Ern, and Puttkammer [Numer. Math. 149, 2021] require a parameter $C_{\mathrm{st},1}$ that is found $\textit{not}$ robust as the polynomial degree $p$ increases. This is related to the $H^1$ stability bound of the $L^2$ projection onto polynomials of degree at most $p$ and its growth $C_{\rm st, 1}\propto (p+1)^{1/2}$ as $p \to \infty$. A similar estimate for the Galerkin projection holds with a $p$-robust constant $C_{\mathrm{st},2}$ and $C_{\mathrm{st},2} \le 2$ for right-isosceles triangles. This paper utilizes the new inequality with the constant $C_{\mathrm{st},2}$ to design a modified hybrid high-order (HHO) eigensolver that directly computes guaranteed lower eigenvalue bounds under the idealized hypothesis of exact solve of the generalized algebraic eigenvalue problem and a mild explicit condition on the maximal mesh-size in the simplicial mesh. A key advance is a $p$-robust parameter selection. The analysis of the new method with a different fine-tuned volume stabilization allows for a priori quasi-best approximation and improved $L^2$ error estimates as well as a stabilization-free reliable and efficient a posteriori error control. The associated adaptive mesh-refining algorithm performs superior in computer benchmarks with striking numerical evidence for optimal higher empirical convergence rates.


title: Age of Information in a Single-Source Generate-at-Will Dual-Server Status Update System

abstract: We study age of information (AoI) in a single-source dual-server status update system for the generate at will (GAW) scenario, consisting of an information source, dual servers, and a monitor. For this system, the method of stochastic hybrid systems (SHS) was used to obtain the mean AoI for the work-conserving ZW (zero wait) policy with out-of-order packet discarding at the monitor. In this paper, we propose a non-work-conserving F/P (freeze/preempt) policy for which the sampling and transmission process is frozen for an Erlang distributed amount of time upon each transmission, and out-of-order packets are preempted immediately at the source, rather than being discarded at the monitor upon reception. We use the absorbing Markov chain (AMC) method to obtain the exact distributions of AoI and also the peak AoI (PAoI) processes, for both ZW and F/P policies. Numerical results are presented for the validation of the proposed analytical model and a comparative evaluation of ZW and F/P policies.


title: LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models

abstract: This paper presents a comprehensive survey of the current status and opportunities for Large Language Models (LLMs) in strategic reasoning, a sophisticated form of reasoning that necessitates understanding and predicting adversary actions in multi-agent settings while adjusting strategies accordingly. Strategic reasoning is distinguished by its focus on the dynamic and uncertain nature of interactions among multi-agents, where comprehending the environment and anticipating the behavior of others is crucial. We explore the scopes, applications, methodologies, and evaluation metrics related to strategic reasoning with LLMs, highlighting the burgeoning development in this area and the interdisciplinary approaches enhancing their decision-making performance. It aims to systematize and clarify the scattered literature on this subject, providing a systematic review that underscores the importance of strategic reasoning as a critical cognitive capability and offers insights into future research directions and potential improvements.


title: Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models

abstract: It is commonplace to produce application-specific models by fine-tuning large pre-trained models using a small bespoke dataset. The widespread availability of foundation model checkpoints on the web poses considerable risks, including the vulnerability to backdoor attacks. In this paper, we unveil a new vulnerability: the privacy backdoor attack. This black-box privacy attack aims to amplify the privacy leakage that arises when fine-tuning a model: when a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model. We conduct extensive experiments on various datasets and models, including both vision-language models (CLIP) and large language models, demonstrating the broad applicability and effectiveness of such an attack. Additionally, we carry out multiple ablation studies with different fine-tuning methods and inference strategies to thoroughly analyze this new threat. Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.


title: Open-Vocabulary Federated Learning with Multimodal Prototyping

abstract: Existing federated learning (FL) studies usually assume the training label space and test label space are identical. However, in real-world applications, this assumption is too ideal to be true. A new user could come up with queries that involve data from unseen classes, and such open-vocabulary queries would directly defect such FL systems. Therefore, in this work, we explicitly focus on the under-explored open-vocabulary challenge in FL. That is, for a new user, the global server shall understand her/his query that involves arbitrary unknown classes. To address this problem, we leverage the pre-trained vision-language models (VLMs). In particular, we present a novel adaptation framework tailored for VLMs in the context of FL, named as Federated Multimodal Prototyping (Fed-MP). Fed-MP adaptively aggregates the local model weights based on light-weight client residuals, and makes predictions based on a novel multimodal prototyping mechanism. Fed-MP exploits the knowledge learned from the seen classes, and robustifies the adapted VLM to unseen categories. Our empirical evaluation on various datasets validates the effectiveness of Fed-MP.


title: Optimal Ridge Regularization for Out-of-Distribution Prediction

abstract: We study the behavior of optimal ridge regularization and optimal ridge risk for out-of-distribution prediction, where the test distribution deviates arbitrarily from the train distribution. We establish general conditions that determine the sign of the optimal regularization level under covariate and regression shifts. These conditions capture the alignment between the covariance and signal structures in the train and test data and reveal stark differences compared to the in-distribution setting. For example, a negative regularization level can be optimal under covariate shift or regression shift, even when the training features are isotropic or the design is underparameterized. Furthermore, we prove that the optimally-tuned risk is monotonic in the data aspect ratio, even in the out-of-distribution setting and when optimizing over negative regularization levels. In general, our results do not make any modeling assumptions for the train or the test distributions, except for moment bounds, and allow for arbitrary shifts and the widest possible range of (negative) regularization levels.


title: GFLean: An Autoformalisation Framework for Lean via GF

abstract: We present an autoformalisation framework for the Lean theorem prover, called GFLean. GFLean uses a high-level grammar writing tool called Grammatical Framework (GF) for parsing and linearisation. GFLean is implemented in Haskell. We explain the functionalities of GFLean, its inner working and discuss its limitations. We also discuss how we can use neural network based translation programs and rule based translation programs together complimenting each other to build robust autoformalisation frameworks.


title: Anomaly Detection and Approximate Similarity Searches of Transients in Real-time Data Streams

abstract: We present LAISS (Lightcurve Anomaly Identification and Similarity Search), an automated pipeline to detect anomalous astrophysical transients in real-time data streams. We deploy our anomaly detection model on the nightly ZTF Alert Stream via the ANTARES broker, identifying a manageable $\sim$1-5 candidates per night for expert vetting and coordinating follow-up observations. Our method leverages statistical light-curve and contextual host-galaxy features within a random forest classifier, tagging transients of rare classes (spectroscopic anomalies), of uncommon host-galaxy environments (contextual anomalies), and of peculiar or interaction-powered phenomena (behavioral anomalies). Moreover, we demonstrate the power of a low-latency ($\sim$ms) approximate similarity search method to find transient analogs with similar light-curve evolution and host-galaxy environments. We use analogs for data-driven discovery, characterization, (re-)classification, and imputation in retrospective and real-time searches. To date we have identified $\sim$50 previously known and previously missed rare transients from real-time and retrospective searches, including but not limited to: SLSNe, TDEs, SNe IIn, SNe IIb, SNe Ia-CSM, SNe Ia-91bg-like, SNe Ib, SNe Ic, SNe Ic-BL, and M31 novae. Lastly, we report the discovery of 325 total transients, all observed between 2018-2021 and absent from public catalogs ($\sim$1% of all ZTF Astronomical Transient reports to the Transient Name Server through 2021). These methods enable a systematic approach to finding the "needle in the haystack" in large-volume data streams. Because of its integration with the ANTARES broker, LAISS is built to detect exciting transients in Rubin data.


title: Cosmic topology. Part IVa. Classification of manifolds using machine learning: a case study with small toroidal universes

abstract: Non-trivial spatial topology of the Universe may give rise to potentially measurable signatures in the cosmic microwave background. We explore different machine learning approaches to classify harmonic-space realizations of the microwave background in the test case of Euclidean $E_1$ topology (the 3-torus) with a cubic fundamental domain of a size scale significantly smaller than the diameter of the last scattering surface. Different machine learning approaches are capable of classifying the harmonic-space realizations with accuracy greater than 99% if the topology scale is half of the diameter of the last-scattering surface and orientation of the topology is known. For distinguishing random rotations of these sky realizations from realizations of the covering space, the extreme gradient boosting classifier algorithm performs best with an accuracy of 88%. Slightly lower accuracies of 83% to 87% are obtained with the random forest classifier along with one- and two-dimensional convolutional neural networks. The techniques presented here can also accurately classify non-rotated cubic $E_1$ topology realizations with a topology scale slightly larger than the diameter of the last-scattering surface, if provided enough training data. This work identifies the prospects and the main challenges for developing machine learning techniques that are capable of accurately classifying observationally viable topologies.


title: FPGA-Accelerated Correspondence-free Point Cloud Registration with PointNet Features

abstract: Point cloud registration serves as a basis for vision and robotic applications including 3D reconstruction and mapping. Despite significant improvements on the quality of results, recent deep learning approaches are computationally expensive and power-hungry, making them difficult to deploy on resource-constrained edge devices. To tackle this problem, in this paper, we propose a fast, accurate, and robust registration for low-cost embedded FPGAs. Based on a parallel and pipelined PointNet feature extractor, we develop custom accelerator cores namely PointLKCore and ReAgentCore, for two different learning-based methods. They are both correspondence-free and computationally efficient as they avoid the costly feature matching step involving nearest-neighbor search. The proposed cores are implemented on the Xilinx ZCU104 board and evaluated using both synthetic and real-world datasets, showing the substantial improvements in the trade-offs between runtime and registration quality. They run 44.08-45.75x faster than ARM Cortex-A53 CPU and offer 1.98-11.13x speedups over Intel Xeon CPU and Nvidia Jetson boards, while consuming less than 1W and achieving 163.11-213.58x energy-efficiency compared to Nvidia GeForce GPU. The proposed cores are more robust to noise and large initial misalignments than the classical methods and quickly find reasonable solutions in less than 15ms, demonstrating the real-time performance.


title: Mass Spectra of Full-Heavy and Double-Heavy Tetraquark States in the Conventional Quark Model

abstract: A comprehensive study of the $S$-wave heavy tetraquark states with identical quarks and antiquarks, specifically $QQ{\bar Q'}\bar Q'$ ($Q, Q'=c,b$), $QQ\bar s\bar s$/$\bar Q\bar Q ss$, and $QQ\bar q\bar q$/$\bar Q\bar Q qq$ ($q=u,d$), are studied in a unified constituent quark model. This model contains the one-gluon exchange and confinement potentials. The latter is modeled as the sum of all two-body linear potentials. We employ the Gaussian expansion method to solve the full four-body Schr\"{o}dinger equations, and search bound and resonant states using the complex-scaling method. We then identify $3$ bound and $62$ resonant states. The bound states are all $QQ\bar q\bar q$ states with the isospin and spin-parity quantum numbers $I(J^P)=0(1^+)$: two bound $bb\bar{q}\bar{q}$ states with the binding energies, 153 MeV and 4 MeV below the $BB^*$ threshold, and a shallow $cc\bar{q}\bar{q}$ state at $-15$ MeV from the $DD^*$ threshold. The deeper $bb\bar q \bar q$ bound state aligns with the lattice QCD predictions, while $cc\bar q\bar q$ bound state, still has a much larger binding energy than the recently observed $T^+_{cc}$ by LHCb collaboration. No bound states are identified for the $QQ\bar Q'\bar Q'$, $QQ\bar s\bar s$ and $QQ\bar q\bar q$ with $I=1$. Our analysis shows that the bound $QQ\bar Q'\bar Q'$ states are more probable with a larger mass ratio, $m_Q/m_{Q'}$. Experimental investigation for these states is desired, which will enrich our understanding of hadron spectroscopy and probe insights into the confinement mechanisms within tetraquarks.


title: Characterization of Atmosphere-Skimming Cosmic-Ray Showers in High-Altitude Experiments

abstract: Atmosphere-skimming showers are initiated by cosmic rays with incoming directions such that the full development of the cascade occurs inside the atmosphere without ever reaching the ground. This new class of showers has been observed in balloon-borne experiments such as ANITA, but a characterisation of their properties is lacking. The interplay between the Earth's magnetic field, the long distances over which atmosphere-skimming showers develop, and the low density of the atmosphere they traverse gives rise to several effects that are not seen in downward-going cascades, and require detailed modeling. In this article, we used the latest version of the ZHAireS-RASPASS shower simulation program to tackle this problem, and dwell on the particular phenomena that arises from the peculiar environment on which these showers develop. We focus in particular on the properties of the longitudinal profile of the shower and its fluctuations as a function of cosmic-ray energy, direction and primary mass. We have also studied the phase-space of cosmic-ray arrival directions where detection in high-altitude experiments is more likely, and have found that only in a small range of directions the showers are sufficiently developed before reaching the altitude of the detector. Our results are relevant for the design of high-altitude and in particular balloon-borne experiments, and for the interpretation of the data they collect.


title: AURORA: Navigating UI Tarpits via Automated Neural Screen Understanding

abstract: Nearly a decade of research in software engineering has focused on automating mobile app testing to help engineers in overcoming the unique challenges associated with the software platform. Much of this work has come in the form of Automated Input Generation tools (AIG tools) that dynamically explore app screens. However, such tools have repeatedly been demonstrated to achieve lower-than-expected code coverage - particularly on sophisticated proprietary apps. Prior work has illustrated that a primary cause of these coverage deficiencies is related to so-called tarpits, or complex screens that are difficult to navigate. In this paper, we take a critical step toward enabling AIG tools to effectively navigate tarpits during app exploration through a new form of automated semantic screen understanding. We introduce AURORA, a technique that learns from the visual and textual patterns that exist in mobile app UIs to automatically detect common screen designs and navigate them accordingly. The key idea of AURORA is that there are a finite number of mobile app screen designs, albeit with subtle variations, such that the general patterns of different categories of UI designs can be learned. As such, AURORA employs a multi-modal, neural screen classifier that is able to recognize the most common types of UI screen designs. After recognizing a given screen, it then applies a set of flexible and generalizable heuristics to properly navigate the screen. We evaluated AURORA both on a set of 12 apps with known tarpits from prior work, and on a new set of five of the most popular apps from the Google Play store. Our results indicate that AURORA is able to effectively navigate tarpit screens, outperforming prior approaches that avoid tarpits by 19.6% in terms of method coverage. The improvements can be attributed to AURORA's UI design classification and heuristic navigation techniques.


title: StructLDM: Structured Latent Diffusion for 3D Human Generation

abstract: Recent 3D human generative models have achieved remarkable progress by learning 3D-aware GANs from 2D images. However, existing 3D human generative methods model humans in a compact 1D latent space, ignoring the articulated structure and semantics of human body topology. In this paper, we explore more expressive and higher-dimensional latent space for 3D human modeling and propose StructLDM, a diffusion-based unconditional 3D human generative model, which is learned from 2D images. StructLDM solves the challenges imposed due to the high-dimensional growth of latent space with three key designs: 1) A semantic structured latent space defined on the dense surface manifold of a statistical human body template. 2) A structured 3D-aware auto-decoder that factorizes the global latent space into several semantic body parts parameterized by a set of conditional structured local NeRFs anchored to the body template, which embeds the properties learned from the 2D training data and can be decoded to render view-consistent humans under different poses and clothing styles. 3) A structured latent diffusion model for generative human appearance sampling. Extensive experiments validate StructLDM's state-of-the-art generation performance and illustrate the expressiveness of the structured latent space over the well-adopted 1D latent space. Notably, StructLDM enables different levels of controllable 3D human generation and editing, including pose/view/shape control, and high-level tasks including compositional generations, part-aware clothing editing, 3D virtual try-on, etc. Our project page is at: https://taohuumd.github.io/projects/StructLDM/.


title: Effectively Prompting Small-sized Language Models for Cross-lingual Tasks via Winning Tickets

abstract: Current soft prompt methods yield limited performance when applied to small-sized models (fewer than a billion parameters). Deep prompt-tuning, which entails prepending parameters in each layer for enhanced efficacy, presents a solution for prompting small-sized models, albeit requiring carefully designed implementation. In this paper, we introduce the Lottery Ticket Prompt-learning (LTP) framework that integrates winning tickets with soft prompts. The LTP offers a simpler implementation and requires only a one-time execution. We demonstrate LTP on cross-lingual tasks, where prior works rely on external tools like human-designed multilingual templates and bilingual dictionaries, which may not be feasible in a low-resource regime. Specifically, we select a subset of parameters that have been changed the most during the fine-tuning with the Masked Language Modeling objective. Then, we prepend soft prompts to the original pre-trained language model and only update the selected parameters together with prompt-related parameters when adapting to the downstream tasks. We verify the effectiveness of our LTP framework on cross-lingual tasks, specifically targeting low-resource languages. Our approach outperforms the baselines by only updating 20\% of the original parameters.


title: A Unified and Interpretable Emotion Representation and Expression Generation

abstract: Canonical emotions, such as happy, sad, and fearful, are easy to understand and annotate. However, emotions are often compound, e.g. happily surprised, and can be mapped to the action units (AUs) used for expressing emotions, and trivially to the canonical ones. Intuitively, emotions are continuous as represented by the arousal-valence (AV) model. An interpretable unification of these four modalities - namely, Canonical, Compound, AUs, and AV - is highly desirable, for a better representation and understanding of emotions. However, such unification remains to be unknown in the current literature. In this work, we propose an interpretable and unified emotion model, referred as C2A2. We also develop a method that leverages labels of the non-unified models to annotate the novel unified one. Finally, we modify the text-conditional diffusion models to understand continuous numbers, which are then used to generate continuous expressions using our unified emotion model. Through quantitative and qualitative experiments, we show that our generated images are rich and capture subtle expressions. Our work allows a fine-grained generation of expressions in conjunction with other textual inputs and offers a new label space for emotions at the same time.


title: Searching for enhancement in coalescence of in-jet (anti-)deuterons in proton-proton collisions

abstract: Recent measurements from ALICE report that $``$in-jet'' nucleons carry a higher probability of forming a deuteron via coalescence than the nucleons from the underlying event (UE). This study makes use of an event shape classifier to separate the $``$in-jet'' deuterons and the deuterons in the UE produced in high multiplicity proton-proton collisions at $\sqrt{s} = 13$ TeV. Event shape variables such as transverse spherocity allow the categorization of hard and soft components of an event, which can be divided into two respective classes; $``$jetty'' and $``$isotropic''. The $``$jetty'' deuterons minus the contribution of the deuterons from the $``$isotropic'' event are taken as $``$in-jet'' deuterons, and the coalescence mechanism is tested. The coalescence is performed with a Wigner function formalism, augmented as an afterburner to \textsc{pythia}8. The possible enhancement of the coalescence probability of $``$in-jet'' deuterons is investigated by calculating the coalescence parameter ($B_{2}$) in different spherocity classes in high-multiplicity $pp$ collisions.


title: A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rules

abstract: Since ChatGPT was introduced in November 2022, embedding (nearly) unnoticeable statistical signals into text generated by large language models (LLMs), also known as watermarking, has been used as a principled approach to provable detection of LLM-generated text from its human-written counterpart. In this paper, we introduce a general and flexible framework for reasoning about the statistical efficiency of watermarks and designing powerful detection rules. Inspired by the hypothesis testing formulation of watermark detection, our framework starts by selecting a pivotal statistic of the text and a secret key -- provided by the LLM to the verifier -- to enable controlling the false positive rate (the error of mistakenly detecting human-written text as LLM-generated). Next, this framework allows one to evaluate the power of watermark detection rules by obtaining a closed-form expression of the asymptotic false negative rate (the error of incorrectly classifying LLM-generated text as human-written). Our framework further reduces the problem of determining the optimal detection rule to solving a minimax optimization program. We apply this framework to two representative watermarks -- one of which has been internally implemented at OpenAI -- and obtain several findings that can be instrumental in guiding the practice of implementing watermarks. In particular, we derive optimal detection rules for these watermarks under our framework. These theoretically derived detection rules are demonstrated to be competitive and sometimes enjoy a higher power than existing detection approaches through numerical experiments.

categories: math.ST cs.CL cs.CR cs.LG stat.ML stat.TH


title: New infrared camera of the Caucasian Mountain Observatory of the SAI MSU: design, main parameters, and first light

abstract: This paper presents a prototype of an infrared photometer, created at SAI of MSU based on the commercial infrared module Gavin-615A. The operating spectral range of the photometer is 3-5 mkm. Investigations of the photometer's detector have shown that its parameters coincide with those stated by the manufacturer. The nonlinearity of the detector does not exceed approx 5% across the entire signal range, and coefficients for correction functions were determined. Additionally, we determined the readout noise RN=1200+-210 e-, the conversion coefficient GAIN=520+-9 e-/ADU, the signal magnitude of the bias frame BIAS = 960.5+-2.2 ADU, and the dark current 9.3e6 +-1.1e6 e-/s, which is the sum of the detector's dark current and the radiation from the entrance window of the detector module. The value of dark current was measured at a window temperature of 6C. Observations commenced with the photometer at the 2.5-m telescope of the Caucasian Mountain Observatory of MSU, with the first results presented in this paper. The unvignetted field of view was 30". In the M band under good atmospheric conditions, an image quality close to the diffraction limit was achieved. Images of a star with brightness L=7.96 and M=6.78 were obtained over the 20-second exposure time and a SNR=10. It is shown that at high image quality with SNR=3 and exposure of 20 seconds it is possible to observe stars up to L=9 mag and M=8 mag. The main module of the photometer was also used in measurements of the sky background brightness.


title: An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevance

abstract: Given the rise of multimedia content, human translators increasingly focus on culturally adapting not only words but also other modalities such as images to convey the same meaning. While several applications stand to benefit from this, machine translation systems remain confined to dealing with language in speech and text. In this work, we take a first step towards translating images to make them culturally relevant. First, we build three pipelines comprising state-of-the-art generative models to do the task. Next, we build a two-part evaluation dataset: i) concept: comprising 600 images that are cross-culturally coherent, focusing on a single concept per image, and ii) application: comprising 100 images curated from real-world applications. We conduct a multi-faceted human evaluation of translated images to assess for cultural relevance and meaning preservation. We find that as of today, image-editing models fail at this task, but can be improved by leveraging LLMs and retrievers in the loop. Best pipelines can only translate 5% of images for some countries in the easier concept dataset and no translation is successful for some countries in the application dataset, highlighting the challenging nature of the task. Our code and data is released here: https://github.com/simran-khanuja/image-transcreation.


title: Scalable Scene Modeling from Perspective Imaging: Physics-based Appearance and Geometry Inference

abstract: 3D scene modeling techniques serve as the bedrocks in the geospatial engineering and computer science, which drives many applications ranging from automated driving, terrain mapping, navigation, virtual, augmented, mixed, and extended reality (for gaming and movie industry etc.). This dissertation presents a fraction of contributions that advances 3D scene modeling to its state of the art, in the aspects of both appearance and geometry modeling. In contrast to the prevailing deep learning methods, as a core contribution, this thesis aims to develop algorithms that follow first principles, where sophisticated physic-based models are introduced alongside with simpler learning and inference tasks. The outcomes of these algorithms yield processes that can consume much larger volume of data for highly accurate reconstructing 3D scenes at a scale without losing methodological generality, which are not possible by contemporary complex-model based deep learning methods. Specifically, the dissertation introduces three novel methodologies that address the challenges of inferring appearance and geometry through physics-based modeling. Overall, the research encapsulated in this dissertation marks a series of methodological triumphs in the processing of complex datasets. By navigating the confluence of deep learning, computational geometry, and photogrammetry, this work lays down a robust framework for future exploration and practical application in the rapidly evolving field of 3D scene reconstruction. The outcomes of these studies are evidenced through rigorous experiments and comparisons with existing state-of-the-art methods, demonstrating the efficacy and scalability of the proposed approaches.


title: FireANTs: Adaptive Riemannian Optimization for Multi-Scale Diffeomorphic Registration

abstract: Diffeomorphic Image Registration is a critical part of the analysis in various imaging modalities and downstream tasks like image translation, segmentation, and atlas building. Registration algorithms based on optimization have stood the test of time in terms of accuracy, reliability, and robustness across a wide spectrum of modalities and acquisition settings. However, these algorithms converge slowly, are prohibitively expensive to run, and their usage requires a steep learning curve, limiting their scalability to larger clinical and scientific studies. In this paper, we develop multi-scale Adaptive Riemannian Optimization algorithms for diffeomorphic image registration. We demonstrate compelling improvements on image registration across a spectrum of modalities and anatomies by measuring structural and landmark overlap of the registered image volumes. Our proposed framework leads to a consistent improvement in performance, and from 300x up to 2000x speedup over existing algorithms. Our modular library design makes it easy to use and allows customization via user-defined cost functions.


title: Image Reconstruction from Electroencephalography Using Latent Diffusion

abstract: In this work, we have adopted the diffusion-based image reconstruction pipeline previously used for fMRI image reconstruction and applied it to Electroencephalography (EEG). The EEG encoding method is very simple, and forms a baseline from which more sophisticated EEG encoding methods can be compared. We have also evaluated the fidelity of the generated image using the same metrics used in the previous functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) works. Our results show that while the reconstruction from EEG recorded to rapidly presented images is not as good as reconstructions from fMRI to slower presented images, it holds a surprising amount of information that could be applied in specific use cases. Also, EEG-based image reconstruction works better in some categories-such as land animals and food-than others, shedding new light on previous findings of EEG's sensitivity to those categories and revealing potential for these methods to further understand EEG responses to human visual coding. More investigation should use longer-duration image stimulations to elucidate the later components that might be salient to the different image categories.


title: Duality based error control for the Signorini problem

abstract: In this paper we study the a posteriori bounds for a conforming piecewise linear finite element approximation of the Signorini problem. We prove new rigorous a posteriori estimates of residual type in $L^{p}$, for $p \in (4,\infty)$ in two spatial dimensions. This new analysis treats the positive and negative parts of the discretisation error separately, requiring a novel sign- and bound-preserving interpolant, which is shown to have optimal approximation properties. The estimates rely on the sharp dual stability results on the problem in $W^{2,(4 - \varepsilon)/3}$ for any $\varepsilon \ll 1$. We summarise extensive numerical experiments aimed at testing the robustness of the estimator to validate the theory.


title: Non-radial oscillations in newly born compact star considering effects of phase transition

abstract: The massive stars end their lives by supernova explosions leaving central compact objects that may evolve into neutron stars. Initially, after birth, the star remains hot and gradually cools down. We explore the matter and star properties during this initial stage of the compact stars considering the possibility of the appearance of deconfined quark matter in the core of the star. At the initial stage after the supernova explosion, the occurrence of non-radial oscillation in the newly born compact object is highly possible. Non-radial oscillations are an important source of GWs. There is a high chance for GWs from these oscillations, especially the nodeless fundamental (f-) mode to be detected by next-generation GW detectors. We study the evolution in frequencies of non-radial oscillation after birth considering phase transition and predicting the possible signature for different possibilities of theoretical compact star models.


title: UniArk: Improving Generalisation and Consistency for Factual Knowledge Extraction through Debiasing

abstract: Several recent papers have investigated the potential of language models as knowledge bases as well as the existence of severe biases when extracting factual knowledge. In this work, we focus on the factual probing performance over unseen prompts from tuning, and using a probabilistic view we show the inherent misalignment between pre-training and downstream tuning objectives in language models for probing knowledge. We hypothesize that simultaneously debiasing these objectives can be the key to generalisation over unseen prompts. We propose an adapter-based framework, UniArk, for generalised and consistent factual knowledge extraction through simple methods without introducing extra parameters. Extensive experiments show that UniArk can significantly improve the model's out-of-domain generalisation as well as consistency under various prompts. Additionally, we construct ParaTrex, a large-scale and diverse dataset for measuring the inconsistency and out-of-domain generation of models. Further, ParaTrex offers a reference method for constructing paraphrased datasets using large language models.


title: Finite groups with some subgroups of prime power order satisfying the partial $ \Pi $-property

abstract: Let $ H $ be a subgroup of a finite group $ G $. We say that $ H $ satisfies the partial $ \Pi $-property in $ G $ if there exists a $G$-chief series $ \varGamma_{G}: 1 =G_{0} < G_{1} < \cdot\cdot\cdot < G_{n}= G $ of $ G $ such that $ | G / G_{i-1} : N_{G/G_{i-1}} (HG_{i-1}/G_{i-1}\cap G_{i}/G_{i-1})| $ is a $ \pi (HG_{i-1}/G_{i-1}\cap G_{i}/G_{i-1}) $-number for every $ G $-chief factor $ G_{i}/G_{i-1} $ of $ \varGamma_{G} $, $1\leq i\leq n$. In this paper, we investigate the structure of a finite group $ G $ under the assumption that some subgroups of prime power order satisfy the partial $ \Pi $-property.


title: Gradient Methods for Scalable Multi-value Electricity Network Expansion Planning

abstract: We consider multi-value expansion planning (MEP), a general bilevel optimization model in which a planner optimizes arbitrary functions of the dispatch outcome in the presence of a partially controllable, competitive electricity market. The MEP problem can be used to jointly plan various grid assets, such as transmission, generation, and battery storage capacities; examples include identifying grid investments that minimize emissions in the absence of a carbon tax, maximizing the profit of a portfolio of renewable investments and long-term energy contracts, or reducing price inequities between different grid stakeholders. The MEP problem, however, is in general nonconvex, making it difficult to solve exactly for large real-world systems. Therefore, we propose a fast stochastic implicit gradient-based heuristic method that scales well to large networks with many scenarios. We use a strong duality reformulation and the McCormick envelope to provide a lower bound on the performance of our algorithm via convex relaxation. We test the performance of our method on a large model of the U.S. Western Interconnect and demonstrate that it scales linearly with network size and number of scenarios and can be efficiently parallelized on large machines. We find that for medium-sized 16 hour cases, gradient descent on average finds a 5.3x lower objective value in 16.5x less time compared to a traditional reformulation-based approach solved with an interior point method. We conclude with a large example in which we jointly plan transmission, generation, and storage for a 768 hour case on 100 node system, showing that emissions penalization leads to additional 40.0% reduction in carbon intensity at an additional cost of $17.1/MWh.


title: The Countable Reals

abstract: We construct a topos in which the Dedekind reals are countable. To accomplish this, we first define a new kind of toposes that we call parameterized realizability toposes. They are built from partial combinatory algebras whose application operation depends on a parameter, and in which realizers operate uniformly with respect to a given parameter set. Our topos is the parameterized realizability topos whose realizers are oracle-computable partial maps, with oracles serving as parameters and ranging over the representations of a non-diagonalizable sequence, discovered by Joseph Miller. It is a sequence of reals in $[0,1]$ that is non-diagonalizable in the sense that any real in $[0,1]$ that is oracle-computable, uniformly in oracles representing the sequence, must already appear in the sequence. The Dedekind reals are countable in the topos because the non-diagonalizable sequence appears in it as an epimorphism. The topos is intuitionistic, as it invalidates both the law of excluded middle and the axiom of countable choice. The Cauchy reals are uncountable. The Hilbert cube is countable, from which Brouwer's fixed-point theorem follows as an easy corollary of Lawvere's fixed-point theorem. From the 1-dimensional Brouwer's fixed-point theorem we obtain the intermediate value theorem and the lesser limited principle of omniscience. The Kreisel-Lacombe-Shoenfield-Tseitin theorem stating that all real-valued maps are continuous is valid, because the usual proof is uniform with respect to oracles. Lastly, the closed interval $[0,1]$, being countable, can trivially be covered by a sequence of open intervals whose lengths add up to any prescribed $0 < \epsilon < 1$, and such a cover has no finite subcover. However, we show that any sequence of open intervals with rational endpoints covering $[0,1]$ must has a finite subcover.


title: New logarithmic step size for stochastic gradient descent

abstract: In this paper, we propose a novel warm restart technique using a new logarithmic step size for the stochastic gradient descent (SGD) approach. For smooth and non-convex functions, we establish an $O(\frac{1}{\sqrt{T}})$ convergence rate for the SGD. We conduct a comprehensive implementation to demonstrate the efficiency of the newly proposed step size on the ~FashionMinst,~ CIFAR10, and CIFAR100 datasets. Moreover, we compare our results with nine other existing approaches and demonstrate that the new logarithmic step size improves test accuracy by $0.9\%$ for the CIFAR100 dataset when we utilize a convolutional neural network (CNN) model.


title: Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward

abstract: Preference modeling techniques, such as direct preference optimization (DPO), has shown effective in enhancing the generalization abilities of large language model (LLM). However, in tasks involving video instruction-following, providing informative feedback, especially for detecting hallucinations in generated responses, remains a significant challenge. Previous studies have explored using large large multimodal models (LMMs) as reward models to guide preference modeling, but their ability to accurately assess the factuality of generated responses compared to corresponding videos has not been conclusively established. This paper introduces a novel framework that utilizes detailed video captions as a proxy of video content, enabling language models to incorporate this information as supporting evidence for scoring video Question Answering (QA) predictions. Our approach demonstrates robust alignment with OpenAI GPT-4V model's reward mechanism, which directly takes video frames as input. Furthermore, we show that applying this tailored reward through DPO significantly improves the performance of video LMMs on video QA tasks.


title: Dynamics and Optimization in Spatially Distributed Electrical Vehicle Charging

abstract: We consider a spatially distributed demand for electrical vehicle recharging, that must be covered by a fixed set of charging stations. Arriving EVs receive feedback on transport times to each station, and waiting times at congested ones, based on which they make a selfish selection. This selection determines total arrival rates in station queues, which are represented by a fluid state; departure rates are modeled under the assumption that clients have a given sojourn time in the system. The resulting differential equation system is analyzed with tools of optimization. We characterize the equilibrium as the solution to a specific convex program, which has connections to optimal transport problems, and also with road traffic theory. In particular a price of anarchy appears with respect to a social planner's allocation. From a dynamical perspective, global convergence to equilibrium is established, with tools of Lagrange duality and Lyapunov theory. An extension of the model that makes customer demand elastic to observed delays is also presented, and analyzed with extensions of the optimization machinery. Simulations to illustrate the global behavior are presented, which also help validate the model beyond the fluid approximation.


title: Bridging Remote Sensors with Multisensor Geospatial Foundation Models

abstract: In the realm of geospatial analysis, the diversity of remote sensors, encompassing both optical and microwave technologies, offers a wealth of distinct observational capabilities. Recognizing this, we present msGFM, a multisensor geospatial foundation model that effectively unifies data from four key sensor modalities. This integration spans an expansive dataset of two million multisensor images. msGFM is uniquely adept at handling both paired and unpaired sensor data. For data originating from identical geolocations, our model employs an innovative cross-sensor pretraining approach in masked image modeling, enabling the synthesis of joint representations from diverse sensors. msGFM, incorporating four remote sensors, upholds strong performance, forming a comprehensive model adaptable to various sensor types. msGFM has demonstrated enhanced proficiency in a range of both single-sensor and multisensor downstream tasks. These include scene classification, segmentation, cloud removal, and pan-sharpening. A key discovery of our research is that representations derived from natural images are not always compatible with the distinct characteristics of geospatial remote sensors, underscoring the limitations of existing representations in this field. Our work can serve as a guide for developing multisensor geospatial pretraining models, paving the way for more advanced geospatial capabilities.


title: FABLES: Evaluating faithfulness and content selection in book-length summarization

abstract: While long-context large language models (LLMs) can technically summarize book-length documents (>100K tokens), the length and complexity of the documents have so far prohibited evaluations of input-dependent aspects like faithfulness. In this paper, we conduct the first large-scale human evaluation of faithfulness and content selection on LLM-generated summaries of fictional books. Our study mitigates the issue of data contamination by focusing on summaries of books published in 2023 or 2024, and we hire annotators who have fully read each book prior to the annotation task to minimize cost and cognitive burden. We collect FABLES, a dataset of annotations on 3,158 claims made in LLM-generated summaries of 26 books, at a cost of $5.2K USD, which allows us to rank LLM summarizers based on faithfulness: Claude-3-Opus significantly outperforms all closed-source LLMs, while the open-source Mixtral is on par with GPT-3.5-Turbo. An analysis of the annotations reveals that most unfaithful claims relate to events and character states, and they generally require indirect reasoning over the narrative to invalidate. While LLM-based auto-raters have proven reliable for factuality and coherence in other settings, we implement several LLM raters of faithfulness and find that none correlates strongly with human annotations, especially with regard to detecting unfaithful claims. Our experiments suggest that detecting unfaithful claims is an important future direction not only for summarization evaluation but also as a testbed for long-context understanding. Finally, we move beyond faithfulness by exploring content selection errors in book-length summarization: we develop a typology of omission errors related to crucial narrative elements and also identify a systematic over-emphasis on events occurring towards the end of the book.


title: Engineered Graviton Condensates in a Room-Temperature Superconductor for a Unified Quantum Fibonacci Field Theory

abstract: In the quest to unify quantum mechanics with general relativity, the concept of gravitons as quantum carriers of gravitational force stands as a pivotal yet unproven hypothesis. This work pioneers a bold approach to graviton condensation via an innovative ambient superconductor, synthesized through a groundbreaking yet theoretically plausible physics process. Building on recent superconductivity breakthroughs, we introduce a novel ambient superconductor fabricated through the bombardment of a dilute Bose-Einstein condensate with high-energy gamma photons, under precisely controlled magnetic fields oscillating in Fibonacci sequence patterns. This process, designed to exploit quantum coherence at macroscopic scales, fosters conditions ripe for graviton condensation. Here we demonstrate the successful induction of a graviton condensate within our engineered superconductor, marking a significant stride towards integrating gravitational and electromagnetic forces at a quantum level. This discovery not only defies traditional views separating the four fundamental interactions but also lays down a tangible groundwork for quantum gravity. Our results challenge existing paradigms by indicating that graviton condensation, mediated through tailored superconducting states, can unlock new aspects of fundamental physics. This graviton condensate represents a critical step towards a unified field theory, bridging the gap between quantum mechanics and general relativity. This investigation not only extends the boundaries of theoretical physics but also hints at potential future technologies harnessing the quantum interplay of gravitational and electromagnetic forces. Through imaginative yet rigorous scientific inquiry, this study underscores the limitless potential of physics to explore and unify the natural world's most fundamental forces.


title: Artificial Intelligence and the Spatial Documentation of Languages

abstract: The advancement in technology has made interdisciplinary research more accessible. Particularly the breakthrough in Artificial Intelligence AI has given huge advantages to researchers working in interdisciplinary and multidisciplinary fields. This study investigates the ability of AI models, particularly GPT4 and GPT Data Analyst in creating language maps for language documentation. The study Integrates documentary linguistics linguistic geography and AI by showcasing how AI models facilitate the spatial documentation of languages through the creation of language maps with minimal cartographic expertise. The study is conducted using a CSV file and a GeoJSON file both obtained from HDX and from the researchers fieldwork. The study data is then applied in realtime conversations with the AI models in order to generate the language distribution maps. The study highlights the two AI models capabilities in generating highquality static and interactive web maps and streamlining the mapmaking process, despite facing challenges like inconsistencies and difficulties in adding legends. The findings suggest a promising future for AI in generating language maps and enhancing the work of documentary linguists as they collect their data in the field pointing towards the need for further development to fully harness AI potential in this field.


title: JWST COMPASS: A NIRSpec/G395H Transmission Spectrum of the Sub-Neptune TOI-836c

abstract: Planets between the sizes of Earth and Neptune are the most common in the Galaxy, bridging the gap between the terrestrial and giant planets in our Solar System. Now that we are firmly in the era of JWST, we can begin to measure, in more detail, the atmospheres of these ubiquitous planets to better understand their evolutionary trajectories. The two planets in the TOI-836 system are ideal candidates for such a study, as they fall on either side of the radius valley, allowing for direct comparisons of the present-day atmospheres of planets that formed in the same environment but had different ultimate end states. We present results from the JWST NIRSpec G395H transit observation of the larger and outer of the planets in this system, TOI-836c (2.587 R$_{\oplus}$, 9.6 M$_{\oplus}$, T$_{\rm eq}$$\sim$665 K). While we measure average 30-pixel binned precisions of $\sim$24ppm for NRS1 and $\sim$43ppm for NRS2 per spectral bin, we do find residual correlated noise in the data, which we attempt to correct using the JWST Engineering Database. We find a featureless transmission spectrum for this sub-Neptune planet, and are able to rule out atmospheric metallicities $<$175$\times$ Solar in the absence of aerosols at $\lesssim$1 millibar. We leverage microphysical models to determine that aerosols at such low pressures are physically plausible. The results presented herein represent the first observation from the COMPASS (Compositions of Mini-Planet Atmospheres for Statistical Study) JWST program, which also includes TOI-836b and will ultimately compare the presence and compositions of atmospheres for 12 super-Earths/sub-Neptunes.


title: Review of Distributed Quantum Computing. From single QPU to High Performance Quantum Computing

abstract: The emerging field of quantum computing has shown it might change how we process information by using the unique principles of quantum mechanics. As researchers continue to push the boundaries of quantum technologies to unprecedented levels, distributed quantum computing raises as an obvious path to explore with the aim of boosting the computational power of current quantum systems. This paper presents a comprehensive survey of the current state of the art in the distributed quantum computing field, exploring its foundational principles, landscape of achievements, challenges, and promising directions for further research. From quantum communication protocols to entanglement-based distributed algorithms, each aspect contributes to the mosaic of distributed quantum computing, making it an attractive approach to address the limitations of classical computing. Our objective is to provide an exhaustive overview for experienced researchers and field newcomers.


title: IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representations

abstract: Current foundation models exhibit impressive capabilities when prompted either with text only or with both image and text inputs. But do their capabilities change depending on the input modality? In this work, we propose $\textbf{IsoBench}$, a benchmark dataset containing problems from four major areas: math, science, algorithms, and games. Each example is presented with multiple $\textbf{isomorphic representations}$ of inputs, such as visual, textual, and mathematical presentations. IsoBench provides fine-grained feedback to diagnose performance gaps caused by the form of the representation. Across various foundation models, we observe that on the same problem, models have a consistent preference towards textual representations. Most prominently, when evaluated on all IsoBench problems, Claude-3 Opus performs 28.7 points worse when provided with images instead of text; similarly, GPT-4 Turbo is 18.7 points worse and Gemini Pro is 14.9 points worse. Finally, we present two prompting techniques, $\textit{IsoCombination}$ and $\textit{IsoScratchPad}$, which improve model performance by considering combinations of, and translations between, different input representations.


title: Non-asymptotic Global Convergence Rates of BFGS with Exact Line Search

abstract: In this paper, we explore the non-asymptotic global convergence rates of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method implemented with exact line search. Notably, due to Dixon's equivalence result, our findings are also applicable to other quasi-Newton methods in the convex Broyden class employing exact line search, such as the Davidon-Fletcher-Powell (DFP) method. Specifically, we focus on problems where the objective function is strongly convex with Lipschitz continuous gradient and Hessian. Our results hold for any initial point and any symmetric positive definite initial Hessian approximation matrix. The analysis unveils a detailed three-phase convergence process, characterized by distinct linear and superlinear rates, contingent on the iteration progress. Additionally, our theoretical findings demonstrate the trade-offs between linear and superlinear convergence rates for BFGS when we modify the initial Hessian approximation matrix, a phenomenon further corroborated by our numerical experiments.


title: Mapping the Increasing Use of LLMs in Scientific Papers

abstract: Scientific publishing lays the foundation of science by disseminating research findings, fostering collaboration, encouraging reproducibility, and ensuring that scientific knowledge is accessible, verifiable, and built upon over time. Recently, there has been immense speculation about how many people are using large language models (LLMs) like ChatGPT in their academic writing, and to what extent this tool might have an effect on global scientific practices. However, we lack a precise measure of the proportion of academic writing substantially modified or produced by LLMs. To address this gap, we conduct the first systematic, large-scale analysis across 950,965 papers published between January 2020 and February 2024 on the arXiv, bioRxiv, and Nature portfolio journals, using a population-level statistical framework to measure the prevalence of LLM-modified content over time. Our statistical estimation operates on the corpus level and is more robust than inference on individual instances. Our findings reveal a steady increase in LLM usage, with the largest and fastest growth observed in Computer Science papers (up to 17.5%). In comparison, Mathematics papers and the Nature portfolio showed the least LLM modification (up to 6.3%). Moreover, at an aggregate level, our analysis reveals that higher levels of LLM-modification are associated with papers whose first authors post preprints more frequently, papers in more crowded research areas, and papers of shorter lengths. Our findings suggests that LLMs are being broadly used in scientific writings.


title: Metallicity and $\alpha$-abundance for 48 million stars in low-extinction regions in the Milky Way

abstract: We estimate ([M/H], [$\alpha$/M]) for 48 million giants and dwarfs in low dust extinction regions from the Gaia DR3 XP spectra by using tree-based machine-learning models trained on APOGEE DR17 and metal-poor star sample of Li et al. The root mean square error of our estimation is 0.0890 dex for [M/H] and 0.0436 dex for [$\alpha$/M], when we evaluate our models with the test data. Our estimation is most reliable for giants and metal-rich stars, because the training data are dominated by such stars. The high-[$\alpha$/M] stars and low-[$\alpha$/M] stars selected by our ([M/H], [$\alpha$/M]) show different kinematical properties for giants and low-temperature dwarfs. We further investigate how our machine-learning models extract information on ([M/H], [$\alpha$/M]). Intriguingly, our models seem to extract information on [$\alpha$/M] from Na D lines (589 nm) and Mg I line (516 nm). This result is understandable given the observed correlation between Na and Mg abundances in the literature. The catalog of ([M/H], [$\alpha$/M]) as well as their associated uncertainties will be publicly available online.


title: Decentralized Collaborative Learning Framework with External Privacy Leakage Analysis

abstract: This paper presents two methodological advancements in decentralized multi-task learning under privacy constraints, aiming to pave the way for future developments in next-generation Blockchain platforms. First, we expand the existing framework for collaborative dictionary learning (CollabDict), which has previously been limited to Gaussian mixture models, by incorporating deep variational autoencoders (VAEs) into the framework, with a particular focus on anomaly detection. We demonstrate that the VAE-based anomaly score function shares the same mathematical structure as the non-deep model, and provide comprehensive qualitative comparison. Second, considering the widespread use of "pre-trained models," we provide a mathematical analysis on data privacy leakage when models trained with CollabDict are shared externally. We show that the CollabDict approach, when applied to Gaussian mixtures, adheres to a Renyi differential privacy criterion. Additionally, we propose a practical metric for monitoring internal privacy breaches during the learning process.


title: Energy, strength, and alpha width measurements of $E_{\rm{c.m.}} = 1323$ and $1487$ keV resonances in $^{15}$N($\alpha,\gamma$)$^{19}$F

abstract: The $^{15}$N($\alpha,\gamma$)$^{19}$F reaction produces $^{19}$F in asymptotic giant branch (AGB) stars, where the low energy tails of two resonances at $E_{\rm{c.m.}} = 1323 \pm 2$ and $1487 \pm 1.7$ keV are estimated to contribute about $30\%$ of the total reaction rate in these environments. However, recent measurements have shown discrepancies in the energies, the strengths, and the corresponding alpha widths of these two resonances, resulting in an increase in the systematic uncertainty of the extrapolated cross section to helium burning energies. With this motivation, we have undertaken new measurements of the $^{15}$N$(\alpha,\gamma)^{19}$F at the University of Notre Dame Nuclear Science Laboratory. The setup consisted of an alpha particle beam impinged on a solid Ti$^{15}$N target with gamma-ray spectroscopy accomplished using a high purity germanium detector. Using the Doppler corrected gamma-ray energies, we confirmed the lower resonance energy to be $1321.6 \pm 0.6$ keV and found a value for the higher one of $1479.4 \pm 0.6$ keV that is more consistent with those found from previous elastic scattering studies. We found that the resonance strengths for both were consistent with most values found in the literature, but a larger alpha width has been recommended for the $E_{\rm{c.m.}} = 1487$ keV resonance. The larger alpha width suggests a reaction rate increase of about $15\%$ at temperatures $T < 0.1$ GK relevant to low mass AGB stars. The impact of the increased reaction rate requires further investigations.


title: Language Guided Domain Generalized Medical Image Segmentation

abstract: Single source domain generalization (SDG) holds promise for more reliable and consistent image segmentation across real-world clinical settings particularly in the medical domain, where data privacy and acquisition cost constraints often limit the availability of diverse datasets. Depending solely on visual features hampers the model's capacity to adapt effectively to various domains, primarily because of the presence of spurious correlations and domain-specific characteristics embedded within the image features. Incorporating text features alongside visual features is a potential solution to enhance the model's understanding of the data, as it goes beyond pixel-level information to provide valuable context. Textual cues describing the anatomical structures, their appearances, and variations across various imaging modalities can guide the model in domain adaptation, ultimately contributing to more robust and consistent segmentation. In this paper, we propose an approach that explicitly leverages textual information by incorporating a contrastive learning mechanism guided by the text encoder features to learn a more robust feature representation. We assess the effectiveness of our text-guided contrastive feature alignment technique in various scenarios, including cross-modality, cross-sequence, and cross-site settings for different segmentation tasks. Our approach achieves favorable performance against existing methods in literature. Our code and model weights are available at https://github.com/ShahinaKK/LG_SDG.git.


title: TWIN-GPT: Digital Twins for Clinical Trials via Large Language Model

abstract: Recently, there has been a burgeoning interest in virtual clinical trials, which simulate real-world scenarios and hold the potential to significantly enhance patient safety, expedite development, reduce costs, and contribute to the broader scientific knowledge in healthcare. Existing research often focuses on leveraging electronic health records (EHRs) to support clinical trial outcome prediction. Yet, trained with limited clinical trial outcome data, existing approaches frequently struggle to perform accurate predictions. Some research has attempted to generate EHRs to augment model development but has fallen short in personalizing the generation for individual patient profiles. Recently, the emergence of large language models has illuminated new possibilities, as their embedded comprehensive clinical knowledge has proven beneficial in addressing medical issues. In this paper, we propose a large language model-based digital twin creation approach, called TWIN-GPT. TWIN-GPT can establish cross-dataset associations of medical information given limited data, generating unique personalized digital twins for different patients, thereby preserving individual patient characteristics. Comprehensive experiments show that using digital twins created by TWIN-GPT can boost clinical trial outcome prediction, exceeding various previous prediction approaches. Besides, we also demonstrate that TWIN-GPT can generate high-fidelity trial data that closely approximate specific patients, aiding in more accurate result predictions in data-scarce situations. Moreover, our study provides practical evidence for the application of digital twins in healthcare, highlighting its potential significance.


title: Growth of regular partitions 1: Improved bounds for small slicewise VC-dimension

abstract: This is Part 1 in a series of papers about sizes of regular partitions of $3$-uniform hypergraphs. Previous work of the author and Wolf, and independently Chernikov and Towsner, showed that $3$-uniform hypergraphs of small slicewise VC-dimension admit homogeneous partitions. The results of Chernikov and Towsner did not produce explicit bounds, while the work of the author and Wolf relied on a strong version of hypergraph regularity, and consequently produced a Wowzer type bound on the size of the partition. This paper gives a new proof of this result, yielding $\e$-homogeneous partitions of size at most $2^{2^{\epsilon^{-K}}}$, where $K$ is a constant depending on the slicewise VC-dimension. This result is a crucial ingredient in Part 2 of the series, which investigates the growth of weak regular partitions in hereditary properties of $3$-uniform hypergraphs.


title: Limit-case admissibility for positive infinite-dimensional systems

abstract: In the context of positive infinite-dimensional linear systems, we systematically study $L^p$-admissible control and observation operators with respect to the limit-cases $p=\infty$ and $p=1$, respectively. This requires an in-depth understanding of the order structure on the extrapolation space $X_{-1}$, which we provide. These properties of $X_{-1}$ also enable us to discuss when zero-class admissibility is automatic. While those limit-cases are the weakest form of admissibility on the $L^p$-scale, it is remarkable that they sometimes follow from order theoretic and geometric assumptions. Our assumptions on the geometries of the involved spaces are minimal.


title: Variable-Length Stop-Feedback Coding for Minimum Age of Incorrect Information

abstract: The Age of Incorrect Information (AoII) is studied within the context of remote monitoring a Markov source using variable-length stop-feedback (VLSF) coding. Leveraging recent results on the non-asymptotic channel coding rate, we consider sources with small cardinality, where feedback is non-instantaneous as the transmitted information and feedback message have comparable lengths. We focus on the feedback sequence, i.e. the times of feedback transmissions, and derive AoII-optimal and delay-optimal feedback sequences. Our results showcase the impact of the feedback sequence on the AoII, revealing that a lower average delay does not necessarily correspond to a lower average AoII. We discuss the implications of our findings and suggest directions for coding scheme design.


title: Direct detection of quasiparticle tunneling with a charge-sensitive superconducting sensor coupled to a waveguide

abstract: Detecting quasiparticle tunneling events in superconducting circuits provides information about the population and dynamics of non-equilibrium quasiparticles. Such events can be detected by monitoring changes in the frequency of an offset-charge-sensitive superconducting qubit. This monitoring has so far been performed by Ramsey interferometry assisted by a readout resonator. Here, we demonstrate a quasiparticle detector based on a superconducting qubit directly coupled to a waveguide. We directly measure quasiparticle number parity on the qubit island by probing the coherent scattering of a microwave tone, offering simplicity of operation, fast detection speed, and a large signal-to-noise ratio. We observe tunneling rates between 0.8 and $7~\rm{s}^{-1}$, depending on the average occupation of the detector qubit, and achieve a temporal resolution below $10~\mu\rm{s}$ without a quantum-limited amplifier. Our simple and efficient detector lowers the barrier to perform studies of quasiparticle population and dynamics, facilitating progress in fundamental science, quantum information processing, and sensing.


title: BiPer: Binary Neural Networks using a Periodic Function

abstract: Quantized neural networks employ reduced precision representations for both weights and activations. This quantization process significantly reduces the memory requirements and computational complexity of the network. Binary Neural Networks (BNNs) are the extreme quantization case, representing values with just one bit. Since the sign function is typically used to map real values to binary values, smooth approximations are introduced to mimic the gradients during error backpropagation. Thus, the mismatch between the forward and backward models corrupts the direction of the gradient, causing training inconsistency problems and performance degradation. In contrast to current BNN approaches, we propose to employ a binary periodic (BiPer) function during binarization. Specifically, we use a square wave for the forward pass to obtain the binary values and employ the trigonometric sine function with the same period of the square wave as a differentiable surrogate during the backward pass. We demonstrate that this approach can control the quantization error by using the frequency of the periodic function and improves network performance. Extensive experiments validate the effectiveness of BiPer in benchmark datasets and network architectures, with improvements of up to 1% and 0.69% with respect to state-of-the-art methods in the classification task over CIFAR-10 and ImageNet, respectively. Our code is publicly available at https://github.com/edmav4/BiPer.


title: Lower bounds and integrality gaps in simplicial decomposition

abstract: Let $\mathcal{K}$ be a finite pure simplicial $d$-complex, with oriented facets $\{F_i\}$, which is boundaryless in the sense that $\sum\partial F_i=0$. We call such a $\mathcal{K}$ an \textit{admissible $d$-complex}. Given an admissible $d$-complex, one can ask for the smallest collection $\{T_i\}$ of oriented $(d+1)$-simplices on the vertices of $\mathcal{K}$ which decomposes $\mathcal{K}$ in the sense that $\sum \partial T_i = \mathcal{K}$. Let the minimum size of such a collection be $V_\mathbb{Z}(\mathcal{K})$, and let $V_\mathbb{Q}(\mathcal{K})$ be the relaxed analog where fractional $(d+1)$-simplices may be used. We explain how these quantities may be computed via integer and linear programming, and show how lower bounds may be obtained by exploiting LP-duality. We then prove that $V_\mathbb{Q}$ and $V_\mathbb{Z}$ are both additive under disjoint union and connected sum along a $d$-simplex. The remainder of the paper explores integrality gaps between $V_\mathbb{Z}$ and $V_\mathbb{Q}$ in dimension 1, where we share what we believe is the simplest admissible complex with an integrality gap; and in dimension 2, where we collect some results on integrality gaps for triangulations of the 2-sphere for a companion paper with Zili Wang and Peter Doyle.


title: Is $\gamma_{KLS}$-generalized statistical field theory complete?

abstract: In this Letter we introduce some field-theoretic approach for computing the critical properties of $\gamma_{KLS}$-generalized systems undergoing continuous phase transitions, namely $\gamma_{KLS}$-statistical field theory. From this new approach emerges the new generalized O($N$)$_{\gamma_{KLS}}$ universality class, which is capable of encompassing nonconventional critical exponents for real imperfect systems known as manganites not described by standard statistical field theory. We compare the generalized results with those obtained from measurements in manganites. The agreement was satisfactory, where the relative errors are $< 5\%$ for the most of manganites used. Although the present approach describes the aforementioned nonconventional critical indices, we show that it is not complete. For example, it does not explain the results for some other manganites, being explained only for nonextensive statistical field theory recently introduced in literature. So, $\gamma_{KLS}$-statistical field theory has to be discarded for statistical mechanics generalization purposes.

categories: hep-th cond-mat.stat-mech math-ph math.MP


title: The pullback theorem for relative monads

abstract: A fundamental result in the theory of monads is the characterisation of the category of algebras for a monad in terms of a pullback of the category of presheaves on the category of free algebras: intuitively, this expresses that every algebra is a colimit of free algebras. We establish an analogous result for enriched relative monads with dense roots, and explain how it generalises the nerve theorems for monads with arities and nervous monads. As an application, we derive sufficient conditions for the existence of algebraic colimits of relative monads. More generally, we show that the pullback theorem for relative monads holds in any exact virtual equipment. In doing so, we are led to study the relationship between a $j$-relative monad $T$ and its associated loose-monad $E(j, T)$, and consequently show that the opalgebra object and the algebra object for $T$ may be constructed from certain double categorical limits and colimits associated to $E(j, T)$.


title: LoSA: Long-Short-range Adapter for Scaling End-to-End Temporal Action Localization

abstract: Temporal Action Localization (TAL) involves localizing and classifying action snippets in an untrimmed video. The emergence of large video foundation models has led RGB-only video backbones to outperform previous methods needing both RGB and optical flow modalities. Leveraging these large models is often limited to training only the TAL head due to the prohibitively large GPU memory required to adapt the video backbone for TAL. To overcome this limitation, we introduce LoSA, the first memory-and-parameter-efficient backbone adapter designed specifically for TAL to handle untrimmed videos. LoSA specializes for TAL by introducing Long-Short-range Adapters that adapt the intermediate layers of the video backbone over different temporal ranges. These adapters run parallel to the video backbone to significantly reduce memory footprint. LoSA also includes Long-Short-range Fusion that strategically combines the output of these adapters from the video backbone layers to enhance the video features provided to the TAL head. Experiments show that LoSA significantly outperforms all existing methods on standard TAL benchmarks, THUMOS-14 and ActivityNet-v1.3, by scaling end-to-end backbone adaptation to billion-parameter-plus models like VideoMAEv2~(ViT-g) and leveraging them beyond head-only transfer learning.


title: Evaluating Privacy Perceptions, Experience, and Behavior of Software Development Teams

abstract: With the increase in the number of privacy regulations, small development teams are forced to make privacy decisions on their own. In this paper, we conduct a mixed-method survey study, including statistical and qualitative analysis, to evaluate the privacy perceptions, practices, and knowledge of members involved in various phases of the Software Development Life Cycle (SDLC). Our survey includes 362 participants from 23 countries, encompassing roles such as product managers, developers, and testers. Our results show diverse definitions of privacy across SDLC roles, emphasizing the need for a holistic privacy approach throughout SDLC. We find that software teams, regardless of their region, are less familiar with privacy concepts (such as anonymization), relying on self-teaching and forums. Most participants are more familiar with GDPR and HIPAA than other regulations, with multi-jurisdictional compliance being their primary concern. Our results advocate the need for role-dependent solutions to address the privacy challenges, and we highlight research directions and educational takeaways to help improve privacy-aware SDLC.


title: Large Motion Model for Unified Multi-Modal Motion Generation

abstract: Human motion generation, a cornerstone technique in animation and video production, has widespread applications in various tasks like text-to-motion and music-to-dance. Previous works focus on developing specialist models tailored for each task without scalability. In this work, we present Large Motion Model (LMM), a motion-centric, multi-modal framework that unifies mainstream motion generation tasks into a generalist model. A unified motion model is appealing since it can leverage a wide range of motion data to achieve broad generalization beyond a single task. However, it is also challenging due to the heterogeneous nature of substantially different motion data and tasks. LMM tackles these challenges from three principled aspects: 1) Data: We consolidate datasets with different modalities, formats and tasks into a comprehensive yet unified motion generation dataset, MotionVerse, comprising 10 tasks, 16 datasets, a total of 320k sequences, and 100 million frames. 2) Architecture: We design an articulated attention mechanism ArtAttention that incorporates body part-aware modeling into Diffusion Transformer backbone. 3) Pre-Training: We propose a novel pre-training strategy for LMM, which employs variable frame rates and masking forms, to better exploit knowledge from diverse training data. Extensive experiments demonstrate that our generalist LMM achieves competitive performance across various standard motion generation tasks over state-of-the-art specialist models. Notably, LMM exhibits strong generalization capabilities and emerging properties across many unseen tasks. Additionally, our ablation studies reveal valuable insights about training and scaling up large motion models for future research.


title: Weak-coupling limits of the quantum Langevin equation for an oscillator

abstract: The quantum Langevin equation as obtained from the independent-oscillator model describes a strong-coupling situation, devoid of the Born-Markov approximation that is employed in the context of the Gorini-Kossakowski-Sudarshan-Lindblad equation. The question we address is what happens when we implement such `Born-Markov'-like approximations at the level of the quantum Langevin equation for a harmonic oscillator which carries a noise term satisfying a fluctuation-dissipation theorem. In this backdrop, we also comment on the rotating-wave approximation.


title: Beyond Linear Response: Equivalence between Thermodynamic Geometry and Optimal Transport

abstract: A fundamental result of thermodynamic geometry is that the optimal, minimal-work protocol that drives a nonequilibrium system between two thermodynamic states in the slow-driving limit is given by a geodesic of the friction tensor, a Riemannian metric defined on control space. For overdamped dynamics in arbitrary dimensions, we demonstrate that thermodynamic geometry is equivalent to $L^2$ optimal transport geometry defined on the space of equilibrium distributions corresponding to the control parameters. We show that obtaining optimal protocols past the slow-driving or linear response regime is computationally tractable as the sum of a friction tensor geodesic and a counterdiabatic term related to the Fisher information metric. These geodesic-counterdiabatic optimal protocols are exact for parameteric harmonic potentials, reproduce the surprising non-monotonic behavior recently discovered in linearly-biased double well optimal protocols, and explain the ubiquitous discontinuous jumps observed at the beginning and end times.


title: Understanding and utilizing the inner bonds of process tensors

abstract: Process tensor matrix product operators (PT-MPOs) enable numerically exact simulations for an unprecedentedly broad range of open quantum systems. By representing environment influences in MPO form, they can be efficiently compressed using established algorithms. The dimensions of inner bonds of the compressed PT-MPO may be viewed as an indicator of the complexity of the environment. Here, we show that the inner bonds themselves, not only their dimensions, have a concrete physical meaning: They represent the subspace of the full environment Liouville space which hosts environment excitations that may influence the subsequent open quantum systems dynamics the most. This connection can be expressed in terms of lossy linear transformations, whose pseudoinverses facilitate the extraction of environment observables. We demonstrate this by extracting the environment spin of a central spin problem, the current through a quantum system coupled to two leads, the number of photons emitted from quantum emitters into a structured environment, and the distribution of the total absorbed energy in a driven non-Markovian quantum system into system, environment, and interaction energy terms. Numerical tests further indicate that different PT-MPO algorithms compress environments to similar subspaces. Thus, the physical interpretation of inner bonds of PT-MPOs both provides a conceptional understanding and it enables new practical applications.


title: Large Language Models are Capable of Offering Cognitive Reappraisal, if Guided

abstract: Large language models (LLMs) have offered new opportunities for emotional support, and recent work has shown that they can produce empathic responses to people in distress. However, long-term mental well-being requires emotional self-regulation, where a one-time empathic response falls short. This work takes a first step by engaging with cognitive reappraisals, a strategy from psychology practitioners that uses language to targetedly change negative appraisals that an individual makes of the situation; such appraisals is known to sit at the root of human emotional experience. We hypothesize that psychologically grounded principles could enable such advanced psychology capabilities in LLMs, and design RESORT which consists of a series of reappraisal constitutions across multiple dimensions that can be used as LLM instructions. We conduct a first-of-its-kind expert evaluation (by clinical psychologists with M.S. or Ph.D. degrees) of an LLM's zero-shot ability to generate cognitive reappraisal responses to medium-length social media messages asking for support. This fine-grained evaluation showed that even LLMs at the 7B scale guided by RESORT are capable of generating empathic responses that can help users reappraise their situations.


title: The galaxy cluster AC114 III. The role of galaxy clusters in the mass-metallicity relation

abstract: We study the role of galaxy clusters in the mass-metallicity relation by using a sample of galaxies belonging to the AC114 (z=0.317) galaxy cluster. The data was taken from archival VIMOS-MOS spectroscopy to estimate gas-phase metallicities by using strong-line methods. On the other hand, the data obtained from DECaLS DR10 and VIKING DR4 ESO surveys were used to estimate photometric masses. We found that passive galaxies dominate the population of AC114 in the cluster center, whereas the star-forming members tend to be located outside this region. The constructed MZR from the latter indicates that AC114 SF galaxies have a lower metallicity content than foreground galaxies (z up to 0.28) and the same or even lower metallicities with respect to background galaxies (spanning redshifts 0.34 to 0.70), producing slightly a shallower slope compared to the field MZR at z~0.3. Additionally, it shows a higher scatter of 0.17 dex, consistent with MZRs of galaxy clusters reported in the literature. The responsible mechanisms behind the scatter are consistent with simulations and observations where dynamic and environmental effects such as galaxy-galaxy interactions and ram-pressure stripping are shaping the chemical evolution of the AC114 SF members. The former mechanism can decrease gas-phase metallicities because of pristine gas infall to the galaxy's outskirts, and the latter increases gas-phase metallicities because of the stripping of the metal-poor gas located at the outskirts of the SF members. The AC114 SF members are metal-poorer than field galaxies at the same redshift, so the cluster dominated by galaxy-galaxy interactions is the best scenario that explains the slightly shallower slope and the higher scatter of the AC114-MZR compared to the field MZR at z~0.3.


title: On the potential of Light-Cone Sum Rules without semi-global Quark-Hadron Duality

abstract: This work addresses the calculation of local form factors involved in the theoretical predictions of semileptonic $B$-meson decays at low-$q^2$. We present a new approach to the method of QCD Light-Cone Sum Rule with $B-$meson Light-Cone Distribution Amplitudes. In our strategy, we bypass the semi-global quark-hadron duality (QHD) approximation which usually contributes an unknown and potentially large systematic error to the prediction of form factors. We trade this improvement for an increased reliance on higher-order contributions in perturbation theory. Unlike the systematic error from semi-global QHD, truncation errors are assessable and systematically improvable, hence allowing robust predictions of form factors. For the transitions $B\to \pi,\rho,K^{(*)}$, our predictions agree with the literature for all form factors at $q^2=0$.


title: Evaluating Text-to-Visual Generation with Image-to-Text Generation

abstract: Despite significant progress in generative AI, comprehensive evaluation remains challenging because of the lack of effective metrics and standardized benchmarks. For instance, the widely-used CLIPScore measures the alignment between a (generated) image and text prompt, but it fails to produce reliable scores for complex prompts involving compositions of objects, attributes, and relations. One reason is that text encoders of CLIP can notoriously act as a "bag of words", conflating prompts such as "the horse is eating the grass" with "the grass is eating the horse". To address this, we introduce the VQAScore, which uses a visual-question-answering (VQA) model to produce an alignment score by computing the probability of a "Yes" answer to a simple "Does this figure show '{text}'?" question. Though simpler than prior art, VQAScore computed with off-the-shelf models produces state-of-the-art results across many (8) image-text alignment benchmarks. We also compute VQAScore with an in-house model that follows best practices in the literature. For example, we use a bidirectional image-question encoder that allows image embeddings to depend on the question being asked (and vice versa). Our in-house model, CLIP-FlanT5, outperforms even the strongest baselines that make use of the proprietary GPT-4V. Interestingly, although we train with only images, VQAScore can also align text with video and 3D models. VQAScore allows researchers to benchmark text-to-visual generation using complex texts that capture the compositional structure of real-world prompts. We introduce GenAI-Bench, a more challenging benchmark with 1,600 compositional text prompts that require parsing scenes, objects, attributes, relationships, and high-order reasoning like comparison and logic. GenAI-Bench also offers over 15,000 human ratings for leading image and video generation models such as Stable Diffusion, DALL-E 3, and Gen2.


title: Measuring Style Similarity in Diffusion Models

abstract: Generative models are now widely used by graphic designers and artists. Prior works have shown that these models remember and often replicate content from their training data during generation. Hence as their proliferation increases, it has become important to perform a database search to determine whether the properties of the image are attributable to specific training data, every time before a generated image is used for professional purposes. Existing tools for this purpose focus on retrieving images of similar semantic content. Meanwhile, many artists are concerned with style replication in text-to-image models. We present a framework for understanding and extracting style descriptors from images. Our framework comprises a new dataset curated using the insight that style is a subjective property of an image that captures complex yet meaningful interactions of factors including but not limited to colors, textures, shapes, etc. We also propose a method to extract style descriptors that can be used to attribute style of a generated image to the images used in the training dataset of a text-to-image model. We showcase promising results in various style retrieval tasks. We also quantitatively and qualitatively analyze style attribution and matching in the Stable Diffusion model. Code and artifacts are available at https://github.com/learn2phoenix/CSD.


title: Growth of regular partitions 2: Weak regularity

abstract: This is Part 2 in a series of papers about the growth of regular partitions in hereditary properties $3$-uniform hypergraphs. The focus of this paper is the notion of weak hypergraph regularity, first developed by Chung, Chung-Graham, and Haviland-Thomason. Given a hereditary property of $3$-uniform hypergraphs $\mathcal{H}$, we define a function $M_{\mathcal{H}}:(0,1)\rightarrow \mathbb{N}$ by letting $M_{\mathcal{H}}(\epsilon)$ be the smallest integer $M$ such that all sufficiently large elements of $\mathcal{H}$ admit weak regular partitions of size at most $M$. We show the asymptotic growth rate of such a function falls into one of four categories: constant, polynomial, between single and double exponentials, or tower. These results are a crucial component in Part 3 of the series, which considers vertex partitions associated to a stronger notion of hypergraph regularity.


title: CosmicMan: A Text-to-Image Foundation Model for Humans

abstract: We present CosmicMan, a text-to-image foundation model specialized for generating high-fidelity human images. Unlike current general-purpose foundation models that are stuck in the dilemma of inferior quality and text-image misalignment for humans, CosmicMan enables generating photo-realistic human images with meticulous appearance, reasonable structure, and precise text-image alignment with detailed dense descriptions. At the heart of CosmicMan's success are the new reflections and perspectives on data and models: (1) We found that data quality and a scalable data production flow are essential for the final results from trained models. Hence, we propose a new data production paradigm, Annotate Anyone, which serves as a perpetual data flywheel to produce high-quality data with accurate yet cost-effective annotations over time. Based on this, we constructed a large-scale dataset, CosmicMan-HQ 1.0, with 6 Million high-quality real-world human images in a mean resolution of 1488x1255, and attached with precise text annotations deriving from 115 Million attributes in diverse granularities. (2) We argue that a text-to-image foundation model specialized for humans must be pragmatic -- easy to integrate into down-streaming tasks while effective in producing high-quality human images. Hence, we propose to model the relationship between dense text descriptions and image pixels in a decomposed manner, and present Decomposed-Attention-Refocusing (Daring) training framework. It seamlessly decomposes the cross-attention features in existing text-to-image diffusion model, and enforces attention refocusing without adding extra modules. Through Daring, we show that explicitly discretizing continuous text space into several basic groups that align with human body structure is the key to tackling the misalignment problem in a breeze.


title: Towards Safety and Helpfulness Balanced Responses via Controllable Large Language Models

abstract: As large language models (LLMs) become easily accessible nowadays, the trade-off between safety and helpfulness can significantly impact user experience. A model that prioritizes safety will cause users to feel less engaged and assisted while prioritizing helpfulness will potentially cause harm. Possible harms include teaching people how to build a bomb, exposing youth to inappropriate content, and hurting users' mental health. In this work, we propose to balance safety and helpfulness in diverse use cases by controlling both attributes in LLM. We explore training-free and fine-tuning methods that do not require extra human annotations and analyze the challenges of controlling safety and helpfulness in LLMs. Our experiments demonstrate that our method can rewind a learned model and unlock its controllability.


title: MagicMirror: Fast and High-Quality Avatar Generation with a Constrained Search Space

abstract: We introduce a novel framework for 3D human avatar generation and personalization, leveraging text prompts to enhance user engagement and customization. Central to our approach are key innovations aimed at overcoming the challenges in photo-realistic avatar synthesis. Firstly, we utilize a conditional Neural Radiance Fields (NeRF) model, trained on a large-scale unannotated multi-view dataset, to create a versatile initial solution space that accelerates and diversifies avatar generation. Secondly, we develop a geometric prior, leveraging the capabilities of Text-to-Image Diffusion Models, to ensure superior view invariance and enable direct optimization of avatar geometry. These foundational ideas are complemented by our optimization pipeline built on Variational Score Distillation (VSD), which mitigates texture loss and over-saturation issues. As supported by our extensive experiments, these strategies collectively enable the creation of custom avatars with unparalleled visual quality and better adherence to input text prompts. You can find more results and videos in our website: https://syntec-research.github.io/MagicMirror


title: Streaming Dense Video Captioning

abstract: An ideal model for dense video captioning -- predicting captions localized temporally in a video -- should be able to handle long input videos, predict rich, detailed textual descriptions, and be able to produce outputs before processing the entire video. Current state-of-the-art models, however, process a fixed number of downsampled frames, and make a single full prediction after seeing the whole video. We propose a streaming dense video captioning model that consists of two novel components: First, we propose a new memory module, based on clustering incoming tokens, which can handle arbitrarily long videos as the memory is of a fixed size. Second, we develop a streaming decoding algorithm that enables our model to make predictions before the entire video has been processed. Our model achieves this streaming ability, and significantly improves the state-of-the-art on three dense video captioning benchmarks: ActivityNet, YouCook2 and ViTT. Our code is released at https://github.com/google-research/scenic.


title: Noise2Image: Noise-Enabled Static Scene Recovery for Event Cameras

abstract: Event cameras capture changes of intensity over time as a stream of 'events' and generally cannot measure intensity itself; hence, they are only used for imaging dynamic scenes. However, fluctuations due to random photon arrival inevitably trigger noise events, even for static scenes. While previous efforts have been focused on filtering out these undesirable noise events to improve signal quality, we find that, in the photon-noise regime, these noise events are correlated with the static scene intensity. We analyze the noise event generation and model its relationship to illuminance. Based on this understanding, we propose a method, called Noise2Image, to leverage the illuminance-dependent noise characteristics to recover the static parts of a scene, which are otherwise invisible to event cameras. We experimentally collect a dataset of noise events on static scenes to train and validate Noise2Image. Our results show that Noise2Image can robustly recover intensity images solely from noise events, providing a novel approach for capturing static scenes in event cameras, without additional hardware.