title: Existence and asymptotic autonomous robustness of random attractors for three-dimensional stochastic globally modified Navier-Stokes equations on unbounded domains

abstract: In this article, we discuss the existence and asymptotically autonomous robustness (AAR) (almost surely) of random attractors for 3D stochastic globally modified Navier-Stokes equations (SGMNSE) on Poincar\'e domains (which may be bounded or unbounded). Our aim is to investigate the existence and AAR of random attractors for 3D SGMNSE when the time-dependent forcing converges to a time-independent function under the perturbation of linear multiplicative noise as well as additive noise. The main approach is to provide a way to justify that, on some uniformly tempered universe, the usual pullback asymptotic compactness of the solution operators is uniform across an infinite time-interval $(-\infty,\tau]$. The backward uniform ``tail-smallness'' and ``flattening-property'' of the solutions over $(-\infty,\tau]$ have been demonstrated to achieve this goal. To the best of our knowledge, this is the first attempt to establish the existence as well as AAR of random attractors for 3D SGMNSE on unbounded domains.


title: Noise-robust Speech Separation with Fast Generative Correction

abstract: Speech separation, the task of isolating multiple speech sources from a mixed audio signal, remains challenging in noisy environments. In this paper, we propose a generative correction method to enhance the output of a discriminative separator. By leveraging a generative corrector based on a diffusion model, we refine the separation process for single-channel mixture speech by removing noises and perceptually unnatural distortions. Furthermore, we optimize the generative model using a predictive loss to streamline the diffusion model's reverse process into a single step and rectify any associated errors by the reverse process. Our method achieves state-of-the-art performance on the in-domain Libri2Mix noisy dataset, and out-of-domain WSJ with a variety of noises, improving SI-SNR by 22-35% relative to SepFormer, demonstrating robustness and strong generalization capabilities.


title: Rayleigh surface waves of extremal elastic materials

abstract: Extremal elastic materials here refer to a specific class of elastic materials whose elastic matrices exhibit one or more zero eigenvalues, resulting in soft deformation modes that, in principle, cost no energy. They can be approximated through artificially designed solid microstructures. Extremal elastic materials have exotic bulk wave properties unavailable with conventional solids due to the soft modes, offering unprecedented opportunities for manipulating bulk waves, e.g., acting as phonon polarizers for elastic waves or invisibility cloaks for underwater acoustic waves. Despite their potential, Rayleigh surface waves, crucially linked to bulk wave behaviors of such extremal elastic materials, have largely remained unexplored so far. In this paper, we theoretically investigate the propagation of Rayleigh waves in extremal elastic materials based on continuum theory and verify our findings with designed microstructure metamaterials based on pantographic structures. Dispersion relations and polarizations of Rayleigh waves in extremal elastic materials are derived, and the impact of higher order gradient effects is also investigated by using strain gradient theory. This study provides a continuum model for exploring surface waves in extremal elastic materials and may stimulate applications of extremal elastic materials for controlling surface waves.


title: Reconfigurable Intelligent Surfaces in Dynamic Rich Scattering Environments: BiLSTM-Based Optimization for Accurate User Localization

abstract: The integration of reconfigurable intelligent surfaces (RIS) in wireless environments offers channel programmability and dynamic control over propagation channels, which is expected to play a crucial role in sixth generation (6G) networks. The majority of RIS-related research has focused on simpler, quasi-free-space conditions, where wireless channels are typically modeled analytically. However, many practical localization scenarios unfold in environments characterized by rich scattering that also change over time. These dynamic and complex conditions pose significant challenges in determining the optimal RIS configuration to maximize localization accuracy. In this paper, we present our approach to overcoming this challenge. This paper introduces a novel approach that leverages a bidirectional long-short term memory (biLSTM) network, trained with a simulator that accurately reflects wave physics, to capture the relationship between wireless channels and the RIS configuration under dynamic, rich-scattering conditions. We use this approach to optimize RIS configurations for enhanced user equipment (UE) localization, measured by mean squared error (MSE). Through extensive simulations, we demonstrate that our approach adapts RIS configurations to significantly improve localization accuracy in such dynamically changing rich scattering environments.


title: Convex ordering for stochastic control: the swing contracts case

abstract: We investigate propagation of convexity and convex ordering on a typical stochastic optimal control problem, namely the pricing of \q{\emph{Take-or-Pay}} swing option, a financial derivative product commonly traded on energy markets. The dynamics of the underlying asset is modelled by an \emph{ARCH} model with convex coefficients. We prove that the value function associated to the stochastic optimal control problem is a convex function of the underlying asset price. We also introduce a domination criterion offering insights into the monotonicity of the value function with respect to parameters of the underlying \emph{ARCH} coefficients. We particularly focus on the one-dimensional setting where, by means of Stein's formula and regularization techniques, we show that the convexity assumption for the \emph{ARCH} coefficients can be relaxed with a semi-convexity assumption. To validate the results presented in this paper, we also conduct numerical illustrations.


title: Microbiomes Through The Looking Glass

abstract: Bacterial communities are pivotal to maintaining ecological function and preserving the rich tapestry of biological diversity. The rapid development of environmental sequencing technologies, such as metagenomics, has revolutionized our capacity to probe such diversity. However, despite these advances, a theoretical understanding connecting empirical data with ecosystem modelling, in particular in the framework of disordered systems akin to spin glasses, is still in its infancy. Here, we present a comprehensive framework using theories of disordered systems to decode microbiome data, which offers insight into the ecological forces that shape macroecological states. By employing the quenched disordered generalized Lotka-Volterra model, we analyze species abundance data in healthy and diseased human gut microbiomes. Results reveal the emergence of two distinct patterns of species-interaction networks, elucidating the pathways through which dysbiosis may drive microbiome instability. Interaction patterns thus provide a window into the systemic shifts accompanying the transition from health to disease, offering a new perspective on the dynamics of the microbial community. Our findings suggest the potential of disordered systems theory to characterize microbiomes by capturing the essence of ecological interactions and their consequences on stability and functioning, leveraging our understanding of the linkages of dysbiosis and microbial dynamics.


title: Multimodal Belief Prediction

abstract: Recognizing a speaker's level of commitment to a belief is a difficult task; humans do not only interpret the meaning of the words in context, but also understand cues from intonation and other aspects of the audio signal. Many papers and corpora in the NLP community have approached the belief prediction task using text-only approaches. We are the first to frame and present results on the multimodal belief prediction task. We use the CB-Prosody corpus (CBP), containing aligned text and audio with speaker belief annotations. We first report baselines and significant features using acoustic-prosodic features and traditional machine learning methods. We then present text and audio baselines for the CBP corpus fine-tuning on BERT and Whisper respectively. Finally, we present our multimodal architecture which fine-tunes on BERT and Whisper and uses multiple fusion methods, improving on both modalities alone.


title: Anomaly Detection on Unstable Logs with GPT Models

abstract: Log-based anomaly detection has been widely studied in the literature as a way to increase the dependability of software-intensive systems. In reality, logs can be unstable due to changes made to the software during its evolution. This, in turn, degrades the performance of downstream log analysis activities, such as anomaly detection. The critical challenge in detecting anomalies on these unstable logs is the lack of information about the new logs, due to insufficient log data from new software versions. The application of Large Language Models (LLMs) to many software engineering tasks has revolutionized various domains. In this paper, we report on an experimental comparison of a fine-tuned LLM and alternative models for anomaly detection on unstable logs. The main motivation is that the pre-training of LLMs on vast datasets may enable a robust understanding of diverse patterns and contextual information, which can be leveraged to mitigate the data insufficiency issue in the context of software evolution. Our experimental results on the two-version dataset of LOGEVOL-Hadoop show that the fine-tuned LLM (GPT-3) fares slightly better than supervised baselines when evaluated on unstable logs. The difference between GPT-3 and other supervised approaches tends to become more significant as the degree of changes in log sequences increases. However, it is unclear whether the difference is practically significant in all cases. Lastly, our comparison of prompt engineering (with GPT-4) and fine-tuning reveals that the latter provides significantly superior performance on both stable and unstable logs, offering valuable insights into the effective utilization of LLMs in this domain.


title: On functions of low differential uniformity in characteristic 2: A close look (I)

abstract: We introduce a new concept, the APN-defect, which can be thought of as measuring the distance of a given function $G:\mathbb{F}_{2^n} \rightarrow \mathbb{F}_{2^n}$ to the set of almost perfect nonlinear (APN) functions. This concept is motivated by the detailed analysis of the differential behaviour of non-APN functions (of low differential uniformity) $G$ using the so-called difference squares. We describe the relations between the APN-defect and other recent concepts of similar nature. Upper and lower bounds for the values of APN-defect for several classes of functions of interest, including Dembowski-Ostrom polynomials are given. Its exact values in some cases are also calculated. The difference square corresponding to a modification of the inverse function is determined, its APN-defect depending on $n$ is evaluated and the implications are discussed. In the forthcoming second part of this work we further examine modifications of the inverse function. We also study modifications of classes of functions of low uniformity over infinitely many extensions of $\mathbb{F}_{2^n}$. We present quantitative results on their differential behaviour, especially in connection with their APN-defects.


title: Vanishing properties of Kloosterman sums and Dyson's conjectures

abstract: In a previous paper arXiv:2406.06294 [math.NT], the author proved the exact formulae for ranks of partitions modulo each prime $p\geq 5$. In this paper, for $p=5$ and $7$, we prove special vanishing properties of the Kloosterman sums appearing in the exact formulae. These vanishing properties imply a new proof of Dyson's rank conjectures. Specifically, we give a new proof of Ramanujan's congruences $p(5n+4)\equiv 0\pmod 5$ and $p(7n+5)\equiv 0\pmod 7$.


title: Exploring non-radial oscillation modes in dark matter admixed neutron stars

abstract: Because of their extreme densities and consequently, gravitational potential, compact objects such as neutron stars can prove to be excellent captors of dark matter particles. Considering purely gravitational interactions between dark and hadronic matter, we construct dark matter admixed stars composed of two-fluid matter subject to current astrophysical constraints of maximum mass and tidal deformability. We choose a wide range of parameters to construct the dark matter equation of state, and the DDME2 parameterization for the hadronic equation of state. We then examine the effect of dark matter on the stellar structure, tidal deformability and non-radial modes considering the relativistic Cowling approximation. We find the effect on $p$-modes is substantial, with frequencies decreasing up to the typical $f-$mode frequency range for most stars with a dark matter halo. The effects on the $f-$mode frequency are less extreme. Finally, we find the most probable and $1\sigma$ values of the dark matter parameters used in this study.


title: OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding

abstract: Surgical scene perception via videos are critical for advancing robotic surgery, telesurgery, and AI-assisted surgery, particularly in ophthalmology. However, the scarcity of diverse and richly annotated video datasets has hindered the development of intelligent systems for surgical workflow analysis. Existing datasets for surgical workflow analysis, which typically face challenges such as small scale, a lack of diversity in surgery and phase categories, and the absence of time-localized annotations, limit the requirements for action understanding and model generalization validation in complex and diverse real-world surgical scenarios. To address this gap, we introduce OphNet, a large-scale, expert-annotated video benchmark for ophthalmic surgical workflow understanding. OphNet features: 1) A diverse collection of 2,278 surgical videos spanning 66 types of cataract, glaucoma, and corneal surgeries, with detailed annotations for 102 unique surgical phases and 150 granular operations; 2) It offers sequential and hierarchical annotations for each surgery, phase, and operation, enabling comprehensive understanding and improved interpretability; 3) Moreover, OphNet provides time-localized annotations, facilitating temporal localization and prediction tasks within surgical workflows. With approximately 205 hours of surgical videos, OphNet is about 20 times larger than the largest existing surgical workflow analysis benchmark. Our dataset and code have been made available at: \url{https://github.com/minghu0830/OphNet-benchmark}.


title: 4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models

abstract: Existing dynamic scene generation methods mostly rely on distilling knowledge from pre-trained 3D generative models, which are typically fine-tuned on synthetic object datasets. As a result, the generated scenes are often object-centric and lack photorealism. To address these limitations, we introduce a novel pipeline designed for photorealistic text-to-4D scene generation, discarding the dependency on multi-view generative models and instead fully utilizing video generative models trained on diverse real-world datasets. Our method begins by generating a reference video using the video generation model. We then learn the canonical 3D representation of the video using a freeze-time video, delicately generated from the reference video. To handle inconsistencies in the freeze-time video, we jointly learn a per-frame deformation to model these imperfections. We then learn the temporal deformation based on the canonical representation to capture dynamic interactions in the reference video. The pipeline facilitates the generation of dynamic scenes with enhanced photorealism and structural integrity, viewable from multiple perspectives, thereby setting a new standard in 4D scene generation.


title: Choreographing the Rhythms of Observation: Dynamics for Ranged Observer Bipartite-Unipartite SpatioTemporal (ROBUST) Networks

abstract: Existing network analysis methods struggle to optimize observer placements in dynamic environments with limited visibility. This dissertation introduces the novel ROBUST (Ranged Observer Bipartite-Unipartite SpatioTemporal) framework, offering a significant advancement in modeling, analyzing, and optimizing observer networks within complex spatiotemporal domains. ROBUST leverages a unique bipartite-unipartite approach, distinguishing between observer and observable entities while incorporating spatial constraints and temporal dynamics. This research extends spatiotemporal network theory by introducing novel graph-based measures, including myopic degree, spatial closeness centrality, and edge length proportion. These measures, coupled with advanced clustering techniques like Proximal Recurrence, provide insights into network structure, resilience, and the effectiveness of observer placements. The ROBUST framework demonstrates superior resource allocation and strategic responsiveness compared to conventional models. Case studies in oceanographic monitoring, urban safety networks, and multi-agent path planning showcases its practical applicability and adaptability. Results demonstrate significant improvements in coverage, response times, and overall network efficiency. This work paves the way for future research in incorporating imperfect knowledge, refining temporal pathing methodologies, and expanding the scope of applications. By bridging theoretical advancements with practical solutions, ROBUST stands as a significant contribution to the field, promising to inform and inspire ongoing and future endeavors in network optimization and multi-agent system planning.


title: Quantifying Local Model Validity using Active Learning

abstract: Real-world applications of machine learning models are often subject to legal or policy-based regulations. Some of these regulations require ensuring the validity of the model, i.e., the approximation error being smaller than a threshold. A global metric is generally too insensitive to determine the validity of a specific prediction, whereas evaluating local validity is costly since it requires gathering additional data.We propose learning the model error to acquire a local validity estimate while reducing the amount of required data through active learning. Using model validation benchmarks, we provide empirical evidence that the proposed method can lead to an error model with sufficient discriminative properties using a relatively small amount of data. Furthermore, an increased sensitivity to local changes of the validity bounds compared to alternative approaches is demonstrated.


title: Partially Observed Trajectory Inference using Optimal Transport and a Dynamics Prior

abstract: Trajectory inference seeks to recover the temporal dynamics of a population from snapshots of its (uncoupled) temporal marginals, i.e. where observed particles are not tracked over time. Lavenant et al. arXiv:2102.09204 addressed this challenging problem under a stochastic differential equation (SDE) model with a gradient-driven drift in the observed space, introducing a minimum entropy estimator relative to the Wiener measure. Chizat et al. arXiv:2205.07146 then provided a practical grid-free mean-field Langevin (MFL) algorithm using Schr\"odinger bridges. Motivated by the overwhelming success of observable state space models in the traditional paired trajectory inference problem (e.g. target tracking), we extend the above framework to a class of latent SDEs in the form of observable state space models. In this setting, we use partial observations to infer trajectories in the latent space under a specified dynamics model (e.g. the constant velocity/acceleration models from target tracking). We introduce PO-MFL to solve this latent trajectory inference problem and provide theoretical guarantees by extending the results of arXiv:2102.09204 to the partially observed setting. We leverage the MFL framework of arXiv:2205.07146, yielding an algorithm based on entropic OT between dynamics-adjusted adjacent time marginals. Experiments validate the robustness of our method and the exponential convergence of the MFL dynamics, and demonstrate significant outperformance over the latent-free method of arXiv:2205.07146 in key scenarios.


title: VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

abstract: In this paper, we present the VideoLLaMA 2, a set of Video Large Language Models (Video-LLMs) designed to enhance spatial-temporal modeling and audio understanding in video and audio-oriented tasks. Building upon its predecessor, VideoLLaMA 2 incorporates a tailor-made Spatial-Temporal Convolution (STC) connector, which effectively captures the intricate spatial and temporal dynamics of video data. Additionally, we integrate an Audio Branch into the model through joint training, thereby enriching the multimodal understanding capabilities of the model by seamlessly incorporating audio cues. Comprehensive evaluations on multiple-choice video question answering (MC-VQA), open-ended video question answering (OE-VQA), and video captioning (VC) tasks demonstrate that VideoLLaMA 2 consistently achieves competitive results among open-source models and even gets close to some proprietary models on several benchmarks. Furthermore, VideoLLaMA 2 exhibits reasonable improvements in audio-only and audio-video question-answering (AQA & OE-AVQA) benchmarks over existing models. These advancements underline VideoLLaMA 2's superior performance in multimodal comprehension, setting a new standard for intelligent video analysis systems. All models are public to facilitate further research.


title: Neutrino magnetic dipole portal with low energy neutrino nucleus scattering data

abstract: Sterile neutrinos that couple to the Standard Model via the neutrino magnetic dipole portals have been extensively studied at various experiments. In this work, we scrutinize these interactions for sterile neutrinos in the mass range of $\unit[0.1]{}-\unit[50]{MeV}$ through the nuclear and electron recoils at various neutrino scattering experiments. For the $e$-flavor specific dipole portal, we demonstrate that Dresden-II can provide leading constraints for $m_N \lesssim \unit[0.5]{MeV}$, setting aside currently unresolved theoretical uncertainties. For the $\mu$-flavor case, we show that the COHERENT experiment can probe a unique parameter region for $m_N$ in the range of $\unit[10]{}-\unit[40]{MeV}$ with the full dataset collected by the CsI[Na] scintillation detector, including both the energy and timing structure of the neutrino beam. We also present limits on the parameter regions of the $\tau$-flavor dipole portal using measurements of the solar neutrino flux from dark matter direct detection experiments.


title: Incompressibility and spectral gaps of random circuits

abstract: Random reversible and quantum circuits form random walks on the alternating group $\mathrm{Alt}(2^n)$ and unitary group $\mathrm{SU}(2^n)$, respectively. Known bounds on the spectral gap for the $t$-th moment of these random walks have inverse-polynomial dependence in both $n$ and $t$. We prove that the gap for random reversible circuits is $\Omega(n^{-3})$ for all $t\geq 1$, and the gap for random quantum circuits is $\Omega(n^{-3})$ for $t \leq \Theta(2^{n/2})$. These gaps are independent of $t$ in the respective regimes. We can further improve both gaps to $n^{-1}/\mathrm{polylog}(n, t)$ for $t\leq 2^{\Theta(n)}$, which is tight up to polylog factors. Our spectral gap results have a number of consequences: 1) Random reversible circuits with $\mathcal{O}(n^4 t)$ gates form multiplicative-error $t$-wise independent (even) permutations for all $t\geq 1$; for $t \leq \Theta(2^{n/6.1})$, we show that $\tilde{\mathcal{O}}(n^2 t)$ gates suffice. 2) Random quantum circuits with $\mathcal{O}(n^4 t)$ gates form multiplicative-error unitary $t$-designs for $t \leq \Theta(2^{n/2})$; for $t\leq \Theta(2^{2n/5})$, we show that $\tilde{\mathcal{O}}(n^2t)$ gates suffice. 3) The robust quantum circuit complexity of random circuits grows linearly for an exponentially long time, proving the robust Brown--Susskind conjecture [BS18,BCHJ+21]. Our spectral gap bounds are proven by reducing random quantum circuits to a more structured walk: a modification of the ``$\mathrm{PFC}$ ensemble'' from [MPSY24] together with an expander on the alternating group due to Kassabov [Kas07a], for which we give an efficient implementation using reversible circuits. In our reduction, we approximate the structured walk with local random circuits without losing the gap, which uses tools from the study of frustration-free Hamiltonians.


title: Lower bounds for sphere packing in arbitrary norms

abstract: We show that in any $d$-dimensional real normed space, unit balls can be packed with density at least \[\frac{(1-o(1))d\log d}{2^{d+1}},\] improving a result of Schmidt from 1958 by a logarithmic factor and generalizing the recent result of Campos, Jenssen, Michelen, and Sahasrabudhe in the $\ell_2$ norm. Our main tools are the graph-theoretic result used in the $\ell_2$ construction and recent progress on the Bourgain slicing problem.


title: Image Neural Field Diffusion Models

abstract: Diffusion models have shown an impressive ability to model complex data distributions, with several key advantages over GANs, such as stable training, better coverage of the training distribution's modes, and the ability to solve inverse problems without extra training. However, most diffusion models learn the distribution of fixed-resolution images. We propose to learn the distribution of continuous images by training diffusion models on image neural fields, which can be rendered at any resolution, and show its advantages over fixed-resolution models. To achieve this, a key challenge is to obtain a latent space that represents photorealistic image neural fields. We propose a simple and effective method, inspired by several recent techniques but with key changes to make the image neural fields photorealistic. Our method can be used to convert existing latent diffusion autoencoders into image neural field autoencoders. We show that image neural field diffusion models can be trained using mixed-resolution image datasets, outperform fixed-resolution diffusion models followed by super-resolution models, and can solve inverse problems with conditions applied at different scales efficiently.


title: The end of multiple choice tests: using AI to enhance assessment

abstract: Effective teaching relies on knowing what students know-or think they know. Revealing student thinking is challenging. Often used because of their ease of grading, even the best multiple choice (MC) tests, those using research based distractors (wrong answers) are intrinsically limited in the insights they provide due to two factors. When distractors do not reflect student beliefs they can be ignored, increasing the likelihood that the correct answer will be chosen by chance. Moreover, making the correct choice does not guarantee that the student understands why it is correct. To address these limitations, we recommend asking students to explain why they chose their answer, and why "wrong" choices are wrong. Using a discipline-trained artificial intelligence-based bot it is possible to analyze their explanations, identifying the concepts and scientific principles that maybe missing or misapplied. The bot also makes suggestions for how instructors can use these data to better guide student thinking. In a small "proof of concept" study, we tested this approach using questions from the Biology Concepts Instrument (BCI). The result was rapid, informative, and provided actionable feedback on student thinking. It appears that the use of AI addresses the weaknesses of conventional MC test. It seems likely that incorporating AI-analyzed formative assessments will lead to improved overall learning outcomes.


title: Comparing Deep Learning Models for Rice Mapping in Bhutan Using High Resolution Satellite Imagery

abstract: The Bhutanese government is increasing its utilization of technological approaches such as including Remote Sensing-based knowledge in their decision-making process. This study focuses on crop type and crop extent in Paro, one of the top rice-yielding districts in Bhutan, and employs publicly available NICFI high-resolution satellite imagery from Planet. Two Deep Learning (DL) approaches, point-based (DNN) and patch-based (U-Net), models were used in conjunction with cloud-computing platforms. Three different models per DL approaches (DNN and U-Net) were trained: 1) RGBN channels from Planet; 2) RGBN and elevation data (RGBNE); 3) RGBN and Sentinel-1 (S1) data (RGBNS), and RGBN with E and S1 data (RGBNES). From this comprehensive analysis, the U-Net displayed higher performance metrics across both model training and model validation efforts. Among the U-Net model sets, the RGBN, RGBNE, RGBNS, and RGBNES models had an F1-score of 0.8546, 0.8563, 0.8467, and 0.8500 respectively. An independent model evaluation was performed and found a high level of performance variation across all the metrics. For this independent model evaluation, the U-Net RGBN, RGBNE, RGBNES, and RGBN models displayed the F1-scores of 0.5935, 0.6154, 0.5882, and 0.6582, suggesting U-Net RGBNES as the best model. The study shows that the DL approaches can predict rice. Also, DL methods can be used with the survey-based approaches currently utilized by the Bhutan Department of Agriculture. Further, this study demonstrated the usage of regional land cover products such as SERVIR's RLCMS as a weak label approach to capture different strata addressing the class imbalance problem and improving the sampling design for DL application. Finally, through preliminary model testing and comparisons outlined it was shown that using additional features such as NDVI, EVI, and NDWI did not drastically improve model performance.


title: Advancing Annotation of Stance in Social Media Posts: A Comparative Analysis of Large Language Models and Crowd Sourcing

abstract: In the rapidly evolving landscape of Natural Language Processing (NLP), the use of Large Language Models (LLMs) for automated text annotation in social media posts has garnered significant interest. Despite the impressive innovations in developing LLMs like ChatGPT, their efficacy, and accuracy as annotation tools are not well understood. In this paper, we analyze the performance of eight open-source and proprietary LLMs for annotating the stance expressed in social media posts, benchmarking their performance against human annotators' (i.e., crowd-sourced) judgments. Additionally, we investigate the conditions under which LLMs are likely to disagree with human judgment. A significant finding of our study is that the explicitness of text expressing a stance plays a critical role in how faithfully LLMs' stance judgments match humans'. We argue that LLMs perform well when human annotators do, and when LLMs fail, it often corresponds to situations in which human annotators struggle to reach an agreement. We conclude with recommendations for a comprehensive approach that combines the precision of human expertise with the scalability of LLM predictions. This study highlights the importance of improving the accuracy and comprehensiveness of automated stance detection, aiming to advance these technologies for more efficient and unbiased analysis of social media.


title: Towards Generalized Hydrological Forecasting using Transformer Models for 120-Hour Streamflow Prediction

abstract: This study explores the efficacy of a Transformer model for 120-hour streamflow prediction across 125 diverse locations in Iowa, US. Utilizing data from the preceding 72 hours, including precipitation, evapotranspiration, and discharge values, we developed a generalized model to predict future streamflow. Our approach contrasts with traditional methods that typically rely on location-specific models. We benchmarked the Transformer model's performance against three deep learning models (LSTM, GRU, and Seq2Seq) and the Persistence approach, employing Nash-Sutcliffe Efficiency (NSE), Kling-Gupta Efficiency (KGE), Pearson's r, and Normalized Root Mean Square Error (NRMSE) as metrics. The study reveals the Transformer model's superior performance, maintaining higher median NSE and KGE scores and exhibiting the lowest NRMSE values. This indicates its capability to accurately simulate and predict streamflow, adapting effectively to varying hydrological conditions and geographical variances. Our findings underscore the Transformer model's potential as an advanced tool in hydrological modeling, offering significant improvements over traditional and contemporary approaches.


title: PITCH: Productivity and Mental Well-being Coaching through Daily Conversational Interaction

abstract: Efficient task planning is essential for productivity and mental well-being, yet individuals often struggle to create realistic plans and reflect upon their productivity. Leveraging the advancement in artificial intelligence (AI), conversational agents have emerged as a promising tool for enhancing productivity. Our work focuses on externalizing plans through conversation, aiming to solidify intentions and foster focused action, thereby positively impacting their productivity and mental well-being. We share our plan of designing a conversational agent to offer insightful questions and reflective prompts for increasing plan adherence by leveraging the social interactivity of natural conversations. Previous studies have shown the effectiveness of such agents, but many interventions remain static, leading to decreased user engagement over time. To address this limitation, we propose a novel rotation and context-aware prompting strategy, providing users with varied interventions daily. Our system, PITCH, utilizes large language models (LLMs) to facilitate externalization and reflection on daily plans. Through this study, we investigate the impact of externalizing tasks with conversational agents on productivity and mental well-being, and the effectiveness of a rotation strategy in maintaining user engagement.


title: Novel Optimized Designs of Modulo $2n+1$ Adder for Quantum Computing

abstract: Quantum modular adders are one of the most fundamental yet versatile quantum computation operations. They help implement functions of higher complexity, such as subtraction and multiplication, which are used in applications such as quantum cryptanalysis, quantum image processing, and securing communication. To the best of our knowledge, there is no existing design of quantum modulo $(2n+1)$ adder. In this work, we propose four quantum adders targeted specifically for modulo $(2n+1)$ addition. These adders can provide both regular and modulo $(2n+1)$ sum concurrently, enhancing their application in residue number system based arithmetic. Our first design, QMA1, is a novel quantum modulo $(2n+1)$ adder. The second proposed adder, QMA2, optimizes the utilization of quantum gates within the QMA1, resulting in 37.5% reduced CNOT gate count, 46.15% reduced CNOT depth, and 26.5% decrease in both Toffoli gates and depth. We propose a third adder QMA3 that uses zero resets, a dynamic circuits based feature that reuses qubits, leading to 25% savings in qubit count. Our fourth design, QMA4, demonstrates the benefit of incorporating additional zero resets to achieve a purer zero state, reducing quantum state preparation errors. Notably, we conducted experiments using 5-qubit configurations of the proposed modulo $(2n+1)$ adders on the IBM Washington, a 127-qubit quantum computer based on the Eagle R1 architecture, to demonstrate a 28.8% reduction in QMA1's error of which: (i) 18.63% error reduction happens due to gate and depth reduction in QMA2, and (ii) 2.53% drop in error due to qubit reduction in QMA3, and (iii) 7.64% error decreased due to application of additional zero resets in QMA4.


title: GLAD: Towards Better Reconstruction with Global and Local Adaptive Diffusion Models for Unsupervised Anomaly Detection

abstract: Diffusion models have shown superior performance on unsupervised anomaly detection tasks. Since trained with normal data only, diffusion models tend to reconstruct normal counterparts of test images with certain noises added. However, these methods treat all potential anomalies equally, which may cause two main problems. From the global perspective, the difficulty of reconstructing images with different anomalies is uneven. Therefore, instead of utilizing the same setting for all samples, we propose to predict a particular denoising step for each sample by evaluating the difference between image contents and the priors extracted from diffusion models. From the local perspective, reconstructing abnormal regions differs from normal areas even in the same image. Theoretically, the diffusion model predicts a noise for each step, typically following a standard Gaussian distribution. However, due to the difference between the anomaly and its potential normal counterpart, the predicted noise in abnormal regions will inevitably deviate from the standard Gaussian distribution. To this end, we propose introducing synthetic abnormal samples in training to encourage the diffusion models to break through the limitation of standard Gaussian distribution, and a spatial-adaptive feature fusion scheme is utilized during inference. With the above modifications, we propose a global and local adaptive diffusion model (abbreviated to GLAD) for unsupervised anomaly detection, which introduces appealing flexibility and achieves anomaly-free reconstruction while retaining as much normal information as possible. Extensive experiments are conducted on three commonly used anomaly detection datasets (MVTec-AD, MPDD, and VisA) and a printed circuit board dataset (PCB-Bank) we integrated, showing the effectiveness of the proposed method.


title: ReduceFormer: Attention with Tensor Reduction by Summation

abstract: Transformers have excelled in many tasks including vision. However, efficient deployment of transformer models in low-latency or high-throughput applications is hindered by the computation in the attention mechanism which involves expensive operations such as matrix multiplication and Softmax. To address this, we introduce ReduceFormer, a family of models optimized for efficiency with the spirit of attention. ReduceFormer leverages only simple operations such as reduction and element-wise multiplication, leading to greatly simplified architecture and improved inference performance, with up to 37% reduction in latency and 44% improvement in throughput, while maintaining competitive accuracy comparable to other recent methods. The proposed model family is suitable for edge devices where compute resource and memory bandwidth are limited, as well as for cloud computing where high throughput is sought after.


title: Prospects for the detection of Dark Matter with Long-lived Mediators in the Sun using the Southern Wide-field Gamma-ray Observatory

abstract: The operation of the next generation of gamma-ray observatories will lead to a great advance in dark matter searches. In this paper, we use the hidden sectors hypothesis within the so-called secluded models to calculate the capabilities of the Southern Wide-field Gamma-ray Observatory (SWGO) to detect gamma-ray signatures produced by dark matter particles concentrated in the Sun. We assume the dark matter particle annihilates into metastable mediators which decay into $\gamma\gamma$, $e^+e^-$, $\tau^+\tau^-$, and $\bar{b}b$ outside the Sun. We found that the SWGO will be able to probe a spin-dependent cross-section of about $10^{-46}$ cm$^2$ for dark matter masses smaller than 5 TeV. This result shows an unprecedented sensitivity surpassing the current instruments by more than one order of magnitude.


title: Transition from decaying to decayless kink oscillations of solar coronal loops

abstract: The transition of an impulsively excited kink oscillation of a solar coronal loop to an oscillation with a stationary amplitude, i.e., the damping pattern, is determined using the low-dimensional self-oscillation model. In the model, the decayless kink oscillations are sustained by the interaction of the oscillating loop with an external quasi-steady flow. The analytical solution is based on the assumption that the combined effect of the effective dissipation, for example, by resonant absorption, and interaction with an external flow, is weak. The effect is characterised by a dimensionless coupling parameter. The damping pattern is found to depend upon the initial amplitude and the coupling parameter. The approximate expression shows a good agreement with a numerical solution of the self-oscillation equation. The plausibility of the established damping pattern is demonstrated by an observational example. Notably, the damping pattern is not exponential, and the characteristic decay time is different from the time determined by the traditionally used exponential damping fit. Implications of this finding for seismology of the solar coronal plasmas are discussed. In particular, it is suggested that a very rapid, in less than the oscillation period, decay of the oscillation to the stationary level, achieved for larger values of the coupling parameter, can explain the relative rareness of the kink oscillation events.


title: Spread Complexity of High Energy Neutrino Propagation over Astrophysical Distances

abstract: Spread complexity measures the minimized spread of quantum states over all choices of basis. It generalizes Krylov operator complexity to quantum states under continuous Hamiltonian evolution. In this paper, we study spread complexity in the context of high-energy astrophysical neutrinos and propose a new flavor ratio based on complexity. Our findings indicate that our proposal might favor an initial ratio of fluxes as $\phi_{\nu_e}^0: \phi_{\nu_\mu}^0: \phi_{\nu_\tau}^0 = 1:0:0$ over a more generally expected ratio of $1:2:0$, when the IceCube neutrino observatory achieves its projected sensitivity to discriminate between flavors. Additionally, complexity-based definitions of flavor ratios exhibit a slight but nonzero sensitivity to the neutrino mass ordering, which traditional flavor ratios cannot capture.


title: Paraphrasing in Affirmative Terms Improves Negation Understanding

abstract: Negation is a common linguistic phenomenon. Yet language models face challenges with negation in many natural language understanding tasks such as question answering and natural language inference. In this paper, we experiment with seamless strategies that incorporate affirmative interpretations (i.e., paraphrases without negation) to make models more robust against negation. Crucially, our affirmative interpretations are obtained automatically. We show improvements with CondaQA, a large corpus requiring reasoning with negation, and five natural language understanding tasks.


title: A model-independent test of pre-recombination New Physics: Machine Learning based estimate of the Sound Horizon from Gravitational Wave Standard Sirens and the Baryon Acoustic Oscillation Angular Scale

abstract: In any cosmological model where spacetime is described by a pseudo-Riemannian manifold, photons propagate along null geodesics, and their number is conserved, upcoming Gravitational Wave (GW) observations can be combined with measurements of the Baryon Acoustic Oscillation (BAO) angular scale to provide model-independent estimates of the sound horizon at the baryon-drag epoch. By focusing on the accuracy expected from forthcoming surveys such as LISA GW standard sirens and DESI or Euclid angular BAO measurements, we forecast a relative precision of $\sigma_{r_{\rm d}} /r_{\rm d} \sim 1.5\%$ within the redshift range $z \lesssim 1$. This approach will offer a unique model-independent measure of a fundamental scale characterizing the early universe, which is competitive with model-dependent values inferred within specific theoretical frameworks. These measurements can serve as a consistency test for $\Lambda$CDM, potentially clarifying the nature of the Hubble tension and confirming or ruling out new physics prior to recombination with a statistical significance of $\sim 4\sigma$.


title: CADS: A Systematic Literature Review on the Challenges of Abstractive Dialogue Summarization

abstract: Abstractive dialogue summarization is the task of distilling conversations into informative and concise summaries. Although reviews have been conducted on this topic, there is a lack of comprehensive work detailing the challenges of dialogue summarization, unifying the differing understanding of the task, and aligning proposed techniques, datasets, and evaluation metrics with the challenges. This article summarizes the research on Transformer-based abstractive summarization for English dialogues by systematically reviewing 1262 unique research papers published between 2019 and 2024, relying on the Semantic Scholar and DBLP databases. We cover the main challenges present in dialog summarization (i.e., language, structure, comprehension, speaker, salience, and factuality) and link them to corresponding techniques such as graph-based approaches, additional training tasks, and planning strategies, which typically overly rely on BART-based encoder-decoder models. We find that while some challenges, like language, have seen considerable progress, mainly due to training methods, others, such as comprehension, factuality, and salience, remain difficult and hold significant research opportunities. We investigate how these approaches are typically assessed, covering the datasets for the subdomains of dialogue (e.g., meeting, medical), the established automatic metrics and human evaluation approaches for assessing scores and annotator agreement. We observe that only a few datasets span across all subdomains. The ROUGE metric is the most used, while human evaluation is frequently reported without sufficient detail on inner-annotator agreement and annotation guidelines. Additionally, we discuss the possible implications of the recently explored large language models and conclude that despite a potential shift in relevance and difficulty, our described challenge taxonomy remains relevant.


title: Some real Rel trajectories in $\mathcal{H}(1,1)$ that are not recurrent

abstract: We study the real Rel orbits of some translation surfaces in the stratum $\mathcal{H}(1,1)$. Specifically, surfaces that are tremors of the locus of branched double covers of tori. We give necessary and sufficient conditions on tremors of a surface so that the real Rel orbit is recurrent. As a consequence, we are able to provide explicit examples of trajectories of real Rel that are not recurrent.


title: TextGrad: Automatic "Differentiation" via Text

abstract: AI is undergoing a paradigm shift, with breakthroughs achieved by systems orchestrating multiple large language models (LLMs) and other complex components. As a result, developing principled and automated optimization methods for compound AI systems is one of the most important new challenges. Neural networks faced a similar challenge in its early days until backpropagation and automatic differentiation transformed the field by making optimization turn-key. Inspired by this, we introduce TextGrad, a powerful framework performing automatic ``differentiation'' via text. TextGrad backpropagates textual feedback provided by LLMs to improve individual components of a compound AI system. In our framework, LLMs provide rich, general, natural language suggestions to optimize variables in computation graphs, ranging from code snippets to molecular structures. TextGrad follows PyTorch's syntax and abstraction and is flexible and easy-to-use. It works out-of-the-box for a variety of tasks, where the users only provide the objective function without tuning components or prompts of the framework. We showcase TextGrad's effectiveness and generality across a diverse range of applications, from question answering and molecule optimization to radiotherapy treatment planning. Without modifying the framework, TextGrad improves the zero-shot accuracy of GPT-4o in Google-Proof Question Answering from $51\%$ to $55\%$, yields $20\%$ relative performance gain in optimizing LeetCode-Hard coding problem solutions, improves prompts for reasoning, designs new druglike small molecules with desirable in silico binding, and designs radiation oncology treatment plans with high specificity. TextGrad lays a foundation to accelerate the development of the next-generation of AI systems.


title: A pilot protocol and cohort for the investigation of non-pathological variability in speech

abstract: Background Speech-based biomarkers have potential as a means for regular, objective assessment of symptom severity, remotely and in-clinic in combination with advanced analytical models. However, the complex nature of speech and the often subtle changes associated with health mean that findings are highly dependent on methodological and cohort choices. These are often not reported adequately in studies investigating speech-based health assessment Objective To develop and apply an exemplar protocol to generate a pilot dataset of healthy speech with detailed metadata for the assessment of factors in the speech recording-analysis pipeline, including device choice, speech elicitation task and non-pathological variability. Methods We developed our collection protocol and choice of exemplar speech features based on a thematic literature review. Our protocol includes the elicitation of three different speech types. With a focus towards remote applications, we also choose to collect speech with three different microphone types. We developed a pipeline to extract a set of 14 exemplar speech features. Results We collected speech from 28 individuals three times in one day, repeated at the same times 8-11 weeks later, and from 25 healthy individuals three times in one week. Participant characteristics collected included sex, age, native language status and voice use habits of the participant. A preliminary set of 14 speech features covering timing, prosody, voice quality, articulation and spectral moment characteristics were extracted that provide a resource of normative values. Conclusions There are multiple methodological factors involved in the collection, processing and analysis of speech recordings. Consistent reporting and greater harmonisation of study protocols are urgently required to aid the translation of speech processing into clinical research and practice.


title: RaD-Net 2: A causal two-stage repairing and denoising speech enhancement network with knowledge distillation and complex axial self-attention

abstract: In real-time speech communication systems, speech signals are often degraded by multiple distortions. Recently, a two-stage Repair-and-Denoising network (RaD-Net) was proposed with superior speech quality improvement in the ICASSP 2024 Speech Signal Improvement (SSI) Challenge. However, failure to use future information and constraint receptive field of convolution layers limit the system's performance. To mitigate these problems, we extend RaD-Net to its upgraded version, RaD-Net 2. Specifically, a causality-based knowledge distillation is introduced in the first stage to use future information in a causal way. We use the non-causal repairing network as the teacher to improve the performance of the causal repairing network. In addition, in the second stage, complex axial self-attention is applied in the denoising network's complex feature encoder/decoder. Experimental results on the ICASSP 2024 SSI Challenge blind test set show that RaD-Net 2 brings 0.10 OVRL DNSMOS improvement compared to RaD-Net.


title: Trim 3D Gaussian Splatting for Accurate Geometry Representation

abstract: In this paper, we introduce Trim 3D Gaussian Splatting (TrimGS) to reconstruct accurate 3D geometry from images. Previous arts for geometry reconstruction from 3D Gaussians mainly focus on exploring strong geometry regularization. Instead, from a fresh perspective, we propose to obtain accurate 3D geometry of a scene by Gaussian trimming, which selectively removes the inaccurate geometry while preserving accurate structures. To achieve this, we analyze the contributions of individual 3D Gaussians and propose a contribution-based trimming strategy to remove the redundant or inaccurate Gaussians. Furthermore, our experimental and theoretical analyses reveal that a relatively small Gaussian scale is a non-negligible factor in representing and optimizing the intricate details. Therefore the proposed TrimGS maintains relatively small Gaussian scales. In addition, TrimGS is also compatible with the effective geometry regularization strategies in previous arts. When combined with the original 3DGS and the state-of-the-art 2DGS, TrimGS consistently yields more accurate geometry and higher perceptual quality. Our project page is https://trimgs.github.io


title: SPIN: Spacecraft Imagery for Navigation

abstract: Data acquired in space operational conditions is scarce due to the costs and complexity of space operations. This poses a challenge to learning-based visual-based navigation algorithms employed in autonomous spacecraft navigation. Existing datasets, which largely depend on computer-simulated data, have partially filled this gap. However, the image generation tools they use are proprietary, which limits the evaluation of methods to unseen scenarios. Furthermore, these datasets provide limited ground-truth data, primarily focusing on the spacecraft's translation and rotation relative to the camera. To address these limitations, we present SPIN (SPacecraft Imagery for Navigation), an open-source realistic spacecraft image generation tool for relative navigation between two spacecrafts. SPIN provides a wide variety of ground-truth data and allows researchers to employ custom 3D models of satellites, define specific camera-relative poses, and adjust various settings such as camera parameters and environmental illumination conditions. For the task of spacecraft pose estimation, we compare the results of training with a SPIN-generated dataset against existing synthetic datasets. We show a %50 average error reduction in common testbed data (that simulates realistic space conditions). Both the SPIN tool (and source code) and our enhanced version of the synthetic datasets will be publicly released upon paper acceptance on GitHub https://github.com/vpulab/SPIN.


title: How big is a tiling's return module?

abstract: The rank of a tiling's return module depends on the geometry of its tiles and is not a topological invariant. However, the rank of the first \v Cech cohomology $\check H^1(\Omega)$ gives upper and lower bounds for the size of the return module. For all sufficiently large patches, the rank of the return module is at most the same as the rank of the cohomology. For a generic choice of tile shapes and an arbitrary reference patch, the rank of the return module is at least the rank of $\check H^1(\Omega)$. Therefore, for generic tile shapes and sufficiently large patches, the rank of the return module is equal to the rank of $\check H^1(\Omega)$.


title: Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions

abstract: Image description datasets play a crucial role in the advancement of various applications such as image understanding, text-to-image generation, and text-image retrieval. Currently, image description datasets primarily originate from two sources. One source is the scraping of image-text pairs from the web. Despite their abundance, these descriptions are often of low quality and noisy. Another is through human labeling. Datasets such as COCO are generally very short and lack details. Although detailed image descriptions can be annotated by humans, the high annotation cost limits the feasibility. These limitations underscore the need for more efficient and scalable methods to generate accurate and detailed image descriptions. In this paper, we propose an innovative framework termed Image Textualization (IT), which automatically produces high-quality image descriptions by leveraging existing multi-modal large language models (MLLMs) and multiple vision expert models in a collaborative manner, which maximally convert the visual information into text. To address the current lack of benchmarks for detailed descriptions, we propose several benchmarks for comprehensive evaluation, which verifies the quality of image descriptions created by our framework. Furthermore, we show that LLaVA-7B, benefiting from training on IT-curated descriptions, acquire improved capability to generate richer image descriptions, substantially increasing the length and detail of their output with less hallucination.


title: Hybrid Machine Learning Approach for Cyberattack Mitigation of Parallel Converters in a DC Microgrid

abstract: Cyberattack susceptibilities are introduced as the communication requirement increases with the incorporation of more renewable energy sources into DC microgrids. Parallel DC-DC converters are utilized to provide high current and supply the load. Nevertheless, these systems are susceptible to cyberattacks that have the potential to disrupt operations and jeopardize stability. Voltage instability may result from the manipulation of communication commands and low-layer control signals. Therefore, in this paper, a cyberattack that specifically targets parallel DC-DC converters is examined in a DC microgrid. A hybrid machine learning-based detection and mitigation strategy is suggested as a means to counteract this threat. The false data injection (FDI) attack targeting the converters is investigated within a DC microgrid. The efficacy of the suggested approach is verified via simulations executed for various scenarios within the MATLAB/Simulink environment. The technique successfully identifies and blocks FDI attacks, preventing cyberattacks and ensuring the safe operation of the DC microgrid.


title: Just Because We Camp, Doesn't Mean We Should: The Ethics of Modelling Queer Voices

abstract: Modern voice cloning models claim to be able to capture a diverse range of voices. We test the ability of a typical pipeline to capture the style known colloquially as "gay voice" and notice a homogenisation effect: synthesised speech is rated as sounding significantly "less gay" (by LGBTQ+ participants) than its corresponding ground-truth for speakers with "gay voice", but ratings actually increase for control speakers. Loss of "gay voice" has implications for accessibility. We also find that for speakers with "gay voice", loss of "gay voice" corresponds to lower similarity ratings. However, we caution that improving the ability of such models to synthesise ``gay voice'' comes with a great number of risks. We use this pipeline as a starting point for a discussion on the ethics of modelling queer voices more broadly. Collecting "clean" queer data has safety and fairness ramifications, and the resulting technology may cause harms from mockery to death.


title: THaLLE: Text Hyperlocally Augmented Large Language Extension -- Technical Report

abstract: Recent advancements in Large Language Models (LLMs) have revealed new capabilities and opportunities across the technological landscape. However, the practicality of very large LLMs is challenged by their high compute cost, which does not justify the benefits given their limited capability compared to humans. While smaller, more practical LLMs have shown potential in financial analysis, though they are not yet fully proficient, as evidenced by their near-passing performance on the Chartered Financial Analyst (CFA) exam. In this work, we present Financial Analyst Extension to our Text Hyperlocally Augmented Large Language Extension (THaLLE), a series of 8B LLMs consistently achieving highest performance on mock CFA exams against models of comparable size. We thoroughly document the fine-tuning techniques used to facilitate future research. Additionally, we introduce the use of Flare CFA, a publicly available dataset for evaluating LLMs as a financial advisor.


title: Understanding Visual Concepts Across Models

abstract: Large multimodal models such as Stable Diffusion can generate, detect, and classify new visual concepts after fine-tuning just a single word embedding. Do models learn similar words for the same concepts (i.e. <orange-cat> = orange + cat)? We conduct a large-scale analysis on three state-of-the-art models in text-to-image generation, open-set object detection, and zero-shot classification, and find that new word embeddings are model-specific and non-transferable. Across 4,800 new embeddings trained for 40 diverse visual concepts on four standard datasets, we find perturbations within an $\epsilon$-ball to any prior embedding that generate, detect, and classify an arbitrary concept. When these new embeddings are spliced into new models, fine-tuning that targets the original model is lost. We show popular soft prompt-tuning approaches find these perturbative solutions when applied to visual concept learning tasks, and embeddings for visual concepts are not transferable. Code for reproducing our work is available at: https://visual-words.github.io.


title: Flow Map Matching

abstract: Generative models based on dynamical transport of measure, such as diffusion models, flow matching models, and stochastic interpolants, learn an ordinary or stochastic differential equation whose trajectories push initial conditions from a known base distribution onto the target. While training is cheap, samples are generated via simulation, which is more expensive than one-step models like GANs. To close this gap, we introduce flow map matching -- an algorithm that learns the two-time flow map of an underlying ordinary differential equation. The approach leads to an efficient few-step generative model whose step count can be chosen a-posteriori to smoothly trade off accuracy for computational expense. Leveraging the stochastic interpolant framework, we introduce losses for both direct training of flow maps and distillation from pre-trained (or otherwise known) velocity fields. Theoretically, we show that our approach unifies many existing few-step generative models, including consistency models, consistency trajectory models, progressive distillation, and neural operator approaches, which can be obtained as particular cases of our formalism. With experiments on CIFAR-10 and ImageNet 32x32, we show that flow map matching leads to high-quality samples with significantly reduced sampling cost compared to diffusion or stochastic interpolant methods.


title: $(J/\psi, J/\psi)$, and $(\eta_c, \eta_c)$ production through two intermediate photons in electron-positron annihilation at B-factories

abstract: We study the processes, $e^- e^+ \rightarrow \gamma^* \gamma^* \rightarrow J/\psi +J/\psi$, and $e^- e^+ \rightarrow \gamma^* \gamma^* \rightarrow \eta_c+ \eta_c$ at $\sqrt{s}=10.6$ GeV in the framework of $4\times 4$ Bethe-Salpeter equation. For $J/\psi+J/\psi$ production, the dominant contribution is through fragmentation process, while for $\eta_c+\eta_c$ production, the quark rearrangement diagrams contribute. Our results of cross section for $J/\psi+J/\psi$ and $\psi(2S)+\psi(2S)$ are compatible with the experimental upper limits set by Belle Collaboration, while in the absence of experimental data for $\eta_c(1S)+\eta_c(1S)$, and $\eta_c(2S)+\eta_c(2S)$ production, we have given theoretical prediction of their cross sections, and compared with NRQCD prediction.


title: Uniqueness on average of large isoperimetric sets in noncompact manifolds with nonnegative Ricci curvature

abstract: Let $(M^n,g)$ be a complete Riemannian manifold which is not isometric to $\mathbb{R}^n$, has nonnegative Ricci curvature, Euclidean volume growth, and quadratic Riemann curvature decay. We prove that there is a set $\mathcal{G}\subset (0,\infty)$ with density $1$ at infinity such that for every $V\in \mathcal{G}$ there is a unique isoperimetric set of volume $V$ in $M$, and its boundary is strictly volume preserving stable. The latter result cannot be improved to uniqueness or strict stability for every large volume. Indeed, we construct a complete Riemannian surface that satisfies the previous assumptions, and with the following property: there are arbitrarily large and diverging intervals $I_n\subset (0,\infty)$ such that isoperimetric sets with volumes $V\in I_n$ exist, but they are neither unique nor they have strictly volume preserving stable boundaries. The proof relies on a set of new ideas, as the present setting goes beyond the range of applicability of the methods based on the implicit function theorem, and no symmetry is assumed.


title: COMAP Pathfinder -- Season 2 results I. Improved data selection and processing

abstract: The CO Mapping Array Project (COMAP) Pathfinder is performing line intensity mapping of CO emission to trace the distribution of unresolved galaxies at redshift $z \sim 3$. We present an improved version of the COMAP data processing pipeline and apply this to the first two seasons of observations. This analysis improves on the COMAP Early Science (ES) results in several key aspects. On the observational side, all second season scans were made in constant-elevation mode, after noting that the previous Lissajous scans were associated with increased systematic errors; those scans accounted for 50% of the total Season 1 data volume. Secondly, all new observations were restricted to an elevation range of 35-65 degrees, to minimize sidelobe ground pickup. On the data processing side, more effective data cleaning in both the time- and map-domain has allowed us to eliminate all data-driven power spectrum-based cuts. This increases the overall data retention and reduces the risk of signal subtraction bias. On the other hand, due to the increased sensitivity, two new pointing-correlated systematic errors have emerged, and we introduce a new map-domain PCA filter to suppress these. Subtracting only 5 out of 256 PCA modes, we find that the standard deviation of the cleaned maps decreases by 67% on large angular scales, and after applying this filter, the maps appear consistent with instrumental noise. Combining all these improvements, we find that each hour of raw Season 2 observations yields on average 3.2 times more cleaned data compared to ES analysis. Combining this with the increase in raw observational hours, the effective amount of data available for high-level analysis is a factor of 8 higher than in ES. The resulting maps have reached an uncertainty of $25$-$50\,\mu K$ per voxel, providing by far the strongest constraints on cosmological CO line emission published to date.


title: COMAP Pathfinder -- Season 2 results II. Updated constraints on the CO(1-0) power spectrum

abstract: We present updated constraints on the cosmological 3D power spectrum of carbon monoxide CO(1-0) emission in the redshift range $2.4$-$3.4$. The constraints are derived from the two first seasons of Carbon monOxide Mapping Array Project (COMAP) Pathfinder line-intensity mapping observations aiming to trace star-formation during the Epoch of Galaxy Assembly. These results improve on the previous Early Science (ES) results through both increased data volume and improved data processing methodology. On the methodological side, we now perform cross-correlations between groups of detectors (''feed-groups''), as opposed to cross-correlations between single feeds, and this new feed-group pseudo power spectrum (FGPXS) is constructed to be more robust against systematic effects. In terms of data volume, the effective mapping speed is significantly increased due to an improved observational strategy as well as better data selection methodology. The updated spherically- and field-averaged FGPXS, $\tilde{C}(k)$, is consistent with zero, at a probability-to-exceed of around $34\,\%$, with an excess of $2.7\,\sigma$ in the most sensitive bin. Our power spectrum estimate is about an order of magnitude more sensitive in our six deepest bins across ${0.09\,\mathrm{Mpc}^{-1} < k < 0.73\,\mathrm{Mpc}^{-1}}$, as compared to the feed-feed pseudo power spectrum (FPXS) of COMAP ES. Each of these bins individually constrains the CO power spectrum to ${kP_\mathrm{CO}(k)< 2400-4900\,\mathrm{\mu K^2 Mpc^{2}}}$ at $95\,\%$ confidence. To monitor potential contamination from residual systematic effects, we analyze a set of 312 difference-map null tests and find that these are consistent with the instrumental noise prediction. In sum, these results provide the strongest direct constraints on the cosmological 3D CO(1-0) power spectrum published to date.


title: COMAP Pathfinder -- Season 2 results III. Implications for cosmic molecular gas content at "Cosmic Half-past Eleven"

abstract: The Carbon monOxide Mapping Array Project (COMAP) Pathfinder survey continues to demonstrate the feasibility of line-intensity mapping using high-redshift carbon monoxide (CO) line emission traced at cosmological scales. The latest COMAP Pathfinder power spectrum analysis is based on observations through the end of Season 2, covering the first three years of Pathfinder operations. We use our latest constraints on the CO(1-0) line-intensity power spectrum at $z\sim3$ to update corresponding constraints on the cosmological clustering of CO line emission and thus the cosmic molecular gas content at a key epoch of galaxy assembly. We first mirror the COMAP Early Science interpretation, considering how Season 2 results translate to limits on the shot noise power of CO fluctuations and the bias of CO emission as a tracer of the underlying dark matter distribution. The COMAP Season 2 results place the most stringent limits on the CO tracer bias to date, at $\langle{Tb}\rangle<4.8$ $\mu$K. These limits narrow the model space significantly compared to previous CO line-intensity mapping results while maintaining consistency with small-volume interferometric surveys of resolved line candidates. The results also express a weak preference for CO emission models used to guide fiducial forecasts from COMAP Early Science, including our data-driven priors. We also consider directly constraining a model of the halo-CO connection, and show qualitative hints of capturing the total contribution of faint CO emitters through the improved sensitivity of COMAP data. With continued observations and matching improvements in analysis, the COMAP Pathfinder remains on track for a detection of cosmological clustering of CO emission.


title: Accurate Current Sharing in a DC Microgrid Using Modified Droop Control Algorithm

abstract: Due to the increasing popularity of DC loads and the potential for higher efficiency, DC microgrids are gaining significant attention. DC microgrids utilize multiple parallel converters to deliver sufficient power to the load. However, a key challenge arises when connecting these converters to a common DC bus: maintaining voltage regulation and accurate current sharing. Unequal cable resistances can cause uneven power sharing and lead to power losses. Conventional droop control methods, which employ a virtual resistor to address this issue, have limitations in achieving good performance across the entire converter operating range. This paper proposes a modified droop control algorithm to address this issue. This method modifies the virtual resistor in a way that ensures power sharing aligns with each converter-rated capacity. The algorithm is simple to implement and uses local measurements to update the droop gain. This paper presents simulation studies and experimental tests to analyze the performance of the proposed method, considering scenarios with equal and unequal converter ratings. The results successfully validate the accuracy and effectiveness of this innovative approach.


title: Scintillation Light in SBND: Simulation, Reconstruction, and Expected Performance of the Photon Detection System

abstract: SBND is the near detector of the Short-Baseline Neutrino program at Fermilab. Its location near to the Booster Neutrino Beam source and relatively large mass will allow the study of neutrino interactions on argon with unprecedented statistics. This paper describes the expected performance of the SBND photon detection system, using a simulated sample of beam neutrinos and cosmogenic particles. Its design is a dual readout concept combining a system of 120 photomultiplier tubes, used for triggering, with a system of 192 X-ARAPUCA devices, located behind the anode wire planes. Furthermore, covering the cathode plane with highly-reflective panels coated with a wavelength-shifting compound recovers part of the light emitted towards the cathode, where no optical detectors exist. We show how this new design provides a high light yield and a more uniform detection efficiency, an excellent timing resolution and an independent 3D-position reconstruction using only the scintillation light. Finally, the whole reconstruction chain is applied to recover the temporal structure of the beam spill, which is resolved with a resolution on the order of nanoseconds.


title: Beyond Model Collapse: Scaling Up with Synthesized Data Requires Reinforcement

abstract: Synthesized data from generative models is increasingly considered as an alternative to human-annotated data for fine-tuning Large Language Models. This raises concerns about model collapse: a drop in performance of models fine-tuned on generated data. Considering that it is easier for both humans and machines to tell between good and bad examples than to generate high-quality samples, we investigate the use of feedback on synthesized data to prevent model collapse. We derive theoretical conditions under which a Gaussian mixture classification model can achieve asymptotically optimal performance when trained on feedback-augmented synthesized data, and provide supporting simulations for finite regimes. We illustrate our theoretical predictions on two practical problems: computing matrix eigenvalues with transformers and news summarization with large language models, which both undergo model collapse when trained on model-generated data. We show that training from feedback-augmented synthesized data, either by pruning incorrect predictions or by selecting the best of several guesses, can prevent model collapse, validating popular approaches like RLHF.


title: Instant 3D Human Avatar Generation using Image Diffusion Models

abstract: We present AvatarPopUp, a method for fast, high quality 3D human avatar generation from different input modalities, such as images and text prompts and with control over the generated pose and shape. The common theme is the use of diffusion-based image generation networks that are specialized for each particular task, followed by a 3D lifting network. We purposefully decouple the generation from the 3D modeling which allow us to leverage powerful image synthesis priors, trained on billions of text-image pairs. We fine-tune latent diffusion networks with additional image conditioning to solve tasks such as image generation and back-view prediction, and to support qualitatively different multiple 3D hypotheses. Our partial fine-tuning approach allows to adapt the networks for each task without inducing catastrophic forgetting. In our experiments, we demonstrate that our method produces accurate, high-quality 3D avatars with diverse appearance that respect the multimodal text, image, and body control signals. Our approach can produce a 3D model in as few as 2 seconds, a four orders of magnitude speedup w.r.t. the vast majority of existing methods, most of which solve only a subset of our tasks, and with fewer controls, thus enabling applications that require the controlled 3D generation of human avatars at scale. The project website can be found at https://www.nikoskolot.com/avatarpopup/.


title: The canonical trace of Cohen-Macaulay algebras of codimension 2

abstract: In the present paper, we investigate a conjecture of J\"urgen Herzog. Let $S$ be a local regular ring with residue field $K$ or a positively graded $K$-algebra, $I\subset S$ be a perfect ideal of grade two, and let $R=S/I$ with canonical module $\omega_R$. Herzog conjectured that the canonical trace $\text{tr}(\omega_R)$ is obtained by specialization from the generic case of maximal minors. We prove this conjecture in several cases, and present a criterion that guarantees that the canonical trace specializes under some additional assumptions. As the final conclusion of all of our results, we classify the nearly Gorenstein monomial ideals of height two.


title: On constant mean curvature 1-immersions of surfaces into hyperbolic 3-manifolds

abstract: Motivated by the work of Bryant on constant mean curvature (CMC) $1$-immersions of surfaces into the hyperbolic space H^3 and after the results of Tarantello (2023), we pursue a possible parametrization for the moduli space of (CMC) 1-immersions of a surface S (closed, orientable and of genus >1) into hyperbolic 3-manifolds. Those immersions enter as "critical" object in our analysis. In fact, they can be attained only as limits of the (CMC) c-immersions (as c tends to 1), obtained in Huang-Lucia-Tarantello (2022), for |c|<1. However, such passage to the limit can be prevented by possible blow-up phenomena, so that the pullback metrics of the (CMC) c-immersions may yield (at the limit) to a singular metric with conical singularities at finitely many points (the blow-up points). In case of genus g=2, blow up can occur only at a single point, and in Tarantello (2023) it was shown how it could be prevented and the passage to the limit ensured in terms of the Kodaira map. In this note we sharpen this result and for genus g=2, we obtaina condition (we believe sharp) which involves only the Kodaira map on the six Weierstrass points. In addition we tackle the case of higher genus, where multiple blow-up points occur. In this case, we need to identify a suitable replacement of the Kodaira map, now defined on the space of non-zero effective divisors. More importantly, we need to improve in a substantial way the asymptotic analysis of Tarantello (2023) limited to the case of "blow-up" with minimal mass. In this direction we give a contribution which best applies to the case of genus g=3, but also provides a relevant step and a convincing indication on what should happen in the general case.


title: Physics-guided weak-form discovery of reduced-order models for trapped ultracold hydrodynamics

abstract: We study the relaxation of a highly collisional, ultracold but nondegenerate gas of polar molecules. Confined within a harmonic trap, the gas is subject to fluid-gaseous coupled dynamics that lead to a breakdown of first-order hydrodynamics. An attempt to treat these higher-order hydrodynamic effects was previously made with a Gaussian ansatz and coarse-graining model parameter [R. R. W. Wang & J. L. Bohn, Phys. Rev. A 108, 013322 (2023)], leading to an approximate set of equations for a few collective observables accessible to experiments. Here we present substantially improved reduced-order models for these same observables, admissible beyond previous parameter regimes, discovered directly from particle simulations using the WSINDy algorithm (Weak-form Sparse Identification of Nonlinear Dynamics). The interpretable nature of the learning algorithm enables estimation of previously unknown physical quantities and discovery of model terms with candidate physical mechanisms, revealing new physics in mixed collisional regimes. Our approach constitutes a general framework for data-driven model identification leveraging known physics.

categories: cond-mat.quant-gas cond-mat.stat-mech cs.LG math.DS


title: Neural Gaffer: Relighting Any Object via Diffusion

abstract: Single-image relighting is a challenging task that involves reasoning about the complex interplay between geometry, materials, and lighting. Many prior methods either support only specific categories of images, such as portraits, or require special capture conditions, like using a flashlight. Alternatively, some methods explicitly decompose a scene into intrinsic components, such as normals and BRDFs, which can be inaccurate or under-expressive. In this work, we propose a novel end-to-end 2D relighting diffusion model, called Neural Gaffer, that takes a single image of any object and can synthesize an accurate, high-quality relit image under any novel environmental lighting condition, simply by conditioning an image generator on a target environment map, without an explicit scene decomposition. Our method builds on a pre-trained diffusion model, and fine-tunes it on a synthetic relighting dataset, revealing and harnessing the inherent understanding of lighting present in the diffusion model. We evaluate our model on both synthetic and in-the-wild Internet imagery and demonstrate its advantages in terms of generalization and accuracy. Moreover, by combining with other generative methods, our model enables many downstream 2D tasks, such as text-based relighting and object insertion. Our model can also operate as a strong relighting prior for 3D tasks, such as relighting a radiance field.


title: Faster Spectral Density Estimation and Sparsification in the Nuclear Norm

abstract: We consider the problem of estimating the spectral density of the normalized adjacency matrix of an $n$-node undirected graph. We provide a randomized algorithm that, with $O(n\epsilon^{-2})$ queries to a degree and neighbor oracle and in $O(n\epsilon^{-3})$ time, estimates the spectrum up to $\epsilon$ accuracy in the Wasserstein-1 metric. This improves on previous state-of-the-art methods, including an $O(n\epsilon^{-7})$ time algorithm from [Braverman et al., STOC 2022] and, for sufficiently small $\epsilon$, a $2^{O(\epsilon^{-1})}$ time method from [Cohen-Steiner et al., KDD 2018]. To achieve this result, we introduce a new notion of graph sparsification, which we call nuclear sparsification. We provide an $O(n\epsilon^{-2})$-query and $O(n\epsilon^{-2})$-time algorithm for computing $O(n\epsilon^{-2})$-sparse nuclear sparsifiers. We show that this bound is optimal in both its sparsity and query complexity, and we separate our results from the related notion of additive spectral sparsification. Of independent interest, we show that our sparsification method also yields the first deterministic algorithm for spectral density estimation that scales linearly with $n$ (sublinear in the representation size of the graph).


title: Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling

abstract: Efficiently modeling sequences with infinite context length has been a long-standing problem. Past works suffer from either the quadratic computation complexity or the limited extrapolation ability on length generalization. In this work, we present Samba, a simple hybrid architecture that layer-wise combines Mamba, a selective State Space Model (SSM), with Sliding Window Attention (SWA). Samba selectively compresses a given sequence into recurrent hidden states while still maintaining the ability to precisely recall memories with the attention mechanism. We scale Samba up to 3.8B parameters with 3.2T training tokens and show that Samba substantially outperforms the state-of-the-art models based on pure attention or SSMs on a wide range of benchmarks. When trained on 4K length sequences, Samba can be efficiently extrapolated to 256K context length with perfect memory recall and show improved token predictions up to 1M context length. As a linear-time sequence model, Samba enjoys a 3.73x higher throughput compared to Transformers with grouped-query attention when processing user prompts of 128K length, and 3.64x speedup when generating 64K tokens with unlimited streaming. A sample implementation of Samba is publicly available in https://github.com/microsoft/Samba.


title: Change of numeraire for weak martingale transport

abstract: Change of numeraire is a classical tool in mathematical finance. Campi-Laachir-Martini established its applicability to martingale optimal transport. We note that the results of Campi-Laachir-Martini extend to the case of weak martingale transport. We apply this to shadow couplings, continuous time martingale transport problems in the framework of Huesmann-Trevisan and in particular to establish the correspondence between stretched Brownian motion with its geometric counterpart. Note: We emphasize that we learned about the geometric stretched Brownian motion gSBM (defined in PDE terms) in a presentation of Loeper \cite{Lo23} before our work on this topic started. We noticed that a change of numeraire transformation in the spirit of \cite{CaLaMa14} allows for an alternative viewpoint in the weak optimal transport framework. We make our work public following the publication of Backhoff-Loeper-Obloj's work \cite{BaLoOb24} on arxiv.org. The article \cite{BaLoOb24} derives gSBM using PDE techniques as well as through an independent probabilistic approach which is close to the one we give in the present article.


title: Simple and Effective Masked Diffusion Language Models

abstract: While diffusion models excel at generating high-quality images, prior work reports a significant performance gap between diffusion and autoregressive (AR) methods in language modeling. In this work, we show that simple masked discrete diffusion is more performant than previously thought. We apply an effective training recipe that improves the performance of masked diffusion models and derive a simplified, Rao-Blackwellized objective that results in additional improvements. Our objective has a simple form -- it is a mixture of classical masked language modeling losses -- and can be used to train encoder-only language models that admit efficient samplers, including ones that can generate arbitrary lengths of text semi-autoregressively like a traditional language model. On language modeling benchmarks, a range of masked diffusion models trained with modern engineering practices achieves a new state-of-the-art among diffusion models, and approaches AR perplexity. We release our code at: https://github.com/kuleshov-group/mdlm


title: Will Southeast Asia be the next global manufacturing hub? A multiway cointegration, causality, and dynamic connectedness analyses on factors influencing offshore decisions

abstract: The COVID-19 pandemic has compelled multinational corporations to diversify their global supply chain risk and to relocate their factories to Southeast Asian countries beyond China. Such recent phenomena provide a good opportunity to understand the factors that influenced offshore decisions in the last two decades. We propose a new conceptual framework based on econometric approaches to examine the relationships between these factors. Firstly, the Vector Auto Regression (VAR) for multi-way cointegration analysis by a Johansen test as well as the embedding Granger causality analysis to examine offshore decisions--innovation, technology readiness, infrastructure, foreign direct investment (FDI), and intermediate imports. Secondly, a Quantile Vector Autoregressive (QVAR) model is used to assess the dynamic connectedness among Southeast Asian countries based on the offshore factors. This study explores a system-wide experiment to evaluate the spillover effects of offshore decisions. It reports a comprehensive analysis using time-series data collected from the World Bank. The results of the cointegration, causality, and dynamic connectedness analyses show that a subset of Southeast Asian countries have spillover effects on each other. These countries present a multi-way cointegration and dynamic connectedness relationship. The study contributes to policymaking by providing a data-driven innovative approach through a new conceptual framework.


title: Cosmological constraints on $\Lambda_{\rm s}$CDM scenario in a type II minimally modified gravity

abstract: The idea of a rapid sign-switching cosmological constant (mirror AdS-dS transition) in the late universe at $z\sim1.7$, known as the $\Lambda_{\rm s}$CDM model, has significantly improved the fit to observational data and provides a promising scenario for alleviating major cosmological tensions, such as the $H_0$ and $S_8$ tensions. However, in the absence of a fully predictive model, implementing this fit required conjecturing that the dynamics of the linear perturbations are governed by general relativity. Recent work embedding the $\Lambda_{\rm s}$CDM model with the Lagrangian of a type-II minimally modified gravity known as VCDM has propelled $\Lambda_{\rm s}$CDM to a fully predictive model, removing the uncertainty related to the aforementioned assumption; we call this new model $\Lambda_{\rm s}$VCDM. In this work, we demonstrate that not only does $\Lambda_{\rm s}$CDM fit the data better than the standard $\Lambda$CDM model, but the new model, $\Lambda_{\rm s}$VCDM, performs even better in alleviating cosmological tensions while also providing a better fit to the data, including CMB, BAO, SNe Ia, and Cosmic Shear measurements. Our findings highlight the $\Lambda_{\rm s}$CDM framework, particularly the $\Lambda_{\rm s}$VCDM model, as a compelling alternative to the standard $\Lambda$CDM model, especially by successfully alleviating the $H_0$ tension. Additionally, these models predict higher values for $\sigma_8$, indicating enhanced structuring, albeit with lower present-day matter density parameter values and consequently reduced $S_8$ values, alleviating the $S_8$ tension as well. This demonstrates that the data are well fit by a combination of background and linear perturbations, both having dynamics differing from those of $\Lambda$CDM. This paves the way for further exploration of new ways for embedding the sign-switching cosmological constant into other models.


title: Interpolating between Hausdorff and box dimension

abstract: Hausdorff and box dimension are two familiar notions of fractal dimension. Box dimension can be larger than Hausdorff dimension, because in the definition of box dimension, all sets in the cover have the same diameter, but for Hausdorff dimension there is no such restriction. This thesis focuses on a family of dimensions parameterised by $\theta \in (0,1)$, called the intermediate dimensions, which are defined by requiring that $\mbox{diam}(U) \leq (\mbox{diam}(V))^{\theta}$ for all sets $U,V$ in the cover. We begin by generalising the intermediate dimensions to allow for greater refinement in how the relative sizes of the covering sets are restricted. These new dimensions can recover the interpolation between Hausdorff and box dimension for compact sets whose intermediate dimensions do not tend to the Hausdorff dimension as $\theta \to 0$. We also use a Moran set construction to prove a necessary and sufficient condition, in terms of Dini derivatives, for a given function to be realised as the intermediate dimensions of a set. We proceed to prove that the intermediate dimensions of limit sets of infinite conformal iterated function systems are given by the maximum of the Hausdorff dimension of the limit set and the intermediate dimensions of the set of fixed points of the contractions. This applies to sets defined using continued fraction expansions, and has applications to dimensions of projections, fractional Brownian images, and general H\"older images. Finally, we determine a formula for the intermediate dimensions of all self-affine Bedford-McMullen carpets. The functions display features not witnessed in previous examples, such as having countably many phase transitions. We deduce that two carpets have equal intermediate dimensions if and only if the multifractal spectra of the corresponding uniform Bernoulli measures coincide.


title: QuickLLaMA: Query-aware Inference Acceleration for Large Language Models

abstract: The capacity of Large Language Models (LLMs) to comprehend and reason over long contexts is pivotal for advancements in diverse fields. Yet, they still stuggle with capturing long-distance dependencies within sequences to deeply understand semantics. To address this issue, we introduce Query-aware Inference for LLMs (Q-LLM), a system designed to process extensive sequences akin to human cognition. By focusing on memory data relevant to a given query, Q-LLM can accurately capture pertinent information within a fixed window size and provide precise answers to queries. It doesn't require extra training and can be seamlessly integrated with any LLMs. Q-LLM using LLaMA3 (QuickLLaMA) can read Harry Potter within 30s and accurately answer the questions. Q-LLM improved by 7.17% compared to the current state-of-the-art on LLaMA3, and by 3.26% on Mistral on the $\infty$-bench. In the Needle-in-a-Haystack task, On widely recognized benchmarks, Q-LLM improved upon the current SOTA by 7.0% on Mistral and achieves 100% on LLaMA3. Our code can be found in https://github.com/dvlab-research/Q-LLM.


title: MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation

abstract: Model merging has emerged as an effective approach to combine multiple single-task models, fine-tuned from the same pre-trained model, into a multitask model. This process typically involves computing a weighted average of the model parameters without any additional training. Existing model-merging methods focus on enhancing average task accuracy. However, interference and conflicts between the objectives of different tasks can lead to trade-offs during model merging. In real-world applications, a set of solutions with various trade-offs can be more informative, helping practitioners make decisions based on diverse preferences. In this paper, we introduce a novel low-compute algorithm, Model Merging with Amortized Pareto Front (MAP). MAP identifies a Pareto set of scaling coefficients for merging multiple models to reflect the trade-offs. The core component of MAP is approximating the evaluation metrics of the various tasks using a quadratic approximation surrogate model derived from a pre-selected set of scaling coefficients, enabling amortized inference. Experimental results on vision and natural language processing tasks show that MAP can accurately identify the Pareto front. To further reduce the required computation of MAP, we propose (1) a Bayesian adaptive sampling algorithm and (2) a nested merging scheme with multiple stages.


title: Coherent Three-Photon Excitation of the Strontium Clock Transition

abstract: We recently demonstrated a continuous Bose-Einstein condensate of strontium atoms. We could turn this into a continuous-wave atom laser if an efficient outcoupling mechanism is found. Here we show a coherent three-photon excitation of the clock transition in a strontium BEC with contrast of 44.6(3.5)%. We follow it up with a demonstration of three-photon STIRAP-like transfer. Our work constitutes an essential step towards the outcoupling of a continuous atom laser beam and provides a robust excitation mechanism for quantum simulation.


title: Interacting-bath dynamical embedding for capturing non-local electron correlation in solids

abstract: Quantitative simulation of electronic structure of solids requires the treatment of local and non-local electron correlations on an equal footing. Dynamical mean-field theory, a widely-used quantum embedding algorithm that assumes local self-energy approximation, is challenging to extend to capture long-range electron correlation. In this work, we present a new formulation of Green's function embedding that, instead of embedding the impurity in non-interacting bath through the hybridization function, derives bath representation with general two-particle interactions in a fully ab initio and systematically improvable manner. The resulting interacting-bath dynamical embedding theory (ibDET) simulates local and non-local self-energies using high-level quantum chemistry solvers, such as the coupled-cluster theory. We demonstrate that ibDET combined with the GW theory (GW+ibDET) achieves good agreements with experimental band structure and photoemission spectra while preserving translational symmetry across a range of semiconducting, insulating, and metallic materials. Furthermore, our approach allows quantifying the role of non-local electron correlation in determining spectral properties of materials and addressing the long-standing debate over the bandwidth narrowing of metallic sodium.


title: Hearing Anything Anywhere

abstract: Recent years have seen immense progress in 3D computer vision and computer graphics, with emerging tools that can virtualize real-world 3D environments for numerous Mixed Reality (XR) applications. However, alongside immersive visual experiences, immersive auditory experiences are equally vital to our holistic perception of an environment. In this paper, we aim to reconstruct the spatial acoustic characteristics of an arbitrary environment given only a sparse set of (roughly 12) room impulse response (RIR) recordings and a planar reconstruction of the scene, a setup that is easily achievable by ordinary users. To this end, we introduce DiffRIR, a differentiable RIR rendering framework with interpretable parametric models of salient acoustic features of the scene, including sound source directivity and surface reflectivity. This allows us to synthesize novel auditory experiences through the space with any source audio. To evaluate our method, we collect a dataset of RIR recordings and music in four diverse, real environments. We show that our model outperforms state-ofthe-art baselines on rendering monaural and binaural RIRs and music at unseen locations, and learns physically interpretable parameters characterizing acoustic properties of the sound source and surfaces in the scene.


title: Interpreting DESI 2024 BAO: late-time dynamical dark energy or a local effect?

abstract: We perform fits to DESI, CMB and supernova data to understand the physical origin of the DESI hint for dynamical dark energy. We find that the linear parametrization of the equation of state $w$ may guide to misleading interpretations, such as the hint for a phantom Universe, which are not preferred by the data. Instead, physical quintessence models fit the data well. Model-independently, present observations prefer deviations from the constant dark energy, $w=-1$, only at very low redshifts, $z < \mathcal{O}(0.1)$. We find that this result is driven by low-$z$ supernova data. Therefore, either the fundamental properties of our Universe, characterised by the equation of state $w$ and the Hubble parameter $H$, underwent dramatic changes very recently or, alternatively, we do not fully understand the systematics of our local Universe in a radius of about $300\,h^{-1}\rm Mpc$.