title: Direct estimates of nitrogen abundance for Seyfert 2 nuclei

abstract: We derive the nitrogen and oxygen abundances in the Narrow Line Regions (NLRs) of a sample of 38 local ($z \: < \: 0.4$) Seyfert~2 nuclei. For that, we consider narrow optical emission line intensities and direct estimates of the electron temperatures ($T_{\rm e}$-method). We find nitrogen abundances in the range $7.6 \: < \: \rm 12+log(N/H) \: < \: 8.6$ (mean value $8.06\pm0.22$) or $\rm 0.4 \: < \: (N/N_{\odot}) \: < 4.7$, in the metallicity regime $8.3 \: < \: \rm 12+log(O/H) \: < \: 9.0$. Our results indicate that the dispersion in N/H abundance for a fixed O/H value in AGNs is in agreement with that for disc \ion{H}{ii} regions with similar metallicity. We show that Seyfert~2 nuclei follow a similar (N/O)-(O/H) relation to the one followed by star-forming objects. Finally, we find that active galaxies called as 'nitrogen-loud' observed at very high redshift ($z \: > \: 5$) show N/O values in consonance with those derived for local NLRs. This result indicates that the main star-formation event is completed in the early evolution stages of active galaxies.


title: Just rephrase it! Uncertainty estimation in closed-source language models via multiple rephrased queries

abstract: State-of-the-art large language models are sometimes distributed as open-source software but are also increasingly provided as a closed-source service. These closed-source large-language models typically see the widest usage by the public, however, they often do not provide an estimate of their uncertainty when responding to queries. As even the best models are prone to ``hallucinating" false information with high confidence, a lack of a reliable estimate of uncertainty limits the applicability of these models in critical settings. We explore estimating the uncertainty of closed-source LLMs via multiple rephrasings of an original base query. Specifically, we ask the model, multiple rephrased questions, and use the similarity of the answers as an estimate of uncertainty. We diverge from previous work in i) providing rules for rephrasing that are simple to memorize and use in practice ii) proposing a theoretical framework for why multiple rephrased queries obtain calibrated uncertainty estimates. Our method demonstrates significant improvements in the calibration of uncertainty estimates compared to the baseline and provides intuition as to how query strategies should be designed for optimal test calibration.


title: Spin-Peierls transition in the frustrated spinels ZnCr2O4 and MgCr2O4

abstract: The chromium spinels MgCr2O4 and ZnCr2O4 are prime examples of the highly frustrated pyrochlore lattice antiferromagnet. Experiment has carefully established that both materials, upon cooling, distort to lower symmetry and order magnetically. We study the nature of this process by a combination of density-functional-theory based energy mapping and classical Monte Carlo simulations. We first computationally establish precise Heisenberg Hamiltonian parameters for the high temperature cubic and the low temperature tetragonal and orthorhombic structures of both spinels. We then investigate the respective ordering temperatures of high symmetry and low symmetry structures. We carefully compare our results with experimental facts and find that our simulations are remarkably consistent with a type of spin-Peierls mechanism, adapted to three dimensions, where the structural distortion is mediated by a magnetic energy gain due to a lower degree of frustration.


title: Causal Energy-Momentum Tensors and Relativistic Fluids

abstract: In this paper, we consider a theory defined by an energy-momentum tensor depending on a set of general fields, including the space-time metric. We prove that if the theory is causal, bounded and transforms appropriately under diffeomorphism, it will depend only on the local values of the independent fields and their covariant derivatives up to a finite order. The implications are that the energy-momentum tensor of a causal relativistic fluid can only depend on covariant derivatives only up to a finite order.


title: Learning Latent Space Hierarchical EBM Diffusion Models

abstract: This work studies the learning problem of the energy-based prior model and the multi-layer generator model. The multi-layer generator model, which contains multiple layers of latent variables organized in a top-down hierarchical structure, typically assumes the Gaussian prior model. Such a prior model can be limited in modelling expressivity, which results in a gap between the generator posterior and the prior model, known as the prior hole problem. Recent works have explored learning the energy-based (EBM) prior model as a second-stage, complementary model to bridge the gap. However, the EBM defined on a multi-layer latent space can be highly multi-modal, which makes sampling from such marginal EBM prior challenging in practice, resulting in ineffectively learned EBM. To tackle the challenge, we propose to leverage the diffusion probabilistic scheme to mitigate the burden of EBM sampling and thus facilitate EBM learning. Our extensive experiments demonstrate a superior performance of our diffusion-learned EBM prior on various challenging tasks.


title: TOPA: Extend Large Language Models for Video Understanding via Text-Only Pre-Alignment

abstract: Recent advancements in image understanding have benefited from the extensive use of web image-text pairs. However, video understanding remains a challenge despite the availability of substantial web video-text data. This difficulty primarily arises from the inherent complexity of videos and the inefficient language supervision in recent web-collected video-text datasets. In this paper, we introduce Text-Only Pre-Alignment (TOPA), a novel approach to extend large language models (LLMs) for video understanding, without the need for pre-training on real video data. Specifically, we first employ an advanced LLM to automatically generate Textual Videos comprising continuous textual frames, along with corresponding annotations to simulate real video-text data. Then, these annotated textual videos are used to pre-align a language-only LLM with the video modality. To bridge the gap between textual and real videos, we employ the CLIP model as the feature extractor to align image and text modalities. During text-only pre-alignment, the continuous textual frames, encoded as a sequence of CLIP text features, are analogous to continuous CLIP image features, thus aligning the LLM with real video representation. Extensive experiments, including zero-shot evaluation and finetuning on various video understanding tasks, demonstrate that TOPA is an effective and efficient framework for aligning video content with LLMs. In particular, without training on any video data, the TOPA-Llama2-13B model achieves a Top-1 accuracy of 51.0% on the challenging long-form video understanding benchmark, Egoschema. This performance surpasses previous video-text pre-training approaches and proves competitive with recent GPT-3.5-based video agents.


title: Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methods

abstract: We study the matrix denoising problem of estimating the singular vectors of a rank-$1$ signal corrupted by noise with both column and row correlations. Existing works are either unable to pinpoint the exact asymptotic estimation error or, when they do so, the resulting approaches (e.g., based on whitening or singular value shrinkage) remain vastly suboptimal. On top of this, most of the literature has focused on the special case of estimating the left singular vector of the signal when the noise only possesses row correlation (one-sided heteroscedasticity). In contrast, our work establishes the information-theoretic and algorithmic limits of matrix denoising with doubly heteroscedastic noise. We characterize the exact asymptotic minimum mean square error, and design a novel spectral estimator with rigorous optimality guarantees: under a technical condition, it attains positive correlation with the signals whenever information-theoretically possible and, for one-sided heteroscedasticity, it also achieves the Bayes-optimal error. Numerical experiments demonstrate the significant advantage of our theoretically principled method with the state of the art. The proofs draw connections with statistical physics and approximate message passing, departing drastically from standard random matrix theory techniques.

categories: math.ST cs.IT cs.LG math.IT math.PR stat.ML stat.TH


title: A geometrical description of non-Hermitian dynamics: speed limits in finite rank density operators

abstract: Non-Hermitian dynamics in quantum systems preserves the rank of the state density operator. We use this insight to develop its geometrical description. In particular, we identify mutually orthogonal coherent and incoherent directions and give their physical interpretation. This understanding allows us to optimize the success rate for implementing non-Hermitian driving along prescribed trajectories. We show its significance for shortcuts to adiabaticity. We introduce the geometrical interpretation of a speed limit for non-Hermitian Hamiltonians and analyze its tightness. We derive the explicit expression that saturates such a speed limit and illustrate our results on a minimal example of a dissipative qubit.


title: The chromatic number of very dense random graphs

abstract: The chromatic number of a very dense random graph $G(n,p)$, with $p \ge 1 - n^{-c}$ for some constant $c > 0$, was first studied by Surya and Warnke, who conjectured that the typical deviation of $\chi(G(n,p))$ from its mean is of order $\sqrt{\mu_r}$, where $\mu_r$ is the expected number of independent sets of size $r$, and $r$ is maximal such that $\mu_r > 1$, except when $\mu_r = O(\log n)$. They moreover proved their conjecture in the case $n^{-2} \ll 1 - p = O(n^{-1})$. In this paper, we study $\chi(G(n,p))$ in the range $n^{-1}\log n \ll 1 - p \ll n^{-2/3}$, that is, when the largest independent set of $G(n,p)$ is typically of size 3. We prove in this case that $\chi(G(n,p))$ is concentrated on some interval of length $O(\sqrt{\mu_3})$, and for sufficiently `smooth' functions $p = p(n)$, that there are infinitely many values of $n$ such that $\chi(G(n,p))$ is not concentrated on any interval of size $o(\sqrt{\mu_3})$. We also show that $\chi(G(n,p))$ satisfies a central limit theorem in the range $n^{-1} \log n \ll 1 - p \ll n^{-7/9}$.


title: HeteGraph-Mamba: Heterogeneous Graph Learning via Selective State Space Model

abstract: We propose a heterogeneous graph mamba network (HGMN) as the first exploration in leveraging the selective state space models (SSSMs) for heterogeneous graph learning. Compared with the literature, our HGMN overcomes two major challenges: (i) capturing long-range dependencies among heterogeneous nodes and (ii) adapting SSSMs to heterogeneous graph data. Our key contribution is a general graph architecture that can solve heterogeneous nodes in real-world scenarios, followed an efficient flow. Methodologically, we introduce a two-level efficient tokenization approach that first captures long-range dependencies within identical node types, and subsequently across all node types. Empirically, we conduct comparisons between our framework and 19 state-of-the-art methods on the heterogeneous benchmarks. The extensive comparisons demonstrate that our framework outperforms other methods in both the accuracy and efficiency dimensions.


title: Projective Space in Synthetic Algebraic Geometry

abstract: Working in an abstract, homotopy type theory based axiomatization of the higher Zariski-topos called synthetic algebraic geometry, we show that the Picard group of projective n-space is the integers, the automorphism group of projective n-space is PGL(n+1) and morphisms between projective standard spaces are given by homogenous polynomials in the usual way.


title: Numerical spectral analysis of standing waves in quantum hydrodynamics with viscosity

abstract: We study the spectrum of the linearization around standing wave profiles for two quantum hydrodynamics systems with linear and nonlinear viscosity. The essential spectrum for such profiles is stable; we investigate the point spectrum using an Evans function technique. For both systems we show numerically that there exists a real unstable eigenvalue, thus providing numerical evidence for spectral instability.


title: An Abstraction Hierarchy Toward Productive Quantum Programming

abstract: Experience from seven decades of classical computing suggests that a sustainable computer industry depends on a community of software engineers writing programs to address a wide variety of specific end-user needs, achieving both performance and utility in the process. Quantum computing is an emerging technology, and we do not yet have the insight to understand what quantum software tools and practices will best support researchers, software engineers, or applications specialists. Developers for today's quantum computers are grappling with the low-level details of the hardware, and progress towards scalable devices does not yet suggest what higher-level abstractions may look like. In this paper, we analyze and reframe the current state of the quantum software stack using the language of programming models. We propose an abstraction hierarchy to support quantum software engineering and discuss the consequences of overlaps across the programming, execution, and hardware models found in current technologies. We exercise this hierarchy for solving the eigenvalue estimation problem in two ways (a variational algorithm with error mitigation, and phase estimation with error correction) and pinpoint key differences in these approaches in terms of these layered models and their overlaps. While our work points to concrete conceptual challenges and gaps in quantum programming and proposes some specific steps forward, our primary thesis is that progress hinges on thinking about the abstraction hierarchy holistically, and not just about its components.


title: Fair Online Bilateral Trade

abstract: In online bilateral trade, a platform posts prices to incoming pairs of buyers and sellers that have private valuations for a certain good. If the price is lower than the buyers' valuation and higher than the sellers' valuation, then a trade takes place. Previous work focused on the platform perspective, with the goal of setting prices maximizing the gain from trade (the sum of sellers' and buyers' utilities). Gain from trade is, however, potentially unfair to traders, as they may receive highly uneven shares of the total utility. In this work we enforce fairness by rewarding the platform with the fair gain from trade, defined as the minimum between sellers' and buyers' utilities. After showing that any no-regret learning algorithm designed to maximize the sum of the utilities may fail badly with fair gain from trade, we present our main contribution: a complete characterization of the regret regimes for fair gain from trade when, after each interaction, the platform only learns whether each trader accepted the current price. Specifically, we prove the following regret bounds: $\Theta(\ln T)$ in the deterministic setting, $\Omega(T)$ in the stochastic setting, and $\tilde{\Theta}(T^{2/3})$ in the stochastic setting when sellers' and buyers' valuations are independent of each other. We conclude by providing tight regret bounds when, after each interaction, the platform is allowed to observe the true traders' valuations.


title: On the boundedness of generalized integration operators on Hardy spaces

abstract: We study the boundedness and compactness properties of the generalized integration operator $T_{g,a}$ when it acts between distinct Hardy spaces in the unit disc of the complex plane. This operator has been introduced by the first author in connection to a theorem of Cohn about factorization of higher order derivatives of functions in Hardy spaces. We answer in the affirmative a conjecture stated in the same work, therefore giving a complete characterization of the class of symbols $g$ for which the operator is bounded from the Hardy space $H^p$ to $H^q, \, 0<p,q<\infty.$


title: Algebraic Conditions for Stability in Runge-Kutta Methods and Their Certification via Semidefinite Programming

abstract: In this work, we present approaches to rigorously certify $A$- and $A(\alpha)$-stability in Runge-Kutta methods through the solution of convex feasibility problems defined by linear matrix inequalities. We adopt two approaches. The first is based on sum-of-squares programming applied to the Runge-Kutta $E$-polynomial and is applicable to both $A$- and $A(\alpha)$-stability. In the second, we sharpen the algebraic conditions for $A$-stability of Cooper, Scherer, T{\"u}rke, and Wendler to incorporate the Runge-Kutta order conditions. We demonstrate how the theoretical improvement enables the practical use of these conditions for certification of $A$-stability within a computational framework. We then use both approaches to obtain rigorous certificates of stability for several diagonally implicit schemes devised in the literature.


title: Towards Certification of Uncertainty Calibration under Adversarial Attacks

abstract: Since neural classifiers are known to be sensitive to adversarial perturbations that alter their accuracy, \textit{certification methods} have been developed to provide provable guarantees on the insensitivity of their predictions to such perturbations. Furthermore, in safety-critical applications, the frequentist interpretation of the confidence of a classifier (also known as model calibration) can be of utmost importance. This property can be measured via the Brier score or the expected calibration error. We show that attacks can significantly harm calibration, and thus propose certified calibration as worst-case bounds on calibration under adversarial perturbations. Specifically, we produce analytic bounds for the Brier score and approximate bounds via the solution of a mixed-integer program on the expected calibration error. Finally, we propose novel calibration attacks and demonstrate how they can improve model calibration through \textit{adversarial calibration training}.


title: Why Not Transform Chat Large Language Models to Non-English?

abstract: The scarcity of non-English data limits the development of non-English large language models (LLMs). Transforming English-centric LLMs to non-English has been identified as an effective and resource-efficient method. Previous works start from base LLMs and perform knowledge distillation (KD) with data generated by stronger LLMs, e.g. GPT-4. Compared to base LLMs, chat LLMs are further optimized for advanced abilities, e.g. multi-turn conversation and human preference alignment, and thus more powerful in both helpfulness and safety. However, transforming a chat LLM involves two critical issues: (1) How can we effectively transfer advanced abilities without their supervised data? (2) How can we prevent the original knowledge from catastrophic forgetting during transformation? We target these issues by introducing a simple framework called TransLLM. For the first issue, TransLLM divides the transfer problem into some common sub-tasks with the translation chain-of-thought, which uses the translation as the bridge between English and non-English step-by-step. We further enhance the performance of sub-tasks with publicly available data. For the second issue, we propose a method comprising two synergistic components: low-rank adaptation for training to maintain the original LLM parameters, and recovery KD, which utilizes data generated by the chat LLM itself to recover the original knowledge from the frozen parameters. In the experiments, we transform the LLaMA-2-chat-7B to the Thai language. Our method, using only single-turn data, outperforms strong baselines and ChatGPT on multi-turn benchmark MT-bench. Furthermore, our method, without safety data, rejects more harmful queries of safety benchmark AdvBench than both ChatGPT and GPT-4.


title: Narrative Review of Support for Emotional Expressions in Virtual Reality: Psychophysiology of speech-to-text interfaces

abstract: This narrative review on emotional expression in Speech-to-Text (STT) interfaces with Virtual Reality (VR) aims to identify advancements, limitations, and research gaps in incorporating emotional expression into transcribed text generated by STT systems. Using a rigorous search strategy, relevant articles published between 2020 and 2024 are extracted and categorized into themes such as communication enhancement technologies, innovations in captioning, emotion recognition in AR and VR, and empathic machines. The findings reveal the evolution of tools and techniques to meet the needs of individuals with hearing impairments, showcasing innovations in live transcription, closed captioning, AR, VR, and emotion recognition technologies. Despite improvements in accessibility, the absence of emotional nuance in transcribed text remains a significant communication challenge. The study underscores the urgency for innovations in STT technology to capture emotional expressions. The research discusses integrating emotional expression into text through strategies like animated text captions, emojilization tools, and models associating emotions with animation properties. Extending these efforts into AR and VR environments opens new possibilities for immersive and emotionally resonant experiences, especially in educational contexts. The study also explores empathic applications in healthcare, education, and human-robot interactions, highlighting the potential for personalized and effective interactions. The multidisciplinary nature of the literature underscores the potential for collaborative and interdisciplinary research.


title: Collective oscillations in a three-dimensional spin model with non-reciprocal interactions

abstract: We study the onset of collective oscillations at low temperature in a three-dimensional spin model with non-reciprocal short-range interactions. Performing numerical simulations of the model, the presence of a continuous phase transition to global oscillations is confirmed by a finite-size scaling analysis. By systematically varying the interaction range, we show that collective oscillations in this spin model actually result from two successive phase transitions: a mean-field phase transition over finite-size neighborhoods, which leads to the emergence of local noisy oscillators, and a synchronization transition of local noisy oscillators, which generates coherent macroscopic oscillations. Using a Fokker-Planck equation under a local mean-field approximation, we derive from the spin dynamics coupled Langevin equations for the complex amplitudes describing noisy oscillations on a mesoscopic scale. The phase diagram of these coupled equations is qualitatively obtained from a fully-connected (mean-field) approximation. This analytical approach allows us to clearly disentangle the onset of local and global oscillations, and to identify the two main control parameters, expressed as combinations of the microscopic parameters of the spin dynamics, that control the phase diagram of the model.


title: Some models are useful, but for how long?: A decision theoretic approach to choosing when to refit large-scale prediction models

abstract: Large-scale prediction models (typically using tools from artificial intelligence, AI, or machine learning, ML) are increasingly ubiquitous across a variety of industries and scientific domains. Such methods are often paired with detailed data from sources such as electronic health records, wearable sensors, and omics data (high-throughput technology used to understand biology). Despite their utility, implementing AI and ML tools at the scale necessary to work with this data introduces two major challenges. First, it can cost tens of thousands of dollars to train a modern AI/ML model at scale. Second, once the model is trained, its predictions may become less relevant as patient and provider behavior change, and predictions made for one geographical area may be less accurate for another. These two challenges raise a fundamental question: how often should you refit the AI/ML model to optimally trade-off between cost and relevance? Our work provides a framework for making decisions about when to {\it refit} AI/ML models when the goal is to maintain valid statistical inference (e.g. estimating a treatment effect in a clinical trial). Drawing on portfolio optimization theory, we treat the decision of {\it recalibrating} versus {\it refitting} the model as a choice between ''investing'' in one of two ''assets.'' One asset, recalibrating the model based on another model, is quick and relatively inexpensive but bears uncertainty from sampling and the possibility that the other model is not relevant to current circumstances. The other asset, {\it refitting} the model, is costly but removes the irrelevance concern (though not the risk of sampling error). We explore the balancing act between these two potential investments in this paper.


title: Memory Scraping Attack on Xilinx FPGAs: Private Data Extraction from Terminated Processes

abstract: FPGA-based hardware accelerators are becoming increasingly popular due to their versatility, customizability, energy efficiency, constant latency, and scalability. FPGAs can be tailored to specific algorithms, enabling efficient hardware implementations that effectively leverage algorithm parallelism. This can lead to significant performance improvements over CPUs and GPUs, particularly for highly parallel applications. For example, a recent study found that Stratix 10 FPGAs can achieve up to 90\% of the performance of a TitanX Pascal GPU while consuming less than 50\% of the power. This makes FPGAs an attractive choice for accelerating machine learning (ML) workloads. However, our research finds privacy and security vulnerabilities in existing Xilinx FPGA-based hardware acceleration solutions. These vulnerabilities arise from the lack of memory initialization and insufficient process isolation, which creates potential avenues for unauthorized access to private data used by processes. To illustrate this issue, we conducted experiments using a Xilinx ZCU104 board running the PetaLinux tool from Xilinx. We found that PetaLinux does not effectively clear memory locations associated with a terminated process, leaving them vulnerable to memory scraping attack (MSA). This paper makes two main contributions. The first contribution is an attack methodology of using the Xilinx debugger from a different user space. We find that we are able to access process IDs, virtual address spaces, and pagemaps of one user from a different user space because of lack of adequate process isolation. The second contribution is a methodology for characterizing terminated processes and accessing their private data. We illustrate this on Xilinx ML application library.


title: Counting the number of inequivalent arithmetic expressions on $n$ variables

abstract: An expression is any mathematical formula that contains certain formal variables and operations to be executed in a specified order. In computer science, it is usually convenient to represent each expression in the form of an expression tree. Here, we consider only arithmetic expressions, i.e., those that contain only the four standard arithmetic operations: addition, subtraction, multiplication and division, alongside additive inversion. We first provide certain theoretical results concerning the equivalence of such expressions and then disclose a $\Theta(n^2)$ algorithm that computes the number of inequivalent arithmetic expressions on $n$ distinct variables.


title: Vikhr: The Family of Open-Source Instruction-Tuned Large Language Models for Russian

abstract: There has been a surge in the development of various Large Language Models (LLMs). However, text generation for languages other than English often faces significant challenges, including poor generation quality and the reduced computational performance due to the disproportionate representation of tokens in model's vocabulary. In this work, we address these issues and introduce Vikhr, a new state-of-the-art open-source instruction-tuned LLM designed specifically for the Russian language. Unlike previous efforts for Russian that utilize computationally inexpensive LoRA adapters on top of English-oriented models, Vikhr features an adapted tokenizer vocabulary and undergoes the continued pre-training and instruction tuning of all weights. This approach not only enhances the model's performance but also significantly improves its computational and contextual efficiency. The remarkable performance of Vikhr across various Russian-language benchmarks can also be attributed to our efforts in expanding instruction datasets and corpora for continued pre-training. Vikhr not only sets the new state of the art among open-source LLMs for Russian, but even outperforms some proprietary closed-source models on certain benchmarks. The model weights, instruction sets, and code are publicly available


title: AlabOS: A Python-based Reconfigurable Workflow Management Framework for Autonomous Laboratories

abstract: The recent advent of autonomous laboratories, coupled with algorithms for high-throughput screening and active learning, promises to accelerate materials discovery and innovation. As these autonomous systems grow in complexity, the demand for robust and efficient workflow management software becomes increasingly critical. In this paper, we introduce AlabOS, a general-purpose software framework for orchestrating experiments and managing resources, with an emphasis on automated laboratories for materials synthesis and characterization. We demonstrate the implementation of AlabOS in a prototype autonomous materials laboratory. AlabOS features a reconfigurable experiment workflow model, enabling the simultaneous execution of varied workflows composed of modular tasks. Therefore, AlabOS is well-suited to handle the rapidly changing experimental protocols defining the progress of self-driving laboratory development for materials research.


title: A Methodology to Identify Physical or Computational Experiment Conditions for Uncertainty Mitigation

abstract: Complex engineering systems require integration of simulation of sub-systems and calculation of metrics to drive design decisions. This paper introduces a methodology for designing computational or physical experiments for system-level uncertainty mitigation purposes. The methodology follows a previously determined problem ontology, where physical, functional and modeling architectures are decided upon. By carrying out sensitivity analysis techniques utilizing system-level tools, critical epistemic uncertainties can be identified. Afterwards, a framework is introduced to design specific computational and physical experimentation for generating new knowledge about parameters, and for uncertainty mitigation. The methodology is demonstrated through a case study on an early-stage design Blended-Wing-Body (BWB) aircraft concept, showcasing how aerostructures analyses can be leveraged for mitigating system-level uncertainty, by computer experiments or guiding physical experimentation. The proposed methodology is versatile enough to tackle uncertainty management across various design challenges, highlighting the potential for more risk-informed design processes.


title: Chain of Targeted Verification Questions to Improve the Reliability of Code Generated by LLMs

abstract: LLM-based assistants, such as GitHub Copilot and ChatGPT, have the potential to generate code that fulfills a programming task described in a natural language description, referred to as a prompt. The widespread accessibility of these assistants enables users with diverse backgrounds to generate code and integrate it into software projects. However, studies show that code generated by LLMs is prone to bugs and may miss various corner cases in task specifications. Presenting such buggy code to users can impact their reliability and trust in LLM-based assistants. Moreover, significant efforts are required by the user to detect and repair any bug present in the code, especially if no test cases are available. In this study, we propose a self-refinement method aimed at improving the reliability of code generated by LLMs by minimizing the number of bugs before execution, without human intervention, and in the absence of test cases. Our approach is based on targeted Verification Questions (VQs) to identify potential bugs within the initial code. These VQs target various nodes within the Abstract Syntax Tree (AST) of the initial code, which have the potential to trigger specific types of bug patterns commonly found in LLM-generated code. Finally, our method attempts to repair these potential bugs by re-prompting the LLM with the targeted VQs and the initial code. Our evaluation, based on programming tasks in the CoderEval dataset, demonstrates that our proposed method outperforms state-of-the-art methods by decreasing the number of targeted errors in the code between 21% to 62% and improving the number of executable code instances to 13%.


title: Resurrection Attack: Defeating Xilinx MPU's Memory Protection

abstract: Memory protection units (MPUs) are hardware-assisted security features that are commonly used in embedded processors such as the ARM 940T, Infineon TC1775, and Xilinx Zynq. MPUs partition the memory statically, and set individual protection attributes for each partition. MPUs typically define two protection domains: user mode and supervisor mode. Normally, this is sufficient for protecting the kernel and applications. However, we have discovered a way to access a process memory due to a vulnerability in Xilinx MPU (XMPU) implementation that we call Resurrection Attack. We find that XMPU security policy protects user memory from unauthorized access when the user is active. However, when a user's session is terminated, the contents of the memory region of the terminated process are not cleared. An attacker can exploit this vulnerability by gaining access to the memory region after it has been reassigned. The attacker can read the data from the previous user's memory region, thereby compromising the confidentiality. To prevent the Resurrection Attack, the memory region of a terminated process must be cleared. However, this is not the case in the XMPU implementation, which allows our attack to succeed. The Resurrection Attack is a serious security flaw that could be exploited to steal sensitive data or gain unauthorized access to a system. It is important for users of Xilinx FPGAs to be aware of this vulnerability until this flaw is addressed.


title: Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models

abstract: Given the ubiquity of graph data, it is intriguing to ask: Is it possible to train a graph foundation model on a broad range of graph data across diverse domains? A major hurdle toward this goal lies in the fact that graphs from different domains often exhibit profoundly divergent characteristics. Although there have been some initial efforts in integrating multi-domain graphs for pre-training, they primarily rely on textual descriptions to align the graphs, limiting their application to text-attributed graphs. Moreover, different source domains may conflict or interfere with each other, and their relevance to the target domain can vary significantly. To address these issues, we propose MDGPT, a text free Multi-Domain Graph Pre-Training and adaptation framework designed to exploit multi-domain knowledge for graph learning. First, we propose a set of domain tokens to to align features across source domains for synergistic pre-training. Second, we propose a dual prompts, consisting of a unifying prompt and a mixing prompt, to further adapt the target domain with unified multi-domain knowledge and a tailored mixture of domain-specific knowledge. Finally, we conduct extensive experiments involving six public datasets to evaluate and analyze MDGPT, which outperforms prior art by up to 37.9%.


title: Out-of-plane magnetic phase diagram of Kitaev quantum spin liquid candidate Na2Co2TeO6

abstract: We have investigated the magnetic properties and mapped out the phase diagram of the honeycomb magnet Na2Co2TeO6 with Co 3d7 in out-of-plane magnetic fields. This material has previously been proposed to show nearest-neighbor Kitaev interactions between Co spins and maybe even Kitaev quantum spin liquid behavior in high fields. At low magnetic fields, we observe a thermal phase transition at TN = 27 K, transitioning from a paramagnetic state to a canonical ferrimagnetic state. Under the application of magnetic fields, a spin flop-like phase transition occurred before saturation of J = 1/2 between 10 K and TN. Below 10 K, a peak-dip-peak structure emerges between 10 and 17 T in the magnetic susceptibility (dM/dH) before the magnetic saturation, reminiscent of magnetic plateau behavior. The measurement of the magnetocaloric effect also shows dip-peak-dip behavior in this field range. Our data can be explained by an XXZ model with a single ion anisotropy and possibly small Kitaev and {\Gamma} exchange interactions. We also unambiguously determined the magnetization saturation field that helps constrain the energy scale of the exchange interactions


title: Nonisothermal Cahn-Hilliard Navier-Stokes system

abstract: In this research, we introduce and investigate an approximation method that preserves the structural integrity of the non-isothermal Cahn-Hilliard-Navier-Stokes system. Our approach extends a previously proposed technique [1], which utilizes conforming (inf-sup stable) finite elements in space, coupled with implicit time discretization employing convex-concave splitting. Expanding upon this method, we incorporate the unstable P1|P1 pair for the Navier-Stokes contributions, integrating Brezzi-Pitk\"aranta stabilization. Additionally, we improve the enforcement of incompressibility conditions through grad div stabilization. While these techniques are well-established for Navier-Stokes equations, it becomes apparent that for non-isothermal models, they introduce additional coupling terms to the equation governing internal energy. To ensure the conservation of total energy and maintain entropy production, these stabilization terms are appropriately integrated into the internal energy equation.


title: DyGPrompt: Learning Feature and Time Prompts on Dynamic Graphs

abstract: Dynamic graphs are pervasive in the real world, modeling dynamic relations between objects across various fields. For dynamic graph modeling, dynamic graph neural networks (DGNNs) have emerged as a mainstream technique, which are generally pre-trained on the link prediction task, leaving a significant gap from the objectives of downstream tasks such as node classification. To bridge the gap, prompt-based learning has gained traction on graphs. However, existing efforts focus on static graphs, neglecting the evolution of dynamic graphs. In this paper, we propose DyGPrompt, a novel pre-training and prompting framework for dynamic graph modeling. First, we design dual prompts to address the gap in both task objectives and dynamic variations across pre-training and downstream tasks. Second, we recognize that node and time features mutually characterize each other, and propose dual condition-nets to model the evolving node-time patterns in downstream tasks. Finally, we thoroughly evaluate and analyze DyGPrompt through extensive experiments on three public datasets.


title: eXmY: A Data Type and Technique for Arbitrary Bit Precision Quantization

abstract: eXmY is a novel data type for quantization of ML models. It supports both arbitrary bit widths and arbitrary integer and floating point formats. For example, it seamlessly supports 3, 5, 6, 7, 9 bit formats. For a specific bit width, say 7, it defines all possible formats e.g. e0m6, e1m5, e2m4, e3m3, e4m2, e5m1 and e6m0. For non-power of two bit widths e.g. 5, 6, 7, we created a novel encoding and decoding scheme which achieves perfect compression, byte addressability and is amenable to sharding and vector processing. We implemented libraries for emulation, encoding and decoding tensors and checkpoints in C++, TensorFlow, JAX and PAX. For optimal performance, the codecs use SIMD instructions on CPUs and vector instructions on TPUs and GPUs. eXmY is also a technique and exploits the statistical distribution of exponents in tensors. It can be used to quantize weights, static and dynamic activations, gradients, master weights and optimizer state. It can reduce memory (CPU DRAM and accelerator HBM), network and disk storage and transfers. It can increase multi tenancy and accelerate compute. eXmY has been deployed in production for almost 2 years.


title: Principal eigenstate classical shadows

abstract: Given many copies of an unknown quantum state $\rho$, we consider the task of learning a classical description of its principal eigenstate. Namely, assuming that $\rho$ has an eigenstate $|\phi\rangle$ with (unknown) eigenvalue $\lambda > 1/2$, the goal is to learn a (classical shadows style) classical description of $|\phi\rangle$ which can later be used to estimate expectation values $\langle \phi |O| \phi \rangle$ for any $O$ in some class of observables. We consider the sample-complexity setting in which generating a copy of $\rho$ is expensive, but joint measurements on many copies of the state are possible. We present a protocol for this task scaling with the principal eigenvalue $\lambda$ and show that it is optimal within a space of natural approaches, e.g., applying quantum state purification followed by a single-copy classical shadows scheme. Furthermore, when $\lambda$ is sufficiently close to $1$, the performance of our algorithm is optimal--matching the sample complexity for pure state classical shadows.


title: High-dimensional (Group) Adversarial Training in Linear Regression

abstract: Adversarial training can achieve robustness against adversarial perturbations and has been widely used in machine learning models. This paper delivers a non-asymptotic consistency analysis of the adversarial training procedure under $\ell_\infty$-perturbation in high-dimensional linear regression. It will be shown that the associated convergence rate of prediction error can achieve the minimax rate up to a logarithmic factor in the high-dimensional linear regression on the class of sparse parameters. Additionally, the group adversarial training procedure is analyzed. Compared with classic adversarial training, it will be proved that the group adversarial training procedure enjoys a better prediction error upper bound under certain group-sparsity patterns.


title: Distributed and Decentralized Control and Task Allocation for Flexible Swarms

abstract: This paper introduces a novel bio-mimetic approach for distributed control of robotic swarms, inspired by the collective behaviors of swarms in nature such as schools of fish and flocks of birds. The agents are assumed to have limited sensory perception, lack memory, be Identical, anonymous, and operate without interagent explicit communication. Despite these limitations, we demonstrate that collaborative exploration and task allocation can be executed by applying simple local rules of interactions between the agents. A comprehensive model comprised of agent, formation, and swarm layers is proposed in this paper, where each layer performs a specific function in shaping the swarm's collective behavior, thereby contributing to the emergence of the anticipated behaviors. We consider four principles combined in the design of the distributed control process: Cohesiveness, Flexibility, Attraction-Repulsion, and Peristaltic Motion. We design the control algorithms as reactive behaviour that enables the swarm to maintain connectivity, adapt to dynamic environments, spread out and cover a region with a size determined by the number of agents, and respond to various local task requirements. We explore some simple broadcast control-based steering methods, that result in inducing "anonymous ad-hoc leaders" among the agents, capable of guiding the swarm towards yet unexplored regions with further tasks. Our analysis is complemented by simulations, validating the efficacy of our algorithms. The experiments with various scenarios showcase the swarm`s capability to self-organize and perform tasks effectively under the proposed framework. The possible implementations include domains that necessitate emergent coordination and control in multi-agent systems, without the need for advanced individual abilities or direct communication.


title: Saturation properties for compositional truth with propositional correctness

abstract: It is an open question whether compositional truth with the principle of propositional soundness ,,all arithmetical sentences which are propositional tautologies are true'' is conservative over its arithmetical base theory. In this article, we show that the principle of propositional soundness imposes some saturation-like properties on the underlying model, thus showing significant limitations to the possible conservativity proof.


title: DoGaussian: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensus

abstract: The recent advances in 3D Gaussian Splatting (3DGS) show promising results on the novel view synthesis (NVS) task. With its superior rendering performance and high-fidelity rendering quality, 3DGS is excelling at its previous NeRF counterparts. The most recent 3DGS method focuses either on improving the instability of rendering efficiency or reducing the model size. On the other hand, the training efficiency of 3DGS on large-scale scenes has not gained much attention. In this work, we propose DoGaussian, a method that trains 3DGS distributedly. Our method first decomposes a scene into K blocks and then introduces the Alternating Direction Method of Multipliers (ADMM) into the training procedure of 3DGS. During training, our DoGaussian maintains one global 3DGS model on the master node and K local 3DGS models on the slave nodes. The K local 3DGS models are dropped after training and we only query the global 3DGS model during inference. The training time is reduced by scene decomposition, and the training convergence and stability are guaranteed through the consensus on the shared 3D Gaussians. Our method accelerates the training of 3DGS by 6+ times when evaluated on large-scale scenes while concurrently achieving state-of-the-art rendering quality. Our project page is available at https://aibluefisher.github.io/DoGaussian.


title: A Survey on Design-space Dimensionality Reduction Methods for Shape Optimization

abstract: The rapidly evolving field of engineering design of functional surfaces necessitates sophisticated tools to manage the inherent complexity of high-dimensional design spaces. This review delves into the field of design-space dimensionality reduction techniques tailored for shape optimization, bridging traditional methods and cutting-edge technologies. Dissecting the spectrum of these techniques, from classical linear approaches like principal component analysis to more nuanced nonlinear methods such as autoencoders, the discussion extends to innovative physics-informed methods that integrate physical data into the dimensionality reduction process, enhancing the predictive accuracy and relevance of reduced models. By integrating these methods into optimization frameworks, it is shown how they significantly mitigate the curse of dimensionality, streamline computational processes, and refine the exploration and optimization of complex functional surfaces. The survey provides a classification of method and highlights the transformative impact of these techniques in simplifying design challenges, thereby fostering more efficient and effective engineering solutions.


title: Exogenous Consideration and Extended Random Utility

abstract: In a consideration set model, an individual maximizes utility among the considered alternatives. I relate a consideration set additive random utility model to classic discrete choice and the extended additive random utility model, in which utility can be $-\infty$ for infeasible alternatives. When observable utility shifters are bounded, all three models are observationally equivalent. Moreover, they have the same counterfactual bounds and welfare formulas for changes in utility shifters like price. For attention interventions, welfare cannot change in the full consideration model but is completely unbounded in the limited consideration model. The identified set for consideration set probabilities has a minimal width for any bounded support of shifters, but with unbounded support it is a point: identification "towards" infinity does not resemble identification "at" infinity.


title: Coded Computing Meets Quantum Circuit Simulation: Coded Parallel Tensor Network Contraction Algorithm

abstract: Parallel tensor network contraction algorithms have emerged as the pivotal benchmarks for assessing the classical limits of computation, exemplified by Google's demonstration of quantum supremacy through random circuit sampling. However, the massive parallelization of the algorithm makes it vulnerable to computer node failures. In this work, we apply coded computing to a practical parallel tensor network contraction algorithm. To the best of our knowledge, this is the first attempt to code tensor network contractions. Inspired by matrix multiplication codes, we provide two coding schemes: 2-node code for practicality in quantum simulation and hyperedge code for generality. Our 2-node code successfully achieves significant gain for $f$-resilient number compared to naive replication, proportional to both the number of node failures and the dimension product of sliced indices. Our hyperedge code can cover tensor networks out of the scope of quantum, with degraded gain in the exchange of its generality.


title: Leader Reward for POMO-Based Neural Combinatorial Optimization

abstract: Deep neural networks based on reinforcement learning (RL) for solving combinatorial optimization (CO) problems are developing rapidly and have shown a tendency to approach or even outperform traditional solvers. However, existing methods overlook an important distinction: CO problems differ from other traditional problems in that they focus solely on the optimal solution provided by the model within a specific length of time, rather than considering the overall quality of all solutions generated by the model. In this paper, we propose Leader Reward and apply it during two different training phases of the Policy Optimization with Multiple Optima (POMO) model to enhance the model's ability to generate optimal solutions. This approach is applicable to a variety of CO problems, such as the Traveling Salesman Problem (TSP), the Capacitated Vehicle Routing Problem (CVRP), and the Flexible Flow Shop Problem (FFSP), but also works well with other POMO-based models or inference phase's strategies. We demonstrate that Leader Reward greatly improves the quality of the optimal solutions generated by the model. Specifically, we reduce the POMO's gap to the optimum by more than 100 times on TSP100 with almost no additional computational overhead.


title: Embodied Design for Enhanced Flipper-Based Locomotion in Complex Terrains

abstract: Robots are becoming increasingly essential for traversing complex environments such as disaster areas, extraterrestrial terrains, and marine environments. Yet, their potential is often limited by mobility and adaptability constraints. In nature, various animals have evolved finely tuned designs and anatomical features that enable efficient locomotion in diverse environments. Sea turtles, for instance, possess specialized flippers that facilitate both long-distance underwater travel and adept maneuvers across a range of coastal terrains. Building on the principles of embodied intelligence and drawing inspiration from sea turtle hatchings, this paper examines the critical interplay between a robot's physical form and its environmental interactions, focusing on how morphological traits and locomotive behaviors affect terrestrial navigation. We present a bio-inspired robotic system and study the impacts of flipper/body morphology and gait patterns on its terrestrial mobility across diverse terrains ranging from sand to rocks. Evaluating key performance metrics such as speed and cost of transport, our experimental results highlight adaptive designs as crucial for multi-terrain robotic mobility to achieve not only speed and efficiency but also the versatility needed to tackle the varied and complex terrains encountered in real-world applications.


title: PitVQA: Image-grounded Text Embedding LLM for Visual Question Answering in Pituitary Surgery

abstract: Visual Question Answering (VQA) within the surgical domain, utilizing Large Language Models (LLMs), offers a distinct opportunity to improve intra-operative decision-making and facilitate intuitive surgeon-AI interaction. However, the development of LLMs for surgical VQA is hindered by the scarcity of diverse and extensive datasets with complex reasoning tasks. Moreover, contextual fusion of the image and text modalities remains an open research challenge due to the inherent differences between these two types of information and the complexity involved in aligning them. This paper introduces PitVQA, a novel dataset specifically designed for VQA in endonasal pituitary surgery and PitVQA-Net, an adaptation of the GPT2 with a novel image-grounded text embedding for surgical VQA. PitVQA comprises 25 procedural videos and a rich collection of question-answer pairs spanning crucial surgical aspects such as phase and step recognition, context understanding, tool detection and localization, and tool-tissue interactions. PitVQA-Net consists of a novel image-grounded text embedding that projects image and text features into a shared embedding space and GPT2 Backbone with an excitation block classification head to generate contextually relevant answers within the complex domain of endonasal pituitary surgery. Our image-grounded text embedding leverages joint embedding, cross-attention and contextual representation to understand the contextual relationship between questions and surgical images. We demonstrate the effectiveness of PitVQA-Net on both the PitVQA and the publicly available EndoVis18-VQA dataset, achieving improvements in balanced accuracy of 8% and 9% over the most recent baselines, respectively. Our code and dataset is available at https://github.com/mobarakol/PitVQA.


title: Actor-critic algorithms for fiber sampling problems

abstract: We propose an actor-critic algorithm for a family of complex problems arising in algebraic statistics and discrete optimization. The core task is to produce a sample from a finite subset of the non-negative integer lattice defined by a high-dimensional polytope. We translate the problem into a Markov decision process and devise an actor-critic reinforcement learning (RL) algorithm to learn a set of good moves that can be used for sampling. We prove that the actor-critic algorithm converges to an approximately optimal sampling policy. To tackle complexity issues that typically arise in these sampling problems, and to allow the RL to function at scale, our solution strategy takes three steps: decomposing the starting point of the sample, using RL on each induced subproblem, and reconstructing to obtain a sample in the original polytope. In this setup, the proof of convergence applies to each subproblem in the decomposition. We test the method in two regimes. In statistical applications, a high-dimensional polytope arises as the support set for the reference distribution in a model/data fit test for a broad family of statistical models for categorical data. We demonstrate how RL can be used for model fit testing problems for data sets for which traditional MCMC samplers converge too slowly due to problem size and sparsity structure. To test the robustness of the algorithm and explore its generalization properties, we apply it to synthetically generated data of various sizes and sparsity levels.


title: Text Prompting for Multi-Concept Video Customization by Autoregressive Generation

abstract: We present a method for multi-concept customization of pretrained text-to-video (T2V) models. Intuitively, the multi-concept customized video can be derived from the (non-linear) intersection of the video manifolds of the individual concepts, which is not straightforward to find. We hypothesize that sequential and controlled walking towards the intersection of the video manifolds, directed by text prompting, leads to the solution. To do so, we generate the various concepts and their corresponding interactions, sequentially, in an autoregressive manner. Our method can generate videos of multiple custom concepts (subjects, action and background) such as a teddy bear running towards a brown teapot, a dog playing violin and a teddy bear swimming in the ocean. We quantitatively evaluate our method using videoCLIP and DINO scores, in addition to human evaluation. Videos for results presented in this paper can be found at https://github.com/divyakraman/MultiConceptVideo2024.


title: Spectral Adapter: Fine-Tuning in Spectral Space

abstract: Recent developments in Parameter-Efficient Fine-Tuning (PEFT) methods for pretrained deep neural networks have captured widespread interest. In this work, we study the enhancement of current PEFT methods by incorporating the spectral information of pretrained weight matrices into the fine-tuning procedure. We investigate two spectral adaptation mechanisms, namely additive tuning and orthogonal rotation of the top singular vectors, both are done via first carrying out Singular Value Decomposition (SVD) of pretrained weights and then fine-tuning the top spectral space. We provide a theoretical analysis of spectral fine-tuning and show that our approach improves the rank capacity of low-rank adapters given a fixed trainable parameter budget. We show through extensive experiments that the proposed fine-tuning model enables better parameter efficiency and tuning performance as well as benefits multi-adapter fusion. The code will be open-sourced for reproducibility.


title: Decay of excess for the abelian Higgs model

abstract: In this article we prove that entire critical points $(u,\nabla)$ of the self-dual $U(1)$-Yang-Mills-Higgs functional $E_1$, with energy $$E_1(u,\nabla;B_R):=\int_{B_R}\left[|\nabla u|^2+\frac{(1-|u|^2)^2}{4}+|F_\nabla|^2\right]\leq(2\pi+\tau(n)) \omega_{n-2}R^{n-2}$$ for all $R>0$, have unique blow-down. Moreover, we show that they are two-dimensional in ambient dimension $2\leq n\leq4$, or in any dimension $n\ge2$ assuming that $(u,\nabla)$ is a local minimizer, thus establishing a co-dimension-two analogue of Savin's theorem. The main ingredient is an Allard-type improvement of flatness.


title: What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions

abstract: Large language models (LLMs) are trained on a vast amount of human-written data, but data providers often remain uncredited. In response to this issue, data valuation (or data attribution), which quantifies the contribution or value of each data to the model output, has been discussed as a potential solution. Nevertheless, applying existing data valuation methods to recent LLMs and their vast training datasets has been largely limited by prohibitive compute and memory costs. In this work, we focus on influence functions, a popular gradient-based data valuation method, and significantly improve its scalability with an efficient gradient projection strategy called LoGra that leverages the gradient structure in backpropagation. We then provide a theoretical motivation of gradient projection approaches to influence functions to promote trust in the data valuation process. Lastly, we lower the barrier to implementing data valuation systems by introducing LogIX, a software package that can transform existing training code into data valuation code with minimal effort. In our data valuation experiments, LoGra achieves competitive accuracy against more expensive baselines while showing up to 6,500x improvement in throughput and 5x reduction in GPU memory usage when applied to Llama3-8B-Instruct and the 1B-token dataset.


title: Cognitive Internet of Vulnerable Road Users in Traffic: Predictive Neural Modulations of Road Crossing Intention

abstract: Vulnerable Road Users (VRUs) present a significant challenge for road safety due to the frequent unpredictability of their behaviors. In typical Intelligent Transportation Systems, vision-based approaches supported by networked cameras are often used to anticipate VRUs motion intentions and trajectories. However, several limitations posed by occlusions and distractions set a boundary for the efficacy of such methods. To address these challenges, this study introduces a framework that leverages data collected using wearable neurophysiological sensors on VRUs to integrate them seamlessly into the Vehicle-to-Everything communication framework. This integration empowers VRUs to autonomously broadcast their intended movements to other road agents, especially autonomous vehicles, thereby bridging a critical gap in current vehicular communication systems. To validate this concept, we conducted an experiment involving 12 participants, from whom EEG signals were collected as they engaged in road-crossing decisions within simulated environments. Employing Hidden Markov Models, we identified four cognitive stages intrinsic to a pedestrian's decision-making process. Our statistical analysis further revealed significant variations in EEG activities across these stages, shedding light on the neural correlates and cognitive dynamics underpinning pedestrian road-crossing behavior. We then developed a predictive cognitive model using dynamic time warping and K-nearest neighbors algorithms, optimized through a data-driven sliding window approach. This model demonstrated high predictive accuracy, evidenced by an Area Under the Curve of 0.91, indicating its capability to anticipate pedestrian road-crossing actions approximately 1 second in advance of any pedestrian movement. This research paves the way for a novel VRU-Vehicle interaction paradigm and signifies a shift towards a forward-thinking ecosystem.


title: Attention as an RNN

abstract: The advent of Transformers marked a significant breakthrough in sequence modelling, providing a highly performant architecture capable of leveraging GPU parallelism. However, Transformers are computationally expensive at inference time, limiting their applications, particularly in low-resource settings (e.g., mobile and embedded devices). Addressing this, we (1) begin by showing that attention can be viewed as a special Recurrent Neural Network (RNN) with the ability to compute its \textit{many-to-one} RNN output efficiently. We then (2) show that popular attention-based models such as Transformers can be viewed as RNN variants. However, unlike traditional RNNs (e.g., LSTMs), these models cannot be updated efficiently with new tokens, an important property in sequence modelling. Tackling this, we (3) introduce a new efficient method of computing attention's \textit{many-to-many} RNN output based on the parallel prefix scan algorithm. Building on the new attention formulation, we (4) introduce \textbf{Aaren}, an attention-based module that can not only (i) be trained in parallel (like Transformers) but also (ii) be updated efficiently with new tokens, requiring only constant memory for inferences (like traditional RNNs). Empirically, we show Aarens achieve comparable performance to Transformers on $38$ datasets spread across four popular sequential problem settings: reinforcement learning, event forecasting, time series classification, and time series forecasting tasks while being more time and memory-efficient.


title: Exploring the Relationship Between Feature Attribution Methods and Model Performance

abstract: Machine learning and deep learning models are pivotal in educational contexts, particularly in predicting student success. Despite their widespread application, a significant gap persists in comprehending the factors influencing these models' predictions, especially in explainability within education. This work addresses this gap by employing nine distinct explanation methods and conducting a comprehensive analysis to explore the correlation between the agreement among these methods in generating explanations and the predictive model's performance. Applying Spearman's correlation, our findings reveal a very strong correlation between the model's performance and the agreement level observed among the explanation methods.


title: Construction of a Geometric Basis of K\"{a}hler differentials for plane branches

abstract: We show that there is a basis of the set of K\"{a}hler differentials of an irreducible germ of holomorphic plane curve whose non-trivial elements correspond to dicritical foliations. Indeed, we discuss several concepts of generation for the semimodule of values of K\"{a}hler differentials of the curve and provide basis of K\"{a}hler differentials, for every of these concepts, whose geometric properties are described. Moreover, we give an algorithmic construction of the bases.


title: Decision Trees for Intuitive Intraday Trading Strategies

abstract: This research paper aims to investigate the efficacy of decision trees in constructing intraday trading strategies using existing technical indicators for individual equities in the NIFTY50 index. Unlike conventional methods that rely on a fixed set of rules based on combinations of technical indicators developed by a human trader through their analysis, the proposed approach leverages decision trees to create unique trading rules for each stock, potentially enhancing trading performance and saving time. By extensively backtesting the strategy for each stock, a trader can determine whether to employ the rules generated by the decision tree for that specific stock. While this method does not guarantee success for every stock, decision treebased strategies outperform the simple buy-and-hold strategy for many stocks. The results highlight the proficiency of decision trees as a valuable tool for enhancing intraday trading performance on a stock-by-stock basis and could be of interest to traders seeking to improve their trading strategies.


title: Learning To Play Atari Games Using Dueling Q-Learning and Hebbian Plasticity

abstract: In this work, an advanced deep reinforcement learning architecture is used to train neural network agents playing atari games. Given only the raw game pixels, action space, and reward information, the system can train agents to play any Atari game. At first, this system uses advanced techniques like deep Q-networks and dueling Q-networks to train efficient agents, the same techniques used by DeepMind to train agents that beat human players in Atari games. As an extension, plastic neural networks are used as agents, and their feasibility is analyzed in this scenario. The plasticity implementation was based on backpropagation and the Hebbian update rule. Plastic neural networks have excellent features like lifelong learning after the initial training, which makes them highly suitable in adaptive learning environments. As a new analysis of plasticity in this context, this work might provide valuable insights and direction for future works.


title: SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data

abstract: Decentralized training enables learning with distributed datasets generated at different locations without relying on a central server. In realistic scenarios, the data distribution across these sparsely connected learning agents can be significantly heterogeneous, leading to local model over-fitting and poor global model generalization. Another challenge is the high communication cost of training models in such a peer-to-peer fashion without any central coordination. In this paper, we jointly tackle these two-fold practical challenges by proposing SADDLe, a set of sharpness-aware decentralized deep learning algorithms. SADDLe leverages Sharpness-Aware Minimization (SAM) to seek a flatter loss landscape during training, resulting in better model generalization as well as enhanced robustness to communication compression. We present two versions of our approach and conduct extensive experiments to show that SADDLe leads to 1-20% improvement in test accuracy compared to other existing techniques. Additionally, our proposed approach is robust to communication compression, with an average drop of only 1% in the presence of up to 4x compression.


title: Learning heavy-tailed distributions with Wasserstein-proximal-regularized $\alpha$-divergences

abstract: In this paper, we propose Wasserstein proximals of $\alpha$-divergences as suitable objective functionals for learning heavy-tailed distributions in a stable manner. First, we provide sufficient, and in some cases necessary, relations among data dimension, $\alpha$, and the decay rate of data distributions for the Wasserstein-proximal-regularized divergence to be finite. Finite-sample convergence rates for the estimation in the case of the Wasserstein-1 proximal divergences are then provided under certain tail conditions. Numerical experiments demonstrate stable learning of heavy-tailed distributions -- even those without first or second moment -- without any explicit knowledge of the tail behavior, using suitable generative models such as GANs and flow-based models related to our proposed Wasserstein-proximal-regularized $\alpha$-divergences. Heuristically, $\alpha$-divergences handle the heavy tails and Wasserstein proximals allow non-absolute continuity between distributions and control the velocities of flow-based algorithms as they learn the target distribution deep into the tails.


title: Active dynamics of charged macromolecules

abstract: We study the role of active coupling on the transport properties of homogeneously charged macromolecules in an infinitely dilute solution. An enzyme becomes actively bound to a segment of the macromolecule, exerting an electrostatic force on it. Eventually, thermal fluctuations cause it to become unbound, introducing active coupling into the system. We study the mean-squared displacement (MSD) and find a new scaling regime compared to the thermal counterpart in the presence of hydrodynamic and segment-segment electrostatic interactions. Furthermore, the study of segment-segment equal-time correlation reveals the swelling of the macromolecule. Further, we derive the concentration equation of the macromolecule with active binding and study how the cooperative diffusivity of the macromolecules get modified by its environment, including the macromolecules itself. It turns out that these active fluctuations enhance the effective diffusivity of the macromolecules. The derived closed-form expression for diffusion constant is pertinent to the accurate interpretation of light scattering data in multi-component systems with binding-unbinding equilibria.


abstract: Offline model-based optimization (MBO) aims to maximize a black-box objective function using only an offline dataset of designs and scores. A prevalent approach involves training a conditional generative model on existing designs and their associated scores, followed by the generation of new designs conditioned on higher target scores. However, these newly generated designs often underperform due to the lack of high-scoring training data. To address this challenge, we introduce a novel method, Design Editing for Offline Model-based Optimization (DEMO), which consists of two phases. In the first phase, termed pseudo-target distribution generation, we apply gradient ascent on the offline dataset using a trained surrogate model, producing a synthetic dataset where the predicted scores serve as new labels. A conditional diffusion model is subsequently trained on this synthetic dataset to capture a pseudo-target distribution, which enhances the accuracy of the conditional diffusion model in generating higher-scoring designs. Nevertheless, the pseudo-target distribution is susceptible to noise stemming from inaccuracies in the surrogate model, consequently predisposing the conditional diffusion model to generate suboptimal designs. We hence propose the second phase, existing design editing, to directly incorporate the high-scoring features from the offline dataset into design generation. In this phase, top designs from the offline dataset are edited by introducing noise, which are subsequently refined using the conditional diffusion model to produce high-scoring designs. Overall, high-scoring designs begin with inheriting high-scoring features from the second phase and are further refined with a more accurate conditional diffusion model in the first phase. Empirical evaluations on 7 offline MBO tasks show that DEMO outperforms various baseline methods.


title: Unleashing the Power of Unlabeled Data: A Self-supervised Learning Framework for Cyber Attack Detection in Smart Grids

abstract: Modern power grids are undergoing significant changes driven by information and communication technologies (ICTs), and evolving into smart grids with higher efficiency and lower operation cost. Using ICTs, however, comes with an inevitable side effect that makes the power system more vulnerable to cyber attacks. In this paper, we propose a self-supervised learning-based framework to detect and identify various types of cyber attacks. Different from existing approaches, the proposed framework does not rely on large amounts of well-curated labeled data but makes use of the massive unlabeled data in the wild which are easily accessible. Specifically, the proposed framework adopts the BERT model from the natural language processing domain and learns generalizable and effective representations from the unlabeled sensing data, which capture the distinctive patterns of different attacks. Using the learned representations, together with a very small amount of labeled data, we can train a task-specific classifier to detect various types of cyber attacks. Meanwhile, real-world training datasets are usually imbalanced, i.e., there are only a limited number of data samples containing attacks. In order to cope with such data imbalance, we propose a new loss function, separate mean error (SME), which pays equal attention to the large and small categories to better train the model. Experiment results in a 5-area power grid system with 37 buses demonstrate the superior performance of our framework over existing approaches, especially when a very limited portion of labeled data are available, e.g., as low as 0.002\%. We believe such a framework can be easily adopted to detect a variety of cyber attacks in other power grid scenarios.


title: On the Brittle Foundations of ReAct Prompting for Agentic Large Language Models

abstract: The reasoning abilities of Large Language Models (LLMs) remain a topic of debate. Some methods such as ReAct-based prompting, have gained popularity for claiming to enhance sequential decision-making abilities of agentic LLMs. However, it is unclear what is the source of improvement in LLM reasoning with ReAct based prompting. In this paper we examine these claims of ReAct based prompting in improving agentic LLMs for sequential decision-making. By introducing systematic variations to the input prompt we perform a sensitivity analysis along the claims of ReAct and find that the performance is minimally influenced by the "interleaving reasoning trace with action execution" or the content of the generated reasoning traces in ReAct, contrary to original claims and common usage. Instead, the performance of LLMs is driven by the similarity between input example tasks and queries, implicitly forcing the prompt designer to provide instance-specific examples which significantly increases the cognitive burden on the human. Our investigation shows that the perceived reasoning abilities of LLMs stem from the exemplar-query similarity and approximate retrieval rather than any inherent reasoning abilities.


title: DeTox: Toxic Subspace Projection for Model Editing

abstract: Recent alignment algorithms such as direct preference optimization (DPO) have been developed to improve the safety of large language models (LLMs) by training these models to match human behaviors exemplified by preference data. However, these methods are both computationally intensive and lacking in controllability and transparency, making them prone to jailbreaking and inhibiting their widespread use. Furthermore, these tuning-based methods require large-scale preference data for training and are susceptible to noisy preference data. In this paper, we introduce a tuning-free alignment alternative (DeTox) and demonstrate its effectiveness under the use case of toxicity reduction. Grounded on theory from factor analysis, DeTox is a sample-efficient model editing approach that identifies a toxic subspace in the model parameter space and reduces model toxicity by projecting away the detected subspace. The toxic sub-space is identified by extracting preference data embeddings from the language model, and removing non-toxic information from these embeddings. We show that DeTox is more sample-efficient than DPO, further showcasing greater robustness to noisy data. Finally, we establish both theoretical and empirical connections between DeTox and DPO, showing that DeTox can be interpreted as a denoised version of a single DPO step.


title: TaleMate: Exploring the use of Voice Agents for Parent-Child Joint Reading Experiences

abstract: Joint reading is a key activity for early learners, with caregiver-child interactions such as questioning and feedback playing an essential role in children's cognitive and linguistic development. However, for some parents, actively engaging children in storytelling can be challenging. To address this, we introduce TaleMate a platform designed to enhance shared reading by leveraging conversational agents that have been shown to support children's engagement and learning. TaleMate enables a dynamic, participatory reading experience where parents and children can choose which characters they wish to embody. Moreover, the system navigates the challenges posed by digital reading tools, such as decreased parent-child interaction, and builds upon the benefits of traditional and digital reading techniques. TaleMate offers an innovative approach to fostering early reading habits, bridging the gap between traditional joint reading practices and the digital reading landscape.


title: Uncertainty-Aware DRL for Autonomous Vehicle Crowd Navigation in Shared Space

abstract: Safe, socially compliant, and efficient navigation of low-speed autonomous vehicles (AVs) in pedestrian-rich environments necessitates considering pedestrians' future positions and interactions with the vehicle and others. Despite the inevitable uncertainties associated with pedestrians' predicted trajectories due to their unobserved states (e.g., intent), existing deep reinforcement learning (DRL) algorithms for crowd navigation often neglect these uncertainties when using predicted trajectories to guide policy learning. This omission limits the usability of predictions when diverging from ground truth. This work introduces an integrated prediction and planning approach that incorporates the uncertainties of predicted pedestrian states in the training of a model-free DRL algorithm. A novel reward function encourages the AV to respect pedestrians' personal space, decrease speed during close approaches, and minimize the collision probability with their predicted paths. Unlike previous DRL methods, our model, designed for AV operation in crowded spaces, is trained in a novel simulation environment that reflects realistic pedestrian behaviour in a shared space with vehicles. Results show a 40% decrease in collision rate and a 15% increase in minimum distance to pedestrians compared to the state of the art model that does not account for prediction uncertainty. Additionally, the approach outperforms model predictive control methods that incorporate the same prediction uncertainties in terms of both performance and computational time, while producing trajectories closer to human drivers in similar scenarios.


title: Conformal uncertainty quantification using kernel depth measures in separable Hilbert spaces

abstract: Depth measures have gained popularity in the statistical literature for defining level sets in complex data structures like multivariate data, functional data, and graphs. Despite their versatility, integrating depth measures into regression modeling for establishing prediction regions remains underexplored. To address this gap, we propose a novel method utilizing a model-free uncertainty quantification algorithm based on conditional depth measures and conditional kernel mean embeddings. This enables the creation of tailored prediction and tolerance regions in regression models handling complex statistical responses and predictors in separable Hilbert spaces. Our focus in this paper is exclusively on examples where the response is a functional data object. To enhance practicality, we introduce a conformal prediction algorithm, providing non-asymptotic guarantees in the derived prediction region. Additionally, we establish both conditional and unconditional consistency results and fast convergence rates in some special homoscedastic cases. We evaluate the model finite sample performance in extensive simulation studies with different function objects as probability distributions and functional data. Finally, we apply the approach in a digital health application related to physical activity, aiming to offer personalized recommendations in the US. population based on individuals' characteristics.


title: Cellular-resolution X-ray microtomography of an entire mouse brain

abstract: Purpose: Histology is the gold standard for sub-cellular visualization of the mouse brain. It offers excellent in-plane resolution, but a comparably low out-of-plane resolution due to physical sectioning. X-ray microtomography does not require this trade-off. Tomographic imaging of the entire mouse brain with isotropic cellular resolution produces datasets of multiple terabytes in size. These data must be processed and made accessible to domain experts who may have only limited image processing knowledge. Approach: Extended-field X-ray microtomography covering an entire mouse brain was performed. The 4,495 projections from 8 $\times$ 8 offset acquisitions were stitched to reconstruct a volume of 15,000$^3$ voxels. The microtomography volume was non-rigidly registered to the Allen Mouse Brain Common Coordinate Framework v3 based on a combination of image intensity and landmark pairs. Results: We present a 3.3 teravoxel dataset covering a full mouse brain with 0.65 $\mu$m voxel size. The data were blockwise transformed to a common coordinate system, then stored in a public repository with a hierarchical format for navigation and overlay with anatomical annotations in online viewers such as Neuroglancer or siibra-explorer. Conclusions: This study demonstrates X-ray imaging and data processing for a full mouse brain, augmenting current atlases by improving resolution in the third dimension by an order of magnitude. The data are publicly available and easily accessible for domain experts via browser-based viewers.


title: Infinite-Dimensional Feature Interaction

abstract: The past neural network design has largely focused on feature representation space dimension and its capacity scaling (e.g., width, depth), but overlooked the feature interaction space scaling. Recent advancements have shown shifted focus towards element-wise multiplication to facilitate higher-dimensional feature interaction space for better information transformation. Despite this progress, multiplications predominantly capture low-order interactions, thus remaining confined to a finite-dimensional interaction space. To transcend this limitation, classic kernel methods emerge as a promising solution to engage features in an infinite-dimensional space. We introduce InfiNet, a model architecture that enables feature interaction within an infinite-dimensional space created by RBF kernel. Our experiments reveal that InfiNet achieves new state-of-the-art, owing to its capability to leverage infinite-dimensional interactions, significantly enhancing model performance.


title: Numerical Simulations of 3D Ion Crystal Dynamics in a Penning Trap using the Fast Multipole Method

abstract: We simulate the dynamics, including laser cooling, of 3D ion crystals confined in a Penning trap using a newly developed molecular dynamics-like code. The numerical integration of the ions' equations of motion is accelerated using the fast multipole method to calculate the Coulomb interaction between ions, which allows us to efficiently study large ion crystals with thousands of ions. In particular, we show that the simulation time scales linearly with ion number, rather than with the square of the ion number. By treating the ions' absorption of photons as a Poisson process, we simulate individual photon scattering events to study laser cooling of 3D ellipsoidal ion crystals. Initial simulations suggest that these crystals can be efficiently cooled to ultracold temperatures, aided by the mixing of the easily cooled axial motional modes with the low frequency planar modes. In our simulations of a spherical crystal of 1,000 ions, the planar kinetic energy is cooled to several millikelvin in a few milliseconds while the axial kinetic energy and total potential energy are cooled even further. This suggests that 3D ion crystals could be well-suited as platforms for future quantum science experiments.


title: CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Models

abstract: This paper introduces the "CIVICS: Culturally-Informed & Values-Inclusive Corpus for Societal impacts" dataset, designed to evaluate the social and cultural variation of Large Language Models (LLMs) across multiple languages and value-sensitive topics. We create a hand-crafted, multilingual dataset of value-laden prompts which address specific socially sensitive topics, including LGBTQI rights, social welfare, immigration, disability rights, and surrogacy. CIVICS is designed to generate responses showing LLMs' encoded and implicit values. Through our dynamic annotation processes, tailored prompt design, and experiments, we investigate how open-weight LLMs respond to value-sensitive issues, exploring their behavior across diverse linguistic and cultural contexts. Using two experimental set-ups based on log-probabilities and long-form responses, we show social and cultural variability across different LLMs. Specifically, experiments involving long-form responses demonstrate that refusals are triggered disparately across models, but consistently and more frequently in English or translated statements. Moreover, specific topics and sources lead to more pronounced differences across model answers, particularly on immigration, LGBTQI rights, and social welfare. As shown by our experiments, the CIVICS dataset aims to serve as a tool for future research, promoting reproducibility and transparency across broader linguistic settings, and furthering the development of AI technologies that respect and reflect global cultural diversities and value pluralism. The CIVICS dataset and tools will be made available upon publication under open licenses; an anonymized version is currently available at https://huggingface.co/CIVICS-dataset.


title: There is HOPE to Avoid HiPPOs for Long-memory State Space Models

abstract: State-space models (SSMs) that utilize linear, time-invariant (LTI) systems are known for their effectiveness in learning long sequences. However, these models typically face several challenges: (i) they require specifically designed initializations of the system matrices to achieve state-of-the-art performance, (ii) they require training of state matrices on a logarithmic scale with very small learning rates to prevent instabilities, and (iii) they require the model to have exponentially decaying memory in order to ensure an asymptotically stable LTI system. To address these issues, we view SSMs through the lens of Hankel operator theory, which provides us with a unified theory for the initialization and training of SSMs. Building on this theory, we develop a new parameterization scheme, called HOPE, for LTI systems that utilizes Markov parameters within Hankel operators. This approach allows for random initializations of the LTI systems and helps to improve training stability, while also provides the SSMs with non-decaying memory capabilities. Our model efficiently implements these innovations by nonuniformly sampling the transfer functions of LTI systems, and it requires fewer parameters compared to canonical SSMs. When benchmarked against HiPPO-initialized models such as S4 and S4D, an SSM parameterized by Hankel operators demonstrates improved performance on Long-Range Arena (LRA) tasks. Moreover, we use a sequential CIFAR-10 task with padded noise to empirically corroborate our SSM's long memory capacity.


title: EchoSpike Predictive Plasticity: An Online Local Learning Rule for Spiking Neural Networks

abstract: The drive to develop artificial neural networks that efficiently utilize resources has generated significant interest in bio-inspired Spiking Neural Networks (SNNs). These networks are particularly attractive due to their potential in applications requiring low power and memory. This potential is further enhanced by the ability to perform online local learning, enabling them to adapt to dynamic environments. This requires the model to be adaptive in a self-supervised manner. While self-supervised learning has seen great success in many deep learning domains, its application for online local learning in multi-layer SNNs remains underexplored. In this paper, we introduce the "EchoSpike Predictive Plasticity" (ESPP) learning rule, a pioneering online local learning rule designed to leverage hierarchical temporal dynamics in SNNs through predictive and contrastive coding. We validate the effectiveness of this approach using benchmark datasets, demonstrating that it performs on par with current state-of-the-art supervised learning rules. The temporal and spatial locality of ESPP makes it particularly well-suited for low-cost neuromorphic processors, representing a significant advancement in developing biologically plausible self-supervised learning models for neuromorphic computing at the edge.


title: Removing Bias from Maximum Likelihood Estimation with Model Autophagy

abstract: We propose autophagy penalized likelihood estimation (PLE), an unbiased alternative to maximum likelihood estimation (MLE) which is more fair and less susceptible to model autophagy disorder (madness). Model autophagy refers to models trained on their own output; PLE ensures the statistics of these outputs coincide with the data statistics. This enables PLE to be statistically unbiased in certain scenarios where MLE is biased. When biased, MLE unfairly penalizes minority classes in unbalanced datasets and exacerbates the recently discovered issue of self-consuming generative modeling. Theoretical and empirical results show that 1) PLE is more fair to minority classes and 2) PLE is more stable in a self-consumed setting. Furthermore, we provide a scalable and portable implementation of PLE with a hypernetwork framework, allowing existing deep learning architectures to be easily trained with PLE. Finally, we show PLE can bridge the gap between Bayesian and frequentist paradigms in statistics.


title: Mitigating Interference in the Knowledge Continuum through Attention-Guided Incremental Learning

abstract: Continual learning (CL) remains a significant challenge for deep neural networks, as it is prone to forgetting previously acquired knowledge. Several approaches have been proposed in the literature, such as experience rehearsal, regularization, and parameter isolation, to address this problem. Although almost zero forgetting can be achieved in task-incremental learning, class-incremental learning remains highly challenging due to the problem of inter-task class separation. Limited access to previous task data makes it difficult to discriminate between classes of current and previous tasks. To address this issue, we propose `Attention-Guided Incremental Learning' (AGILE), a novel rehearsal-based CL approach that incorporates compact task attention to effectively reduce interference between tasks. AGILE utilizes lightweight, learnable task projection vectors to transform the latent representations of a shared task attention module toward task distribution. Through extensive empirical evaluation, we show that AGILE significantly improves generalization performance by mitigating task interference and outperforming rehearsal-based approaches in several CL scenarios. Furthermore, AGILE can scale well to a large number of tasks with minimal overhead while remaining well-calibrated with reduced task-recency bias.


title: Optimizing Curvature Learning for Robust Hyperbolic Deep Learning in Computer Vision

abstract: Hyperbolic deep learning has become a growing research direction in computer vision for the unique properties afforded by the alternate embedding space. The negative curvature and exponentially growing distance metric provide a natural framework for capturing hierarchical relationships between datapoints and allowing for finer separability between their embeddings. However, these methods are still computationally expensive and prone to instability, especially when attempting to learn the negative curvature that best suits the task and the data. Current Riemannian optimizers do not account for changes in the manifold which greatly harms performance and forces lower learning rates to minimize projection errors. Our paper focuses on curvature learning by introducing an improved schema for popular learning algorithms and providing a novel normalization approach to constrain embeddings within the variable representative radius of the manifold. Additionally, we introduce a novel formulation for Riemannian AdamW, and alternative hybrid encoder techniques and foundational formulations for current convolutional hyperbolic operations, greatly reducing the computational penalty of the hyperbolic embedding space. Our approach demonstrates consistent performance improvements across both direct classification and hierarchical metric learning tasks while allowing for larger hyperbolic models.


title: Rank Reduction Autoencoders -- Enhancing interpolation on nonlinear manifolds

abstract: The efficiency of classical Autoencoders (AEs) is limited in many practical situations. When the latent space is reduced through autoencoders, feature extraction becomes possible. However, overfitting is a common issue, leading to ``holes'' in AEs' interpolation capabilities. On the other hand, increasing the latent dimension results in a better approximation with fewer non-linearly coupled features (e.g., Koopman theory or kPCA), but it doesn't necessarily lead to dimensionality reduction, which makes feature extraction problematic. As a result, interpolating using Autoencoders gets harder. In this work, we introduce the Rank Reduction Autoencoder (RRAE), an autoencoder with an enlarged latent space, which is constrained to have a small pre-specified number of dominant singular values (i.e., low-rank). The latent space of RRAEs is large enough to enable accurate predictions while enabling feature extraction. As a result, the proposed autoencoder features a minimal rank linear latent space. To achieve what's proposed, two formulations are presented, a strong and a weak one, that build a reduced basis accurately representing the latent space. The first formulation consists of a truncated SVD in the latent space, while the second one adds a penalty term to the loss function. We show the efficiency of our formulations by using them for interpolation tasks and comparing the results to other autoencoders on both synthetic data and MNIST.


title: Benchmarking the Non-flow Contributions to the Elliptic Flow Parameter ($v_{_2}$) in Proton-Proton Collisions

abstract: This manuscript reports on the elliptic flow parameter, $v_{_2}$, in proton-proton (p-p) collisions using PYTHIA8 event generator simulations. The typical Event Plane (EP) method $v_{_2}$(EP) besides the algorithm used for the real data analysis has been adopted to measure the $v_{_2}$ of particles composed of different quark flavors. The $v_{_2}$ of the aforementioned particles was measured at the center of mass energy $\sqrt{S_{_{NN}}}$ = 200 GeV and $\sqrt{S_{_{NN}}}$ = 13 TeV. The pseudorapidity gap techniques, theoretically defined as $|\Delta\eta| = |\eta_{_{particle}} - \eta_{_{EP}}|$, between the particle of interest and the soft particles, $p_{_{t}}$ $\leq$ 2 GeV/c, used for the event plane determinations, have been implemented in the current measurements. The $v_{_2}^{\pi^{^\pm,}\pi^{^0}}$, $v_{_2}^{J/\psi,\Upsilon, K, D, B}$, and the $v_{_2}^{\gamma_{dir}}$ show similar pattern dependence on the transverse momentum for the same $|\Delta\eta|$, at both $\sqrt{S_{_{NN}}}$ = 200 GeV and $\sqrt{S_{_{NN}}}$ = 13 TeV. At each center-of-mass energy, the $v_{_2}^{\gamma_{dir}}$ $\le$ $v_{_2}^{J/\psi, \Upsilon, K, D, B}$ $\le$ $v_{_2}^{\pi^{^\pm,}\pi^{^0}}$ for similar $|\Delta\eta|$. The measured $v_{_2}$ shows slightly higher values at smaller $\sqrt{S_{_{NN}}}$ for particles of similar $|\Delta\eta|$, particularly at small $|\Delta\eta|$. The measured $v_{_2}$ for all particles shows systematic decreases with increasing the $|\Delta\eta|$ gaps as expected. Since PYTHIA8 does not include final state interactions, the reported $v_{_2}$(EP) is mainly due to the non-flow contributions which should be subtracted from the signal in the real data analysis that uses the same event plane determination algorithm.


title: Diagramatics of $A_1 \times A_1$ Folded Soergel Bimodules

abstract: There is an action of $\mathbb{Z}/2$ on the category of Soergel Bimodules of type $A_1 \times A_1$ induced by the nontrivial automorphism of its Dynkin diagram. We give an isotopy presentation by local generators and relations of the equivariantization of the category of Soergel Bimodules of type $A_1 \times A_1$ under this action. This is the first step in a program to describe and study the categories of equivariant Soergel Bimodules.