title: A novel model-based parameters estimation combining local optimization and global optimization of nonlinear ship models with physical experiment dataset

abstract: Designing an autonomous precise controller for ships requires accurate and reliable ship models, including the ship dynamic model and actuator model. However, selecting a suitable model for controller design and determining its parameters pose a significant challenge, considering factors such as ship actuation, input constraints, environmental disturbances, and others. This challenge is further amplified for underactuated ships, as obtaining decoupled experiment data is not feasible, and the limited data available may not adequately represent the motion characteristics of ships. To address this issue, we propose a novel model-based parameter estimation approach, called MBPE-LOGO, for underactuated ship motion models. This method combines local optimization and global optimization methods to solve the model identification problem using a dataset generated from real experiments. The effectiveness of the identified model is verified through extensive comparisons of different trajectories and prediction steps.


title: Sample-Optimal Locally Private Hypothesis Selection and the Provable Benefits of Interactivity

abstract: We study the problem of hypothesis selection under the constraint of local differential privacy. Given a class $\mathcal{F}$ of $k$ distributions and a set of i.i.d. samples from an unknown distribution $h$, the goal of hypothesis selection is to pick a distribution $\hat{f}$ whose total variation distance to $h$ is comparable with the best distribution in $\mathcal{F}$ (with high probability). We devise an $\varepsilon$-locally-differentially-private ($\varepsilon$-LDP) algorithm that uses $\Theta\left(\frac{k}{\alpha^2\min \{\varepsilon^2,1\}}\right)$ samples to guarantee that $d_{TV}(h,\hat{f})\leq \alpha + 9 \min_{f\in \mathcal{F}}d_{TV}(h,f)$ with high probability. This sample complexity is optimal for $\varepsilon<1$, matching the lower bound of Gopi et al. (2020). All previously known algorithms for this problem required $\Omega\left(\frac{k\log k}{\alpha^2\min \{ \varepsilon^2 ,1\}} \right)$ samples to work. Moreover, our result demonstrates the power of interaction for $\varepsilon$-LDP hypothesis selection. Namely, it breaks the known lower bound of $\Omega\left(\frac{k\log k}{\alpha^2\min \{ \varepsilon^2 ,1\}} \right)$ for the sample complexity of non-interactive hypothesis selection. Our algorithm breaks this barrier using only $\Theta(\log \log k)$ rounds of interaction. To prove our results, we define the notion of \emph{critical queries} for a Statistical Query Algorithm (SQA) which may be of independent interest. Informally, an SQA is said to use a small number of critical queries if its success relies on the accuracy of only a small number of queries it asks. We then design an LDP algorithm that uses a smaller number of critical queries.


title: Eigenvalues asymptotics of unbounded operators. Two-photon quantum Rabi model

abstract: In this work the general results about asymptotics of eigenvalues of unbounded operators are obtained. We consider here different cases of compact, relatively compact, selfadjoint or nonselfadjoint perturbations. In particular we prove a generalization of Janas-Naboko lemma about eigenvalues asymptotics of unbounded operators at compact perturbation. A generalization of our previous result about noncompact perturbation of oscillator spectrum is also given. As an example we consider two-photon quantum Rabi model. We obtain tree-term asymptotic formula for large eigenvalues of the energy operator of this model. The asymptotics of related to this model polynomials is found. We give also an original proof of the Perelomov factorization theorem for contraction operator of quantum optics.


title: Avoiding matrix exponentials for large transition rate matrices

abstract: Exact methods for exponentiation of matrices of dimension $N$ can be computationally expensive in terms of execution time ($N^{3}$) and memory requirements ($N^{2}$) not to mention numerical precision issues. A type of matrix often exponentiated in the sciences is the rate matrix. Here we explore five methods to exponentiate rate matrices some of which apply even more broadly to other matrix types. Three of the methods leverage a mathematical analogy between computing matrix elements of a matrix exponential and computing transition probabilities of a dynamical processes (technically a Markov jump process, MJP, typically simulated using Gillespie). In doing so, we identify a novel MJP-based method relying on restricting the number of "trajectory" jumps based on the magnitude of the matrix elements with favorable computational scaling. We then discuss this method's downstream implications on mixing properties of Monte Carlo posterior samplers. We also benchmark two other methods of matrix exponentiation valid for any matrix (beyond rate matrices and, more generally, positive definite matrices) related to solving differential equations: Runge-Kutta integrators and Krylov subspace methods. Under conditions where both the largest matrix element and the number of non-vanishing elements scale linearly with $N$ -- reasonable conditions for rate matrices often exponentiated -- computational time scaling with the most competitive methods (Krylov and one of the MJP-based methods) reduces to $N^2$ with total memory requirements of $N$.


title: Learning Arbitrary Complex Matrices by Interlacing Amplitude and Phase Masks with Fixed Unitary Operations

abstract: Programmable photonic integrated circuits represent an emerging technology that amalgamates photonics and electronics, paving the way for light-based information processing at high speeds and low power consumption. Considering their wide range of applications as one of the most fundamental mathematical operations there has been a particular interest in programmable photonic circuits that perform matrix-vector multiplication. In this regard, there has been great interest in developing novel circuit architectures for performing matrix operations that are compatible with the existing photonic integrated circuit technology which can thus be reliably implemented. Recently, it has been shown that discrete linear unitary operations can be parameterized through diagonal phase parameters interlaced with a fixed operator that enables efficient photonic realization of unitary operations by cascading phase shifter arrays interlaced with a multiport component. Here, we show that such a decomposition is only a special case of a much broader class of factorizations that allow for parametrizing arbitrary complex matrices in terms of diagonal matrices alternating with a fixed unitary matrix. Thus, we introduce a novel architecture for physically implementing discrete linear operations. The proposed architecture is built on representing an $N \times N$ matrix operator in terms of $N+1$ amplitude-and-phase modulation layers interlaced with a fixed unitary layer that could be implemented via a coupled waveguide array. The proposed architecture enables the development of novel families of programmable photonic circuits for on-chip analog information processing.


title: Micro pixelated halide perovskite photodiodes fabricated with ultraviolet laser scribing

abstract: In this study, we present a complex investigation for miniaturizing of perovskite photodiodes (PPDs) in various geometries with use of ultraviolet laser scribing (UV-LS). Employing a 355 nm (3.5 eV) pulsed laser at 30 kHz, we successfully manufactured PPDs with pixel configurations of 70x130 um2, 520x580 um2, and 2000x2000 um2. The utilization of UV-LS has a proven efficiency in achieving relevant diode characteristics, such as low dark currents and high shunt resistance, as well as ultrafast response. The multi-step scribing cycle provided precise patterning of perovskite photodiodes (PPDs) in a string design. The dark current densities demonstrated exceptional uniformity, ranging from 10-10 A/cm2 for 2000x2000 um2 pixelated PPDs to 10-9 A/cm2 for the 70x130 um2 configuration. The string PPDs, consisting of 10 pixels per string, displayed homogenous dark current values, ensuring effective isolation between devices. Under green light illumination (540 nm), all PPD types exhibited a broad Linear Dynamic Range (LDR). Specifically, LDR values reached 110 dB, 117 dB, and 136 dB for 70x130, 520x580, and 2000x2000 devices, respectively, spanning an illumination intensity range from 2*10-3 mW/cm2 to 2 mW/cm2. High responsivity values up to 0.38 A/W, depending on the PPDs geometry, highlight the potential of laser scribing devices for sensing in the visible range. The calculated specific detectivity performance (from 1011 to 1013 Jones) surpasses commercial analogs, while the sub-microsecond response of 70x130 um2 and 520x580 um2 miniaturized devices underscores their suitability for precise time resolution detection systems.


title: An embedding theorem for multidimensional subshifts

abstract: Krieger's embedding theorem provides necessary and sufficient conditions for an arbitrary subshift to embed in a given topologically mixing $\mathbb{Z}$-subshift of finite type. For some $\mathbb{Z}^d$-subshifts of finite type, Lightwood characterized the \emph{aperiodic} subsystems. In the current paper we prove a new embedding theorem for a class of subshifts of finite type over any countable abelian group. Our main theorem provides necessary and sufficient conditions for an arbitrary subshift $X$ to embed inside a given subshift of finite type $Y$ that satisfies a certain condition. For the particular case of $\mathbb{Z}$-subshifts, our new theorem coincides with Krieger's theorem. In particular, our result gives the first complete characterization of the subsystems of the multidimensional full shift $Y= A^{\mathbb{Z}^d}$. The natural condition on the target subshift $Y$, introduced explicitly for the first time in the current paper, is called the map extension property. It was introduced implicitly by Mike Boyle in the early 1980's for $\mathbb{Z}$-subshifts, and is closely related to the notion of an absolute retract, introduced by Borsuk in the 1930's. A $\mathbb{Z}$-subshift has the map extension property if and only if it is a topologically mixing subshift of finite type. Over abelian groups, a subshift has the map extension property if and only if it is a contractible SFT as shown in work of Poirier and Salo. We also establish a new theorem regarding lower entropy factors of multidimensional subshifts, that extends Boyle's lower entropy factor theorem from the one-dimensional case.


title: Set-valued recursions arising from vantage-point trees

abstract: We study vantage-point trees constructed using an independent sample from the uniform distribution on a fixed convex body $K$ in $(\mathbb{R}^d,\|\cdot\|)$, where $\|\cdot\|$ is an arbitrary homogeneous norm on $\mathbb{R}^d$. We prove that a sequence of sets, associated with the left boundary of a vantage-point tree, forms a recurrent Harris chain on the space of convex bodies in $(\mathbb{R}^d,\|\cdot\|)$. The limiting object is a ball polyhedron, that is, an a.s.~finite intersection of closed balls in $(\mathbb{R}^d,\|\cdot\|)$ of possibly different radii. As a consequence, we derive a limit theorem for the length of the leftmost path of a vantage-point tree.


title: One Gate Scheme to Rule Them All: Introducing a Complex Yet Reduced Instruction Set for Quantum Computing

abstract: The design and architecture of a quantum instruction set are paramount to the performance of a quantum computer. This work introduces a gate scheme for qubits with $XX+YY$ coupling that directly and efficiently realizes any two-qubit gate up to single-qubit gates. First, this scheme enables high-fidelity execution of quantum operations and achieves minimum possible gate times. Second, since the scheme spans the entire $\textbf{SU}(4)$ group of two-qubit gates, we can use it to attain the optimal two-qubit gate count for algorithm implementation. These two advantages in synergy give rise to a quantum Complex yet Reduced Instruction Set Computer (CRISC). Though the gate scheme is compact, it supports a comprehensive array of quantum operations. This may seem paradoxical but is realizable due to the fundamental differences between quantum and classical computer architectures. Using our gate scheme, we observe marked improvements across various applications, including generic $n$-qubit gate synthesis, quantum volume, and qubit routing. Furthermore, the proposed scheme also realizes a gate locally equivalent to the commonly used CNOT gate with a gate time of $\frac{\pi}{2g}$, where $g$ is the two-qubit coupling. The AshN scheme is also completely impervious to $ZZ$ error, the main coherent error in transversely coupled systems, as the control parameters implementing the gates can be easily adjusted to take the $ZZ$ term into account.


title: Multifractal nonlinearity as a robust estimator of multiplicative cascade dynamics

abstract: Multifractal formalisms provide an apt framework to study random cascades in which multifractal spectrum width $\Delta\alpha$ fluctuates depending on the number of estimable power-law relationships. Then again, multifractality without surrogate comparison can be ambiguous: the original measurement series' multifractal spectrum width $\Delta\alpha_\mathrm{Orig}$ can be sensitive to the series length, ergodicity-breaking linear temporal correlations (e.g., fractional Gaussian noise, $fGn$), or additive cascade dynamics. To test these threats, we built a suite of random cascades that differ by the length, type of noise (i.e., additive white Gaussian noise, $awGn$, or $fGn$), and mixtures of $awGn$ or $fGn$ across generations (progressively more $awGn$, progressively more $fGn$, and a random sampling by generation), and operations applying noise (i.e., addition vs. multiplication). The so-called ``multifractal nonlinearity'' $t_\mathrm{MF}$ (i.e., a $t$-statistic comparing $\Delta\alpha_\mathrm{Orig}$ and multifractal spectra width for phase-randomized linear surrogates $\Delta\alpha_\mathrm{Surr}$) is a robust indicator of random multiplicative rather than random additive cascade processes irrespective of the series length or type of noise. $t_\mathrm{MF}$ is more sensitive to the number of generations than the series length. Furthermore, the random additive cascades exhibited much stronger ergodicity breaking than all multiplicative analogs. Instead, ergodicity breaking in random multiplicative cascades more closely followed the ergodicity-breaking of the constituent noise types -- breaking ergodicity much less when arising from ergodic $awGn$ and more so for noise incorporating relatively more correlated $fGn$. Hence, $t_\mathrm{MF}$ is a robust multifractal indicator of multiplicative cascade processes and not spuriously sensitive to ergodicity breaking.


title: Spectral methods for Neural Integral Equations

abstract: Neural integral equations are deep learning models based on the theory of integral equations, where the model consists of an integral operator and the corresponding equation (of the second kind) which is learned through an optimization procedure. This approach allows to leverage the nonlocal properties of integral operators in machine learning, but it is computationally expensive. In this article, we introduce a framework for neural integral equations based on spectral methods that allows us to learn an operator in the spectral domain, resulting in a cheaper computational cost, as well as in high interpolation accuracy. We study the properties of our methods and show various theoretical guarantees regarding the approximation capabilities of the model, and convergence to solutions of the numerical methods. We provide numerical experiments to demonstrate the practical effectiveness of the resulting model.


title: A novel scaling approach for unbiased adjustment of risk estimators

abstract: The assessment of risk based on historical data faces many challenges, in particular due to the limited amount of available data, lack of stationarity, and heavy tails. While estimation on a short-term horizon for less extreme percentiles tends to be reasonably accurate, extending it to longer time horizons or extreme percentiles poses significant difficulties. The application of theoretical risk scaling laws to address this issue has been extensively explored in the literature. This paper presents a novel approach to scaling a given risk estimator, ensuring that the estimated capital reserve is robust and conservatively estimates the risk. We develop a simple statistical framework that allows efficient risk scaling and has a direct link to backtesting performance. Our method allows time scaling beyond the conventional square-root-of-time rule, enables risk transfers, such as those involved in economic capital allocation, and could be used for unbiased risk estimation in small sample settings. To demonstrate the effectiveness of our approach, we provide various examples related to the estimation of value-at-risk and expected shortfall together with a short empirical study analysing the impact of our method.


title: Plasmonic skyrmion quantum thermodynamics

abstract: The primary obstacle in the field of quantum thermodynamics revolves around the development and practical implementation of quantum heat engines operating at the nanoscale. One of the key challenges associated with quantum working bodies is the occurrence of "quantum friction," which refers to irreversible wasted work resulting from quantum inter-level transitions. Consequently, the construction of a reversible quantum cycle necessitates the utilization of adiabatic shortcuts. However, the experimental realization of such shortcuts for realistic quantum substances is exceedingly complex and often unattainable. In this study, we propose a quantum heat engine that capitalizes on the plasmonic skyrmion lattice. Through rigorous analysis, we demonstrate that the quantum skyrmion substance, owing to its topological protection, exhibits zero irreversible work. Consequently, our engine operates without the need for adiabatic shortcuts. We checked by numerical calculations and observed that when the system is in the quantum skyrmion phase, the propagated states differ from the initial states only by the geometricl and dynamical phases. The adiabacit evoluation leads to the zero transition matrix elements and zero irreversible work. By employing plasmonic mods and an electric field, we drive the quantum cycle. The fundamental building blocks for constructing the quantum working body are individual skyrmions within the plasmonic lattice. As a result, one can precisely control the output power of the engine and the thermodynamic work accomplished by manipulating the number of quantum skyrmions present.


title: Leveraging Reinforcement Learning and Large Language Models for Code Optimization

abstract: Code optimization is a daunting task that requires a significant level of expertise from experienced programmers. This level of expertise is not sufficient when compared to the rapid development of new hardware architectures. Towards advancing the whole code optimization process, recent approaches rely on machine learning and artificial intelligence techniques. This paper introduces a new framework to decrease the complexity of code optimization. The proposed framework builds on large language models (LLMs) and reinforcement learning (RL) and enables LLMs to receive feedback from their environment (i.e., unit tests) during the fine-tuning process. We compare our framework with existing state-of-the-art models and show that it is more efficient with respect to speed and computational usage, as a result of the decrement in training steps and its applicability to models with fewer parameters. Additionally, our framework reduces the possibility of logical and syntactical errors. Toward evaluating our approach, we run several experiments on the PIE dataset using a CodeT5 language model and RRHF, a new reinforcement learning algorithm. We adopt a variety of evaluation metrics with regards to optimization quality, and speedup. The evaluation results demonstrate that the proposed framework has similar results in comparison with existing models using shorter training times and smaller pre-trained models. In particular, we accomplish an increase of 5.6% and 2.2 over the baseline models concerning the %OP T and SP metrics.


title: Monoid Theory in Alonzo: A Little Theories Formalization in Simple Type Theory

abstract: Alonzo is a practice-oriented classical higher-order logic that extends first-order logic and that admits undefined expressions. Named in honor of Alonzo Church, Alonzo is based on Church's type theory, Church's formulation of simple type theory. The little theories method is a method for formalizing mathematical knowledge as a network of theories called a theory graph consisting of theories as nodes and theory morphisms as directed edges. The development of a mathematical topic is done in the "little theory" in the theory graph that has the most convenient level of abstraction and the most convenient vocabulary, and then the definitions and theorems produced in the development are transported, as needed, to other theories via the theory morphisms in the theory graph. The purpose of this paper is to illustrate how a body of mathematical knowledge can be formalized in Alonzo using the little theories method. This is done by formalizing monoid theory -- the body of mathematical knowledge about monoids -- in Alonzo.


title: Optimal Unbiased Randomizers for Regression with Label Differential Privacy

abstract: We propose a new family of label randomizers for training regression models under the constraint of label differential privacy (DP). In particular, we leverage the trade-offs between bias and variance to construct better label randomizers depending on a privately estimated prior distribution over the labels. We demonstrate that these randomizers achieve state-of-the-art privacy-utility trade-offs on several datasets, highlighting the importance of reducing bias when training neural networks with label DP. We also provide theoretical results shedding light on the structural properties of the optimal unbiased randomizers.


title: Galois Cohomology of Reductive Groups over Function Fields over $\mathbb{C}((t))$

abstract: Over a global field (number field or function field of a curve over a finite field), theorems for the Galois cohomology of algebraic groups have long been known. For $F$ the function field of a curve over the formal series field $\mathbb{C}((t))$, under an additional assumption on the curve, we establish an explicit description of the localization map in Galois cohomology. This is achieved by extending the theory of $B(F,G)$ due to Kottwitz.


title: Optimized Multifidelity Machine Learning for Quantum Chemistry

abstract: Machine learning (ML) provides access to fast and accurate quantum chemistry (QC) calculations for various properties of interest such as excitation energies. It is often the case that high accuracy in prediction using an ML model, demands a large and costly training set. Various solutions and procedures have been presented to reduce this cost. These include methods such as $\Delta$-ML, hierarchical-ML, and multifidelity machine learning (MFML). MFML combines various $\Delta$-ML like sub-models for various fidelities according to a fixed scheme derived from the sparse grid combination technique. In this work we implement an optimization procedure to combine multifidelity models in a flexible scheme resulting in optimized MFML (o-MFML) that provides superior prediction capabilities. This hyper-parameter optimization is carried out on a holdout validation set of the property of interest. This work benchmarks the o-MFML method in predicting the atomization energies on the QM7b dataset, and again in the prediction of excitation energies for three molecules of growing size. The results indicate that o-MFML is a strong methodological improvement over MFML and provides lower error of prediction. Even in cases of poor data distributions and lack of clear hierarchies among the fidelities, which were previously identified as issues for multifidelity methods, the o-MFML provides advantage to the prediction of quantum chemical properties.


title: Understanding the Effect of Model Compression on Social Bias in Large Language Models

abstract: Large Language Models (LLMs) trained with self-supervision on vast corpora of web text fit to the social biases of that text. Without intervention, these social biases persist in the model's predictions in downstream tasks, leading to representational harm. Many strategies have been proposed to mitigate the effects of inappropriate social biases learned during pretraining. Simultaneously, methods for model compression have become increasingly popular to reduce the computational burden of LLMs. Despite the popularity and need for both approaches, little work has been done to explore the interplay between these two. We perform a carefully controlled study of the impact of model compression via quantization and knowledge distillation on measures of social bias in LLMs. Longer pretraining and larger models led to higher social bias, and quantization showed a regularizer effect with its best trade-off around 20% of the original pretraining time.


title: Bridging colorings of virtual links from virtual biquandles to biquandles

abstract: A biquandle is a solution to the set-theoretical Yang-Baxter equation, which yields invariants for virtual knots such as the coloring number and the state-sum invariant. A virtual biquandle enriches the structure of a biquandle by incorporating an invertible unary map. This unary operator plays a crucial role in defining the action of virtual crossings on the labels of incoming arcs in a virtual link diagram. This leads to extensions of invariants from biquandles to virtual biquandles, thereby enhancing their strength. In this article, we establish a connection between the coloring invariant derived from biquandles and virtual biquandles. We prove that the number of colorings of a virtual link $L$ by virtual biquandles can be recovered from colorings by biquandles. We achieve this by proving the equivalence between two different representations of virtual braid groups. Furthermore, we introduce a new set of labeling rules using which one can construct a presentation of the associated fundamental virtual biquandle of $L$ using only the relations coming from the classical crossings. This is an improvement to the traditional method, where writing down a presentation of the associated fundamental virtual biquandle necessitates noting down the relations arising from the classical and virtual crossings.


title: CoGS: Controllable Gaussian Splatting

abstract: Capturing and re-animating the 3D structure of articulated objects present significant barriers. On one hand, methods requiring extensively calibrated multi-view setups are prohibitively complex and resource-intensive, limiting their practical applicability. On the other hand, while single-camera Neural Radiance Fields (NeRFs) offer a more streamlined approach, they have excessive training and rendering costs. 3D Gaussian Splatting would be a suitable alternative but for two reasons. Firstly, existing methods for 3D dynamic Gaussians require synchronized multi-view cameras, and secondly, the lack of controllability in dynamic scenarios. We present CoGS, a method for Controllable Gaussian Splatting, that enables the direct manipulation of scene elements, offering real-time control of dynamic scenes without the prerequisite of pre-computing control signals. We evaluated CoGS using both synthetic and real-world datasets that include dynamic objects that differ in degree of difficulty. In our evaluations, CoGS consistently outperformed existing dynamic and controllable neural representations in terms of visual fidelity.


title: Enhancing Modbus TCP Protocol Security with eBPF Technology

abstract: The core component of an Industrial Control System (ICS) is often a Programmable Logic Controller (PLC) combined with various modules. In such systems, the communication between devices is mainly based on the Modbus protocol, which was developed by Modicon (now Schneider Electric) in 1979 as an application-level communication protocol and has become a de facto standard for ICS for the past 40 years. Modbus TCP is a variant of this protocol for communications over the TCP/IP network. However, the Modbus protocol was not designed with security in mind, and the use of plaintext transmissions during communication makes information easily accessible to the attackers, while the lack of an authentication mechanism gives any protocol-compliant device the ability to take over control. In this study, we use the eBPF technology to shift the process of protocol change to the lower level of the operating system, making the change transparent to the existing software, and enhancing the security of the Modbus TCP protocol without affecting the existing software ecosystem as much as possible.


title: Towards a Graph Neural Network-Based Approach for Estimating Hidden States in Cyber Attack Simulations

abstract: This work-in-progress paper introduces a prototype for a novel Graph Neural Network (GNN) based approach to estimate hidden states in cyber attack simulations. Utilizing the Meta Attack Language (MAL) in conjunction with Relational Dynamic Decision Language (RDDL) conformant simulations, our framework aims to map the intricate complexity of cyber attacks with a vast number of possible vectors in the simulations. While the prototype is yet to be completed and validated, we discuss its foundational concepts, the architecture, and the potential implications for the field of computer security.


title: Bayesian Optimization Algorithms for Accelerator Physics

abstract: Accelerator physics relies on numerical algorithms to solve optimization problems in online accelerator control and tasks such as experimental design and model calibration in simulations. The effectiveness of optimization algorithms in discovering ideal solutions for complex challenges with limited resources often determines the problem complexity these methods can address. The accelerator physics community has recognized the advantages of Bayesian optimization algorithms, which leverage statistical surrogate models of objective functions to effectively address complex optimization challenges, especially in the presence of noise during accelerator operation and in resource-intensive physics simulations. In this review article, we offer a conceptual overview of applying Bayesian optimization techniques towards solving optimization problems in accelerator physics. We begin by providing a straightforward explanation of the essential components that make up Bayesian optimization techniques. We then give an overview of current and previous work applying and modifying these techniques to solve accelerator physics challenges. Finally, we explore practical implementation strategies for Bayesian optimization algorithms to maximize their performance, enabling users to effectively address complex optimization challenges in real-time beam control and accelerator design.


title: Polarization in Decentralized Online Social Networks

abstract: Centralized social media platforms are currently experiencing a shift in user engagement, drawing attention to alternative paradigms like Decentralized Online Social Networks (DOSNs). The rising popularity of DOSNs finds its root in the accessibility of open-source software, enabling anyone to create a new instance (i.e., server) and participate in a decentralized network known as Fediverse. Despite this growing momentum, there has been a lack of studies addressing the effect of positive and negative interactions among instances within DOSNs. This work aims to fill this gap by presenting a preliminary examination of instances' polarization in DOSNs, focusing on Mastodon -- the most widely recognized decentralized social media platform, boasting over 10M users and nearly 20K instances to date. Our results suggest that polarization in the Fediverse emerges in unique ways, influenced by the desire to foster a federated environment between instances, also facilitating the isolation of instances that may pose potential risks to the Fediverse.


title: Relevance Feedback with Brain Signals

abstract: The Relevance Feedback (RF) process relies on accurate and real-time relevance estimation of feedback documents to improve retrieval performance. Since collecting explicit relevance annotations imposes an extra burden on the user, extensive studies have explored using pseudo-relevance signals and implicit feedback signals as substitutes. However, such signals are indirect indicators of relevance and suffer from complex search scenarios where user interactions are absent or biased. Recently, the advances in portable and high-precision brain-computer interface (BCI) devices have shown the possibility to monitor user's brain activities during search process. Brain signals can directly reflect user's psychological responses to search results and thus it can act as additional and unbiased RF signals. To explore the effectiveness of brain signals in the context of RF, we propose a novel RF framework that combines BCI-based relevance feedback with pseudo-relevance signals and implicit signals to improve the performance of document re-ranking. The experimental results on the user study dataset show that incorporating brain signals leads to significant performance improvement in our RF framework. Besides, we observe that brain signals perform particularly well in several hard search scenarios, especially when implicit signals as feedback are missing or noisy. This reveals when and how to exploit brain signals in the context of RF.


title: Bounds for the sampling discretization error and their applications to universal sampling discretization

abstract: In the first part of the paper we study absolute error of sampling discretization of the integral $L_p$-norm for functional classes of continuous functions. We use chaining technique to provide a general bound for the error of sampling discretization of the $L_p$-norm on a given functional class in terms of entropy numbers in the uniform norm of this class. The general result yields new error bounds for sampling discretization of the $L_p$-norms on classes of multivariate functions with mixed smoothness. In the second part of the paper we study universal sampling discretization and the problem of optimal sampling recovery.


title: Hate Speech and Offensive Content Detection in Indo-Aryan Languages: A Battle of LSTM and Transformers

abstract: Social media platforms serve as accessible outlets for individuals to express their thoughts and experiences, resulting in an influx of user-generated data spanning all age groups. While these platforms enable free expression, they also present significant challenges, including the proliferation of hate speech and offensive content. Such objectionable language disrupts objective discourse and can lead to radicalization of debates, ultimately threatening democratic values. Consequently, organizations have taken steps to monitor and curb abusive behavior, necessitating automated methods for identifying suspicious posts. This paper contributes to Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages (HASOC) 2023 shared tasks track. We, team Z-AGI Labs, conduct a comprehensive comparative analysis of hate speech classification across five distinct languages: Bengali, Assamese, Bodo, Sinhala, and Gujarati. Our study encompasses a wide range of pre-trained models, including Bert variants, XLM-R, and LSTM models, to assess their performance in identifying hate speech across these languages. Results reveal intriguing variations in model performance. Notably, Bert Base Multilingual Cased emerges as a strong performer across languages, achieving an F1 score of 0.67027 for Bengali and 0.70525 for Assamese. At the same time, it significantly outperforms other models with an impressive F1 score of 0.83009 for Bodo. In Sinhala, XLM-R stands out with an F1 score of 0.83493, whereas for Gujarati, a custom LSTM-based model outshined with an F1 score of 0.76601. This study offers valuable insights into the suitability of various pre-trained models for hate speech detection in multilingual settings. By considering the nuances of each, our research contributes to an informed model selection for building robust hate speech detection systems.


title: Dynamics and Clouds in Planetary Atmospheres from Telescopic Observations

abstract: This review presents an insight into our current knowledge of the atmospheres of the planets Venus, Mars, Jupiter, Saturn, Uranus and Neptune, the satellite Titan, and those of exoplanets. It deals with the thermal structure, aerosol properties (hazes and clouds, dust in the case of Mars), chemical composition, global winds and selected dynamical phenomena in these objects. Our understanding of atmospheres is greatly benefitting from the discovery in the last three decades of thousands of exoplanets. The exoplanet properties span a broad range of conditions, and it is fair to expect as much variety for their atmospheres. This complexity is driving unprecedented investigations of the atmospheres, where those of the solar systems bodies are the obvious reference. We are witnessing a significant transfer of knowledge in both directions between the investigations dedicated to Solar System and exoplanet atmospheres, and there are reasons to think that this exchange will intensity in the future. We identify and select a list of research subjects that can be conducted at optical and infrared wavelengths with future and currently available ground-based and space-based telescopes, but excluding those from the space missions to solar system bodies.


title: Modeling Homophily in Exponential-Family Random Graph Models for Bipartite Networks

abstract: Homophily, the tendency of individuals who are alike to form ties with one another, is an important concept in the study of social networks. Yet accounting for homophily effects is complicated in the context of bipartite networks where ties connect individuals not with one another but rather with a separate set of nodes, which might also be individuals but which are often an entirely different type of objects. As a result, much work on the effect of homophily in a bipartite network proceeds by first eliminating the bipartite structure, collapsing a two-mode network to a one-mode network and thereby ignoring potentially meaningful structure in the data. We introduce a set of methods to model homophily on bipartite networks without losing information in this way, then we demonstrate that these methods allow for substantively interesting findings in management science not possible using standard techniques. These methods are implemented in the widely-used ergm package for R.


title: Position control of an acoustic cavitation bubble by reinforcement learning

abstract: A control technique is developed via Reinforcement Learning that allows arbitrary controlling of the position of an acoustic cavitation bubble in a dual-frequency standing acoustic wave field. The agent must choose the optimal pressure amplitude values to manipulate the bubble position in the range of $x/\lambda_0\in[0.05, 0.25]$. To train the agent an actor-critic off-policy algorithm (Deep Deterministic Policy Gradient) was used that supports continuous action space, which allows setting the pressure amplitude values continuously within $0$ and $1\, \mathrm{bar}$. A shaped reward function is formulated that minimizes the distance between the bubble and the target position and implicitly encourages the agent to perform the position control within the shortest amount of time. In some cases, the optimal control can be 7 times faster than the solution expected from the linear theory.


title: Using Think-Aloud Data to Understand Relations between Self-Regulation Cycle Characteristics and Student Performance in Intelligent Tutoring Systems

abstract: Numerous studies demonstrate the importance of self-regulation during learning by problem-solving. Recent work in learning analytics has largely examined students' use of SRL concerning overall learning gains. Limited research has related SRL to in-the-moment performance differences among learners. The present study investigates SRL behaviors in relationship to learners' moment-by-moment performance while working with intelligent tutoring systems for stoichiometry chemistry. We demonstrate the feasibility of labeling SRL behaviors based on AI-generated think-aloud transcripts, identifying the presence or absence of four SRL categories (processing information, planning, enacting, and realizing errors) in each utterance. Using the SRL codes, we conducted regression analyses to examine how the use of SRL in terms of presence, frequency, cyclical characteristics, and recency relate to student performance on subsequent steps in multi-step problems. A model considering students' SRL cycle characteristics outperformed a model only using in-the-moment SRL assessment. In line with theoretical predictions, students' actions during earlier, process-heavy stages of SRL cycles exhibited lower moment-by-moment correctness during problem-solving than later SRL cycle stages. We discuss system re-design opportunities to add SRL support during stages of processing and paths forward for using machine learning to speed research depending on the assessment of SRL based on transcription of think-aloud data.


title: Torsional oscillations of magnetized neutron stars: Impacts of Landau-Rabi quantization of electron motion

abstract: Torsional oscillations of magnetized neutron stars have been well studied since they may be excited in magnetar starquakes and relevant to the observed quasiperiodic oscillations in the magnetar giant flares. In the crustal region of a magnetar, the strong magnetic field can alter the equation of state and composition of the crust due to the Landau-Rabi quantization of electron motion. In this paper, we study this effect on the torsional oscillation modes of neutron stars with mixed poloidal-toroidal magnetic fields in general relativity under the Cowling approximation. Furthermore, the inner and outer crusts are treated consistently based on the nuclear-energy density functional theory. Depending on the magnetic-field configurations, we find that the Landau-Rabi quantization of electron can change the frequencies of the fundamental torsional oscillation mode of $1.4 M_\odot$ neutron star models with a normal fluid core by about $10\%$ when the magnetic field strength at the pole reaches the order of $10^{16}$ G. The shift can even approach $20\%$ at a field strength of $10^{15}$ G for neutron stars with a simple model of superconducting core where the magnetic field is assumed to be expelled completely from the core and confined only in the crust.


title: Batched Low-Rank Adaptation of Foundation Models

abstract: Low-Rank Adaptation (LoRA) has recently gained attention for fine-tuning foundation models by incorporating trainable low-rank matrices, thereby reducing the number of trainable parameters. While LoRA offers numerous advantages, its applicability for real-time serving to a diverse and global user base is constrained by its incapability to handle multiple task-specific adapters efficiently. This imposes a performance bottleneck in scenarios requiring personalized, task-specific adaptations for each incoming request. To mitigate this constraint, we introduce Fast LoRA (FLoRA), a framework in which each input example in a minibatch can be associated with its unique low-rank adaptation weights, allowing for efficient batching of heterogeneous requests. We empirically demonstrate that FLoRA retains the performance merits of LoRA, showcasing competitive results on the MultiPL-E code generation benchmark spanning over 8 languages and a multilingual speech recognition task across 6 languages.


title: Measuring sampling plan utility in post-market surveillance of medical product supply chains

abstract: Ensuring product quality is critical to combating the global challenge of substandard and falsified medical products. Post-market surveillance is a quality-assurance activity in which products are tested at consumer-facing locations. Regulators in low-resource settings use post-market surveillance to evaluate product quality across locations and determine corrective actions. Part of post-market surveillance is developing a sampling plan, which specifies the number of tests to conduct at each location. With limited resources, it is important to base decisions for test allocations on the utility of the samples tested. We propose a Bayesian approach to generate a comprehensive utility for sampling plans. This sampling plan utility integrates regulatory risk assessments with prior testing data, available supply-chain information, and valuations of regulatory objectives. We illustrate the value of this metric with a case study based on de-identified post-market surveillance data from a low-resource setting. To do so, we develop a fast method for calculating utility that is used in a greedy heuristic to form sampling plans. The resulting plans focus surveillance on locations more likely to impact regulator decisions. These locations can be distinct from locations with higher assessed risk.


title: Schr\"odinger's control and estimation paradigm with spatio-temporal distributions on graphs

abstract: The problem of reconciling a prior probability law with data was introduced by E. Schr\"odinger in 1931/32. It represents an early formulation of a maximum likelihood problem. The specific formulation can also be seen as the control problem to modify the law of a diffusion process so as to match specifications on marginal distributions at given times. Thereby, in recent years, this so-called {\em Schr\"odinger Bridge problem} has been at the center of the development of uncertainty control. However, an unstudied facet of this program has been to address uncertainty in space and time, modeling the effect of tasks being completed, instead of imposing specifications at fixed times. The present work is a first study to extend Schr\"odinger's paradigm on such an issue. It is developed in the context of Markov chains and random walks on graphs. Specifically, we study the case where one marginal distribution represents the initial state occupation of a Markov chain, while others represent first-arrival time distributions at absorbing states signifying completion of tasks. We establish that when the prior is Markov, a Markov policy is once again optimal with respect to a likelihood cost that follows Schr\"odinger's dictum.


title: Bi-Quadratic Improvement in Conditional Quantum Search

abstract: The Grover search algorithm performs an unstructured search of a marked item in a database quadratically faster than classical algorithms and is shown to be optimal. Here, we show that if the search space is divided into two blocks with the local query operators and the global operators satisfy certain condition, then it is possible to achieve an improvement of bi-quadratic speed-up. Furthermore, we investigate the bi-quadratic speed-up in the presence of noise and show that it can tolerate noisy scenario. This may have potential applications for diverse fields, including database searching, and optimization, where efficient search algorithms play a pivotal role in solving complex computational problems.


title: Can silica nanoparticles improve lithium transport in polymer electrolytes?

abstract: The question of if silica nanoparticles can enhance the ionic conductivity of a polymer electrolyte above its crystallization temperature has remained unclear for the two decades following the first experiments on these systems. We use Molecular Dynamics simulations to decipher the atomic scale mechanisms affecting the properties of LiTFSI-poly(ethylene oxide) electrolytes upon the addition of silica nanoparticles. At any ionic concentration, adding nanoparticles significantly decreases the conductivity. Most of this reduction can be simply accounted for by the diffusion equation, resulting from the fact that the space occupied by the nanoparticles is made inactive and unable to sustain ionic diffusion. We identify two distinct regimes, above and below a concentration threshold, corresponding to very different ionic distributions and coordination features of the various species. The lack of conductivity enhancement observed in the simulations supports the conclusions of some recent measurements, and disagrees with the earliest experimental reports on hybrid silica/polyethylene-oxide electrolytes.


title: Valid Cross-Covariance Models via Multivariate Mixtures with an Application to the Confluent Hypergeometric Class

abstract: Modeling of multivariate random fields through Gaussian processes calls for the construction of valid cross-covariance functions describing the dependence between any two component processes at different spatial locations. The required validity conditions often present challenges that lead to complicated restrictions on the parameter space. The purpose of this work is to present techniques using multivariate mixtures for establishing validity that are simultaneously simplified and comprehensive. This is accomplished using results on conditionally negative semidefinite matrices and the Schur product theorem. For illustration, we use the recently-introduced Confluent Hypergeometric (CH) class of covariance functions. In addition, we establish the spectral density of the Confluent Hypergeometric covariance and use this to construct valid multivariate models as well as propose new cross-covariances. Our approach leads to valid multivariate cross-covariance models that inherit the desired marginal properties of the Confluent Hypergeometric model and outperform the multivariate Mat\'ern model in out-of-sample prediction under slowly-decaying correlation of the underlying multivariate random field. We also establish properties of the new models, including results on equivalence of Gaussian measures. We demonstrate the new model's use for multivariate oceanography dataset consisting of temperature, salinity and oxygen, as measured by autonomous floats in the Southern Ocean.


title: Cosmological FLRW phase transitions and micro-structure under Kaniadakis statistics

abstract: This article is devoted to the study of the thermodynamics phase transitions and critical phenomena of an FLRW cosmological model under the so-called Kaniadakis's statistics. The equation of state is derived from the corrected Friedmann field equations and the thermodynamics unified first law. This reveals the existence of non-trivial critical points where a first-order phase transition takes place. The system behaves as an "inverted" van der Waals fluid in this concern. Interestingly, the numerical values of the critical exponents are the same as those of the van der Waals system. Besides, to obtain more insights into the thermodynamics description, the so-called Ruppeiner's geometry is studied through the normalized scalar curvature, disclosing this invariant zone where the system undergoes repulsive/attractive interactions. Near the critical point, this curvature provides again the same critical exponent and universal constant value as for van der Waals fluid.


title: Light detection and Cosmic Rejection in the ICARUS LArTPC at Fermilab

abstract: The ICARUS-T600 detector is a 760-ton Liquid Argon Time Projection Chamber (LArTPC) currently operating at Fermilab as the Far Detector in the Short Baseline Neutrino (SBN) program. The SBN program is composed of three LArTPCs with a central goal of testing the sterile neutrino hypothesis. After operating for 3-years in the Gran Sasso Underground Laboratory, the ICARUS detector was shipped to CERN where it was outfitted with 360 8" Photomultiplier Tubes (PMTs) for a new optical detection system. The PMT system detects fast scintillation light from charged particles interacting in the Liquid Argon, generating the trigger signal for the full detector and allows 3D reconstruction of events. Now operating at shallow depth, the detector is exposed to a high flux of cosmic rays that can fake neutrino interactions. To mitigate this effect a Cosmic Ray Tagger (CRT) and a 3-meter-thick concrete were installed. Precise timing information from both the PMT and CRT subsystems can help to identify whether an interaction originated from inside or outside of the ICARUS cryostat. This paper reviews a method for cosmogenic background reduction and timing calibration of the CRT and PMT light detection systems in ICARUS.


title: On KB and Levi operators in Banach lattices

abstract: We prove that an order continuous Banach lattice E is a KB-space if and only if each positive compact operator on E is a KB operator. We give conditions on quasi-KB (resp., quasi-Levi) operators to be KB (resp., Levi), study norm completeness and domination for these operators, and show that neither KB nor Levi operators are stable under rank one perturbations.


title: Privacy Preserving Multi-Agent Reinforcement Learning in Supply Chains

abstract: This paper addresses privacy concerns in multi-agent reinforcement learning (MARL), specifically within the context of supply chains where individual strategic data must remain confidential. Organizations within the supply chain are modeled as agents, each seeking to optimize their own objectives while interacting with others. As each organization's strategy is contingent on neighboring strategies, maintaining privacy of state and action-related information is crucial. To tackle this challenge, we propose a game-theoretic, privacy-preserving mechanism, utilizing a secure multi-party computation (MPC) framework in MARL settings. Our major contribution is the successful implementation of a secure MPC framework, SecFloat on EzPC, to solve this problem. However, simply implementing policy gradient methods such as MADDPG operations using SecFloat, while conceptually feasible, would be programmatically intractable. To overcome this hurdle, we devise a novel approach that breaks down the forward and backward pass of the neural network into elementary operations compatible with SecFloat , creating efficient and secure versions of the MADDPG algorithm. Furthermore, we present a learning mechanism that carries out floating point operations in a privacy-preserving manner, an important feature for successful learning in MARL framework. Experiments reveal that there is on average 68.19% less supply chain wastage in 2 PC compared to no data share, while also giving on average 42.27% better average cumulative revenue for each player. This work paves the way for practical, privacy-preserving MARL, promising significant improvements in secure computation within supply chain contexts and broadly.


title: Stellar Spectra Fitting with Amortized Neural Posterior Estimation and nbi

abstract: Modern surveys often deliver hundreds of thousands of stellar spectra at once, which are fit to spectral models to derive stellar parameters/labels. Therefore, the technique of Amortized Neural Posterior Estimation (ANPE) stands out as a suitable approach, which enables the inference of large number of targets as sub-linear/constant computational costs. Leveraging our new nbi software package, we train an ANPE model for the APOGEE survey and demonstrate its efficacy on both mock and real APOGEE stellar spectra. Unique to the nbi package is its out-of-the-box functionality on astronomical inverse problems with sequential data. As such, we have been able to acquire the trained model with minimal effort. We introduce an effective approach to handling the measurement noise properties inherent in spectral data, which utilizes the actual uncertainties in the observed data. This allows training data to resemble observed data, an aspect that is crucial for ANPE applications. Given the association of spectral data properties with the observing instrument, we discuss the utility of an ANPE "model zoo," where models are trained for specific instruments and distributed under the nbi framework to facilitate real-time stellar parameter inference.

categories: astro-ph.SR astro-ph.GA astro-ph.IM cs.LG


title: NLLG Quarterly arXiv Report 09/23: What are the most influential current AI Papers?

abstract: Artificial Intelligence (AI) has witnessed rapid growth, especially in the subfields Natural Language Processing (NLP), Machine Learning (ML) and Computer Vision (CV). Keeping pace with this rapid progress poses a considerable challenge for researchers and professionals in the field. In this arXiv report, the second of its kind, which covers the period from January to September 2023, we aim to provide insights and analysis that help navigate these dynamic areas of AI. We accomplish this by 1) identifying the top-40 most cited papers from arXiv in the given period, comparing the current top-40 papers to the previous report, which covered the period January to June; 2) analyzing dataset characteristics and keyword popularity; 3) examining the global sectoral distribution of institutions to reveal differences in engagement across geographical areas. Our findings highlight the continued dominance of NLP: while only 16% of all submitted papers have NLP as primary category (more than 25% have CV and ML as primary category), 50% of the most cited papers have NLP as primary category, 90% of which target LLMs. Additionally, we show that i) the US dominates among both top-40 and top-9k papers, followed by China; ii) Europe clearly lags behind and is hardly represented in the top-40 most cited papers; iii) US industry is largely overrepresented in the top-40 most influential papers.


title: Heat Transfer in Gold Interfaces Capped with Thiolated Polyethylene Glycol: A Molecular Dynamics Study

abstract: Reverse non-equilibrium molecular dynamics (RNEMD) simulations were used to study heat transport in solvated gold interfaces which have been functionalized with a low molecular weight thiolated polyethylene glycol (PEG). The gold interfaces studied included (111), (110), and (100) facets, as well as spherical nanoparticles with radii of 10 and 20 {\AA}. The embedded atom model (EAM) and the polarizable density-readjusted embedded atom model (DR-EAM) were implemented to determine the effect of metal polarizability on heat transport properties. We find that the interfacial thermal conductance values for thiolated PEG capped interfaces are higher than those for pristine gold interfaces. Hydrogen bonding between the thiolated PEG and solvent differs between planar facets and the nanospheres, suggesting one mechanism for enhanced transfer of energy, while the covalent gold sulfur bond appears to create the largest barrier to thermal conduction. Through analysis of vibrational power spectra, we find an enhanced population at low frequency heat carrying modes for the nanospheres, which may also explain the higher mean G value.


title: Is Ignorance Bliss? The Role of Post Hoc Explanation Faithfulness and Alignment in Model Trust in Laypeople and Domain Experts

abstract: Post hoc explanations have emerged as a way to improve user trust in machine learning models by providing insight into model decision-making. However, explanations tend to be evaluated based on their alignment with prior knowledge while the faithfulness of an explanation with respect to the model, a fundamental criterion, is often overlooked. Furthermore, the effect of explanation faithfulness and alignment on user trust and whether this effect differs among laypeople and domain experts is unclear. To investigate these questions, we conduct a user study with computer science students and doctors in three domain areas, controlling the laypeople and domain expert groups in each setting. The results indicate that laypeople base their trust in explanations on explanation faithfulness while domain experts base theirs on explanation alignment. To our knowledge, this work is the first to show that (1) different factors affect laypeople and domain experts' trust in post hoc explanations and (2) domain experts are subject to specific biases due to their expertise when interpreting post hoc explanations. By uncovering this phenomenon and exposing this cognitive bias, this work motivates the need to educate end users about how to properly interpret explanations and overcome their own cognitive biases, and motivates the development of simple and interpretable faithfulness metrics for end users. This research is particularly important and timely as post hoc explanations are increasingly being used in high-stakes, real-world settings such as medicine.


title: Improving the representation of the atmospheric boundary layer by direct assimilation of ground-based microwave radiometer observations

abstract: In a joint effort, MeteoSwiss and Deutscher Wetterdienst (DWD) address the need for improving the initial state of the atmospheric boundary layer (ABL) by exploiting ground-based profiling observations that aim to fill the existing observational gap in the ABL. We implemented the brightness temperature observations from ground-based microwave radiometers (MWRs) into our data assimilation systems using a local ensemble transform Kalman filter (LETKF) with RTTOV-gb (Radiative Transfer for TOVS, ground-based) as a forward operator. We were able to obtain a positive impact on the brightness temperature first guess and analysis as well as a slight impact on the ABL humidity using two MWRs at MeteoSwiss. These results led to a subsequent operational implementation of the observing system at MeteoSwiss. Furthermore, we performed an extensive set of assimilation experiments at DWD to further investigate various aspects such as the vertical localisation of selected single channels. We obtained a positive impact on the 6h-forecast of ABL temperature and humidity by assimilating two channels employing a dynamical localisation based on the sensitivity functions of RTTOV-gb but also with a static localisation in a single-channel setup. Our experiments indicate the importance of the vertical localisation when using more than one channel, although reliable improvements are challenging to obtain without a larger number of observations for both assimilation and verification.


title: Decay estimates for Cayley transforms and inverses of semigroup generators via the $\mathcal{B}$-calculus

abstract: Let $-A$ be the generator of a bounded $C_0$-semigroup $(e^{-tA})_{t \geq 0}$ on a Hilbert space. First we study the long-time asymptotic behavior of the Cayley transform $V_{\omega}(A) := (A-\omega I) (A+\omega I)^{-1}$ with $\omega >0$. We give a decay estimate for $\|V_{\omega}(A)^nA^{-1}\|$ when $(e^{-tA})_{t \geq 0}$ is polynomially stable. Considering the case where the parameter $\omega$ varies, we estimate $\|(\prod_{k=1}^n V_{\omega_k}(A))A^{-1}\|$ for exponentially stable $C_0$-semigroups $(e^{-tA})_{t \geq 0}$. Next we show that if the generator $-A$ of the bounded $C_0$-semigroup has a bounded inverse, then $\sup_{t \geq 0} \|e^{-tA^{-1}} A^{-\alpha} \| < \infty$ for all $\alpha >0$. We also present an estimate for the rate of decay of $\|e^{-tA^{-1}} A^{-1} \|$, assuming that $(e^{-tA})_{t \geq 0}$ is polynomially stable. To obtain these results, we use operator norm estimates offered by a functional calculus called the $\mathcal{B}$-calculus.


title: Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge

abstract: Large Language Models (LLMs) stand out for their impressive performance in intricate language modeling tasks. However, their demanding computational and memory needs pose obstacles for broad use on edge devices. Quantization is then introduced to boost LLMs' on-device efficiency. Recent works show that 8-bit or lower weight quantization is feasible with minimal impact on end-to-end task performance, while the activation is still not quantized. On the other hand, mainstream commodity edge devices still struggle to execute these sub-8-bit quantized networks effectively. In this paper, we propose Agile-Quant, an activation-guided quantization framework for popular Large Language Models (LLMs), and implement an end-to-end accelerator on multiple edge devices for faster inference. Considering the hardware profiling and activation analysis, we first introduce a basic activation quantization strategy to balance the trade-off of task performance and real inference speed. Then we leverage the activation-aware token pruning technique to reduce the outliers and the adverse impact on attentivity. Ultimately, we utilize the SIMD-based 4-bit multiplier and our efficient TRIP matrix multiplication to implement the accelerator for LLMs on the edge. We apply our framework on different scales of LLMs including LLaMA, OPT, and BLOOM with 4-bit or 8-bit for the activation and 4-bit for the weight quantization. Experiments show that Agile-Quant achieves simultaneous quantization of model weights and activations while maintaining task performance comparable to existing weight-only quantization methods. Moreover, in the 8- and 4-bit scenario, Agile-Quant achieves an on-device speedup of up to 2.55x compared to its FP16 counterparts across multiple edge devices, marking a pioneering advancement in this domain.


title: Numerical 1-loop correction from a potential yielding ultra-slow-roll dynamics

abstract: Single-field models of inflation might lead to amplified scalar fluctuations on small scales due, for example, to a transient ultra-slow-roll phase. It was argued by Kristiano $\&$ Yokoyama in arXiv:2211.03395 that the enhanced amplitude of the scalar power spectrum on small scales has the potential to induce a sizeable 1-loop correction to the spectrum at large scales. In this work, we repeat the calculation for the 1-loop correction presented in arXiv:2211.03395. We closely follow their assumptions but evaluate the loop numerically. This allows us to consider both instantaneous and smooth transitions between the slow-roll and ultra-slow-roll phases. In particular, we generate models featuring realistic, smooth evolution from an analytic inflationary potential. We find that, upon fixing the amplitude of the peak in the power spectrum at short scales, the resulting 1-loop correction is not significantly reduced by considering a smooth evolution. In particular, for a power spectrum with a tree-level peak amplitude potentially relevant for small-scale phenomenology, e.g. primordial black hole production, the 1-loop correction on large scales is a few percent of the tree-level power spectrum.


title: The Counterattack of CNNs in Self-Supervised Learning: Larger Kernel Size might be All You Need

abstract: Vision Transformers have been rapidly uprising in computer vision thanks to their outstanding scaling trends, and gradually replacing convolutional neural networks (CNNs). Recent works on self-supervised learning (SSL) introduce siamese pre-training tasks, on which Transformer backbones continue to demonstrate ever stronger results than CNNs. People come to believe that Transformers or self-attention modules are inherently more suitable than CNNs in the context of SSL. However, it is noteworthy that most if not all prior arts of SSL with CNNs chose the standard ResNets as their backbones, whose architecture effectiveness is known to already lag behind advanced Vision Transformers. Therefore, it remains unclear whether the self-attention operation is crucial for the recent advances in SSL - or CNNs can deliver the same excellence with more advanced designs, too? Can we close the SSL performance gap between Transformers and CNNs? To answer these intriguing questions, we apply self-supervised pre-training to the recently proposed, stronger lager-kernel CNN architecture and conduct an apple-to-apple comparison with Transformers, in their SSL performance. Our results show that we are able to build pure CNN SSL architectures that perform on par with or better than the best SSL-trained Transformers, by just scaling up convolutional kernel sizes besides other small tweaks. Impressively, when transferring to the downstream tasks \texttt{MS COCO} detection and segmentation, our SSL pre-trained CNN model (trained in 100 epochs) achieves the same good performance as the 300-epoch pre-trained Transformer counterpart. We hope this work can help to better understand what is essential (or not) for self-supervised learning backbones.


title: GPT-4 and Safety Case Generation: An Exploratory Analysis

abstract: In the ever-evolving landscape of software engineering, the emergence of large language models (LLMs) and conversational interfaces, exemplified by ChatGPT, is nothing short of revolutionary. While their potential is undeniable across various domains, this paper sets out on a captivating expedition to investigate their uncharted territory, the exploration of generating safety cases. In this paper, our primary objective is to delve into the existing knowledge base of GPT-4, focusing specifically on its understanding of the Goal Structuring Notation (GSN), a well-established notation allowing to visually represent safety cases. Subsequently, we perform four distinct experiments with GPT-4. These experiments are designed to assess its capacity for generating safety cases within a defined system and application domain. To measure the performance of GPT-4 in this context, we compare the results it generates with ground-truth safety cases created for an X-ray system system and a Machine-Learning (ML)-enabled component for tire noise recognition (TNR) in a vehicle. This allowed us to gain valuable insights into the model's generative capabilities. Our findings indicate that GPT-4 demonstrates the capacity to produce safety arguments that are moderately accurate and reasonable. Furthermore, it exhibits the capability to generate safety cases that closely align with the semantic content of the reference safety cases used as ground-truths in our experiments.


title: Environment induced dynamical quantum phase transitions in two-qubit Rabi model

abstract: The physics of quantum states beyond thermodynamic equilibrium represents a fascinating and cutting-edge research. Using numerical state-of-the-art approaches, we observe dynamical quantum phase transitions in the dissipative two-qubit Rabi model. By quenching the qubits-oscillator coupling, the system (Rabi + Environment) exhibits dynamical quantum phase transitions signalled by kinks of Loschmidt echo's rate function at parameter values close to thermodynamic transition. Notably, these transitions also manifest in two-qubit entanglement. While at equilibrium one class of Beretzinski-Kosterlitz-Thouless-type transitions occurs, non-equilibrium conditions reveal two classes of dynamical critical phenomena, depending on qubits' interactions and entanglement. When qubits directly interact, the kink critical exponent describes a linear behavior, reminiscent of nearest neighbors Ising chains, with short-range interactions dominating at short times. Conversely, non-interacting qubits exhibit critical exponents much smaller than unity due to bath-induced long-range interactions. These findings shed light on the complex behavior of dynamical quantum phase transitions in non-integrable models, showing unusual entanglement features and the environment's significant role.


title: Unsupervised Multi-modal Feature Alignment for Time Series Representation Learning

abstract: In recent times, the field of unsupervised representation learning (URL) for time series data has garnered significant interest due to its remarkable adaptability across diverse downstream applications. Unsupervised learning goals differ from downstream tasks, making it tricky to ensure downstream task utility by focusing only on temporal feature characterization. Researchers have proposed multiple transformations to extract discriminative patterns implied in informative time series, trying to fill the gap. Despite the introduction of a variety of feature engineering techniques, e.g. spectral domain, wavelet transformed features, features in image form and symbolic features etc. the utilization of intricate feature fusion methods and dependence on heterogeneous features during inference hampers the scalability of the solutions. To address this, our study introduces an innovative approach that focuses on aligning and binding time series representations encoded from different modalities, inspired by spectral graph theory, thereby guiding the neural encoder to uncover latent pattern associations among these multi-modal features. In contrast to conventional methods that fuse features from multiple modalities, our proposed approach simplifies the neural architecture by retaining a single time series encoder, consequently leading to preserved scalability. We further demonstrate and prove mechanisms for the encoder to maintain better inductive bias. In our experimental evaluation, we validated the proposed method on a diverse set of time series datasets from various domains. Our approach outperforms existing state-of-the-art URL methods across diverse downstream tasks.


title: Products of curves as ball quotients

abstract: For any $g_1, g_2 \ge 0$, this paper shows that there is a cocompact lattice $\Gamma < \mathrm{PU}(2,1)$ such that the ball quotient $\Gamma \backslash \mathbb{B}^2$ is birational to a product $C_1 \times C_2$ of smooth projective curves $C_j$ of genus $g_j$. The only prior examples were $\mathbb{P}^1 \times \mathbb{P}^1$, due to Deligne--Mostow and rediscovered by many others, and a lesser-known product of elliptic curves whose existence follows from work of Hirzebruch. Combined with related new examples, this answers the rational variant of a question of Gromov in the positive for surfaces of Kodaira dimension $\kappa \le 0$, namely that they admit deformations $V^\prime$ such that there is a compact ball quotient $\Gamma \backslash \mathbb{B}^2$ with a rational map $\Gamma \backslash \mathbb{B}^2 \dashrightarrow V^\prime$. Often the proof gives the stronger conclusion that $V^\prime$ is birational to a ball quotient orbifold. All examples are shown to be arithmetic, and even arithmeticity of Hirzebruch's example appears to be new.


title: Influence Analysis with Panel Data

abstract: The presence of units with extreme values in the dependent and/or independent variables (i.e., vertical outliers, leveraged data) has the potential to severely bias regression coefficients and/or standard errors. This is common with short panel data because the researcher cannot advocate asymptotic theory. Example include cross-country studies, cell-group analyses, and field or laboratory experimental studies, where the researcher is forced to use few cross-sectional observations repeated over time due to the structure of the data or research design. Available diagnostic tools may fail to properly detect these anomalies, because they are not designed for panel data. In this paper, we formalise statistical measures for panel data models with fixed effects to quantify the degree of leverage and outlyingness of units, and the joint and conditional influences of pairs of units. We first develop a method to visually detect anomalous units in a panel data set, and identify their type. Second, we investigate the effect of these units on LS estimates, and on other units' influence on the estimated parameters. To illustrate and validate the proposed method, we use a synthetic data set contaminated with different types of anomalous units. We also provide an empirical example.


title: Optical transmission of a moving Fabry-Perot interferometer

abstract: Fabry-Perot interferometers have been widely studied and used for well over a century. However, they have always been treated as stationary devices in the past. In this paper, we investigate the optical transmission of a longitudinally moving Fabry-Perot interferometer within the framework of relativity and establish a general relation between the transmission coefficient and the velocity for uniform motions. Several features of the transmission spectrum are analyzed, including velocity-dependent frequency rescaling, the Fabry-Perot transmittance, and the transmission phase near resonance. Special attentions are given to the non-relativistic regime, where application prospects are evaluated. Potential new interferometric schemes, such as velocity-scanning interferometry and hybrid interferometers based on nested configurations, are proposed. Finally, a special case of non-uniform motion is also investigated.


title: Intrinsic Grain Boundary Shear Coupling Tensor

abstract: Grain boundary (GB) migration stands as a linchpin process governing microstructural evolution in polycrystalline materials. Over the past decade, the concept of shear coupling, quantified through the shear coupling factor, has transformed our understanding and driven the development of theoretical frameworks for unifying GB behaviors. In this study, we unveil a critical shortcoming in the definition of shear coupling factor and introduce a mathematically redefined descriptor - shear coupling strength - along with its corresponding tensor. This innovation accurately characterizes intrinsic shear coupling properties across diverse GBs and reveals complex dynamics within the GB mobility tensor, with a mathematical proof affirming its symmetry that is assumed in previous studies. Additionally, an efficient methodology has been developed for streamlined extraction of both shear coupling and GB mobility tensors from atomistic simulations. This advancement holds the potential to sample GB behavior across extensive datasets, significantly enhancing our ability to predict structure-property relationships within the expansive 5-parameter space of GBs.


title: Grid-free stochastic simulations of reaction-diffusion processes at cell-cell contacts

abstract: Biological cells can exchange messages through soluble molecules or membrane-bound receptors. In particular in the latter case, the interaction is usually located in specific regions of the interacting cells and may depend on or induce local morphological features or reorganizations of the membrane-associated or membrane-proximal biochemistry. Examples are interactions among cells that probe their environment and the surfaces of neighboring cells with dendrite-like protrusions. Previously, we introduced an algorithm capable of creating spatially-resolved, stochastic particle-based computational models of cellular biochemistry. However, its applicability was limited to the case of soluble ligands. Here, building on this earlier approach, we introduce a simulation method that accounts for the unique features that distinguish the sites of cell-cell contact from other regions of the cell membrane. Employing a smooth, grid-free computational representation of the contact sites, we extend the applicability of Green's function-based stochastic simulation algorithms to study intermembrane bimolecular interactions between particles on adjacent curved surfaces. We demonstrate the utility of this computational approach by exploring the effects of contact geometry and local biochemistry for the early phase of T-cell receptor activation.


title: On the Ground and in the Sky: A Tutorial on Radio Localization in Ground-Air-Space Networks

abstract: The inherent limitations in scaling up ground infrastructure for future wireless networks, combined with decreasing operational costs of aerial and space networks, are driving considerable research interest in multisegment ground-air-space (GAS) networks. In GAS networks, where ground and aerial users share network resources, ubiquitous and accurate user localization becomes indispensable, not only as an end-user service but also as an enabler for location-aware communications. This breaks the convention of having localization as a byproduct in networks primarily designed for communications. To address these imperative localization needs, the design and utilization of ground, aerial, and space anchors require thorough investigation. In this tutorial, we provide an in-depth systemic analysis of the radio localization problem in GAS networks, considering ground and aerial users as targets to be localized. Starting from a survey of the most relevant works, we then define the key characteristics of anchors and targets in GAS networks. Subsequently, we detail localization fundamentals in GAS networks, considering 3D positions, orientations, and velocities. Afterward, we thoroughly analyze radio localization systems in GAS networks, detailing the system model, design aspects, and considerations for each of the three GAS anchors. Preliminary results are presented to provide a quantifiable perspective on key design aspects in GAS-based localization scenarios. We then identify the vital roles 6G enablers are expected to play in radio localization in GAS networks.


title: Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC

abstract: Second-order methods such as KFAC can be useful for neural net training. However, they are often memory-inefficient since their preconditioning Kronecker factors are dense, and numerically unstable in low precision as they require matrix inversion or decomposition. These limitations render such methods unpopular for modern mixed-precision training. We address them by (i) formulating an inverse-free KFAC update and (ii) imposing structures in the Kronecker factors, resulting in structured inverse-free natural gradient descent (SINGD). On modern neural networks, we show that SINGD is memory-efficient and numerically robust, in contrast to KFAC, and often outperforms AdamW even in half precision. Our work closes a gap between first- and second-order methods in modern low-precision training.


title: Bit Blasting Probabilistic Programs

abstract: Probabilistic programming languages (PPLs) are expressive means for creating and reasoning about probabilistic models. Unfortunately hybrid probabilistic programs, involving both continuous and discrete structures, are not well supported by today's PPLs. In this paper we develop a new approximate inference algorithm for hybrid probabilistic programs that first discretizes the continuous distributions and then performs discrete inference on the resulting program. The key novelty is a form of discretization that we call bit blasting, which uses a binary representation of numbers such that a domain of $2^b$ discretized points can be succinctly represented as a discrete probabilistic program over poly($b$) Boolean random variables. Surprisingly, we prove that many common continuous distributions can be bit blasted in a manner that incurs no loss of accuracy over an explicit discretization and supports efficient probabilistic inference. We have built a probabilistic programming system for hybrid programs called HyBit, which employs bit blasting followed by discrete probabilistic inference. We empirically demonstrate the benefits of our approach over existing sampling-based and symbolic inference approaches.


title: Non-Cartesian Self-Supervised Physics-Driven Deep Learning Reconstruction for Highly-Accelerated Multi-Echo Spiral fMRI

abstract: Functional MRI (fMRI) is an important tool for non-invasive studies of brain function. Over the past decade, multi-echo fMRI methods that sample multiple echo times has become popular with potential to improve quantification. While these acquisitions are typically performed with Cartesian trajectories, non-Cartesian trajectories, in particular spiral acquisitions, hold promise for denser sampling of echo times. However, such acquisitions require very high acceleration rates for sufficient spatiotemporal resolutions. In this work, we propose to use a physics-driven deep learning (PD-DL) reconstruction to accelerate multi-echo spiral fMRI by 10-fold. We modify a self-supervised learning algorithm for optimized training with non-Cartesian trajectories and use it to train the PD-DL network. Results show that the proposed self-supervised PD-DL reconstruction achieves high spatio-temporal resolution with meaningful BOLD analysis.

categories: eess.IV cs.CV cs.LG eess.SP physics.med-ph


title: Context Tuning for Retrieval Augmented Generation

abstract: Large language models (LLMs) have the remarkable ability to solve new tasks with just a few examples, but they need access to the right tools. Retrieval Augmented Generation (RAG) addresses this problem by retrieving a list of relevant tools for a given task. However, RAG's tool retrieval step requires all the required information to be explicitly present in the query. This is a limitation, as semantic search, the widely adopted tool retrieval method, can fail when the query is incomplete or lacks context. To address this limitation, we propose Context Tuning for RAG, which employs a smart context retrieval system to fetch relevant information that improves both tool retrieval and plan generation. Our lightweight context retrieval model uses numerical, categorical, and habitual usage signals to retrieve and rank context items. Our empirical results demonstrate that context tuning significantly enhances semantic search, achieving a 3.5-fold and 1.5-fold improvement in Recall@K for context retrieval and tool retrieval tasks respectively, and resulting in an 11.6% increase in LLM-based planner accuracy. Additionally, we show that our proposed lightweight model using Reciprocal Rank Fusion (RRF) with LambdaMART outperforms GPT-4 based retrieval. Moreover, we observe context augmentation at plan generation, even after tool retrieval, reduces hallucination.


title: Global Centers in a Class of Quintic Polynomial Differential System

abstract: A center of a differential system in the plane $\mathbb{R}^2$ is an equilibrium point $p$ having a neighborhood $U$ such that $U\setminus \{p\}$ is filled of periodic orbits. A center $p$ is global when $\mathbb{R}^2\setminus \{p\}$ is filled of periodic orbits. In general is a difficult problem to distinguish the centers from the foci for a given class of differential systems, and also it is difficult to distinguish the global centers inside the centers. The goal of this paper is to classify the centers and the global centers of the following class of quintic polynomial differential systems $$ \dot{x}= y,\quad \dot{y}=-x+a_{05}\,y^5+a_{14}\,x\,y^4+a_{23}\,x^2\,y^3+a_{32}\,x^3\,y^2+a_{41}\,x^4\,y+a_{50}\,x^5, $$ in the plane $\mathbb{R}^2$.


title: A Novel Discrete Phase Shift Design for RIS-Assisted Multi-User MIMO

abstract: Reconfigurable intelligent surface (RIS) is a newly-emerged technology that might fundamentally change how wireless networks are operated. Though extensively studied in recent years, the practical limitations of RIS are often neglected when assessing the performance of RIS-assisted communication networks. One of these limitations is that each RIS element is restricted to incur a controllable phase shift to the reflected signal from a predefined discrete set. This paper studies an RIS-assisted multi-user multiple-input multiple-output (MIMO) system, where an RIS with discrete phase shifts assists in simultaneous uplink data transmission from multiple user equipments (UEs) to a base station (BS). We aim to maximize the sum rate by optimizing the receive beamforming vectors and RIS phase shift configuration. To this end, we transform the original sum-rate maximization problem into a minimum mean square error (MMSE) minimization problem and employ the block coordinate descent (BCD) technique for iterative optimization of the variables until convergence. We formulate the discrete RIS phase shift optimization problem as a mixed-integer least squares problem and propose a novel method based on sphere decoding (SD) to solve it. Through numerical evaluation, we show that the proposed discrete phase shift design outperforms the conventional nearest point mapping method, which is prevalently used in previous works.


title: Spectral Data of Special Orthogonal Higgs Bundles and Hecke Modification

abstract: We give a complete, self-contained computation of the spectral data parametrising Higgs bundles in the generic fibres of the $\mathrm{SO}_{2n+1}$-Hitchin fibration where the Higgs fields are $L$-twisted endomorphisms. Although the spectral data is known in the literature, we develop a new approach to spectral data, which takes advantage of Hecke modification. Further, we present the computation for the $\mathrm{Sp}_{2n}$ and $\mathrm{SO}_{2n}$ cases while clarifying some aspects of the correspondence which are not well explained in the pre-existing literature. We also compute the number of connected components of the generic fibres, and demonstrate Langlands duality in the fibres via the canonical duality in the fibres.


title: Entanglement Entropy Distributions of a Muon Decay

abstract: Divergences that occur in density matrices of decay and scattering processes are shown to be regularized by tracing and unitarity or the optical theorem. These divergences are regularized by the lifetime of the decaying particle or the total scattering cross section. Also, this regularization is shown to give the expected helicities of final particles. The density matrix is derived for the weak decay of a polarized muon at rest, $\mu^- \rightarrow \nu_{\mu} (e^- \bar \nu_e)$, with Lorentz invariant density matrix entries and unitarity upheld at tree level. The electron's von Neumann entanglement entropy distributions are calculated with respect to both the electron's emission angle and energy. The angular entropy distribution favors an electron emitted backwards with respect to the muon's polarization given a minimum volume regularization. The kinematic entropy distribution is maximal at half the muon's rest mass energy. These results are similar to the electron's angular and kinematic decay rate distributions. Both the density matrix and entanglement entropy can be cast either in terms of ratios of areas or volumes.


title: LOFAR discovery and wide-band characterisation of an ultra-steep spectrum AGN radio remnant associated with Abell 1318

abstract: We present the discovery of a very extended (550 kpc) and low-surface-brightness ($ 3.3 \mu \mathrm{Jy} \, arcsec^{-2} $ at 144 MHz) radio emission region in Abell 1318. These properties are consistent with its characterisation as an active galactic nucleus (AGN) remnant radio plasma, based on its morphology and radio spectral properties. We performed a broad-band (54 - 1400 MHz) radio spectral index and curvature analysis using LOFAR, uGMRT, and WSRT-APERTIF data. We also derived the radiative age of the detected emission, estimating a maximum age of 250 Myr. The morphology of the source is remarkably intriguing, with two larger, oval-shaped components and a thinner, elongated, and filamentary structure in between, plausibly reminiscent of two aged lobes and a jet. Based on archival {\it Swift} as well as SDSS data we performed an X-ray and optical characterisation of the system, whose virial mass was estimated to be $ \sim 7.4 \times 10^{13} \, \mathrm{M} _{\odot}$. This places A1318 in the galaxy group regime. Interestingly, the radio source does not have a clear optical counterpart embedded in it, thus, we propose that it is most likely an unusual AGN remnant of previous episode(s) of activity of the AGN hosted by the brightest group galaxy ($ \sim 2.6 \times 10^{12} \, \mathrm{M} _{\odot}$), which is located at a projected distance of $\sim$170 kpc in the current epoch. This relatively high offset may be a result of IGrM sloshing sourced by a minor merger. The filamentary morphology of the source may suggest that the remnant plasma has been perturbed by the system dynamics, however, only future deeper X-ray observations will be able to address this question.


title: Vivisecting the Dissection: On the Role of Trusted Components in BFT Protocols

abstract: A recent paper by Gupta et al. (EuroSys'23) challenged the usefulness of trusted component (TC) based Byzantine fault-tolerant (BFT) protocols to lower the replica group size from $3f+1$ to $2f+1$, identifying three limitations of such protocols and proposing that TCs should be used instead to improve the performance of BFT protocols. Here, we point out flaws in both arguments and advocate that the most worthwhile use of TCs in BFT protocols is indeed to make them as resilient as crash fault-tolerant (CFT) protocols, which can tolerate up to $f$ faulty replicas using $2f+1$ replicas.


title: Micro-Macro Consistency in Multiscale Modeling: Score-Based Model Assisted Sampling of Fast/Slow Dynamical Systems

abstract: A valuable step in the modeling of multiscale dynamical systems in fields such as computational chemistry, biology, materials science and more, is the representative sampling of the phase space over long timescales of interest; this task is not, however, without challenges. For example, the long term behavior of a system with many degrees of freedom often cannot be efficiently computationally explored by direct dynamical simulation; such systems can often become trapped in local free energy minima. In the study of physics-based multi-time-scale dynamical systems, techniques have been developed for enhancing sampling in order to accelerate exploration beyond free energy barriers. On the other hand, in the field of Machine Learning, a generic goal of generative models is to sample from a target density, after training on empirical samples from this density. Score based generative models (SGMs) have demonstrated state-of-the-art capabilities in generating plausible data from target training distributions. Conditional implementations of such generative models have been shown to exhibit significant parallels with long-established -- and physics based -- solutions to enhanced sampling. These physics-based methods can then be enhanced through coupling with the ML generative models, complementing the strengths and mitigating the weaknesses of each technique. In this work, we show that that SGMs can be used in such a coupling framework to improve sampling in multiscale dynamical systems.


title: Initialization Matters for Adversarial Transfer Learning

abstract: With the prevalence of the Pretraining-Finetuning paradigm in transfer learning, the robustness of downstream tasks has become a critical concern. In this work, we delve into adversarial robustness in transfer learning and reveal the critical role of initialization, including both the pretrained model and the linear head. First, we discover the necessity of an adversarially robust pretrained model. Specifically, we reveal that with a standard pretrained model, Parameter-Efficient Finetuning (PEFT) methods either fail to be adversarially robust or continue to exhibit significantly degraded adversarial robustness on downstream tasks, even with adversarial training during finetuning. Leveraging a robust pretrained model, surprisingly, we observe that a simple linear probing can outperform full finetuning and other PEFT methods with random initialization on certain datasets. We further identify that linear probing excels in preserving robustness from the robust pretraining. Based on this, we propose Robust Linear Initialization (RoLI) for adversarial finetuning, which initializes the linear head with the weights obtained by adversarial linear probing to maximally inherit the robustness from pretraining. Across five different image classification datasets, we demonstrate the effectiveness of RoLI and achieve new state-of-the-art results. Our code is available at \url{https://github.com/DongXzz/RoLI}.


title: Forecasting Lithium-Ion Battery Longevity with Limited Data Availability: Benchmarking Different Machine Learning Algorithms

abstract: As the use of Lithium-ion batteries continues to grow, it becomes increasingly important to be able to predict their remaining useful life. This work aims to compare the relative performance of different machine learning algorithms, both traditional machine learning and deep learning, in order to determine the best-performing algorithms for battery cycle life prediction based on minimal data. We investigated 14 different machine learning models that were fed handcrafted features based on statistical data and split into 3 feature groups for testing. For deep learning models, we tested a variety of neural network models including different configurations of standard Recurrent Neural Networks, Gated Recurrent Units, and Long Short Term Memory with and without attention mechanism. Deep learning models were fed multivariate time series signals based on the raw data for each battery across the first 100 cycles. Our experiments revealed that the machine learning algorithms on handcrafted features performed particularly well, resulting in 10-20% average mean absolute percentage error. The best-performing algorithm was the Random Forest Regressor, which gave a minimum 9.8% mean absolute percentage error. Traditional machine learning models excelled due to their capability to comprehend general data set trends. In comparison, deep learning models were observed to perform particularly poorly on raw, limited data. Algorithms like GRU and RNNs that focused on capturing medium-range data dependencies were less adept at recognizing the gradual, slow trends critical for this task. Our investigation reveals that implementing machine learning models with hand-crafted features proves to be more effective than advanced deep learning models for predicting the remaining useful Lithium-ion battery life with limited data availability.