title: Convolutional L2LFlows: Generating Accurate Showers in Highly Granular Calorimeters Using Convolutional Normalizing Flows

abstract: In the quest to build generative surrogate models as computationally efficient alternatives to rule-based simulations, the quality of the generated samples remains a crucial frontier. So far, normalizing flows have been among the models with the best fidelity. However, as the latent space in such models is required to have the same dimensionality as the data space, scaling up normalizing flows to high dimensional datasets is not straightforward. The prior L2LFlows approach successfully used a series of separate normalizing flows and sequence of conditioning steps to circumvent this problem. In this work, we extend L2LFlows to simulate showers with a 9-times larger profile in the lateral direction. To achieve this, we introduce convolutional layers and U-Net-type connections, move from masked autoregressive flows to coupling layers, and demonstrate the successful modelling of showers in the ILD Electromagnetic Calorimeter as well as Dataset 3 from the public CaloChallenge dataset.

categories: physics.ins-det cs.LG hep-ex hep-ph physics.data-an


title: Quantum encoder for fixed Hamming-weight subspaces

abstract: We present an exact $n$-qubit computational-basis amplitude encoder of real- or complex-valued data vectors of $d=\binom{n}{k}$ components into a subspace of fixed Hamming weight $k$. This represents a polynomial space compression. The circuit is optimal in that it expresses an arbitrary data vector using only $d-1$ (controlled) Reconfigurable Beam Splitter (RBS) gates and is constructed by an efficient classical algorithm that sequentially generates all bitstrings of weight $k$ and identifies all gate parameters. An explicit compilation into CNOTs and single-qubit gates is presented, with the total CNOT-gate count of $\mathcal{O}(k\, d)$ provided in analytical form. In addition, we show how to load data in the binary basis by sequentially stacking encoders of different Hamming weights using $\mathcal{O}(d\,\log(d))$ CNOT gates. Moreover, using generalized RBS gates that mix states of different Hamming weights, we extend the construction to efficiently encode arbitrary sparse vectors. Finally, we perform an experimental proof-of-principle demonstration of our scheme on a commercial trapped-ion quantum computer. We successfully upload a $q$-Gaussian probability distribution in the non-log-concave regime with $n = 6$ and $k = 2$. We also showcase how the effect of hardware noise can be alleviated by quantum error mitigation. Our results constitute a versatile framework for quantum data compression with various potential applications in fields such as quantum chemistry, quantum machine learning, and constrained combinatorial optimizations.


title: ZTF SN Ia DR2: Peculiar velocities impact on the Hubble diagram

abstract: SNe Ia are used to determine the distance-redshift relation and build the Hubble diagram. Neglecting their host-galaxy peculiar velocities (PVs) may bias the measurement of cosmological parameters. The smaller the redshift, the larger the effect is. We use realistic simulations of SNe Ia observed by the Zwicky Transient Facility (ZTF) to investigate the effect of different methods to take into account PVs. We study the impact of neglecting galaxy PVs and their correlations in an analysis of the SNe Ia Hubble diagram. We find that it is necessary to use the PV full covariance matrix computed from the velocity power spectrum to take into account the sample variance. Considering the results we have obtained using simulations, we determine the PV systematic effects in the context of the ZTF DR2 SNe Ia sample. We determine the PV impact on the intercept of the Hubble diagram, $a_B$, which is directly linked to the measurement of $H_0$. We show that not taking into account PVs and their correlations results in a shift of the $H_0$ value of about $1.0$km.s$^{-1}$.Mpc$^{-1}$ and a slight underestimation of the $H_0$ error bar.


title: SeamlessExpressiveLM: Speech Language Model for Expressive Speech-to-Speech Translation with Chain-of-Thought

abstract: Expressive speech-to-speech translation (S2ST) is a key research topic in seamless communication, which focuses on the preservation of semantics and speaker vocal style in translated speech. Early works synthesized speaker style aligned speech in order to directly learn the mapping from speech to target speech spectrogram. Without reliance on style aligned data, recent studies leverage the advances of language modeling (LM) and build cascaded LMs on semantic and acoustic tokens. This work proposes SeamlessExpressiveLM, a single speech language model for expressive S2ST. We decompose the complex source-to-target speech mapping into intermediate generation steps with chain-of-thought prompting. The model is first guided to translate target semantic content and then transfer the speaker style to multi-stream acoustic units. Evaluated on Spanish-to-English and Hungarian-to-English translations, SeamlessExpressiveLM outperforms cascaded LMs in both semantic quality and style transfer, meanwhile achieving better parameter efficiency.


title: Asteroid Kamo`oalewa's journey from the lunar Giordano Bruno crater to Earth 1:1 resonance

abstract: Among the nearly 30,000 known near-Earth asteroids (NEAs), only tens of them possess Earth co-orbital characteristics with semi-major axes $\sim$1 au. In particular, 469219 Kamo`oalewa (2016 HO3), upcoming target of China's Tianwen-2 asteroid sampling mission, exhibits a meta-stable 1:1 mean-motion resonance with Earth. Intriguingly, recent ground-based observations show that Kamo`oalewa has spectroscopic characteristics similar to space-weathered lunar silicates, hinting at a lunar origin instead of an asteroidal one like the vast majority of NEAs. Here we use numerical simulations to demonstrate that Kamo`oalewa's physical and orbital properties are compatible with a fragment from a crater larger than 10--20 km formed on the Moon in the last few million years. The impact could have ejected sufficiently large fragments into heliocentric orbits, some of which could be transferred to Earth 1:1 resonance and persist today. This leads us to suggest the young lunar crater Giordano Bruno (22 km diameter, 1--10 Ma age) as the most likely source, linking a specific asteroid in space to its source crater on the Moon. The hypothesis will be tested by the Tianwen-2 mission when it returns a sample of Kamo`oalewa. And the upcoming NEO Surveyor mission will possibly help us to identify such a lunar-derived NEA population.


title: Audio2Rig: Artist-oriented deep learning tool for facial animation

abstract: Creating realistic or stylized facial and lip sync animation is a tedious task. It requires lot of time and skills to sync the lips with audio and convey the right emotion to the character's face. To allow animators to spend more time on the artistic and creative part of the animation, we present Audio2Rig: a new deep learning based tool leveraging previously animated sequences of a show, to generate facial and lip sync rig animation from an audio file. Based in Maya, it learns from any production rig without any adjustment and generates high quality and stylized animations which mimic the style of the show. Audio2Rig fits in the animator workflow: since it generates keys on the rig controllers, the animation can be easily retaken. The method is based on 3 neural network modules which can learn an arbitrary number of controllers. Hence, different configurations can be created for specific parts of the face (such as the tongue, lips or eyes). With Audio2Rig, animators can also pick different emotions and adjust their intensities to experiment or customize the output, and have high level controls on the keyframes setting. Our method shows excellent results, generating fine animation details while respecting the show style. Finally, as the training relies on the studio data and is done internally, it ensures data privacy and prevents from copyright infringement.


title: Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters

abstract: Large Language Models (LLMs) are typically harmless but remain vulnerable to carefully crafted prompts known as ``jailbreaks'', which can bypass protective measures and induce harmful behavior. Recent advancements in LLMs have incorporated moderation guardrails that can filter outputs, which trigger processing errors for certain malicious questions. Existing red-teaming benchmarks often neglect to include questions that trigger moderation guardrails, making it difficult to evaluate jailbreak effectiveness. To address this issue, we introduce JAMBench, a harmful behavior benchmark designed to trigger and evaluate moderation guardrails. JAMBench involves 160 manually crafted instructions covering four major risk categories at multiple severity levels. Furthermore, we propose a jailbreak method, JAM (Jailbreak Against Moderation), designed to attack moderation guardrails using jailbreak prefixes to bypass input-level filters and a fine-tuned shadow model functionally equivalent to the guardrail model to generate cipher characters to bypass output-level filters. Our extensive experiments on four LLMs demonstrate that JAM achieves higher jailbreak success ($\sim$ $\times$ 19.88) and lower filtered-out rates ($\sim$ $\times$ 1/6) than baselines.


title: The Impact of Ontology on the Prediction of Cardiovascular Disease Compared to Machine Learning Algorithms

abstract: Cardiovascular disease is one of the chronic diseases that is on the rise. The complications occur when cardiovascular disease is not discovered early and correctly diagnosed at the right time. Various machine learning approaches, including ontology-based Machine Learning techniques, have lately played an essential role in medical science by building an automated system that can identify heart illness. This paper compares and reviews the most prominent machine learning algorithms, as well as ontology-based Machine Learning classification. Random Forest, Logistic regression, Decision Tree, Naive Bayes, k-Nearest Neighbours, Artificial Neural Network, and Support Vector Machine were among the classification methods explored. The dataset used consists of 70000 instances and can be downloaded from the Kaggle website. The findings are assessed using performance measures generated from the confusion matrix, such as F-Measure, Accuracy, Recall, and Precision. The results showed that the ontology outperformed all the machine learning algorithms.


title: Differentially Private Boxplots

abstract: Despite the potential of differentially private data visualization to harmonize data analysis and privacy, research in this area remains relatively underdeveloped. Boxplots are a widely popular visualization used for summarizing a dataset and for comparison of multiple datasets. Consequentially, we introduce a differentially private boxplot. We evaluate its effectiveness for displaying location, scale, skewness and tails of a given empirical distribution. In our theoretical exposition, we show that the location and scale of the boxplot are estimated with optimal sample complexity, and the skewness and tails are estimated consistently. In simulations, we show that this boxplot performs similarly to a non-private boxplot, and it outperforms a boxplot naively constructed from existing differentially private quantile algorithms. Additionally, we conduct a real data analysis of Airbnb listings, which shows that comparable analysis can be achieved through differentially private boxplot visualization.


title: First Tree-like Quantum Data Structure: Quantum B+ Tree

abstract: Quantum computing is a popular topic in computer science, which has recently attracted many studies in various areas such as machine learning and network. However, the topic of quantum data structures seems neglected. There is an open problem in the database area: Can we improve existing data structures by quantum techniques? Consider a dataset of key-record pairs. Given an interval as a query range, a classical B+ tree can report all the records with keys within this interval, which is called a range query, in O(log N + k) time, where N is the total number of records and k is the output size. It is asymptotically optimal in a classical computer but not efficient enough in a quantum computer, because it is expected that the execution time and the output size are linear in a quantum computer. In this paper, we propose the quantum range query problem. Different from the classical range queries, a quantum range query returns the results in quantum bits, which has broad potential applications due to the foreseeable advance of quantum computers and quantum algorithms. To the best of our knowledge, we design the first tree-like quantum data structure called the quantum B+ tree. Based on this data structure, we propose a hybrid quantum-classical algorithm to do the range search. It answers a static quantum range query in O(log_B N) time, which is asymptotically optimal in quantum computers. Since the execution time does not depend on the output size (i.e., k, which could be as large as O(N)), it is significantly faster than the classical data structure. Moreover, we extend our quantum B+ tree to answer the dynamic and d-dimensional quantum range queries efficiently in O(log^2_B N) and O(log^d_B N) time, respectively. Our experimental results show that our proposed quantum data structures achieve up to 1000x improvement in the number of memory accesses compared to their classical competitors.


title: Orderly divergence of Levy Gamma integrals

abstract: ``Orderly divergence'' deals with limit theorems for weighted stochastic Gamma integrals of otherwise nonintegrable functions. Although for monotonic functions this category usually coincides with the classical notion of weighted limit theorems for sums of i.i.d. random variables but there are exceptions and the lack of monotonicity reveals new aspects that are absent in the discrete case.


title: A Bayesian joint model of multiple nonlinear longitudinal and competing risks outcomes for dynamic prediction in multiple myeloma: joint estimation and corrected two-stage approaches

abstract: Predicting cancer-associated clinical events is challenging in oncology. In Multiple Myeloma (MM), a cancer of plasma cells, disease progression is determined by changes in biomarkers, such as serum concentration of the paraprotein secreted by plasma cells (M-protein). Therefore, the time-dependent behaviour of M-protein and the transition across lines of therapy (LoT) that may be a consequence of disease progression should be accounted for in statistical models to predict relevant clinical outcomes. Furthermore, it is important to understand the contribution of the patterns of longitudinal biomarkers, upon each LoT initiation, to time-to-death or time-to-next-LoT. Motivated by these challenges, we propose a Bayesian joint model for trajectories of multiple longitudinal biomarkers, such as M-protein, and the competing risks of death and transition to next LoT. Additionally, we explore two estimation approaches for our joint model: simultaneous estimation of all parameters (joint estimation) and sequential estimation of parameters using a corrected two-stage strategy aiming to reduce computational time. Our proposed model and estimation methods are applied to a retrospective cohort study from a real-world database of patients diagnosed with MM in the US from January 2015 to February 2022. We split the data into training and test sets in order to validate the joint model using both estimation approaches and make dynamic predictions of times until clinical events of interest, informed by longitudinally measured biomarkers and baseline variables available up to the time of prediction.


title: Enhancing Antibiotic Stewardship using a Natural Language Approach for Better Feature Representation

abstract: The rapid emergence of antibiotic-resistant bacteria is recognized as a global healthcare crisis, undermining the efficacy of life-saving antibiotics. This crisis is driven by the improper and overuse of antibiotics, which escalates bacterial resistance. In response, this study explores the use of clinical decision support systems, enhanced through the integration of electronic health records (EHRs), to improve antibiotic stewardship. However, EHR systems present numerous data-level challenges, complicating the effective synthesis and utilization of data. In this work, we transform EHR data into a serialized textual representation and employ pretrained foundation models to demonstrate how this enhanced feature representation can aid in antibiotic susceptibility predictions. Our results suggest that this text representation, combined with foundation models, provides a valuable tool to increase interpretability and support antibiotic stewardship efforts.


title: Back to the Basics on Predicting Transfer Performance

abstract: In the evolving landscape of deep learning, selecting the best pre-trained models from a growing number of choices is a challenge. Transferability scorers propose alleviating this scenario, but their recent proliferation, ironically, poses the challenge of their own assessment. In this work, we propose both robust benchmark guidelines for transferability scorers, and a well-founded technique to combine multiple scorers, which we show consistently improves their results. We extensively evaluate 13 scorers from literature across 11 datasets, comprising generalist, fine-grained, and medical imaging datasets. We show that few scorers match the predictive performance of the simple raw metric of models on ImageNet, and that all predictors suffer on medical datasets. Our results highlight the potential of combining different information sources for reliably predicting transferability across varied domains.


title: Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQA

abstract: Large Multimodal Models (LMMs) have shown remarkable progress in medical Visual Question Answering (Med-VQA), achieving high accuracy on existing benchmarks. However, their reliability under robust evaluation is questionable. This study reveals that when subjected to simple probing evaluation, state-of-the-art models perform worse than random guessing on medical diagnosis questions. To address this critical evaluation problem, we introduce the Probing Evaluation for Medical Diagnosis (ProbMed) dataset to rigorously assess LMM performance in medical imaging through probing evaluation and procedural diagnosis. Particularly, probing evaluation features pairing original questions with negation questions with hallucinated attributes, while procedural diagnosis requires reasoning across various diagnostic dimensions for each image, including modality recognition, organ identification, clinical findings, abnormalities, and positional grounding. Our evaluation reveals that top-performing models like GPT-4o, GPT-4V, and Gemini Pro perform worse than random guessing on specialized diagnostic questions, indicating significant limitations in handling fine-grained medical inquiries. Besides, models like LLaVA-Med struggle even with more general questions, and results from CheXagent demonstrate the transferability of expertise across different modalities of the same organ, showing that specialized domain knowledge is still crucial for improving performance. This study underscores the urgent need for more robust evaluation to ensure the reliability of LMMs in critical fields like medical diagnosis, and current LMMs are still far from applicable to those fields.


title: Quantum-corrected anti-de Sitter space-time

abstract: We study the back-reaction of a quantum scalar field on anti-de Sitter (AdS) space-time. The renormalized expectation value of the stress-energy tensor operator for a massless, conformally-coupled quantum scalar field on global AdS space-time in four space-time dimensions acts as a source term on the right-hand-side of the Einstein equations for the quantum-corrected metric. We solve the quantum-corrected Einstein equations numerically and find deviations from pure AdS which increase as the temperature of the quantum scalar field state increases. We interpret these quantum-corrected metrics as asymptotically-AdS solitons, and study the mass of these solitons as a function of the temperature of the quantum scalar field.


title: Dynamics and Contracts for an Agent with Misspecified Beliefs

abstract: We study a single-agent contracting environment where the agent has misspecified beliefs about the outcome distributions for each chosen action. First, we show that for a myopic Bayesian learning agent with only two possible actions, the empirical frequency of the chosen actions converges to a Berk-Nash equilibrium. However, through a constructed example, we illustrate that this convergence in action frequencies fails when the agent has three or more actions. Furthermore, with multiple actions, even computing an $\varepsilon$-Berk-Nash equilibrium requires at least quasi-polynomial time under the Exponential Time Hypothesis (ETH) for the PPAD-class. This finding poses a significant challenge to the existence of simple learning dynamics that converge in action frequencies. Motivated by this challenge, we focus on the contract design problems for an agent with misspecified beliefs and two possible actions. We show that the revenue-optimal contract, under a Berk-Nash equilibrium, can be computed in polynomial time. Perhaps surprisingly, we show that even a minor degree of misspecification can result in a significant reduction in optimal revenue.


title: Euclidean Maximum Matchings in the Plane---Local to Global

abstract: Let $M$ be a perfect matching on a set of points in the plane where every edge is a line segment between two points. We say that $M$ is globally maximum if it is a maximum-length matching on all points. We say that $M$ is $k$-local maximum if for any subset $M'=\{a_1b_1,\dots,a_kb_k\}$ of $k$ edges of $M$ it holds that $M'$ is a maximum-length matching on points $\{a_1,b_1,\dots,a_k,b_k\}$. We show that local maximum matchings are good approximations of global ones. Let $\mu_k$ be the infimum ratio of the length of any $k$-local maximum matching to the length of any global maximum matching, over all finite point sets in the Euclidean plane. It is known that $\mu_k\geqslant \frac{k-1}{k}$ for any $k\geqslant 2$. We show the following improved bounds for $k\in\{2,3\}$: $\sqrt{3/7}\leqslant\mu_2< 0.93 $ and $\sqrt{3}/2\leqslant\mu_3< 0.98$. We also show that every pairwise crossing matching is unique and it is globally maximum. Towards our proof of the lower bound for $\mu_2$ we show the following result which is of independent interest: If we increase the radii of pairwise intersecting disks by factor $2/\sqrt{3}$, then the resulting disks have a common intersection.


title: Condensation in scale-free geometric graphs with excess edges

abstract: We identify the upper large deviation probability for the number of edges in scale-free geometric random graph models as the space volume goes to infinity. Our result covers the models of scale-free percolation, the Boolean model with heavy-tailed radius distribution, and the age-dependent random connection model. In all these cases the mechanism behind the large deviation is based on a condensation effect. Loosely speaking, the mechanism randomly selects a finite number of vertices and increases their power, so that they connect to a macroscopic number of vertices in the graph, while the other vertices retain a degree close to their expectation and thus make no more than the expected contribution to the large deviation event. We verify this intuition by means of limit theorems for the empirical distributions of degrees and edge-lengths under the conditioning. We observe that at large finite volumes, the edge-length distribution splits into a bulk and travelling wave part of asymptotically positive proportions.


title: Quality of Non-Convergent Best Response Processes in Multi-Agent Systems through Sink Equilibrium

abstract: Examining the behavior of multi-agent systems is vitally important to many emerging distributed applications - game theory has emerged as a powerful tool set in which to do so. The main approach of game-theoretic techniques is to model agents as players in a game, and predict the emergent behavior through the relevant Nash equilibrium. The virtue from this viewpoint is that by assuming that self-interested decision-making processes lead to Nash equilibrium, system behavior can then be captured by Nash equilibrium without studying the decision-making processes explicitly. This approach has seen success in a wide variety of domains, such as sensor coverage, traffic networks, auctions, and network coordination. However, in many other problem settings, Nash equilibrium are not necessarily guaranteed to exist or emerge from self-interested processes. Thus the main focus of the paper is on the study of sink equilibrium, which are defined as the attractors of these decision-making processes. By classifying system outcomes through a global objective function, we can analyze the resulting approximation guarantees that sink equilibrium have for a given game. Our main result is an approximation guarantee on the sink equilibrium through defining an introduced metric of misalignment, which captures how uniform agents are in their self-interested decision making. Overall, sink equilibrium are naturally occurring in many multi-agent contexts, and we display our results on their quality with respect to two practical problem settings.


title: Measurement of Energy Resolution with the NEXT-White Silicon Photomultipliers

abstract: The NEXT-White detector, a high-pressure gaseous xenon time projection chamber, demonstrated the excellence of this technology for future neutrinoless double beta decay searches using photomultiplier tubes (PMTs) to measure energy and silicon photomultipliers (SiPMs) to extract topology information. This analysis uses $^{83m}\text{Kr}$ data from the NEXT-White detector to measure and understand the energy resolution that can be obtained with the SiPMs, rather than with PMTs. The energy resolution obtained of (10.9 $\pm$ 0.6) $\%$, full-width half-maximum, is slightly larger than predicted based on the photon statistics resulting from very low light detection coverage of the SiPM plane in the NEXT-White detector. The difference in the predicted and measured resolution is attributed to poor corrections, which are expected to be improved with larger statistics. Furthermore, the noise of the SiPMs is shown to not be a dominant factor in the energy resolution and may be negligible when noise subtraction is applied appropriately, for high-energy events or larger SiPM coverage detectors. These results, which are extrapolated to estimate the response of large coverage SiPM planes, are promising for the development of future, SiPM-only, readout planes that can offer imaging and achieve similar energy resolution to that previously demonstrated with PMTs.


title: Relatively hyperbolic groups with planar boundaries

abstract: In this article, we prove a version of Martin and Skora's conjecture that convergence groups on the $2$-sphere are covered by Kleinian groups. Given a relatively hyperbolic group pair $(G,\mathcal{P})$ with planar boundary and no Sierpinski carpet or cut points in its boundary, and with $G$ one ended and virtually having no $2$-torsion, we show that $G$ is virtually Kleinian. We also give applications to various versions of the Cannon conjecture and to convergence groups acting on $S^2$.


title: Quantum Preference Query

abstract: Given a large dataset of many tuples, it is hard for users to pick out their preferred tuples. Thus, the preference query problem, which is to find the most preferred tuples from a dataset, is widely discussed in the database area. In this problem, a utility function is given by the user to evaluate to what extent the user prefers a tuple. However, considering a dataset consisting of N tuples, the existing algorithms need O(N) time to answer a query, or need O(N) time for a cold start to answer a query. The reason is that in a classical computer, a linear time is needed to evaluate the utilities by the utility function for N tuples. In this paper, we discuss the Quantum Preference Query (QPQ) problem, where the dataset is given in a quantum memory, and we use a quantum computer to return the answers. Due to quantum parallelism, the quantum algorithm can theoretically perform better than their classical competitors. We discuss this problem in different kinds of input and output. In the QPQ problem, the input can be a number k or a threshold theta. Given k, the problem is to return k tuples with the highest utilities. Given theta, the problem is to return all the tuples with utilities higher than theta. Also, in QPQ problem, the output can be classical (i.e., a list of tuples) or quantum (i.e., a superposition in quantum bits). We proposed four quantum algorithms to solve the problems in the above four scenarios. We analyze the number of memory accesses needed for each quantum algorithm, which shows that the proposed quantum algorithms are at least quadratically faster than their classical competitors. In our experiments, we show that to answer a QPQ problem, the quantum algorithms achieve up to 1000x improvement in number of memory accesses than their classical competitors, which proved that QPQ problem could be a future direction of the study of preference query problems.


title: Enhancing Performance for Highly Imbalanced Medical Data via Data Regularization in a Federated Learning Setting

abstract: The increased availability of medical data has significantly impacted healthcare by enabling the application of machine / deep learning approaches in various instances. However, medical datasets are usually small and scattered across multiple providers, suffer from high class-imbalance, and are subject to stringent data privacy constraints. In this paper, the application of a data regularization algorithm, suitable for learning under high class-imbalance, in a federated learning setting is proposed. Specifically, the goal of the proposed method is to enhance model performance for cardiovascular disease prediction by tackling the class-imbalance that typically characterizes datasets used for this purpose, as well as by leveraging patient data available in different nodes of a federated ecosystem without compromising their privacy and enabling more resource sensitive allocation. The method is evaluated across four datasets for cardiovascular disease prediction, which are scattered across different clients, achieving improved performance. Meanwhile, its robustness under various hyperparameter settings, as well as its ability to adapt to different resource allocation scenarios, is verified.


title: Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspective

abstract: Federated Learning (FL) is a promising paradigm that offers significant advancements in privacy-preserving, decentralized machine learning by enabling collaborative training of models across distributed devices without centralizing data. However, the practical deployment of FL systems faces a significant bottleneck: the communication overhead caused by frequently exchanging large model updates between numerous devices and a central server. This communication inefficiency can hinder training speed, model performance, and the overall feasibility of real-world FL applications. In this survey, we investigate various strategies and advancements made in communication-efficient FL, highlighting their impact and potential to overcome the communication challenges inherent in FL systems. Specifically, we define measures for communication efficiency, analyze sources of communication inefficiency in FL systems, and provide a taxonomy and comprehensive review of state-of-the-art communication-efficient FL methods. Additionally, we discuss promising future research directions for enhancing the communication efficiency of FL systems. By addressing the communication bottleneck, FL can be effectively applied and enable scalable and practical deployment across diverse applications that require privacy-preserving, decentralized machine learning, such as IoT, healthcare, or finance.


title: Supernova Neutrinos: flavour conversion mechanisms and new physics scenarios

abstract: A core-collapse supernova (SN) releases almost all of its energy in the form of neutrinos, which provide a unique opportunity to probe the working machinery of a SN. These sites are prone to neutrino-neutrino refractive effects, which can lead to fascinating collective flavour oscillations among neutrinos. This causes rapid neutrino flavour conversions deep inside the SN even for suppressed mixing angles, with intriguing consequences for the explosion mechanism as well as nucleosynthesis. We review the physics of collective oscillations of neutrinos -- both slow and fast, along with the well-known resonant flavour conversion effects, and discuss the current state-of-the-art of the field. Furthermore, we discuss how neutrinos from a SN can be used to probe novel particle physics properties, extreme values of which are otherwise inaccessible in laboratories.


title: Efficient Industrial Refrigeration Scheduling with Peak Pricing

abstract: The widespread use of industrial refrigeration systems across various sectors contribute significantly to global energy consumption, highlighting substantial opportunities for energy conservation through intelligent control design. As such, this work focuses on control algorithm design in industrial refrigeration that minimize operational costs and provide efficient heat extraction. By adopting tools from inventory control, we characterize the structure of these optimal control policies, exploring the impact of different energy cost-rate structures such as time-of-use (TOU) pricing and peak pricing. While classical threshold policies are optimal under TOU costs, introducing peak pricing challenges their optimality, emphasizing the need for carefully designed control strategies in the presence of significant peak costs. We provide theoretical findings and simulation studies on this phenomenon, offering insights for more efficient industrial refrigeration management.


title: Facilitating Human-LLM Collaboration through Factuality Scores and Source Attributions

abstract: While humans increasingly rely on large language models (LLMs), they are susceptible to generating inaccurate or false information, also known as "hallucinations". Technical advancements have been made in algorithms that detect hallucinated content by assessing the factuality of the model's responses and attributing sections of those responses to specific source documents. However, there is limited research on how to effectively communicate this information to users in ways that will help them appropriately calibrate their trust toward LLMs. To address this issue, we conducted a scenario-based study (N=104) to systematically compare the impact of various design strategies for communicating factuality and source attribution on participants' ratings of trust, preferences, and ease in validating response accuracy. Our findings reveal that participants preferred a design in which phrases within a response were color-coded based on the computed factuality scores. Additionally, participants increased their trust ratings when relevant sections of the source material were highlighted or responses were annotated with reference numbers corresponding to those sources, compared to when they received no annotation in the source material. Our study offers practical design guidelines to facilitate human-LLM collaboration and it promotes a new human role to carefully evaluate and take responsibility for their use of LLM outputs.


title: Deep Learning for Computing Convergence Rates of Markov Chains

abstract: Convergence rate analysis for general state-space Markov chains is fundamentally important in areas such as Markov chain Monte Carlo and algorithmic analysis (for computing explicit convergence bounds). This problem, however, is notoriously difficult because traditional analytical methods often do not generate practically useful convergence bounds for realistic Markov chains. We propose the Deep Contractive Drift Calculator (DCDC), the first general-purpose sample-based algorithm for bounding the convergence of Markov chains to stationarity in Wasserstein distance. The DCDC has two components. First, inspired by the new convergence analysis framework in (Qu et.al, 2023), we introduce the Contractive Drift Equation (CDE), the solution of which leads to an explicit convergence bound. Second, we develop an efficient neural-network-based CDE solver. Equipped with these two components, DCDC solves the CDE and converts the solution into a convergence bound. We analyze the sample complexity of the algorithm and further demonstrate the effectiveness of the DCDC by generating convergence bounds for realistic Markov chains arising from stochastic processing networks as well as constant step-size stochastic optimization.


title: Three-flavor Collective Neutrino Oscillations on D-Wave's {\tt Advantage} Quantum Annealer

abstract: In extreme environments such as core-collapse supernovae, neutron-star mergers, and the early Universe, neutrinos are dense enough that their self-interactions significantly affect, if not dominate, their flavor dynamics. In order to develop techniques for characterizing the resulting quantum entanglement, I present the results of simulations of Dirac neutrino-neutrino interactions that include all three physical neutrino flavors and were performed on D-Wave Inc.'s {\tt Advantage} 5000+ qubit quantum annealer. These results are checked against those from exact classical simulations, which are also used to compare the Dirac neutrino-neutrino interactions to neutrino-antineutrino and Majorana neutrino-neutrino interactions. The D-Wave {\tt Advantage} annealer is shown to be able to reproduce time evolution with the precision of a classical machine for small number of neutrinos and to do so without Trotter errors. However, it suffers from poor scaling in qubit-count with the number of neutrinos. Two approaches to improving the qubit-scaling are discussed, but only one of the two shows promise.


title: The four-components link invariant in the framework of topological field theories

abstract: In this work, we undertake a perturbative analysis of the topological non-Abelian Chern-Simons-Wong model with the aim to explicitly construct the second-order on-shell action. The resulting action is a topological quantity depending solely on closed curves, so it correspond to an analytical expression of a link invariant. Additionally, we construct an Abelian model that reproduces the same second-order on-shell action as its non-Abelian Chern-Simons-Wong counterpart so it functions as an intermediate model, featuring Abelian fields generated by currents supported on closed paths. By geometrically analyzing each term, we demonstrate that this topological invariant effectively detects the knotting of a four-component link.


title: The Boltzmann-Poisson equation with a central body: analytical solutions in one and two dimensions

abstract: We consider an isothermal self-gravitating system surrounding a central body. This model can represent a galaxy or a globular cluster harboring a central black hole. It can also represent a gaseous atmosphere surrounding a protoplanet. In three dimensions, the Boltzmann-Poisson equation must be solved numerically in order to obtain the density profile of the gas [Chavanis {\it et al.}, Phys. Rev. E {\bf 109}, 014118 (2024)]. In one and two dimensions, we show that the Boltzmann-Poisson equation can be solved analytically. We obtain explicit analytical expressions of the density profile around a central body which generalize the analytical solutions found by Camm (1950) and Ostriker (1964) in the absence of a central body. Our results also have applications for self-gravitating Brownian particles (Smoluchowski-Poisson system), for the chemotaxis of bacterial populations (Keller-Segel model), and for two-dimensional point vortices (Onsager's model). In the case of bacterial populations, the central body could represent a supply of ``food'' that attracts the bacteria (chemoattractant). In the case of two-dimensional vortices, the central body could be a central vortex


title: Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learning

abstract: Sharpness-Aware Minimization (SAM) has emerged as a promising alternative optimizer to stochastic gradient descent (SGD). The originally-proposed motivation behind SAM was to bias neural networks towards flatter minima that are believed to generalize better. However, recent studies have shown conflicting evidence on the relationship between flatness and generalization, suggesting that flatness does fully explain SAM's success. Sidestepping this debate, we identify an orthogonal effect of SAM that is beneficial out-of-distribution: we argue that SAM implicitly balances the quality of diverse features. SAM achieves this effect by adaptively suppressing well-learned features which gives remaining features opportunity to be learned. We show that this mechanism is beneficial in datasets that contain redundant or spurious features where SGD falls for the simplicity bias and would not otherwise learn all available features. Our insights are supported by experiments on real data: we demonstrate that SAM improves the quality of features in datasets containing redundant or spurious features, including CelebA, Waterbirds, CIFAR-MNIST, and DomainBed.


title: Characterization of a Longwave HgCdTe GeoSnap Detector

abstract: New longwave HgCdTe detectors are critical to upcoming plans for ground-based infrared astronomy. These detectors, with fast-readouts and deep well-depths, will be key components of extremely large telescope instruments and therefore must be well understood prior to deployment. We analyze one such HgCdTe detector, a Teledyne Imaging Sensors GeoSnap, at the University of Michigan. We find that the properties of the GeoSnap are consistent with expectations from analysis of past devices. The GeoSnap has a well-depth of 2.75 million electrons per pixel, a read noise of 360 e-/pix, and a dark current of 330,000 e-/s/pix at 45 K. The device experiences 1/f noise which can be mitigated relative to half-well shot noise with modest frequency image differencing. The GeoSnap's quantum efficiency is calculated to be 79.7 +- 8.3 % at 10.6 microns. Although the GeoSnap's bad pixel fraction, on the order of 3%, is consistent with other GeoSnap devices, close to a third of the bad pixels in this detector are clustered in a series of 31 "leopard" spots spread across the detector plane. We report these properties and identify additional analyses that will be performed on future GeoSnap detectors.


title: SECURE: Benchmarking Generative Large Language Models for Cybersecurity Advisory

abstract: Large Language Models (LLMs) have demonstrated potential in cybersecurity applications but have also caused lower confidence due to problems like hallucinations and a lack of truthfulness. Existing benchmarks provide general evaluations but do not sufficiently address the practical and applied aspects of LLM performance in cybersecurity-specific tasks. To address this gap, we introduce the SECURE (Security Extraction, Understanding \& Reasoning Evaluation), a benchmark designed to assess LLMs performance in realistic cybersecurity scenarios. SECURE includes six datasets focussed on the Industrial Control System sector to evaluate knowledge extraction, understanding, and reasoning based on industry-standard sources. Our study evaluates seven state-of-the-art models on these tasks, providing insights into their strengths and weaknesses in cybersecurity contexts, and offer recommendations for improving LLMs reliability as cyber advisory tools.


title: Spontaneous scalarization in Einstein--power--Maxwell--scalar models

abstract: We study the spontaneous scalarization of charged black holes in Einstein's gravity minimally coupled to power--Maxwell electrodynamics which, in turn, is non--minimally coupled to a real scalar field. We point out the existence of a specific power for which the scalarized solution is well-behaved, and entropically preferred in comparison to the scalar-free charged black hole solution.


title: P-MSDiff: Parallel Multi-Scale Diffusion for Remote Sensing Image Segmentation

abstract: Diffusion models and multi-scale features are essential components in semantic segmentation tasks that deal with remote-sensing images. They contribute to improved segmentation boundaries and offer significant contextual information. U-net-like architectures are frequently employed in diffusion models for segmentation tasks. These architectural designs include dense skip connections that may pose challenges for interpreting intermediate features. Consequently, they might not efficiently convey semantic information throughout various layers of the encoder-decoder architecture. To address these challenges, we propose a new model for semantic segmentation known as the diffusion model with parallel multi-scale branches. This model consists of Parallel Multiscale Diffusion modules (P-MSDiff) and a Cross-Bridge Linear Attention mechanism (CBLA). P-MSDiff enhances the understanding of semantic information across multiple levels of granularity and detects repetitive distribution data through the integration of recursive denoising branches. It further facilitates the amalgamation of data by connecting relevant branches to the primary framework to enable concurrent denoising. Furthermore, within the interconnected transformer architecture, the LA module has been substituted with the CBLA module. This module integrates a semidefinite matrix linked to the query into the dot product computation of keys and values. This integration enables the adaptation of queries within the LA framework. This adjustment enhances the structure for multi-head attention computation, leading to enhanced network performance and CBLA is a plug-and-play module. Our model demonstrates superior performance based on the J1 metric on both the UAVid and Vaihingen Building datasets, showing improvements of 1.60% and 1.40% over strong baseline models, respectively.


title: New Limit on Dark Photon Kinetic Mixing in the 0.2-1.2 $\boldsymbol{\mu}$eV Mass Range From the Dark E-Field Radio Experiment

abstract: We report new limits on the kinetic mixing strength of the dark photon spanning the mass range 0.21 -- 1.24 $\mu$eV corresponding to a frequency span of 50 -- 300 MHz. The Dark E-Field Radio experiment is a wide-band search for dark photon dark matter. In this paper we detail changes in calibration and upgrades since our proof-of-concept pilot run. Our detector employs a wide bandwidth E-field antenna moved to multiple positions in a shielded room, a low noise amplifier, wideband ADC, followed by a $2^{24}$-point FFT. An optimal filter searches for signals with Q $\approx10^6$. In nine days of integration, this system is capable of detecting dark photon signals corresponding to $\epsilon$ several orders of magnitude lower than previous limits. We find a 95% exclusion limit on $\epsilon$ over this mass range between $6\times 10^{-15}$ and $6\times 10^{-13}$, tracking the complex resonant mode structure in the shielded room.


title: GraphAny: A Foundation Model for Node Classification on Any Graph

abstract: Foundation models that can perform inference on any new task without requiring specific training have revolutionized machine learning in vision and language applications. However, applications involving graph-structured data remain a tough nut for foundation models, due to challenges in the unique feature- and label spaces associated with each graph. Traditional graph ML models such as graph neural networks (GNNs) trained on graphs cannot perform inference on a new graph with feature and label spaces different from the training ones. Furthermore, existing models learn functions specific to the training graph and cannot generalize to new graphs. In this work, we tackle these two challenges with a new foundational architecture for inductive node classification named GraphAny. GraphAny models inference on a new graph as an analytical solution to a LinearGNN, thereby solving the first challenge. To solve the second challenge, we learn attention scores for each node to fuse the predictions of multiple LinearGNNs. Specifically, the attention module is carefully parameterized as a function of the entropy-normalized distance-features between multiple LinearGNNs predictions to ensure generalization to new graphs. Empirically, GraphAny trained on the Wisconsin dataset with only 120 labeled nodes can effectively generalize to 30 new graphs with an average accuracy of 67.26\% in an inductive manner, surpassing GCN and GAT trained in the supervised regime, as well as other inductive baselines.


title: Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generation

abstract: Retrieval Augmented Generation (RAG) systems have shown great promise in natural language processing. However, their reliance on data stored in a retrieval database, which may contain proprietary or sensitive information, introduces new privacy concerns. Specifically, an attacker may be able to infer whether a certain text passage appears in the retrieval database by observing the outputs of the RAG system, an attack known as a Membership Inference Attack (MIA). Despite the significance of this threat, MIAs against RAG systems have yet remained under-explored. This study addresses this gap by introducing an efficient and easy-to-use method for conducting MIA against RAG systems. We demonstrate the effectiveness of our attack using two benchmark datasets and multiple generative models, showing that the membership of a document in the retrieval database can be efficiently determined through the creation of an appropriate prompt in both black-box and gray-box settings. Moreover, we introduce an initial defense strategy based on adding instructions to the RAG template, which shows high effectiveness for some datasets and models. Our findings highlight the importance of implementing security countermeasures in deployed RAG systems and developing more advanced defenses to protect the privacy and security of retrieval databases.


title: Algorithmic Fairness in Performative Policy Learning: Escaping the Impossibility of Group Fairness

abstract: In many prediction problems, the predictive model affects the distribution of the prediction target. This phenomenon is known as performativity and is often caused by the behavior of individuals with vested interests in the outcome of the predictive model. Although performativity is generally problematic because it manifests as distribution shifts, we develop algorithmic fairness practices that leverage performativity to achieve stronger group fairness guarantees in social classification problems (compared to what is achievable in non-performative settings). In particular, we leverage the policymaker's ability to steer the population to remedy inequities in the long term. A crucial benefit of this approach is that it is possible to resolve the incompatibilities between conflicting group fairness definitions.


title: Knockout: A simple way to handle missing inputs

abstract: Deep learning models can extract predictive and actionable information from complex inputs. The richer the inputs, the better these models usually perform. However, models that leverage rich inputs (e.g., multi-modality) can be difficult to deploy widely, because some inputs may be missing at inference. Current popular solutions to this problem include marginalization, imputation, and training multiple models. Marginalization can obtain calibrated predictions but it is computationally costly and therefore only feasible for low dimensional inputs. Imputation may result in inaccurate predictions because it employs point estimates for missing variables and does not work well for high dimensional inputs (e.g., images). Training multiple models whereby each model takes different subsets of inputs can work well but requires knowing missing input patterns in advance. Furthermore, training and retaining multiple models can be costly. We propose an efficient way to learn both the conditional distribution using full inputs and the marginal distributions. Our method, Knockout, randomly replaces input features with appropriate placeholder values during training. We provide a theoretical justification of Knockout and show that it can be viewed as an implicit marginalization strategy. We evaluate Knockout in a wide range of simulations and real-world datasets and show that it can offer strong empirical performance.


title: Optimization, guidance, and control of low-thrust transfers from the Lunar Gateway to low lunar orbit

abstract: The Gateway will represent a primary space system useful for the Artemis program, Earth-Moon transportation, and deep space exploration. It is expected to serve as a staging location on the way to the lunar surface. This study focuses on low-thrust transfer dynamics, from the Near-Rectilinear Halo Orbit traveled by Gateway to a specified Low-altitude Lunar Orbit (LLO). This research addresses: (i) determination of the minimum-time low-thrust trajectory and (ii) design, implementation, and testing of a guidance and control architecture, for a space vehicle that travels from Gateway to LLO. Orbit dynamics is described in terms of modified equinoctial elements, in the context of a high-fidelity ephemeris model. The minimum-time trajectory from Gateway to a specified lunar orbit is detected through an indirect heuristic approach, which uses the analytical conditions arising in optimal control theory in conjunction with a heuristic technique. However, future missions will pursue a growing level of autonomy, and this circumstance implies the mandatory design of an efficient feedback guidance scheme, capable of compensating for nonnominal flight conditions. This research proposes nonlinear orbit control as a viable option for autonomous explicit guidance of low-thrust transfers from Gateway to LLO. This approach allows defining a feedback law that enjoys quasi-global stability properties without requiring any offline reference trajectory. The overall spacecraft dynamics is modeled including attitude control and actuation. The latter is demanded to an array of reaction wheels, arranged in a pyramidal configuration. Guidance, attitude control, and actuation are implemented in an iterative scheme. Monte Carlo simulations demonstrate that the guidance and control architecture is effective with random starting points from Gateway and the temporary unavailability of the propulsion system.

categories: eess.SY cs.SY math.DS math.OC physics.space-ph


title: Decentralized AI: Permissionless LLM Inference on POKT Network

abstract: POKT Network's decentralized Remote Procedure Call (RPC) infrastructure, surpassing 740 billion requests since launching on MainNet in 2020, is well-positioned to extend into providing AI inference services with minimal design or implementation modifications. This litepaper illustrates how the network's open-source and permissionless design aligns incentives among model researchers, hardware operators, API providers and users whom we term model Sources, Suppliers, Gateways and Applications respectively. Through its Relay Mining algorithm, POKT creates a transparent marketplace where costs and earnings directly reflect cryptographically verified usage. This decentralized framework offers large model AI researchers a new avenue to disseminate their work and generate revenue without the complexities of maintaining infrastructure or building end-user products. Supply scales naturally with demand, as evidenced in recent years and the protocol's free market dynamics. POKT Gateways facilitate network growth, evolution, adoption, and quality by acting as application-facing load balancers, providing value-added features without managing LLM nodes directly. This vertically decoupled network, battle tested over several years, is set up to accelerate the adoption, operation, innovation and financialization of open-source models. It is the first mature permissionless network whose quality of service competes with centralized entities set up to provide application grade inference.


title: Statistical Properties of Robust Satisficing

abstract: The Robust Satisficing (RS) model is an emerging approach to robust optimization, offering streamlined procedures and robust generalization across various applications. However, the statistical theory of RS remains unexplored in the literature. This paper fills in the gap by comprehensively analyzing the theoretical properties of the RS model. Notably, the RS structure offers a more straightforward path to deriving statistical guarantees compared to the seminal Distributionally Robust Optimization (DRO), resulting in a richer set of results. In particular, we establish two-sided confidence intervals for the optimal loss without the need to solve a minimax optimization problem explicitly. We further provide finite-sample generalization error bounds for the RS optimizer. Importantly, our results extend to scenarios involving distribution shifts, where discrepancies exist between the sampling and target distributions. Our numerical experiments show that the RS model consistently outperforms the baseline empirical risk minimization in small-sample regimes and under distribution shifts. Furthermore, compared to the DRO model, the RS model exhibits lower sensitivity to hyperparameter tuning, highlighting its practicability for robustness considerations.


title: Understanding Encoder-Decoder Structures in Machine Learning Using Information Measures

abstract: We present new results to model and understand the role of encoder-decoder design in machine learning (ML) from an information-theoretic angle. We use two main information concepts, information sufficiency (IS) and mutual information loss (MIL), to represent predictive structures in machine learning. Our first main result provides a functional expression that characterizes the class of probabilistic models consistent with an IS encoder-decoder latent predictive structure. This result formally justifies the encoder-decoder forward stages many modern ML architectures adopt to learn latent (compressed) representations for classification. To illustrate IS as a realistic and relevant model assumption, we revisit some known ML concepts and present some interesting new examples: invariant, robust, sparse, and digital models. Furthermore, our IS characterization allows us to tackle the fundamental question of how much performance (predictive expressiveness) could be lost, using the cross entropy risk, when a given encoder-decoder architecture is adopted in a learning setting. Here, our second main result shows that a mutual information loss quantifies the lack of expressiveness attributed to the choice of a (biased) encoder-decoder ML design. Finally, we address the problem of universal cross-entropy learning with an encoder-decoder design where necessary and sufficiency conditions are established to meet this requirement. In all these results, Shannon's information measures offer new interpretations and explanations for representation learning.


title: Exterior boundary-value Poincare problem for elliptic systems of the second order with two independent variables

abstract: This paper offers a number of examples showing that in the case of two independent variables the uniform ellipticity of a linear system of differential equations with partial derivatives of the second order, which fulfills condition (3), do not always cause the normal solvability of formulated exterior elliptic problems in the sense of Noether. Nevertheless, from the system of differential equations with partial derivatives of elliptic type it is possible to choose, under certain additional conditions, classes which are normally solvable in the sense of Noether. This paper also shows that for the so-called decomposed system of differential equations, with partial derivatives of an elliptic type in the case of exterior regions, the Noether theorems are valid.


title: Geometric Characterization of Rat Urinary Bladder Wall During Ex-Vivo Filling Using Micro-Computed Tomography (Micro-CT)

abstract: This study employs micro-computed tomography (micro-CT) to unravel the geometrical intricacies of the rat urinary bladder wall during various states of ex-vivo filling, contrasting markedly with the commonly held idealizations of uniform bladder geometry. Through precise 3D reconstructions at resolutions between 10-20 micrometers, the research meticulously documents the bladder's morphological transformations across different filling pressures. The findings illuminate substantial deviations from the theoretical model of a uniformly thick, spherical bladder, particularly underlined by variations in wall thickness and bladder volume during the transition from voided to filled states. These results are pivotal for refining mechanical models of bladder function, which have traditionally oversimplified the complex geometrical and biomechanical behavior of the bladder. Moreover, this study underscores the potential of micro-CT in providing a deeper understanding of bladder mechanics, essential for advancing therapeutic strategies for conditions like bladder outlet obstruction (BOO), thereby enhancing both surgical and pharmacological treatment paradigms.


title: DepsRAG: Towards Managing Software Dependencies using Large Language Models

abstract: Managing software dependencies is a crucial maintenance task in software development and is becoming a rapidly growing research field, especially in light of the significant increase in software supply chain attacks. Specialized expertise and substantial developer effort are required to fully comprehend dependencies and reveal hidden properties about the dependencies (e.g., number of dependencies, dependency chains, depth of dependencies). Recent advancements in Large Language Models (LLMs) allow the retrieval of information from various data sources for response generation, thus providing a new opportunity to uniquely manage software dependencies. To highlight the potential of this technology, we present~\tool, a proof-of-concept Retrieval Augmented Generation (RAG) approach that constructs direct and transitive dependencies of software packages as a Knowledge Graph (KG) in four popular software ecosystems. DepsRAG can answer user questions about software dependencies by automatically generating necessary queries to retrieve information from the KG, and then augmenting the input of LLMs with the retrieved information. DepsRAG can also perform Web search to answer questions that the LLM cannot directly answer via the KG. We identify tangible benefits that DepsRAG can offer and discuss its limitations.


title: Scaling Laws for the Value of Individual Data Points in Machine Learning

abstract: Recent works have shown that machine learning models improve at a predictable rate with the total amount of training data, leading to scaling laws that describe the relationship between error and dataset size. These scaling laws can help design a model's training dataset, but they typically take an aggregate view of the data by only considering the dataset's size. We introduce a new perspective by investigating scaling behavior for the value of individual data points: we find that a data point's contribution to model's performance shrinks predictably with the size of the dataset in a log-linear manner. Interestingly, there is significant variability in the scaling exponent among different data points, indicating that certain points are more valuable in small datasets while others are relatively more useful as a part of large datasets. We provide learning theory to support our scaling law, and we observe empirically that it holds across diverse model classes. We further propose a maximum likelihood estimator and an amortized estimator to efficiently learn the individualized scaling behaviors from a small number of noisy observations per data point. Using our estimators, we provide insights into factors that influence the scaling behavior of different data points. Finally, we demonstrate applications of the individualized scaling laws to data valuation and data subset selection. Overall, our work represents a first step towards understanding and utilizing scaling properties for the value of individual data points.


title: Online network topology shapes personal narratives and hashtag generation

abstract: While narratives have shaped cognition and cultures for centuries, digital media and online social networks have introduced new narrative phenomena. With increased narrative agency, networked groups of individuals can directly contribute and steer narratives that center our collective discussions of politics, science, and morality. We report the results of an online network experiment on narrative and hashtag generation, in which networked groups of participants interpreted a text-based narrative of a disaster event, and were incentivized to produce matching hashtags with their network neighbors. We found that network structure not only influences the emergence of dominant beliefs through coordination with network neighbors, but also impacts participants' use of causal language in their personal narratives.


title: Contingency-Aware Station-Keeping Control of Halo Orbits

abstract: We present an algorithm to perform fuel-optimal stationkeeping for spacecraft in unstable halo orbits with additional constraints to ensure safety in the event of a control failure. We formulate a convex trajectory-optimization problem to generate impulsive spacecraft maneuvers to loosely track a halo orbit using a receding-horizon controller. Our solution also provides a safe exit strategy in the event that propulsion is lost at any point in the mission. We validate our algorithm in simulations of the three-body Earth-Moon and Saturn-Enceladus systems, demonstrating both low total delta-v and a safe contingency plan throughout the mission.


title: On Calibration of Object Detectors: Pitfalls, Evaluation and Baselines

abstract: Reliable usage of object detectors require them to be calibrated -- a crucial problem that requires careful attention. Recent approaches towards this involve (1) designing new loss functions to obtain calibrated detectors by training them from scratch, and (2) post-hoc Temperature Scaling (TS) that learns to scale the likelihood of a trained detector to output calibrated predictions. These approaches are then evaluated based on a combination of Detection Expected Calibration Error (D-ECE) and Average Precision. In this work, via extensive analysis and insights, we highlight that these recent evaluation frameworks, evaluation metrics, and the use of TS have notable drawbacks leading to incorrect conclusions. As a step towards fixing these issues, we propose a principled evaluation framework to jointly measure calibration and accuracy of object detectors. We also tailor efficient and easy-to-use post-hoc calibration approaches such as Platt Scaling and Isotonic Regression specifically for object detection task. Contrary to the common notion, our experiments show that once designed and evaluated properly, post-hoc calibrators, which are extremely cheap to build and use, are much more powerful and effective than the recent train-time calibration methods. To illustrate, D-DETR with our post-hoc Isotonic Regression calibrator outperforms the recent train-time state-of-the-art calibration method Cal-DETR by more than 7 D-ECE on the COCO dataset. Additionally, we propose improved versions of the recently proposed Localization-aware ECE and show the efficacy of our method on these metrics as well. Code is available at: https://github.com/fiveai/detection_calibration.


title: Moduli of rank two semistable sheaves on rational Fano threefolds of the main series

abstract: In this paper we investigate the moduli spaces of semistable coherent sheaves of rank two on the projective space $\mathbb{P}^3$ and the following rational Fano manifolds of the main series - the three-dimensional quadric $X_2$, the intersection of two 4-dimensional quadrics $X_4$ and the Fano manifold $X_5$ of degree 5. For the quadric $X_2$, the boundedness of the third Chern class $c_3$ of rank two semistable objects in $\mathrm{D}^b(X_2)$, including sheaves, is proved. An explicit description is given of all the moduli spaces of semistable sheaves of rank two on $X_2$, including reflexive ones, with a maximal third class $c_3\ge0$. These spaces turn out to be irreducible smooth rational manifolds in all cases, except for the following two: $(c_1,c_2,c_3)=(0,2,2)$ or (0,4,8). The first example of a disconnected module space of semistable rank two sheaves with fixed Chern classes on a smooth projective variety is found -- this is the second of these exceptional cases $(c_1,c_2,c_3)= (0,4,8)$ on the quadric $X_2$. Several new infinite series of rational components of the moduli spaces of semistable sheaves of rank two on $\mathbb{P}^3$, $X_2$, $X_4$ and $X_5$ are constructed, as well as a new infinite series of irrational components on $X_4$. The boundedness of the class $c_3$ is proved for $c_1=0$ and any $c_2>0$ for stable reflexive sheaves of general type on manifolds $X_4$ and $X_5$.


title: Scalable Detection of Salient Entities in News Articles

abstract: News articles typically mention numerous entities, a large fraction of which are tangential to the story. Detecting the salience of entities in articles is thus important to applications such as news search, analysis and summarization. In this work, we explore new approaches for efficient and effective salient entity detection by fine-tuning pretrained transformer models with classification heads that use entity tags or contextualized entity representations directly. Experiments show that these straightforward techniques dramatically outperform prior work across datasets with varying sizes and salience definitions. We also study knowledge distillation techniques to effectively reduce the computational cost of these models without affecting their accuracy. Finally, we conduct extensive analyses and ablation experiments to characterize the behavior of the proposed models.


title: Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining

abstract: Self-supervised pretraining on large-scale satellite data has raised great interest in building Earth observation (EO) foundation models. However, many important resources beyond pure satellite imagery, such as land-cover-land-use products that provide free global semantic information, as well as vision foundation models that hold strong knowledge of the natural world, tend to be overlooked. In this work, we show these free additional resources not only help resolve common contrastive learning bottlenecks, but also significantly boost the efficiency and effectiveness of EO pretraining. Specifically, we first propose soft contrastive learning that optimizes cross-scene soft similarity based on land-cover-generated multi-label supervision, naturally solving the issue of multiple positive samples and too strict positive matching in complex scenes. Second, we explore cross-domain continual pretraining for both multispectral and SAR imagery, building efficient EO foundation models from strongest vision models such as DINOv2. Integrating simple weight-initialization and Siamese masking strategies into our soft contrastive learning framework, we demonstrate impressive continual pretraining performance even when the input channels and modalities are not aligned. Without prohibitive training, we produce multispectral and SAR foundation models that achieve significantly better results in 9 out of 10 downstream tasks than most existing SOTA models. For example, our ResNet50/ViT-S achieve 84.8/85.0 linear probing mAP scores on BigEarthNet-10\% which are better than most existing ViT-L models; under the same setting, our ViT-B sets a new record of 86.8 in multispectral, and 82.5 in SAR, the latter even better than many multispectral models. Dataset and models are available at https://github.com/zhu-xlab/softcon.


title: Hyperspatiality for isomorphisms of stabilized automorphism groups of shifts of finite type

abstract: Given a homeomorphism $T \colon X \to X$ of a compact metric space $X$, the stabilized automorphism group $\textrm{Aut}^{\infty}(T)$ of the system $(X,T)$ is the group of self-homeomorphisms of $X$ which commute with some power of $T$. We study the question of spatiality for stabilized automorphism groups of shifts of finite type. We prove that any isomorphism $\Psi \colon \textrm{Aut}^{\infty}(\sigma_{m}) \to \textrm{Aut}^{\infty}(\sigma_{n})$ between stabilized automorphism groups of full shifts is spatially induced by a homeomorphism $\hat{\Psi}$ between respective stabilized spaces of chain recurrent subshifts. This spatialization in particular gives a bijection between the sets of periodic points which intertwines some powers of the shifts, and this bijection recovers the isomorphism at the level of the faithful actions on the sets of periodic points. We also prove that the outer automorphism group of $\textrm{Aut}^{\infty}(\sigma_{n})$ is uncountable, and deduce several other properties of $\textrm{Aut}^{\infty}(\sigma_{n})$ using the spatiality results.


title: Index theory on Pin manifolds

abstract: We give a systematic treatment of index theory on Pin manifolds, based on the Clifford linear Dirac operator and differential KO-theory. This expository article is based on joint work with Mike Hopkins.


title: ENTIRe-ID: An Extensive and Diverse Dataset for Person Re-Identification

abstract: The growing importance of person reidentification in computer vision has highlighted the need for more extensive and diverse datasets. In response, we introduce the ENTIRe-ID dataset, an extensive collection comprising over 4.45 million images from 37 different cameras in varied environments. This dataset is uniquely designed to tackle the challenges of domain variability and model generalization, areas where existing datasets for person re-identification have fallen short. The ENTIRe-ID dataset stands out for its coverage of a wide array of real-world scenarios, encompassing various lighting conditions, angles of view, and diverse human activities. This design ensures a realistic and robust training platform for ReID models. The ENTIRe-ID dataset is publicly available at https://serdaryildiz.github.io/ENTIRe-ID


abstract: Multiscale differentials arise as limits of holomorphic differentials with prescribed orders of zeros on nodal curves. In this paper, we address the conjecture concerning Gorenstein contractions of multiscale differentials, initially proposed by Ranganathan and Wise and further elaborated upon by Battistella and Bozlee. Specifically, we show that multiscale differentials can be contracted into Gorenstein singularities level by level from the top down. At each level, these differentials can descend to become generators of the dualizing bundle at the singularities. Additionally, the global residue condition, which governs the smoothability of multiscale differentials, is a special case of the residue condition for descent differentials.


title: Performance of NPG in Countable State-Space Average-Cost RL

abstract: We consider policy optimization methods in reinforcement learning settings where the state space is arbitrarily large, or even countably infinite. The motivation arises from control problems in communication networks, matching markets, and other queueing systems. We consider Natural Policy Gradient (NPG), which is a popular algorithm for finite state spaces. Under reasonable assumptions, we derive a performance bound for NPG that is independent of the size of the state space, provided the error in policy evaluation is within a factor of the true value function. We obtain this result by establishing new policy-independent bounds on the solution to Poisson's equation, i.e., the relative value function, and by combining these bounds with previously known connections between MDPs and learning from experts.


title: MTEB-French: Resources for French Sentence Embedding Evaluation and Analysis

abstract: Recently, numerous embedding models have been made available and widely used for various NLP tasks. The Massive Text Embedding Benchmark (MTEB) has primarily simplified the process of choosing a model that performs well for several tasks in English, but extensions to other languages remain challenging. This is why we expand MTEB to propose the first massive benchmark of sentence embeddings for French. We gather 15 existing datasets in an easy-to-use interface and create three new French datasets for a global evaluation of 8 task categories. We compare 51 carefully selected embedding models on a large scale, conduct comprehensive statistical tests, and analyze the correlation between model performance and many of their characteristics. We find out that even if no model is the best on all tasks, large multilingual models pre-trained on sentence similarity perform exceptionally well. Our work comes with open-source code, new datasets and a public leaderboard.


title: Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Images

abstract: A long-standing challenge in developing machine learning approaches has been the lack of high-quality labeled data. Recently, models trained with purely synthetic data, here termed synthetic clones, generated using large-scale pre-trained diffusion models have shown promising results in overcoming this annotation bottleneck. As these synthetic clone models progress, they are likely to be deployed in challenging real-world settings, yet their suitability remains understudied. Our work addresses this gap by providing the first benchmark for three classes of synthetic clone models, namely supervised, self-supervised, and multi-modal ones, across a range of robustness measures. We show that existing synthetic self-supervised and multi-modal clones are comparable to or outperform state-of-the-art real-image baselines for a range of robustness metrics - shape bias, background bias, calibration, etc. However, we also find that synthetic clones are much more susceptible to adversarial and real-world noise than models trained with real data. To address this, we find that combining both real and synthetic data further increases the robustness, and that the choice of prompt used for generating synthetic images plays an important part in the robustness of synthetic clones.


title: STHN: Deep Homography Estimation for UAV Thermal Geo-localization with Satellite Imagery

abstract: Accurate geo-localization of Unmanned Aerial Vehicles (UAVs) is crucial for a variety of outdoor applications including search and rescue operations, power line inspections, and environmental monitoring. The vulnerability of Global Navigation Satellite Systems (GNSS) signals to interference and spoofing necessitates the development of additional robust localization methods for autonomous navigation. Visual Geo-localization (VG), leveraging onboard cameras and reference satellite maps, offers a promising solution for absolute localization. Specifically, Thermal Geo-localization (TG), which relies on image-based matching between thermal imagery with satellite databases, stands out by utilizing infrared cameras for effective night-time localization. However, the efficiency and effectiveness of current TG approaches, are hindered by dense sampling on satellite maps and geometric noises in thermal query images. To overcome these challenges, in this paper, we introduce STHN, a novel UAV thermal geo-localization approach that employs a coarse-to-fine deep homography estimation method. This method attains reliable thermal geo-localization within a 512-meter radius of the UAV's last known location even with a challenging 11% overlap between satellite and thermal images, despite the presence of indistinct textures in thermal imagery and self-similar patterns in both spectra. Our research significantly enhances UAV thermal geo-localization performance and robustness against the impacts of geometric noises under low-visibility conditions in the wild. The code will be made publicly available.


title: Equivalent External Noise Temperature of Time-Varying Receivers

abstract: The equivalent external noise temperature of time-varying antennas is studied using the concept of cross-frequency effective aperture, which quantifies the intermodulation conversion of external noise across the frequency spectrum into a receiver's operational bandwidth. The theoretical tools for this approach are laid out following the classical method for describing external noise temperature of linear time-invariant antennas, with generalizations made along the way to capture the effects of time-varying components or materials. The results demonstrate the specific ways that a time-varying system's noise characteristics are dependent on its cross-frequency effective aperture and the broadband noise environment. The general theory is applied to several examples, including abstract models of hypothetical systems, antennas integrated with parametric amplification, and time-modulated arrays.


title: Kicking Co and Rh atoms on a row-wise antiferromagnet

abstract: Diffusion on surfaces is a fundamental process in surface science, governing nanostructure and film growth, molecular self-assembly, and chemical reactions. Atom motion on non-magnetic surfaces has been studied extensively both theoretically and by real-space imaging techniques. For magnetic surfaces density functional theory (DFT) calculations have predicted strong effects of the magnetic state onto adatom diffusion, but to date no corresponding experimental data exists. Here, we investigate Co and Rh atoms on a hexagonal magnetic layer, using scanning tunneling microscopy (STM) and DFT calculations. Experimentally, we "kick" atoms by local voltage pulses and thereby initiate strictly one-dimensional motion which is dictated by the row-wise antiferromagnetic (AFM) state. Our calculations show that the one-dimensional motion of Co and Rh atoms results from conserving the Co spin direction during movement and avoiding high induced Rh spin moments, respectively. These findings demonstrate that magnetism can be a means to control adatom mobility.


title: Solving multi-pole challenges in the GW100 benchmark enables precise low-scaling GW calculations

abstract: The $GW$ approximation is a widely used method for computing electron addition and removal energies of molecules and solids. The computational effort of conventional $GW$ algorithms increases as $O(N^4)$ with the system size $N$, hindering the application of $GW$ to large and complex systems. Low-scaling $GW$ algorithms are currently very actively developed. Benchmark studies at the single-shot $G_0W_0$ level indicate excellent numerical precision for frontier quasiparticle energies, with mean absolute deviations $<10$ meV with respect to standard implementations for the widely used $GW100$ test set. A notable challenge for low-scaling $GW$ algorithms remains in achieving high precision for five molecules within the $GW100$ test set, namely O$_3$, BeO, MgO, BN, and CuCN, for which the deviations are in the range of several hundred meV. This is due to a spurious transfer of spectral weight from the quasiparticle to the satellite spectrum in $G_0W_0$ calculations, resulting in multi-pole features in the self-energy and spectral function, which low-scaling algorithms fail to describe. We show in this work that including eigenvalue self-consistency in the Green's function ($\text{ev}GW_0$) achieves a proper separation between satellite and quasiparticle peak, leading to a single solution of the quasiparticle equation with spectral weight close to one. $\text{ev}GW_0$ quasiparticles energies from low-scaling $GW$ closely align with reference calculations; the mean absolute error is only 12 meV for the five molecules.


title: Principle of structural analogy of solutions and its application to nonlinear PDEs and delay PDEs

abstract: Using the principle of structural analogy of solutions, approaches have been developed for constructing exact solutions of complex nonlinear PDEs, including PDEs with delay, based on the use of special solutions to auxiliary simpler related equations. It is shown that to obtain exact solutions of nonlinear non-autonomous PDEs, the coefficients of which depend on time, it is possible to use generalized and functional separable solutions of simpler autonomous PDEs, the coefficients of which do not depend on time. Specific examples of constructing exact solutions to nonlinear PDEs, the coefficients of which depend arbitrarily on time, are considered. It has been discovered that generalized and functional separable solutions of nonlinear PDEs with constant delay can be used to construct exact solutions of more complex nonlinear PDEs with variable delay of general form. A number of nonlinear reaction-diffusion type PDEs with variable delay are described, which allow exact solutions with generalized separation of variables.


title: What makes a cosmic filament? The dynamical origin and identity of filaments I. fundamentals in 2D

abstract: Cosmic filaments are the main transport channels of matter in the Megaparsec universe, and represent the most prominent structural feature in the matter and galaxy distribution. Here we describe and define the physical and dynamical nature of cosmic filaments. It is based on the realization that the complex spatial pattern and connectivity of the cosmic web are already visible in the primordial random Gaussian density field, in the spatial pattern of the primordial tidal and deformation eigenvalue field. The filaments and other structural features in the cosmic web emerging from this are multistream features and structural singularities in phase-space. The caustic skeleton formalism allows a fully analytical classification, identification, and treatment of the nonlinear cosmic web. The caustic conditions yield the mathematical specification of weblike structures in terms of the primordial deformation tensor eigenvalue and eigenvector fields, in which filaments are identified -- in 2D -- with the so-called cusp caustics. These are centered around points that are maximally stretched as a result of the tidal force field. The resulting mathematical conditions represent a complete characterization of filaments in terms of their formation history, dynamics, and orientation. We illustrate the workings of the formalism on the basis of a set of constrained $N$-body simulations of protofilament realizations. These realizations are analyzed in terms of spatial structure, density profiles, and multistream structure and compared to simpler density or potential field saddle point specifications. The presented formalism, and its 3D generalization, will facilitate the mining of the rich cosmological information contained in the observed weblike galaxy distribution, and be of key significance for the analysis of cosmological surveys such as SDSS, DESI, and Euclid.


title: Coherent coupling of circular Rydberg qubits to inner shell excitations

abstract: Divalent atoms provide excellent means for advancing control in Rydberg atom-based quantum simulation and computing, due to the second optically active valence electron available. Particularly promising in this context are circular Rydberg atoms, for which long-lived ionic core excitations can be exploited without suffering from detrimental autoionization. Here, we report measurements of the electric quadrupole coupling between the metastable 4D$_{3/2}$ level and a very high-$n$ ($n=79$) circular Rydberg qubit, realized in doubly excited $^{88}$Sr atoms. We measure the kHz-scale differential level shift on the circular Rydberg qubit via beat-node Ramsey interferometry comprising spin echo. Observing this coupling requires coherent interrogation of the Rydberg states for more than one hundred microseconds, which is assisted by tweezer trapping and circular state lifetime enhancement in a black-body radiation suppressing capacitor. Our results demonstrate access to weak electron-electron interactions in Rydberg atoms and expand the quantum simulation toolbox for optical control of highly excited circular state qubits via ionic core manipulation.


title: Automated Focused Feedback Generation for Scientific Writing Assistance

abstract: Scientific writing is a challenging task, particularly for novice researchers who often rely on feedback from experienced peers. Recent work has primarily focused on improving surface form and style rather than manuscript content. In this paper, we propose a novel task: automated focused feedback generation for scientific writing assistance. We present SWIF$^{2}$T: a Scientific WrIting Focused Feedback Tool. It is designed to generate specific, actionable and coherent comments, which identify weaknesses in a scientific paper and/or propose revisions to it. Our approach consists of four components - planner, investigator, reviewer and controller - leveraging multiple Large Language Models (LLMs) to implement them. We compile a dataset of 300 peer reviews citing weaknesses in scientific papers and conduct human evaluation. The results demonstrate the superiority in specificity, reading comprehension, and overall helpfulness of SWIF$^{2}$T's feedback compared to other approaches. In our analysis, we also identified cases where automatically generated reviews were judged better than human ones, suggesting opportunities for integration of AI-generated feedback in scientific writing.


title: On the viability of the QCD axion - dark energy transition

abstract: This conference thesis summarizes my presentation at Tartu Tuorla Cosmology meeting based on a collaborative work with E.Nardi and C.Smarra \cite{Muursepp:2024mbb}. It is well known that a pseudo-Nambu Goldstone boson (pNGB) coupled to a confining gauge group obtains a non-zero contribution to its mass through instanton effects. At non-zero temperature, this manifests in temperature-dependent mass $m^{2}(T) \propto T^{-n}$. If these particles come to dominate the energy density of the Universe in the non-relativistic regime they would accelerate the expansion of the Universe for $n>2$, thus providing a dark energy (DE) component. In this work, we outline a scenario in which a pNGB $\phi_{b}$ presently undergoing confinement could realize such a scenario. Using energetic considerations we find that $\phi_{b}$ alone is not enough to produce the experimentally observed amount of DE. However, coupling $\phi_{b}$ to the QCD axion $\phi_a$ allows the transfer of energy from the QCD axion to the pNGB via non-adiabatic level-crossing thus reproducing the observed amount of dark matter (DM) and DE.


title: Black Hole Mass and Eddington Ratio Distribution of Hot Dust-Obscured Galaxies

abstract: Hot Dust-Obscured Galaxies (Hot DOGs) are a rare population of hyper-luminous infrared galaxies discovered by the WISE mission. Despite the significant obscuration of the AGN by dust in these systems, pronounced broad and blue-shifted emission lines are often observed. Previous work has shown that 8 Hot DOGs, referred to as Blue-excess Hot DOGs (BHDs), present a blue excess consistent with type 1 quasar emission in their UV-optical SEDs, which has been shown to originate from the light of the obscured central engine scattered into the line of sight. We present an analysis of the rest-frame optical emission characteristics for 172 Hot DOGs through UV-MIR SED modeling and spectroscopic details, with a particular focus on the identification of BHDs. We find that while the optical emission observed in Hot DOGs is in most cases dominated by a young stellar population, 26% of Hot DOGs show a significant enough blue excess emission to be classified as BHDs. Based on their broad CIV and MgII lines, we find that the $M_{\rm BH}$ in BHDs range from $10^{8.7}$ to $10^{10} \ M_{\odot}$. When using the same emission lines in regular Hot DOGs, we find the $M_{\rm BH}$ estimates cover the entire range found for BHDs while also extending to somewhat lower values. This agreement may imply that the broad lines in regular Hot DOGs also originate from scattered light from the central engine, just as in BHDs, although a more detailed study would be needed to rule out an outflow-driven nature. Similar to $z\sim 6$ quasars, we find that Hot DOGs sit above the local relation between stellar and black hole mass, suggesting either that AGN feedback has not yet significantly suppressed the stellar mass growth in the host galaxies, or that they will be outliers of the relation when reaching $z$=0.


title: Normality, factoriality and strong $F$-regularity of Lov\'asz-Saks-Schrijver rings

abstract: Every simple finite graph $G$ has an associated Lov\'asz-Saks-Schrijver ring $R_G(d)$ that is related to the $d$-dimensional orthogonal representations of $G$. The study of $R_G(d)$ lies at the intersection between algebraic geometry, commutative algebra and combinatorics. We find a link between algebraic properties such as normality, factoriality and strong $F$-regularity of $R_G(d)$ and combinatorial invariants of the graph $G$. In particular we prove that if $d \geq \text{pmd}(G)+k(G)$ then $R_G(d)$ is $F$-regular in finite characteristic and rational singularity in characteristic $0$ and furthermore if $d \geq \text{pmd}(G)+k(G)+1$ then $R_G(d)$ is UFD. Here $\text{pmd}(G)$ is the positive matching decomposition number of $G$ and $k(G)$ is its degeneracy number.


title: On the randomized Euler scheme for SDEs with integral-form drift

abstract: In this paper, we investigate the problem of strong approximation of the solution of SDEs in the case when the drift coefficient is given in the integral form. Such drift often appears when analyzing stochastic dynamics of optimization procedures in machine learning problems. We discuss connections of the defined randomized Euler approximation scheme with the perturbed version of the stochastic gradient descent (SGD) algorithm. We investigate its upper error bounds, in terms of the discretization parameter n and the size M of the random sample drawn at each step of the algorithm, in different subclasses of coefficients of the underlying SDE. Finally, the results of numerical experiments performed by using GPU architecture are also reported.


title: Sparsity regularization via tree-structured environments for disentangled representations

abstract: Many causal systems such as biological processes in cells can only be observed indirectly via measurements, such as gene expression. Causal representation learning -- the task of correctly mapping low-level observations to latent causal variables -- could advance scientific understanding by enabling inference of latent variables such as pathway activation. In this paper, we develop methods for inferring latent variables from multiple related datasets (environments) and tasks. As a running example, we consider the task of predicting a phenotype from gene expression, where we often collect data from multiple cell types or organisms that are related in known ways. The key insight is that the mapping from latent variables driven by gene expression to the phenotype of interest changes sparsely across closely related environments. To model sparse changes, we introduce Tree-Based Regularization (TBR), an objective that minimizes both prediction error and regularizes closely related environments to learn similar predictors. We prove that under assumptions about the degree of sparse changes, TBR identifies the true latent variables up to some simple transformations. We evaluate the theory empirically with both simulations and ground-truth gene expression data. We find that TBR recovers the latent causal variables better than related methods across these settings, even under settings that violate some assumptions of the theory.


title: Hiding Your Awful Online Choices Made More Efficient and Secure: A New Privacy-Aware Recommender System

abstract: Recommender systems are an integral part of online platforms that recommend new content to users with similar interests. However, they demand a considerable amount of user activity data where, if the data is not adequately protected, constitute a critical threat to the user privacy. Privacy-aware recommender systems enable protection of such sensitive user data while still maintaining a similar recommendation accuracy compared to the traditional non-private recommender systems. However, at present, the current privacy-aware recommender systems suffer from a significant trade-off between privacy and computational efficiency. For instance, it is well known that architectures that rely purely on cryptographic primitives offer the most robust privacy guarantees, however, they suffer from substantial computational and network overhead. Thus, it is crucial to improve this trade-off for better performance. This paper presents a novel privacy-aware recommender system that combines privacy-aware machine learning algorithms for practical scalability and efficiency with cryptographic primitives like Homomorphic Encryption and Multi-Party Computation - without assumptions like trusted-party or secure hardware - for solid privacy guarantees. Experiments on standard benchmark datasets show that our approach results in time and memory gains by three orders of magnitude compared to using cryptographic primitives in a standalone for constructing a privacy-aware recommender system. Furthermore, for the first time our method makes it feasible to compute private recommendations for datasets containing 100 million entries, even on memory-constrained low-power SOC (System on Chip) devices.