title: Measuring and Mitigating Bias for Tabular Datasets with Multiple Protected Attributes

abstract: Motivated by the recital (67) of the current corrigendum of the AI Act in the European Union, we propose and present measures and mitigation strategies for discrimination in tabular datasets. We specifically focus on datasets that contain multiple protected attributes, such as nationality, age, and sex. This makes measuring and mitigating bias more challenging, as many existing methods are designed for a single protected attribute. This paper comes with a twofold contribution: Firstly, new discrimination measures are introduced. These measures are categorized in our framework along with existing ones, guiding researchers and practitioners in choosing the right measure to assess the fairness of the underlying dataset. Secondly, a novel application of an existing bias mitigation method, FairDo, is presented. We show that this strategy can mitigate any type of discrimination, including intersectional discrimination, by transforming the dataset. By conducting experiments on real-world datasets (Adult, Bank, Compas), we demonstrate that de-biasing datasets with multiple protected attributes is achievable. Further, the transformed fair datasets do not compromise any of the tested machine learning models' performances significantly when trained on these datasets compared to the original datasets. Discrimination was reduced by up to 83% in our experimentation. For most experiments, the disparity between protected groups was reduced by at least 7% and 27% on average. Generally, the findings show that the mitigation strategy used is effective, and this study contributes to the ongoing discussion on the implementation of the European Union's AI Act.


title: Safe and Efficient Estimation for Robotics through the Optimal Use of Resources

abstract: In order to operate in and interact with the physical world, robots need to have estimates of the current and future state of the environment. We thus equip robots with sensors and build models and algorithms that, given some measurements, produce estimates of the current or future states. Environments can be unpredictable and sensors are not perfect. Therefore, it is important to both use all information available, and to do so optimally: making sure that we get the best possible answer from the amount of information we have. However, in prevalent research, uncommon sensors, such as sound or radio-frequency signals, are commonly ignored for state estimation; and the most popular solvers employed to produce state estimates are only of local nature, meaning they may produce suboptimal estimates for the typically non-convex estimation problems. My research aims to use resources more optimally, by building on 1) multi-modality: using ubiquitous RF transceivers and microphones to support state estimation, 2) building certifiably optimal solvers and 3) learning and improving adequate models from data.


title: Multi-qubit circuit synthesis and Hermitian lattices

abstract: We present new optimal and heuristic algorithms for exact synthesis of multi-qubit unitaries and isometries. For example, our algorithms find Clifford and T circuits for unitaries with entries in $\mathbb{Z}[i,1/\sqrt{2}]$. The optimal algorithms are the A* search instantiated with a new data structure for graph vertices and new consistent heuristic functions. We also prove that for some gate sets, best-first search synthesis relying on the same heuristic is efficient. For example, for two-qubit Clifford and T circuits, our best-first search runtime is proportional to the T-count of the unitary. Our algorithms rely on Hermite and Smith Normal Forms of matrices with entries in a ring of integers of a number field, and we leverage the theory of and algorithms for Hermitian lattices over number fields to prove efficiency. These new techniques are of independent interest for future work on multi-qubit exact circuit synthesis and related questions.


title: Morse Theory for Chromatic Delaunay Triangulations

abstract: The chromatic alpha filtration is a generalization of the alpha filtration that can encode spatial relationships among classes of labelled point cloud data, and has applications in topological data analysis of multi-species data. In this paper we introduce the chromatic Delaunay--\v{C}ech and chromatic Delaunay--Rips filtrations, which are computationally favourable alternatives to the chromatic alpha filtration. We use generalized discrete Morse theory to show that the \v{C}ech, chromatic Delaunay--\v{C}ech, and chromatic alpha filtrations are related by simplicial collapses. Our result generalizes a result of Bauer and Edelsbrunner from the non-chromatic to the chromatic setting. We also show that the chromatic Delaunay--Rips filtration is locally stable to perturbations of the underlying point cloud. Our results provide theoretical justification for the use of chromatic Delaunay--\v{C}ech and chromatic Delaunay--Rips filtrations in applications, and we demonstrate their computational advantage with numerical experiments.


title: Set Descriptive Complexity of Solvable Functions

abstract: In a recent article, we introduced and studied a precise class of dynamical systems called solvable systems. These systems present a dynamic ruled by discontinuous ordinary differential equations with solvable right-hand terms and unique evolution. They correspond to a class of systems for which a transfinite method exist to compute the solution. We also presented several examples including a nontrivial one whose solution yields, at an integer time, a real encoding of the halting set for Turing machines; therefore showcasing that the behavior of solvable systems might describe ordinal Turing computations. In the current article, we study in more depth solvable systems, using tools from descriptive set theory. By establishing a correspondence with the class of well-founded trees, we construct a coanalytic ranking over the set of solvable functions and discuss its relation with other existing rankings for differentiable functions, in particular with the Kechris-Woodin, Denjoy and Zalcwasser ranking. We prove that our ranking is unbounded below the first uncountable ordinal.


title: Real-Time Environment Condition Classification for Autonomous Vehicles

abstract: Current autonomous driving technologies are being rolled out in geo-fenced areas with well-defined operation conditions such as time of operation, area, weather conditions and road conditions. In this way, challenging conditions as adverse weather, slippery road or densely-populated city centers can be excluded. In order to lift the geo-fenced restriction and allow a more dynamic availability of autonomous driving functions, it is necessary for the vehicle to autonomously perform an environment condition assessment in real time to identify when the system cannot operate safely and either stop operation or require the resting passenger to take control. In particular, adverse-weather challenges are a fundamental limitation as sensor performance degenerates quickly, prohibiting the use of sensors such as cameras to locate and monitor road signs, pedestrians or other vehicles. To address this issue, we train a deep learning model to identify outdoor weather and dangerous road conditions, enabling a quick reaction to new situations and environments. We achieve this by introducing an improved taxonomy and label hierarchy for a state-of-the-art adverse-weather dataset, relabelling it with a novel semi-automated labeling pipeline. Using the novel proposed dataset and hierarchy, we train RECNet, a deep learning model for the classification of environment conditions from a single RGB frame. We outperform baseline models by relative 16% in F1- Score, while maintaining a real-time capable performance of 20 Hz.


title: Uniform-in-time estimates on the size of chaos for interacting Brownian particles

abstract: We consider a system of classical Brownian particles interacting via a smooth long-range potential in the mean-field regime, and we analyze the propagation of chaos in form of sharp, uniform-in-time estimates on many-particle correlation functions. Our results cover both the kinetic Langevin setting and the corresponding overdamped Brownian dynamics. The approach is mainly based on so-called Lions expansions, which we combine with new diagrammatic tools to capture many-particle cancellations, as well as with fine ergodic estimates on the linearized mean-field equation, and with discrete stochastic calculus with respect to initial data. In the process, we derive some new ergodic estimates for the linearized Vlasov-Fokker-Planck kinetic equation that are of independent interest. Our analysis also leads to uniform-in-time concentration estimates and to a uniform-in-time quantitative central limit theorem for the empirical measure associated with the particle dynamics.


title: Data Efficient Behavior Cloning for Fine Manipulation via Continuity-based Corrective Labels

abstract: We consider imitation learning with access only to expert demonstrations, whose real-world application is often limited by covariate shift due to compounding errors during execution. We investigate the effectiveness of the Continuity-based Corrective Labels for Imitation Learning (CCIL) framework in mitigating this issue for real-world fine manipulation tasks. CCIL generates corrective labels by learning a locally continuous dynamics model from demonstrations to guide the agent back toward expert states. Through extensive experiments on peg insertion and fine grasping, we provide the first empirical validation that CCIL can significantly improve imitation learning performance despite discontinuities present in contact-rich manipulation. We find that: (1) real-world manipulation exhibits sufficient local smoothness to apply CCIL, (2) generated corrective labels are most beneficial in low-data regimes, and (3) label filtering based on estimated dynamics model error enables performance gains. To effectively apply CCIL to robotic domains, we offer a practical instantiation of the framework and insights into design choices and hyperparameter selection. Our work demonstrates CCIL's practicality for alleviating compounding errors in imitation learning on physical robots.


title: Visualizing the microscopic origins of topology in twisted molybdenum ditelluride

abstract: In moir\'e materials with flat electronic bands and suitable quantum geometry, strong correlations can give rise to novel topological states of matter. The nontrivial band topology of twisted molybdenum ditelluride (tMoTe$_2$) -- responsible for its fractional quantum anomalous Hall (FQAH) states -- is predicted to arise from a layer-pseudospin skyrmion lattice. Tracing the layer polarization of wavefunctions within the moir\'e unit cell can thus offer crucial insights into the band topology. Here, we use scanning tunneling microscopy and spectroscopy (STM/S) to probe the layer-pseudospin skyrmion textures of tMoTe$_2$. We do this by simultaneously visualizing the moir\'e lattice structure and the spatial localization of its electronic states. We find that the wavefunctions associated with the topological flat bands exhibit a spatially-dependent layer polarization within the moir\'e unit cell. This is in excellent agreement with our theoretical modeling, thereby revealing a direct microscopic connection between the structural properties of tMoTe$_2$ and its band topology. Our work enables new pathways for engineering FQAH states with strain, as well as future STM studies of the intertwined correlated and topological states arising in gate-tunable devices.


title: SDPRLayers: Certifiable Backpropagation Through Polynomial Optimization Problems in Robotics

abstract: Differentiable optimization is a powerful new paradigm capable of reconciling model-based and learning-based approaches in robotics. However, the majority of robotics optimization problems are non-convex and current differentiable optimization techniques are therefore prone to convergence to local minima. When this occurs, the gradients provided by these existing solvers can be wildly inaccurate and will ultimately corrupt the training process. On the other hand, many non-convex robotics problems can be framed as polynomial optimization problems and, in turn, admit convex relaxations that can be used to recover a global solution via so-called certifiably correct methods. We present SDPRLayers, an approach that leverages these methods as well as state-of-the-art convex implicit differentiation techniques to provide certifiably correct gradients throughout the training process. We introduce this approach and showcase theoretical results that provide conditions under which correctness of the gradients is guaranteed. We first demonstrate our approach on two simple-but-demonstrative simulated examples, which expose the potential pitfalls of existing, state-of-the-art, differentiable optimization methods. We then apply our method in a real-world application: we train a deep neural network to detect image keypoints for robot localization in challenging lighting conditions. We provide our open-source, PyTorch implementation of SDPRLayers and our differentiable localization pipeline.


title: Network Connectivity--Information Freshness Tradeoff in Information Dissemination Over Networks

abstract: We consider a gossip network consisting of a source generating updates and $n$ nodes connected according to a given graph structure. The source keeps updates of a process, that might be generated or observed, and shares them with the gossiping network. The nodes in the network communicate with their neighbors and disseminate these version updates using a push-style gossip strategy. We use the version age metric to quantify the timeliness of information at the nodes. We first find an upper bound for the average version age for a set of nodes in a general network. Using this, we find the average version age scaling of a node in several network graph structures, such as two-dimensional grids, generalized rings and hyper-cubes. Prior to our work, it was known that when $n$ nodes are connected on a ring the version age scales as $O(n^{\frac{1}{2}})$, and when they are connected on a fully-connected graph the version age scales as $O(\log n)$. Ours is the first work to show an age scaling result for a connectivity structure other than the ring and the fully-connected network, which constitute the two extremes of network connectivity. Our work helps fill the gap between these two extremes by analyzing a large variety of graphs with intermediate connectivity, thus providing insight into the relationship between the connectivity structure of the network and the version age, and uncovering a network connectivity--information freshness tradeoff.


title: The Effect of Tornadic Supercell Thunderstorms on the Atmospheric Muon Flux

abstract: Tornadoes are severe weather phenomena characterized by a violently rotating column of air connecting the ground to a parent storm. Within the United States, hundreds of tornadoes occur every year. Despite this, the dynamics of tornado formation and propagation are not particularly well understood, in part due to the challenge of instrumentation: many existing instruments for measuring atmospheric properties are in-situ detectors, making deployment in or near an active or developing tornado difficult. Here, we combine local atmospheric and cosmic ray air shower simulation to explore the potential for remote measurement of the pressure field within tornado-producing supercell thunderstorms by examining directional variations of the atmospheric muon flux.


title: Causal Inference for Balanced Incomplete Block Designs

abstract: Researchers often turn to block randomization to increase the precision of their inference or due to practical considerations, such as in multi-site trials. However, if the number of treatments under consideration is large it might not be practical or even feasible to assign all treatments within each block. We develop novel inference results under the finite-population design-based framework for a natural alternative to the complete block design that does not require reducing the number of treatment arms, the balanced incomplete block design (BIBD). This includes deriving the properties of two estimators for BIBDs and proposing conservative variance estimators. To assist practitioners in understanding the trade-offs of using BIBDs over other designs, the precisions of resulting estimators are compared to standard estimators for the complete block design, the cluster-randomized design, and the completely randomized design. Simulations and a data illustration demonstrate the strengths and weaknesses of using BIBDs. This work highlights BIBDs as practical and currently underutilized designs.


title: Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice

abstract: The observed similarities in the behavior of humans and Large Language Models (LLMs) have prompted researchers to consider the potential of using LLMs as models of human cognition. However, several significant challenges must be addressed before LLMs can be legitimately regarded as cognitive models. For instance, LLMs are trained on far more data than humans typically encounter, and may have been directly trained on human data in specific cognitive tasks or aligned with human preferences. Consequently, the origins of these behavioral similarities are not well understood. In this paper, we propose a novel way to enhance the utility of LLMs as cognitive models. This approach involves (i) leveraging computationally equivalent tasks that both an LLM and a rational agent need to master for solving a cognitive problem and (ii) examining the specific task distributions required for an LLM to exhibit human-like behaviors. We apply this approach to decision-making -- specifically risky and intertemporal choice -- where the key computationally equivalent task is the arithmetic of expected value calculations. We show that an LLM pretrained on an ecologically valid arithmetic dataset, which we call Arithmetic-GPT, predicts human behavior better than many traditional cognitive models. Pretraining LLMs on ecologically valid arithmetic datasets is sufficient to produce a strong correspondence between these models and human decision-making. Our results also suggest that LLMs used as cognitive models should be carefully investigated via ablation studies of the pretraining data.


title: Higgs Physics at a $\sqrt{s}=3$ TeV Muon Collider with detailed detector simulation

abstract: The Muon Collider is one of the most promising future collider facilities with the potential to reach multi-TeV center-of-mass energy and high luminosity. Due to the significant Higgs boson production cross section in muon collisions at these high energies, the collider can be considered a Higgs factory. It holds the capability to significantly advance our understanding of the Higgs sector to an unprecedented level of precision. However, the presence of beam-induced background resulting from the decay of the beam muons poses unique challenges for detector development and event reconstruction. In this paper, the prospects for various measurements of the Higgs boson production cross sections at a $\sqrt{s}=3$ TeV collider are presented using a detailed detector simulation in a realistic environment. The study demonstrates the feasibility of achieving high precision measurements of the Higgs boson production cross sections with the current state-of-the-art detector design. In addition, the paper discusses the detector requirements necessary for obtaining such resolutions and for measuring the Higgs trilinear self-coupling.


title: Matryoshka Query Transformer for Large Vision-Language Models

abstract: Large Vision-Language Models (LVLMs) typically encode an image into a fixed number of visual tokens (e.g., 576) and process these tokens with a language model. Despite their strong performance, LVLMs face challenges in adapting to varying computational constraints. This raises the question: can we achieve flexibility in the number of visual tokens to suit different tasks and computational resources? We answer this with an emphatic yes. Inspired by Matryoshka Representation Learning, we introduce the Matryoshka Query Transformer (MQT), capable of encoding an image into m visual tokens during inference, where m can be any number up to a predefined maximum. This is achieved by employing a query transformer with M latent query tokens to compress the visual embeddings. During each training step, we randomly select m <= M latent query tokens and train the model using only these first m tokens, discarding the rest. Combining MQT with LLaVA, we train a single model once, and flexibly and drastically reduce the number of inference-time visual tokens while maintaining similar or better performance compared to training independent models for each number of tokens. Our model, MQT-LLAVA, matches LLaVA-1.5 performance across 11 benchmarks using a maximum of 256 tokens instead of LLaVA's fixed 576. Reducing to 16 tokens (8x less TFLOPs) only sacrifices the performance by 2.4 points on MMBench. On certain tasks such as ScienceQA and MMMU, we can even go down to only 2 visual tokens with performance drops of just 3% and 6% each. Our exploration of the trade-off between the accuracy and computational cost brought about by the number of visual tokens facilitates future research to achieve the best of both worlds.


title: Robust Preference Optimization through Reward Model Distillation

abstract: Language model (LM) post-training (or alignment) involves maximizing a reward function that is derived from preference annotations. Direct Preference Optimization (DPO) is a popular offline alignment method that trains a policy directly on preference data without the need to train a reward model or apply reinforcement learning. However, typical preference datasets have only a single, or at most a few, annotation per preference pair, which causes DPO to overconfidently assign rewards that trend towards infinite magnitude. This frequently leads to degenerate policies, sometimes causing even the probabilities of the preferred generations to go to zero. In this work, we analyze this phenomenon and propose distillation to get a better proxy for the true preference distribution over generation pairs: we train the LM to produce probabilities that match the distribution induced by a reward model trained on the preference data. Moreover, to account for uncertainty in the reward model we are distilling from, we optimize against a family of reward models that, as a whole, is likely to include at least one reasonable proxy for the preference distribution. Our results show that distilling from such a family of reward models leads to improved robustness to distribution shift in preference annotations, while preserving the simple supervised nature of DPO.


title: Adaptive Generalized Neyman Allocation: Local Asymptotic Minimax Optimal Best Arm Identification

abstract: This study investigates a local asymptotic minimax optimal strategy for fixed-budget best arm identification (BAI). We propose the Adaptive Generalized Neyman Allocation (AGNA) strategy and show that its worst-case upper bound of the probability of misidentifying the best arm aligns with the worst-case lower bound under the small-gap regime, where the gap between the expected outcomes of the best and suboptimal arms is small. Our strategy corresponds to a generalization of the Neyman allocation for two-armed bandits (Neyman, 1934; Kaufmann et al., 2016) and a refinement of existing strategies such as the ones proposed by Glynn & Juneja (2004) and Shin et al. (2018). Compared to Komiyama et al. (2022), which proposes a minimax rate-optimal strategy, our proposed strategy has a tighter upper bound that exactly matches the lower bound, including the constant terms, by restricting the class of distributions to the class of small-gap distributions. Our result contributes to the longstanding open issue about the existence of asymptotically optimal strategies in fixed-budget BAI, by presenting the local asymptotic minimax optimal strategy.


title: Cosmology Based on Finsler and Finsler-like Metric Structure of Gravitational Field

abstract: In this article, we review some aspects of gravitational field and cosmology based on Finsler and Finsler-like generalized metric structures. The geometrical framework of these spaces allows further investigation of locally-anisotropic phenomena related to the gravitational field and cosmological considerations, e.g the extracted geodesics, deflection of light, Finsler-Einstein gravitational field equations , the Friedmann equations and the Raychaudhuri equations include extra anisotropic terms that in the Riemannian framework of General Relativity (GR) are not interpreted. This approach gives us the opportunity to extend the research with more degrees of freedom on the tangent bundle of a spacetime manifold. In the above mentioned generalizations omitting the extra anisotropic terms we recover the framework of GR. In addition, we study the gravitational Magnus effect in a generalized metric framework. Based on this approach, we describe further properties of Finsler-Randers (FR) and Schwarzschild Finsler Randers (SFR) cosmological models which are useful for the description and evolution of the universe.


title: ACE: A general-purpose non-Markovian open quantum systems simulation toolkit based on process tensors

abstract: We describe a general-purpose computational toolkit for simulating open quantum systems, which provides numerically exact solutions for composites of zero-dimensional quantum systems that may be strongly coupled to multiple, quite general non-Markovian environments. It is based on process tensor matrix product operators (PT-MPOs), which efficiently encapsulate environment influences. The code features implementations of several PT-MPO algorithms, in particular, Automated Compression of Environments (ACE) for general environments comprised of independent modes as well as schemes for generalized spin boson models. The latter includes a divide-and-conquer scheme for periodic PT-MPOs, which enable million time step simulations for realistic models. PT-MPOs can be precalculated and reused for efficiently probing different time-dependent system Hamiltonians. They can also be stacked together and combined to provide numerically complete solutions of small networks of open quantum systems. The code is written in C++ and is fully controllable by configuration files, for which we have developed a versatile and compact human-readable format.


title: Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF

abstract: Reinforcement learning from human feedback (RLHF) has demonstrated great promise in aligning large language models (LLMs) with human preference. Depending on the availability of preference data, both online and offline RLHF are active areas of investigation. A key bottleneck is understanding how to incorporate uncertainty estimation in the reward function learned from the preference data for RLHF, regardless of how the preference data is collected. While the principles of optimism or pessimism under uncertainty are well-established in standard reinforcement learning (RL), a practically-implementable and theoretically-grounded form amenable to large language models is not yet available, as standard techniques for constructing confidence intervals become intractable under arbitrary policy parameterizations. In this paper, we introduce a unified approach to online and offline RLHF -- value-incentivized preference optimization (VPO) -- which regularizes the maximum-likelihood estimate of the reward function with the corresponding value function, modulated by a $\textit{sign}$ to indicate whether the optimism or pessimism is chosen. VPO also directly optimizes the policy with implicit reward modeling, and therefore shares a simpler RLHF pipeline similar to direct preference optimization. Theoretical guarantees of VPO are provided for both online and offline settings, matching the rates of their standard RL counterparts. Moreover, experiments on text summarization and dialog verify the practicality and effectiveness of VPO.


title: DGD: Dynamic 3D Gaussians Distillation

abstract: We tackle the task of learning dynamic 3D semantic radiance fields given a single monocular video as input. Our learned semantic radiance field captures per-point semantics as well as color and geometric properties for a dynamic 3D scene, enabling the generation of novel views and their corresponding semantics. This enables the segmentation and tracking of a diverse set of 3D semantic entities, specified using a simple and intuitive interface that includes a user click or a text prompt. To this end, we present DGD, a unified 3D representation for both the appearance and semantics of a dynamic 3D scene, building upon the recently proposed dynamic 3D Gaussians representation. Our representation is optimized over time with both color and semantic information. Key to our method is the joint optimization of the appearance and semantic attributes, which jointly affect the geometric properties of the scene. We evaluate our approach in its ability to enable dense semantic 3D object tracking and demonstrate high-quality results that are fast to render, for a diverse set of scenes. Our project webpage is available on https://isaaclabe.github.io/DGD-Website/


title: Optical superlattice for engineering Hubbard couplings in quantum simulation

abstract: Quantum simulations of Hubbard models with ultracold atoms rely on the exceptional control of coherent motion provided by optical lattices. Here we demonstrate enhanced tunability using an optical superlattice in a fermionic quantum gas microscope. With our phase-stable bichromatic design, we achieve a precise control of tunneling and tilt throughout the lattice, as evidenced by long-lived coherent double-well oscillations and next-nearest-neighbor quantum walks in a staggered configuration. We furthermore present correlated quantum walks of two particles initiated through a resonant pair-breaking mechanism. Finally, we engineer tunable spin couplings through local offsets and create a spin ladder with ferromagnetic and antiferromagnetic couplings along the rungs and legs, respectively. Our work underscores the high potential of optical superlattices for engineering, simulating, and detecting strongly correlated many-body quantum states.


title: Are Large Language Models Chameleons?

abstract: Do large language models (LLMs) have their own worldviews and personality tendencies? Simulations in which an LLM was asked to answer subjective questions were conducted more than 1 million times. Comparison of the responses from different LLMs with real data from the European Social Survey (ESS) suggests that the effect of prompts on bias and variability is fundamental, highlighting major cultural, age, and gender biases. Methods for measuring the difference between LLMs and survey data are discussed, such as calculating weighted means and a new proposed measure inspired by Jaccard similarity. We conclude that it is important to analyze the robustness and variability of prompts before using LLMs to model individual decisions or collective behavior, as their imitation abilities are approximate at best.


title: X-ray and Radio campaign of the Z-source GX 340+0: discovery of X-ray polarization and its implications

abstract: We present the discovery of X-ray polarization from the neutron star low-mass X-ray binary and Z-source, GX~340$+$0, using an Imaging X-ray Polarimetry Explorer (IXPE) observation in March 2024. Along with the IXPE observation, we conducted an extensive X-ray and radio monitoring campaign to ascertain the source properties during and around the IXPE observation. The source was within the horizontal branch throughout the multiwavelength campaign. We measured a significant X-ray polarization in 2--8 keV with polarization degree (PD) = $4.02 \pm 0.35$% and polarization angle (PA) = $37.6 \pm 2.5^\circ$. The energy-dependent polarization indicates that in the 2-2.5 keV energy range, the PA is much lower, $\sim9\pm8^\circ$, while other energy bands are consistent with the PA found over 2.5--8 keV. The simultaneous AstroSat-IXPE spectro-polarimetric observations provide some evidence for independent polarization from various spectral components, hinting at a disparity in the PA from the accretion disk and the Comptonized emission, while suggesting an unpolarized emission from the blackbody component. Radio observations in the 0.7--9 GHz frequency range reveal a non-detection of radio emission in 0.7-1.5 GHz and a significant detection in 5.5--9 GHz, suggesting the presence of a spectral break in 1.5-5.5 GHz. Using ATCA observation we place upper limits on the radio polarization at $<$6% on the linear polarization and $<$4% on the circular polarization at 3$\sigma$ level. We discuss the origin of the X-ray polarization and its implications on the geometry of the spectral components.


title: Nearest Neighbor Speculative Decoding for LLM Generation and Attribution

abstract: Large language models (LLMs) often hallucinate and lack the ability to provide attribution for their generations. Semi-parametric LMs, such as kNN-LM, approach these limitations by refining the output of an LM for a given prompt using its nearest neighbor matches in a non-parametric data store. However, these models often exhibit slow inference speeds and produce non-fluent texts. In this paper, we introduce Nearest Neighbor Speculative Decoding (NEST), a novel semi-parametric language modeling approach that is capable of incorporating real-world text spans of arbitrary length into the LM generations and providing attribution to their sources. NEST performs token-level retrieval at each inference step to compute a semi-parametric mixture distribution and identify promising span continuations in a corpus. It then uses an approximate speculative decoding procedure that accepts a prefix of the retrieved span or generates a new token. NEST significantly enhances the generation quality and attribution rate of the base LM across a variety of knowledge-intensive tasks, surpassing the conventional kNN-LM method and performing competitively with in-context retrieval augmentation. In addition, NEST substantially improves the generation speed, achieving a 1.8x speedup in inference time when applied to Llama-2-Chat 70B.


title: Reasoning3D -- Grounding and Reasoning in 3D: Fine-Grained Zero-Shot Open-Vocabulary 3D Reasoning Part Segmentation via Large Vision-Language Models

abstract: In this paper, we introduce a new task: Zero-Shot 3D Reasoning Segmentation for parts searching and localization for objects, which is a new paradigm to 3D segmentation that transcends limitations for previous category-specific 3D semantic segmentation, 3D instance segmentation, and open-vocabulary 3D segmentation. We design a simple baseline method, Reasoning3D, with the capability to understand and execute complex commands for (fine-grained) segmenting specific parts for 3D meshes with contextual awareness and reasoned answers for interactive segmentation. Specifically, Reasoning3D leverages an off-the-shelf pre-trained 2D segmentation network, powered by Large Language Models (LLMs), to interpret user input queries in a zero-shot manner. Previous research have shown that extensive pre-training endows foundation models with prior world knowledge, enabling them to comprehend complex commands, a capability we can harness to "segment anything" in 3D with limited 3D datasets (source efficient). Experimentation reveals that our approach is generalizable and can effectively localize and highlight parts of 3D objects (in 3D mesh) based on implicit textual queries, including these articulated 3d objects and real-world scanned data. Our method can also generate natural language explanations corresponding to these 3D models and the decomposition. Moreover, our training-free approach allows rapid deployment and serves as a viable universal baseline for future research of part-level 3d (semantic) object understanding in various fields including robotics, object manipulation, part assembly, autonomous driving applications, augment reality and virtual reality (AR/VR), and medical applications. The code, the model weight, the deployment guide, and the evaluation protocol are: http://tianrun-chen.github.io/Reason3D/


title: MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series

abstract: Large Language Models (LLMs) have made great strides in recent years to achieve unprecedented performance across different tasks. However, due to commercial interest, the most competitive models like GPT, Gemini, and Claude have been gated behind proprietary interfaces without disclosing the training details. Recently, many institutions have open-sourced several strong LLMs like LLaMA-3, comparable to existing closed-source LLMs. However, only the model's weights are provided with most details (e.g., intermediate checkpoints, pre-training corpus, and training code, etc.) being undisclosed. To improve the transparency of LLMs, the research community has formed to open-source truly open LLMs (e.g., Pythia, Amber, OLMo), where more details (e.g., pre-training corpus and training code) are being provided. These models have greatly advanced the scientific study of these large models including their strengths, weaknesses, biases and risks. However, we observe that the existing truly open LLMs on reasoning, knowledge, and coding tasks are still inferior to existing state-of-the-art LLMs with similar model sizes. To this end, we open-source MAP-Neo, a highly capable and transparent bilingual language model with 7B parameters trained from scratch on 4.5T high-quality tokens. Our MAP-Neo is the first fully open-sourced bilingual LLM with comparable performance compared to existing state-of-the-art LLMs. Moreover, we open-source all details to reproduce our MAP-Neo, where the cleaned pre-training corpus, data cleaning pipeline, checkpoints, and well-optimized training/evaluation framework are provided. Finally, we hope our MAP-Neo will enhance and strengthen the open research community and inspire more innovations and creativities to facilitate the further improvements of LLMs.


title: Normative Modules: A Generative Agent Architecture for Learning Norms that Supports Multi-Agent Cooperation

abstract: Generative agents, which implement behaviors using a large language model (LLM) to interpret and evaluate an environment, has demonstrated the capacity to solve complex tasks across many social and technological domains. However, when these agents interact with other agents and humans in presence of social structures such as existing norms, fostering cooperation between them is a fundamental challenge. In this paper, we develop the framework of a 'Normative Module': an architecture designed to enhance cooperation by enabling agents to recognize and adapt to the normative infrastructure of a given environment. We focus on the equilibrium selection aspect of the cooperation problem and inform our agent design based on the existence of classification institutions that implement correlated equilibrium to provide effective resolution of the equilibrium selection problem. Specifically, the normative module enables agents to learn through peer interactions which of multiple candidate institutions in the environment, does a group treat as authoritative. By enabling normative competence in this sense, agents gain ability to coordinate their sanctioning behaviour; coordinated sanctioning behaviour in turn shapes primary behaviour within a social environment, leading to higher average welfare. We design a new environment that supports institutions and evaluate the proposed framework based on two key criteria derived from agent interactions with peers and institutions: (i) the agent's ability to disregard non-authoritative institutions and (ii) the agent's ability to identify authoritative institutions among several options. We show that these capabilities allow the agent to achieve more stable cooperative outcomes compared to baseline agents without the normative module, paving the way for research in a new avenue of designing environments and agents that account for normative infrastructure.


title: Ion Correlation-Driven Hysteretic Adhesion and Repulsion between Opposing Polyelectrolyte Brushes

abstract: Polyelectrolyte (PE) brushes are widely used in biomaterials and nanotechnology to regulate surface properties and interactions. Here, we apply the electrostatic correlation augmented self-consistent field theory to investigate the interactions between opposing PE brushes in a mixture of 1:1 and 3:1 salt solutions. Our theory predicts hysteretic feature of the normal stress induced by strong ion correlations. In the presence of trivalent ions, the force profile is discontinuous: repulsive in the compression branch and adhesive in the separation branch. The molecular origin of the hysteretic force is the coexistence of two collapsed modes: two separated condensed layer on each surface in the compression and a single bundled condensed layer in the separation. With the systematic inclusion of ion correlations, our theory fully captures the hysteretic force, adhesive separation, ``jump-in'' and ``jump-out'' features, and the ``specific ion effect'', all in good agreement with the reported experimental results.

categories: cond-mat.soft cond-mat.mtrl-sci cond-mat.stat-mech physics.chem-ph physics.comp-ph


title: Barium stars as tracers of s-process nucleosynthesis in AGB stars III. Systematic deviations from the AGB models

abstract: Barium (Ba) stars help to verify asymptotic giant branch (AGB) star nucleosynthesis models since they experienced pollution from an AGB binary companion and thus their spectra carry the signatures of the slow neutron capture process (s process). For 180 Ba stars, we searched for AGB stellar models that match the observed abundance patterns. We employed three machine learning algorithms as classifiers: a Random Forest method, developed for this work, and the two classifiers used in our previous study. We studied the statistical behaviour of the s-process elements in the observational sample to investigate if the AGB models systematically under- or overpredict the abundances observed in the Ba stars and show the results in the form of violin plots of the residuals between spectroscopic abundances and model predictions. We find a significant trend in the residuals that implies an underproduction of the elements Nb, Mo, and Ru in the models relative to the observations. This may originate from a process (e.g. the intermediate neutron-capture process, i process) at the metallicity of the Ba stars not yet included in the AGB models. Correlations are found between the residuals of these elements, suggesting a common origin for the deviations. In addition, there is a weak metallicity dependence of their residuals. The s-process temperatures derived with the [Zr/Fe] - [Nb/Fe] thermometer have an unrealistic value for the majority of our stars. The most likely explanation is that at least a fraction of these elements are not produced in a steady-state s process, and instead may be due to processes not included in the AGB models. The mass distribution of the identified models confirms that our sample of Ba stars was polluted by low-mass AGB stars. Most of the matching AGB models require low accreted mass, but a few systems with high accreted mass are needed to explain the observations. (abridged)


title: NPGA: Neural Parametric Gaussian Avatars

abstract: The creation of high-fidelity, digital versions of human heads is an important stepping stone in the process of further integrating virtual components into our everyday lives. Constructing such avatars is a challenging research problem, due to a high demand for photo-realism and real-time rendering performance. In this work, we propose Neural Parametric Gaussian Avatars (NPGA), a data-driven approach to create high-fidelity, controllable avatars from multi-view video recordings. We build our method around 3D Gaussian Splatting for its highly efficient rendering and to inherit the topological flexibility of point clouds. In contrast to previous work, we condition our avatars' dynamics on the rich expression space of neural parametric head models (NPHM), instead of mesh-based 3DMMs. To this end, we distill the backward deformation field of our underlying NPHM into forward deformations which are compatible with rasterization-based rendering. All remaining fine-scale, expression-dependent details are learned from the multi-view videos. To increase the representational capacity of our avatars, we augment the canonical Gaussian point cloud using per-primitive latent features which govern its dynamic behavior. To regularize this increased dynamic expressivity, we propose Laplacian terms on the latent features and predicted dynamics. We evaluate our method on the public NeRSemble dataset, demonstrating that NPGA significantly outperforms the previous state-of-the-art avatars on the self-reenactment task by 2.6 PSNR. Furthermore, we demonstrate accurate animation capabilities from real-world monocular videos.


title: Self-Exploring Language Models: Active Preference Elicitation for Online Alignment

abstract: Preference optimization, particularly through Reinforcement Learning from Human Feedback (RLHF), has achieved significant success in aligning Large Language Models (LLMs) to adhere to human intentions. Unlike offline alignment with a fixed dataset, online feedback collection from humans or AI on model generations typically leads to more capable reward models and better-aligned LLMs through an iterative process. However, achieving a globally accurate reward model requires systematic exploration to generate diverse responses that span the vast space of natural language. Random sampling from standard reward-maximizing LLMs alone is insufficient to fulfill this requirement. To address this issue, we propose a bilevel objective optimistically biased towards potentially high-reward responses to actively explore out-of-distribution regions. By solving the inner-level problem with the reparameterized reward function, the resulting algorithm, named Self-Exploring Language Models (SELM), eliminates the need for a separate RM and iteratively updates the LLM with a straightforward objective. Compared to Direct Preference Optimization (DPO), the SELM objective reduces indiscriminate favor of unseen extrapolations and enhances exploration efficiency. Our experimental results demonstrate that when finetuned on Zephyr-7B-SFT and Llama-3-8B-Instruct models, SELM significantly boosts the performance on instruction-following benchmarks such as MT-Bench and AlpacaEval 2.0, as well as various standard academic benchmarks in different settings. Our code and models are available at https://github.com/shenao-zhang/SELM.


title: Multi-Modal Generative Embedding Model

abstract: Most multi-modal tasks can be formulated into problems of either generation or embedding. Existing models usually tackle these two types of problems by decoupling language modules into a text decoder for generation, and a text encoder for embedding. To explore the minimalism of multi-modal paradigms, we attempt to achieve only one model per modality in this work. We propose a Multi-Modal Generative Embedding Model (MM-GEM), whereby the generative and embedding objectives are encapsulated in one Large Language Model. We also propose a PoolAggregator to boost efficiency and enable the ability of fine-grained embedding and generation. A surprising finding is that these two objectives do not significantly conflict with each other. For example, MM-GEM instantiated from ViT-Large and TinyLlama shows competitive performance on benchmarks for multimodal embedding models such as cross-modal retrieval and zero-shot classification, while has good ability of image captioning. Additionally, MM-GEM can seamlessly execute region-level image caption generation and retrieval tasks. Besides, the advanced text model in MM-GEM brings over 5% improvement in Recall@1 for long text and image retrieval.


title: LLMs Meet Multimodal Generation and Editing: A Survey

abstract: With the recent advancement in large language models (LLMs), there is a growing interest in combining LLMs with multimodal learning. Previous surveys of multimodal large language models (MLLMs) mainly focus on multimodal understanding. This survey elaborates on multimodal generation and editing across various domains, comprising image, video, 3D, and audio. Specifically, we summarize the notable advancements with milestone works in these fields and categorize these studies into LLM-based and CLIP/T5-based methods. Then, we summarize the various roles of LLMs in multimodal generation and exhaustively investigate the critical technical components behind these methods and the multimodal datasets utilized in these studies. Additionally, we dig into tool-augmented multimodal agents that can leverage existing generative models for human-computer interaction. Lastly, we discuss the advancements in the generative AI safety field, investigate emerging applications, and discuss future prospects. Our work provides a systematic and insightful overview of multimodal generation and processing, which is expected to advance the development of Artificial Intelligence for Generative Content (AIGC) and world models. A curated list of all related papers can be found at https://github.com/YingqingHe/Awesome-LLMs-meet-Multimodal-Generation


title: X-VILA: Cross-Modality Alignment for Large Language Model

abstract: We introduce X-VILA, an omni-modality model designed to extend the capabilities of large language models (LLMs) by incorporating image, video, and audio modalities. By aligning modality-specific encoders with LLM inputs and diffusion decoders with LLM outputs, X-VILA achieves cross-modality understanding, reasoning, and generation. To facilitate this cross-modality alignment, we curate an effective interleaved any-to-any modality instruction-following dataset. Furthermore, we identify a significant problem with the current cross-modality alignment method, which results in visual information loss. To address the issue, we propose a visual alignment mechanism with a visual embedding highway module. We then introduce a resource-efficient recipe for training X-VILA, that exhibits proficiency in any-to-any modality conversation, surpassing previous approaches by large margins. X-VILA also showcases emergent properties across modalities even in the absence of similar training data. The project will be made open-source.


title: Image-based retrieval of all-day cloud physical parameters for FY4A/AGRI and its application over the Tibetan Plateau

abstract: Satellite remote sensing serves as a crucial means to acquire cloud physical parameters. However, existing official cloud products derived from the advanced geostationary radiation imager (AGRI) onboard the Fengyun-4A geostationary satellite suffer from limitations in computational precision and efficiency. In this study, an image-based transfer learning model (ITLM) was developed to realize all-day and high-precision retrieval of cloud physical parameters using AGRI thermal infrared measurements and auxiliary data. Combining the observation advantages of geostationary and polar-orbiting satellites, ITLM was pre-trained and transfer-trained with official cloud products from advanced Himawari imager (AHI) and Moderate Resolution Imaging Spectroradiometer (MODIS), respectively. Taking official MODIS products as the benchmarks, ITLM achieved an overall accuracy of 79.93% for identifying cloud phase and root mean squared errors of 1.85 km, 6.72 um, and 12.79 for estimating cloud top height, cloud effective radius, and cloud optical thickness, outperforming the precision of official AGRI and AHI products. Compared to the pixel-based random forest model, ITLM utilized the spatial information of clouds to significantly improve the retrieval performance and achieve more than a 6-fold increase in speed for a single full-disk retrieval. Moreover, the AGRI ITLM products with spatiotemporal continuity and high precision were used to accurately describe the spatial distribution characteristics of cloud fractions and cloud properties over the Tibetan Plateau (TP) during both daytime and nighttime, and for the first time provide insights into the diurnal variation of cloud cover and cloud properties for total clouds and deep convective clouds across different seasons.


title: Information-theoretic language of proteinoid gels: Boolean gates and QR codes

abstract: With an aim to build analog computers out of soft matter fluidic systems in future, this work attempts to invent a new information-theoretic language, in the form of two-dimensional Quick Response (QR) codes. This language is, effectively, a digital representation of the analog signals shown by the proteinoids. We use two different experimental techniques: (i) a voltage-sensitive dye and (ii) a pair of differential electrodes, to record the analog signals. The analog signals are digitally approximatied (synthesised) by sampling the analog signals into a series of discrete values, which are then converted into binary representations. We have shown the AND-OR-NOT-XOR-NOR-NAND-XNOR gate representation of the digitally sampled signal of proteinoids. Additional encoding schemes are applied to convert the binary code identified above to a two-dimensional QR code. As a result, the QR code becomes a digital, unique marker of a given proteinoid network. We show that it is possible to retrieve the analog signal from the QR code by scanning the QR code using a mobile phone. Our work shows that soft matter fluidic systems, such as proteinoids, can have a fundamental informatiom-theoretic language, unique to their internal information transmission properties (electrical activity in this case) - such a language can be made universal and accessible to everyone using 2D QR codes, which can digitally encode their internal properties and give an option to recover the original signal when required. On a more fundamental note, this study identifies the techniques of approximating continuum properties of soft matter fluidic systems using a series representation of gates and QR codes, which are a piece-wise digital representation, and thus one step closer to programming the fluids using information-theoretic methods, as suggested almost a decade ago by Tao's fluid program.

categories: cs.ET cs.IT math.IT physics.bio-ph physics.chem-ph


title: Accurate Patient Alignment without Unnecessary Imaging Dose via Synthesizing Patient-specific 3D CT Images from 2D kV Images

abstract: In radiotherapy, 2D orthogonally projected kV images are used for patient alignment when 3D-on-board imaging(OBI) unavailable. But tumor visibility is constrained due to the projection of patient's anatomy onto a 2D plane, potentially leading to substantial setup errors. In treatment room with 3D-OBI such as cone beam CT(CBCT), the field of view(FOV) of CBCT is limited with unnecessarily high imaging dose, thus unfavorable for pediatric patients. A solution to this dilemma is to reconstruct 3D CT from kV images obtained at the treatment position. Here, we propose a dual-models framework built with hierarchical ViT blocks. Unlike a proof-of-concept approach, our framework considers kV images as the solo input and can synthesize accurate, full-size 3D CT in real time(within milliseconds). We demonstrate the feasibility of the proposed approach on 10 patients with head and neck (H&N) cancer using image quality(MAE: <45HU), dosimetrical accuracy(Gamma passing rate (2%/2mm/10%)>97%) and patient position uncertainty(shift error: <0.4mm). The proposed framework can generate accurate 3D CT faithfully mirroring real-time patient position, thus significantly improving patient setup accuracy, keeping imaging dose minimum, and maintaining treatment veracity.


title: MidSurfer: A Parameter-Free Approach for Mid-Surface Extraction from Segmented Volumetric Data

abstract: In the field of volumetric data processing and analysis, extracting mid-surfaces from thinly bounded compartments is crucial for tasks such as surface area estimation and accurate modeling of biological structures, yet it has lacked a standardized approach. To bridge this gap, we introduce MidSurfer--a novel parameter-free method for extracting mid-surfaces from segmented volumetric data. Our method produces smooth, uniformly triangulated meshes that accurately capture the structural features of interest. The process begins with the Ridge Field Transformation step that transforms the segmented input data, followed by the Mid-Polyline Extraction Algorithm that works on individual volume slices. Based on the connectivity of components, this step can result in either single or multiple polyline segments that represent the structural features. These segments form a coherent series across the volume, creating a backbone of regularly distributed points on each slice that represents the mid-surface. Subsequently, we employ a Polyline Zipper Algorithm for triangulation that connects these polyline segments across neighboring slices, yielding a detailed triangulated mid-surface mesh. Our findings demonstrate that this method surpasses previous techniques in versatility, simplicity of use, and accuracy. Our approach is now publicly available as a plugin for ParaView, a widely-used multi-platform tool for data analysis and visualization, and can be found at https://github.com/kaust-vislab/MidSurfer .


title: Obtaining physical layer data of latest generation networks for investigating adversary attacks

abstract: The field of machine learning is developing rapidly and is being used in various fields of science and technology. In this way, machine learning can be used to optimize the functions of latest generation data networks such as 5G and 6G. This also applies to functions at a lower level. A feature of the use of machine learning in the radio path for targeted radiation generation in modern ultra-massive MIMO, reconfigurable intelligent interfaces and other technologies is the complex acquisition and processing of data from the physical layer. Additionally, adversarial measures that manipulate the behaviour of intelligent machine learning models are becoming a major concern, as many machine learning models are sensitive to incorrect input data. To obtain data on attacks directly from processing service information, a simulation model is proposed that works in conjunction with machine learning applications.


title: Spatial Impulse Response Analysis and Ensemble Learning for Efficient Precision Level Sensing

abstract: In this paper, we propose an innovative method for determining the fill level of containers, such as trash cans, addressing a critical aspect of waste management. The method combines spatial impulse response analysis with machine learning (ML) techniques, offering a unique and effective approach for sound-based classification that can be extended to various domains beyond waste management. By employing a buzzer-generated sine sweep signal, we create a distinctive signature specific to the fill level of the waste container. This signature, once accurately decoded, is then interpreted by a specially developed ensemble learning algorithm. Our approach achieves a classification accuracy of over 90% when implemented locally on a development board, optimizing operational efficiencies and eliminating the need to delegate complex classification tasks to external entities. Using low-cost and energy-efficient hardware components, our method offers a cost-effective approach that contributes to sustainable and efficient waste management practices, providing a reliable and locally deployable solution.


title: Sonos Voice Control Bias Assessment Dataset: A Methodology for Demographic Bias Assessment in Voice Assistants

abstract: Recent works demonstrate that voice assistants do not perform equally well for everyone, but research on demographic robustness of speech technologies is still scarce. This is mainly due to the rarity of large datasets with controlled demographic tags. This paper introduces the Sonos Voice Control Bias Assessment Dataset, an open dataset composed of voice assistant requests for North American English in the music domain (1,038 speakers, 166 hours, 170k audio samples, with 9,040 unique labelled transcripts) with a controlled demographic diversity (gender, age, dialectal region and ethnicity). We also release a statistical demographic bias assessment methodology, at the univariate and multivariate levels, tailored to this specific use case and leveraging spoken language understanding metrics rather than transcription accuracy, which we believe is a better proxy for user experience. To demonstrate the capabilities of this dataset and statistical method to detect demographic bias, we consider a pair of state-of-the-art Automatic Speech Recognition and Spoken Language Understanding models. Results show statistically significant differences in performance across age, dialectal region and ethnicity. Multivariate tests are crucial to shed light on mixed effects between dialectal region, gender and age.


title: Luganda Speech Intent Recognition for IoT Applications

abstract: The advent of Internet of Things (IoT) technology has generated massive interest in voice-controlled smart homes. While many voice-controlled smart home systems are designed to understand and support widely spoken languages like English, speakers of low-resource languages like Luganda may need more support. This research project aimed to develop a Luganda speech intent classification system for IoT applications to integrate local languages into smart home environments. The project uses hardware components such as Raspberry Pi, Wio Terminal, and ESP32 nodes as microcontrollers. The Raspberry Pi processes Luganda voice commands, the Wio Terminal is a display device, and the ESP32 nodes control the IoT devices. The ultimate objective of this work was to enable voice control using Luganda, which was accomplished through a natural language processing (NLP) model deployed on the Raspberry Pi. The NLP model utilized Mel Frequency Cepstral Coefficients (MFCCs) as acoustic features and a Convolutional Neural Network (Conv2D) architecture for speech intent classification. A dataset of Luganda voice commands was curated for this purpose and this has been made open-source. This work addresses the localization challenges and linguistic diversity in IoT applications by incorporating Luganda voice commands, enabling users to interact with smart home devices without English proficiency, especially in regions where local languages are predominant.


title: On the modelling of short and intermediate water waves

abstract: The propagation of water waves of finite depth and flat bottom is studied in the case when the depth is not small in comparison to the wavelength. This propagation regime is complementary to the long-wave regime described by the famous KdV equation. The Hamiltonian approach is employed in the derivation of a model equation in evolutionary form, which is both nonlinear and nonlocal, and most likely not integrable. Possible implications for the numerical solutions are discussed.


title: Review of Deep Representation Learning Techniques for Brain-Computer Interfaces and Recommendations

abstract: In the field of brain-computer interfaces (BCIs), the potential for leveraging deep learning techniques for representing electroencephalogram (EEG) signals has gained substantial interest. This review synthesizes empirical findings from a collection of articles using deep representation learning techniques for BCI decoding, to provide a comprehensive analysis of the current state-of-the-art. Each article was scrutinized based on three criteria: (1) the deep representation learning technique employed, (2) the underlying motivation for its utilization, and (3) the approaches adopted for characterizing the learned representations. Among the 81 articles finally reviewed in depth, our analysis reveals a predominance of 31 articles using autoencoders. We identified 13 studies employing self-supervised learning (SSL) techniques, among which ten were published in 2022 or later, attesting to the relative youth of the field. However, at the time being, none of these have led to standard foundation models that are picked up by the BCI community. Likewise, only a few studies have introspected their learned representations. We observed that the motivation in most studies for using representation learning techniques is for solving transfer learning tasks, but we also found more specific motivations such as to learn robustness or invariances, as an algorithmic bridge, or finally to uncover the structure of the data. Given the potential of foundation models to effectively tackle these challenges, we advocate for a continued dedication to the advancement of foundation models specifically designed for EEG signal decoding by using SSL techniques. We also underline the imperative of establishing specialized benchmarks and datasets to facilitate the development and continuous improvement of such foundation models.


title: Subject-Adaptive Transfer Learning Using Resting State EEG Signals for Cross-Subject EEG Motor Imagery Classification

abstract: Electroencephalography (EEG) motor imagery (MI) classification is a fundamental, yet challenging task due to the variation of signals between individuals i.e., inter-subject variability. Previous approaches try to mitigate this using task-specific (TS) EEG signals from the target subject in training. However, recording TS EEG signals requires time and limits its applicability in various fields. In contrast, resting state (RS) EEG signals are a viable alternative due to ease of acquisition with rich subject information. In this paper, we propose a novel subject-adaptive transfer learning strategy that utilizes RS EEG signals to adapt models on unseen subject data. Specifically, we disentangle extracted features into task- and subject-dependent features and use them to calibrate RS EEG signals for obtaining task information while preserving subject characteristics. The calibrated signals are then used to adapt the model to the target subject, enabling the model to simulate processing TS EEG signals of the target subject. The proposed method achieves state-of-the-art accuracy on three public benchmarks, demonstrating the effectiveness of our method in cross-subject EEG MI classification. Our findings highlight the potential of leveraging RS EEG signals to advance practical brain-computer interface systems.


title: Near-Field Spot Beamfocusing: A Correlation-Aware Transfer Learning Approach

abstract: 3D spot beamfocusing (SBF), in contrast to conventional angular-domain beamforming, concentrates radiating power within very small volume in both radial and angular domains in the near-field zone. Recently the implementation of channel-state-information (CSI)-independent machine learning (ML)-based approaches have been developed for effective SBF using extremely-largescale-programable-metasurface (ELPMs). These methods involve dividing the ELPMs into subarrays and independently training them with Deep Reinforcement Learning to jointly focus the beam at the Desired Focal Point (DFP). This paper explores near-field SBF using ELPMs, addressing challenges associated with lengthy training times resulting from independent training of subarrays. To achieve a faster CSIindependent solution, inspired by the correlation between the beamfocusing matrices of the subarrays, we leverage transfer learning techniques. First, we introduce a novel similarity criterion based on the Phase Distribution Image of subarray apertures. Then we devise a subarray policy propagation scheme that transfers the knowledge from trained to untrained subarrays. We further enhance learning by introducing Quasi-Liquid-Layers as a revised version of the adaptive policy reuse technique. We show through simulations that the proposed scheme improves the training speed about 5 times. Furthermore, for dynamic DFP management, we devised a DFP policy blending process, which augments the convergence rate up to 8-fold.


title: NERULA: A Dual-Pathway Self-Supervised Learning Framework for Electrocardiogram Signal Analysis

abstract: Electrocardiogram (ECG) signals are critical for diagnosing heart conditions and capturing detailed cardiac patterns. As wearable single-lead ECG devices become more common, efficient analysis methods are essential. We present NERULA (Non-contrastive ECG and Reconstruction Unsupervised Learning Algorithm), a self-supervised framework designed for single-lead ECG signals. NERULA's dual-pathway architecture combines ECG reconstruction and non-contrastive learning to extract detailed cardiac features. Our 50% masking strategy, using both masked and inverse-masked signals, enhances model robustness against real-world incomplete or corrupted data. The non-contrastive pathway aligns representations of masked and inverse-masked signals, while the reconstruction pathway comprehends and reconstructs missing features. We show that combining generative and discriminative paths into the training spectrum leads to better results by outperforming state-of-the-art self-supervised learning benchmarks in various tasks, demonstrating superior performance in ECG analysis, including arrhythmia classification, gender classification, age regression, and human activity recognition. NERULA's dual-pathway design offers a robust, efficient solution for comprehensive ECG signal interpretation.


title: Beyond Isolated Frames: Enhancing Sensor-Based Human Activity Recognition through Intra- and Inter-Frame Attention

abstract: Human Activity Recognition (HAR) has become increasingly popular with ubiquitous computing, driven by the popularity of wearable sensors in fields like healthcare and sports. While Convolutional Neural Networks (ConvNets) have significantly contributed to HAR, they often adopt a frame-by-frame analysis, concentrating on individual frames and potentially overlooking the broader temporal dynamics inherent in human activities. To address this, we propose the intra- and inter-frame attention model. This model captures both the nuances within individual frames and the broader contextual relationships across multiple frames, offering a comprehensive perspective on sequential data. We further enrich the temporal understanding by proposing a novel time-sequential batch learning strategy. This learning strategy preserves the chronological sequence of time-series data within each batch, ensuring the continuity and integrity of temporal patterns in sensor-based HAR.


title: Approximation by $T$ means with respect to Vilenkin system in Lebesgue spaces and Lipschitz classes

abstract: In this paper we present and prove some new results concerning approximation properties of $T$ means with respect to the Vilenkin system in Lebesgue spaces and Lipschitz classes for any $1\leq p<\infty$. As applications, we obtain extension of some known approximation inequalities.


title: Resonate-and-Fire Spiking Neurons for Target Detection and Hand Gesture Recognition: A Hybrid Approach

abstract: Hand gesture recognition using radar often relies on computationally expensive fast Fourier transforms. This paper proposes an alternative approach that bypasses fast Fourier transforms using resonate-and-fire neurons. These neurons directly detect the hand in the time-domain signal, eliminating the need for fast Fourier transforms to retrieve range information. Following detection, a simple Goertzel algorithm is employed to extract five key features, eliminating the need for a second fast Fourier transform. These features are then fed into a recurrent neural network, achieving an accuracy of 98.21% for classifying five gestures. The proposed approach demonstrates competitive performance with reduced complexity compared to traditional methods


title: Weighted Schreier-type Sets and the Fibonacci Sequence

abstract: For a finite set $A\subset\mathbb{N}$ and $k\in \mathbb{N}$, let $\omega_k(A) = \sum_{i\in A, i\neq k}1$. For each $n\in \mathbb{N}$, define $$a_{k, n}\ =\ |\{E\subset \mathbb{N}\,:\, E = \emptyset\mbox{ or } \omega_k(E) < \min E\leqslant \max E\leqslant n\}|.$$ First, we prove that $$a_{k,k+\ell} \ =\ 2F_{k+\ell},\mbox{ for all }\ell\geqslant 0\mbox{ and }k\geqslant \ell+2,$$ where $F_n$ is the $n$th Fibonacci number. Second, we show that $$|\{E\subset \mathbb{N}\,:\, \max E = n+1, \min E > \omega_{2,3}(E), \mbox{ and }|E|\neq 2\}|\ =\ F_{n},$$ where $\omega_{2,3}(E) = \sum_{i\in E, i\neq 2, 3}1$.


title: Putatively optimal projective spherical designs with little apparent symmetry

abstract: We give some new explicit examples of putatively optimal projective spherical designs. i.e., ones for which there is numerical evidence that they are of minimal size. These form continuous families, and so have little apparent symmetry in general, which requires the introduction of new techniques for their construction. New examples of interest include an 11-point spherical (3, 3)-design for R 3 , and a 12-point spherical (2, 2)-design for R 4 given by four Mercedes-Benz frames that lie on equi-isoclinic planes. We also give results of an extensive numerical study to determine the nature of the real algebraic variety of optimal projective real spherical designs, and in particular when it is a single point (a unique design) or corresponds to an infinite family of designs.


title: Rotations of G\"odel algebras with modal operators

abstract: The present paper is devoted to study the effect of connected and disconnected rotations of G\"odel algebras with operators grounded on directly indecomposable structures. The structures resulting from this construction we will present are nilpotent minimum (with or without negation fixpoint, depending on whether the rotation is connected or disconnected) with special modal operators defined on a directly indecomposable algebra. In this paper we will present a (quasi-)equational definition of these latter structures. Our main results show that directly indecomposable nilpotent minimum algebras (with or without negation fixpoint) with modal operators are fully characterized as connected and disconnected rotations of directly indecomposable G\"odel algebras endowed with modal operators.


title: Enhancing Trust and Security in the Vehicular Metaverse: A Reputation-Based Mechanism for Participants with Moral Hazard

abstract: In this paper, we tackle the issue of moral hazard within the realm of the vehicular Metaverse. A pivotal facilitator of the vehicular Metaverse is the effective orchestration of its market elements, primarily comprised of sensing internet of things (SIoT) devices. These SIoT devices play a critical role by furnishing the virtual service provider (VSP) with real-time sensing data, allowing for the faithful replication of the physical environment within the virtual realm. However, SIoT devices with intentional misbehavior can identify a loophole in the system post-payment and proceeds to deliver falsified content, which cause the whole vehicular Metaverse to collapse. To combat this significant problem, we propose an incentive mechanism centered around a reputation-based strategy. Specifically, the concept involves maintaining reputation scores for participants based on their interactions with the VSP. These scores are derived from feedback received by the VSP from Metaverse users regarding the content delivered by the VSP and are managed using a subjective logic model. Nevertheless, to prevent ``good" SIoT devices with false positive ratings to leave the Metaverse market, we build a vanishing-like system of previous ratings so that the VSP can make informed decisions based on the most recent and accurate data available. Finally, we validate our proposed model through extensive simulations. Our primary results show that our mechanism can efficiently prevent malicious devices from starting their poisoning attacks. At the same time, trustworthy SIoT devices that had a previous miss-classification are not banned from the market.


title: An LSTM Feature Imitation Network for Hand Movement Recognition from sEMG Signals

abstract: Surface Electromyography (sEMG) is a non-invasive signal that is used in the recognition of hand movement patterns, the diagnosis of diseases, and the robust control of prostheses. Despite the remarkable success of recent end-to-end Deep Learning approaches, they are still limited by the need for large amounts of labeled data. To alleviate the requirement for big data, researchers utilize Feature Engineering, which involves decomposing the sEMG signal into several spatial, temporal, and frequency features. In this paper, we propose utilizing a feature-imitating network (FIN) for closed-form temporal feature learning over a 300ms signal window on Ninapro DB2, and applying it to the task of 17 hand movement recognition. We implement a lightweight LSTM-FIN network to imitate four standard temporal features (entropy, root mean square, variance, simple square integral). We then explore transfer learning capabilities by applying the pre-trained LSTM-FIN for tuning to a downstream hand movement recognition task. We observed that the LSTM network can achieve up to 99\% R2 accuracy in feature reconstruction and 80\% accuracy in hand movement recognition. Our results also showed that the model can be robustly applied for both within- and cross-subject movement recognition, as well as simulated low-latency environments. Overall, our work demonstrates the potential of the FIN modeling paradigm in data-scarce scenarios for sEMG signal processing.


title: Tracking Dynamical Transitions using Link Density of Recurrence Networks

abstract: We present Link Density (LD) computed from the Recurrence Network (RN) of a time series data as an effective measure that can detect dynamical transitions in a system. We illustrate its use using time series from the standard Rossler system in the period doubling transitions and the transition to chaos. Moreover, we find that the standard deviation of LD can be more effective in highlighting the transition points. We also consider the variations in data when the parameter of the system is varying due to internal or intrinsic perturbations but at a time scale much slower than that of the dynamics. In this case also, the measure LD and its standard deviation correctly detect transition points in the underlying dynamics of the system. The computation of LD requires minimal computing resources and time, and works well with short data sets. Hence, we propose this measure as a tool to track transitions in dynamics from data, facilitating quicker and more effective analysis of large number of data sets.


title: Robustifying Safety-Aligned Large Language Models through Clean Data Curation

abstract: Large language models (LLMs) are vulnerable when trained on datasets containing harmful content, which leads to potential jailbreaking attacks in two scenarios: the integration of harmful texts within crowdsourced data used for pre-training and direct tampering with LLMs through fine-tuning. In both scenarios, adversaries can compromise the safety alignment of LLMs, exacerbating malfunctions. Motivated by the need to mitigate these adversarial influences, our research aims to enhance safety alignment by either neutralizing the impact of malicious texts in pre-training datasets or increasing the difficulty of jailbreaking during downstream fine-tuning. In this paper, we propose a data curation framework designed to counter adversarial impacts in both scenarios. Our method operates under the assumption that we have no prior knowledge of attack details, focusing solely on curating clean texts. We introduce an iterative process aimed at revising texts to reduce their perplexity as perceived by LLMs, while simultaneously preserving their text quality. By pre-training or fine-tuning LLMs with curated clean texts, we observe a notable improvement in LLM robustness regarding safety alignment against harmful queries. For instance, when pre-training LLMs using a crowdsourced dataset containing 5\% harmful instances, adding an equivalent amount of curated texts significantly mitigates the likelihood of providing harmful responses in LLMs and reduces the attack success rate by 71\%. Our study represents a significant step towards mitigating the risks associated with training-based jailbreaking and fortifying the secure utilization of LLMs.


title: Modally Reduced Representation Learning of Multi-Lead ECG Signals through Simultaneous Alignment and Reconstruction

abstract: Electrocardiogram (ECG) signals, profiling the electrical activities of the heart, are used for a plethora of diagnostic applications. However, ECG systems require multiple leads or channels of signals to capture the complete view of the cardiac system, which limits their application in smartwatches and wearables. In this work, we propose a modally reduced representation learning method for ECG signals that is capable of generating channel-agnostic, unified representations for ECG signals. Through joint optimization of reconstruction and alignment, we ensure that the embeddings of the different channels contain an amalgamation of the overall information across channels while also retaining their specific information. On an independent test dataset, we generated highly correlated channel embeddings from different ECG channels, leading to a moderate approximation of the 12-lead signals from a single-channel embedding. Our generated embeddings can work as competent features for ECG signals for downstream tasks.


title: ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users

abstract: Large-scale pre-trained generative models are taking the world by storm, due to their abilities in generating creative content. Meanwhile, safeguards for these generative models are developed, to protect users' rights and safety, most of which are designed for large language models. Existing methods primarily focus on jailbreak and adversarial attacks, which mainly evaluate the model's safety under malicious prompts. Recent work found that manually crafted safe prompts can unintentionally trigger unsafe generations. To further systematically evaluate the safety risks of text-to-image models, we propose a novel Automatic Red-Teaming framework, ART. Our method leverages both vision language model and large language model to establish a connection between unsafe generations and their prompts, thereby more efficiently identifying the model's vulnerabilities. With our comprehensive experiments, we reveal the toxicity of the popular open-source text-to-image models. The experiments also validate the effectiveness, adaptability, and great diversity of ART. Additionally, we introduce three large-scale red-teaming datasets for studying the safety risks associated with text-to-image models. Datasets and models can be found in https://github.com/GuanlinLee/ART.


title: Decreasing and complete monotonicity of two functions defined by three derivatives of a completely monotonic function involving the trigamma function

abstract: In the paper, by convolution theorem of the Laplace transforms, a monotonicity rule for the ratio of two Laplace transforms, Bernstein's theorem for completely monotonic functions, and other analytic techniques, the authors verify decreasing property of a ratio between three derivatives of a function involving trigamma function and find necessary and sufficient conditions for a function defined by three derivatives of a function involving trigamma function to be completely monotonic. These results confirm previous guesses posed by Qi and generalize corresponding known conclusions.


title: On Tempered Ultradistributions in Classical Morrey Spaces

abstract: We introduce the notion of tempered ultradistributions in classical Morrey spaces by preserving their respective properties. Moreover we investigate some embedding results within the scale of classical Morrey spaces (local Morrey space $\mathcal{L}^{p,\beta}(\mathbb{C}^{n},\mu)$ or global Morrey space $L^{p,\beta}(\mathbb{C}^{n})$ where the underlying functions $\mu$ satisfy the growth condition by polxnomials. Finally we present some results on the new classical Morrey spaces described by tempered ultradistributions satisfying both the vanishing and slow growth conditions in form of embedding theorems.


title: Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classification

abstract: Medical time series data, such as Electroencephalography (EEG) and Electrocardiography (ECG), play a crucial role in healthcare, such as diagnosing brain and heart diseases. Existing methods for medical time series classification primarily rely on handcrafted biomarkers extraction and CNN-based models, with limited exploration of transformers tailored for medical time series. In this paper, we introduce Medformer, a multi-granularity patching transformer tailored specifically for medical time series classification. Our method incorporates three novel mechanisms to leverage the unique characteristics of medical time series: cross-channel patching to leverage inter-channel correlations, multi-granularity embedding for capturing features at different scales, and two-stage (intra- and inter-granularity) multi-granularity self-attention for learning features and correlations within and among granularities. We conduct extensive experiments on five public datasets under both subject-dependent and challenging subject-independent setups. Results demonstrate Medformer's superiority over 10 baselines, achieving top averaged ranking across five datasets on all six evaluation metrics. These findings underscore the significant impact of our method on healthcare applications, such as diagnosing Myocardial Infarction, Alzheimer's, and Parkinson's disease. We release the source code at \url{https://github.com/DL4mHealth/Medformer}.


title: Essential Self-Adjointness of Semi-Bounded Schrodinger Operators on Birth-Death Chains

abstract: We study the essential self-adjointness of semi-bounded Schr\"{o}dinger operators on birth-death chains. First, we offer a general characterization which originates from studying a second order linear recurrence with variational coefficients which comes from the Schr\"{o}dinger operator. As this characterization is algebraically complicated, we present an additional theorem discussing the failure of essential self-adjointness. Finally, we study two specific cases of solutions to equations involving the Schr\"{o}dinger operator over birth-death chains and derive explicit formulas in these cases.


title: Hydrodynamic modulation instability triggered by a two-wave system

abstract: The modulation instability (MI) is responsible for the disintegration of a regular nonlinear wave train and can lead to strong localizations in a from of rogue waves. This mechanism has been studied in a variety of nonlinear dispersive media, such as hydrodynamics, optics, plasma, mechanical systems, electric transmission lines, and Bose-Einstein condensates, while its impact on applied sciences is steadily growing. Following the linear stability analysis of weakly nonlinear waves, the classical MI dynamics, can be triggered when a pair of small-amplitude sidebands are excited within a particular frequency range around the main peak frequency. That is, a three-wave system is usually required to initiate the wave focusing process. Breather solutions of the nonlinear Schr\"odinger equation (NLSE) revealed that MI can generate much more complex localized structures, beyond the three-wave system initialization approach or by means of a continuous spectrum. In this work, we report an experimental study for deep-water surface gravity waves asserting that a MI process can be triggered by a single unstable sideband only, and thus, from a two-wave process when including the contribution of the peak frequency. The experimental data are validated against fully nonlinear hydrodynamic numerical wave tank simulations and show very good agreement. The long-term evolution of such unstable wave trains shows a distinct shift in the recurrent Fermi-Pasta-Ulam-Tsingou focusing cycles, which are captured by the NLSE and fully nonlinear hydrodynamic simulations with minor distinctions.


title: ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text

abstract: The utilization of deep learning on electrocardiogram (ECG) analysis has brought the advanced accuracy and efficiency of cardiac healthcare diagnostics. By leveraging the capabilities of deep learning in semantic understanding, especially in feature extraction and representation learning, this study introduces a new multimodal contrastive pretaining framework that aims to improve the quality and robustness of learned representations of 12-lead ECG signals. Our framework comprises two key components, including Cardio Query Assistant (CQA) and ECG Semantics Integrator(ESI). CQA integrates a retrieval-augmented generation (RAG) pipeline to leverage large language models (LLMs) and external medical knowledge to generate detailed textual descriptions of ECGs. The generated text is enriched with information about demographics and waveform patterns. ESI integrates both contrastive and captioning loss to pretrain ECG encoders for enhanced representations. We validate our approach through various downstream tasks, including arrhythmia detection and ECG-based subject identification. Our experimental results demonstrate substantial improvements over strong baselines in these tasks. These baselines encompass supervised and self-supervised learning methods, as well as prior multimodal pretraining approaches.


title: Soft convex structures

abstract: In this manuscript the idea of soft convex structures is given and some of their properties are investigated. Also, soft convex sets, soft concave sets and soft convex hull operator are defined and their properties are studied. Moreover, the concepts of soft convexly derived operator and soft convex base are studied and their relationship to convex structures are explored.


title: The BiGamma Function and some of its Related Inequalities

abstract: In this article, we define a special function called the Bigamma function. It provides a generalization of Euler's gamma function. Several algebraic properties of this new function are studied. In particular, results linking this new function to the standard Beta function have been provided. We have also established inequalities, which allow to approximate this new function.


title: Sublinear Cuts are the Exception in BDF-GIRGs

abstract: The introduction of geometry has proven instrumental in the efforts towards more realistic models for real-world networks. In Geometric Inhomogeneous Random Graphs (GIRGs), Euclidean Geometry induces clustering of the vertices, which is widely observed in networks in the wild. Euclidean Geometry in multiple dimensions however restricts proximity of vertices to those cases where vertices are close in each coordinate. We introduce a large class of GIRG extensions, called BDF-GIRGs, which capture arbitrary hierarchies of the coordinates within the distance function of the vertex feature space. These distance functions have the potential to allow more realistic modeling of the complex formation of social ties in real-world networks, where similarities between people lead to connections. Here, similarity with respect to certain features, such as familial kinship or a shared workplace, suffices for the formation of ties. It is known that - while many key properties of GIRGs, such as log-log average distance and sparsity, are independent of the distance function - the Euclidean metric induces small separators, i.e. sublinear cuts of the unique giant component in GIRGs, whereas no such sublinear separators exist under the component-wise minimum distance. Building on work of Lengler and Todorovi\'{c}, we give a complete classification for the existence of small separators in BDF-GIRGs. We further show that BDF-GIRGs all fulfill a stochastic triangle inequality and thus also exhibit clustering.


title: Interaction and adiabatic evolution of orthodromic and antidromic impulses in the axoplasmic fluid

abstract: Unlike expected from the Hodgkin-Huxley model predictions, in which there is annihilation once orthodromic and antidromic impulses collide, the Heimburg-Jackson model demonstrates that both impulses penetrate each other as it has been shown experimentally. These impulses can be depicted as low amplitude nonlinear excitations in a weakly dissipative soliton model described by the damped NLSE. In view of the above, the Karpman-Solov'ev-Maslov perturbation theory turns out to be ideal to study the interaction and adiabatic evolution of orthodromic and antidromic impulses once axoplasmic fluid is present.


title: Unifying trigonometric and hyperbolic function derivatives via negative integer order polylogarithms

abstract: Special functions like the polygamma, Hurwitz zeta, and Lerch zeta functions have sporadically been connected with the nth derivatives of trigonometric functions. We show the polylogarithm $\text{Li}_s(z)$, a function of complex argument and order $z$ and $s$, encodes the nth derivatives of the cotangent, tangent, cosecant and secant functions, and their hyperbolic equivalents, at negative integer orders $s = -n$. We then show how at the same orders, the polylogarithm represents the nth application of the operator $x \frac{d}{dx}$ on the inverse trigonometric and hyperbolic functions. Finally, we construct a sum relating two polylogarithms of order $-n$ to a linear combination of polylogarithms of orders $s = 0, -1, -2, ..., -n$.


title: On a conjecture about generalized integration operators on Hardy spaces

abstract: A conjecture posed by Chalmoukis in 2020 states that if $T_{g,a}:H^p\to H^q(0<q<p<\infty)$ is bounded, then $g$ must be in $H^{\frac{pq}{p-q}}$. In this article, we provide a positive answer to the aforementioned conjecture. We also consider the compactness of $T_{g,a}:H^p\to H^q(0<q<p<\infty)$.


title: Multi-modal Mood Reader: Pre-trained Model Empowers Cross-Subject Emotion Recognition

abstract: Emotion recognition based on Electroencephalography (EEG) has gained significant attention and diversified development in fields such as neural signal processing and affective computing. However, the unique brain anatomy of individuals leads to non-negligible natural differences in EEG signals across subjects, posing challenges for cross-subject emotion recognition. While recent studies have attempted to address these issues, they still face limitations in practical effectiveness and model framework unity. Current methods often struggle to capture the complex spatial-temporal dynamics of EEG signals and fail to effectively integrate multimodal information, resulting in suboptimal performance and limited generalizability across subjects. To overcome these limitations, we develop a Pre-trained model based Multimodal Mood Reader for cross-subject emotion recognition that utilizes masked brain signal modeling and interlinked spatial-temporal attention mechanism. The model learns universal latent representations of EEG signals through pre-training on large scale dataset, and employs Interlinked spatial-temporal attention mechanism to process Differential Entropy(DE) features extracted from EEG data. Subsequently, a multi-level fusion layer is proposed to integrate the discriminative features, maximizing the advantages of features across different dimensions and modalities. Extensive experiments on public datasets demonstrate Mood Reader's superior performance in cross-subject emotion recognition tasks, outperforming state-of-the-art methods. Additionally, the model is dissected from attention perspective, providing qualitative analysis of emotion-related brain areas, offering valuable insights for affective research in neural signal processing.


title: Optimal Multiclass U-Calibration Error and Beyond

abstract: We consider the problem of online multiclass U-calibration, where a forecaster aims to make sequential distributional predictions over $K$ classes with low U-calibration error, that is, low regret with respect to all bounded proper losses simultaneously. Kleinberg et al. (2023) developed an algorithm with U-calibration error $O(K\sqrt{T})$ after $T$ rounds and raised the open question of what the optimal bound is. We resolve this question by showing that the optimal U-calibration error is $\Theta(\sqrt{KT})$ -- we start with a simple observation that the Follow-the-Perturbed-Leader algorithm of Daskalakis and Syrgkanis (2016) achieves this upper bound, followed by a matching lower bound constructed with a specific proper loss (which, as a side result, also proves the optimality of the algorithm of Daskalakis and Syrgkanis (2016) in the context of online learning against an adversary with finite choices). We also strengthen our results under natural assumptions on the loss functions, including $\Theta(\log T)$ U-calibration error for Lipschitz proper losses, $O(\log T)$ U-calibration error for a certain class of decomposable proper losses, U-calibration error bounds for proper losses with a low covering number, and others.