title: Construction Repetition Reduces Information Rate in Dialogue

abstract: Speakers repeat constructions frequently in dialogue. Due to their peculiar information-theoretic properties, repetitions can be thought of as a strategy for cost-effective communication. In this study, we focus on the repetition of lexicalised constructions -- i.e., recurring multi-word units -- in English open-domain spoken dialogues. We hypothesise that speakers use construction repetition to mitigate information rate, leading to an overall decrease in utterance information content over the course of a dialogue. We conduct a quantitative analysis, measuring the information content of constructions and that of their containing utterances, estimating information content with an adaptive neural language model. We observe that construction usage lowers the information content of utterances. This facilitating effect (i) increases throughout dialogues, (ii) is boosted by repetition, (iii) grows as a function of repetition frequency and density, and (iv) is stronger for repetitions of referential constructions.


title: A journey into the proton structure: Progresses and challenges

abstract: Unraveling the inner dynamics of gluons and quarks inside nucleons is a primary target of studies at new-generation colliding machines. Finding an answer to fundamental problems of Quantum ChromoDynamics, such as the origin of nucleon mass and spin, strongly depends on our ability of reconstructing the 3D motion of partons inside the parent hadrons. We present progresses and challenges in the extraction of TMD parton densities, with particular attention to the ones describing polarization states of gluons, which still represent an largely unexplored field. Then, we highlight connections with corresponding parton densities in the high-energy limit, the so-called unintegrated gluon distributions or UGDs and, more in general, to recent developments in high-energy physics.


title: A Policy-Guided Imitation Approach for Offline Reinforcement Learning

abstract: Offline reinforcement learning (RL) methods can generally be categorized into two types: RL-based and Imitation-based. RL-based methods could in principle enjoy out-of-distribution generalization but suffer from erroneous off-policy evaluation. Imitation-based methods avoid off-policy evaluation but are too conservative to surpass the dataset. In this study, we propose an alternative approach, inheriting the training stability of imitation-style methods while still allowing logical out-of-distribution generalization. We decompose the conventional reward-maximizing policy in offline RL into a guide-policy and an execute-policy. During training, the guide-poicy and execute-policy are learned using only data from the dataset, in a supervised and decoupled manner. During evaluation, the guide-policy guides the execute-policy by telling where it should go so that the reward can be maximized, serving as the \textit{Prophet}. By doing so, our algorithm allows \textit{state-compositionality} from the dataset, rather than \textit{action-compositionality} conducted in prior imitation-style methods. We dumb this new approach Policy-guided Offline RL (\texttt{POR}). \texttt{POR} demonstrates the state-of-the-art performance on D4RL, a standard benchmark for offline RL. We also highlight the benefits of \texttt{POR} in terms of improving with supplementary suboptimal data and easily adapting to new tasks by only changing the guide-poicy.


title: Variational problems in thin elastic structures

abstract: For two different scenarios regarding thin elastic structures, described by 2d-F\"oppl-von K\'arm\'an plate models, we obtain energy scaling laws. Firstly, assuming the reference geometry being that of a singular excess-cone, we obtain fairly optimal upper- and lower energy bounds and we highlight how those bounds scale wrt. the thickness-parameter $h.$ Secondly, we consider the half sphere, while being indented by a thin object at the top and perpendicular to its surface. In this situation we provide an energy-scaling law, for radial symmetric admissible maps, this time, depending on $h$ and the indentation depth $d.$


title: Azimuthal anisotropy of dijet events in PbPb collisions at $\sqrt{s_\mathrm{NN}}$ = 5.02 TeV

abstract: The path-length dependent parton energy loss within the dense partonic medium created in lead-lead collisions at a nucleon-nucleon center-of-mass energy of $\sqrt{s_\mathrm{NN}}$ = 5.02 TeV is studied by determining the azimuthal anisotropies for dijets with high transverse momentum. The data were collected by the CMS experiment in 2018 and correspond to an integrated luminosity of 1.69 nb$^{-1}$. For events containing back-to-back jets, correlations in relative azimuthal angle and pseudorapidity ($\eta$) between jets and hadrons, and between two hadrons, are constructed. The anisotropies are expressed as the Fourier expansion coefficients $v_n$, $n = $ 2-4 of these azimuthal distributions. The dijet $v_n$ values are extracted from long-range (1.5 $\lt$ $\vert\Delta\eta\vert$ $\lt$ 2.5) components of these correlations, which suppresses the background contributions from jet fragmentation processes. Positive dijet $v_2$ values are observed which increase from central to more peripheral events, while the $v_3$ and $v_4$ values are consistent with zero within experimental uncertainties.


title: Distributionally Robust Causal Inference with Observational Data

abstract: We consider the estimation of average treatment effects in observational studies and propose a new framework of robust causal inference with unobserved confounders. Our approach is based on distributionally robust optimization and proceeds in two steps. We first specify the maximal degree to which the distribution of unobserved potential outcomes may deviate from that of observed outcomes. We then derive sharp bounds on the average treatment effects under this assumption. Our framework encompasses the popular marginal sensitivity model as a special case, and we demonstrate how the proposed methodology can address a primary challenge of the marginal sensitivity model that it produces uninformative results when unobserved confounders substantially affect treatment and outcome. Specifically, we develop an alternative sensitivity model, called the distributional sensitivity model, under the assumption that heterogeneity of treatment effect due to unobserved variables is relatively small. Unlike the marginal sensitivity model, the distributional sensitivity model allows for potential lack of overlap and often produces informative bounds even when unobserved variables substantially affect both treatment and outcome. Finally, we show how to extend the distributional sensitivity model to difference-in-differences designs and settings with instrumental variables. Through simulation and empirical studies, we demonstrate the applicability of the proposed methodology.


title: A graph-database approach to assess the impact of demand-responsive services on public transit accessibility

abstract: This paper proposes an approach to analyze the impact of multimodal Public Transit (PT), combining conventional fixed-route transit and Demand-Responsive Transit (DRT), on equality in transport accessibility distribution. We construct a graph model of multimodal PT in Neo4j, based on General Transit Feed Specification (GTFS) data, modeling DRT analytically (using continuous approximation). We quantify service quality on any location using an accessibility measure, indicating how easily other places or opportunities can be reached. We quantify inequality of accessibility distribution. We state the problem of allocating a DRT fleet to minimize inequality and show its NP-completeness. We showcase our approach on the transportation network of the French town of Royan.


title: A Group Public Goods Game with Position Uncertainty

abstract: We model a dynamic public good contribution game, where players are (naturally) formed into groups. The groups are exogenously placed in a sequence, with limited information available to players about their groups' position in the sequence. Contribution decisions are made by players simultaneously and independently, and the groups' total contribution is made sequentially. We try to capture both inter and intra-group behaviors and analyze different situations where players observe partial history about total contributions of their predecessor groups. Given this framework, we show that even when players observe a history of defection (no contribution), a cooperative outcome is achievable. This is particularly interesting in the situation when players observe only their immediate predecessor groups' contribution, where we observe that players play an important role in motivating others to contribute.


title: Multilevel Bayesian Quadrature

abstract: Multilevel Monte Carlo is a key tool for approximating integrals involving expensive scientific models. The idea is to use approximations of the integrand to construct an estimator with improved accuracy over classical Monte Carlo. We propose to further enhance multilevel Monte Carlo through Bayesian surrogate models of the integrand, focusing on Gaussian process models and the associated Bayesian quadrature estimators. We show, using both theory and numerical experiments, that our approach can lead to significant improvements in accuracy when the integrand is expensive and smooth, and when the dimensionality is small or moderate. We conclude the paper with a case study illustrating the potential impact of our method in landslide-generated tsunami modelling, where the cost of each integrand evaluation is typically too large for operational settings.


title: Aplicaci\'on de redes neuronales convolucionales profundas al diagn\'ostico asistido de la enfermedad de Alzheimer

abstract: Currently, the diagnosis of Alzheimer's disease is a complex and error-prone process. Improving this diagnosis could allow earlier detection of the disease and improve the quality of life of patients and their families. For this work, we will use 249 brain images from two modalities: PET and MRI, taken from the ADNI database, and labelled into three classes according to the degree of development of Alzheimer's disease. We propose the development of a convolutional neural network to perform the classification of these images, during which, we will study the appropriate depth of the networks for this problem, the importance of pre-processing medical images, the use of transfer learning and data augmentation techniques as tools to reduce the effects of the problem of having too little data, and the simultaneous use of multiple medical imaging modalities. We also propose the application of an evaluation method that guarantees a good degree of repeatability of the results even when using a small dataset. Following this evaluation method, our best final model, which makes use of transfer learning with COVID-19 data, achieves an accuracy d 68\%. In addition, in an independent test set, this same model achieves 70\% accuracy, a promising result given the small size of our dataset. We further conclude that augmenting the depth of the networks helps with this problem, that image pre-processing is a fundamental process to address this type of medical problem, and that the use of data augmentation and the use of pre-trained networks with images of other diseases can provide significant improvements.


title: Combination Of Convolution Neural Networks And Deep Neural Networks For Fake News Detection

abstract: Nowadays, People prefer to follow the latest news on social media, as it is cheap, easily accessible, and quickly disseminated. However, it can spread fake or unreliable, low-quality news that intentionally contains false information. The spread of fake news can have a negative effect on people and society. Given the seriousness of such a problem, researchers did their best to identify patterns and characteristics that fake news may exhibit to design a system that can detect fake news before publishing. In this paper, we have described the Fake News Challenge stage #1 (FNC-1) dataset and given an overview of the competitive attempts to build a fake news detection system using the FNC-1 dataset. The proposed model was evaluated with the FNC-1 dataset. A competitive dataset is considered an open problem and a challenge worldwide. This system's procedure implies processing the text in the headline and body text columns with different natural language processing techniques. After that, the extracted features are reduced using the elbow truncated method, finding the similarity between each pair using the soft cosine similarity method. The new feature is entered into CNN and DNN deep learning approaches. The proposed system detects all the categories with high accuracy except the disagree category. As a result, the system achieves up to 84.6 % accuracy, classifying it as the second ranking based on other competitive studies regarding this dataset.


title: Code Recommendation for Open Source Software Developers

abstract: Open Source Software (OSS) is forming the spines of technology infrastructures, attracting millions of talents to contribute. Notably, it is challenging and critical to consider both the developers' interests and the semantic features of the project code to recommend appropriate development tasks to OSS developers. In this paper, we formulate the novel problem of code recommendation, whose purpose is to predict the future contribution behaviors of developers given their interaction history, the semantic features of source code, and the hierarchical file structures of projects. Considering the complex interactions among multiple parties within the system, we propose CODER, a novel graph-based code recommendation framework for open source software developers. CODER jointly models microscopic user-code interactions and macroscopic user-project interactions via a heterogeneous graph and further bridges the two levels of information through aggregation on file-structure graphs that reflect the project hierarchy. Moreover, due to the lack of reliable benchmarks, we construct three large-scale datasets to facilitate future research in this direction. Extensive experiments show that our CODER framework achieves superior performance under various experimental settings, including intra-project, cross-project, and cold-start recommendation. We will release all the datasets, code, and utilities for data retrieval upon the acceptance of this work.


title: Characterization of elastic topological states using dynamic mode decomposition

abstract: Elastic topological states have been receiving increased intention in numerous scientific and engineering fields due to their defect-immune nature, resulting in applications of vibration control and information processing. Here, we present the data-driven discovery of elastic topological states using dynamic mode decomposition (DMD). The DMD spectrum and DMD modes are retrieved from the propagation of the relevant states along the topological boundary, where their nature is learned by DMD. Applications such as classification and prediction can be achieved by the underlying characteristics from DMD. We demonstrate the classification between topological and traditional metamaterials using DMD modes. Moreover, the model enabled by the DMD modes realizes the prediction of topological state propagation along the given interface. Our approach to characterizing topological states using DMD can pave the way towards data-driven discovery of topological phenomena in material physics and more broadly lattice systems.


title: Two families of circulant nut graphs

abstract: A circulant nut graph is a non-trivial simple graph whose adjacency matrix is a circulant matrix of nullity one such that its non-zero null space vectors have no zero elements. The study of circulant nut graphs was originally initiated by Ba\v{s}i\'c et al. [Art Discrete Appl. Math. 5(2) (2021) #P2.01], where a conjecture was made regarding the existence of all the possible pairs $(n, d)$ for which there exists a $d$-regular circulant nut graph of order $n$. Later on, it was proved by Damnjanovi\'c and Stevanovi\'c [Linear Algebra Appl. 633 (2022) 127-151] that for each odd $t \ge 3$ such that $t\not\equiv_{10}1$ and $t\not\equiv_{18}15$, the $4t$-regular circulant graph of order $n$ with the generator set $\{ 1, 2, 3, \ldots, 2t+1 \} \setminus \{t\})$ must necessarily be a nut graph for each even $n \ge 4t + 4$. In this paper, we extend these results by constructing two families of circulant nut graphs. The first family comprises the $4t$-regular circulant graphs of order $n$ which correspond to the generator sets $\{1, 2, \ldots, t-1\} \cup \left\{\frac{n}{4}, \frac{n}{4} + 1 \right\} \cup \left\{\frac{n}{2} - (t-1), \ldots, \frac{n}{2} - 2, \frac{n}{2} - 1 \right\}$, for each odd $t \in \mathbb{N}$ and $n \ge 4t + 4$ divisible by four. The second family consists of the $4t$-regular circulant graphs of order $n$ which correspond to the generator sets $\{1, 2, \ldots, t-1\} \cup \left\{\frac{n+2}{4}, \frac{n+6}{4} \right\} \cup \left\{\frac{n}{2} - (t-1), \ldots, \frac{n}{2} - 2, \frac{n}{2}-1 \right\}$, for each $t \in \mathbb{N}$ and $n \ge 4t + 6$ such that $n \equiv_{4} 2$. We prove that all of the graphs which belong to these families are indeed nut graphs, thereby fully resolving the $4t$-regular circulant nut graph order-degree existence problem whenever $t$ is odd and partially solving this problem for even values of $t$ as well.


title: NOMA Made Practical: Removing the Receive SIC Processing through Interference Exploitation

abstract: Non-orthogonal multiple access (NOMA) is a powerful transmission technique that enhances the spectral efficiency of communication links, and is being investigated for 5G standards and beyond. A major drawback of NOMA is the need to apply successive interference cancellation (SIC) at the receiver on a symbol-by-symbol basis, which limits its practicality. To circumvent this, in this paper a novel constructive multiple access (CoMA) scheme is proposed and investigated. CoMA aligns the superimposed signals to the different users constructively to the signal of interest. Since the superimposed signal aligns with the data signal, there is no need to remove it at the receiver using SIC. Accordingly, SIC component can be removed at the receiver side. In this regard and in order to provide a comprehensive investigation and comparison, different optimization problems for user paring NOMA multiple-input-single-output (MISO) systems are considered. Firstly, an optimal precoder to minimize the total transmission power for CoMA subject to a quality-of-service constraint is obtained, and compared to conventional NOMA. Then, a precoder that minimizes the CoMA symbol error rate (SER) subject to power constraint is investigated. Further, the computational complexity of CoMA is considered and compared with conventional NOMA scheme in terms of total number of complex operations. The results in this paper prove the superiority of the proposed CoMA scheme over the conventional NOMA technique, and demonstrate that CoMA is an attractive solution for user paring NOMA MISO systems with low number of BS antennas, while circumventing the receive SIC complexity.


title: Decoupling Deep Learning for Interpretable Image Recognition

abstract: The interpretability of neural networks has recently received extensive attention. Previous prototype-based explainable networks involved prototype activation in both reasoning and interpretation processes, requiring specific explainable structures for the prototype, thus making the network less accurate as it gains interpretability. Therefore, the decoupling prototypical network (DProtoNet) was proposed to avoid this problem. This new model contains encoder, inference, and interpretation modules. As regards the encoder module, unrestricted feature masks were presented to generate expressive features and prototypes. Regarding the inference module, a multi-image prototype learning method was introduced to update prototypes so that the network can learn generalized prototypes. Finally, concerning the interpretation module, a multiple dynamic masks (MDM) decoder was suggested to explain the neural network, which generates heatmaps using the consistent activation of the original image and mask image at the detection nodes of the network. It decouples the inference and interpretation modules of a prototype-based network by avoiding the use of prototype activation to explain the network's decisions in order to simultaneously improve the accuracy and interpretability of the neural network. The multiple public general and medical datasets were tested, and the results confirmed that our method could achieve a 5% improvement in accuracy and state-of-the-art interpretability compared with previous methods.


title: Spectral properties of balanced trees and dendrimers

abstract: We investigate the spectral properties of balanced trees and dendrimers, with a view toward unifying and improving the existing results. Here we find a semi-factorized formula for their characteristic polynomials. Afterwards, we determine their spectra via the aforementioned factors. In the end, we analyze the behavior of the energy of dendrimers and compute lower and upper bound approximations for it.


title: Fair Effect Attribution in Parallel Online Experiments

abstract: A/B tests serve the purpose of reliably identifying the effect of changes introduced in online services. It is common for online platforms to run a large number of simultaneous experiments by splitting incoming user traffic randomly in treatment and control groups. Despite a perfect randomization between different groups, simultaneous experiments can interact with each other and create a negative impact on average population outcomes such as engagement metrics. These are measured globally and monitored to protect overall user experience. Therefore, it is crucial to measure these interaction effects and attribute their overall impact in a fair way to the respective experimenters. We suggest an approach to measure and disentangle the effect of simultaneous experiments by providing a cost sharing approach based on Shapley values. We also provide a counterfactual perspective, that predicts shared impact based on conditional average treatment effects making use of causal inference techniques. We illustrate our approach in real world and synthetic data experiments.


title: Reachable Polyhedral Marching (RPM): An Exact Analysis Tool for Deep-Learned Control Systems

abstract: We present a tool for computing exact forward and backward reachable sets of deep neural networks with rectified linear unit (ReLU) activation. We then develop algorithms using this tool to compute invariant sets and regions of attraction (ROAs) for control systems with neural networks in the feedback loop. Our algorithm is unique in that it builds the reachable sets by incrementally enumerating polyhedral regions in the input space, rather than iterating layer-by-layer through the network as in other methods. When performing safety verification, if an unsafe region is found, our algorithm can return this result without completing the full reachability computation, thus giving an anytime property that accelerates safety verification. Furthermore, we introduce a method to accelerate the computation of ROAs in the case that deep learned components are homeomorphisms, which we find is surprisingly common in practice. We demonstrate our tool in several test cases. We compute a ROA for a learned van der Pol oscillator model. We find a control invariant set for a learned torque-controlled pendulum model. We also verify specific safety properties for multiple deep networks related to the ACAS Xu aircraft collision advisory system. Finally, we apply our algorithm to find ROAs for an image-based aircraft runway taxi problem. Algorithm source code: https://github.com/StanfordMSL/Neural-Network-Reach .


title: Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution

abstract: Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in NeuroAI. A core component of this is the embodied Turing test, which challenges AI animal models to interact with the sensorimotor world at skill levels akin to their living counterparts. The embodied Turing test shifts the focus from those capabilities like game playing and language that are especially well-developed or uniquely human to those capabilities, inherited from over 500 million years of evolution, that are shared with all animals. Building models that can pass the embodied Turing test will provide a roadmap for the next generation of AI.


title: Time-weighted estimates for the Blackstock equation in nonlinear ultrasonics

abstract: High frequencies at which ultrasonic waves travel give rise to nonlinear phenomena. In thermoviscous fluids, these are captured by Blackstock's acoustic wave equation with strong damping. We revisit in this work its well-posedness analysis. By exploiting the parabolic-like character of this equation due to strong dissipation, we construct a time-weighted energy framework for investigating its local solvability. In this manner, we obtain the small-data well-posedness on bounded domains under less restrictive regularity assumptions on the initial conditions compared to the known results. Furthermore, we prove that such initial boundary-value problems for the Blackstock equation are globally solvable and that their solution decays exponentially fast to the steady state.


title: Well-definedness of Physical Law Learning: The Uniqueness Problem

abstract: Physical law learning is the ambiguous attempt at automating the derivation of governing equations with the use of machine learning techniques. The current literature focuses however solely on the development of methods to achieve this goal, and a theoretical foundation is at present missing. This paper shall thus serve as a first step to build a comprehensive theoretical framework for learning physical laws, aiming to provide reliability to according algorithms. One key problem consists in the fact that the governing equations might not be uniquely determined by the given data. We will study this problem in the common situation that a physical law is described by an ordinary or partial differential equation. For various different classes of differential equations, we provide both necessary and sufficient conditions for a function to uniquely determine the differential equation which is governing the phenomenon. We then use our results to devise numerical algorithms to determine whether a function solves a differential equation uniquely. Finally, we provide extensive numerical experiments showing that our algorithms in combination with common approaches for learning physical laws indeed allow to guarantee that a unique governing differential equation is learnt, without assuming any knowledge about the function, thereby ensuring reliability.


title: Modular machine learning-based elastoplasticity: generalization in the context of limited data

abstract: The development of accurate constitutive models for materials that undergo path-dependent processes continues to be a complex challenge in computational solid mechanics. Challenges arise both in considering the appropriate model assumptions and from the viewpoint of data availability, verification, and validation. Recently, data-driven modeling approaches have been proposed that aim to establish stress-evolution laws that avoid user-chosen functional forms by relying on machine learning representations and algorithms. However, these approaches not only require a significant amount of data but also need data that probes the full stress space with a variety of complex loading paths. Furthermore, they rarely enforce all necessary thermodynamic principles as hard constraints. Hence, they are in particular not suitable for low-data or limited-data regimes, where the first arises from the cost of obtaining the data and the latter from the experimental limitations of obtaining labeled data, which is commonly the case in engineering applications. In this work, we discuss a hybrid framework that can work on a variable amount of data by relying on the modularity of the elastoplasticity formulation where each component of the model can be chosen to be either a classical phenomenological or a data-driven model depending on the amount of available information and the complexity of the response. The method is tested on synthetic uniaxial data coming from simulations as well as cyclic experimental data for structural materials. The discovered material models are found to not only interpolate well but also allow for accurate extrapolation in a thermodynamically consistent manner far outside the domain of the training data. Training aspects and details of the implementation of these models into Finite Element simulations are discussed and analyzed.


title: How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders

abstract: Masked Autoencoders (MAE) based on a reconstruction task have risen to be a promising paradigm for self-supervised learning (SSL) and achieve state-of-the-art performance across different benchmark datasets. However, despite its impressive empirical success, there is still limited theoretical understanding of it. In this paper, we propose a theoretical understanding of how masking matters for MAE to learn meaningful features. We establish a close connection between MAE and contrastive learning, which shows that MAE implicit aligns the mask-induced positive pairs. Built upon this connection, we develop the first downstream guarantees for MAE methods, and analyze the effect of mask ratio. Besides, as a result of the implicit alignment, we also point out the dimensional collapse issue of MAE, and propose a Uniformity-enhanced MAE (U-MAE) loss that can effectively address this issue and bring significant improvements on real-world datasets, including CIFAR-10, ImageNet-100, and ImageNet-1K. Code is available at (https://github.com/zhangq327/U-MAE).


title: Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representations

abstract: The pretasks are mainly built on mutual information estimation, which requires data augmentation to construct positive samples with similar semantics to learn invariant signals and negative samples with dissimilar semantics in order to empower representation discriminability. However, an appropriate data augmentation configuration depends heavily on lots of empirical trials such as choosing the compositions of data augmentation techniques and the corresponding hyperparameter settings. We propose an augmentation-free graph contrastive learning method, invariant-discriminative graph contrastive learning (iGCL), that does not intrinsically require negative samples. iGCL designs the invariant-discriminative loss (ID loss) to learn invariant and discriminative representations. On the one hand, ID loss learns invariant signals by directly minimizing the mean square error between the target samples and positive samples in the representation space. On the other hand, ID loss ensures that the representations are discriminative by an orthonormal constraint forcing the different dimensions of representations to be independent of each other. This prevents representations from collapsing to a point or subspace. Our theoretical analysis explains the effectiveness of ID loss from the perspectives of the redundancy reduction criterion, canonical correlation analysis, and information bottleneck principle. The experimental results demonstrate that iGCL outperforms all baselines on 5 node classification benchmark datasets. iGCL also shows superior performance for different label ratios and is capable of resisting graph attacks, which indicates that iGCL has excellent generalization and robustness. The source code is available at https://github.com/lehaifeng/T-GCN/tree/master/iGCL.


title: Fisher's Noncentral Hypergeometric Distribution for Population Size Estimation

abstract: We introduce a method to make inference on the subgroups' sizes of a heterogeneous population using survey data, even in the presence of a single list. To this aim, we use Fisher's noncentral hypergeometric distribution, which allows us to account for the possibility that capture heterogeneity is related to key survey variables. We propose a Bayesian approach for estimating the population sizes posterior distributions, exploiting both extra-experimental information, e.g., coming from administrative data, and the computational efficiency of MCMC and ABC methods. The motivating case study deals with the size estimation of the population of Italian youngsters who are not employed one year after graduating by gender and degree program. We account for the possibility that surveys' response rates differ according to individuals' employment status, implying a not-at-random missing data scenario. We find that employed persons are generally more inclined to answer the questionnaire; this behavior might imply the overestimation of the employment rate.


title: Mini-Batch Learning Strategies for modeling long term temporal dependencies: A study in environmental applications

abstract: In many environmental applications, recurrent neural networks (RNNs) are often used to model physical variables with long temporal dependencies. However, due to mini-batch training, temporal relationships between training segments within the batch (intra-batch) as well as between batches (inter-batch) are not considered, which can lead to limited performance. Stateful RNNs aim to address this issue by passing hidden states between batches. Since Stateful RNNs ignore intra-batch temporal dependency, there exists a trade-off between training stability and capturing temporal dependency. In this paper, we provide a quantitative comparison of different Stateful RNN modeling strategies, and propose two strategies to enforce both intra- and inter-batch temporal dependency. First, we extend Stateful RNNs by defining a batch as a temporally ordered set of training segments, which enables intra-batch sharing of temporal information. While this approach significantly improves the performance, it leads to much larger training times due to highly sequential training. To address this issue, we further propose a new strategy which augments a training segment with an initial value of the target variable from the timestep right before the starting of the training segment. In other words, we provide an initial value of the target variable as additional input so that the network can focus on learning changes relative to that initial value. By using this strategy, samples can be passed in any order (mini-batch training) which significantly reduces the training time while maintaining the performance. In demonstrating our approach in hydrological modeling, we observe that the most significant gains in predictive accuracy occur when these methods are applied to state variables whose values change more slowly, such as soil water and snowpack, rather than continuously moving flux variables such as streamflow.


title: A note on representations of $SL_n$

abstract: In this note, we study irreducible unitary representations of special linear groups of lower ranks, in terms of the matrix models of Gelfand-Naimark and Gelfand-Graev. Review of existing literature is provided. We also add some new calculation based on existing theory.


title: When to Update Your Model: Constrained Model-based Reinforcement Learning

abstract: Designing and analyzing model-based RL (MBRL) algorithms with guaranteed monotonic improvement has been challenging, mainly due to the interdependence between policy optimization and model learning. Existing discrepancy bounds generally ignore the impacts of model shifts, and their corresponding algorithms are prone to degrade performance by drastic model updating. In this work, we first propose a novel and general theoretical scheme for a non-decreasing performance guarantee of MBRL. Our follow-up derived bounds reveal the relationship between model shifts and performance improvement. These discoveries encourage us to formulate a constrained lower-bound optimization problem to permit the monotonicity of MBRL. A further example demonstrates that learning models from a dynamically-varying number of explorations benefit the eventual returns. Motivated by these analyses, we design a simple but effective algorithm CMLO (Constrained Model-shift Lower-bound Optimization), by introducing an event-triggered mechanism that flexibly determines when to update the model. Experiments show that CMLO surpasses other state-of-the-art methods and produces a boost when various policy optimization methods are employed.


title: Self-Improving SLAM in Dynamic Environments: Learning When to Mask

abstract: Visual SLAM - Simultaneous Localization and Mapping - in dynamic environments typically relies on identifying and masking image features on moving objects to prevent them from negatively affecting performance. Current approaches are suboptimal: they either fail to mask objects when needed or, on the contrary, mask objects needlessly. Thus, we propose a novel SLAM that learns when masking objects improves its performance in dynamic scenarios. Given a method to segment objects and a SLAM, we give the latter the ability of Temporal Masking, i.e., to infer when certain classes of objects should be masked to maximize any given SLAM metric. We do not make any priors on motion: our method learns to mask moving objects by itself. To prevent high annotations costs, we created an automatic annotation method for self-supervised training. We constructed a new dataset, named ConsInv, which includes challenging real-world dynamic sequences respectively indoors and outdoors. Our method reaches the state of the art on the TUM RGB-D dataset and outperforms it on KITTI and ConsInv datasets.


title: Reducing climate risk in energy system planning: a posteriori time series aggregation for models with storage

abstract: The growth in variable renewables such as solar and wind is increasing the impact of climate uncertainty in energy system planning. Addressing this ideally requires high-resolution time series spanning at least a few decades. However, solving capacity expansion planning models across such datasets often requires too much computing time or memory. To reduce computational cost, users often employ time series aggregation to compress demand and weather time series into a smaller number of time steps. Methods are usually a priori, employing information about the input time series only. Recent studies highlight the limitations of this approach, since reducing statistical error metrics on input time series does not in general lead to more accurate model outputs. Furthermore, many aggregation schemes are unsuitable for models with storage since they distort chronology. In this paper, we introduce a posteriori time series aggregation schemes for models with storage. Our methods adapt to the underlying energy system model; aggregation may differ in systems with different technologies or topologies even with the same time series inputs. Furthermore, they preserve chronology and hence allow modelling of storage technologies. We investigate a number of approaches. We find that a posteriori methods can perform better than a priori ones, primarily through a systematic identification and preservation of relevant extreme events. We hope that these tools render long demand and weather time series more manageable in capacity expansion planning studies. We make our models, data, and code publicly available.


title: Production of activated and Graphene-Like Carbon Materials from rice husk

abstract: The main research efforts are aimed at finding inexpensive materials that can be converted into nanostructured carbon-containing materials, such as activated carbon and graphene. Rice husks are one such material, especially in developing countries, where more than 95% of Rice husks are produced worldwide. Although numerous studies have been conducted on the production of activated carbon and graphene structures from Rice husk, the existing scientific information is still widely scattered in the literature. Therefore, this review article provides extensive information on made by various researchers, including the Institute of combustion problems (Almaty, Kazakhstan), regarding the production of nanostructured carbon-containing materials from rice husk and its adsorption characteristics. The properties and pre-treatment of rice husk in relation to the production of activated carbon and multilayer graphene are discussed. The activation of rice husk by physical and chemical methods under different conditions is considered.


title: MGNNI: Multiscale Graph Neural Networks with Implicit Layers

abstract: Recently, implicit graph neural networks (GNNs) have been proposed to capture long-range dependencies in underlying graphs. In this paper, we introduce and justify two weaknesses of implicit GNNs: the constrained expressiveness due to their limited effective range for capturing long-range dependencies, and their lack of ability to capture multiscale information on graphs at multiple resolutions. To show the limited effective range of previous implicit GNNs, We first provide a theoretical analysis and point out the intrinsic relationship between the effective range and the convergence of iterative equations used in these models. To mitigate the mentioned weaknesses, we propose a multiscale graph neural network with implicit layers (MGNNI) which is able to model multiscale structures on graphs and has an expanded effective range for capturing long-range dependencies. We conduct comprehensive experiments for both node classification and graph classification to show that MGNNI outperforms representative baselines and has a better ability for multiscale modeling and capturing of long-range dependencies.


title: Dispersive $\pi\pi\rightarrow K\bar K$ amplitude and giant CP violation in B to three light-meson decays at LHCb

abstract: The LHCb collaboration has recently reported the largest CP violation effect from a single amplitude, as well as other giant CP asymmetries in several $B$-meson decays into three charmless light mesons. It is also claimed that this is predominantly due to $\pi\pi\rightarrow K\bar K$ rescattering in the final state, particularly in the 1 to 1.5 GeV region. In these analyses the $\pi\pi\rightarrow K\bar K$ amplitude is by default estimated from the $\pi\pi$ elastic scattering amplitude and does not describe the existing $\pi\pi\rightarrow K\bar K$ scattering data. Here we show how the recent model-independent dispersive analysis of $\pi\pi\rightarrow K\bar K$ data can be easily implemented in the LHCb formalism. This leads to a more accurate description of the asymmetry, while being consistent with the measured scattering amplitude and confirming the prominent role of hadronic final state interactions, paving the way for more elaborated analyses.


title: A Simple and Strong Baseline for End-to-End Neural RST-style Discourse Parsing

abstract: To promote and further develop RST-style discourse parsing models, we need a strong baseline that can be regarded as a reference for reporting reliable experimental results. This paper explores a strong baseline by integrating existing simple parsing strategies, top-down and bottom-up, with various transformer-based pre-trained language models. The experimental results obtained from two benchmark datasets demonstrate that the parsing performance strongly relies on the pretrained language models rather than the parsing strategies. In particular, the bottom-up parser achieves large performance gains compared to the current best parser when employing DeBERTa. We further reveal that language models with a span-masking scheme especially boost the parsing performance through our analysis within intra- and multi-sentential parsing, and nuclearity prediction.


title: HUDD: A tool to debug DNNs for safety analysis

abstract: We present HUDD, a tool that supports safety analysis practices for systems enabled by Deep Neural Networks (DNNs) by automatically identifying the root causes for DNN errors and retraining the DNN. HUDD stands for Heatmap-based Unsupervised Debugging of DNNs, it automatically clusters error-inducing images whose results are due to common subsets of DNN neurons. The intent is for the generated clusters to group error-inducing images having common characteristics, that is, having a common root cause. HUDD identifies root causes by applying a clustering algorithm to matrices (i.e., heatmaps) capturing the relevance of every DNN neuron on the DNN outcome. Also, HUDD retrains DNNs with images that are automatically selected based on their relatedness to the identified image clusters. Our empirical evaluation with DNNs from the automotive domain have shown that HUDD automatically identifies all the distinct root causes of DNN errors, thus supporting safety analysis. Also, our retraining approach has shown to be more effective at improving DNN accuracy than existing approaches. A demo video of HUDD is available at https://youtu.be/drjVakP7jdU.


title: Vehicle Risk Assessment and Control for Lane-Keeping and Collision Avoidance in Urban and Highway Driving Scenarios

abstract: This article examines a symbolic numerical approach to optimize a vehicle's track for autonomous driving and collision avoidance. The new approach uses the classical cost function definition incorporating the essential aspects of the dynamic state of the vehicle as position, orientation, time sampling, and constraints on slip angles of tires. The optimization processes minimize the cost function and simultaneously determine the optimal track by varying steering and breaking amplitudes. The current velocity of the vehicle is limited to a maximal velocity, thus, allowing a stable search of the optimal track. The parametric definition of obstacles generates a flexible environment for low and high speed simulations. The minimal number of influential optimization variables guarantees a stable and direct generation of optimal results. By the current new approach to control a vehicle on an optimal track, we are able to autonomously move the vehicle on an arbitrary track approximated by low order polynomials. The optimization approach is also able to deal with a variety of different obstacles and the corresponding optimal smooth obstacle path. The computations demonstrate the effective control of a four wheel vehicle in normal operation and exceptional obstacle avoidance with continuously differentiable obstacle avoidance tracks. Simulation tests are done using vehicle's velocities of 3m/s, 6m/s, 7.6m/s, 10m/s, 12m/s, and 18m/s. At higher vehicle's velocities, a mathematical-only approach is not sufficient and a mechanical intervention for tires is needed as a complimentary part to control the slip angle. The results shows that the cost function reached a considerably high average convergence-to-zero rate success in most of the tested scenarios.


title: Particle Injection and Nonthermal Particle Acceleration in Relativistic Magnetic Reconnection

abstract: Magnetic reconnection in the relativistic regime has been proposed as an important process for the efficient production of nonthermal particles and high-energy emissions. Using fully kinetic particle-in-cell simulations, we investigate how guide-field strength and domain size affect characteristic spectral features and acceleration processes. We study two stages of acceleration: energization up until the injection energy $\gamma_{\rm inj}$ and further acceleration that generates a power-law spectrum. Stronger guide fields increase the power-law index and $\gamma_{\rm inj}$, which suppresses acceleration efficiency. These quantities seemingly converge with increasing domain size, suggesting that our findings can be extended to large-scale systems. We find that three distinct mechanisms contribute to acceleration during injection: particle streaming along the parallel electric field, Fermi reflection, and the pickup process. Fermi and pickup processes, related to the electric field perpendicular to the magnetic field, govern the injection for weak guide fields and larger domains. Meanwhile, parallel electric fields are important for injection in the strong guide field regime. In the post-injection stage, we find that perpendicular electric fields dominate particle acceleration in the weak guide field regime, whereas parallel electric fields control acceleration for strong guide fields. These findings will help explain the nonthermal acceleration and emissions in high-energy astrophysics, including black hole jets and pulsar wind nebulae.


title: The Influence of Multiple Classes on Learning Online Classifiers from Imbalanced and Concept Drifting Data Streams

abstract: This work is aimed at the experimental studying the influence of local data characteristics and drifts on the difficulties of learning various online classifiers from multi-class imbalanced data streams. Firstly we present a categorization of these data factors and drifts in the context of imbalanced streams, then we introduce the generators of synthetic streams that model these factors and drifts. The results of many experiments with synthetically generated data streams have shown a much greater role of the overlapping between many minority classes (the type of borderline examples) than for streams with one minority class. The presence of rare examples in the stream is the most difficult single factor. The local drift of splitting minority classes is the third influential factor. Unlike binary streams, the specialized UOB and OOB classifiers perform well enough for even high imbalance ratios. The most challenging for all classifiers are complex scenarios integrating the drifts of the identified factors simultaneously, which worsen the evaluation measures in the case of a several minority classes stronger than for binary ones. This is an extended version of the short paper presented at LIDTA'2022 workshop at ECMLPKDD2022.


title: MIXER: Multiattribute, Multiway Fusion of Uncertain Pairwise Affinities

abstract: We present a multiway fusion algorithm capable of directly processing uncertain pairwise affinities. In contrast to existing works that require initial pairwise associations, our MIXER algorithm improves accuracy by leveraging the additional information provided by pairwise affinities. Our main contribution is a multiway fusion formulation that is particularly suited to processing non-binary affinities and a novel continuous relaxation whose solutions are guaranteed to be binary, thus avoiding the typical, but potentially problematic, solution binarization steps that may cause infeasibility. A crucial insight of our formulation is that it allows for three modes of association, ranging from non-match, undecided, and match. Exploiting this insight allows fusion to be delayed for some data pairs until more information is available, which is an effective feature for fusion of data with multiple attributes/information sources. We evaluate MIXER on typical synthetic data and benchmark datasets and show increased accuracy against the state of the art in multiway matching, especially in noisy regimes with low observation redundancy. Additionally, we collect RGB data of cars in a parking lot to demonstrate MIXER's ability to fuse data having multiple attributes (color, visual appearance, and bounding box). On this challenging dataset, MIXER achieves 74% F1 accuracy and is 49x faster than the next best algorithm, which has 42% accuracy. Open source code is available at https://github.com/mit-acl/mixer.


title: A Nearly Optimal Size Coreset Algorithm with Nearly Linear Time

abstract: A coreset is a point set containing information about geometric properties of a larger point set. A series of previous works show that in many machine learning problems, especially in clustering problems, coreset could be very useful to build efficient algorithms. Two main measures of an coreset construction algorithm's performance are the running time of the algorithm and the size of the coreset output by the algorithm. In this paper we study the construction of coresets for the $(k,z)$-clustering problem, which is a generalization of $k$-means and $k$-median problem. By properly designing a sketching-based distance estimation data structure, we propose faster algorithms that construct coresets with matching size of the state-of-the-art results.


title: PAR: Political Actor Representation Learning with Social Context and Expert Knowledge

abstract: Modeling the ideological perspectives of political actors is an essential task in computational political science with applications in many downstream tasks. Existing approaches are generally limited to textual data and voting records, while they neglect the rich social context and valuable expert knowledge for holistic ideological analysis. In this paper, we propose \textbf{PAR}, a \textbf{P}olitical \textbf{A}ctor \textbf{R}epresentation learning framework that jointly leverages social context and expert knowledge. Specifically, we retrieve and extract factual statements about legislators to leverage social context information. We then construct a heterogeneous information network to incorporate social context and use relational graph neural networks to learn legislator representations. Finally, we train PAR with three objectives to align representation learning with expert knowledge, model ideological stance consistency, and simulate the echo chamber phenomenon. Extensive experiments demonstrate that PAR is better at augmenting political text understanding and successfully advances the state-of-the-art in political perspective detection and roll call vote prediction. Further analysis proves that PAR learns representations that reflect the political reality and provide new insights into political behavior.


title: Data-Efficient Augmentation for Training Neural Networks

abstract: Data augmentation is essential to achieve state-of-the-art performance in many deep learning applications. However, the most effective augmentation techniques become computationally prohibitive for even medium-sized datasets. To address this, we propose a rigorous technique to select subsets of data points that when augmented, closely capture the training dynamics of full data augmentation. We first show that data augmentation, modeled as additive perturbations, improves learning and generalization by relatively enlarging and perturbing the smaller singular values of the network Jacobian, while preserving its prominent directions. This prevents overfitting and enhances learning the harder to learn information. Then, we propose a framework to iteratively extract small subsets of training data that when augmented, closely capture the alignment of the fully augmented Jacobian with labels/residuals. We prove that stochastic gradient descent applied to the augmented subsets found by our approach has similar training dynamics to that of fully augmented data. Our experiments demonstrate that our method achieves 6.3x speedup on CIFAR10 and 2.2x speedup on SVHN, and outperforms the baselines by up to 10% across various subset sizes. Similarly, on TinyImageNet and ImageNet, our method beats the baselines by up to 8%, while achieving up to 3.3x speedup across various subset sizes. Finally, training on and augmenting 50% subsets using our method on a version of CIFAR10 corrupted with label noise even outperforms using the full dataset. Our code is available at: https://github.com/tianyu139/data-efficient-augmentation


title: Stochastic modeling of physical drag coefficient -- its impact on orbit prediction and space traffic management

abstract: Ambitious satellite constellation projects by commercial entities and the ease of access to space in recent times have led to a dramatic proliferation of low-Earth space traffic. It jeopardizes space safety and long-term sustainability, necessitating better space traffic management (STM). Correct modeling of uncertainties in force models and orbital states, among other things, is an essential part of STM. For objects in the low-Earth orbit (LEO) region, the uncertainty in the orbital dynamics mainly emanate from limited knowledge of the atmospheric drag-related parameters and variables. In this paper, which extends the work by Paul et al. [2021], we develop a feed-forward deep neural network model for the prediction of the satellite drag coefficient for the full range of satellite attitude (i.e., satellite pitch $\in$ ($-90^0$, $+90^0$) and satellite yaw $\in$ ($0^0$, $+360^0$)). The model simultaneously predicts the mean and the standard deviation and is well-calibrated. We use numerically simulated physical drag coefficient data for training our neural network. The numerical simulations are carried out using the test particle Monte Carlo method using the diffuse reflection with incomplete accommodation gas-surface interaction model. Modeling is carried out for the well-known CHAllenging Minisatellite Payload (CHAMP) satellite. Finally, we use the Monte Carlo approach to propagate CHAMP over a three-day period under various modeling scenarios to investigate the distribution of radial, in-track, and cross-track orbital errors caused by drag coefficient uncertainty.


title: Classification of Hopf superalgebra structures on Drinfeld super Yangians

abstract: We construct a minimalistic presentation of Drinfeld super Yangians in the case of special linear superalgebra associated with an arbitrary Dynkin diagram. This gives us a possibility to introduce Hopf superalgebra structure on Drinfeld super Yangians. Using complete Weyl group we classify Drinfeld super Yangians endowed with mentioned Hopf superalgebra structures. Also it is constructed an isomorphism between completions of Drinfeld super Yangians and quantum loop superalgebras.


title: Simultaneous Bright- and Dark-Field X-ray Microscopy at X-ray Free Electron Lasers

abstract: The structures, strain fields, and defect distributions in solid materials underlie the mechanical and physical properties across numerous applications. Many modern microstructural microscopy tools characterize crystal grains, domains and defects required to map lattice distortions or deformation, but are limited to studies of the (near) surface. Generally speaking, such tools cannot probe the structural dynamics in a way that is representative of bulk behavior. Synchrotron X-ray diffraction based imaging has long mapped the deeply embedded structural elements, and with enhanced resolution, Dark Field X-ray Microscopy (DFXM) can now map those features with the requisite nm-resolution. However, these techniques still suffer from the required integration times due to limitations from the source and optics. This work extends DFXM to X-ray free electron lasers, showing how the $10^{12}$ photons per pulse available at these sources offer structural characterization down to 100 fs resolution (orders of magnitude faster than current synchrotron images). We introduce the XFEL DFXM setup with simultaneous bright field microscopy to probe density changes within the same volume. This work presents a comprehensive guide to the multi-modal ultrafast high-resolution X-ray microscope that we constructed and tested at two XFELs, and shows initial data demonstrating two timing strategies to study associated reversible or irreversible lattice dynamics.


title: Active Learning with Neural Networks: Insights from Nonparametric Statistics

abstract: Deep neural networks have great representation power, but typically require large numbers of training examples. This motivates deep active learning methods that can significantly reduce the amount of labeled training data. Empirical successes of deep active learning have been recently reported in the literature, however, rigorous label complexity guarantees of deep active learning have remained elusive. This constitutes a significant gap between theory and practice. This paper tackles this gap by providing the first near-optimal label complexity guarantees for deep active learning. The key insight is to study deep active learning from the nonparametric classification perspective. Under standard low noise conditions, we show that active learning with neural networks can provably achieve the minimax label complexity, up to disagreement coefficient and other logarithmic terms. When equipped with an abstention option, we further develop an efficient deep active learning algorithm that achieves $\mathsf{polylog}(\frac{1}{\epsilon})$ label complexity, without any low noise assumptions. We also provide extensions of our results beyond the commonly studied Sobolev/H\"older spaces and develop label complexity guarantees for learning in Radon $\mathsf{BV}^2$ spaces, which have recently been proposed as natural function spaces associated with neural networks.


title: Moire Potential, Lattice Relaxation and Layer Polarization in Marginally Twisted MoS2 Bilayers

abstract: Artificially twisted heterostructures of semiconducting transition metal dichalcogenides (TMDs) offer unprecedented control over their electronic and optical properties via the spatial modulation of interlayer interactions and structural reconstruction. Here we study twisted MoS2 bilayers in a wide range of twist angles near 0{\deg} using Scanning Tunneling Microscopy/Spectroscopy. We investigate the twist angle-dependence of the moir\'e pattern which is dominated by lattice reconstruction for small angles (<2{\deg}) leading to large triangular domains with rhombohedral stacking. Local spectroscopy measurements reveal a large moir\'e-potential strength of 100-200 meV for angles <3{\deg}. In reconstructed regions we see a bias-dependent asymmetry between neighboring triangular domains which we relate to the vertical polarization which is intrinsic to rhombohedral stacked TMDs. This viewpoint is urther supported by spectroscopy maps and ambient Piezoresponse measurements. Our results provide a microscopic perspective on this new class of interfacial ferroelectrics and can offer clues for designing novel heterostructures which harness this effect.


title: Metasurface Smart Glass for Object Recognition

abstract: Recent years have seen a considerable surge of research on developing heuristic approaches to realize analog computing using physical waves. Among these, neuromorphic computing using light waves is envisioned to feature performance metrics such as computational speed and energy efficiency exceeding those of conventional digital techniques by many orders of magnitude. Yet, neuromorphic computing based on photonics remains a challenge due to the difficulty of training and manufacturing sophisticated photonic structures to support neural networks with adequate expressive power. Here, we realize a diffractive optical neural network (ONN) based on metasurfaces that can recognize objects by directly processing light waves scattered from the objects. Metasurfaces composed of a two-dimensional array of millions of meta-units can realize precise control of optical wavefront with subwavelength resolution; thus, when used as constitutive layers of an ONN, they can provide exceptionally high expressive power. We experimentally demonstrate ONNs based on single-layered metasurfaces that modulate the phase and polarization over optical wavefront for recognizing optically coherent binary objects, including hand-written digits and English alphabetic letters. We further demonstrate, in simulation, ONNs based on metasurface doublets for human facial verification. The advantageous traits of metasurface-based ONNs, including ultra-compact form factors, zero power consumption, ultra-fast and parallel data processing capabilities, and physics-guaranteed data security, make them suitable as "edge" perception devices that can transform the future of image collection and analysis.


title: A Proof of the $(n,k,t)$ Conjectures

abstract: An $(n,k,t)$-graph is a graph on $n$ vertices in which every set of $k$ vertices contain a clique on $t$ vertices. Tur\'an's Theorem (complemented) states that the unique minimum $(n,k,2)$-graph is a disjoint union of cliques. We prove that minimum $(n,k,t)$-graphs are always disjoint unions of cliques for any $t$ (despite nonuniqueness of extremal examples), thereby generalizing Tur\'an's Theorem and confirming two conjectures of Hoffman et al.


title: Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerability

abstract: Sketching is one of the most fundamental tools in large-scale machine learning. It enables runtime and memory saving via randomly compressing the original large problem into lower dimensions. In this paper, we propose a novel sketching scheme for the first order method in large-scale distributed learning setting, such that the communication costs between distributed agents are saved while the convergence of the algorithms is still guaranteed. Given gradient information in a high dimension $d$, the agent passes the compressed information processed by a sketching matrix $R\in \mathbb{R}^{s\times d}$ with $s\ll d$, and the receiver de-compressed via the de-sketching matrix $R^\top$ to ``recover'' the information in original dimension. Using such a framework, we develop algorithms for federated learning with lower communication costs. However, such random sketching does not protect the privacy of local data directly. We show that the gradient leakage problem still exists after applying the sketching technique by presenting a specific gradient attack method. As a remedy, we prove rigorously that the algorithm will be differentially private by adding additional random noises in gradient information, which results in a both communication-efficient and differentially private first order approach for federated learning tasks. Our sketching scheme can be further generalized to other learning settings and might be of independent interest itself.


title: An Efficient and Decentralized Blockchain-based Commercial Alternative (Full Version)

abstract: While online interactions and exchanges have grown exponentially over the past decade, most commercial infrastructures still operate through centralized protocols, and their success essentially depends on trust between different economic actors. Digital advances such as blockchain technology has led to a massive wave of \textit{Decentralized Ledger Technology} (\textit{DLT}) initiatives, protocols and solutions. This advance makes it possible to implement trustless systems in the real world, which, combined with appropriate economic and participatory incentives, would foster the proper functioning and drive the adoption of a decentralized platform among different actors. This paper describes an alternative to current commercial structures and networks by introducing \textit{Lyzis Labs}, which is is an incentive-driven and democratic protocol designed to support a decentralized online marketplace, based on blockchain technology. The proposal, \textit{Lyzis Marketplace}, allows to connect two or more people in a decentralized and secure way without having to rely on a \textit{Trusted Third Party} (\textit{TTP}) in order to perform physical asset exchanges while mainly providing transparent and fully protected data storage. This approach can potentially lead to the creation of a permissionless, efficient, secure and transparent business environment where each user can gain purchasing and decision-making power by supporting the collective welfare while following their personal interests during their various interactions on the network.


title: Further clarification on quasinormal modes/circular null geodesics correspondence

abstract: The well-known duality between quasinormal modes of any stationary, spherically symmetric and asymptotically flat or de Sitter black hole and parameters of the circular null geodesic was initially claimed for gravitational and test field perturbations. According to this duality the real and imaginary parts of the $\ell \gg n$ quasinormal mode (where $\ell$ and $n$ are multipole and overtone numbers respectively) are multiples of the frequency and instability timescale of the circular null geodesics respectively. Later it was shown that the duality is guaranteed only for test fields and may be broken for gravitational and other non-minimally coupled fields. Here, we farther specify the duality and prove that even when the duality is guaranteed it may not represent the full spectrum of the $\ell \gg n$ modes, missing the quasinormal frequencies which cannot be found by the standard WKB method. In particular we show that this always happens for an arbitrary asymptotically de Sitter black holes and further argue that, in general, this might be related to sensitivity of the quasinormal spectrum to geometry deformations near the boundaries.


title: Beyond the Surface: Investigating Malicious CVE Proof of Concept Exploits on GitHub

abstract: Exploit proof-of-concepts (PoCs) for known vulnerabilities are widely shared in the security community. They help security analysts to learn from each other and they facilitate security assessments and red teaming tasks. In the recent years, PoCs have been widely distributed, e.g., via dedicated websites and platforms, and public code repositories such as GitHub. However, there is no guarantee that PoCs in public code repositories come from trustworthy sources or even that they do what they are supposed to do. In this work we investigate GitHub-hosted PoCs for known vulnerabilities discovered in 2017--2021. We discovered that not all PoCs are trustworthy. Some proof-of-concepts are malicious, e.g., they attempt to exfiltrate data from the system they are being run on, or they try to install malware on this system, and in some cases they have hard-coded reverse shell listener. To measure the prevalence of this threat, we propose an approach to detecting malicious PoCs. Our approach relies on the maliciousness symptoms we have observed in our PoC dataset: calls to malicious IP addresses, encoded malicious code, and included Trojanized binaries. With this approach, we have discovered 899 malicious repositories out of 47,285 repositories that have been downloaded and checked (i.e., 1.9% of the studied repositories have indicators of malicious intent). This figure shows a worrying prevalence of dangerous malicious PoCs among the exploit code distributed on GitHub.


title: Improving the Intra-class Long-tail in 3D Detection via Rare Example Mining

abstract: Continued improvements in deep learning architectures have steadily advanced the overall performance of 3D object detectors to levels on par with humans for certain tasks and datasets, where the overall performance is mostly driven by common examples. However, even the best performing models suffer from the most naive mistakes when it comes to rare examples that do not appear frequently in the training data, such as vehicles with irregular geometries. Most studies in the long-tail literature focus on class-imbalanced classification problems with known imbalanced label counts per class, but they are not directly applicable to the intra-class long-tail examples in problems with large intra-class variations such as 3D object detection, where instances with the same class label can have drastically varied properties such as shapes and sizes. Other works propose to mitigate this problem using active learning based on the criteria of uncertainty, difficulty, or diversity. In this study, we identify a new conceptual dimension - rareness - to mine new data for improving the long-tail performance of models. We show that rareness, as opposed to difficulty, is the key to data-centric improvements for 3D detectors, since rareness is the result of a lack in data support while difficulty is related to the fundamental ambiguity in the problem. We propose a general and effective method to identify the rareness of objects based on density estimation in the feature space using flow models, and propose a principled cost-aware formulation for mining rare object tracks, which improves overall model performance, but more importantly - significantly improves the performance for rare objects (by 30.97\%


title: Variant Parallelism: Lightweight Deep Convolutional Models for Distributed Inference on IoT Devices

abstract: Two major techniques are commonly used to meet real-time inference limitations when distributing models across resource-constrained IoT devices: (1) model parallelism (MP) and (2) class parallelism (CP). In MP, transmitting bulky intermediate data (orders of magnitude larger than input) between devices imposes huge communication overhead. Although CP solves this problem, it has limitations on the number of sub-models. In addition, both solutions are fault intolerant, an issue when deployed on edge devices. We propose variant parallelism (VP), an ensemble-based deep learning distribution method where different variants of a main model are generated and can be deployed on separate machines. We design a family of lighter models around the original model, and train them simultaneously to improve accuracy over single models. Our experimental results on six common mid-sized object recognition datasets demonstrate that our models can have 5.8-7.1x fewer parameters, 4.3-31x fewer multiply-accumulations (MACs), and 2.5-13.2x less response time on atomic inputs compared to MobileNetV2 while achieving comparable or higher accuracy. Our technique easily generates several variants of the base architecture. Each variant returns only 2k outputs 1 <= k <= (#classes/2), representing Top-k classes, instead of tons of floating point values required in MP. Since each variant provides a full-class prediction, our approach maintains higher availability compared with MP and CP in presence of failure.


title: Does a larger neural network mean greater information transmission efficiency?

abstract: Realistic modeling of brain involves large number of neurons. The important question is how this size affects transmission efficiency? Here, these issue is studied in terms of Shannon's Theory. Mutual Information between input and output signals for simple class of networks with an increasing number of neurons is analyzed theoretically and numerically. Levy-Baxter neural model is applied. It turned out, that for these networks the Mutual Information converges, with increasing size, asymptotically very slowly to saturation level. This suggests that from certain level, the increase of neurons number does not imply significant increase in transmission efficiency, contributes rather to reliability.


title: Memory-Efficient Differentiable Programming for Quantum Optimal Control of Discrete Lattices

abstract: Quantum optimal control problems are typically solved by gradient-based algorithms such as GRAPE, which suffer from exponential growth in storage with increasing number of qubits and linear growth in memory requirements with increasing number of time steps. Employing QOC for discrete lattices reveals that these memory requirements are a barrier for simulating large models or long time spans. We employ a nonstandard differentiable programming approach that significantly reduces the memory requirements at the cost of a reasonable amount of recomputation. The approach exploits invertibility properties of the unitary matrices to reverse the computation during back-propagation. We utilize QOC software written in the differentiable programming framework JAX that implements this approach, and demonstrate its effectiveness for lattice gauge theory.


title: Self-Tuning Transmitter for Quantum Key Distribution Using Machine Intelligence

abstract: The development and performance of quantum technologies heavily relies on the properties of the quantum states, which often require careful optimization of the driving conditions of all underlying components. In quantum key distribution (QKD), optical injection locking (OIL) of pulsed lasers has recently been shown as a promising technique to realize high-speed quantum transmitters with efficient system design. However, due to the complex underlying laser dynamics, tuning such laser system is both a challenging and time-consuming task. Here, we experimentally demonstrate an OIL-based QKD transmitter that can be automatically tuned to its optimum operating state by employing a genetic algorithm. Starting with minimal knowledge of the laser operating parameters, the phase coherence and the quantum bit error rate of the system are optimized autonomously to a level matching the state of the art.


title: Suppression of scattering from slow to fast subsystems and application to resonantly Floquet-driven impurities in strongly interacting systems

abstract: We study solutions to the Lippmann-Schwinger equation in systems where a slow subsystem is coupled to a fast subsystem via an impurity. Such situations appear when a high-frequency Floquet-driven impurity is introduced into a low-energy system, but the driving frequency is at resonance with a high-energy band. In contrast to the case of resonant bulk driving, where the particles in the low-energy system are excited into the high-energy band, we surprisingly find that these excitations are suppressed for resonantly driven impurities. Still, the transmission through the impurity is strongly affected by the presence of the high-energy band in a universal way that does not depend on the details of the high-energy band. We apply our general result to two examples and show the suppression of excitations from the low-energy band into the high-energy band: a) bound pairs in a Fermi-Hubbard chain scattering at a driven impurity, which is at resonance with the Hubbard interaction and b) particles in a deep optical lattice described by the tight-binding approximation, which scatter at a driven impurity, whose driving frequency equals the band gap between the two lowest energy bands.


title: Study on electro-optic noise in crystalline coatings toward future gravitational wave detectors

abstract: Thermal noise in high-reflectivity mirror coatings is a limiting factor in ground-based gravitational wave detectors. Reducing this coating thermal noise improves the sensitivity of detectors and enriches the scientific outcome of observing runs. Crystalline gallium arsenide and aluminum-alloyed gallium arsenide (referred to as AlGaAs) coatings are promising coating candidates for future upgrades of gravitational wave detectors because of their low coating thermal noise. However, AlGaAs-based crystalline coatings may be susceptible to an electro-optic noise induced by fluctuations in an electric field. We investigated the electro-optic effect in an AlGaAs coating by using a Fabry-Perot cavity, and concluded that the noise level is well below the sensitivity of current and planned gravitational-wave detectors.


title: Machine-Learning Love: classifying the equation of state of neutron stars with Transformers

abstract: The use of the Audio Spectrogram Transformer (AST) model for gravitational-wave data analysis is investigated. The AST machine-learning model is a convolution-free classifier that captures long-range global dependencies through a purely attention-based mechanism. In this paper a model is applied to a simulated dataset of inspiral gravitational wave signals from binary neutron star coalescences, built from five distinct, cold equations of state (EOS) of nuclear matter. From the analysis of the mass dependence of the tidal deformability parameter for each EOS class it is shown that the AST model achieves a promising performance in correctly classifying the EOS purely from the gravitational wave signals, especially when the component masses of the binary system are in the range $[1,1.5]M_{\odot}$. Furthermore, the generalization ability of the model is investigated by using gravitational-wave signals from a new EOS not used during the training of the model, achieving fairly satisfactory results. Overall, the results, obtained using the simplified setup of noise-free waveforms, show that the AST model, once trained, might allow for the instantaneous inference of the cold nuclear matter EOS directly from the inspiral gravitational-wave signals produced in binary neutron star coalescences.

categories: astro-ph.IM astro-ph.HE cs.CV cs.LG gr-qc


title: Comment: The Essential Role of Policy Evaluation for the 2020 Census Disclosure Avoidance System

abstract: In "Differential Perspectives: Epistemic Disconnects Surrounding the US Census Bureau's Use of Differential Privacy," boyd and Sarathy argue that empirical evaluations of the Census Disclosure Avoidance System (DAS), including our published analysis, failed to recognize how the benchmark data against which the 2020 DAS was evaluated is never a ground truth of population counts. In this commentary, we explain why policy evaluation, which was the main goal of our analysis, is still meaningful without access to a perfect ground truth. We also point out that our evaluation leveraged features specific to the decennial Census and redistricting data, such as block-level population invariance under swapping and voter file racial identification, better approximating a comparison with the ground truth. Lastly, we show that accurate statistical predictions of individual race based on the Bayesian Improved Surname Geocoding, while not a violation of differential privacy, substantially increases the disclosure risk of private information the Census Bureau sought to protect. We conclude by arguing that policy makers must confront a key trade-off between data utility and privacy protection, and an epistemic disconnect alone is insufficient to explain disagreements between policy choices.


title: Revisiting the Roles of "Text" in Text Games

abstract: Text games present opportunities for natural language understanding (NLU) methods to tackle reinforcement learning (RL) challenges. However, recent work has questioned the necessity of NLU by showing random text hashes could perform decently. In this paper, we pursue a fine-grained investigation into the roles of text in the face of different RL challenges, and reconcile that semantic and non-semantic language representations could be complementary rather than contrasting. Concretely, we propose a simple scheme to extract relevant contextual information into an approximate state hash as extra input for an RNN-based text agent. Such a lightweight plug-in achieves competitive performance with state-of-the-art text agents using advanced NLU techniques such as knowledge graph and passage retrieval, suggesting non-NLU methods might suffice to tackle the challenge of partial observability. However, if we remove RNN encoders and use approximate or even ground-truth state hash alone, the model performs miserably, which confirms the importance of semantic function approximation to tackle the challenge of combinatorially large observation and action spaces. Our findings and analysis provide new insights for designing better text game task setups and agents.


title: A Joint Modeling Approach for Clustering Mixed-Type Multivariate Longitudinal Data: Application to the CHILD Cohort Study

abstract: In epidemiological and clinical studies, identifying patients' phenotypes based on longitudinal profiles is critical to understanding the disease's developmental patterns. The current study was motivated by data from a Canadian birth cohort study, the CHILD Cohort Study. Our goal was to use multiple longitudinal respiratory traits to cluster the participants into subgroups with similar longitudinal respiratory profiles in order to identify clinically relevant disease phenotypes. To appropriately account for distinct structures and types of these longitudinal markers, we proposed a novel joint model for clustering mixed-type (continuous, discrete and categorical) multivariate longitudinal data. We also developed a Markov Chain Monte Carlo algorithm to estimate the posterior distribution of model parameters. Analysis of the CHILD Cohort data and simulated data were presented and discussed. Our study demonstrated that the proposed model serves as a useful analytical tool for clustering multivariate mixed-type longitudinal data. We developed an R package BCClong to implement the proposed model efficiently.


title: Simulating dirty bosons on a quantum computer

abstract: The physics of dirty bosons highlights the intriguing interplay of disorder and interactions in quantum systems, playing a central role in describing, for instance, ultracold gases in a random potential, doped quantum magnets, and amorphous superconductors. Here, we demonstrate how quantum computers can be used to elucidate the physics of dirty bosons in one and two dimensions. Specifically, we explore the disorder-induced delocalized-to-localized transition using adiabatic state preparation. In one dimension, the quantum circuits can be compressed to small enough depths for execution on currently available quantum computers. In two dimensions, the compression scheme is no longer applicable, thereby requiring the use of large-scale classical state vector simulations to emulate quantum computer performance. In addition, simulating interacting bosons via emulation of a noisy quantum computer allowed us to study the effect of quantum hardware noise on the physical properties of the simulated system. Our results suggest that scaling laws control how noise modifies observables versus its strength, the circuit depth, and the number of qubits. Moreover, we observe that noise impacts the delocalized and localized phases differently. A better understanding of how noise alters the genuine properties of the simulated system is essential for leveraging noisy intermediate-scale quantum devices for simulation of dirty bosons, and indeed for condensed matter systems in general.

categories: cond-mat.dis-nn cond-mat.quant-gas cond-mat.str-el quant-ph


title: Time-Varying Semidefinite Programming: Path Following a Burer-Monteiro Factorization

abstract: We present an online algorithm for time-varying semidefinite programs (TV-SDPs), based on the tracking of the solution trajectory of a low-rank matrix factorization, also known as the Burer-Monteiro factorization, in a path-following procedure. There, a predictor-corrector algorithm solves a sequence of linearized systems. This requires the introduction of a horizontal space constraint to ensure the local injectivity of the low-rank factorization. The method produces a sequence of approximate solutions for the original TV-SDP problem, for which we show that they stay close to the optimal solution path if properly initialized. Numerical experiments for a time-varying max-cut SDP relaxation demonstrate the computational advantages of the proposed method for tracking TV-SDPs in terms of runtime compared to off-the-shelf interior point methods.


title: RoS-KD: A Robust Stochastic Knowledge Distillation Approach for Noisy Medical Imaging

abstract: AI-powered Medical Imaging has recently achieved enormous attention due to its ability to provide fast-paced healthcare diagnoses. However, it usually suffers from a lack of high-quality datasets due to high annotation cost, inter-observer variability, human annotator error, and errors in computer-generated labels. Deep learning models trained on noisy labelled datasets are sensitive to the noise type and lead to less generalization on the unseen samples. To address this challenge, we propose a Robust Stochastic Knowledge Distillation (RoS-KD) framework which mimics the notion of learning a topic from multiple sources to ensure deterrence in learning noisy information. More specifically, RoS-KD learns a smooth, well-informed, and robust student manifold by distilling knowledge from multiple teachers trained on overlapping subsets of training data. Our extensive experiments on popular medical imaging classification tasks (cardiopulmonary disease and lesion classification) using real-world datasets, show the performance benefit of RoS-KD, its ability to distill knowledge from many popular large networks (ResNet-50, DenseNet-121, MobileNet-V2) in a comparatively small network, and its robustness to adversarial attacks (PGD, FSGM). More specifically, RoS-KD achieves >2% and >4% improvement on F1-score for lesion classification and cardiopulmonary disease classification tasks, respectively, when the underlying student is ResNet-18 against recent competitive knowledge distillation baseline. Additionally, on cardiopulmonary disease classification task, RoS-KD outperforms most of the SOTA baselines by ~1% gain in AUC score.


title: Semantic Video Moments Retrieval at Scale: A New Task and a Baseline

abstract: Motivated by the increasing need of saving search effort by obtaining relevant video clips instead of whole videos, we propose a new task, named Semantic Video Moments Retrieval at scale (SVMR), which aims at finding relevant videos coupled with re-localizing the video clips in them. Instead of a simple combination of video retrieval and video re-localization, our task is more challenging because of several essential aspects. In the 1st stage, our SVMR should take into account the fact that: 1) a positive candidate long video can contain plenty of irrelevant clips which are also semantically meaningful. 2) a long video can be positive to two totally different query clips if it contains clips relevant to two queries. The 2nd re-localization stage also exhibits different assumptions from existing video re-localization tasks, which hold an assumption that the reference video must contain semantically similar segments corresponding to the query clip. Instead, in our scenario, the retrieved long video can be a false positive one due to the inaccuracy of the first stage. To address these challenges, we propose our two-stage baseline solution of candidate videos retrieval followed by a novel attention-based query-reference semantically alignment framework to re-localize target clips from candidate videos. Furthermore, we build two more appropriate benchmark datasets from the off-the-shelf ActivityNet-1.3 and HACS for a thorough evaluation of SVMR models. Extensive experiments are carried out to show that our solution outperforms several reference solutions.


title: SOCIALMAPF: Optimal and Efficient Multi-Agent Path Finding with Strategic Agents for Social Navigation

abstract: We propose an extension to the MAPF formulation, called SocialMAPF, to account for private incentives of agents in constrained environments such as doorways, narrow hallways, and corridor intersections. SocialMAPF is able to, for instance, accurately reason about the urgent incentive of an agent rushing to the hospital over another agent's less urgent incentive of going to a grocery store; MAPF ignores such agent-specific incentives. Our proposed formulation addresses the open problem of optimal and efficient path planning for agents with private incentives. To solve SocialMAPF, we propose a new class of algorithms that use mechanism design during conflict resolution to simultaneously optimize agents' private local utilities and the global system objective. We perform an extensive array of experiments that show that optimal search-based MAPF techniques lead to collisions and increased time-to-goal in SocialMAPF compared to our proposed method using mechanism design. Furthermore, we empirically demonstrate that mechanism design results in models that maximizes agent utility and minimizes the overall time-to-goal of the entire system. We further showcase the capabilities of mechanism design-based planning by successfully deploying it in environments with static obstacles. To conclude, we briefly list several research directions using the SocialMAPF formulation, such as exploring motion planning in the continuous domain for agents with private incentives.


title: Video in 10 Bits: Few-Bit VideoQA for Efficiency and Privacy

abstract: In Video Question Answering (VideoQA), answering general questions about a video requires its visual information. Yet, video often contains redundant information irrelevant to the VideoQA task. For example, if the task is only to answer questions similar to "Is someone laughing in the video?", then all other information can be discarded. This paper investigates how many bits are really needed from the video in order to do VideoQA by introducing a novel Few-Bit VideoQA problem, where the goal is to accomplish VideoQA with few bits of video information (e.g., 10 bits). We propose a simple yet effective task-specific feature compression approach to solve this problem. Specifically, we insert a lightweight Feature Compression Module (FeatComp) into a VideoQA model which learns to extract task-specific tiny features as little as 10 bits, which are optimal for answering certain types of questions. We demonstrate more than 100,000-fold storage efficiency over MPEG4-encoded videos and 1,000-fold over regular floating point features, with just 2.0-6.6% absolute loss in accuracy, which is a surprising and novel finding. Finally, we analyze what the learned tiny features capture and demonstrate that they have eliminated most of the non-task-specific information, and introduce a Bit Activation Map to visualize what information is being stored. This decreases the privacy risk of data by providing k-anonymity and robustness to feature-inversion techniques, which can influence the machine learning community, allowing us to store data with privacy guarantees while still performing the task effectively.


title: The Effects of Partitioning Strategies on Energy Consumption in Distributed CNN Inference at The Edge

abstract: Nowadays, many AI applications utilizing resource-constrained edge devices (e.g., small mobile robots, tiny IoT devices, etc.) require Convolutional Neural Network (CNN) inference on a distributed system at the edge due to limited resources of a single edge device to accommodate and execute a large CNN. There are four main partitioning strategies that can be utilized to partition a large CNN model and perform distributed CNN inference on multiple devices at the edge. However, to the best of our knowledge, no research has been conducted to investigate how these four partitioning strategies affect the energy consumption per edge device. Such an investigation is important because it will reveal the potential of these partitioning strategies to be used effectively for reduction of the per-device energy consumption when a large CNN model is deployed for distributed inference at the edge. Therefore, in this paper, we investigate and compare the per-device energy consumption of CNN model inference at the edge on a distributed system when the four partitioning strategies are utilized. The goal of our investigation and comparison is to find out which partitioning strategies (and under what conditions) have the highest potential to decrease the energy consumption per edge device when CNN inference is performed at the edge on a distributed system.


title: Distributed Estimation and Inference for Semi-parametric Binary Response Models

abstract: The development of modern technology has enabled data collection of unprecedented size, which poses new challenges to many statistical estimation and inference problems. This paper studies the maximum score estimator of a semi-parametric binary choice model under a distributed computing environment without pre-specifying the noise distribution. An intuitive divide-and-conquer estimator is computationally expensive and restricted by a non-regular constraint on the number of machines, due to the highly non-smooth nature of the objective function. We propose (1) a one-shot divide-and-conquer estimator after smoothing the objective to relax the constraint, and (2) a multi-round estimator to completely remove the constraint via iterative smoothing. We specify an adaptive choice of kernel smoother with a sequentially shrinking bandwidth to achieve the superlinear improvement of the optimization error over the multiple iterations. The improved statistical accuracy per iteration is derived, and a quadratic convergence up to the optimal statistical error rate is established. We further provide two generalizations to handle the heterogeneity of datasets and high-dimensional problems where the parameter of interest is sparse.


title: mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors

abstract: The ability to estimate 3D human body pose and movement, also known as human pose estimation (HPE), enables many applications for home-based health monitoring, such as remote rehabilitation training. Several possible solutions have emerged using sensors ranging from RGB cameras, depth sensors, millimeter-Wave (mmWave) radars, and wearable inertial sensors. Despite previous efforts on datasets and benchmarks for HPE, few dataset exploits multiple modalities and focuses on home-based health monitoring. To bridge the gap, we present mRI, a multi-modal 3D human pose estimation dataset with mmWave, RGB-D, and Inertial Sensors. Our dataset consists of over 160k synchronized frames from 20 subjects performing rehabilitation exercises and supports the benchmarks of HPE and action detection. We perform extensive experiments using our dataset and delineate the strength of each modality. We hope that the release of mRI can catalyze the research in pose estimation, multi-modal learning, and action understanding, and more importantly facilitate the applications of home-based health monitoring.