title: Transformer for Partial Differential Equations' Operator Learning

abstract: Data-driven learning of partial differential equations' solution operators has recently emerged as a promising paradigm for approximating the underlying solutions. The solution operators are usually parameterized by deep learning models that are built upon problem-specific inductive biases. An example is a convolutional or a graph neural network that exploits the local grid structure where functions' values are sampled. The attention mechanism, on the other hand, provides a flexible way to implicitly exploit the patterns within inputs, and furthermore, relationship between arbitrary query locations and inputs. In this work, we present an attention-based framework for data-driven operator learning, which we term Operator Transformer (OFormer). Our framework is built upon self-attention, cross-attention, and a set of point-wise multilayer perceptrons (MLPs), and thus it makes few assumptions on the sampling pattern of the input function or query locations. We show that the proposed framework is competitive on standard benchmark problems and can flexibly be adapted to randomly sampled input.


title: Discriminative Dimensionality Reduction using Deep Neural Networks for Clustering of LIGO Data

abstract: In this paper, leveraging the capabilities of neural networks for modeling the non-linearities that exist in the data, we propose several models that can project data into a low dimensional, discriminative, and smooth manifold. The proposed models can transfer knowledge from the domain of known classes to a new domain where the classes are unknown. A clustering algorithm is further applied in the new domain to find potentially new classes from the pool of unlabeled data. The research problem and data for this paper originated from the Gravity Spy project which is a side project of Advanced Laser Interferometer Gravitational-wave Observatory (LIGO). The LIGO project aims at detecting cosmic gravitational waves using huge detectors. However non-cosmic, non-Gaussian disturbances known as "glitches", show up in gravitational-wave data of LIGO. This is undesirable as it creates problems for the gravitational wave detection process. Gravity Spy aids in glitch identification with the purpose of understanding their origin. Since new types of glitches appear over time, one of the objective of Gravity Spy is to create new glitch classes. Towards this task, we offer a methodology in this paper to accomplish this.


title: Diffusion of Community Fact-Checked Misinformation on Twitter

abstract: The spread of misinformation on social media is a pressing societal problem that platforms, policymakers, and researchers continue to grapple with. As a countermeasure, recent works have proposed to employ non-expert fact-checkers in the crowd to fact-check social media content. While experimental studies suggest that crowds might be able to accurately assess the veracity of social media content, an understanding of how crowd fact-checked (mis-)information spreads is missing. In this work, we empirically analyze the spread of misleading vs. not misleading community fact-checked posts on social media. For this purpose, we employ a dataset of community-created fact-checks from Twitter's Birdwatch pilot and map them to resharing cascades on Twitter. Different from earlier studies analyzing the spread of misinformation listed on third-party fact-checking websites (e.g., Snopes), we find that community fact-checked misinformation is less viral. Specifically, misleading posts are estimated to receive 36.62% fewer retweets than not misleading posts. A partial explanation may lie in differences in the fact-checking targets: community fact-checkers tend to fact-check posts from influential user accounts with many followers, while expert fact-checks tend to target posts that are shared by less influential users. We further find that there are significant differences in virality across different sub-types of misinformation (e.g., factual errors, missing context, manipulated media). Moreover, we conduct a user study to assess the perceived reliability of (real-world) community-created fact-checks. Here, we find that users, to a large extent, agree with community-created fact-checks. Altogether, our findings offer insights into how misleading vs. not misleading posts spread and highlight the crucial role of sample selection when studying misinformation on social media.


title: Global Normalization for Streaming Speech Recognition in a Modular Framework

abstract: We introduce the Globally Normalized Autoregressive Transducer (GNAT) for addressing the label bias problem in streaming speech recognition. Our solution admits a tractable exact computation of the denominator for the sequence-level normalization. Through theoretical and empirical results, we demonstrate that by switching to a globally normalized model, the word error rate gap between streaming and non-streaming speech-recognition models can be greatly reduced (by more than 50\% on the Librispeech dataset). This model is developed in a modular framework which encompasses all the common neural speech recognition models. The modularity of this framework enables controlled comparison of modelling choices and creation of new models.


title: Reinforcement Learning Approach for Mapping Applications to Dataflow-Based Coarse-Grained Reconfigurable Array

abstract: The Streaming Engine (SE) is a Coarse-Grained Reconfigurable Array which provides programming flexibility and high-performance with energy efficiency. An application program to be executed on the SE is represented as a combination of Synchronous Data Flow (SDF) graphs, where every instruction is represented as a node. Each node needs to be mapped to the right slot and array in the SE to ensure the correct execution of the program. This creates an optimization problem with a vast and sparse search space for which finding a mapping manually is impractical because it requires expertise and knowledge of the SE micro-architecture. In this work we propose a Reinforcement Learning framework with Global Graph Attention (GGA) module and output masking of invalid placements to find and optimize instruction schedules. We use Proximal Policy Optimization in order to train a model which places operations into the SE tiles based on a reward function that models the SE device and its constraints. The GGA module consists of a graph neural network and an attention module. The graph neural network creates embeddings of the SDFs and the attention block is used to model sequential operation placement. We show results on how certain workloads are mapped to the SE and the factors affecting mapping quality. We find that the addition of GGA, on average, finds 10% better instruction schedules in terms of total clock cycles taken and masking improves reward obtained by 20%.


title: Forward variable selection enables fast and accurate dynamic system identification with Karhunen-Lo\`eve decomposed Gaussian processes

abstract: A promising approach for scalable Gaussian processes (GPs) is the Karhunen-Lo\`eve (KL) decomposition, in which the GP kernel is represented by a set of basis functions which are the eigenfunctions of the kernel operator. Such decomposed kernels have the potential to be very fast, and do not depend on the selection of a reduced set of inducing points. However KL decompositions lead to high dimensionality, and variable selection becomes paramount. This paper reports a new method of forward variable selection, enabled by the ordered nature of the basis functions in the KL expansion of the Bayesian Smoothing Spline ANOVA kernel (BSS-ANOVA), coupled with fast Gibbs sampling in a fully Bayesian approach. It quickly and effectively limits the number of terms, yielding a method with competitive accuracies, training and inference times for tabular datasets of low feature set dimensionality. The inference speed and accuracy makes the method especially useful for dynamic systems identification, by modeling the dynamics in the tangent space as a static problem, then integrating the learned dynamics using a high-order scheme. The methods are demonstrated on two dynamic datasets: a `Susceptible, Infected, Recovered' (SIR) toy problem, with the transmissibility used as forcing function, along with the experimental `Cascaded Tanks' benchmark dataset. Comparisons on the static prediction of time derivatives are made with a random forest (RF), a residual neural network (ResNet), and the Orthogonal Additive Kernel (OAK) inducing points scalable GP, while for the timeseries prediction comparisons are made with LSTM and GRU recurrent neural networks (RNNs) along with the SINDy package.


title: Hadamard Tail from Initial Data on the Light Cone

abstract: Field perturbations of a curved background spacetime generally propagate not only at the speed of light but also at all smaller velocities. This so-called $Hadamard\,tail$ contribution to wave propagation is relevant in various settings, from classical self-force calculations to communication between quantum particle detectors. One method for calculating this tail contribution is by integrating the homogeneous wave equation using Characteristic Initial Data on the light cone. However, to the best of our knowledge, this method has never been implemented before except in flat or conformally-flat spacetimes, where null geodesics emanating from a point do not cross. In this work, we implement this method on the black hole toy model Pleba\'nski-Hacyan spacetime, $\mathbb{M}_2\times\mathbb{S}^2$. We obtain new results in this spacetime by calculating the Hadamard tail of a scalar field everywhere where it is defined (namely, in the maximal normal neighbourhood of an arbitrary point) and investigate how it varies for various values of the coupling constant. This serves as a proof-of-concept for the Characteristic Initial Data method on spacetimes where null geodesics emanating from a point $do$ cross.


title: Pass Evaluation in Women's Olympic Hockey

abstract: Passing during power plays in hockey is a crucial component to move one's team closer to scoring a goal. With the use of women's ice hockey event and tracking data from the elimination round games during the 2022 Winter Olympics, we evaluate passing and assess players' risk-reward behaviours in these high intensity moments. We develop a model for probabilistic passing that accounts for the order of arrival to a desired location and potential interceptions along the way. This model is based on a player-specific motion model and a puck motion model that determines how far each player can reach in the time it takes the puck to get to a target. In addition, we model the rink control for each team and the scoring probability of the offensive team. These models are then combined into novel metrics for quantifying where a pass should be made such that it would result in a high scoring opportunity or result in a high chance of maintaining possession. Finally, we use various metrics to evaluate passes made throughout the available power plays and compare them to the optimal options at that time. This can be used to identify players' risk-reward tendencies and can be used by coaches when selecting which players are best suited for a power play given the circumstances of the game.


title: SeedGNN: Graph Neural Networks for Supervised Seeded Graph Matching

abstract: There is a growing interest in designing Graph Neural Networks (GNNs) for seeded graph matching, which aims to match two unlabeled graphs using only topological information and a small set of seed nodes. However, most previous GNNs for this task use a semi-supervised approach, which requires a large number of seeds and cannot learn knowledge that is transferable to unseen graphs. In contrast, this paper proposes a new supervised approach that can learn from a training set how to match unseen graphs with only a few seeds. Our SeedGNN architecture incorporates several novel designs, inspired by theoretical studies of seeded graph matching: 1) it can learn to compute and use witness-like information from different hops, in a way that can be generalized to graphs of different sizes; 2) it can use easily-matched node-pairs as new seeds to improve the matching in subsequent layers. We evaluate SeedGNN on synthetic and real-world graphs and demonstrate significant performance improvements over both non-learning and learning algorithms in the existing literature. Furthermore, our experiments confirm that the knowledge learned by SeedGNN from training graphs can be generalized to test graphs of different sizes and categories.


title: Membership Inference Attack Using Self Influence Functions

abstract: Member inference (MI) attacks aim to determine if a specific data sample was used to train a machine learning model. Thus, MI is a major privacy threat to models trained on private sensitive data, such as medical records. In MI attacks one may consider the black-box settings, where the model's parameters and activations are hidden from the adversary, or the white-box case where they are available to the attacker. In this work, we focus on the latter and present a novel MI attack for it that employs influence functions, or more specifically the samples' self-influence scores, to perform the MI prediction. We evaluate our attack on CIFAR-10, CIFAR-100, and Tiny ImageNet datasets, using versatile architectures such as AlexNet, ResNet, and DenseNet. Our attack method achieves new state-of-the-art results for both training with and without data augmentations. Code is available at https://github.com/giladcohen/sif_mi_attack.


title: Sequential Nature of Recommender Systems Disrupts the Evaluation Process

abstract: Datasets are often generated in a sequential manner, where the previous samples and intermediate decisions or interventions affect subsequent samples. This is especially prominent in cases where there are significant human-AI interactions, such as in recommender systems. To characterize the importance of this relationship across samples, we propose to use adversarial attacks on popular evaluation processes. We present sequence-aware boosting attacks and provide a lower bound on the amount of extra information that can be exploited from a confidential test set solely based on the order of the observed data. We use real and synthetic data to test our methods and show that the evaluation process on the MovieLense-100k dataset can be affected by $\sim1\%$ which is important when considering the close competition. Codes are publicly available.


title: ANISE: Assembly-based Neural Implicit Surface rEconstruction

abstract: We present ANISE, a method that reconstructs a 3D~shape from partial observations (images or sparse point clouds) using a part-aware neural implicit shape representation. The shape is formulated as an assembly of neural implicit functions, each representing a different part instance. In contrast to previous approaches, the prediction of this representation proceeds in a coarse-to-fine manner. Our model first reconstructs a structural arrangement of the shape in the form of geometric transformations of its part instances. Conditioned on them, the model predicts part latent codes encoding their surface geometry. Reconstructions can be obtained in two ways: (i) by directly decoding the part latent codes to part implicit functions, then combining them into the final shape; or (ii) by using part latents to retrieve similar part instances in a part database and assembling them in a single shape. We demonstrate that, when performing reconstruction by decoding part representations into implicit functions, our method achieves state-of-the-art part-aware reconstruction results from both images and sparse point clouds.When reconstructing shapes by assembling parts retrieved from a dataset, our approach significantly outperforms traditional shape retrieval methods even when significantly restricting the database size. We present our results in well-known sparse point cloud reconstruction and single-view reconstruction benchmarks.


title: Looking at Creative ML Blindspots with a Sociological Lens

abstract: How can researchers from the creative ML/AI community and sociology of culture engage in fruitful collaboration? How do researchers from both fields think (differently) about creativity and the production of creative work? While the ML community considers creativity as a matter of technical expertise and acumen, social scientists have emphasized the role of embeddedness in cultural production. This perspective aims to bridge both disciplines and proposes a conceptual and methodological toolkit for collaboration. We provide a systematic review of recent research in both fields and offer three perspectives around which to structure interdisciplinary research on cultural production: people, processes, and products. We thereby provide necessary grounding work to support multidisciplinary researchers to navigate conceptual and methodological hurdles in their collaboration. Our research will be of interest to ML researchers and sociologists interested in creativity that aim to conduct innovative research by bridging both fields.


title: Learning with Stochastic Orders

abstract: Learning high-dimensional distributions is often done with explicit likelihood modeling or implicit modeling via minimizing integral probability metrics (IPMs). In this paper, we expand this learning paradigm to stochastic orders, namely, the convex or Choquet order between probability measures. Towards this end, exploiting the relation between convex orders and optimal transport, we introduce the Choquet-Toland distance between probability measures, that can be used as a drop-in replacement for IPMs. We also introduce the Variational Dominance Criterion (VDC) to learn probability measures with dominance constraints, that encode the desired stochastic order between the learned measure and a known baseline. We analyze both quantities and show that they suffer from the curse of dimensionality and propose surrogates via input convex maxout networks (ICMNs), that enjoy parametric rates. We provide a min-max framework for learning with stochastic orders and validate it experimentally on synthetic and high-dimensional image generation, with promising results. Finally, our ICMNs class of convex functions and its derived Rademacher Complexity are of independent interest beyond their application in convex orders.


title: Adversarial attacks and defenses in Speaker Recognition Systems: A survey

abstract: Speaker recognition has become very popular in many application scenarios, such as smart homes and smart assistants, due to ease of use for remote control and economic-friendly features. The rapid development of SRSs is inseparable from the advancement of machine learning, especially neural networks. However, previous work has shown that machine learning models are vulnerable to adversarial attacks in the image domain, which inspired researchers to explore adversarial attacks and defenses in Speaker Recognition Systems (SRS). Unfortunately, existing literature lacks a thorough review of this topic. In this paper, we fill this gap by performing a comprehensive survey on adversarial attacks and defenses in SRSs. We first introduce the basics of SRSs and concepts related to adversarial attacks. Then, we propose two sets of criteria to evaluate the performance of attack methods and defense methods in SRSs, respectively. After that, we provide taxonomies of existing attack methods and defense methods, and further review them by employing our proposed criteria. Finally, based on our review, we find some open issues and further specify a number of future directions to motivate the research of SRSs security.


title: coCartesian fibrations and homotopy colimits

abstract: The main objective of this paper is to show that the homotopy colimit of a diagram of quasi-categories and indexed by a small category is a localization of Lurie's higher Grothendieck construction of the diagram. We thereby generalize Thomason's classical result which states that the homotopy colimit of a diagram of categories has the homotopy type of (the classifying space of) the Grothendieck construction of the diagram of categories.


title: Statistical Inference of Constrained Stochastic Optimization via Sketched Sequential Quadratic Programming

abstract: We consider online statistical inference of constrained stochastic nonlinear optimization problems. We apply the Stochastic Sequential Quadratic Programming (StoSQP) method to solve these problems, which can be regarded as applying second-order Newton's method to the Karush-Kuhn-Tucker (KKT) conditions. In each iteration, the StoSQP method computes the Newton direction by solving a quadratic program, and then selects a proper adaptive stepsize $\bar{\alpha}_t$ to update the primal-dual iterate. To reduce dominant computational cost of the method, we inexactly solve the quadratic program in each iteration by employing an iterative sketching solver. Notably, the approximation error of the sketching solver need not vanish as iterations proceed, meaning that the per-iteration computational cost does not blow up. For the above StoSQP method, we show that under mild assumptions, the rescaled primal-dual sequence $1/\sqrt{\bar{\alpha}_t}\cdot (x_t - x^\star, \lambda_t - \lambda^\star)$ converges to a mean-zero Gaussian distribution with a nontrivial covariance matrix depending on the underlying sketching distribution. To perform inference in practice, we also analyze a plug-in covariance matrix estimator. We illustrate the asymptotic normality result of the method both on benchmark nonlinear problems in CUTEst test set and on linearly/nonlinearly constrained regression problems.


title: Efficient Adiabatic Rapid Passage in the Presence of Noise

abstract: Adiabatic Rapid Passage (ARP) is a powerful technique for efficient transfer of population between quantum states. In the lab, the efficiency of ARP is often limited by noise on either the energies of the states or the frequency of the driving field. We study ARP in the simple setting of a two-level system subject to sinusoidal fluctuations on the energy level separation by numerically solving the optical Bloch equations in the absence of damping. We investigate the dependence of the efficiency of population transfer on the frequency and amplitude of the perturbation, and find that it is predominantly affected by resonant coupling when the detuning matches the frequency of the noise. We present intuitive principles for when ARP becomes inefficient within this model, and provide a sufficient condition for the population transfer to be above an arbitrary threshold.


title: Safety Aware Changepoint Detection for Piecewise i.i.d. Bandits

abstract: In this paper, we consider the setting of piecewise i.i.d. bandits under a safety constraint. In this piecewise i.i.d. setting, there exists a finite number of changepoints where the mean of some or all arms change simultaneously. We introduce the safety constraint studied in \citet{wu2016conservative} to this setting such that at any round the cumulative reward is above a constant factor of the default action reward. We propose two actively adaptive algorithms for this setting that satisfy the safety constraint, detect changepoints, and restart without the knowledge of the number of changepoints or their locations. We provide regret bounds for our algorithms and show that the bounds are comparable to their counterparts from the safe bandit and piecewise i.i.d. bandit literature. We also provide the first matching lower bounds for this setting. Empirically, we show that our safety-aware algorithms perform similarly to the state-of-the-art actively adaptive algorithms that do not satisfy the safety constraint.


title: Banach fixed-point between SEM image and EBSD diffraction pattern from a cylindrically symmetric rotating crystal

abstract: The Kikuchi bands arise from Bragg diffraction of incoherent electrons scattered within a crystalline specimen and can be observed in both the transmission and reflection modes of scanning electron microscopy (SEM). Converging, rocking, or grazing incidence beams must be used to generate divergent electron sources to obtain the Kikuchi pattern. This paper report the observation of Kikuchi pattern from SEM images of an exceptional rotating crystal with continuous rotation in the local crystal direction and satisfying cylindrical symmetry, named a cylindrically symmetric rotating crystal. SEM images of cylindrically symmetric rotating crystals reflect the interactions between electrons and the sample in both the real- and momentum-space. Furthermore, we identify an unexpected mathematical relationship between the electron backscattered diffraction (EBSD) Kikuchi pattern matrix map and the SEM image of the present sample which can be rationalized as a concrete example of the Banach fixed-point theorems in the field of EBSD technique.


title: Suppression of Peierls-like, nesting-based instabilities in solids

abstract: The understanding of lattice instabilities is of vast importance in material science. The famous example is the Peierls instability of one-dimensional metals and for strongly-nested Fermi surfaces in two and three dimensions. Through an analysis of H and Li chains in band theory, we find that the Bloch wave nature of the wavefunctions, if involving strong k-dependent hybridization of oppositeparity atomic states, strongly suppresses susceptibility peaks and associated instabilities and is thus essential to consider in searching for materials with strong responses to external perturbations.


title: FedAvg with Fine Tuning: Local Updates Lead to Representation Learning

abstract: The Federated Averaging (FedAvg) algorithm, which consists of alternating between a few local stochastic gradient updates at client nodes, followed by a model averaging update at the server, is perhaps the most commonly used method in Federated Learning. Notwithstanding its simplicity, several empirical studies have illustrated that the output model of FedAvg, after a few fine-tuning steps, leads to a model that generalizes well to new unseen tasks. This surprising performance of such a simple method, however, is not fully understood from a theoretical point of view. In this paper, we formally investigate this phenomenon in the multi-task linear representation setting. We show that the reason behind generalizability of the FedAvg's output is its power in learning the common data representation among the clients' tasks, by leveraging the diversity among client data distributions via local updates. We formally establish the iteration complexity required by the clients for proving such result in the setting where the underlying shared representation is a linear map. To the best of our knowledge, this is the first such result for any setting. We also provide empirical evidence demonstrating FedAvg's representation learning ability in federated image classification with heterogeneous data.


title: A novel buckling pattern in periodically porous elastomers with applications to elastic wave regulations

abstract: This paper proposes a new metamaterial structure consisting of a periodically porous elastomer with pore coatings. This design enables us to engender finite deformation by a contactless load. As a case study, we apply thermal load to the pore coating and carry out a finite element analysis to probe instabilities and the associated phononic properties. It turns out that a novel buckling mode, preserving the nature of surface wrinkling in tubular structures, can be induced under a plane-strain setup, and a smaller size of the unit cell is attained compared to the counterpart of traditional buckled profile in soft porous elastomers. In particular, this buckling pattern is able to produce several bandgaps in different frequency ranges as the macroscopic mean strain increases. We further introduce a metallic core as local resonator, and the updated metamaterial allows a low-frequency bandgap, the bandgap width of which can be estimated by a simplified theoretical model. As more free parameters are involved in the structure, we perform a detailed parametric study to elucidate the influences of the modulus ratio between coating and matrix, the porosity, the core radius, and the macroscopic mean strain on the buckling initiation and the evolution of bandgap. Remarkably, a stiffer surface coating is prone to enhance the stability of the structure, which is contrary to existing results in film/substrate bilayers. It is expected that the current study could shed light on new insight into pattern formation and wave manipulation in porous elastomers.


title: On the equidistribution of closed geodesics and geodesic nets

abstract: We show that given a closed $n$-manifold $M$, for a generic set of Riemannian metrics $g$ on $M$ there exists a sequence of closed geodesics that are equidistributed in $M$ if $n=2$; and an equidistributed sequence of embedded stationary geodesic nets if $n=3$. One of the main tools that we use is the Weyl Law for the volume spectrum for $1$-cycles, proved by Liokumovich, Marques and Neves for $n=2$ and more recently by Guth and Liokumovich for $n=3$. We show that our proof of the equidistribution of geodesic nets can be generalized for any dimension $n\geq 2$ provided the Weyl Law for $1$-cycles in $n$-manifolds holds.


title: Multi-input model uncertainty analysis for long-range wind farm noise predictions

abstract: One of the major sources of uncertainty in predictions of wind farm noise (WFN) reflect parametric and model structure uncertainty. The model structure uncertainty is a systematic uncertainty, which relates to uncertainty about the appropriate mathematical structure of the models. Here we quantified the model structure uncertainty in predicting WFN arising from multi-input models, including nine ground impedance and four wind speed profile models. We used a numerical ray tracing sound propagation model for predicting the noise level at different receivers. We found that variations between different ground impedance models and wind speed profile models were significant sources of uncertainty, and that these sources contributed to predicted noise level differences in excess of 10 dBA at distances greater than 3.5 km. We also found that differences between atmospheric vertical wind speed profile models were the main source of uncertainty in predicting WFN at long-range distances. When predicting WFN, it is important to acknowledge variability associated with different models as this contributes to the uncertainty of the predicted values.


title: Image of Bonnor black dihole with a thin accretion disk and its polarization information

abstract: We have studied the image of Bonnor black dihole surrounded by a thin accretion disk where the electromagnetic emission is assumed to be dominated respectively by black body radiation and synchrotron radiation. Our results show that the intensity of Bonnor black dihole image increases with the magnetic parameter and the inclination angle in both radiation models. The image of Bonnor black dihole in the synchrotron radiation model is one order of magnitude brighter than that in the black body radiation model, but its intensity in the former decreases more rapidly with the radial coordinate. Especially, for the synchrotron radiation model, the intensity of the secondary image is stronger than that of the direct image at certain an inclination angle. We also present the polarization patterns for the images of Bonnor black dihole arising from the synchrotron radiation, which depend sharply on the magnetic parameter and inclination angle. Finally, we make a comparison between the polarimetric images of Bonnor black dihole and M87*. Our result further confirms that the image of black hole depends on the black hole's properties itself, the matter around black hole and the corresponding radiation occurred in the accretion disk.


title: FedFormer: Contextual Federation with Attention in Reinforcement Learning

abstract: A core issue in multi-agent federated reinforcement learning is defining how to aggregate insights from multiple agents. This is commonly done by taking the average of each participating agent's model weights into one common model (FedAvg). We instead propose FedFormer, a novel federation strategy that utilizes Transformer Attention to contextually aggregate embeddings from models originating from different learner agents. In so doing, we attentively weigh the contributions of other agents with respect to the current agent's environment and learned relationships, thus providing a more effective and efficient federation. We evaluate our methods on the Meta-World environment and find that our approach yields significant improvements over FedAvg and non-federated Soft Actor-Critic single-agent methods. Our results compared to Soft Actor-Critic show that FedFormer achieves higher episodic return while still abiding by the privacy constraints of federated learning. Finally, we also demonstrate improvements in effectiveness with increased agent pools across all methods in certain tasks. This is contrasted by FedAvg, which fails to make noticeable improvements when scaled.


title: Characterizing the robustness of Bayesian adaptive experimental designs to active learning bias

abstract: Bayesian adaptive experimental design is a form of active learning, which chooses samples to maximize the information they give about uncertain parameters. Prior work has shown that other forms of active learning can suffer from active learning bias, where unrepresentative sampling leads to inconsistent parameter estimates. We show that active learning bias can also afflict Bayesian adaptive experimental design, depending on model misspecification. We analyze the case of estimating a linear model, and show that worse misspecification implies more severe active learning bias. At the same time, model classes incorporating more "noise" - i.e., specifying higher inherent variance in observations - suffer less from active learning bias. Finally, we demonstrate empirically that insights from the linear model can predict the presence and degree of active learning bias in nonlinear contexts, namely in a (simulated) preference learning experiment.


title: Maximum Likelihood Training of Implicit Nonlinear Diffusion Models

abstract: Whereas diverse variations of diffusion models exist, extending the linear diffusion into a nonlinear diffusion process is investigated by very few works. The nonlinearity effect has been hardly understood, but intuitively, there would be promising diffusion patterns to efficiently train the generative distribution towards the data distribution. This paper introduces a data-adaptive nonlinear diffusion process for score-based diffusion models. The proposed Implicit Nonlinear Diffusion Model (INDM) learns by combining a normalizing flow and a diffusion process. Specifically, INDM implicitly constructs a nonlinear diffusion on the \textit{data space} by leveraging a linear diffusion on the \textit{latent space} through a flow network. This flow network is key to forming a nonlinear diffusion, as the nonlinearity depends on the flow network. This flexible nonlinearity improves the learning curve of INDM to nearly Maximum Likelihood Estimation (MLE) against the non-MLE curve of DDPM++, which turns out to be an inflexible version of INDM with the flow fixed as an identity mapping. Also, the discretization of INDM shows the sampling robustness. In experiments, INDM achieves the state-of-the-art FID of 1.75 on CelebA. We release our code at https://github.com/byeonghu-na/INDM.


title: ES-GNN: Generalizing Graph Neural Networks Beyond Homophily with Edge Splitting

abstract: While Graph Neural Networks (GNNs) have achieved enormous success in multiple graph analytical tasks, modern variants mostly rely on the strong inductive bias of homophily. However, real-world networks typically exhibit both homophilic and heterophilic linking patterns, wherein adjacent nodes may share dissimilar attributes and distinct labels. Therefore, GNNs smoothing node proximity holistically may aggregate both task-relevant and irrelevant (even harmful) information, limiting their ability to generalize to heterophilic graphs and potentially causing non-robustness. In this work, we propose a novel Edge Splitting GNN (ES-GNN) framework to adaptively distinguish between graph edges either relevant or irrelevant to learning tasks. This essentially transfers the original graph into two subgraphs with the same node set but complementary edge sets dynamically. Given that, information propagation separately on these subgraphs and edge splitting are alternatively conducted, thus disentangling the task-relevant and irrelevant features. Theoretically, we show that our ES-GNN can be regarded as a solution to a disentangled graph denoising problem, which further illustrates our motivations and interprets the improved generalization beyond homophily. Extensive experiments over 11 benchmark and 1 synthetic datasets not only demonstrate the effective performance of ES-GNN but also highlight its robustness to adversarial graphs and mitigation of the over-smoothing problem.


title: Evolution of Quantum Nonequilibrium for Coupled Harmonic Oscillators

abstract: In the context of de Broglie-Bohm pilot-wave theory, violations of the Born rule are allowed and can be considered as describing nonequilibrium distributions. We study the effects of interactions on quantum relaxation towards equilibrium for a system of one-dimensional coupled harmonic oscillators. We show by numerical simulations that interactions can delay or even prevent complete relaxation for some initial states. We also discuss how this effect might be relevant for cosmological scenarios and how nonequilibrium could be detected in some models.


title: R-HTDetector: Robust Hardware-Trojan Detection Based on Adversarial Training

abstract: Hardware Trojans (HTs) have become a serious problem, and extermination of them is strongly required for enhancing the security and safety of integrated circuits. An effective solution is to identify HTs at the gate level via machine learning techniques. However, machine learning has specific vulnerabilities, such as adversarial examples. In reality, it has been reported that adversarial modified HTs greatly degrade the performance of a machine learning-based HT detection method. Therefore, we propose a robust HT detection method using adversarial training (R-HTDetector). We formally describe the robustness of R-HTDetector in modifying HTs. Our work gives the world-first adversarial training for HT detection with theoretical backgrounds. We show through experiments with Trust-HUB benchmarks that R-HTDetector overcomes adversarial examples while maintaining its original accuracy.


title: Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matters

abstract: Motivated by the success of ensembles for uncertainty estimation in supervised learning, we take a renewed look at how ensembles of $Q$-functions can be leveraged as the primary source of pessimism for offline reinforcement learning (RL). We begin by identifying a critical flaw in a popular algorithmic choice used by many ensemble-based RL algorithms, namely the use of shared pessimistic target values when computing each ensemble member's Bellman error. Through theoretical analyses and construction of examples in toy MDPs, we demonstrate that shared pessimistic targets can paradoxically lead to value estimates that are effectively optimistic. Given this result, we propose MSG, a practical offline RL algorithm that trains an ensemble of $Q$-functions with independently computed targets based on completely separate networks, and optimizes a policy with respect to the lower confidence bound of predicted action values. Our experiments on the popular D4RL and RL Unplugged offline RL benchmarks demonstrate that on challenging domains such as antmazes, MSG with deep ensembles surpasses highly well-tuned state-of-the-art methods by a wide margin. Additionally, through ablations on benchmarks domains, we verify the critical significance of using independently trained $Q$-functions, and study the role of ensemble size. Finally, as using separate networks per ensemble member can become computationally costly with larger neural network architectures, we investigate whether efficient ensemble approximations developed for supervised learning can be similarly effective, and demonstrate that they do not match the performance and robustness of MSG with separate networks, highlighting the need for new efforts into efficient uncertainty estimation directed at RL.


title: Polynomials with core entropy zero

abstract: This paper studies polynomials with core entropy zero. We give several characterizations of polynomials with core entropy zero. In particular, we show that a degree d post-critically finite polynomial f has core entropy zero if and only if f is in the degree d main molecule. The characterizations define several quantities which measure the complexities of polynomials with core entropy zero. We show that these measures are all comparable.


title: Heterogeneous Collaborative Learning for Personalized Healthcare Analytics via Messenger Distillation

abstract: In this paper, we propose a Similarity-Quality-based Messenger Distillation (SQMD) framework for heterogeneous asynchronous on-device healthcare analytics. By introducing a preloaded reference dataset, SQMD enables all participant devices to distill knowledge from peers via messengers (i.e., the soft labels of the reference dataset generated by clients) without assuming the same model architecture. Furthermore, the messengers also carry important auxiliary information to calculate the similarity between clients and evaluate the quality of each client model, based on which the central server creates and maintains a dynamic collaboration graph (communication graph) to improve the personalization and reliability of SQMD under asynchronous conditions. Extensive experiments on three real-life datasets show that SQMD achieves superior performance.


title: On concentrated traveling vortex pairs with prescribed impulse

abstract: In this paper, we consider a constrained maximization problem related to planar vortex pairs with prescribed impulse. We prove existence, stability and asymptotic behavior for the maximizers, hence obtain a family of stable traveling vortex pairs approaching a pair of point vortices with equal magnitude and opposite signs. As a corollary, we get fine asymptotic estimates for Burton's vortex pairs with large impulse. For the non-concentrated case, we prove a form of stability for the Chaplygin-Lamb dipole.


title: A Hybrid Josephson Transmission Line and Passive Transmission Line Routing Framework for Single Flux Quantum Logic

abstract: The Single Flux Quantum (SFQ) logic family is a novel digital logic as it provides ultra-fast and energy-efficient circuits. For large-scale SFQ circuit design, specialized electronic design automation (EDA) tools are required due to the differences in logic type, timing constraints and circuit architecture, in contrast to the CMOS logic. In order to improve the overall performance of an SFQ circuit, an efficient routing algorithm should be applied during the layout design to perform accurate timing adjustment for fixing hold violations and optimizing critical paths. Thus, a hybrid Josephson transmission line and passive transmission line routing framework is proposed. It consists of four main modules and an exploration of the potential timing performance based on the given layout placement. The proposed routing tool is demonstrated on seven testbench circuits. The obtained results demonstrate that the operating frequency is greatly improved, and all the hold violations are eliminated for each circuit.


title: HiJoNLP at SemEval-2022 Task 2: Detecting Idiomaticity of Multiword Expressions using Multilingual Pretrained Language Models

abstract: This paper describes an approach to detect idiomaticity only from the contextualized representation of a MWE over multilingual pretrained language models. Our experiments find that larger models are usually more effective in idiomaticity detection. However, using a higher layer of the model may not guarantee a better performance. In multilingual scenarios, the convergence of different languages are not consistent and rich-resource languages have big advantages over other languages.


title: DP-PCA: Statistically Optimal and Differentially Private PCA

abstract: We study the canonical statistical task of computing the principal component from $n$ i.i.d.~data in $d$ dimensions under $(\varepsilon,\delta)$-differential privacy. Although extensively studied in literature, existing solutions fall short on two key aspects: ($i$) even for Gaussian data, existing private algorithms require the number of samples $n$ to scale super-linearly with $d$, i.e., $n=\Omega(d^{3/2})$, to obtain non-trivial results while non-private PCA requires only $n=O(d)$, and ($ii$) existing techniques suffer from a non-vanishing error even when the randomness in each data point is arbitrarily small. We propose DP-PCA, which is a single-pass algorithm that overcomes both limitations. It is based on a private minibatch gradient ascent method that relies on {\em private mean estimation}, which adds minimal noise required to ensure privacy by adapting to the variance of a given minibatch of gradients. For sub-Gaussian data, we provide nearly optimal statistical error rates even for $n=\tilde O(d)$. Furthermore, we provide a lower bound showing that sub-Gaussian style assumption is necessary in obtaining the optimal error rate.

categories: cs.LG cs.CR cs.IT math.IT math.ST stat.ML stat.TH


title: Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Loss

abstract: A central issue in machine learning is how to train models on sensitive user data. Industry has widely adopted a simple algorithm: Stochastic Gradient Descent with noise (a.k.a. Stochastic Gradient Langevin Dynamics). However, foundational theoretical questions about this algorithm's privacy loss remain open -- even in the seemingly simple setting of smooth convex losses over a bounded domain. Our main result resolves these questions: for a large range of parameters, we characterize the differential privacy up to a constant factor. This result reveals that all previous analyses for this setting have the wrong qualitative behavior. Specifically, while previous privacy analyses increase ad infinitum in the number of iterations, we show that after a small burn-in period, running SGD longer leaks no further privacy. Our analysis departs from previous approaches based on fast mixing, instead using techniques based on optimal transport (namely, Privacy Amplification by Iteration) and the Sampled Gaussian Mechanism (namely, Privacy Amplification by Sampling). Our techniques readily extend to other settings, e.g., strongly convex losses, non-uniform stepsizes, arbitrary batch sizes, and random or cyclic choice of batches.


title: Incorporating the Barzilai-Borwein Adaptive Step Size into Sugradient Methods for Deep Network Training

abstract: In this paper, we incorporate the Barzilai-Borwein step size into gradient descent methods used to train deep networks. This allows us to adapt the learning rate using a two-point approximation to the secant equation which quasi-Newton methods are based upon. Moreover, the adaptive learning rate method presented here is quite general in nature and can be applied to widely used gradient descent approaches such as Adagrad and RMSprop. We evaluate our method using standard example network architectures on widely available datasets and compare against alternatives elsewhere in the literature. In our experiments, our adaptive learning rate shows a smoother and faster convergence than that exhibited by the alternatives, with better or comparable performance.


title: An Expanding Shell of Neutral Hydrogen Associated with SN 1006: Hints for the Single-Degenerate Origin and Faint Hadronic Gamma-Rays

abstract: We report new HI observations of the Type Ia supernova remnant SN 1006 using the Australia Telescope Compact Array with an angular resolution of $4.5' \times 1.4'$ ($\sim$2 pc at the assumed SNR distance of 2.2 kpc). We find an expanding gas motion in position-velocity diagrams of HI with an expansion velocity of $\sim$4 km s$^{-1}$ and a mass of $\sim$1000 $M_\odot$. The spatial extent of the expanding shell is roughly the same as that of SN 1006. We here propose a hypothesis that SN 1006 exploded inside the wind-blown bubble formed by accretion winds from the progenitor system consisting of a white dwarf and a companion star, and then the forward shock has already reached the wind wall. This scenario is consistent with the single-degenerate model. We also derived the total energy of cosmic-ray protons $W_\mathrm{p}$ to be only $\sim$1.2-$2.0 \times 10^{47}$ erg by adopting the averaged interstellar proton density of $\sim$25 cm$^{-3}$. The small value is compatible with the relation between the age and $W_\mathrm{p}$ of other gamma-ray supernova remnants with ages below $\sim$6 kyr. The $W_\mathrm{p}$ value in SN 1006 will possibly increase up to several 10$^{49}$ erg in the next $\sim$5 kyr via the cosmic-ray diffusion into the HI wind-shell.


title: PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences

abstract: Point cloud sequences are irregular and unordered in the spatial dimension while exhibiting regularities and order in the temporal dimension. Therefore, existing grid based convolutions for conventional video processing cannot be directly applied to spatio-temporal modeling of raw point cloud sequences. In this paper, we propose a point spatio-temporal (PST) convolution to achieve informative representations of point cloud sequences. The proposed PST convolution first disentangles space and time in point cloud sequences. Then, a spatial convolution is employed to capture the local structure of points in the 3D space, and a temporal convolution is used to model the dynamics of the spatial regions along the time dimension. Furthermore, we incorporate the proposed PST convolution into a deep network, namely PSTNet, to extract features of point cloud sequences in a hierarchical manner. Extensive experiments on widely-used 3D action recognition and 4D semantic segmentation datasets demonstrate the effectiveness of PSTNet to model point cloud sequences.


title: Distributed Gaussian Process Based Cooperative Visual Pursuit Control for Drone Networks

abstract: In this paper, we propose a control law for camera-equipped drone networks to pursue a target rigid body with unknown motion based on distributed Gaussian process. First, we consider the situation where each drone has its own dataset, and learned the unknown target motion in a distributed manner. Second, we propose a control law using the distributed Gaussian processes, and show that the estimation and control errors are ultimately bounded. Furthermore, the effectiveness of the proposed method is verified by using simulations and experiments with actual drones.


title: Vetting the Lobster Diagram: Searching for Unseen Companions in Wide Binaries using NASA Space Exoplanet Missions

abstract: Over the past decade, the number of known wide binary systems has exponentially expanded thanks to the release of data from the Gaia Mission. Some of these wide binary systems are actually higher-order multiples, where one of the components is an unresolved binary itself. One way to search for these systems is by identifying overluminous components in the systems. In this study, we examine 4947 K+K wide binary pairs from the SUPERWIDE catalog and quantify the relative color and luminosity of the components to find evidence for additional, unresolved companions. The method is best illustrated in a graph we call the "Lobster diagram." To confirm that the identified overluminous components are close binary systems, we cross-match our wide binaries with the TESS, K2 and Kepler archives and search for the signs of eclipses and fast stellar rotation modulation in the light curves. We find that $78.9\%\pm20.7\%$ of the wide binaries which contain an eclipsing system are identified to be overluminous in the "Lobster Diagram" and $73.5\%\pm12.4\%$ of the wide binaries which contain a component showing fast rotation ($P<5$) days also show an overluminous component. From these results, we calculate a revised lower limit on the higher-order multiplicity fraction for K+K wide binaries of $40.0\%\pm1.6\%$. We also examine the higher-order multiplicity fraction as a function of projected physical separation and metallicity. The fraction is unusually constant as a function of projected physical separation while we see no statistically significant evidence that the fraction varies with metallicity.


title: Clustering Functional Data via Variational Inference

abstract: Functional data analysis deals with data recorded densely over time (or any other continuum) with one or more observed curves per subject. Conceptually, functional data are continuously defined, but in practice, they are usually observed at discrete points. Among different kinds of functional data analyses, clustering analysis aims to determine underlying groups of curves in the dataset when there is no information on the group membership of each individual curve. In this work, we propose a new model-based approach for clustering and smoothing functional data simultaneously via variational inference. We derive coordinate ascent mean-field variational Bayes algorithms to approximate the posterior distribution of our model parameters by finding the variational distribution with the smallest Kullback-Leibler divergence to the posterior. The performance of our proposed method is evaluated using simulated data and publicly available datasets.


title: Hazard Gradient Penalty for Survival Analysis

abstract: Survival analysis appears in various fields such as medicine, economics, engineering, and business. Recent studies showed that the Ordinary Differential Equation (ODE) modeling framework unifies many existing survival models while the framework is flexible and widely applicable. However, naively applying the ODE framework to survival analysis problems may model fiercely changing density function which may worsen the model's performance. Though we can apply L1 or L2 regularizers to the ODE model, their effect on the ODE modeling framework is barely known. In this paper, we propose hazard gradient penalty (HGP) to enhance the performance of a survival analysis model. Our method imposes constraints on local data points by regularizing the gradient of hazard function with respect to the data point. Our method applies to any survival analysis model including the ODE modeling framework and is easy to implement. We theoretically show that our method is related to minimizing the KL divergence between the density function at a data point and that of the neighborhood points. Experimental results on three public benchmarks show that our approach outperforms other regularization methods.


title: Off-Beat Multi-Agent Reinforcement Learning

abstract: We investigate model-free multi-agent reinforcement learning (MARL) in environments where off-beat actions are prevalent, i.e., all actions have pre-set execution durations. During execution durations, the environment changes are influenced by, but not synchronised with, action execution. Such a setting is ubiquitous in many real-world problems. However, most MARL methods assume actions are executed immediately after inference, which is often unrealistic and can lead to catastrophic failure for multi-agent coordination with off-beat actions. In order to fill this gap, we develop an algorithmic framework for MARL with off-beat actions. We then propose a novel episodic memory, LeGEM, for model-free MARL algorithms. LeGEM builds agents' episodic memories by utilizing agents' individual experiences. It boosts multi-agent learning by addressing the challenging temporal credit assignment problem raised by the off-beat actions via our novel reward redistribution scheme, alleviating the issue of non-Markovian reward. We evaluate LeGEM on various multi-agent scenarios with off-beat actions, including Stag-Hunter Game, Quarry Game, Afforestation Game, and StarCraft II micromanagement tasks. Empirical results show that LeGEM significantly boosts multi-agent coordination and achieves leading performance and improved sample efficiency.


title: Estimating and Analyzing Neural Information Flow Using Signal Processing on Graphs

abstract: Correlating neural communication in brain networks with behavior and cognition can provide fundamental insights into the functionality of both healthy and diseased brains. We demonstrate how communication in the brain can be estimated from recorded neural activity using concepts from graph signal processing. The communication is modeled as a flow signals on the edges of a graph and naturally arises from a graph diffusion process. We apply the diffusion model to micro-electrocorticography (ECoG) recordings from sensorimotor cortex of two non-human primates to estimate the neural communication flow during excitatory optogenetics. Comparisons with a baseline model demonstrate that adding the neural flow can improve ECoG predictions. Finally, we demonstrate how the neural flow can be decomposed into a gradient and rotational component and show that the gradient component depends on the location of stimulation. This technique, for the first time, offers the opportunity to study neural communication on an unprecedented spatiotemporal scale.


title: Effective Abstract Reasoning with Dual-Contrast Network

abstract: As a step towards improving the abstract reasoning capability of machines, we aim to solve Raven's Progressive Matrices (RPM) with neural networks, since solving RPM puzzles is highly correlated with human intelligence. Unlike previous methods that use auxiliary annotations or assume hidden rules to produce appropriate feature representation, we only use the ground truth answer of each question for model learning, aiming for an intelligent agent to have a strong learning capability with a small amount of supervision. Based on the RPM problem formulation, the correct answer filled into the missing entry of the third row/column has to best satisfy the same rules shared between the first two rows/columns. Thus we design a simple yet effective Dual-Contrast Network (DCNet) to exploit the inherent structure of RPM puzzles. Specifically, a rule contrast module is designed to compare the latent rules between the filled row/column and the first two rows/columns; a choice contrast module is designed to increase the relative differences between candidate choices. Experimental results on the RAVEN and PGM datasets show that DCNet outperforms the state-of-the-art methods by a large margin of 5.77%. Further experiments on few training samples and model generalization also show the effectiveness of DCNet. Code is available at https://github.com/visiontao/dcnet.


title: Residual Intersections and Core of Modules

abstract: We introduce the notion of residual intersections of modules and prove their existence. We show that projective dimension one modules have Cohen-Macaulay residual intersections, namely they satisfy the relevant Artin-Nagata property. We then establish a formula for the core of orientable modules satisfying certain homological conditions, extending previous results of Corso, Polini, and Ulrich on the core of projective one modules. Finally, we provide examples of classes of modules that satisfy our assumptions.


title: Can Foundation Models Help Us Achieve Perfect Secrecy?

abstract: A key promise of machine learning is the ability to assist users with personal tasks. Because the personal context required to make accurate predictions is often sensitive, we require systems that protect privacy. A gold standard privacy-preserving system will satisfy perfect secrecy, meaning that interactions with the system provably reveal no private information. However, privacy and quality appear to be in tension in existing systems for personal tasks. Neural models typically require copious amounts of training to perform well, while individual users typically hold a limited scale of data, so federated learning (FL) systems propose to learn from the aggregate data of multiple users. FL does not provide perfect secrecy, but rather practitioners apply statistical notions of privacy -- i.e., the probability of learning private information about a user should be reasonably low. The strength of the privacy guarantee is governed by privacy parameters. Numerous privacy attacks have been demonstrated on FL systems and it can be challenging to reason about the appropriate privacy parameters for a privacy-sensitive use case. Therefore our work proposes a simple baseline for FL, which both provides the stronger perfect secrecy guarantee and does not require setting any privacy parameters. We initiate the study of when and where an emerging tool in ML -- the in-context learning abilities of recent pretrained models -- can be an effective baseline alongside FL. We find in-context learning is competitive with strong FL baselines on 6 of 7 popular benchmarks from the privacy literature and a real-world case study, which is disjoint from the pretraining data. We release our code here: https://github.com/simran-arora/focus


title: DLTTA: Dynamic Learning Rate for Test-time Adaptation on Cross-domain Medical Images

abstract: Test-time adaptation (TTA) has increasingly been an important topic to efficiently tackle the cross-domain distribution shift at test time for medical images from different institutions. Previous TTA methods have a common limitation of using a fixed learning rate for all the test samples. Such a practice would be sub-optimal for TTA, because test data may arrive sequentially therefore the scale of distribution shift would change frequently. To address this problem, we propose a novel dynamic learning rate adjustment method for test-time adaptation, called DLTTA, which dynamically modulates the amount of weights update for each test image to account for the differences in their distribution shift. Specifically, our DLTTA is equipped with a memory bank based estimation scheme to effectively measure the discrepancy of a given test sample. Based on this estimated discrepancy, a dynamic learning rate adjustment strategy is then developed to achieve a suitable degree of adaptation for each test sample. The effectiveness and general applicability of our DLTTA is extensively demonstrated on three tasks including retinal optical coherence tomography (OCT) segmentation, histopathological image classification, and prostate 3D MRI segmentation. Our method achieves effective and fast test-time adaptation with consistent performance improvement over current state-of-the-art test-time adaptation methods. Code is available at: https://github.com/med-air/DLTTA.


title: V-Doc : Visual questions answers with Documents

abstract: We propose V-Doc, a question-answering tool using document images and PDF, mainly for researchers and general non-deep learning experts looking to generate, process, and understand the document visual question answering tasks. The V-Doc supports generating and using both extractive and abstractive question-answer pairs using documents images. The extractive QA selects a subset of tokens or phrases from the document contents to predict the answers, while the abstractive QA recognises the language in the content and generates the answer based on the trained model. Both aspects are crucial to understanding the documents, especially in an image format. We include a detailed scenario of question generation for the abstractive QA task. V-Doc supports a wide range of datasets and models, and is highly extensible through a declarative, framework-agnostic platform.


title: Low-Degree Polynomials Extract from Local Sources

abstract: We continue a line of work on extracting random bits from weak sources that are generated by simple processes. We focus on the model of locally samplable sources, where each bit in the source depends on a small number of (hidden) uniformly random input bits. Also known as local sources, this model was introduced by De and Watson (TOCT 2012) and Viola (SICOMP 2014), and is closely related to sources generated by $\mathsf{AC}^0$ circuits and bounded-width branching programs. In particular, extractors for local sources also work for sources generated by these classical computational models. Despite being introduced a decade ago, little progress has been made on improving the entropy requirement for extracting from local sources. The current best explicit extractors require entropy $n^{1/2}$, and follow via a reduction to affine extractors. To start, we prove a barrier showing that one cannot hope to improve this entropy requirement via a black-box reduction of this form. In particular, new techniques are needed. In our main result, we seek to answer whether low-degree polynomials (over $\mathbb{F}_2$) hold potential for breaking this barrier. We answer this question in the positive, and fully characterize the power of low-degree polynomials as extractors for local sources. More precisely, we show that a random degree $r$ polynomial is a low-error extractor for $n$-bit local sources with min-entropy $\Omega(r(n\log n)^{1/r})$, and we show that this is tight. Our result leverages several new ingredients, which may be of independent interest. Our existential result relies on a new reduction from local sources to a more structured family, known as local non-oblivious bit-fixing sources. To show its tightness, we prove a "local version" of a structural result by Cohen and Tal (RANDOM 2015), which relies on a new "low-weight" Chevalley-Warning theorem.


title: A Robust, Multiple Control Barrier Function Framework for Input Constrained Systems

abstract: We propose a novel (Type-II) zeroing control barrier function (ZCBF) for safety-critical control, which generalizes the original ZCBF approach. Our method allows for applications to a larger class of systems (e.g. passivity-based) while still ensuring robustness, for which the construction of conventional ZCBFs is difficult. We also propose a locally Lipschitz continuous control law that handles multiple ZCBFs, while respecting input constraints, which is not currently possible with existing ZCBF methods. We apply the proposed concept for unicycle navigation in an obstacle-rich environment.


title: High-Order Incremental Potential Contact for Elastodynamic Simulation on Curved Meshes

abstract: High-order bases provide major advantages over linear ones in terms of efficiency, as they provide (for the same physical model) higher accuracy for the same running time, and reliability, as they are less affected by locking artifacts and mesh quality. Thus, we introduce a high-order finite element (FE) formulation (high-order bases) for elastodynamic simulation on high-order (curved) meshes with contact handling based on the recently proposed Incremental Potential Contact (IPC) model. Our approach is based on the observation that each IPC optimization step used to minimize the elasticity, contact, and friction potentials leads to linear trajectories even in the presence of nonlinear meshes or nonlinear FE bases. It is thus possible to retain the strong non-penetration guarantees and large time steps of the original formulation while benefiting from the high-order bases and high-order geometry. We accomplish this by mapping displacements and resulting contact forces between a linear collision proxy and the underlying high-order representation. We demonstrate the effectiveness of our approach in a selection of problems from graphics, computational fabrication, and scientific computing.


title: GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesis

abstract: Despite achieving superior performance in human-level control problems, unlike humans, deep reinforcement learning (DRL) lacks high-order intelligence (e.g., logic deduction and reuse), thus it behaves ineffectively than humans regarding learning and generalization in complex problems. Previous works attempt to directly synthesize a white-box logic program as the DRL policy, manifesting logic-driven behaviors. However, most synthesis methods are built on imperative or declarative programming, and each has a distinct limitation, respectively. The former ignores the cause-effect logic during synthesis, resulting in low generalizability across tasks. The latter is strictly proof-based, thus failing to synthesize programs with complex hierarchical logic. In this paper, we combine the above two paradigms together and propose a novel Generalizable Logic Synthesis (GALOIS) framework to synthesize hierarchical and strict cause-effect logic programs. GALOIS leverages the program sketch and defines a new sketch-based hybrid program language for guiding the synthesis. Based on that, GALOIS proposes a sketch-based program synthesis method to automatically generate white-box programs with generalizable and interpretable cause-effect logic. Extensive evaluations on various decision-making tasks with complex logic demonstrate the superiority of GALOIS over mainstream baselines regarding the asymptotic performance, generalizability, and great knowledge reusability across different environments.


title: Chern classes and unitary equivalence of normal matrices over topological spaces

abstract: This paper continues the authors' work on the question of unitary equivalence of matrices with entries in the complex-valued functions of a topological space (matrices over spaces). Specifically, we here consider the question of unitary equivalence for pairs of normal matrices over a space that share a common characteristic polynomial that can be globally factored into distinct linear factors. We show that such a matrix is diagonalizable if and only if the first Chern classes of its eigenbundles all vanish and derive as an application that all such matrices over $\mathbb{C}P^m$ are diagonalizable for $m > 1$. Next, given a CW complex $X$ and a polynomial $\mu$ in $C(X)[\lambda]$ that globally splits into distinct linear factors, we prove that the number of unitary equivalence classes of matrices with $\mu$ as a characteristic polynomial depends only on the space $X$ and the degree of $\mu$, and we give some estimates on how many unitary equivalence classes there can be. In the case that $X$ is a CW complex of dimension at most three, we demonstrate a bijection between the unitary equivalence classes of $n \times n$ normal matrices with characteristic polynomial $\mu$ and elements of the group $(H^2(X))^{n-1}$. Finally, when $X$ is a smooth manifold and we restrict to matrices with smooth entries, we construct a de Rham cohomology class whose nonvanishing is an obstruction to unitary equivalence.


title: Understanding Long Programming Languages with Structure-Aware Sparse Attention

abstract: Programming-based Pre-trained Language Models (PPLMs) such as CodeBERT have achieved great success in many downstream code-related tasks. Since the memory and computational complexity of self-attention in the Transformer grow quadratically with the sequence length, PPLMs typically limit the code length to 512. However, codes in real-world applications are generally long, such as code searches, which cannot be processed efficiently by existing PPLMs. To solve this problem, in this paper, we present SASA, a Structure-Aware Sparse Attention mechanism, which reduces the complexity and improves performance for long code understanding tasks. The key components in SASA are top-$k$ sparse attention and Abstract Syntax Tree (AST)-based structure-aware attention. With top-$k$ sparse attention, the most crucial attention relation can be obtained with a lower computational cost. As the code structure represents the logic of the code statements, which is a complement to the code sequence characteristics, we further introduce AST structures into attention. Extensive experiments on CodeXGLUE tasks show that SASA achieves better performance than the competing baselines.


title: Accurate Tree Roots Positioning and Sizing over Undulated Ground Surfaces by Common Offset GPR Measurements

abstract: Tree roots detection is a popular application of the Ground-penetrating radar (GPR). Normally, the ground surface above the tree roots is assumed to be flat, and standard processing methods based on hyperbolic fitting are applied to the hyperbolae reflection patterns of tree roots for detection purposes. When the surface of the land is undulating (not flat), these typical hyperbolic fitting methods becomes inaccurate. This is because, the reflection patterns change with the uneven ground surfaces. When the soil surface is not flat, it is inaccurate to use the peak point of an asymmetric reflection pattern to identify the depth and horizontal position of the underground target. The reflection patterns of the complex shapes due to extreme surface variations results in analysis difficulties. Furthermore, when multiple objects are buried under an undulating ground, it is hard to judge their relative positions based on a B-scan that assumes a flat ground. In this paper, a roots fitting method based on electromagnetic waves (EM) travel time analysis is proposed to take into consideration the realistic undulating ground surface. A wheel-based (WB) GPR and an antenna-height-fixed (AHF) GPR System are presented, and their corresponding fitting models are proposed. The effectiveness of the proposed method is demonstrated and validated through numerical examples and field experiments.


title: Explicit method to make shortened stabilizer EAQECC from stabilizer QECC

abstract: In the previous research by Grassl, Huber and Winter, they proved a theorem which can make entanglement-assisted quantum error-correcting codes (EAQECC) from general quantum error-correcting codes (QECC). In this paper, we prove that the shortened EAQECC is a stabilizer code if the original EAQECC is a stabilizer code.


title: Towards Faithful and Consistent Explanations for Graph Neural Networks

abstract: Uncovering rationales behind predictions of graph neural networks (GNNs) has received increasing attention over recent years. Instance-level GNN explanation aims to discover critical input elements, like nodes or edges, that the target GNN relies upon for making predictions. Though various algorithms are proposed, most of them formalize this task by searching the minimal subgraph which can preserve original predictions. However, an inductive bias is deep-rooted in this framework: several subgraphs can result in the same or similar outputs as the original graphs. Consequently, they have the danger of providing spurious explanations and fail to provide consistent explanations. Applying them to explain weakly-performed GNNs would further amplify these issues. To address this problem, we theoretically examine the predictions of GNNs from the causality perspective. Two typical reasons of spurious explanations are identified: confounding effect of latent variables like distribution shift, and causal factors distinct from the original input. Observing that both confounding effects and diverse causal rationales are encoded in internal representations, we propose a simple yet effective countermeasure by aligning embeddings. Concretely, concerning potential shifts in the high-dimensional space, we design a distribution-aware alignment algorithm based on anchors. This new objective is easy to compute and can be incorporated into existing techniques with no or little effort. Theoretical analysis shows that it is in effect optimizing a more faithful explanation objective in design, which further justifies the proposed approach.


title: An efficient tensor regression for high-dimensional data

abstract: Most currently used tensor regression models for high-dimensional data are based on Tucker decomposition, which has good properties but loses its efficiency in compressing tensors very quickly as the order of tensors increases, say greater than four or five. However, for the simplest tensor autoregression in handling time series data, its coefficient tensor already has the order of six. This paper revises a newly proposed tensor train (TT) decomposition and then applies it to tensor regression such that a nice statistical interpretation can be obtained. The new tensor regression can well match the data with hierarchical structures, and it even can lead to a better interpretation for the data with factorial structures, which are supposed to be better fitted by models with Tucker decomposition. More importantly, the new tensor regression can be easily applied to the case with higher order tensors since TT decomposition can compress the coefficient tensors much more efficiently. The methodology is also extended to tensor autoregression for time series data, and nonasymptotic properties are derived for the ordinary least squares estimations of both tensor regression and autoregression. A new algorithm is introduced to search for estimators, and its theoretical justification is also discussed. Theoretical and computational properties of the proposed methodology are verified by simulation studies, and the advantages over existing methods are illustrated by two real examples.


title: Electric Vehicle Traveling Salesman Problem with Drone with Partial recharge Policy

abstract: In (Zhu et al., 2022), it proposes an electric vehicle traveling salesman problem with drone while assuming that the electric vehicle (EV) is a battery-electric vehicle whose energy could be refreshed in a battery swap station in minutes. In this paper, we extend the work in (Zhu et al., 2022) by relaxing the fixed-time-full-charge assumption, assuming that the EV is a plug-in hybrid electric vehicle that could be partially recharged in a charging station. This problem is named electric vehicle traveling salesman problem with drone with partial recharge policy (EVTSPD-P). A three-index MILP formulation is proposed to solve the EVTSPD-P with linear and non-linear charging functions where the concave time-state-of-charge (SoC) function is approximated using piecewise linear functions, a technique proposed in Montoya et al. (2017) and Zuo et al. (2019). Furthermore, a specially designed adaptive large neighborhood search (ALNS) meta-heuristic, which incorporates constraint programming (CP), is presented to solve EVTSPD-P problem instances of practical size. The numerical analysis results indicate that the proposed ALNS method is more efficient than variable neighborhood search and has an average optimality gap of about 3% when solving instances with ten nodes. Besides, using a piecewise linear function with a six-line-segments approximation has an average of 10.8% less cost than a linear approximation.


title: Observable effect of quantized cylindrical gravitational waves

abstract: We investigate the response of a model gravitational wave detector consisting of two particles to the quantized cylindrical gravitational waves and obtain a relation between the standard deviation of the distance between two particles and the distance from the source to the detector. It is found that the quantum effect carried by the cylindrical gravitational wave can be observed above Planck scale even though the source is as far as the cosmological horizon. The equation of motion for the change of the distance between two particles is obtained when the cylindrical gravitational waves pass. It is surprising that the dissipative term does not exist up to the first order approximation due to cylindrical symmetry of the gravitational wave.


title: Continuous modeling of creased annuli with tunable bistable and looping behaviors

abstract: Creases are purposely introduced to thin structures for designing deployable origami, artistic geometries, and functional structures with tunable nonlinear mechanics. Modeling the mechanics of creased structures is challenging because creases introduce geometric discontinuity and often have complex mechanical responses due to the local material damage. In this work, we propose a continuous description of the sharp geometry of creases and apply it to the study of \emph{creased annuli}, made by introducing radial creases to annular strips with the creases annealed to behave elastically. We find creased annuli have generic bistability and can be folded into various compact shapes, depending on the crease pattern and the overcurvature of the flat annulus. We use a regularized Dirac delta function (RDDF) to describe the geometry of a crease, with the finite spike of the RDDF capturing the localized curvature. Together with anisotropic rod theory, we solve the nonlinear mechanics of creased annuli, with its stability determined by the standard conjugate point test. We find excellent agreement between precision tabletop models, numerical predictions from our analytical framework, and modeling results from finite element simulations. We further show that by varying the rest curvature of the thin strip, dynamic switches between different states of creased annuli can be achieved, which could inspire the design of novel deployable and morphable structures. We believe our smooth description of discontinuous geometries will benefit to the mechanical modeling and design of a wide spectrum of engineering structures that embrace geometric and material discontinuities.


title: Image Reconstruction of Multi Branch Feature Multiplexing Fusion Network with Mixed Multi-layer Attention

abstract: Image super-resolution reconstruction achieves better results than traditional methods with the help of the powerful nonlinear representation ability of convolution neural network. However, some existing algorithms also have some problems, such as insufficient utilization of phased features, ignoring the importance of early phased feature fusion to improve network performance, and the inability of the network to pay more attention to high-frequency information in the reconstruction process. To solve these problems, we propose a multi-branch feature multiplexing fusion network with mixed multi-layer attention (MBMFN), which realizes the multiple utilization of features and the multistage fusion of different levels of features. To further improve the networks performance, we propose a lightweight enhanced residual channel attention (LERCA), which can not only effectively avoid the loss of channel information but also make the network pay more attention to the key channel information and benefit from it. Finally, the attention mechanism is introduced into the reconstruction process to strengthen the restoration of edge texture and other details. A large number of experiments on several benchmark sets show that, compared with other advanced reconstruction algorithms, our algorithm produces highly competitive objective indicators and restores more image detail texture information.


title: Curvature estimates for hypersurfaces of constant curvature in hyperbolic space

abstract: In this note, we prove that for every $0<\sigma<1$, there exists a smooth complete hypersurface $\Sigma$ in $\mathbb{H}^{n+1}$ with prescribed asymptotic boundary $\partial \Sigma=\Gamma$ at infinity, whose principal curvatures $\kappa=(\kappa_1,\ldots,\kappa_n)$ lie in a general cone $K$ and satisfy $f(\kappa)=\sigma$ at each point of $\Sigma$. Previously, the problem has been studied by Guan-Spruck in [J. Eur. Math. Soc. (JEMS) 12 (2010), no. 3, 797-817], and they proved the existence result for $\sigma \in (\sigma_0,1)$, where $\sigma_0>0$. A major ingredient of our proof is a refined curvature estimate of theirs that is applicable when the curvature function $f(\kappa)$ has controllable partial derivatives, but it is adequate for our purpose; specifically, we solve the problem for $f=H_{k}/H_{k-1}$ in the $k$-th Garding cone where $H_k$ is the normalized $k$-th elementary symmetric polynomial and $1 \leq k \leq n$.


title: Subverting machines, fluctuating identities: Re-learning human categorization

abstract: Most machine learning systems that interact with humans construct some notion of a person's "identity," yet the default paradigm in AI research envisions identity with essential attributes that are discrete and static. In stark contrast, strands of thought within critical theory present a conception of identity as malleable and constructed entirely through interaction; a doing rather than a being. In this work, we distill some of these ideas for machine learning practitioners and introduce a theory of identity as autopoiesis, circular processes of formation and function. We argue that the default paradigm of identity used by the field immobilizes existing identity categories and the power differentials that co$\unicode{x2010}$occur, due to the absence of iterative feedback to our models. This includes a critique of emergent AI fairness practices that continue to impose the default paradigm. Finally, we apply our theory to sketch approaches to autopoietic identity through multilevel optimization and relational learning. While these ideas raise many open questions, we imagine the possibilities of machines that are capable of expressing human identity as a relationship perpetually in flux.


title: Generating multivariate time series with COmmon Source CoordInated GAN (COSCI-GAN)

abstract: Generating multivariate time series is a promising approach for sharing sensitive data in many medical, financial, and IoT applications. A common type of multivariate time series originates from a single source such as the biometric measurements from a medical patient. This leads to complex dynamical patterns between individual time series that are hard to learn by typical generation models such as GANs. There is valuable information in those patterns that machine learning models can use to better classify, predict or perform other downstream tasks. We propose a novel framework that takes time series' common origin into account and favors channel/feature relationships preservation. The two key points of our method are: 1) the individual time series are generated from a common point in latent space and 2) a central discriminator favors the preservation of inter-channel/feature dynamics. We demonstrate empirically that our method helps preserve channel/feature correlations and that our synthetic data performs very well in downstream tasks with medical and financial data.


title: Optimal control with a strong harmonic trap

abstract: Quadratic trapping potentials are widely used to experimentally probe biopolymers and molecular machines and drive transitions in steered molecular-dynamics simulations. Approximating energy landscapes as locally quadratic, we design multidimensional trapping protocols that minimize dissipation. The designed protocols are easily solvable and applicable to a wide range of systems. The approximation does not rely on either fast or slow limits and is valid for any duration provided the trapping potential is sufficiently strong. We demonstrate the utility of the designed protocols with a simple model of a periodically driven rotary motor. Our results elucidate principles of effective single-molecule manipulation and efficient nonequilibrium free-energy estimation.


title: Personalized Algorithmic Recourse with Preference Elicitation

abstract: Algorithmic Recourse (AR) is the problem of computing a sequence of actions that -- once performed by a user -- overturns an undesirable machine decision. It is paramount that the sequence of actions does not require too much effort for users to implement. Yet, most approaches to AR assume that actions cost the same for all users, and thus may recommend unfairly expensive recourse plans to certain users. Prompted by this observation, we introduce PEAR, the first human-in-the-loop approach capable of providing personalized algorithmic recourse tailored to the needs of any end-user. PEAR builds on insights from Bayesian Preference Elicitation to iteratively refine an estimate of the costs of actions by asking choice set queries to the target user. The queries themselves are computed by maximizing the Expected Utility of Selection, a principled measure of information gain accounting for uncertainty on both the cost estimate and the user's responses. PEAR integrates elicitation into a Reinforcement Learning agent coupled with Monte Carlo Tree Search to quickly identify promising recourse plans. Our empirical evaluation on real-world datasets highlights how PEAR produces high-quality personalized recourse in only a handful of iterations.


title: Learning Instance Representation Banks for Aerial Scene Classification

abstract: Aerial scenes are more complicated in terms of object distribution and spatial arrangement than natural scenes due to the bird view, and thus remain challenging to learn discriminative scene representation. Recent solutions design \textit{local semantic descriptors} so that region of interests (RoIs) can be properly highlighted. However, each local descriptor has limited description capability and the overall scene representation remains to be refined. In this paper, we solve this problem by designing a novel representation set named \textit{instance representation bank} (IRB), which unifies multiple local descriptors under the multiple instance learning (MIL) formulation. This unified framework is not trivial as all the local semantic descriptors can be aligned to the same scene scheme, enhancing the scene representation capability. Specifically, our IRB learning framework consists of a backbone, an instance representation bank, a semantic fusion module and a scene scheme alignment loss function. All the components are organized in an end-to-end manner. Extensive experiments on three aerial scene benchmarks demonstrate that our proposed method outperforms the state-of-the-art approaches by a large margin.


title: DeepSAT: An EDA-Driven Learning Framework for SAT

abstract: We present DeepSAT, a novel end-to-end learning framework for the Boolean satisfiability (SAT) problem. Unlike existing solutions trained on random SAT instances with relatively weak supervision, we propose applying the knowledge of the well-developed electronic design automation (EDA) field for SAT solving. Specifically, we first resort to logic synthesis algorithms to pre-process SAT instances into optimized and-inverter graphs (AIGs). By doing so, the distribution diversity among various SAT instances can be dramatically reduced, which facilitates improving the generalization capability of the learned model. Next, we regard the distribution of SAT solutions being a product of conditional Bernoulli distributions. Based on this observation, we approximate the SAT solving procedure with a conditional generative model, leveraging a novel directed acyclic graph neural network (DAGNN) with two polarity prototypes for conditional SAT modeling. To effectively train the generative model, with the help of logic simulation tools, we obtain the probabilities of nodes in the AIG being logic `1' as rich supervision. We conduct comprehensive experiments on various SAT problems. Our results show that, DeepSAT achieves significant accuracy improvements over state-of-the-art learning-based SAT solutions, especially when generalized to SAT instances that are relatively large or with diverse distributions.


title: Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Games

abstract: We study the problem of finding the Nash equilibrium in a two-player zero-sum Markov game. Due to its formulation as a minimax optimization program, a natural approach to solve the problem is to perform gradient descent/ascent with respect to each player in an alternating fashion. However, due to the non-convexity/non-concavity of the underlying objective function, theoretical understandings of this method are limited. In our paper, we consider solving an entropy-regularized variant of the Markov game. The regularization introduces structure into the optimization landscape that make the solutions more identifiable and allow the problem to be solved more efficiently. Our main contribution is to show that under proper choices of the regularization parameter, the gradient descent ascent algorithm converges to the Nash equilibrium of the original unregularized problem. We explicitly characterize the finite-time performance of the last iterate of our algorithm, which vastly improves over the existing convergence bound of the gradient descent ascent algorithm without regularization. Finally, we complement the analysis with numerical simulations that illustrate the accelerated convergence of the algorithm.


title: Inferring Quadrupolar Dynamo Mode from Sunspot Statistics

abstract: Observations of long-term north-south asymmetry in solar activity demand the equator-symmetric (quadrupolar) mode be present in the solar magnetic field in line with the dominant antisymmetric (dipolar) mode. This paper proposes treating the sunspot area as a proxy for subsurface toroidal magnetic flux to infer the quadrupolar mode of the solar dynamo from sunspot data. Toroidal pseudo-fluxes (PF) in the northern and southern hemispheres are defined as a signed sunspot area with plus or minus sign prescribed to them in accord with the Hale's sunspot polarity rules. Statistical correlation analysis and wavelet analysis of so-defined PFs reveal quadrupolar oscillations with a period of about 16 yr and amplitude of about 0.17 relative to the amplitude of the dominant 22-yr dipolar mode.


title: Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture

abstract: Physics-informed neural networks (PINNs) are revolutionizing science and engineering practice by bringing together the power of deep learning to bear on scientific computation. In forward modeling problems, PINNs are meshless partial differential equation (PDE) solvers that can handle irregular, high-dimensional physical domains. Naturally, the neural architecture hyperparameters have a large impact on the efficiency and accuracy of the PINN solver. However, this remains an open and challenging problem because of the large search space and the difficulty of identifying a proper search objective for PDEs. Here, we propose Auto-PINN, the first systematic, automated hyperparameter optimization approach for PINNs, which employs Neural Architecture Search (NAS) techniques to PINN design. Auto-PINN avoids manually or exhaustively searching the hyperparameter space associated with PINNs. A comprehensive set of pre-experiments using standard PDE benchmarks allows us to probe the structure-performance relationship in PINNs. We find that the different hyperparameters can be decoupled, and that the training loss function of PINNs is a good search objective. Comparison experiments with baseline methods demonstrate that Auto-PINN produces neural architectures with superior stability and accuracy over alternative baselines.


title: Electronic structures and stability investigation of the new class of large band gap topological insulators MTl$_4$Te$_3$ (M = Cd, Hg)

abstract: By means of ternary chemical potential phase diagram and phonon spectrum calculations, we propose that $M$Tl$_4$Te$_3$ ($M$ = Cd, Hg), the derivatives of Tl$_5$Te$_3$, are thermodynamically and dynamically stable in the body centered tetragonal crystal structure with $I$4/$mcm$ symmetry. Our electronic structures calculations confirm that a robust $s$-$p$ band inversion occurs near the Fermi level in $M$Tl$_4$Te$_3$, and a topological band gap about 0.13 eV in CdTl$_4$Te$_3$ is induced by the spin-orbit coupling. These results suggest that $M$Tl$_4$Te$_3$ are a new class of large band gap 3D strong topological insulators that are stable and synthesizable in experiments, which could be used to design efficient spin torque equipment and spin device.


title: Attention Awareness Multiple Instance Neural Network

abstract: Multiple instance learning is qualified for many pattern recognition tasks with weakly annotated data. The combination of artificial neural network and multiple instance learning offers an end-to-end solution and has been widely utilized. However, challenges remain in two-folds. Firstly, current MIL pooling operators are usually pre-defined and lack flexibility to mine key instances. Secondly, in current solutions, the bag-level representation can be inaccurate or inaccessible. To this end, we propose an attention awareness multiple instance neural network framework in this paper. It consists of an instance-level classifier, a trainable MIL pooling operator based on spatial attention and a bag-level classification layer. Exhaustive experiments on a series of pattern recognition tasks demonstrate that our framework outperforms many state-of-the-art MIL methods and validates the effectiveness of our proposed attention MIL pooling operators.


title: Computations about formal multiple zeta spaces defined by binary extended double shuffle relations

abstract: The formal multiple zeta space we consider with a computer is an $\mathbb{F}_2$-vector space generated by $2^{k-2}$ formal symbols for a given weight $k$, where the symbols satisfy binary extended double shuffle relations. Up to weight $k=22$, we compute the dimensions of the formal multiple zeta spaces, and verify the dimension conjecture on original extended double shuffle relations of real multiple zeta values. Our computations adopt Gaussian forward elimination and give information for spaces filtered by depth. We can observe that the dimensions of the depth-graded formal multiple zeta spaces have a Pascal triangle pattern expected by the Hoffman mult-indices.


title: On Lyndon-Word Representable Graphs

abstract: In this short note, we first associate a new simple undirected graph with a given word over an ordered alphabet of $n$-letters. We will call it the Lyndon graph of that word. Then, we introduce the concept of the Lyndon-word representable graph as a graph isomorphic to a Lyndon graph of some word. Then, we introduce the generalized Stirling cycle number $S(N;n,k)$ as the number words of length $N$ with $k$ distinct Lydon words in their Lyndon factorization over an ordered alphabet of $n$-letters . Finally, we conclude the paper with several interesting open questions and conjectures for interested audiences.