title: Attention improves concentration when learning node embeddings

abstract: We consider the problem of predicting edges in a graph from node attributes in an e-commerce setting. Specifically, given nodes labelled with search query text, we want to predict links to related queries that share products. Experiments with a range of deep neural architectures show that simple feedforward networks with an attention mechanism perform best for learning embeddings. The simplicity of these models allows us to explain the performance of attention. We propose an analytically tractable model of query generation, AttEST, that views both products and the query text as vectors embedded in a latent space. We prove (and empirically validate) that the point-wise mutual information (PMI) matrix of the AttEST query text embeddings displays a low-rank behavior analogous to that observed in word embeddings. This low-rank property allows us to derive a loss function that maximizes the mutual information between related queries which is used to train an attention network to learn query embeddings. This AttEST network beats traditional memory-based LSTM architectures by over 20% on F-1 score. We justify this out-performance by showing that the weights from the attention mechanism correlate strongly with the weights of the best linear unbiased estimator (BLUE) for the product vectors, and conclude that attention plays an important role in variance reduction.


title: Adaptive Gradient Methods Converge Faster with Over-Parameterization (but you should do a line-search)

abstract: Adaptive gradient methods are typically used for training over-parameterized models. To better understand their behaviour, we study a simplistic setting -- smooth, convex losses with models over-parameterized enough to interpolate the data. In this setting, we prove that AMSGrad with constant step-size and momentum converges to the minimizer at a faster $O(1/T)$ rate. When interpolation is only approximately satisfied, constant step-size AMSGrad converges to a neighbourhood of the solution at the same rate, while AdaGrad is robust to the violation of interpolation. However, even for simple convex problems satisfying interpolation, the empirical performance of both methods heavily depends on the step-size and requires tuning, questioning their adaptivity. We alleviate this problem by automatically determining the step-size using stochastic line-search or Polyak step-sizes. With these techniques, we prove that both AdaGrad and AMSGrad retain their convergence guarantees, without needing to know problem-dependent constants. Empirically, we demonstrate that these techniques improve the convergence and generalization of adaptive gradient methods across tasks, from binary classification with kernel mappings to multi-class classification with deep networks.


title: Probing dark photons with plasma haloscopes

abstract: Dark photons (DPs) produced in the early Universe are well-motivated dark matter (DM) candidates. We show that the recently proposed tunable plasma haloscopes are particularly advantageous for DP searches. While in-medium effects suppress the DP signal in conventional searches, plasma haloscopes make use of metamaterials that enable resonant absorption of the DP by matching its mass to a tunable plasma frequency and thus enable efficient plasmon production. Using thermal field theory, we confirm the in-medium DP absorption rate within the detector. This scheme allows to competitively explore a significant part of the DP DM parameter space in the DP mass range of $6-400$ $\mu$eV. If a signal is observed, the observation of a daily or annual modulation of the signal would be crucial to clearly identify the signal as due to DP DM and could shed light on the production mechanism.


title: Stuttering Conway Sequences Are Still Conway Sequences

abstract: A look-and-say sequence is obtained iteratively by reading off the digits of the current value, grouping identical digits together: starting with 1, the sequence reads: 1, 11, 21, 1211, 111221, 312211, etc. (OEIS A005150). Starting with any digit $d \neq 1$ gives Conway's sequence: $d$, $1d$, $111d$, $311d$, $13211d$, etc. (OEIS A006715). Conway popularised these sequences and studied some of their properties. In this paper we consider a variant subbed "look-and-say again" where digits are repeated twice. We prove that the look-and-say again sequence contains only the digits $1, 2, 4, 6, d$, where $d$ represents the starting digit. Such sequences decompose and the ratio of successive lengths converges to Conway's constant. In fact, these properties result from a commuting diagram between look-and-say again sequences and "classical" look-and-say sequences. Similar results apply to the "look-and-say three times" sequence.


title: A new relationship between Erd\H{o}s-R\'{e}nyi graphs, epidemic models and Brownian motion with parabolic drift

abstract: In the Reed-Frost model, an example of an SIR epidemic model, one can examine a statistic that counts the number of concurrently infected individuals. This statistic can be reformulated as a statistic on the \ER random graph $G(n,p)$. Within the critical window of Aldous and Martin-L\"{o}f, i.e. when $p = p(n) = n^{-1}+\lambda n^{-4/3}$, the cumulative sum of this statistic converges weakly to the integral of a Brownian motion with parabolic drift. This same statistic exhibits a deterministic scaling limit when $p = (1+\lambda \varepsilon_n)/n$ whenever $\varepsilon_n\to 0$ and $n^{1/3}\varepsilon_n\to\infty$.


title: Jamming and percolation of dimers in restricted-valence random sequential adsorption

abstract: Restricted-valence random sequential adsorption~(RSA) is studied in its pure and disordered versions, on the square and triangular lattices. For the simplest case~(pure on the square lattice) we prove the absence of percolation for maximum valence $V_{\rm max}=2$. In other cases, Monte Carlo simulations are used to investigate the percolation threshold, universality class, and jamming limit. Our results reveal a continuous transition for the majority of the cases studied. The percolation threshold is computed through finite-size scaling analysis of seven properties; its value increases with the average valency. Scaling plots and data-collapse analyses show that the transition belongs to the standard percolation universality class even in disordered cases


title: Nowcasting Covid-19 statistics reported withdelay: a case-study of Sweden

abstract: The new corona virus disease -- COVID-2019 -- is rapidly spreading through the world. The availability of unbiased timely statistics of trends in disease events are a key to effective responses. But due to reporting delays, the most recently reported numbers are frequently underestimating of the total number of infections, hospitalizations and deaths creating an illusion of a downward trend. Here we describe a statistical methodology for predicting true daily quantities and their uncertainty, estimated using historical reporting delays. The methodology takes into account the observed distribution pattern of the lag. It is derived from the removal method, a well-established estimation framework in the field of ecology.


title: Backdoors in Neural Models of Source Code

abstract: Deep neural networks are vulnerable to a range of adversaries. A particularly pernicious class of vulnerabilities are backdoors, where model predictions diverge in the presence of subtle triggers in inputs. An attacker can implant a backdoor by poisoning the training data to yield a desired target prediction on triggered inputs. We study backdoors in the context of deep-learning for source code. (1) We define a range of backdoor classes for source-code tasks and show how to poison a dataset to install such backdoors. (2) We adapt and improve recent algorithms from robust statistics for our setting, showing that backdoors leave a spectral signature in the learned representation of source code, thus enabling detection of poisoned data. (3) We conduct a thorough evaluation on different architectures and languages, showing the ease of injecting backdoors and our ability to eliminate them.


title: Tunable Photodetectors via in situ Thermal Conversion of TiS$_3$ to TiO$_2$

abstract: In two-dimensional materials research, oxidation is usually considered as a common source for the degradation of electronic and optoelectronic devices or even device failure. However, in some cases a controlled oxidation can open the possibility to widely tune the band structure of 2D materials. In particular, we demonstrate the controlled oxidation of titanium trisulfide (TiS$_3$), a layered semiconductor that has attracted much attention recently thanks to its quasi-1D electronic and optoelectronic properties and its direct bandgap of 1.1 eV. Heating TiS$_3$ in air above 300 {\deg}C gradually converts it into TiO$_2$, a semiconductor with a wide bandgap of 3.2 eV with ap-plications in photo-electrochemistry and catalysis. In this work, we investigate the controlled thermal oxidation of individual TiS$_3$ nanoribbons and its influence on the optoelectronic properties of TiS$_3$-based photodetectors. We observe a step-wise change in the cut-off wavelength from its pristine value ~1000 nm to 450 nm after subjecting the TiS$_3$ devices to subsequent thermal treatment cycles. Ab-initio and many-body calculations confirm an increase in the bandgap of titanium oxysulfide (TiO$_{2-x}$S$_x$) when increasing the amount of oxygen and reducing the amount of sulfur.


title: Robust Optimization and Inference on Manifolds

abstract: We propose a robust and scalable procedure for general optimization and inference problems on manifolds leveraging the classical idea of `median-of-means' estimation. This is motivated by ubiquitous examples and applications in modern data science in which a statistical learning problem can be cast as an optimization problem over manifolds. Being able to incorporate the underlying geometry for inference while addressing the need for robustness and scalability presents great challenges. We address these challenges by first proving a key lemma that characterizes some crucial properties of geometric medians on manifolds. In turn, this allows us to prove robustness and tighter concentration of our proposed final estimator in a subsequent theorem. This estimator aggregates a collection of subset estimators by taking their geometric median over the manifold. We illustrate bounds on this estimator via calculations in explicit examples. The robustness and scalability of the procedure is illustrated in numerical examples on both simulated and real data sets.


title: GOAL-DTU: Development of Distributed Intelligence for the Multi-Agent Programming Contest

abstract: We provide a brief description of the GOAL-DTU system for the agent contest, including the overall strategy and how the system is designed to apply this strategy. Our agents are implemented using the GOAL programming language. We evaluate the performance of our agents for the contest, and finally also discuss how to improve the system based on analysis of its strengths and weaknesses.


title: Higher Kiss Terms

abstract: We show that the modular term condition higher commutator is equal to the modular hypercommutator. As a consequence, we arrive at a new proof that HC8 holds for modular varieties. Next, we develop a procedure for a modular variety for producing the higher dimensional congruences that characterize the hypercommutator. This procedure allows us to demonstrate that every modular variety has an infinite sequence of what we call higher dimensional Kiss terms. We use these results to extend the scope of a theorem of Opr\v{s}al from permutable varieties to modular varieties.


title: Direct transformation of crystalline MoO$_3$ into few-layers MoS$_2$

abstract: We fabricate large-area atomically thin MoS$_2$ layers through the direct transformation of crystalline molybdenum MoS$_2$ (MoO$_3$) by sulfurization at relatively low temperatures. The obtained MoS2 sheets are polycrystalline (~10-20 nm single-crystal domain size) with areas of up to 300x300 um$^2$ with 2-4 layers in thickness and show a marked p-type behaviour. The synthesized films are characterized by a combination of complementary techniques: Raman spectroscopy, X-ray diffraction, transmission electron microscopy and electronic transport measurements.


title: Weak Quasicircles Have Lipschitz Dimension 1

abstract: We prove that the Lipschitz dimension of any bounded turning Jordan circle or arc is equal to 1. In particular, the Lipschitz dimension of any weak quasicircle or arc is equal to 1.


title: Getting a CLUE: A Method for Explaining Uncertainty Estimates

abstract: Both uncertainty estimation and interpretability are important factors for trustworthy machine learning systems. However, there is little work at the intersection of these two areas. We address this gap by proposing a novel method for interpreting uncertainty estimates from differentiable probabilistic models, like Bayesian Neural Networks (BNNs). Our method, Counterfactual Latent Uncertainty Explanations (CLUE), indicates how to change an input, while keeping it on the data manifold, such that a BNN becomes more confident about the input's prediction. We validate CLUE through 1) a novel framework for evaluating counterfactual explanations of uncertainty, 2) a series of ablation experiments, and 3) a user study. Our experiments show that CLUE outperforms baselines and enables practitioners to better understand which input patterns are responsible for predictive uncertainty.


title: New Rigid-foldable Developable Quadrilateral Creased Papers

abstract: This article extends the range of 1-DOF rigid-foldable developable quadrilateral creased papers. In a previous article, we put forward a sufficient and necessary condition for a quadrilateral creased paper to be rigid-foldable, and introduce a special sufficient condition that is convenient for practical use, generating quadrilateral creased paper by stitching basic units. In this article we develop new flat-foldable units and show how these can be used to construct a series of more complex rigid-foldable developable quadrilateral creased papers.


title: List Learning with Attribute Noise

abstract: We introduce and study the model of list learning with attribute noise. Learning with attribute noise was introduced by Shackelford and Volper (COLT 1988) as a variant of PAC learning, in which the algorithm has access to noisy examples and uncorrupted labels, and the goal is to recover an accurate hypothesis. Sloan (COLT 1988) and Goldman and Sloan (Algorithmica 1995) discovered information-theoretic limits to learning in this model, which have impeded further progress. In this article we extend the model to that of list learning, drawing inspiration from the list-decoding model in coding theory, and its recent variant studied in the context of learning. On the positive side, we show that sparse conjunctions can be efficiently list learned under some assumptions on the underlying ground-truth distribution. On the negative side, our results show that even in the list-learning model, efficient learning of parities and majorities is not possible regardless of the representation used.


title: Pore Scale Study of Gas Bubble Nucleation and Migration in Porous Media

abstract: The behavior of gas bubbles in porous media includes bubble nucleation and growth, migration, coalescence, and trapping. These processes are significantly affected by the pore-scale characteristics and heterogeneity of the sediment. In this study, experiments are performed using a microfluidic chip in which different gas bubble behavior mechanisms are observed. Based on the microscale experiments, eight major gas bubble behavior mechanisms are identified. In addition, a mesoscale experiment is designed and performed to study the impacts of soil layering and heterogeneity on the formation and distribution of biogenic gas generation via denitrification. The results show that the pore-scale characteristics of soil significantly affect the distribution and migration pattern of biogenic gas bubbles. As a result, the water saturation may vary locally between fully saturated (in fine sand), to about 80% in zones where the gas is allowed to migrate freely (coarse sand), to even close to 0% in the zone where the gas gets trapped under layers with higher air entry value.


title: Group-Fair Online Allocation in Continuous Time

abstract: The theory of discrete-time online learning has been successfully applied in many problems that involve sequential decision-making under uncertainty. However, in many applications including contractual hiring in online freelancing platforms and server allocation in cloud computing systems, the outcome of each action is observed only after a random and action-dependent time. Furthermore, as a consequence of certain ethical and economic concerns, the controller may impose deadlines on the completion of each task, and require fairness across different groups in the allocation of total time budget $B$. In order to address these applications, we consider continuous-time online learning problem with fairness considerations, and present a novel framework based on continuous-time utility maximization. We show that this formulation recovers reward-maximizing, max-min fair and proportionally fair allocation rules across different groups as special cases. We characterize the optimal offline policy, which allocates the total time between different actions in an optimally fair way (as defined by the utility function), and impose deadlines to maximize time-efficiency. In the absence of any statistical knowledge, we propose a novel online learning algorithm based on dual ascent optimization for time averages, and prove that it achieves $\tilde{O}(B^{-1/2})$ regret bound.


title: Maximal Objectives in the Multi-armed Bandit with Applications

abstract: In several applications of the stochastic multi-armed bandit problem, the traditional objective of maximizing the expected total reward can be inappropriate. In this paper, motivated by certain operational concerns in online platforms, we consider a new objective in the classical setup. Given $K$ arms, instead of maximizing the expected total reward from $T$ pulls (the traditional "sum" objective), we consider the vector of total rewards earned from each of the $K$ arms at the end of $T$ pulls and aim to maximize the expected highest total reward across arms (the "max" objective). For this objective, we show that any policy must incur an instance-dependent asymptotic regret of $\Omega(\log T)$ (with a higher instance-dependent constant compared to the traditional objective) and a worst-case regret of $\Omega(K^{1/3}T^{2/3})$. We then design an adaptive explore-then-commit policy featuring exploration based on appropriately tuned confidence bounds on the mean reward and an adaptive stopping criterion, which adapts to the problem difficulty and achieves these bounds (up to logarithmic factors). We then generalize our algorithmic insights to the problem of maximizing the expected value of the average total reward of the top $m$ arms with the highest total rewards. Our numerical experiments demonstrate the efficacy of our policies compared to several natural alternatives in practical parameter regimes. We discuss applications of these new objectives to the problem of grooming an adequate supply of value-providing market participants (workers/sellers/service providers) in online platforms.


title: Resonance locking in giant planets indicated by the rapid orbital expansion of Titan

abstract: Tidal effects in planetary systems are the main driver in the orbital migration of natural satellites. They result from physical processes occurring deep inside celestial bodies, whose effects are rarely observable from surface imaging. For giant planet systems, the tidal migration rate is determined by poorly understood dissipative processes in the planet, and standard theories suggest an orbital expansion rate inversely proportional to the power 11/2 in distance, implying little migration for outer moons such as Saturn's largest moon, Titan. Here, we use two independent measurements obtained with the Cassini spacecraft to measure Titan's orbital expansion rate. We find Titan migrates away from Saturn at 11.3 $\pm$ 2.0 cm/year, corresponding to a tidal quality factor of Saturn of Q $\simeq$ 100, and a migration timescale of roughly 10 Gyr. This rapid orbital expansion suggests Titan formed significantly closer to Saturn and has migrated outward to its current position. Our results for Titan and five other moons agree with the predictions of a resonance locking tidal theory, sustained by excitation of inertial waves inside the planet. The associated tidal expansion is only weakly sensitive to orbital distance, motivating a revision of the evolutionary history of Saturn's moon system. The resonance locking mechanism could operate in other systems such as stellar binaries and exoplanet systems, and it may allow for tidal dissipation to occur at larger orbital separations than previously believed.


title: Threshold for weak saturation stability

abstract: We study the weak $K_s$-saturation number of the Erd\H{o}s--R\'{e}nyi random graph $\mathbbmsl{G}(n, p)$, denoted by $\mathrm{wsat}(\mathbbmsl{G}(n, p), K_s)$, where $K_s$ is the complete graph on $s$ vertices. Kor\'{a}ndi and Sudakov in 2017 proved that the weak $K_s$-saturation number of $K_n$ is stable, in the sense that it remains the same after removing edges with constant probability. In this paper, we prove that there exists a threshold for this stability property and give upper and lower bounds on the threshold. This generalizes the result of Kor\'{a}ndi and Sudakov. A general upper bound for $\mathrm{wsat}(\mathbbmsl{G}(n, p), K_s)$ is also provided.


title: BanditPAM: Almost Linear Time $k$-Medoids Clustering via Multi-Armed Bandits

abstract: Clustering is a ubiquitous task in data science. Compared to the commonly used $k$-means clustering, $k$-medoids clustering requires the cluster centers to be actual data points and support arbitrary distance metrics, which permits greater interpretability and the clustering of structured objects. Current state-of-the-art $k$-medoids clustering algorithms, such as Partitioning Around Medoids (PAM), are iterative and are quadratic in the dataset size $n$ for each iteration, being prohibitively expensive for large datasets. We propose BanditPAM, a randomized algorithm inspired by techniques from multi-armed bandits, that reduces the complexity of each PAM iteration from $O(n^2)$ to $O(n \log n)$ and returns the same results with high probability, under assumptions on the data that often hold in practice. As such, BanditPAM matches state-of-the-art clustering loss while reaching solutions much faster. We empirically validate our results on several large real-world datasets, including a coding exercise submissions dataset, the 10x Genomics 68k PBMC single-cell RNA sequencing dataset, and the MNIST handwritten digits dataset. In these experiments, we observe that BanditPAM returns the same results as state-of-the-art PAM-like algorithms up to 4x faster while performing up to 200x fewer distance computations. The improvements demonstrated by BanditPAM enable $k$-medoids clustering on a wide range of applications, including identifying cell types in large-scale single-cell data and providing scalable feedback for students learning computer science online. We also release highly optimized Python and C++ implementations of our algorithm.


title: Are the beginning and ending phases of epidemics provided by next generation matrices? -- Revisiting drug sensitive and resistant tuberculosis model

abstract: In epidemiological modelings, the spectral radius of the next generation matrix evaluated at the trivial equilibrium was considered as the basic reproduction number. Also, the global stability of the trivial equilibrium point was determined by the left eigenvector associated to that next generation matrix. More recently, the fraction of susceptible individuals was also obtained from the next generation matrix. By revisiting drug sensitive and resistant tuberculosis model, the gross reproduction number and the fraction of susceptible individuals are calculated. Hence, the next generation matrices shed light to the evolution of the dynamics: the beginning of the epidemics via the reproduction number and the approaching to the epidemics level via the asymptotic fraction of susceptible individuals.


title: Kinetic Mixing, Dark Photons and Extra Dimensions III: Brane Localized Dark Matter

abstract: Extra dimensions have proven to be a very useful tool in constructing new physics models. In earlier work, we began investigating toy models for the 5-D analog of the kinetic mixing/vector portal scenario where the interactions of dark matter, taken to be, e.g., a complex scalar, with the brane-localized fields of the Standard Model (SM) are mediated by a massive $U(1)_D$ dark photon living in the bulk. These models were shown to have many novel features differentiating them from their 4-D analogs and which, in several cases, avoided some well-known 4-D model building constraints. However, these gains were obtained at the cost of the introduction of a fair amount of model complexity, e.g., dark matter Kaluza-Klein excitations. In the present paper, we consider an alternative setup wherein the dark matter and the dark Higgs, responsible for $U(1)_D$ breaking, are both localized to the 'dark' brane at the opposite end of the 5-D interval from where the SM fields are located with only the dark photon now being a 5-D field. The phenomenology of such a setup is explored for both flat and warped extra dimensions and compared to the previous more complex models.


title: On The Hecke Orbit Conjecture for PEL Type Shimura Varieties

abstract: The Hecke orbit conjecture asserts that every prime-to-$p$ Hecke orbit in a Shimura variety is dense in the central leaf containing it. In this paper, we prove the conjecture for certain irreducible components of Newton strata in Shimura varieties of PEL type A and C, when $p$ is an unramified prime of good reduction. Our approach generalizes Chai and Oort's method for Siegel modular varieties.


title: Observation of modulation instability and rogue breathers on stationary periodic waves

abstract: We present both theoretical description and experimental observation of the modulation instability process and related rogue breathers in the case of stationary periodic background waves, namely cnoidal and dnoidal envelopes. Despite being well-known solutions of the nonlinear Schrodinger equation, the stability of such background waves has remained unexplored experimentally until now, unlike the fundamental plane wave. By means of two experimental setups, namely, in nonlinear optics and hydrodynamics, we report on quantitative measurements of spontaneous modulation instability gain seeded by input random noise, as well as the formation of rogue breather solutions induced by a coherent perturbation. Our results confirm the generalization of modulation instability when more complex background waves are involved.


title: Robustness to Adversarial Attacks in Learning-Enabled Controllers

abstract: Learning-enabled controllers used in cyber-physical systems (CPS) are known to be susceptible to adversarial attacks. Such attacks manifest as perturbations to the states generated by the controller's environment in response to its actions. We consider state perturbations that encompass a wide variety of adversarial attacks and describe an attack scheme for discovering adversarial states. To be useful, these attacks need to be natural, yielding states in which the controller can be reasonably expected to generate a meaningful response. We consider shield-based defenses as a means to improve controller robustness in the face of such perturbations. Our defense strategy allows us to treat the controller and environment as black-boxes with unknown dynamics. We provide a two-stage approach to construct this defense and show its effectiveness through a range of experiments on realistic continuous control domains such as the navigation control-loop of an F16 aircraft and the motion control system of humanoid robots.


title: Deep Learning-based Stress Determinator for Mouse Psychiatric Analysis using Hippocampus Activity

abstract: Decoding neurons to extract information from transmission and employ them into other use is the goal of neuroscientists' study. Due to that the field of neuroscience is utilizing the traditional methods presently, we hence combine the state-of-the-art deep learning techniques with the theory of neuron decoding to discuss its potential of accomplishment. Besides, the stress level that is related to neuron activity in hippocampus is statistically examined as well. The experiments suggest that our state-of-the-art deep learning-based stress determinator provides good performance with respect to its model prediction accuracy and additionally, there is strong evidence against equivalence of mouse stress level under diverse environments.


title: Few-shot Neural Architecture Search

abstract: Efficient evaluation of a network architecture drawn from a large search space remains a key challenge in Neural Architecture Search (NAS). Vanilla NAS evaluates each architecture by training from scratch, which gives the true performance but is extremely time-consuming. Recently, one-shot NAS substantially reduces the computation cost by training only one supernetwork, a.k.a. supernet, to approximate the performance of every architecture in the search space via weight-sharing. However, the performance estimation can be very inaccurate due to the co-adaption among operations. In this paper, we propose few-shot NAS that uses multiple supernetworks, called sub-supernet, each covering different regions of the search space to alleviate the undesired co-adaption. Compared to one-shot NAS, few-shot NAS improves the accuracy of architecture evaluation with a small increase of evaluation cost. With only up to 7 sub-supernets, few-shot NAS establishes new SoTAs: on ImageNet, it finds models that reach 80.5% top-1 accuracy at 600 MB FLOPS and 77.5% top-1 accuracy at 238 MFLOPS; on CIFAR10, it reaches 98.72% top-1 accuracy without using extra data or transfer learning. In Auto-GAN, few-shot NAS outperforms the previously published results by up to 20%. Extensive experiments show that few-shot NAS significantly improves various one-shot methods, including 4 gradient-based and 6 search-based methods on 3 different tasks in NasBench-201 and NasBench1-shot-1.


title: Nonrigid registration using Gaussian processes and local likelihood estimation

abstract: Surface registration, the task of aligning several multidimensional point sets, is a necessary task in many scientific fields. In this work, a novel statistical approach is developed to solve the problem of nonrigid registration. While the application of an affine transformation results in rigid registration, using a general nonlinear function to achieve nonrigid registration is necessary when the point sets require deformations that change over space. The use of a local likelihood-based approach using windowed Gaussian processes provides a flexible way to accurately estimate the nonrigid deformation. This strategy also makes registration of massive data sets feasible by splitting the data into many subsets. The estimation results yield spatially-varying local rigid registration parameters. Gaussian process surface models are then fit to the parameter fields, allowing prediction of the transformation parameters at unestimated locations, specifically at observation locations in the unregistered data set. Applying these transformations results in a global, nonrigid registration. A penalty on the transformation parameters is included in the likelihood objective function. Combined with smoothing of the local estimates from the surface models, the nonrigid registration model can prevent the problem of overfitting. The efficacy of the nonrigid registration method is tested in two simulation studies, varying the number of windows and number of points, as well as the type of deformation. The nonrigid method is applied to a pair of massive remote sensing elevation data sets exhibiting complex geological terrain, with improved accuracy and uncertainty quantification in a cross validation study versus two rigid registration methods.


title: Exploring Algorithmic Fairness in Robust Graph Covering Problems

abstract: Fueled by algorithmic advances, AI algorithms are increasingly being deployed in settings subject to unanticipated challenges with complex social effects. Motivated by real-world deployment of AI driven, social-network based suicide prevention and landslide risk management interventions, this paper focuses on robust graph covering problems subject to group fairness constraints. We show that, in the absence of fairness constraints, state-of-the-art algorithms for the robust graph covering problem result in biased node coverage: they tend to discriminate individuals (nodes) based on membership in traditionally marginalized groups. To mitigate this issue, we propose a novel formulation of the robust graph covering problem with group fairness constraints and a tractable approximation scheme applicable to real-world instances. We provide a formal analysis of the price of group fairness (PoF) for this problem, where we show that uncertainty can lead to greater PoF. We demonstrate the effectiveness of our approach on several real-world social networks. Our method yields competitive node coverage while significantly improving group fairness relative to state-of-the-art methods.


title: GPU-Accelerated Periodic Source Identification in Large-Scale Surveys: Measuring $P$ and $\dot{P}$

abstract: Many inspiraling and merging stellar remnants emit both gravitational and electromagnetic radiation as they orbit or collide. These gravitational wave events together with their associated electromagnetic counterparts provide insight about the nature of the merger, allowing us to further constrain properties of the binary. With the future launch of the Laser Interferometer Space Antenna (LISA), follow up observations and models are needed of ultracompact binary (UCB) systems. Current and upcoming long baseline time domain surveys will observe many of these UCBs. We present a new fast periodic object search tool capable of searching for generic periodic signals based on the Conditional Entropy algorithm. This new implementation allows for a grid search over both the period ($P$) and the time derivative of the period ($\dot{P}$). To demonstrate the usage of this tool, we use a small, hand-picked subset of a UCB population generated from the population synthesis code \cosmic, as well as a custom catalog for varying periods at fixed intrinsic parameters. We simulate light curves as likely to be observed by future time domain surveys by using an existing eclipsing binary light curve model accounting for the change in orbital period due to gravitational radiation. We find that a search with $\dot{P}$ values is necessary for detecting binaries at orbital periods less than $\sim$10 min. We also show it is useful in finding and characterizing binaries with longer periods, but at a higher computational cost. Our code is called gce (GPU-Accelerated Conditional Entropy). It is available at https://github.com/mikekatz04/gce.


title: Detection of Novel Social Bots by Ensembles of Specialized Classifiers

abstract: Malicious actors create inauthentic social media accounts controlled in part by algorithms, known as social bots, to disseminate misinformation and agitate online discussion. While researchers have developed sophisticated methods to detect abuse, novel bots with diverse behaviors evade detection. We show that different types of bots are characterized by different behavioral features. As a result, supervised learning techniques suffer severe performance deterioration when attempting to detect behaviors not observed in the training data. Moreover, tuning these models to recognize novel bots requires retraining with a significant amount of new annotations, which are expensive to obtain. To address these issues, we propose a new supervised learning method that trains classifiers specialized for each class of bots and combines their decisions through the maximum rule. The ensemble of specialized classifiers (ESC) can better generalize, leading to an average improvement of 56\% in F1 score for unseen accounts across datasets. Furthermore, novel bot behaviors are learned with fewer labeled examples during retraining. We deployed ESC in the newest version of Botometer, a popular tool to detect social bots in the wild, with a cross-validation AUC of 0.99.


title: SegNBDT: Visual Decision Rules for Segmentation

abstract: The black-box nature of neural networks limits model decision interpretability, in particular for high-dimensional inputs in computer vision and for dense pixel prediction tasks like segmentation. To address this, prior work combines neural networks with decision trees. However, such models (1) perform poorly when compared to state-of-the-art segmentation models or (2) fail to produce decision rules with spatially-grounded semantic meaning. In this work, we build a hybrid neural-network and decision-tree model for segmentation that (1) attains neural network segmentation accuracy and (2) provides semi-automatically constructed visual decision rules such as "Is there a window?". We obtain semantic visual meaning by extending saliency methods to segmentation and attain accuracy by leveraging insights from neural-backed decision trees, a deep learning analog of decision trees for image classification. Our model SegNBDT attains accuracy within ~2-4% of the state-of-the-art HRNetV2 segmentation model while also retaining explainability; we achieve state-of-the-art performance for explainable models on three benchmark datasets -- Pascal-Context (49.12%), Cityscapes (79.01%), and Look Into Person (51.64%). Furthermore, user studies suggest visual decision rules are more interpretable, particularly for incorrect predictions. Code and pretrained models can be found at https://github.com/daniel-ho/SegNBDT.


title: Feudal Steering: Hierarchical Learning for Steering Angle Prediction

abstract: We consider the challenge of automated steering angle prediction for self driving cars using egocentric road images. In this work, we explore the use of feudal networks, used in hierarchical reinforcement learning (HRL), to devise a vehicle agent to predict steering angles from first person, dash-cam images of the Udacity driving dataset. Our method, Feudal Steering, is inspired by recent work in HRL consisting of a manager network and a worker network that operate on different temporal scales and have different goals. The manager works at a temporal scale that is relatively coarse compared to the worker and has a higher level, task-oriented goal space. Using feudal learning to divide the task into manager and worker sub-networks provides more accurate and robust prediction. Temporal abstraction in driving allows more complex primitives than the steering angle at a single time instance. Composite actions comprise a subroutine or skill that can be re-used throughout the driving sequence. The associated subroutine id is the manager network's goal, so that the manager seeks to succeed at the high level task (e.g. a sharp right turn, a slight right turn, moving straight in traffic, or moving straight unencumbered by traffic). The steering angle at a particular time instance is the worker network output which is regulated by the manager's high level task. We demonstrate state-of-the art steering angle prediction results on the Udacity dataset.


title: Multi-Agent Informational Learning Processes

abstract: We introduce a new mathematical model of multi-agent reinforcement learning, the Multi-Agent Informational Learning Processor "MAILP" model. The model is based on the notion that agents have policies for a certain amount of information, models how this information iteratively evolves and propagates through many agents. This model is very general, and the only meaningful assumption made is that learning for individual agents progressively slows over time.


title: Exact resummation of the Holstein-Primakoff expansion and differential equation approach to operator square-roots

abstract: Operator square-roots are ubiquitous in theoretical physics. They appear, for example, in the Holstein-Primakoff representation of spin operators and in the Klein-Gordon equation. Often the use of a perturbative expansion is the only recourse when dealing with them. In this work we show that under certain conditions differential equations can be derived which can be used to find perturbatively inaccessible approximations to operator square-roots. Specifically, for the number operator $\hat n=\hat a^\dag a$ we show that the square-root $\sqrt{\hat n}$ near $\hat n=0$ can be approximated by a polynomial in $\hat n$. This result is unexpected because a Taylor expansion fails. A polynomial expression in $\hat n$ is possible because $\hat n$ is an operator, and its constituents $a$ and $a^\dag$ have a non-trivial commutator $[a,a^\dag]=1$ and do not behave as scalars. We apply our approach to the zero mass Klein-Gordon Hamiltonian in a constant magnetic field, and as a main application, the Holstein-Primakoff representation of spin operators, where we are able to find new expressions that are polynomial in bosonic operators. We prove that these new expressions exactly reproduce spin operators. Our expressions are manifestly Hermitian, which offer an advantage over other methods, such as the Dyson-Maleev representation.


title: The entropy of Hawking radiation

abstract: In this review, we describe recent progress on the black hole information problem that involves a new understanding of how to calculate the entropy of Hawking radiation. We show how the method for computing gravitational fine-grained entropy, developed over the past 15 years, can be extended to capture the entropy of Hawking radiation. This technique reveals large corrections needed for the entropy to be consistent with unitary black hole evaporation.


title: FastPitch: Parallel Text-to-speech with Pitch Prediction

abstract: We present FastPitch, a fully-parallel text-to-speech model based on FastSpeech, conditioned on fundamental frequency contours. The model predicts pitch contours during inference. By altering these predictions, the generated speech can be more expressive, better match the semantic of the utterance, and in the end more engaging to the listener. Uniformly increasing or decreasing pitch with FastPitch generates speech that resembles the voluntary modulation of voice. Conditioning on frequency contours improves the overall quality of synthesized speech, making it comparable to state-of-the-art. It does not introduce an overhead, and FastPitch retains the favorable, fully-parallel Transformer architecture, with over 900x real-time factor for mel-spectrogram synthesis of a typical utterance.


title: Learning to Play by Imitating Humans

abstract: Acquiring multiple skills has commonly involved collecting a large number of expert demonstrations per task or engineering custom reward functions. Recently it has been shown that it is possible to acquire a diverse set of skills by self-supervising control on top of human teleoperated play data. Play is rich in state space coverage and a policy trained on this data can generalize to specific tasks at test time outperforming policies trained on individual expert task demonstrations. In this work, we explore the question of whether robots can learn to play to autonomously generate play data that can ultimately enhance performance. By training a behavioral cloning policy on a relatively small quantity of human play, we autonomously generate a large quantity of cloned play data that can be used as additional training. We demonstrate that a general purpose goal-conditioned policy trained on this augmented dataset substantially outperforms one trained only with the original human data on 18 difficult user-specified manipulation tasks in a simulated robotic tabletop environment. A video example of a robot imitating human play can be seen here: https://learning-to-play.github.io/videos/undirected_play1.mp4


title: The Smoothed Possibility of Social Choice

abstract: We develop a framework that leverages the smoothed complexity analysis by Spielman and Teng to circumvent paradoxes and impossibility theorems in social choice, motivated by modern applications of social choice powered by AI and ML. For Condrocet's paradox, we prove that the smoothed likelihood of the paradox either vanishes at an exponential rate as the number of agents increases, or does not vanish at all. For the ANR impossibility on the non-existence of voting rules that simultaneously satisfy anonymity, neutrality, and resolvability, we characterize the rate for the impossibility to vanish, to be either polynomially fast or exponentially fast. We also propose a novel easy-to-compute tie-breaking mechanism that optimally preserves anonymity and neutrality for even number of alternatives in natural settings. Our results illustrate the smoothed possibility of social choice -- even though the paradox and the impossibility theorem hold in the worst case, they may not be a big concern in practice.


title: Separable algebras and coflasque resolutions

abstract: Over a non-closed field, it is a common strategy to use separable algebras as invariants to distinguish algebraic and geometric objects. The most famous example is the deep connection between Severi-Brauer varieties and central simple algebras. For more general varieties, one might use endomorphism algebras of line bundles, of indecomposable vector bundles, or of exceptional objects in their derived categories. Using Galois cohomology, we describe a new invariant of reductive algebraic groups that captures precisely when this strategy will fail. Our main result characterizes this invariant in terms of coflasque resolutions of linear algebraic groups introduced by Colliot-Th\'el\`ene. We determine whether or not this invariant is trivial for many fields. For number fields, we show it agrees with the Tate-Shafarevich group of the linear algebraic group, up to behavior at real places.


title: High-Precision Extraction of Emerging Concepts from Scientific Literature

abstract: Identification of new concepts in scientific literature can help power faceted search, scientific trend analysis, knowledge-base construction, and more, but current methods are lacking. Manual identification cannot keep up with the torrent of new publications, while the precision of existing automatic techniques is too low for many applications. We present an unsupervised concept extraction method for scientific literature that achieves much higher precision than previous work. Our approach relies on a simple but novel intuition: each scientific concept is likely to be introduced or popularized by a single paper that is disproportionately cited by subsequent papers mentioning the concept. From a corpus of computer science papers on arXiv, we find that our method achieves a Precision@1000 of 99%, compared to 86% for prior work, and a substantially better precision-yield trade-off across the top 15,000 extractions. To stimulate research in this area, we release our code and data (https://github.com/allenai/ForeCite).


title: Optimization Theory for ReLU Neural Networks Trained with Normalization Layers

abstract: The success of deep neural networks is in part due to the use of normalization layers. Normalization layers like Batch Normalization, Layer Normalization and Weight Normalization are ubiquitous in practice, as they improve generalization performance and speed up training significantly. Nonetheless, the vast majority of current deep learning theory and non-convex optimization literature focuses on the un-normalized setting, where the functions under consideration do not exhibit the properties of commonly normalized neural networks. In this paper, we bridge this gap by giving the first global convergence result for two-layer neural networks with ReLU activations trained with a normalization layer, namely Weight Normalization. Our analysis shows how the introduction of normalization layers changes the optimization landscape and can enable faster convergence as compared with un-normalized neural networks.


title: Active Sampling for Min-Max Fairness

abstract: We propose simple active sampling and reweighting strategies for optimizing min-max fairness that can be applied to any classification or regression model learned via loss minimization. The key intuition behind our approach is to use at each timestep a datapoint from the group that is worst off under the current model for updating the model. The ease of implementation and the generality of our robust formulation make it an attractive option for improving model performance on disadvantaged groups. For convex learning problems, such as linear or logistic regression, we provide a fine-grained analysis, proving the rate of convergence to a min-max fair solution.


title: Reintroducing Straight-Through Estimators as Principled Methods for Stochastic Binary Networks

abstract: Training neural networks with binary weights and activations is a challenging problem due to the lack of gradients and difficulty of optimization over discrete weights. Many successful experimental results have been achieved with empirical straight-through (ST) approaches, proposing a variety of ad-hoc rules for propagating gradients through non-differentiable activations and updating discrete weights. At the same time, ST methods can be truly derived as estimators in the stochastic binary network (SBN) model with Bernoulli weights. We advance these derivations to a more complete and systematic study. We analyze properties, estimation accuracy, obtain different forms of correct ST estimators for activations and weights, explain existing empirical approaches and their shortcomings, explain how latent weights arise from the mirror descent method when optimizing over probabilities. This allows to reintroduce ST methods, long known empirically, as sound approximations, apply them with clarity and develop further improvements.


title: A pandemic-resilient open-inquiry physical science lab course which leverages the Maker movement

abstract: Without any major changes, a pilot version of a physical science lab course was able to continue when the COVID-19 crisis necessitated the abrupt suspension of on-campus education. The 'Maker Lab' course, in which students conceive and set up their own experiments using affordable microcontrollers, required students to follow the entire arc of an empirical research cycle twice. The decision for and facilitation of such open-inquiry projects was based on and warranted by the literature on teaching the process of experimental research and methods of scientific research. The flipped classroom approach was used, where contact time is devoted to discussions and the students' actual experiments were carried out independently at home or elsewhere without the in-person supervision of an instructor. Despite the COVID-19 measures, all students were able to produce interesting and successful research projects. While there were of course difficulties encountered due to the abrupt transition to online teaching, we found several counterbalancing advantages that bear consideration for inclusion even when all teaching activities can return to campus. We believe that three components in the design of the course were vital to the resilience of the course: the choice for fully open-inquiry projects, the decision to use Arduinos as measurement tools, and the flipped aspect of the instruction methods. We also include considerations for adapting these pandemic-resilient methods in other courses and programs.


title: Rethinking Pre-training and Self-training

abstract: Pre-training is a dominant paradigm in computer vision. For example, supervised ImageNet pre-training is commonly used to initialize the backbones of object detection and segmentation models. He et al., however, show a surprising result that ImageNet pre-training has limited impact on COCO object detection. Here we investigate self-training as another method to utilize additional data on the same setup and contrast it against ImageNet pre-training. Our study reveals the generality and flexibility of self-training with three additional insights: 1) stronger data augmentation and more labeled data further diminish the value of pre-training, 2) unlike pre-training, self-training is always helpful when using stronger data augmentation, in both low-data and high-data regimes, and 3) in the case that pre-training is helpful, self-training improves upon pre-training. For example, on the COCO object detection dataset, pre-training benefits when we use one fifth of the labeled data, and hurts accuracy when we use all labeled data. Self-training, on the other hand, shows positive improvements from +1.3 to +3.4AP across all dataset sizes. In other words, self-training works well exactly on the same setup that pre-training does not work (using ImageNet to help COCO). On the PASCAL segmentation dataset, which is a much smaller dataset than COCO, though pre-training does help significantly, self-training improves upon the pre-trained model. On COCO object detection, we achieve 54.3AP, an improvement of +1.5AP over the strongest SpineNet model. On PASCAL segmentation, we achieve 90.5 mIOU, an improvement of +1.5% mIOU over the previous state-of-the-art result by DeepLabv3+.


title: Spin gapless semiconductors

abstract: Spin gapless semiconductors (SGSs) are a new class of zero gap materials which have a fully spin polarised electrons and holes. They bridge zero gap materials and half-metals. The band structures of the SGSs can have two types of energy dispersions: Dirac linear dispersion and parabolic dispersion. The Dirac type SGSs exhibit fully spin polarized Dirac cones, and offer a platform for massless and fully spin polarized spintronics as well as dissipationless edge state via quantum anomalous Hall effect. Due to its fascinating spin and charge states, they hold great potential application in spintronics. There have been tremendous efforts worldwide on searching for suitable candidates of SGSs. In particularly, there is an increasing interest in searching for Dirac type SGSs. In the past decade, a large number of Dirac or parabolic type SGSs have been predicted by density functional theory and some of parabolic SGSs have been experimentally demonstrated. The SGSs hold great potential for high speed and low-energy consumption spintronics, electronics and optoelectronics. Here, we review both Dirac and parabolic types of SGSs in different materials systems and outline the concepts of SGSs, novel spin and charge states, and potential applications of SGSs in next generation spintronic devices.


title: Casimir effect in conformally flat spacetimes

abstract: We discuss several approaches to determine the Casimir force in inertial frames of reference in different dimensions. On an example of a simple model involving mirrors in Rindler spacetime we show that Casimir's and Lifschitz's methods are inequivalent and only latter can be generalized to other spacetime geometries. For conformally coupled fields we derive the Casimir force in conformally flat spacetimes utilizing an anomaly and provide explicit examples in the Friedmann-Lemaitre-Robertson-Walker $(k=0)$ models.


title: Uncovering the Folding Landscape of RNA Secondary Structure with Deep Graph Embeddings

abstract: Biomolecular graph analysis has recently gained much attention in the emerging field of geometric deep learning. Here we focus on organizing biomolecular graphs in ways that expose meaningful relations and variations between them. We propose a geometric scattering autoencoder (GSAE) network for learning such graph embeddings. Our embedding network first extracts rich graph features using the recently proposed geometric scattering transform. Then, it leverages a semi-supervised variational autoencoder to extract a low-dimensional embedding that retains the information in these features that enable prediction of molecular properties as well as characterize graphs. We show that GSAE organizes RNA graphs both by structure and energy, accurately reflecting bistable RNA structures. Also, the model is generative and can sample new folding trajectories.


title: Can you take Akemann--Weaver's $\diamondsuit_{\aleph_1}$ away?

abstract: By Glimm's dichotomy, a separable, simple $\textrm{C}^*$-algebra has continuum-many unitarily inequivalent irreducible representations if, and only if, it is non-type I while all of its irreducible representations are unitarily equivalent if, and only if, it is type I. Naimark asked whether the latter equivalence holds for all $\textrm{C}^*$-algebras. In 2004, Akemann and Weaver gave a negative answer to Naimark's problem, using Jensen's diamond axiom $\diamondsuit_{\aleph_1}$, a powerful diagonalization principle that implies the Continuum Hypothesis ($\mathsf{CH}$). By a result of Rosenberg, a separably represented simple $\textrm{C}^*$-algebra with a unique irreducible representation is necessarily of type I. We show that this result is sharp by constructing an example of a separably represented, simple $\textrm{C}^*$-algebra that has exactly two inequivalent irreducible representations, and therefore does not satisfy the conclusion of Glimm's dichotomy. Our construction uses a weakening of Jensen's $\diamondsuit_{\aleph_1}$, denoted $\diamondsuit^\mathsf{Cohen}$, that holds in the original Cohen's model for the negation of $\mathsf{CH}$. We also prove that $\diamondsuit^\mathsf{Cohen}$ suffices to give a negative answer to Naimark's problem. Our main technical tool is a forcing notion that generically adds an automorphism of a given $\textrm{C}^*$-algebra with a prescribed action on its space of pure states.


title: Stochastic Optimization for Performative Prediction

abstract: In performative prediction, the choice of a model influences the distribution of future data, typically through actions taken based on the model's predictions. We initiate the study of stochastic optimization for performative prediction. What sets this setting apart from traditional stochastic optimization is the difference between merely updating model parameters and deploying the new model. The latter triggers a shift in the distribution that affects future data, while the former keeps the distribution as is. Assuming smoothness and strong convexity, we prove rates of convergence for both greedily deploying models after each stochastic update (greedy deploy) as well as for taking several updates before redeploying (lazy deploy). In both cases, our bounds smoothly recover the optimal $O(1/k)$ rate as the strength of performativity decreases. Furthermore, they illustrate how depending on the strength of performative effects, there exists a regime where either approach outperforms the other. We experimentally explore the trade-off on both synthetic data and a strategic classification simulator.


title: SemifreddoNets: Partially Frozen Neural Networks for Efficient Computer Vision Systems

abstract: We propose a system comprised of fixed-topology neural networks having partially frozen weights, named SemifreddoNets. SemifreddoNets work as fully-pipelined hardware blocks that are optimized to have an efficient hardware implementation. Those blocks freeze a certain portion of the parameters at every layer and replace the corresponding multipliers with fixed scalers. Fixing the weights reduces the silicon area, logic delay, and memory requirements, leading to significant savings in cost and power consumption. Unlike traditional layer-wise freezing approaches, SemifreddoNets make a profitable trade between the cost and flexibility by having some of the weights configurable at different scales and levels of abstraction in the model. Although fixing the topology and some of the weights somewhat limits the flexibility, we argue that the efficiency benefits of this strategy outweigh the advantages of a fully configurable model for many use cases. Furthermore, our system uses repeatable blocks, therefore it has the flexibility to adjust model complexity without requiring any hardware change. The hardware implementation of SemifreddoNets provides up to an order of magnitude reduction in silicon area and power consumption as compared to their equivalent implementation on a general-purpose accelerator.


title: Fast Objective & Duality Gap Convergence for Non-Convex Strongly-Concave Min-Max Problems with PL Condition

abstract: This paper focuses on stochastic methods for solving smooth non-convex strongly-concave min-max problems, which have received increasing attention due to their potential applications in deep learning (e.g., deep AUC maximization, distributionally robust optimization). However, most of the existing algorithms are slow in practice, and their analysis revolves around the convergence to a nearly stationary point.We consider leveraging the Polyak-Lojasiewicz (PL) condition to design faster stochastic algorithms with stronger convergence guarantee. Although PL condition has been utilized for designing many stochastic minimization algorithms, their applications for non-convex min-max optimization remain rare. In this paper, we propose and analyze a generic framework of proximal stage-based method with many well-known stochastic updates embeddable. Fast convergence is established in terms of both the primal objective gap and the duality gap. Compared with existing studies, (i) our analysis is based on a novel Lyapunov function consisting of the primal objective gap and the duality gap of a regularized function, and (ii) the results are more comprehensive with improved rates that have better dependence on the condition number under different assumptions. We also conduct deep and non-deep learning experiments to verify the effectiveness of our methods.


title: EMOGI: Efficient Memory-access for Out-of-memory Graph-traversal In GPUs

abstract: Modern analytics and recommendation systems are increasingly based on graph data that capture the relations between entities being analyzed. Practical graphs come in huge sizes, offer massive parallelism, and are stored in sparse-matrix formats such as CSR. To exploit the massive parallelism, developers are increasingly interested in using GPUs for graph traversal. However, due to their sizes, graphs often do not fit into the GPU memory. Prior works have either used input data pre-processing/partitioning or UVM to migrate chunks of data from the host memory to the GPU memory. However, the large, multi-dimensional, and sparse nature of graph data presents a major challenge to these schemes and results in significant amplification of data movement and reduced effective data throughput. In this work, we propose EMOGI, an alternative approach to traverse graphs that do not fit in GPU memory using direct cacheline-sized access to data stored in host memory. This paper addresses the open question of whether a sufficiently large number of overlapping cacheline-sized accesses can be sustained to 1) tolerate the long latency to host memory, 2) fully utilize the available bandwidth, and 3) achieve favorable execution performance. We analyze the data access patterns of several graph traversal applications in GPU over PCIe using an FPGA to understand the cause of poor external bandwidth utilization. By carefully coalescing and aligning external memory requests, we show that we can minimize the number of PCIe transactions and nearly fully utilize the PCIe bandwidth even with direct cache-line accesses to the host memory. EMOGI achieves 2.92$\times$ speedup on average compared to the optimized UVM implementations in various graph traversal applications. We also show that EMOGI scales better than a UVM-based solution when the system uses higher bandwidth interconnects such as PCIe 4.0.


title: Thick string-like braneworlds in f(T) gravity

abstract: We propose a codimension two warped braneworld model within the teleparallel $f(T)$ gravity. By assuming a global vortex as the source, we found a $l=0$ vortex solution that yields to a thick string-like brane. Asymptotically, the bulk geometry converges to an $AdS_6$ spacetime whose cosmological constant is produced by the torsion parameters. Furthermore, the torsion induces an $AdS-dS$ transition on the exterior region. Inside the brane core, the torsion produces an internal structure even for a single complex scalar field. As the torsion parameters vary, the brane undergoes a phase transition leading to the formation of ring-like structures. The bulk-brane Planck mass ration is modified by the torsion. The analysis of the stress energy condition reveals a splitting brane process satisfying the weak and strong energy conditions for some values of the parameters. In addition, we investigate the behaviour of the gravitational perturbations in this scenario. It turns out that the gravitational spectrum has a linear behaviour for small masses and is independent of the torsion parameters for large masses. In the bulk, the torsion keeps a gapless non-localizable and stable tower of massive modes. Inside the brane core the torsion produces new barriers and potential wells leading to small amplitude massive modes and a massless mode localized around the ring structures.


title: Laser-driven Collisionless Shock Acceleration of Ions from Near-critical plasmas

abstract: This paper overviews experimental and numerical results on acceleration of narrow energy spread ion beams by an electrostatic collisionless shockwave driven by 1 um (Omega EP) and 10 um (UCLA Neptune Laboratory) lasers in near critical density CH and He plasmas, respectively. Shock waves in CH targets produced high-energy 50 MeV protons (energy spread of <30%) and 314 MeV C6+ ions (energy spread of <10%). Observation of acceleration of both protons and carbon ions to similar velocities is consistent with reflection of particles off the moving potential of a shock front. For shocks driven by CO2 laser in a gas jet, 30 MeV peak in He ion spectrum was detected. Particle-in-cell simulations indicate that regardless of the target further control over its density profile is needed for optimization of accelerated ion beams in part of energy spread, yield and maximum kinetic energy.


title: Online Sequential Extreme Learning Machines: Features Combined From Hundreds of Midlayers

abstract: In this paper, we develop an algorithm called hierarchal online sequential learning algorithm (H-OS-ELM) for single feed feedforward network with features combined from hundreds of midlayers, the algorithm can learn chunk by chunk with fixed or varying block size, we believe that the diverse selectivity of neurons in top layers which consists of encoded distributed information produced by the other neurons offers better computational advantage over inference accuracy. Thus this paper proposes a Hierarchical model framework combined with Online-Sequential learning algorithm, Firstly the model consists of subspace feature extractor which consists of subnetwork neuron, using the sub-features which is result of the feature extractor in first layer of the hierarchy we get rid of irrelevant factors which are of no use for the learning and iterate this process so that to recast the the subfeatures into the hierarchical model to be processed into more acceptable cognition. Secondly by using OS-Elm we are using non-iterative style for learning we are implementing a network which is wider and shallow which plays a important role in generalizing the overall performance which in turn boosts up the learning speed


title: Google Dataset Search by the Numbers

abstract: Scientists, governments, and companies increasingly publish datasets on the Web. Google's Dataset Search extracts dataset metadata -- expressed using schema.org and similar vocabularies -- from Web pages in order to make datasets discoverable. Since we started the work on Dataset Search in 2016, the number of datasets described in schema.org has grown from about 500K to almost 30M. Thus, this corpus has become a valuable snapshot of data on the Web. To the best of our knowledge, this corpus is the largest and most diverse of its kind. We analyze this corpus and discuss where the datasets originate from, what topics they cover, which form they take, and what people searching for datasets are interested in. Based on this analysis, we identify gaps and possible future work to help make data more discoverable.


title: Injecting Reliable Radio Frequency Fingerprints Using Metasurface for The Internet of Things

abstract: In Internet of Things, where billions of devices with limited resources are communicating with each other, security has become a major stumbling block affecting the progress of this technology. Existing authentication schemes-based on digital signatures have overhead costs associated with them in terms of computation time, battery power, bandwidth, memory, and related hardware costs. Radio frequency fingerprint (RFF), utilizing the unique device-based information, can be a promising solution for IoT. However, traditional RFFs have become obsolete because of low reliability and reduced user capability. Our proposed solution, Metasurface RF-Fingerprinting Injection (MeRFFI), is to inject a carefully-designed radio frequency fingerprint into the wireless physical layer that can increase the security of a stationary IoT device with minimal overhead. The injection of fingerprint is implemented using a low cost metasurface developed and fabricated in our lab, which is designed to make small but detectable perturbations in the specific frequency band in which the IoT devices are communicating. We have conducted comprehensive system evaluations including distance, orientation, multiple channels where the feasibility, effectiveness, and reliability of these fingerprints are validated. The proposed MeRFFI system can be easily integrated into the existing authentication schemes. The security vulnerabilities are analyzed for some of the most threatening wireless physical layer-based attacks.


title: A New Perspective on Learning Context-Specific Independence

abstract: Local structure such as context-specific independence (CSI) has received much attention in the probabilistic graphical model (PGM) literature, as it facilitates the modeling of large complex systems, as well as for reasoning with them. In this paper, we provide a new perspective on how to learn CSIs from data. We propose to first learn a functional and parameterized representation of a conditional probability table (CPT), such as a neural network. Next, we quantize this continuous function, into an arithmetic circuit representation that facilitates efficient inference. In the first step, we can leverage the many powerful tools that have been developed in the machine learning literature. In the second step, we exploit more recently-developed analytic tools from explainable AI, for the purposes of learning CSIs. Finally, we contrast our approach, empirically and conceptually, with more traditional variable-splitting approaches, that search for CSIs more explicitly.


title: MCMC Should Mix: Learning Energy-Based Model with Neural Transport Latent Space MCMC

abstract: Learning energy-based model (EBM) requires MCMC sampling of the learned model as an inner loop of the learning algorithm. However, MCMC sampling of EBMs in high-dimensional data space is generally not mixing, because the energy function, which is usually parametrized by a deep network, is highly multi-modal in the data space. This is a serious handicap for both theory and practice of EBMs. In this paper, we propose to learn an EBM with a flow-based model (or in general a latent variable model) serving as a backbone, so that the EBM is a correction or an exponential tilting of the flow-based model. We show that the model has a particularly simple form in the space of the latent variables of the backbone model, and MCMC sampling of the EBM in the latent space mixes well and traverses modes in the data space. This enables proper sampling and learning of EBMs.


title: Some polynomial maps with Jacobian rank two or three

abstract: In the paper, we first classify all polynomial maps of the form $H=(u(x,y,z),v(x,y,z), h(x,y))$ in the case that $JH$ is nilpotent and $\deg_zv\leq 1$. After that, we generalize the structure of $H$ to $H=\big(H_1(x_1,x_2,\ldots,x_n),\allowbreak b_3x_3+\cdots+b_nx_n+H_2^{(0)}(x_2),H_3(x_1,x_2),\ldots,H_n(x_1,x_2)\big)$.


title: JWST Transit Spectra I: Exploring Potential Biases and Opportunities in Retrievals of Tidally-locked Hot Jupiters with Clouds and Hazes

abstract: Many of the exoplanets for which we can obtain the highest SNR transit spectra are tidally locked. The atmospheres on tidally-locked planets likely exhibit large differences between the day and night side of the planet, the poles, and the morning versus evening terminators. In this paper, we illustrate how the combined effects of aerosols and day-night temperature gradients shape transit spectra of tidally-locked exoplanets when full 3D structures are taken into account and evaluate the implications for retrievals of atmospheric properties. To do this, we have developed a new code, METIS, which can compute transit spectra for an arbitrary longitude-latitude-altitude grid of temperatures and pressures. Using METIS, we pair flexible treatments of clouds and hazes with simple parameterized day-night temperature gradients to compute transit spectra and perform retrieval experiments across a wide array of possible exoplanet atmospheric properties. Our key findings are that: (1) the presence of aerosols typically increases the effects of day-night temperature gradients on transit spectra; (2) ignoring day-night temperature gradients when attempting to perform Bayesian parameter estimation will still return biased results when aerosols are present, as has already been shown for clear atmospheres in the literature; (3) when a day-night temperature gradient is present and accounted for in the retrieval model, some transit spectra can provide sufficient information to constrain temperatures and the width of the transition from day to night. The presence of clouds and hazes can actually tighten such constraints, but also weaken constraints on metallicity. This paper represents a step towards the larger goal of developing models and theory of adequate complexity to match the superior quality data that will soon be available.


title: Improving GAN Training with Probability Ratio Clipping and Sample Reweighting

abstract: Despite success on a wide range of problems related to vision, generative adversarial networks (GANs) often suffer from inferior performance due to unstable training, especially for text generation. To solve this issue, we propose a new variational GAN training framework which enjoys superior training stability. Our approach is inspired by a connection of GANs and reinforcement learning under a variational perspective. The connection leads to (1) probability ratio clipping that regularizes generator training to prevent excessively large updates, and (2) a sample re-weighting mechanism that improves discriminator training by downplaying bad-quality fake samples. Moreover, our variational GAN framework can provably overcome the training issue in many GANs that an optimal discriminator cannot provide any informative gradient to training generator. By plugging the training approach in diverse state-of-the-art GAN architectures, we obtain significantly improved performance over a range of tasks, including text generation, text style transfer, and image generation.


title: Real-time estimation of the optically detected magnetic resonance shift in diamond quantum thermometry

abstract: We investigate the real-time estimation protocols for the frequency shift of optically detected magnetic resonance (ODMR) of nitrogen-vacancy (NV) centers in nanodiamonds (NDs). Efficiently integrating multipoint ODMR measurements and ND particle tracking into fluorescence microscopy has recently demonstrated stable monitoring of the temperature inside living animals. We analyze the multipoint ODMR measurement techniques (3-, 4-, and 6-point methods) in detail and quantify the amount of measurement artifact owing to several systematic errors derived from instrumental errors of experimental hardware and ODMR spectral shape. We propose a practical approach to minimize the effect of these factors, which allows for measuring accurate temperatures of single NDs during dynamic thermal events. We also discuss integration of noise filters, data estimation protocols, and possible artifacts for further developments in real-time temperature estimation. The present study provides technical details of quantum diamond thermometry and discusses factors that may affect the temperature estimation in biological applications.


title: Self-organization of multi-layer spiking neural networks

abstract: Living neural networks in our brains autonomously self-organize into large, complex architectures during early development to result in an organized and functional organic computational device. A key mechanism that enables the formation of complex architecture in the developing brain is the emergence of traveling spatio-temporal waves of neuronal activity across the growing brain. Inspired by this strategy, we attempt to efficiently self-organize large neural networks with an arbitrary number of layers into a wide variety of architectures. To achieve this, we propose a modular tool-kit in the form of a dynamical system that can be seamlessly stacked to assemble multi-layer neural networks. The dynamical system encapsulates the dynamics of spiking units, their inter/intra layer interactions as well as the plasticity rules that control the flow of information between layers. The key features of our tool-kit are (1) autonomous spatio-temporal waves across multiple layers triggered by activity in the preceding layer and (2) Spike-timing dependent plasticity (STDP) learning rules that update the inter-layer connectivity based on wave activity in the connecting layers. Our framework leads to the self-organization of a wide variety of architectures, ranging from multi-layer perceptrons to autoencoders. We also demonstrate that emergent waves can self-organize spiking network architecture to perform unsupervised learning, and networks can be coupled with a linear classifier to perform classification on classic image datasets like MNIST. Broadly, our work shows that a dynamical systems framework for learning can be used to self-organize large computational devices.


title: On Correctness of Automatic Differentiation for Non-Differentiable Functions

abstract: Differentiation lies at the core of many machine-learning algorithms, and is well-supported by popular autodiff systems, such as TensorFlow and PyTorch. Originally, these systems have been developed to compute derivatives of differentiable functions, but in practice, they are commonly applied to functions with non-differentiabilities. For instance, neural networks using ReLU define non-differentiable functions in general, but the gradients of losses involving those functions are computed using autodiff systems in practice. This status quo raises a natural question: are autodiff systems correct in any formal sense when they are applied to such non-differentiable functions? In this paper, we provide a positive answer to this question. Using counterexamples, we first point out flaws in often-used informal arguments, such as: non-differentiabilities arising in deep learning do not cause any issues because they form a measure-zero set. We then investigate a class of functions, called PAP functions, that includes nearly all (possibly non-differentiable) functions in deep learning nowadays. For these PAP functions, we propose a new type of derivatives, called intensional derivatives, and prove that these derivatives always exist and coincide with standard derivatives for almost all inputs. We also show that these intensional derivatives are what most autodiff systems compute or try to compute essentially. In this way, we formally establish the correctness of autodiff systems applied to non-differentiable functions.


title: Hall algebras and quantum symmetric pairs of Kac-Moody type

abstract: We extend our $\imath$Hall algebra construction from acyclic to arbitrary $\imath$quivers, where the $\imath$quiver algebras are infinite-dimensional 1-Gorenstein in general. Then we establish an injective homomorphism from the universal $\imath$quantum group of Kac-Moody type arising from quantum symmetric pairs to the $\imath$Hall algebra associated to a virtually acyclic $\imath$quiver.


title: Majorana corner pairs in a two-dimensional $s$-wave cold atomic superfluid

abstract: We propose a method to prepare Majorana pairs at the corners of imprinted defects on a two-dimensional cold atom optical lattice with $s$-wave superfluid pairing. Different from previous proposals that manipulate the effective Dirac masses, our scheme relies on the sign flip of the spin-orbit coupling at the corners, which can be tuned in experiments by adjusting the angle of incident Raman lasers. The Majorana corner pairs are found to be located at the interface between two regimes with opposite spin orbit coupling strengths in an anticlockwise direction and are robust against certain symmetry-persevered perturbations. Our work provides a new way for implementing and manipulating Majorana pairs with existing cold-atom techniques.


title: 3D Fusion between Fluoroscopy Angiograms and SPECT Myocardial Perfusion Images to Guide Percutaneous Coronary Intervention

abstract: Background. Percutaneous coronary intervention(PCI) in stable coronary artery disease(CAD) is commonly triggered by abnormal myocardial perfusion imaging(MPI). However, due to the possibilities of multivessel disease and variability of coronary artery perfusion distribution, opportunity exists to better align anatomic stenosis with perfusion abnormalities to improve revascularization decisions. This study aims to develop a 3D multi-modality fusion approach to assist decision-making for PCI. Methods. Coronary arteries from fluoroscopic angiography(FA) were reconstructed into 3D artery anatomy. Left ventricular(LV) epicardial surface was extracted from SPECT. The 3D artery anatomy was non-rigidly fused with the LV epicardial surface. The accuracy of the 3D fusion was evaluated via both computer simulation and real patient data. For technical validation, simulated FA and MPI were integrated and then compared with the ground truth from a digital phantom. For clinical validation, FA and SPECT images were integrated and then compared with the ground truth from CT angiograms. Results. In the technical evaluation, the distance-based mismatch error between simulated fluoroscopy and phantom arteries is 1.86(SD:1.43)mm for left coronary arteries(LCA) and 2.21(SD:2.50)mm for right coronary arteries(RCA). In the clinical validation, the distance-based mismatch errors between the fluoroscopy and CT arteries were 3.84(SD:3.15)mm for LCA and 5.55(SD:3.64)mm for RCA. The presence of the corresponding fluoroscopy and CT arteries in the AHA 17-segment model agreed well with a Kappa value of 0.91(95% CI: 0.89-0.93) for LCA and 0.80(CI: 0.67-0.92) for RCA. Conclusions. Our fusion approach is technically accurate to assist PCI decision-making and is clinically feasible to be used in the catheterization laboratory. There is an opportunity to improve the decision-making and outcomes of PCI in stable CAD.


title: Search for gravitational waves from five low mass X-ray binaries in the second Advanced LIGO observing run with an improved hidden Markov model

abstract: Low mass X-ray binaries are prime targets for continuous gravitational wave searches by ground-based interferometers. Results are presented from a search for five low-mass X-ray binaries whose spin frequencies and orbital elements are measured accurately from X-ray pulsations: HETE J1900.1-2455, IGR J00291+5934, SAX J1808.4-3658, XTE J0929-314, and XTE J1814-338. Data are analysed from Observing Run 2 of the Advanced Laser Interferometer Gravitational-wave Observatory (LIGO). The search algorithm uses a hidden Markov model to track spin wandering, the $\mathcal{J}$-statistic maximum likelihood matched filter to track orbital phase, and a suite of five vetoes to reject artefacts from non-Gaussian noise. The search yields a number of low-significance, above threshold candidates consistent with the selected false-alarm probability. The candidates will be followed up in subsequent observing runs.


title: Chiral photocurrent in parity-violating magnet and enhanced response in topological antiferromagnet

abstract: Rectified electric current induced by irradiating light, so-called photocurrent, is an established phenomenon in optoelectronic physics. In this paper, we present a comprehensive classification of the photocurrent response arising from the parity violation in bulk systems. We clarify the contrasting role of $\mathcal{T}$- and $\mathcal{PT}$-symmetries and consequently find a new type of photocurrent phenomena characteristic of parity-violating magnets, intrinsic Fermi surface effect and gyration current. Especially, the gyration current is induced by the circularly-polarized light and it is the counterpart of the shift current caused by the linearly-polarized light. This photocurrent adds a new functionality of materials studied in various fields of condensed matter physics such as multiferroics and spintronics. A list of materials is provided. Furthermore, we show that the gyration current is strongly enhanced by topologically nontrivial band dispersion. On the basis of the microscopic analysis of Dirac models, we demonstrate the divergent photocurrent response and elucidate the importance of tilting of Dirac cones.


title: Weisfeiler-Lehman Embedding for Molecular Graph Neural Networks

abstract: A graph neural network (GNN) is a good choice for predicting the chemical properties of molecules. Compared with other deep networks, however, the current performance of a GNN is limited owing to the "curse of depth." Inspired by long-established feature engineering in the field of chemistry, we expanded an atom representation using Weisfeiler-Lehman (WL) embedding, which is designed to capture local atomic patterns dominating the chemical properties of a molecule. In terms of representability, we show WL embedding can replace the first two layers of ReLU GNN -- a normal embedding and a hidden GNN layer -- with a smaller weight norm. We then demonstrate that WL embedding consistently improves the empirical performance over multiple GNN architectures and several molecular graph datasets.


title: Hybrid Attentional Memory Network for Computational drug repositioning

abstract: Drug repositioning is designed to discover new uses of known drugs, which is an important and efficient method of drug discovery. Researchers only use one certain type of Collaborative Filtering (CF) models for drug repositioning currently, like the neighborhood based approaches which are good at mining the local information contained in few strong drug-disease associations, or the latent factor based models which are effectively capture the global information shared by a majority of drug-disease associations. Few researchers have combined these two types of CF models to derive a hybrid model with the advantages of both of them. Besides, the cold start problem has always been a major challenge in the field of computational drug repositioning, which restricts the inference ability of relevant models. Inspired by the memory network, we propose the Hybrid Attentional Memory Network (HAMN) model, a deep architecture combines two classes of CF model in a nonlinear manner. Firstly, the memory unit and the attention mechanism are combined to generate the neighborhood contribution representation to capture the local structure of few strong drug-disease associations. Then a variant version of the autoencoder is used to extract the latent factor of drugs and diseases to capture the overall information shared by a majority of drug-disease associations. In that process, ancillary information of drugs and diseases can help to alleviate the cold start problem. Finally, in the prediction stage, the neighborhood contribution representation is combined with the drug latent factor and disease latent factor to produce the predicted value. Comprehensive experimental results on two real data sets show that our proposed HAMN model is superior to other comparison models according to the AUC, AUPR and HR indicators.


title: Iterate & Cluster: Iterative Semi-Supervised Action Recognition

abstract: We propose a novel system for active semi-supervised feature-based action recognition. Given time sequences of features tracked during movements our system clusters the sequences into actions. Our system is based on encoder-decoder unsupervised methods shown to perform clustering by self-organization of their latent representation through the auto-regression task. These methods were tested on human action recognition benchmarks and outperformed non-feature based unsupervised methods and achieved comparable accuracy to skeleton-based supervised methods. However, such methods rely on K-Nearest Neighbours (KNN) associating sequences to actions, and general features with no annotated data would correspond to approximate clusters which could be further enhanced. Our system proposes an iterative semi-supervised method to address this challenge and to actively learn the association of clusters and actions. The method utilizes latent space embedding and clustering of the unsupervised encoder-decoder to guide the selection of sequences to be annotated in each iteration. Each iteration, the selection aims to enhance action recognition accuracy while choosing a small number of sequences for annotation. We test the approach on human skeleton-based action recognition benchmarks assuming that only annotations chosen by our method are available and on mouse movements videos recorded in lab experiments. We show that our system can boost recognition performance with only a small percentage of annotations. The system can be used as an interactive annotation tool to guide labeling efforts for 'in the wild' videos of various objects and actions to reach robust recognition.


title: A Marginal Analysis Framework to Incorporate the Externality Effect of Ordering Perishables

abstract: Finding the optimal policy for multi-period perishable inventory systems requires solving computationally-expensive stochastic dynamic programs (DP). To avoid the difficulty of solving DP models, we propose a framework that uses an externality term to capture the long-term impact of ordering decisions on the average cost over an infinite horizon. By approximating the externality term, we yield a tractable approximate optimality condition, which is solved through standard marginal analysis. The resulted policy is near-optimal in long-run average cost and ordering decisions.


title: Collapsing domain walls in the two-Higgs-doublet model and deep insights from the EDM

abstract: We study the domain wall solutions in the general two-Higgs-doublet model (2HDM) with a CP-violating phase. The 2HDM with the spontaneouse CP violation is found to have domain wall solutions whose tensions are $\mathcal{O}(10^6)\,{\rm GeV}^3$, which are excluded by the Zel'dovich-Kobzarev-Okun bound. With the explicit CP-violating (CPV) terms as the so-called biased term in the scalar potential, domain walls can collapse in the early Universe. The sizes of the explicit CP violation can be constrained from the Big Bang nucleosynthesis. This constraint is converted to the CPV mixing of $\alpha_c$, and is mostly sensitive to the mass splittings between two heavy neutral Higgs bosons. We estimate the possible gravitational wave signals and the electric dipole moment (EDM) predictions due to the domain wall collapsing. It turns out that the peak spectrum of the GW from the domain wall collapsing cannot be probed in any future program. In contrast, the untenable regions with very tiny explicit CPV parameter in the Higgs potential has been partially excluded by the latest electron EDM measurements at the ACME-II and will be further confirmed or excluded by the future ACME-III projection.


title: Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses

abstract: Uniform stability is a notion of algorithmic stability that bounds the worst case change in the model output by the algorithm when a single data point in the dataset is replaced. An influential work of Hardt et al. (2016) provides strong upper bounds on the uniform stability of the stochastic gradient descent (SGD) algorithm on sufficiently smooth convex losses. These results led to important progress in understanding of the generalization properties of SGD and several applications to differentially private convex optimization for smooth losses. Our work is the first to address uniform stability of SGD on {\em nonsmooth} convex losses. Specifically, we provide sharp upper and lower bounds for several forms of SGD and full-batch GD on arbitrary Lipschitz nonsmooth convex losses. Our lower bounds show that, in the nonsmooth case, (S)GD can be inherently less stable than in the smooth case. On the other hand, our upper bounds show that (S)GD is sufficiently stable for deriving new and useful bounds on generalization error. Most notably, we obtain the first dimension-independent generalization bounds for multi-pass SGD in the nonsmooth case. In addition, our bounds allow us to derive a new algorithm for differentially private nonsmooth stochastic convex optimization with optimal excess population risk. Our algorithm is simpler and more efficient than the best known algorithm for the nonsmooth case Feldman et al. (2020).


title: How Many Samples is a Good Initial Point Worth in Low-rank Matrix Recovery?

abstract: Given a sufficiently large amount of labeled data, the non-convex low-rank matrix recovery problem contains no spurious local minima, so a local optimization algorithm is guaranteed to converge to a global minimum starting from any initial guess. However, the actual amount of data needed by this theoretical guarantee is very pessimistic, as it must prevent spurious local minima from existing anywhere, including at adversarial locations. In contrast, prior work based on good initial guesses have more realistic data requirements, because they allow spurious local minima to exist outside of a neighborhood of the solution. In this paper, we quantify the relationship between the quality of the initial guess and the corresponding reduction in data requirements. Using the restricted isometry constant as a surrogate for sample complexity, we compute a sharp threshold number of samples needed to prevent each specific point on the optimization landscape from becoming a spurious local minimum. Optimizing the threshold over regions of the landscape, we see that for initial points around the ground truth, a linear improvement in the quality of the initial guess amounts to a constant factor improvement in the sample complexity.


title: Superconductivity mystery turns 25

abstract: In 1994, an unconventional form of superconductivity was detected in strontium ruthenate. The discovery has shed light on the mechanism of unconventional superconductivity at high temperatures. "The great tragedy of Science [is] the slaying of a beautiful hypothesis by an [experimental] fact." - T.H. Huxley, Biogenesis and abiogenesis, (1884)


title: Massive Coded-NOMA for Low-Capacity Channels: A Low-Complexity Recursive Approach

abstract: In this paper, we present a low-complexity recursive approach for massive and scalable code-domain nonorthogonal multiple access (NOMA) with applications to emerging low-capacity scenarios. The problem definition in this paper is inspired by three major requirements of the next generations of wireless networks. Firstly, the proposed scheme is particularly beneficial in low-capacity regimes which is important in practical scenarios of utmost interest such as the Internet-of-Things (IoT) and massive machine-type communication (mMTC). Secondly, we employ code-domain NOMA to efficiently share the scarce common resources among the users. Finally, the proposed recursive approach enables code-domain NOMA with low-complexity detection algorithms that are scalable with the number of users to satisfy the requirements of massive connectivity. To this end, we propose a novel encoding and decoding scheme for code-domain NOMA based on factorizing the pattern matrix, for assigning the available resource elements to the users, as the Kronecker product of several smaller factor matrices. As a result, both the pattern matrix design at the transmitter side and the mixed symbols' detection at the receiver side can be performed over matrices with dimensions that are much smaller than the overall pattern matrix. Consequently, this leads to significant reduction in both the complexity and the latency of the detection. We present the detection algorithm for the general case of factor matrices. The proposed algorithm involves several recursions each involving certain sets of equations corresponding to a certain factor matrix. We then characterize the system performance in terms of average sum rate, latency, and detection complexity. Our latency and complexity analysis confirm the superiority of our proposed scheme in enabling large pattern matrices.