title: Time-Fractional Allen-Cahn Equations: Analysis and Numerical Methods

abstract: In this work, we consider a time-fractional Allen-Cahn equation, where the conventional first order time derivative is replaced by a Caputo fractional derivative with order $\alpha\in(0,1)$. First, the well-posedness and (limited) smoothing property are systematically analyzed, by using the maximal $L^p$ regularity of fractional evolution equations and the fractional Gr\"onwall's inequality. We also show the maximum principle like their conventional local-in-time counterpart. Precisely, the time-fractional equation preserves the property that the solution only takes value between the wells of the double-well potential when the initial data does the same. Second, after discretizing the fractional derivative by backward Euler convolution quadrature, we develop several unconditionally solvable and stable time stepping schemes, i.e., convex splitting scheme, weighted convex splitting scheme and linear weighted stabilized scheme. Meanwhile, we study the discrete energy dissipation property (in a weighted average sense), which is important for gradient flow type models, for the two weighted schemes. Finally, by using a discrete version of fractional Gr\"onwall's inequality and maximal $\ell^p$ regularity, we prove that the convergence rates of those time-stepping schemes are $O(\tau^\alpha)$ without any extra regularity assumption on the solution. We also present extensive numerical results to support our theoretical findings and to offer new insight on the time-fractional Allen-Cahn dynamics.


title: Stability of Asai local factors for $GL(2)$

abstract: Let $F$ be a non-archimedean local field of characteristic not equal to $2$ and let $E/F$ be a quadratic algebra. We prove the stability of local factors attached to (complex) irreducible admissible representations of $GL(2,E)$ via the Rankin-Selberg method under highly ramified twists. This includes both the Asai as well as the Rankin-Selberg local factors attached to pairs. Our method relies on expressing the gamma factor as a Mellin transform using Bessel functions.


title: A New Approach to Real Time Impulsive Sound Detection for Surveillance Applications

abstract: Most of the surveillance systems for public safety are solely based on one or more video cameras. These camera systems have some drawbacks such that they have poor performance in adverse weather conditions or during night time. Therefore most of the time, some other sensors should accompany to video cameras. Although audio surveillance is in its early stage, there has been considerable amount of work in this area in the last decade. In this paper we make a review of impulsive sound detection algorithms. Sounds from dangerous events such as gunshots, explosions, human screaming can be classified as impulsive sounds, so this paper reviews all impulsive sound detection algorithms along with impulsive noise detection algorithms although they progress in their own path. These dangerous sound events have no other detection means except audio. We try to adapt some algorithms used in impulsive noise detection to the area of impulsive sound detection. Tests show that Warped Linear Prediction (WLP) can be used for impulsive sound detection.


title: Importance of nonresonant corrections for the description of atomic spectra

abstract: We demonstrate that the present superaccurate measurements of transition processes between atomic states in hydrogen atom reached the limit of accuracy when transition frequency cannot be defined anymore in a unique way. This was predicted earlier and is due to the necessity to include the nonresonant corrections in the description of resonant processes. The observed spectral line profile becomes asymmetric, and it becomes impossible to extract the value of transition frequency from this profile in a unique way. Nonresonant corrections depend on the type of experiment and on the experimental arrangement. However, the line profile itself for any resonant process can be defined with any desired level of accuracy. A popular trend in modern search for atomic frequency standards and atomic clocks is the search for transitions where the nonresonant corrections are negligible. In this paper we present closed expressions for the resonant photon scattering cross sections on an atomic level with dependence on all atomic quantum numbers including fine and hyperfine structure. These expressions are given for different types of experiments with fixing of the incident (outgoing) photon propagation directions and incident (outgoing) photon polarization. Using these expressions we demonstrate that the transition frequencies in particular cases cannot be derived uniquely if the accuracy of measurement reaches the level quoted in A. Beyer {\it et al}. Our interpretation of the results of this experiment is alternative to the interpretation given by A. Beyer {\it et al}.


title: Reinforcement Learning with Non-uniform State Representations for Adaptive Search

abstract: Efficient spatial exploration is a key aspect of search and rescue. In this paper, we present a search algorithm that generates efficient trajectories that optimize the rate at which probability mass is covered by a searcher. This should allow an autonomous vehicle find one or more lost targets as rapidly as possible. We do this by performing non-uniform sampling of the search region. The path generated minimizes the expected time to locate the missing target by visiting high probability regions using non-myopic path generation based on reinforcement learning. We model the target probability distribution using a classic mixture of Gaussians model with means and mixture coefficients tuned according to the location and time of sightings of the lost target. Key features of our search algorithm are the ability to employ a very general non-deterministic action model and the ability to generate action plans for any new probability distribution using the parameters learned on other similar looking distributions. One of the key contributions of this paper is the use of non-uniform state aggregation for policy search in the context of robotics.


title: Membership Privacy for Machine Learning Models Through Knowledge Transfer

abstract: Large capacity machine learning (ML) models are prone to membership inference attacks (MIAs), which aim to infer whether the target sample is a member of the target model's training dataset. The serious privacy concerns due to the membership inference have motivated multiple defenses against MIAs, e.g., differential privacy and adversarial regularization. Unfortunately, these defenses produce ML models with unacceptably low classification performances. Our work proposes a new defense, called distillation for membership privacy (DMP), against MIAs that preserves the utility of the resulting models significantly better than prior defenses. DMP leverages knowledge distillation to train ML models with membership privacy. We provide a novel criterion to tune the data used for knowledge transfer in order to amplify the membership privacy of DMP. Our extensive evaluation shows that DMP provides significantly better tradeoffs between membership privacy and classification accuracies compared to state-of-the-art MIA defenses. For instance, DMP achieves ~100% accuracy improvement over adversarial regularization for DenseNet trained on CIFAR100, for similar membership privacy (measured using MIA risk): when the MIA risk is 53.7%, adversarially regularized DenseNet is 33.6% accurate, while DMP-trained DenseNet is 65.3% accurate.


title: Query and Resource Optimizations: A Case for Breaking the Wall in Big Data Systems

abstract: Modern big data systems run on cloud environments where resources are shared amongst several users and applications. As a result, declarative user queries in these environments need to be optimized and executed over resources that constantly change and are provisioned on demand for each job. This requires us to rethink traditional query optimizers designed for systems that run on dedicated resources. In this paper, we show evidence that the choice of query plans depends heavily on the available resources, and the current practice of choosing query plans before picking the resources could lead to significant performance loss in two popular big data systems, namely Hive and SparkSQL. Therefore, we make a case for Resource and Query Optimization (or RAQO), i.e., choosing both the query plan and the resource configuration at the same time. We describe rule-based RAQO and present alternate decisions trees to make resource-aware query planning in Hive and Spark. We further present cost-based RAQO that integrates resource planning within a query planner, and show techniques to significantly reduce the resource planning overheads. We evaluate cost-based RAQO using state-of-the-art System R query planner as well as a recently proposed multi-objective query planner. Our evaluation on TPC-H and randomly generated schemas show that: (i) we can reduce the resource planning overhead by up to 16x, and (ii) RAQO can scale to schemas as large as 100 table joins as well as clusters as big as 100K containers with 100GB each.


title: Plane Sweep Algorithms for Data Collection in Wireless Sensor Network using Mobile Sink

abstract: Usage of mobile sink(s) for data gathering in wireless sensor networks(WSNs) improves the performance of WSNs in many respects such as power consumption, lifetime, etc. In some applications, the mobile sink $MS$ travels along a predefined path to collect data from the nearby sensors, which are referred as sub-sinks. Due to the slow speed of the $MS$, the data delivery latency is high. However, optimizing the {\em data gathering schedule}, i.e. optimizing the transmission schedule of the sub-sinks to the $MS$ and the movement speed of the $MS$ can reduce data gathering latency. We formulate two novel optimization problems for data gathering in minimum time. The first problem determines an optimal data gathering schedule of the $MS$ by controlling the data transmission schedule and the speed of the $MS$, where the data availabilities of the sub-sinks are given. The second problem generalizes the first, where the data availabilities of the sub-sinks are unknown. Plane sweep algorithms are proposed for finding optimal data gathering schedule and data availabilities of the sub-sinks. The performances of the proposed algorithms are evaluated through simulations. The simulation results reveal that the optimal distribution of data among the sub-sinks together with optimal data gathering schedule improves the data gathering time.


title: Many-body Green's function theory of electrons and nuclei beyond the Born-Oppenheimer approximation

abstract: The method of many-body Green's functions is developed for arbitrary systems of electrons and nuclei starting from the full (beyond Born-Oppenheimer) Hamiltonian of Coulomb interactions and kinetic energies. The theory presented here resolves the problems arising from the translational and rotational invariance of this Hamiltonian that afflict the existing many-body Green's function theories. We derive a coupled set of exact equations for the electronic and nuclear Green's functions and provide a systematic way to approximately compute the properties of arbitrary many-body systems of electrons and nuclei beyond the Born-Oppenheimer approximation. The case of crystalline solids is discussed in detail.


title: Context is Key: Grammatical Error Detection with Contextual Word Representations

abstract: Grammatical error detection (GED) in non-native writing requires systems to identify a wide range of errors in text written by language learners. Error detection as a purely supervised task can be challenging, as GED datasets are limited in size and the label distributions are highly imbalanced. Contextualized word representations offer a possible solution, as they can efficiently capture compositional information in language and can be optimized on large amounts of unsupervised data. In this paper, we perform a systematic comparison of ELMo, BERT and Flair embeddings (Peters et al., 2017; Devlin et al., 2018; Akbik et al., 2018) on a range of public GED datasets, and propose an approach to effectively integrate such representations in current methods, achieving a new state of the art on GED. We further analyze the strengths and weaknesses of different contextual embeddings for the task at hand, and present detailed analyses of their impact on different types of errors.


title: The True Sample Complexity of Identifying Good Arms

abstract: We consider two multi-armed bandit problems with $n$ arms: (i) given an $\epsilon > 0$, identify an arm with mean that is within $\epsilon$ of the largest mean and (ii) given a threshold $\mu_0$ and integer $k$, identify $k$ arms with means larger than $\mu_0$. Existing lower bounds and algorithms for the PAC framework suggest that both of these problems require $\Omega(n)$ samples. However, we argue that these definitions not only conflict with how these algorithms are used in practice, but also that these results disagree with intuition that says (i) requires only $\Theta(\frac{n}{m})$ samples where $m = |\{ i : \mu_i > \max_{i \in [n]} \mu_i - \epsilon\}|$ and (ii) requires $\Theta(\frac{n}{m}k)$ samples where $m = |\{ i : \mu_i > \mu_0 \}|$. We provide definitions that formalize these intuitions, obtain lower bounds that match the above sample complexities, and develop explicit, practical algorithms that achieve nearly matching upper bounds.


title: Learning Restricted Boltzmann Machines with Arbitrary External Fields

abstract: We study the problem of learning graphical models with latent variables. We give the first algorithm for learning locally consistent (ferromagnetic or antiferromagnetic) Restricted Boltzmann Machines (or RBMs) with {\em arbitrary} external fields. Our algorithm has optimal dependence on dimension in the sample complexity and run time however it suffers from a sub-optimal dependency on the underlying parameters of the RBM. Prior results have been established only for {\em ferromagnetic} RBMs with {\em consistent} external fields (signs must be same)\cite{bresler2018learning}. The proposed algorithm strongly relies on the concavity of magnetization which does not hold in our setting. We show the following key structural property: even in the presence of arbitrary external field, for any two observed nodes that share a common latent neighbor, the covariance is high. This enables us to design a simple greedy algorithm that maximizes covariance to iteratively build the neighborhood of each vertex.


title: Local and Global Homogeneity for Manifolds that admit a Positive Curvature Metric

abstract: In this note we study globally homogeneous Riemannian quotients $\Gamma\backslash (M,ds^2)$ of homogeneous Riemannian manifolds $(M,ds^2)$. The Homogeneity Conjecture is that $\Gamma\backslash (M,ds^2)$ is (globally) homogeneous if and only if $(M,ds^2)$ is homogeneous and every $\gamma \in \Gamma$ is of constant displacement on $(M,ds^2)$. We provide further evidence for that conjecture by (i) verifying it for normal homogeneous Riemannian manifolds that also admit an invariant Riemannian metric of strictly positive sectional curvature and (ii) showing that in most (three or less) cases the normality condition can be dropped.


title: IMP: Instance Mask Projection for High Accuracy Semantic Segmentation of Things

abstract: In this work, we present a new operator, called Instance Mask Projection (IMP), which projects a predicted Instance Segmentation as a new feature for semantic segmentation. It also supports back propagation so is trainable end-to-end. Our experiments show the effectiveness of IMP on both Clothing Parsing (with complex layering, large deformations, and non-convex objects), and on Street Scene Segmentation (with many overlapping instances and small objects). On the Varied Clothing Parsing dataset (VCP), we show instance mask projection can improve 3 points on mIOU from a state-of-the-art Panoptic FPN segmentation approach. On the ModaNet clothing parsing dataset, we show a dramatic improvement of 20.4% absolutely compared to existing baseline semantic segmentation results. In addition, the instance mask projection operator works well on other (non-clothing) datasets, providing an improvement of 3 points in mIOU on Thing classes of Cityscapes, a self-driving dataset, on top of a state-of-the-art approach.


title: On the unramified cohomology of certain quotient varieties

abstract: In this note, we consider unramified cohomology with $\mathbb{Z}/2$ coefficients for some (degree two) quotient varieties and describe a method that allows one to prove the non-vanishing of these groups under certain conditions. We apply this method to prove a non-vanishing statement in the case of Kummer varieties. Combining this with work of Colliot-Th\'el\`ene and Voisin, we obtain a new type of three-dimensional counterexample to the integral Hodge conjecture.


title: Comments on ""Hot" electrons in metallic nanostructures -- non-thermal carriers or heating?" and "Assistance of metal nanoparticles to photo-catalysis -- nothing more than a classical heat source"

abstract: The authors of preprint arXiv:1810.00565 and paper in Light: Science & Applications (2019), Y. Dubi and Y. Sivan, made several wrong and inconsistent comments on several of our papers. In addition, the paper in Faraday Discuss. 214, 215-233 (2018) by the same authors also contains several wrong statements in relationship with our work. Moreover, the authors address in their comments features that were in fact not present in our work. In what follows we present correction to a number of specific points in which they either misrepresented or erroneously interpreted our published work.


title: A Statistical View on Synthetic Aperture Imaging for Occlusion Removal

abstract: Synthetic apertures find applications in many fields, such as radar, radio telescopes, microscopy, sonar, ultrasound, LiDAR, and optical imaging. They approximate the signal of a single hypothetical wide aperture sensor with either an array of static small aperture sensors or a single moving small aperture sensor. Common sense in synthetic aperture sampling is that a dense sampling pattern within a wide aperture is required to reconstruct a clear signal. In this article we show that there exists practical limits to both, synthetic aperture size and number of samples for the application of occlusion removal. This leads to an understanding on how to design synthetic aperture sampling patterns and sensors in a most optimal and practically efficient way. We apply our findings to airborne optical sectioning which uses camera drones and synthetic aperture imaging to computationally remove occluding vegetation or trees for inspecting ground surfaces.


title: 4D X-Ray CT Reconstruction using Multi-Slice Fusion

abstract: There is an increasing need to reconstruct objects in four or more dimensions corresponding to space, time and other independent parameters. The best 4D reconstruction algorithms use regularized iterative reconstruction approaches such as model based iterative reconstruction (MBIR), which depends critically on the quality of the prior modeling. Recently, Plug-and-Play methods have been shown to be an effective way to incorporate advanced prior models using state-of-the-art denoising algorithms designed to remove additive white Gaussian noise (AWGN). However, state-of-the-art denoising algorithms such as BM4D and deep convolutional neural networks (CNNs) are primarily available for 2D and sometimes 3D images. In particular, CNNs are difficult and computationally expensive to implement in four or more dimensions, and training may be impossible if there is no associated high-dimensional training data. In this paper, we present Multi-Slice Fusion, a novel algorithm for 4D and higher-dimensional reconstruction, based on the fusion of multiple low-dimensional denoisers. Our approach uses multi-agent consensus equilibrium (MACE), an extension of Plug-and-Play, as a framework for integrating the multiple lower-dimensional prior models. We apply our method to the problem of 4D cone-beam X-ray CT reconstruction for Non Destructive Evaluation (NDE) of moving parts. This is done by solving the MACE equations using lower-dimensional CNN denoisers implemented in parallel on a heterogeneous cluster. Results on experimental CT data demonstrate that Multi-Slice Fusion can substantially improve the quality of reconstructions relative to traditional 4D priors, while also being practical to implement and train.


title: Coexistence of infinitely many large, stable, rapidly oscillating periodic solutions in time-delayed Duffing oscillators

abstract: We explore stability and instability of rapidly oscillating solutions $x(t)$ for the hard spring delayed Duffing oscillator $$x''(t)+ ax(t)+bx(t-T)+x^3(t)=0.$$ Fix $T>0$. We target periodic solutions $x_n(t)$ of small minimal periods $p_n=2T/n$, for integer $n\rightarrow \infty$, and with correspondingly large amplitudes. Note how $x_n(t)$ are also marginally stable solutions, respectively, of the two standard, non-delayed, Hamiltonian Duffing oscillators $$x''+ ax+(-1)^nbx+x^3=0.$$ Stability changes for the delayed Duffing oscillator. Simultaneously for all sufficiently large $n\geq n_0$, we obtain local exponential stability for $(-1)^nb<0$, and exponential instability for $(-1)^nb>0$, provided that $$0 \neq (-1)^{n+1}b\,T^2< \tfrac{3}{2}\pi^2.$$ We interpret our results in terms of noninvasive delayed feedback stabilization and destabilization for large amplitude rapidly periodic solutions of the standard Duffing oscillators. We conclude with numerical illustrations of our results for small and moderate $n$ which also indicate a Neimark-Sacker torus bifurcation at the validity boundary of our theoretical results.


title: An Overview of In-memory Processing with Emerging Non-volatile Memory for Data-intensive Applications

abstract: The conventional von Neumann architecture has been revealed as a major performance and energy bottleneck for rising data-intensive applications. %, due to the intensive data movements. The decade-old idea of leveraging in-memory processing to eliminate substantial data movements has returned and led extensive research activities. The effectiveness of in-memory processing heavily relies on memory scalability, which cannot be satisfied by traditional memory technologies. Emerging non-volatile memories (eNVMs) that pose appealing qualities such as excellent scaling and low energy consumption, on the other hand, have been heavily investigated and explored for realizing in-memory processing architecture. In this paper, we summarize the recent research progress in eNVM-based in-memory processing from various aspects, including the adopted memory technologies, locations of the in-memory processing in the system, supported arithmetics, as well as applied applications.


title: Hypersonic flow over a wedge in the detached shock range

abstract: In a recent publication Hornung (2019) showed that the shock wave stand-off distance and the drag coefficient of a cone in inviscid hypersonic flow of a perfect gas can be expressed as the product of a function of the inverse normal-shock density ratio and a function of a cone-angle parameter, thus reducing the number of independent parameters from three (Mach number, specific heat ratio and angle) to two. By making a large number of Euler computations, analytic forms of the functions were obtained. In this article the same approach is applied to the symmetrical flow over a wedge. It turns out that the same simplification applies and corresponding analytical forms of the functions are obtained. The functions of the normal-shock density ratio are compared with newly determined corresponding functions for flow over a circular cylinder.


title: Non-linear second-order topological insulators

abstract: We demonstrate, both theoretically and experimentally, the concept of non-linear second-order topological insulators, a class of bulk insulators with quantized Wannier centers and a bulk polarization directly controlled by the level of non-linearity. We show that one-dimensional edge states and zero-dimensional corner states can be induced in a trivial crystal insulator made of evanescently coupled resonators with linear and nonlinear coupling coefficients, simply by tuning the excitation intensity. This allows global external control over topological phase transitions and switching to a phase with non-zero bulk polarization, without requiring any structural or geometrical changes. We further show how these non-linear effects enable dynamic tuning of the spectral properties and localization of the topological edge and corner states. Such self-induced second-order topological insulators, which can be found and implemented in a wide variety of physical platforms ranging from electronics to microwaves, acoustics, and optics, hold exciting promises for reconfigurable topological energy confinement, power harvesting, data storage, and spatial management of high-intensity fields.


title: Multi-Hop Paragraph Retrieval for Open-Domain Question Answering

abstract: This paper is concerned with the task of multi-hop open-domain Question Answering (QA). This task is particularly challenging since it requires the simultaneous performance of textual reasoning and efficient searching. We present a method for retrieving multiple supporting paragraphs, nested amidst a large knowledge base, which contain the necessary evidence to answer a given question. Our method iteratively retrieves supporting paragraphs by forming a joint vector representation of both a question and a paragraph. The retrieval is performed by considering contextualized sentence-level representations of the paragraphs in the knowledge source. Our method achieves state-of-the-art performance over two well-known datasets, SQuAD-Open and HotpotQA, which serve as our single- and multi-hop open-domain QA benchmarks, respectively.


title: Extension property in the tridisc

abstract: Motivated by works on extension sets in standard domains we introduce a notion of the Carath\'eodory set that seems better suited for the methods used in proofs of results on characterization of extension sets. A special stress is put on a class of two dimensional submanifolds in the tridisc which not only turns out to be Carath\'eodory but also provides examples of two dimensional domains for which the celebrated Lempert Theorem holds. Additionally, a recently introduced notion of universal sets for the Carath\'eodory extremal problem is studied and new results on domains admitting (no) finite universal sets are given.


title: Cicho\'n's maximum without large cardinals

abstract: Cicho\'n's diagram lists twelve cardinal characteristics (and the provable inequalities between them) associated with the ideals of null sets, meager sets, countable sets, and $\sigma$-compact subsets of the irrationals. It is consistent that all entries of Cicho\'n's diagram are pairwise different (apart from $\textrm{add}(\mathcal{M})$ and $\textrm{cof}(\mathcal{M})$, which are provably equal to other entries). However, the consistency proofs so far required large cardinal assumptions. In this work, we show the consistency without such assumptions.


title: Transmutation elements Re/Ta effect on vacancy formation and dissociation behavior in W

abstract: Transmutation elements are the essential products in plasma facing materials tungsten, and have further effects on point defects evolution resulted by radiation. Here, transmutation elements Re and Ta atoms have been selected to assess the effects on property of vacancy and vacancy cluster in W material via first-principles calculations. The formation energy indicates mono-vacancy is more likely to form in W-Re system than pure W and W-Ta system. Both Re and Ta have reduced the diffusion barrier energy in the mono-vacancy migration. The calculation presents that vacancy cluster prefers to grow up by combining another vacancy cluster relative to a single mono-vacancy. Re is favorable to the nucleation and growth of vacancies clusters, while Ta has a suppressive effect on the aggregation of small vacancy cluster. The emphasis analysis is obtained according the volumetric dependent strain. Vacancy dissociation calculations show that the dissociation of vacancy clusters is easier to begin with a single vacancy dissociation process.


title: Effect of tungsten on vacancy behaviors in Ta-W alloys from first-principles

abstract: Alloying elements play an important role in the design of plasma facing materials with good comprehensive properties. Based on first-principles calculations, the stability of alloying element W and its interaction with vacancy defects in Ta-W alloys are studied. The results show that W tends to distribute dispersedly in Ta lattice, and is not likely to form precipitation even with the coexistence of vacancy. The aggregation behaviors of W and vacancy can be affected by their concentration competition. The increase of W atoms has a negative effect on the vacancy clustering, as well as delays the vacancy nucleation process, which is favorable to the recovery of point defects. Our results are in consistent with the defect evolution observed in irradiation experiments in Ta-W alloys. Our calculations suggest that Ta is a potential repairing element that can be doped into Ta-based materials to improve their radiation resistance.


title: A parametrized Poincare-Hopf Theorem and Clique Cardinalities of graphs

abstract: Given a locally injective real function g on the vertex set V of a finite simple graph G=(V,E), we prove the Poincare-Hopf formula f_G(t) = 1+t sum_{x in V} f_{S_g(x)}(t), where S_g(x) = { y in S(x), g(y) less than g(x) } and f_G(t)=1+f_0 t + ... + f_{d} t^{d+1} is the f-function encoding the f-vector of a graph G, where f_k counts the number of k-dimensional cliques, complete sub-graphs, in G. The corresponding computation of f reduces the problem recursively to n tasks of graphs of half the size. For t=-1, the parametric Poincare-Hopf formula reduces to the classical Poincare-Hopf result X(G)=sum_x i_g(x), with integer indices i_g(x)=1-X(S_g(x)) and Euler characteristic X. In the new Poincare-Hopf formula, the indices are integer polynomials and the curvatures K_x(t) expressed as index expectations K_x(t) = E[i_x(t)] are polynomials with rational coefficients. Integrating the Poincare-Hopf formula over probability spaces of functions g gives Gauss-Bonnet formulas like f_G(t) = 1+sum_{x} F_{S(x)}(t), where F_G is the anti-derivative of f_G. A similar computation is done for the generating function f_{G,H}(t,s) = sum_{k,l} f_{k,l}(G,H) s^k t^l of the f-intersection matrix f_{k,l}(G,H) counting the number of intersections of k-simplices in G with l-simplices in H. Also here, the computation is reduced to 4 n^2 computations for graphs of half the size: f_{G,H}(t,s) = sum_{v,w} f_{B_g(v),B_g(w)}(t,s) - f_{B_g(v),S_g(w)}(t,s) - f_{S_g(v),B_g(w)}(t,s) + f_{S_g(v),S_g(w)}(t,s), where B_g(v)= S_g(v)+{v} is the unit ball of v.


title: No-regret Learning in Cournot Games

abstract: This paper examines the convergence of no-regret learning in Cournot games with continuous actions. Cournot games are the essential model for many socio-economic systems, where players compete by strategically setting their output quantity. We assume that players do not have full information of the game and thus cannot pre-compute a Nash equilibrium. Two types of feedback are considered: one is bandit feedback and the other is gradient feedback. To study the convergence of the induced sequence of play, we introduce the notion of convergence in measure, and show that the players' actual sequence of action converges to the unique Nash equilibrium. In addition, our results naturally extend the no-regret learning algorithms' time-average regret bounds to obtain the final-iteration convergence rates. Together, our work presents significantly sharper convergence results for learning in games without strong assumptions on game property (e.g., monotonicity) and shows how exploiting the game information feedback can influence the convergence rates.


title: The Price of Local Fairness in Multistage Selection

abstract: The rise of algorithmic decision making led to active researches on how to define and guarantee fairness, mostly focusing on one-shot decision making. In several important applications such as hiring, however, decisions are made in multiple stage with additional information at each stage. In such cases, fairness issues remain poorly understood. In this paper we study fairness in $k$-stage selection problems where additional features are observed at every stage. We first introduce two fairness notions, local (per stage) and global (final stage) fairness, that extend the classical fairness notions to the $k$-stage setting. We propose a simple model based on a probabilistic formulation and show that the locally and globally fair selections that maximize precision can be computed via a linear program. We then define the price of local fairness to measure the loss of precision induced by local constraints; and investigate theoretically and empirically this quantity. In particular, our experiments show that the price of local fairness is generally smaller when the sensitive attribute is observed at the first stage; but globally fair selections are more locally fair when the sensitive attribute is observed at the second stage---hence in both cases it is often possible to have a selection that has a small price of local fairness and is close to locally fair.


title: The Anatomy of Requirements

abstract: Requirements engineering is crucial to software development but lacks a precise definition of its fundamental concepts. Even the basic definitions in the literature and in industry standards are often vague and verbose. To remedy this situation and provide a solid basis for discussions of requirements, this work provides precise definitions of the fundamental requirements concepts and two systematic classifications: a taxonomy of requirement elements (such as components, goals, constraints...) ; and a taxonomy of possible relations between these elements (such as "extends", "excepts", "belongs"...). The discussion evaluates the taxonomies on published requirements documents; readers can test the concepts in two online quizzes. The intended result of this work is to spur new advances in the study and practice of software requirements by clarifying the fundamental concepts.


title: Detecting new signals under background mismodelling

abstract: Searches for new astrophysical phenomena often involve several sources of non-random uncertainties which can lead to highly misleading results. Among these, model-uncertainty arising from background mismodelling can dramatically compromise the sensitivity of the experiment under study. Specifically, overestimating the background distribution in the signal region increases the chances of missing new physics. Conversely, underestimating the background outside the signal region leads to an artificially enhanced sensitivity and a higher likelihood of claiming false discoveries. The aim of this work is to provide a unified statistical strategy to perform modelling, estimation, inference, and signal characterization under background mismodelling. The method proposed allows to incorporate the (partial) scientific knowledge available on the background distribution and provides a data-updated version of it in a purely nonparametric fashion without requiring the specification of prior distributions on the parameters. Applications in the context of dark matter searches and radio surveys show how the tools presented in this article can be used to incorporate non-stochastic uncertainty due to instrumental noise and to overcome violations of classical distributional assumptions in stacking experiments.

categories: physics.data-an astro-ph.HE astro-ph.IM hep-ph stat.ME


title: A Wong-Zakai theorem for the stochastic mass critical NLS

abstract: We prove a Wong-Zakai theorem for the defocusing mass-critical stochastic nonlinear Schr\"odinger equation (NLS) on $\mathbb{R}$. The main ingredient are careful mixtures of bootstrapping arguments at both deterministic and stochastic levels. Several subtleties arising from the proof mark the difference between the dispersive case and corresponding situations in SDEs and parabolic stochastic PDEs, as well as the difference between the large-$n$ case and the limiting ($n=\infty$) case.


title: Social Cost Guarantees in Smart Route Guidance

abstract: We model and study the problem of assigning traffic in an urban road network infrastructure. In our model, each driver submits their intended destination and is assigned a route to follow that minimizes the social cost (i.e., travel distance of all the drivers). We assume drivers are strategic and try to manipulate the system (i.e., misreport their intended destination and/or deviate from the assigned route) if they can reduce their travel distance by doing so. Such strategic behavior is highly undesirable as it can lead to an overall suboptimal traffic assignment and cause congestion. To alleviate this problem, we develop moneyless mechanisms that are resilient to manipulation by the agents and offer provable approximation guarantees on the social cost obtained by the solution. We then empirically test the mechanisms studied in the paper, showing that they can be effectively used in practice in order to compute manipulation resistant traffic allocations.


title: How To Train Your Deep Multi-Object Tracker

abstract: The recent trend in vision-based multi-object tracking (MOT) is heading towards leveraging the representational power of deep learning to jointly learn to detect and track objects. However, existing methods train only certain sub-modules using loss functions that often do not correlate with established tracking evaluation measures such as Multi-Object Tracking Accuracy (MOTA) and Precision (MOTP). As these measures are not differentiable, the choice of appropriate loss functions for end-to-end training of multi-object tracking methods is still an open research problem. In this paper, we bridge this gap by proposing a differentiable proxy of MOTA and MOTP, which we combine in a loss function suitable for end-to-end training of deep multi-object trackers. As a key ingredient, we propose a Deep Hungarian Net (DHN) module that approximates the Hungarian matching algorithm. DHN allows estimating the correspondence between object tracks and ground truth objects to compute differentiable proxies of MOTA and MOTP, which are in turn used to optimize deep trackers directly. We experimentally demonstrate that the proposed differentiable framework improves the performance of existing multi-object trackers, and we establish a new state of the art on the MOTChallenge benchmark. Our code is publicly available from https://github.com/yihongXU/deepMOT.


title: Generating Diverse and Informative Natural Language Fashion Feedback

abstract: Recent advances in multi-modal vision and language tasks enable a new set of applications. In this paper, we consider the task of generating natural language fashion feedback on outfit images. We collect a unique dataset, which contains outfit images and corresponding positive and constructive fashion feedback. We treat each feedback type separately, and train deep generative encoder-decoder models with visual attention, similar to the standard image captioning pipeline. Following this approach, the generated sentences tend to be too general and non-informative. We propose an alternative decoding technique based on the Maximum Mutual Information objective function, which leads to more diverse and detailed responses. We evaluate our model with common language metrics, and also show human evaluation results. This technology is applied within the ``Alexa, how do I look?'' feature, publicly available in Echo Look devices.


title: Joint Visual-Textual Embedding for Multimodal Style Search

abstract: We introduce a multimodal visual-textual search refinement method for fashion garments. Existing search engines do not enable intuitive, interactive, refinement of retrieved results based on the properties of a particular product. We propose a method to retrieve similar items, based on a query item image and textual refinement properties. We believe this method can be leveraged to solve many real-life customer scenarios, in which a similar item in a different color, pattern, length or style is desired. We employ a joint embedding training scheme in which product images and their catalog textual metadata are mapped closely in a shared space. This joint visual-textual embedding space enables manipulating catalog images semantically, based on textual refinement requirements. We propose a new training objective function, Mini-Batch Match Retrieval, and demonstrate its superiority over the commonly used triplet loss. Additionally, we demonstrate the feasibility of adding an attribute extraction module, trained on the same catalog data, and demonstrate how to integrate it within the multimodal search to boost its performance. We introduce an evaluation protocol with an associated benchmark, and compare several approaches.


title: The transition to the ultimate regime of thermal convection from a stochastic one-dimensional turbulence perspective

abstract: The Rayleigh number $Ra$ dependence of the Nusselt number $Nu$ in turbulent Rayleigh--B\'enard convection is numerically investigated for a moderate and low Prandtl number, $Pr=0.7$ and $0.021$, respectively. Here we specifically address the case of a Boussinesq fluid in a planar configuration with smooth horizontal walls and notionally infinite aspect ratio. Numerical simulations up to $Ra=10^{16}$ for $Pr=0.7$ and up to $Ra=8\times10^{13}$ for $Pr=0.021$ are made feasible on state-of-the-art workstations by utilising the stochastic one-dimensional turbulence (ODT) model. The ODT model parameters were estimated once for two combinations $(Pr,Ra)$ in the classical regime and kept fixed afterwards in order to address the predictive capabilities of the model. The ODT results presented exhibit various effective Nusselt number scalings $Nu\propto Ra^b$. The exponent changes from $b\approx1/3$ to $b\approx1/2$ when the $Ra$ number increases beyond the critical value $Ra_*\simeq6\times10^{14}$ ($Pr=0.7$) and $Ra_*\simeq6\times10^{11}$ ($Pr=0.021$), respectively. This is consistent with the literature. Furthermore, our results suggest that the transition to the ultimate regime is correlated with a relative enhancement of the temperature-velocity cross-correlations in the bulk of the fluid as hypothesised by Kraichnan, R. H., Phys. Fluids, 5, 1374 (1962).


title: Achieving Conservation of Energy in Neural Network Emulators for Climate Modeling

abstract: Artificial neural-networks have the potential to emulate cloud processes with higher accuracy than the semi-empirical emulators currently used in climate models. However, neural-network models do not intrinsically conserve energy and mass, which is an obstacle to using them for long-term climate predictions. Here, we propose two methods to enforce linear conservation laws in neural-network emulators of physical models: Constraining (1) the loss function or (2) the architecture of the network itself. Applied to the emulation of explicitly-resolved cloud processes in a prototype multi-scale climate model, we show that architecture constraints can enforce conservation laws to satisfactory numerical precision, while all constraints help the neural-network better generalize to conditions outside of its training set, such as global warming.


title: PET/CT Radiomic Sequencer for Prediction of EGFR and KRAS Mutation Status in NSCLC Patients

abstract: The aim of this study was to develop radiomic models using PET/CT radiomic features with different machine learning approaches for finding best predictive epidermal growth factor receptor (EGFR) and Kirsten rat sarcoma viral oncogene (KRAS) mutation status. Patients images including PET and CT [diagnostic (CTD) and low dose CT (CTA)] were pre-processed using wavelet (WAV), Laplacian of Gaussian (LOG) and 64 bin discretization (BIN) (alone or in combinations) and several features from images were extracted. The prediction performance of model was checked using the area under the receiver operator characteristic (ROC) curve (AUC). Results showed a wide range of radiomic model AUC performances up to 0.75 in prediction of EGFR and KRAS mutation status. Combination of K-Best and variance threshold feature selector with logistic regression (LREG) classifier in diagnostic CT scan led to the best performance in EGFR (CTD-BIN+B-KB+LREG, AUC: mean 0.75 sd 0.10) and KRAS (CTD-BIN-LOG-WAV+B-VT+LREG, AUC: mean 0.75 sd 0.07) respectively. Additionally, incorporating PET, kept AUC values at ~0.74. When considering conventional features only, highest predictive performance was achieved by PET SUVpeak (AUC: 0.69) for EGFR and by PET MTV (AUC: 0.55) for KRAS. In comparison with conventional PET parameters such as standard uptake value, radiomic models were found as more predictive. Our findings demonstrated that non-invasive and reliable radiomics analysis can be successfully used to predict EGFR and KRAS mutation status in NSCLC patients.


title: Scalable Model Compression by Entropy Penalized Reparameterization

abstract: We describe a simple and general neural network weight compression approach, in which the network parameters (weights and biases) are represented in a "latent" space, amounting to a reparameterization. This space is equipped with a learned probability model, which is used to impose an entropy penalty on the parameter representation during training, and to compress the representation using a simple arithmetic coder after training. Classification accuracy and model compressibility is maximized jointly, with the bitrate--accuracy trade-off specified by a hyperparameter. We evaluate the method on the MNIST, CIFAR-10 and ImageNet classification benchmarks using six distinct model architectures. Our results show that state-of-the-art model compression can be achieved in a scalable and general way without requiring complex procedures such as multi-stage training.


title: Some results for the large time behavior of Hamilton-Jacobi Equations with Caputo Time Derivative

abstract: We obtain some H\"older regularity estimates for an Hamilton-Jacobi with fractional time derivative of order $\alpha \in (0,1)$ cast by a Caputo derivative. The H\"older seminorms are independent of time, which allows to investigate the large time behavior of the solutions. We focus on the Namah-Roquejoffre setting whose typical example is the Eikonal equation. Contrary to the classical time derivative case $\alpha=1$, the convergence of the solution on the so-called projected Aubry set, which is an important step to catch the large time behavior, is not straightforward. Indeed, a function with nonpositive Caputo derivative for all time does not necessarily converge; we provide such a counterexample. However, we establish partial results of convergence under some geometrical assumptions.


title: REMAP: Multi-layer entropy-guided pooling of dense CNN features for image retrieval

abstract: This paper addresses the problem of very large-scale image retrieval, focusing on improving its accuracy and robustness. We target enhanced robustness of search to factors such as variations in illumination, object appearance and scale, partial occlusions, and cluttered backgrounds - particularly important when search is performed across very large datasets with significant variability. We propose a novel CNN-based global descriptor, called REMAP, which learns and aggregates a hierarchy of deep features from multiple CNN layers, and is trained end-to-end with a triplet loss. REMAP explicitly learns discriminative features which are mutually-supportive and complementary at various semantic levels of visual abstraction. These dense local features are max-pooled spatially at each layer, within multi-scale overlapping regions, before aggregation into a single image-level descriptor. To identify the semantically useful regions and layers for retrieval, we propose to measure the information gain of each region and layer using KL-divergence. Our system effectively learns during training how useful various regions and layers are and weights them accordingly. We show that such relative entropy-guided aggregation outperforms classical CNN-based aggregation controlled by SGD. The entire framework is trained in an end-to-end fashion, outperforming the latest state-of-the-art results. On image retrieval datasets Holidays, Oxford and MPEG, the REMAP descriptor achieves mAP of 95.5%, 91.5%, and 80.1% respectively, outperforming any results published to date. REMAP also formed the core of the winning submission to the Google Landmark Retrieval Challenge on Kaggle.


title: Representation Quality Of Neural Networks Links To Adversarial Attacks and Defences

abstract: Neural networks have been shown vulnerable to a variety of adversarial algorithms. A crucial step to understanding the rationale for this lack of robustness is to assess the potential of the neural networks' representation to encode the existing features. Here, we propose a method to understand the representation quality of the neural networks using a novel test based on Zero-Shot Learning, entitled Raw Zero-Shot. The principal idea is that, if an algorithm learns rich features, such features should be able to interpret "unknown" classes as an aggregate of previously learned features. This is because unknown classes usually share several regular features with recognised classes, given the features learned are general enough. We further introduce two metrics to assess these learned features to interpret unknown classes. One is based on inter-cluster validation technique (Davies-Bouldin Index), and the other is based on the distance to an approximated ground-truth. Experiments suggest that adversarial defences improve the representation of the classifiers, further suggesting that to improve the robustness of the classifiers, one has to improve the representation quality also. Experiments also reveal a strong association (a high Pearson Correlation and low p-value) between the metrics and adversarial attacks. Interestingly, the results indicate that dynamic routing networks such as CapsNet have better representation while current deeper neural networks are trading off representation quality for accuracy. Code available at http://bit.ly/RepresentationMetrics.


title: General classification of light curves using extreme boosting

abstract: A significant degree of misclassification of variable stars through the application of machine learning methods to survey data motivates a search for more reliable and accurate machine learning procedures, especially in light of the very large data cubes that will be generated by future surveys and the need for immediate production of accurate, formalised catalogues of variable behaviour to enable science to proceed. In this study, the efficiency of an ensemble machine learning procedure utilising extreme boosting was determined by application to a large sample of data from the OGLE III and IV surveys and from the \textit{Kepler} mission. Through recursive training of classifiers, the study developed a variable star classification workflow which produced an average efficiency determined with the average precision of the model (0.81 for \textit{Kepler} and 0.91 for OGLE) and the $f-score$ of predictions on the test sets. This suggests that extreme boosting can be presented as one of the favourable shallow learning methods in developing a variable star classifier for future large survey projects.


title: Robust Federated Learning in a Heterogeneous Environment

abstract: We study a recently proposed large-scale distributed learning paradigm, namely Federated Learning, where the worker machines are end users' own devices. Statistical and computational challenges arise in Federated Learning particularly in the presence of heterogeneous data distribution (i.e., data points on different devices belong to different distributions signifying different clusters) and Byzantine machines (i.e., machines that may behave abnormally, or even exhibit arbitrary and potentially adversarial behavior). To address the aforementioned challenges, first we propose a general statistical model for this problem which takes both the cluster structure of the users and the Byzantine machines into account. Then, leveraging the statistical model, we solve the robust heterogeneous Federated Learning problem \emph{optimally}; in particular our algorithm matches the lower bound on the estimation error in dimension and the number of data points. Furthermore, as a by-product, we prove statistical guarantees for an outlier-robust clustering algorithm, which can be considered as the Lloyd algorithm with robust estimation. Finally, we show via synthetic as well as real data experiments that the estimation error obtained by our proposed algorithm is significantly better than the non-Byzantine-robust algorithms; in particular, we gain at least by 53\% and 33\% for synthetic and real data experiments, respectively, in typical settings.


title: Excursion Set Halos -- ExSHalos: A New Parameter Free Method for Fast Generation of Halo Catalogues

abstract: We develop a new, simple, fast and parameter-free method to construct dark matter halo catalogues. This method requires as inputs only the linear matter power spectrum and the threshold density for halo formation in linear theory. It directly uses excursion set ideas and Lagrangian perturbation theory to produce halo catalogues with the correct abundance, large scale power spectrum, bispectrum and velocity field. These halo catalogues can be used for the fast construction of mock galaxy catalogues, allowing for the evaluation of covariance matrices for multiple observables. Because of its robustness and predictive nature, this method can be easily adapted to produce catalogues with e.g. primordial non-Gaussianities, modified theories of gravity and non-standard dark energy models, enabling detailed studies of these models in the context of next-generation surveys. We implement this method in a C code, and present numerical comparisons with theoretical predictions as well as full N-body cosmological simulations.


title: Pre-Galois theory

abstract: We introduce and study a class of field extensions that we call pre-Galois; viz. extensions that become Galois after some linearly disjoint Galois base change. Among them are geometrically Galois extensions of k(T), with k a field: extensions that become Galois and remain of the same degree after passing to the algebraic closure of k. We develop a pre-Galois theory that includes a Galois correspondence, and investigate the corresponding variants of the inverse Galois problem. We provide answers in situations where the classical analogs are not known. In particular, for every finite simple group, some power is a geometric Galois group over k, and is a pre-Galois group over k if k is Hilbertian. For every finite group G, the same conclusion holds for G itself (n=1) if k is the maximal abelian extension E of the rationals and G has a weakly rigid tuple of conjugacy classes; and then G is a regular Galois group over an extension of E of degree dividing the order of Out(G). We also show that the inverse problem for pre-Galois extensions over a field k (that every finite group is a pre-Galois group over k) is equivalent to the a priori stronger inverse Galois problem over k, and similarly for the geometric vs. regular variants.


title: Image Captioning with Integrated Bottom-Up and Multi-level Residual Top-Down Attention for Game Scene Understanding

abstract: Image captioning has attracted considerable attention in recent years. However, little work has been done for game image captioning which has some unique characteristics and requirements. In this work we propose a novel game image captioning model which integrates bottom-up attention with a new multi-level residual top-down attention mechanism. Firstly, a lower-level residual top-down attention network is added to the Faster R-CNN based bottom-up attention network to address the problem that the latter may lose important spatial information when extracting regional features. Secondly, an upper-level residual top-down attention network is implemented in the caption generation network to better fuse the extracted regional features for subsequent caption prediction. We create two game datasets to evaluate the proposed model. Extensive experiments show that our proposed model outperforms existing baseline models.


title: Mixture separability loss in a deep convolutional network for image classification

abstract: In machine learning, the cost function is crucial because it measures how good or bad a system is. In image classification, well-known networks only consider modifying the network structures and applying cross-entropy loss at the end of the network. However, using only cross-entropy loss causes a network to stop updating weights when all training images are correctly classified. This is the problem of the early saturation. This paper proposes a novel cost function, called mixture separability loss (MSL), which updates the weights of the network even when most of the training images are accurately predicted. MSL consists of between-class and within-class loss. Between-class loss maximizes the differences between inter-class images, whereas within-class loss minimizes the similarities between intra-class images. We designed the proposed loss function to attach to different convolutional layers in the network in order to utilize intermediate feature maps. Experiments show that a network with MSL deepens the learning process and obtains promising results with some public datasets, such as Street View House Number (SVHN), Canadian Institute for Advanced Research (CIFAR), and our self-collected Inha Computer Vision Lab (ICVL) gender dataset.


title: On stabilizer-free weak Galerkin finite element methods on polytopal meshes

abstract: A stabilizing/penalty term is often used in finite element methods with discontinuous approximations to enforce connection of discontinuous functions across element boundaries. Removing stabilizers from discontinuous finite element methods will simplify the formulations and reduce programming complexity significantly. The goal of this paper is to introduce a stabilizer free weak Galerkin finite element method for second order elliptic equations on polytopal meshes in 2D or 3D. This new WG method keeps a simple symmetric positive definite form and can work on polygonal/polyheral meshes. Optimal order error estimates are established for the corresponding WG approximations in both a discrete $H^1$ norm and the $L^2$ norm. Numerical results are presented verifying the theorem.


title: Conditional Computation for Continual Learning

abstract: Catastrophic forgetting of connectionist neural networks is caused by the global sharing of parameters among all training examples. In this study, we analyze parameter sharing under the conditional computation framework where the parameters of a neural network are conditioned on each input example. At one extreme, if each input example uses a disjoint set of parameters, there is no sharing of parameters thus no catastrophic forgetting. At the other extreme, if the parameters are the same for every example, it reduces to the conventional neural network. We then introduce a clipped version of maxout networks which lies in the middle, i.e. parameters are shared partially among examples. Based on the parameter sharing analysis, we can locate a limited set of examples that are interfered when learning a new example. We propose to perform rehearsal on this set to prevent forgetting, which is termed as conditional rehearsal. Finally, we demonstrate the effectiveness of the proposed method in an online non-stationary setup, where updates are made after each new example and the distribution of the received example shifts over time.


title: Experimental Evidence of the Topological Surface States in Mg3Bi2 Films Grown by Molecular Beam Epitaxy

abstract: Type-II nodal line semimetal (NLS) is a new quantum state hosting one-dimensional closed loops formed by the crossing of two bands which have the same sign in their slopes along the radial direction of the loop. According to the theoretical prediction, Mg3Bi2 is an ideal candidate for studying the type-II NLS by tuning its spin-orbit coupling (SOC). In this paper, high quality Mg3Bi2 films are grown by molecular beam epitaxy (MBE). By in-situ angle resolved photoemission spectroscopy (ARPES), a pair of surface resonance bands (SRBs) around Gamma point is clearly seen. It shows that Mg3Bi2 films grown by MBE is Mg(1)-terminated by comparing the ARPES data with the first principles calculations results. And, the temperature dependent weak anti-localization (WAL) effect in Mg3Bi2 films is observed under low magnetic field, which shows a clear two dimensional (2D) e-e scattering characteristics by fitting with the Hikami-Larkin-Nagaoka (HLN) model. Combining with ARPES, magneto-transport measurements and the first principles calculations, this work proves that Mg3Bi2 is a semimetal with topological surface states TSSs, which paves the way for Mg3Bi2 as an ideal materials platform for studying the exotic features of type-II nodal line semimetals (NLSs) and the topological phase transition by tuning its SOC.


title: A Closer Look at Double Backpropagation

abstract: In recent years, an increasing number of neural network models have included derivatives with respect to inputs in their loss functions, resulting in so-called double backpropagation for first-order optimization. However, so far no general description of the involved derivatives exists. Here, we cover a wide array of special cases in a very general Hilbert space framework, which allows us to provide optimized backpropagation rules for many real-world scenarios. This includes the reduction of calculations for Frobenius-norm-penalties on Jacobians by roughly a third for locally linear activation functions. Furthermore, we provide a description of the discontinuous loss surface of ReLU networks both in the inputs and the parameters and demonstrate why the discontinuities do not pose a big problem in reality.


title: ALMA view of the $^{12}$C/$^{13}$C isotopic ratio in starburst galaxies

abstract: We derive molecular-gas-phase $^{12}$C/$^{13}$C isotope ratios for the central few 100 pc of the three nearby starburst galaxies NGC 253, NGC 1068, and NGC 4945 making use of the $\lambda$ $\sim$ 3 mm $^{12}$CN and $^{13}$CN $N$ = 1--0 lines in the ALMA Band 3. The $^{12}$C/$^{13}$C isotopic ratios derived from the ratios of these lines range from 30 to 67 with an average of 41.6 $\pm$ 0.2 in NGC 253, from 24 to 62 with an average of 38.3 $\pm$ 0.4 in NGC 1068, and from 6 to 44 with an average of 16.9 $\pm$ 0.3 in NGC 4945. The highest $^{12}$C/$^{13}$C isotopic ratios are determined in some of the outskirts of the nuclear regions of the three starburst galaxies. The lowest ratios are associated with the northeastern and southwestern molecular peaks of NGC 253, the northeastern and southwestern edge of the mapped region in NGC 1068, and the very center of NGC 4945. In case of NGC 1068, the measured ratios suggest inflow from the outer part of NGC 1068 into the circum-nuclear disk through both the halo and the bar. Low $^{12}$C/$^{13}$C isotopic ratios in the central regions of these starburst galaxies indicate the presence of highly processed material.


title: Reinforcement Learning Driven Heuristic Optimization

abstract: Heuristic algorithms such as simulated annealing, Concorde, and METIS are effective and widely used approaches to find solutions to combinatorial optimization problems. However, they are limited by the high sample complexity required to reach a reasonable solution from a cold-start. In this paper, we introduce a novel framework to generate better initial solutions for heuristic algorithms using reinforcement learning (RL), named RLHO. We augment the ability of heuristic algorithms to greedily improve upon an existing initial solution generated by RL, and demonstrate novel results where RL is able to leverage the performance of heuristics as a learning signal to generate better initialization. We apply this framework to Proximal Policy Optimization (PPO) and Simulated Annealing (SA). We conduct a series of experiments on the well-known NP-complete bin packing problem, and show that the RLHO method outperforms our baselines. We show that on the bin packing problem, RL can learn to help heuristics perform even better, allowing us to combine the best parts of both approaches.


title: Hamilton's Dynamics in Complex Phase Space

abstract: We present the basic formulation of Hamilton dynamics in complex phase space. We extend the Hamilton's function by including the imaginary part and find out the corresponding Hamilton's canonical equation of motion. Example of simple harmonic motion are considered and the corresponding trajectory are plotted on real and complex phase space.


title: Fitting formulae for evolution tracks of massive stars under extreme metal poor environments for population synthesis calculations and star cluster simulations

abstract: We have devised fitting formulae for evolution tracks of massive stars with $8 \lesssim M/M_\odot \lesssim 160$ under extreme metal poor (EMP) environments for $\log (Z/Z_\odot) = -2, -4, -5, -6$, and $-8$, where $M_\odot$ and $Z_\odot$ are the solar mass and metallicity, respectively. Our fitting formulae are based on reference stellar models which we have newly obtained by simulating the time evolutions of EMP stars. Our fitting formulae take into account stars ending with blue supergiant (BSG) stars, and stars skipping Hertzsprung gap (HG) phases and blue loops, which are characteristics of massive EMP stars. In our fitting formulae, stars may remain BSG stars when they finish their core Helium burning (CHeB) phases. Our fitting formulae are in good agreement with our stellar evolution models. We can use these fitting formulae on the SSE, BSE, NBODY4, and NBODY6 codes, which are widely used for population synthesis calculations and star cluster simulations. These fitting formulae should be useful to make theoretical templates of binary black holes formed under EMP environments.


title: A Simple Local Minimal Intensity Prior and An Improved Algorithm for Blind Image Deblurring

abstract: Blind image deblurring is a long standing challenging problem in image processing and low-level vision. Recently, sophisticated priors such as dark channel prior, extreme channel prior, and local maximum gradient prior, have shown promising effectiveness. However, these methods are computationally expensive. Meanwhile, since these priors involved subproblems cannot be solved explicitly, approximate solution is commonly used, which limits the best exploitation of their capability. To address these problems, this work firstly proposes a simplified sparsity prior of local minimal pixels, namely patch-wise minimal pixels (PMP). The PMP of clear images is much more sparse than that of blurred ones, and hence is very effective in discriminating between clear and blurred images. Then, a novel algorithm is designed to efficiently exploit the sparsity of PMP in deblurring. The new algorithm flexibly imposes sparsity inducing on the PMP under the MAP framework rather than directly uses the half quadratic splitting algorithm. By this, it avoids non-rigorous approximation solution in existing algorithms, while being much more computationally efficient. Extensive experiments demonstrate that the proposed algorithm can achieve better practical stability compared with state-of-the-arts. In terms of deblurring quality, robustness and computational efficiency, the new algorithm is superior to state-of-the-arts. Code for reproducing the results of the new method is available at https://github.com/FWen/deblur-pmp.git.


title: Throughput Analysis of CSMA: Technical Report

abstract: In this technical report, the throughput performance of CSMA networks with two representative receiver structures, i.e., the collision model and the capture model, is characterized and optimized. The analysis is further applied to an IEEE 802.11 network, which is a representative wireless network that adopts the CSMA mechanism, where the optimal initial backoff window sizes of nodes to achieve the maximum network throughput are derived and verified against simulation results.


title: Gupta-Bleuler quantization for linearized gravity in de Sitter spacetime

abstract: In a recent Letter, we have pointed out that the linearized Einstein gravity in de Sitter (dS) spacetime besides the spacetime symmetries generated by the Killing vectors and the evident gauge symmetry also possesses a hitherto `hidden' local (gauge-like) symmetry which becomes anomalous on the quantum level. This gauge-like anomaly makes the theory inconsistent and must be canceled at all costs. In this companion paper, we first review our argument and discuss it in more detail. We argue that the cancelation of this anomaly makes it impossible to preserve dS symmetry in linearized quantum gravity through the usual canonical quantization in a consistent manner. Then, demanding that all the classical symmetries to survive in the quantized theory, we set up a coordinate-independent formalism \`{a} \emph{la} Gupta-Bleuler which allows for preserving the (manifest) dS covariance in the presence of the gauge and the gauge-like invariance of the theory. On this basis, considering a new representation of the canonical commutation relations, we present a graviton quantum field on dS space, transforming correctly under isometries, gauge transformations, and gauge-like transformations, which acts on a state space containing a vacuum invariant under all of them. Despite the appearance of negative norm states in this quantization scheme, the energy operator is positive in all physical states, and vanishes in the vacuum.


title: Finding optimal solutions by stochastic cellular automata

abstract: Finding a ground state of a given Hamiltonian is an important but hard problem. One of the potential methods is to use a Markov chain Monte Carlo (MCMC) to sample the Gibbs distribution whose highest peaks correspond to the ground states. In this short paper, we use stochastic cellular automata (SCA) and see if it is possible to find a ground state faster than the conventional MCMCs, such as the Glauber dynamics. We show that, if the temperature is sufficiently high, it is possible for SCA to have more spin-flips per update in average than Glauber and, at the same time, to have an equilibrium distribution ``close" to the one for Glauber, i.e., the Gibbs distribution. During the course, we also propose a new way to characterize how close a probability measure is to the target Gibbs.


title: Sample Size Calculations for SMARTs

abstract: Sequential Multiple Assignment Randomized Trials (SMARTs) are considered the gold standard for estimation and evaluation of treatment regimes. SMARTs are typically sized to ensure sufficient power for a simple comparison, e.g., the comparison of two fixed treatment sequences. Estimation of an optimal treatment regime is conducted as part of a secondary and hypothesis-generating analysis with formal evaluation of the estimated optimal regime deferred to a follow-up trial. However, running a follow-up trial to evaluate an estimated optimal treatment regime is costly and time-consuming; furthermore, the estimated optimal regime that is to be evaluated in such a follow-up trial may be far from optimal if the original trial was underpowered for estimation of an optimal regime. We derive sample size procedures for a SMART that ensure: (i) sufficient power for comparing the optimal treatment regime with standard of care; and (ii) the estimated optimal regime is within a given tolerance of the true optimal regime with high-probability. We establish asymptotic validity of the proposed procedures and demonstrate their finite sample performance in a series of simulation experiments.


abstract: The paper is devoted to weighted $L^p$-Hardy inequalities with best constants on Finsler metric measure manifolds. There are two major ingredients. The first, which is the main part of this paper, is the Hardy inequalities concerned with distance functions in the Finsler setting. In this case, we find that besides the flag curvature, the Ricci curvature together with two non-Riemannian quantities, i.e., reversibility and S-curvature, also play an important role. And we establish the optimal Hardy inequalities not only on noncompact manifolds, but also on closed manifolds. The second ingredient is the Hardy inequalities for Finsler $p$-sub/superharmonic functions, in which we also investigate the existence of extremals and the Brezis-V\'azquez improvement.


title: A Clark-Ocone type formula via Ito calculus and its application to finance

abstract: An explicit martingale representation for random variables described as a functional of a Levy process will be given. The Clark-Ocone theorem shows that integrands appeared in a martingale representation are given by conditional expectations of Malliavin derivatives. Our goal is to extend it to random variables which are not Malliavin differentiable. To this end, we make use of Ito's formula, instead of Malliavin calculus. As an application to mathematical finance, we shall give an explicit representation of locally risk-minimizing strategy of digital options for exponential Levy models. Since the payoff of digital options is described by an indicator function, we also discuss the Malliavin differentiability of indicator functions with respect to Levy processes.


title: Density Evolution Analysis of Partially Information Coupled Turbo Codes on the Erasure Channel

abstract: In this paper, we investigate the performance of a class of spatially coupled codes, namely partially information coupled turbo codes (PIC-TCs) over the binary erasure channel (BEC). This class of codes enjoy flexible code rate adjustment by varying the coupling ratio. Moreover, the coupling method can be directly applied to any component codes without changing the encoding and decoding architectures of the underlying component codes. However, the theoretical performance of PIC-TCs has not been fully investigated. For this work, we consider the codes that have coupling memory $m$ and study the corresponding graph model. We then derive the exact density evolution equations for these code ensembles with any given coupling ratio and coupling memory $m$ to precisely compute their belief propagation decoding thresholds for the BEC. Our simulation results verify the correctness of our theoretical analysis and also show better error performance over uncoupled turbo codes with a variety of code rates on the BEC.


title: Classification of quad-equations on a cuboctahedron

abstract: In this paper, we consider polynomials associated with faces and internal quadrilaterals of a cuboctahedron and classify them under the requirement that they are consistent. These polynomials give rise to a system of partial difference equations on a face-centred cubic lattice. Our results were motivated by $\tau$-functions related to discrete Painlev\'e equations.


title: Lattice Coding for Downlink Multiuser Transmission

abstract: In this thesis, we mainly investigate the lattice coding problem of the downlink communication between a base station and multiple users. The base station broadcasts a message containing each user's intended message. The capacity limit of such a system setting is already well-known while the design of practical coding and modulation schemes to approach the theoretical limit has not been fully studied and investigated in the literature. This thesis attempts to address this problem by providing a systematic design on lattice coding and modulation schemes for downlink multiuser communication systems. The main idea is to exploit the structure property of lattices to harness interference from downlink users.


title: Staggered DG method for coupling of the Stokes and Darcy-Forchheimer problems

abstract: In this paper we develop a staggered discontinuous Galerkin method for the Stokes and Darcy-Forchheimer problems coupled with the \Red{Beavers-Joseph-Saffman} conditions. The method is defined by imposing staggered continuity for all the variables involved and the interface conditions are enforced by switching the roles of the variables met on the interface, which eliminate the hassle of introducing additional variables. This method can be flexibly applied to rough grids such as the highly distorted grids and the polygonal grids. In addition, the method allows nonmatching grids on the interface thanks to the special inclusion of the interface conditions, which is highly appreciated from a practical point of view. A new discrete trace inequality and a generalized Poincar\'{e}-Friedrichs inequality are proved, which enables us to prove the optimal convergence estimates under reasonable regularity assumptions. Finally, several numerical experiments are given to illustrate the performances of the proposed method, and the numerical results indicate that the proposed method is accurate and efficient, in addition, it is a good candidate for practical applications.


title: A Comprehensive Formal Security Analysis and Revision of the Two-phase Key Exchange Primitive of TPM 2.0

abstract: The Trusted Platform Module (TPM) version 2.0 provides a two-phase key exchange primitive which can be used to implement three widely-standardized authenticated key exchange protocols: the Full Unified Model, the Full MQV, and the SM2 key exchange protocols. However, vulnerabilities have been found in all of these protocols. Fortunately, it seems that the protections offered by TPM chips can mitigate these vulnerabilities. In this paper, we present a security model which captures TPM's protections on keys and protocols' computation environments and in which multiple protocols can be analyzed in a unified way. Based on the unified security model, we give the first formal security analysis of the key exchange primitive of TPM 2.0, and the analysis results show that, with the help of hardware protections of TPM chips, the key exchange primitive indeed satisfies the well-defined security property of our security model, but unfortunately under some impractical limiting conditions, which would prevent the application of the key exchange primitive in real-world networks. To make TPM 2.0 applicable to real-world networks, we present a revision of the key exchange primitive of TPM 2.0, which can be secure without the limiting conditions. We give a rigorous analysis of our revision, and the results show that our revision achieves not only the basic security property of modern AKE security models but also some further security properties.


title: Quasinormal modes of three-dimensional rotating Ho\v{r}ava AdS black hole and the approach to thermal equilibrium

abstract: We compute the quasinormal modes (QNMs) of a massive scalar field in the background of a rotating three-dimensional Ho\v{r}ava AdS black hole, and we analyze the effect of the breaking of the Lorentz invariance on the QNMs. Imposing on the horizon that there are only ingoing waves and at infinity the Dirichlet boundary conditions and the Neumann boundary condition, we calculate the oscillatory and the decay modes of the QNMs. We find that the propagation of the scalar field is stable in this background and employing the holographic principle we find the different times of the perturbed system to reach thermal equilibrium for the various branches of solutions.


title: Decoupling PDE Computation with Intrinsic or Inertial Robin Interface Condition

abstract: We study decoupled numerical methods for multi-domain, multi-physics applications. By investigating various stages of numerical approximation and decoupling and tracking how the information is transmitted across the interface for a typical multi-modeling model problem, we derive an approximate intrinsic or inertial type Robin condition for its semi-discrete model. This new interface condition is justified both mathematically and physically in contrast to the classical Robin interface condition conventionally introduced for decoupling multi-modeling problems. Based on the intrinsic or inertial Robin condition, an equivalent semi-discrete model is introduced, which provides a general framework for devising effective decoupled numerical methods. Numerical experiments also confirm the effectiveness of this new decoupling approach.


title: Characterization of the norm-based robust solutions in vector optimization

abstract: In this paper, we study the norm-based robust (efficient) solutions of a Vector Optimization Problem (VOP). We define two kinds of non-ascent directions in terms of Clarke's generalized gradient and characterize norm-based robustness by means of the newly-defined directions. This is done under a basic Constraint Qualification (CQ). We extend the provided characterization to VOPs with conic constraints. Moreover, we derive a necessary condition for norm-based robustness utilizing a nonsmooth gap function.


title: Hypergraphs without exponents

abstract: Here we give a short, concise proof for the following result. There exists a $k$-uniform hypergraph $H$ (for $k\geq 5$) without exponent, i.e., when the Tur\'an function is not polynomial in $n$. More precisely, we have $ex(n,H)=o(n^{k-1})$ but it exceeds $n^{k-1-c}$ for any positive $c$ for $n> n_0(k,c)$. This is an extension (and simplification) of a result of Frankl and the first author from 1987 where the case $k=5$ was proven. We conjecture that it is true for $k\in \{3, 4\}$ as well.


title: A K\"ahler structure for the ${\rm PU}(2, 1)$ configuration space of four points in $S^3$

abstract: We show that an open subset ${\mathfrak F}_4''$ of the ${\rm PU}(2,1)$ configuration space of four points in $S^3$ is in bijection with an open subset of %with a K\"ahler structure which is inherited from the one of ${\mathfrak H}^{\star}\times\mathbb{R}_{>0}$, where ${\mathfrak H}^\star$ is the affine-rotational group. Since the latter is a Sasakian manifold, the cone ${\mathfrak H}^\star\times\mathbb{R}_{>0}$ is K\"ahler and thus ${\mathfrak F}_4''$ inherits this K\"ahler structure.


title: A Generalized Minimax Q-learning Algorithm for Two-Player Zero-Sum Stochastic Games

abstract: We consider the problem of two-player zero-sum games. This problem is formulated as a min-max Markov game in the literature. The solution of this game, which is the min-max payoff, starting from a given state is called the min-max value of the state. In this work, we compute the solution of the two-player zero-sum game utilizing the technique of successive relaxation that has been successfully applied in the literature to compute a faster value iteration algorithm in the context of Markov Decision Processes. We extend the concept of successive relaxation to the setting of two-player zero-sum games. We show that, under a special structure on the game, this technique facilitates faster computation of the min-max value of the states. We then derive a generalized minimax Q-learning algorithm that computes the optimal policy when the model information is not known. Finally, we prove the convergence of the proposed generalized minimax Q-learning algorithm utilizing stochastic approximation techniques, under an assumption on the boundedness of iterates. Through experiments, we demonstrate the effectiveness of our proposed algorithm.


title: QSPR analysis of some novel neighborhood degree based topological descriptors

abstract: Topological index is a numerical value associated with chemical constitution for correlation of chemical structure with various physical properties, chemical reactivity or biological activity. In this work, some new indices based on neighbourhood degree sum of nodes are proposed. To make the computation of the novel indices convenient, an algorithm is designed. QSPR analysis of these newly introduced indices are studied here which reveals their predicting power. Some mathematical properties of these indices are also discussed here.


title: Static Correlation Density Functional Theory

abstract: Over the years, several schemes have been proposed to describe multireference systems with Kohn-Sham Density Functional Theory. Problematic is the combination of two aspects: the Kohn-Sham reference wavefunction is usually taken to be a single Slater determinant, and approximate exchange-correlation functionals are typically derived form the local density approximation. In this work, we develop a theoretical framework that foregoes the single Slater determinant and instead employs thermal states as reference states for zero-temperature interacting systems. We provide convenient definitions of static and dynamic correlation functionals via an adiabatic connection approach. The formalism and computational results indicate that the entropic term of the thermal reference state is a good approximation to the static correlation functional. Hence, this work validates several reported results in the literature and motivates additional developments of static correlation density functionals.


title: Comparison between Variational Monte Carlo and Shell Model Calculations of Neutrinoless Double Beta Decay Matrix Elements in Light Nuclei

abstract: Benchmark comparisons between many-body methods are performed to assess the ingredients necessary for an accurate calculation of neutrinoless double beta decay matrix elements. Shell model and variational Monte Carlo (VMC) calculations are carried out for $A=10$ and $12$ nuclei. Different variational wavefunctions are used to evaluate the uncertainties in the {\it ab initio} calculations, finding fairly small differences between the VMC double beta decay matrix elements. For shell model calculations, the role of model space truncation, radial wavefunction choices, and short-range correlation are investigated and all found to be important. Based on the detailed comparisons between the VMC and shell model approaches, we conclude that accurate descriptions of neutrinoless double beta decay matrix elements require a proper treatment of both long-range and short-range correlations.


title: Sampler for Composition Ratio by Markov Chain Monte Carlo

abstract: Invention involves combination, or more precisely, ratios of composition. According to Thomas Edison, "Genius is one percent inspiration and 99 percent perspiration" is an example. In many situations, researchers and inventors already have a variety of data and manage to create something new by using it, but the key problem is how to select and combine knowledge. In this paper, we propose a new Markov chain Monte Carlo (MCMC) algorithm to generate composition ratios, nonnegative-integer-valued vectors with two properties: (i) the sum of the elements of each vector is constant, and (ii) only a small number of elements is nonzero. These constraints make it difficult for existing MCMC algorithms to sample composition ratios. The key points of our approach are (1) designing an appropriate target distribution by using a condition on the number of nonzero elements, and (2) changing values only between a certain pair of elements in each iteration. Through an experiment on creating a new cocktail, we show that the combination of the proposed method with supervised learning can solve a creative problem.


title: Mesoscale investigations of fluid-solid interaction: Liquid slip flow in a parallel-plate microchannel

abstract: Liquid slip flow with a Knudsen number Kn = 0.001-0.1 plays a dominant role in confined flow channels. Its physical origin can be attributed to the intermolecular fluid-solid (F-S) interaction force. To this end, we propose herein continuous force functions (decaying either exponentially or by a power law) between fluid particles and two confined flat walls in the framework of the mesoscopic lattice Boltzmann model (LBM). The analytical solutions for density profile, velocity profile, slip length, and permeability ratio are derived and related to the mesoscale F-S interaction parameters and the size of the gap of the flow channel. Through nondimensionalization of the analytical solutions, we obtain the dimensionless numbers that indicate the key feature of slip-flow systems for each of the proposed force functions. The analytical solutions are strictly consistent with the LBM solutions. We suggest reasonable ranges for the F-S interaction parameters based on the observed range of density ratio (film fluid to bulk fluid) and the increasing permeability ratios with narrowing gap size. Within the given range of interaction parameters, simple relationships between permeability ratios and dimensionless numbers are provided by fitting. The curves for continuous F-S interaction force with two free parameters are calibrated for a hydrophobic surface by using LBM simulations, which were validated a priori by comparison with the slip velocity profile measured in a benchmark flow experiment. The mesoscopic LBM model based on the proposed F-S interaction force functions provides a robust framework to elucidate the physical process of liquid slip flow.

categories: physics.flu-dyn cond-mat.mes-hall physics.comp-ph physics.geo-ph


title: On the Properties of the Synthetic Control Estimator with Many Periods and Many Controls

abstract: We consider the asymptotic properties of the Synthetic Control (SC) estimator when both the number of pre-treatment periods and control units are large. If potential outcomes follow a linear factor model, we provide conditions under which the factor loadings of the SC unit converge in probability to the factor loadings of the treated unit. This happens when there are weights diluted among an increasing number of control units such that a weighted average of the factor loadings of the control units asymptotically reconstructs the factor loadings of the treated unit. In this case, the SC estimator is asymptotically unbiased even when treatment assignment is correlated with time-varying unobservables. This result can be valid even when the number of control units is larger than the number of pre-treatment periods.


title: Addressing database variability in learning from medical data: an ensemble-based approach using convolutional neural networks and a case of study applied to automatic sleep scoring

abstract: In this work we examine some of the problems associated with the development of machine learning models with the objective to achieve robust generalization capabilities on common-task multiple-database scenarios. Referred to as the "database variability problem", we focus on a specific medical domain (sleep staging in sleep medicine) to show the non-triviality of translating the estimated model's local generalization capabilities into independent external databases. We analyze some of the scalability problems when multiple-database data are used as inputs to train a single learning model. Then, we introduce a novel approach based on an ensemble of local models, and we show its advantages in terms of inter-database generalization performance and data scalability. In addition, we analyze different model configurations and data pre-processing techniques to determine their effects on the overall generalization performance. For this purpose, we carry out experimentation that involves several sleep databases and evaluates different machine learning models based on convolutional neural networks


title: Conformal Symmetry, Accelerated Observers and Nonlocality

abstract: The acceleration transformations form a 4-parameter Abelian subgroup of the conformal group of Minkowski spacetime. The passive interpretation of acceleration transformations leads to a congruence of uniformly accelerated observers in Minkowski spacetime. The properties of this congruence are studied in order to illustrate the kinematics of accelerated observers in relativistic physics. The generalization of this approach under conformal rescaling of the spacetime metric is examined.


title: Principles alone cannot guarantee ethical AI

abstract: AI Ethics is now a global topic of discussion in academic and policy circles. At least 84 public-private initiatives have produced statements describing high-level principles, values, and other tenets to guide the ethical development, deployment, and governance of AI. According to recent meta-analyses, AI Ethics has seemingly converged on a set of principles that closely resemble the four classic principles of medical ethics. Despite the initial credibility granted to a principled approach to AI Ethics by the connection to principles in medical ethics, there are reasons to be concerned about its future impact on AI development and governance. Significant differences exist between medicine and AI development that suggest a principled approach in the latter may not enjoy success comparable to the former. Compared to medicine, AI development lacks (1) common aims and fiduciary duties, (2) professional history and norms, (3) proven methods to translate principles into practice, and (4) robust legal and professional accountability mechanisms. These differences suggest we should not yet celebrate consensus around high-level principles that hide deep political and normative disagreement.


title: One Epoch Is All You Need

abstract: In unsupervised learning, collecting more data is not always a costly process unlike the training. For example, it is not hard to enlarge the 40GB WebText used for training GPT-2 by modifying its sampling methodology considering how many webpages there are in the Internet. On the other hand, given that training on this dataset already costs tens of thousands of dollars, training on a larger dataset naively is not cost-wise feasible. In this paper, we suggest to train on a larger dataset for only one epoch unlike the current practice, in which the unsupervised models are trained for from tens to hundreds of epochs. Furthermore, we suggest to adjust the model size and the number of iterations to be performed appropriately. We show that the performance of Transformer language model becomes dramatically improved in this way, especially if the original number of epochs is greater. For example, by replacing the training for 10 epochs with the one epoch training, this translates to 1.9-3.3x speedup in wall-clock time in our settings and more if the original number of epochs is greater. Under one epoch training, no overfitting occurs, and regularization method does nothing but slows down the training. Also, the curve of test loss over iterations follows power-law extensively. We compare the wall-clock time of the training of models with different parameter budget under one epoch training, and we show that size/iteration adjustment based on our proposed heuristics leads to 1-2.7x speedup in our cases. With the two methods combined, we achieve 3.3-5.1x speedup. Finally, we speculate various implications of one epoch training and size/iteration adjustment. In particular, based on our analysis we believe that we can reduce the cost to train the state-of-the-art models as BERT and GPT-2 dramatically, maybe even by the factor of 10.


title: Point g\'en\'erique et saut du rang du groupe de Mordell-Weil

abstract: Let $k$ be a number field and $U$ a smooth integral $k$-variety. Let $X \to U$ be an abelian scheme. We consider the set $\mathcal{R}$ of rational points $m \in U(k)$ such that the Mordell-Weil rank of the fibre $U_{m}$ is strictly bigger than the Mordell-Weil rank of the generic fibre. We prove the following results. If the $k$-variety $X$ is $k$-unirational, then $\mathcal{R}$ is dense for the Zariski topology on $U$. If $X$ is $k$-rational, then $\mathcal{R}$ is not thin in $U$. This generalizes results of Billard and of Salgado. The main idea goes back to N\'eron's thesis: use the generic point of the generic fibre of the family.


title: Quantifying Dismantlement in Disconnected Networks

abstract: We propose a novel measure to quantify dismantlement of a fragmented network. The existing measure of dismantlement used to study problems like optimal percolation is usually the size of the largest component of the network. We modify the measure of uniformity used to prove the Szemeredi's Regularity Lemma to obtain the proposed measure. The proposed measure incorporates the notion that the measure of dismantlement increases as the number of disconnected components increase and decreases as the variance of sizes of these components increases.


title: On "Optimal" h-Independent Convergence of Parareal and MGRIT using Runge-Kutta Time Integration

abstract: Although convergence of the Parareal and multigrid-reduction-in-time (MGRIT) parallel-in-time algorithms is well studied, results on their optimality is limited. Appealling to recently derived tight bounds of two-level Parareal and MGRIT convergence, this paper proves (or disproves) $h_x$- and $h_t$-independent convergence of two-level Parareal and MGRIT, for linear problems of the form $\mathbf{u}'(t) + \mathcal{L}\mathbf{u}(t) = f(t)$, where $\mathcal{L}$ is symmetric positive definite and Runge-Kutta time integration is used. The theory presented in this paper also encompasses analysis of some modified Parareal algorithms, such as the $\theta$-Parareal method, and shows that not all Runge-Kutta schemes are equal from the perspective of parallel-in-time. Some schemes, particularly L-stable methods, offer significantly better convergence than others as they are guaranteed to converge rapidly at both limits of small and large $h_t\xi$, where $\xi$ denotes an eigenvalue of $\mathcal{L}$ and $h_t$ time-step size. On the other hand, some schemes do not obtain $h$-optimal convergence, and two-level convergence is restricted to certain regimes. In certain cases, an $\mathcal{O}(1)$ factor change in time step $h_t$ or coarsening factor $k$ can be the difference between convergence factors $\rho\approx0.02$ and divergence! The analysis is extended to skew symmetric operators as well, which cannot obtain $h$-independent convergence and, in fact, will generally not converge for a sufficiently large number of time steps. Numerical results confirm the analysis in practice and emphasize the importance of a priori analysis in choosing an effective coarse-grid scheme and coarsening factor. A Mathematica notebook to perform a priori two-grid analysis is available at https://github.com/XBraid/xbraid-convergence-est.