title: Fast estimation of propagation constants in crossed gratings

abstract: Fourier-based modal methods are among the most effective numerical tools for the accurate analysis of crossed gratings. However, leading to computationally expensive eigenvalue equations significantly restricts their applicability, particularly when large truncation orders are required. The resultant eigenvalues are the longitudinal propagation constants of the grating and play a key role in applying the boundary conditions, as well as in the convergence and stability analyses. This paper aims to propose simple techniques for the fast estimation of propagation constants in crossed gratings, predominantly with no need to solve an eigenvalue equation. In particular, we show that for regular optical gratings comprised of lossless dielectrics, nearly every propagation constant appears on the main diagonal of the modal matrix.


title: Vision Transformers with Hierarchical Attention

abstract: This paper tackles the high computational/space complexity associated with Multi-Head Self-Attention (MHSA) in vanilla vision transformers. To this end, we propose Hierarchical MHSA (H-MHSA), a novel approach that computes self-attention in a hierarchical fashion. Specifically, we first divide the input image into patches as commonly done, and each patch is viewed as a token. Then, the proposed H-MHSA learns token relationships within local patches, serving as local relationship modeling. Then, the small patches are merged into larger ones, and H-MHSA models the global dependencies for the small number of the merged tokens. At last, the local and global attentive features are aggregated to obtain features with powerful representation capacity. Since we only calculate attention for a limited number of tokens at each step, the computational load is reduced dramatically. Hence, H-MHSA can efficiently model global relationships among tokens without sacrificing fine-grained information. With the H-MHSA module incorporated, we build a family of Hierarchical-Attention-based Transformer Networks, namely HAT-Net. To demonstrate the superiority of HAT-Net in scene understanding, we conduct extensive experiments on fundamental vision tasks, including image classification, semantic segmentation, object detection, and instance segmentation. Therefore, HAT-Net provides a new perspective for vision transformers. Code and pretrained models are available at https://github.com/yun-liu/HAT-Net.


title: Transient Chaos in BERT

abstract: Language is an outcome of our complex and dynamic human-interactions and the technique of natural language processing (NLP) is hence built on human linguistic activities. Bidirectional Encoder Representations from Transformers (BERT) has recently gained its popularity by establishing the state-of-the-art scores in several NLP benchmarks. A Lite BERT (ALBERT) is literally characterized as a lightweight version of BERT, in which the number of BERT parameters is reduced by repeatedly applying the same neural network called Transformer's encoder layer. By pre-training the parameters with a massive amount of natural language data, ALBERT can convert input sentences into versatile high-dimensional vectors potentially capable of solving multiple NLP tasks. In that sense, ALBERT can be regarded as a well-designed high-dimensional dynamical system whose operator is the Transformer's encoder, and essential structures of human language are thus expected to be encapsulated in its dynamics. In this study, we investigated the embedded properties of ALBERT to reveal how NLP tasks are effectively solved by exploiting its dynamics. We thereby aimed to explore the nature of human language from the dynamical expressions of the NLP model. Our short-term analysis clarified that the pre-trained model stably yields trajectories with higher dimensionality, which would enhance the expressive capacity required for NLP tasks. Also, our long-term analysis revealed that ALBERT intrinsically shows transient chaos, a typical nonlinear phenomenon showing chaotic dynamics only in its transient, and the pre-trained ALBERT model tends to produce the chaotic trajectory for a significantly longer time period compared to a randomly-initialized one. Our results imply that local chaoticity would contribute to improving NLP performance, uncovering a novel aspect in the role of chaotic dynamics in human language behaviors.


title: Large-time asymptotic of heavy tailed renewal processes

abstract: We study the large-time asymptotic of renewal-reward processes with a heavy-tailed waiting time distribution. It is known that the heavy tail of the distribution produces an extremely slow dynamics, resulting in a singular large deviation function. When the singularity takes place, the bottom of the large deviation function is flattened, manifesting anomalous fluctuations of the renewal-reward processes. In this article, we aim to study how these singularities emerge as the time increases. Using a classical result on the sum of random variables with regularly varying tail, we develop an expansion approach to prove an upper bound of the finite-time moment generating function for the Pareto waiting time distribution (power law) with an integer exponent. We perform numerical simulations using Pareto (with a real value exponent), inverse Rayleigh and log-normal waiting time distributions, and demonstrate similar results are anticipated in these waiting time distributions.

categories: math-ph cond-mat.stat-mech math.MP math.PR


title: Minkowski bases, Korkin-Zolotarev bases and successive minima

abstract: Let $\lambda_k$ denote the $k$-th successive minimum of a lattice $L$. We study properties of the lengths of certain bases of $L$. If $v_1, \dots v_n$ is a basis which is reduced in the sense of Minkowski we show that $\lvert v_k \rvert^2 \leq \frac{k}{4} \lambda_{k}^2$ for $k = 6, 7$, confirming a conjecture of Sch\"urmann, and obtaining the first improvement of a classical bound by Van der Waerden. We construct a sequences of lattices where $\lvert v_n \rvert$ is significantly longer than the longest vector in a Korkin-Zolotarev reduced basis, answering a question of Sch\"urmann. In an appendix joint with Lior Hadassi we construct a lattice $L$ with the surprising property that any basis containing the shortest vector of $L$ is not the shortest basis.


title: The law of momentum conservation in free microwave cavities containing coherent radiation

abstract: Using classical electrodynamics, this work analyzes the dynamics of a closed microwave cavity as a function of its center of energy. Starting from the principle of momentum conservation, expressions for the maximum electromagnetic momentum stored in a free microwave cavity are obtained. Next, it is shown that, for coherent fields and special shape conditions, this momentum component may not completely average out to zero when the fields change in the transient regime. Non-zero conditions are illustrated for the asymmetric conical frustum whose exact modes can not be calculated analytically. One concludes that the electromagnetic momentum can be imparted to the mechanical body so as to displace it in relation to the original center of energy. However, the average time range of the effect is much shorter than any time regime of the experimental tests performed to measure presently, suggesting it has not been observed yet in copper-made resonators.


title: Tight Lower Bounds for the RMR Complexity of Recoverable Mutual Exclusion

abstract: We present a tight RMR complexity lower bound for the recoverable mutual exclusion (RME) problem, defined by Golab and Ramaraju \cite{GR2019a}. In particular, we show that any $n$-process RME algorithm using only atomic read, write, fetch-and-store, fetch-and-increment, and compare-and-swap operations, has an RMR complexity of $\Omega(\log n/\log\log n)$ on the CC and DSM model. This lower bound covers all realistic synchronization primitives that have been used in RME algorithms and matches the best upper bounds of algorithms employing swap objects (e.g., [5,6,10]). Algorithms with better RMR complexity than that have only been obtained by either (i) assuming that all failures are system-wide [7], (ii) employing fetch-and-add objects of size $(\log n)^{\omega(1)}$ [12], or (iii) using artificially defined synchronization primitives that are not available in actual systems [6,9].


title: Reverse Engineering the Neural Tangent Kernel

abstract: The development of methods to guide the design of neural networks is an important open challenge for deep learning theory. As a paradigm for principled neural architecture design, we propose the translation of high-performing kernels, which are better-understood and amenable to first-principles design, into equivalent network architectures, which have superior efficiency, flexibility, and feature learning. To this end, we constructively prove that, with just an appropriate choice of activation function, any positive-semidefinite dot-product kernel can be realized as either the NNGP or neural tangent kernel of a fully-connected neural network with only one hidden layer. We verify our construction numerically and demonstrate its utility as a design tool for finite fully-connected networks in several experiments.


title: Tempered Stable Autoregressive Models

abstract: In this article, we introduce and study a one sided tempered stable first order autoregressive model called TAR(1). Under the assumption of stationarity of the model, the marginal probability density function of the error term is found. It is shown that the distribution of the error term is infinitely divisible. Parameter estimation of the introduced TAR(1) process is done by adopting the conditional least square and method of moments based approach and the performance of the proposed methods are evaluated on simulated data. Also we study an autoregressive model of order one with tempered stable innovations. Using appropriate test statistic it is shown that the model fit very well on real and simulated data. Our models generalize the inverse Gaussian and one-sided stable autoregressive models existing in the literature.


title: Combinatorial Optimization for Panoptic Segmentation: A Fully Differentiable Approach

abstract: We propose a fully differentiable architecture for simultaneous semantic and instance segmentation (a.k.a. panoptic segmentation) consisting of a convolutional neural network and an asymmetric multiway cut problem solver. The latter solves a combinatorial optimization problem that elegantly incorporates semantic and boundary predictions to produce a panoptic labeling. Our formulation allows to directly maximize a smooth surrogate of the panoptic quality metric by backpropagating the gradient through the optimization problem. Experimental evaluation shows improvement by backpropagating through the optimization problem w.r.t. comparable approaches on Cityscapes and COCO datasets. Overall, our approach shows the utility of using combinatorial optimization in tandem with deep learning in a challenging large scale real-world problem and showcases benefits and insights into training such an architecture.


title: Discrete-to-Continuous Extensions: Lov\'asz extension, optimizations and eigenvalue problems

abstract: In this paper, we use various versions of Lov\'asz extension to systematically derive continuous formulations of problems from discrete mathematics. This will take place in the following context: (1) For combinatorial optimization problems in quotient form, we systematically develop equivalent continuous versions, thereby making tools from convex optimization, fractional programming and more general continuous algorithms like the stochastic subgradient method available for such optimization problems. (2) For some combinatorial quantities like Cheeger-type constants, we suggest a nonlinear eigenvalue problem for a pair of Lov\'asz extensions of certain functions, which encodes certain combinatorial structures. This theory has several applications to quantitative and combinatorial problems, including: (a) The equivalent continuous representations for the max $k$-cut problem, various Cheeger sets and isoperimetric constants are constructed. This also initiates a study of Dirichlet and Neumann 1-Laplacians on graphs, in which the nodal domain property and Cheeger-type equalities are presented. (b) Also, we derive a new equivalent continuous representation of the graph independence number, which can be compared with the Motzkin-Straus theorem. More importantly, an equivalent continuous optimization for the chromatic number is provided, which seems to be the first continuous representation of the graph vertex coloring number. We provide the first continuous reformulation of the frustration index in signed networks, and we find a connection to the so-called modularity measure. Graph matching numbers, submodular vertex covers and multiway partition problems can also be studied in this framework.


title: Normalized Sombor indices as complexity measures of random graphs

abstract: We perform a detailed computational study of the recently introduced Sombor indices on random graphs. Specifically, we apply Sombor indices on three models of random graphs: Erd\"os-R\'enyi graphs, random geometric graphs, and bipartite random graphs. Within a statistical random matrix theory approach, we show that the average values of Sombor indices, normalized to the order of the graph, scale with the graph average degree. Moreover, we discuss the application of average Sombor indices as complexity measures of random graphs and, as a consequence, we show that selected normalized Sombor indices are highly correlated with the Shannon entropy of the eigenvectors of the graph adjacency matrix.


title: An $L^p$-weak Galerkin method for second order elliptic equations in non-divergence form

abstract: This article presents a new primal-dual weak Galerkin method for second order elliptic equations in non-divergence form. The new method is devised as a constrained $L^p$-optimization problem with constraints that mimic the second order elliptic equation by using the discrete weak Hessian locally on each element. An equivalent min-max characterization is derived to show the existence and uniqueness of the numerical solution. Optimal order error estimates are established for the numerical solution under the discrete $W^{2,p}$ norm, as well as the standard $W^{1,p}$ and $L^p$ norms. An equivalent characterization of the optimization problem in term of a system of fixed-point equations via the proximity operator is presented. An iterative algorithm is designed based on the fixed-point equations to solve the optimization problems. Implementation of the iterative algorithm is studied and convergence of the iterative algorithm is established. Numerical experiments for both smooth and non-smooth coefficients problems are presented to verify the theoretical findings.


title: Let's be explicit about that: Distant supervision for implicit discourse relation classification via connective prediction

abstract: In implicit discourse relation classification, we want to predict the relation between adjacent sentences in the absence of any overt discourse connectives. This is challenging even for humans, leading to shortage of annotated data, a fact that makes the task even more difficult for supervised machine learning approaches. In the current study, we perform implicit discourse relation classification without relying on any labeled implicit relation. We sidestep the lack of data through explicitation of implicit relations to reduce the task to two sub-problems: language modeling and explicit discourse relation classification, a much easier problem. Our experimental results show that this method can even marginally outperform the state-of-the-art, in spite of being much simpler than alternative models of comparable performance. Moreover, we show that the achieved performance is robust across domains as suggested by the zero-shot experiments on a completely different domain. This indicates that recent advances in language modeling have made language models sufficiently good at capturing inter-sentence relations without the help of explicit discourse markers.


title: The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation

abstract: One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider only restricted domains, or are low quality because they are constructed using semi-automatic procedures. In this work, we introduce the FLORES-101 evaluation benchmark, consisting of 3001 sentences extracted from English Wikipedia and covering a variety of different topics and domains. These sentences have been translated in 101 languages by professional translators through a carefully controlled process. The resulting dataset enables better assessment of model quality on the long tail of low-resource languages, including the evaluation of many-to-many multilingual translation systems, as all translations are multilingually aligned. By publicly releasing such a high-quality and high-coverage dataset, we hope to foster progress in the machine translation community and beyond.


title: Robust Implicit Networks via Non-Euclidean Contractions

abstract: Implicit neural networks, a.k.a., deep equilibrium networks, are a class of implicit-depth learning models where function evaluation is performed by solving a fixed point equation. They generalize classic feedforward models and are equivalent to infinite-depth weight-tied feedforward networks. While implicit models show improved accuracy and significant reduction in memory consumption, they can suffer from ill-posedness and convergence instability. This paper provides a new framework, which we call Non-Euclidean Monotone Operator Network (NEMON), to design well-posed and robust implicit neural networks based upon contraction theory for the non-Euclidean norm $\ell_{\infty}$. Our framework includes (i) a novel condition for well-posedness based on one-sided Lipschitz constants, (ii) an average iteration for computing fixed-points, and (iii) explicit estimates on input-output Lipschitz constants. Additionally, we design a training problem with the well-posedness condition and the average iteration as constraints and, to achieve robust models, with the input-output Lipschitz constant as a regularizer. Our $\ell_{\infty}$ well-posedness condition leads to a larger polytopic training search space than existing conditions and our average iteration enjoys accelerated convergence. Finally, we evaluate our framework in image classification through the MNIST and the CIFAR-10 datasets. Our numerical results demonstrate improved accuracy and robustness of the implicit models with smaller input-output Lipschitz bounds. Code is available at https://github.com/davydovalexander/Non-Euclidean_Mon_Op_Net.


title: Meta-Learning Reliable Priors in the Function Space

abstract: When data are scarce meta-learning can improve a learner's accuracy by harnessing previous experience from related learning tasks. However, existing methods have unreliable uncertainty estimates which are often overconfident. Addressing these shortcomings, we introduce a novel meta-learning framework, called F-PACOH, that treats meta-learned priors as stochastic processes and performs meta-level regularization directly in the function space. This allows us to directly steer the probabilistic predictions of the meta-learner towards high epistemic uncertainty in regions of insufficient meta-training data and, thus, obtain well-calibrated uncertainty estimates. Finally, we showcase how our approach can be integrated with sequential decision making, where reliable uncertainty quantification is imperative. In our benchmark study on meta-learning for Bayesian Optimization (BO), F-PACOH significantly outperforms all other meta-learners and standard baselines.


title: Graded Hecke algebras, constructible sheaves and the p-adic Kazhdan--Lusztig conjecture

abstract: Graded Hecke algebras can be constructed in terms of equivariant cohomology and constructible sheaves on nilpotent cones. In earlier work, their standard modules and their irreducible modules where realized with such geometric methods. We pursue this setup to study properties of module categories of (twisted) graded Hecke algebras, in particular what happens geometrically upon formal completion with respect to a central character. We prove a version of the Kazhdan--Lusztig conjecture for (twisted) graded Hecke algebras. It expresses the multiplicity of an irreducible module in a standard module as the multiplicity of an equivariant local system in an equivariant perverse sheaf. This is applied to smooth representations of reductive p-adic groups. Under some conditions, we verify the p-adic Kazhdan--Lusztig conjecture from [Vogan]. Here the equivariant constructible sheaves live on certain varieties of Langlands parameters. The involved conditions are checked for substantial classes of groups and representations.


title: Robustness of Parameter Estimation Procedures for Bulk-Heterojunction Organic Solar Cells

abstract: Parameter estimation procedures provide valuable guidance in the understanding and improvement of organic solar cells and other devices. They often rely on one-dimensional models, but in the case of bulk-heterojunction (BHJ) designs, it is not straightforward that these models' parameters have a consistent physical interpretation. Indeed, contrarily to two- or three-dimensional models, the BHJ morphology is not explicitly described in one-dimensional models and must be implicitly expressed through effective parameters. In order to inform experimental decisions, a helpful parameter estimation method must establish that one can correctly interpret the provided parameters. However, only a few works have been undertaken to reach that objective in the context of BHJ organic solar cells. In this work, a realistic two-dimensional model of BHJ solar cells is used to investigate the behavior of state-of-the-art parameter estimation procedures in situations that emulate experimental conditions. We demonstrate that fitting solely current-voltage characteristics by an effective medium one-dimensional model can yield nonsensical results, which may lead to counter-productive decisions about future design choices. In agreement with previously published literature, we explicitly demonstrate that fitting several characterization results together can drastically improve the robustness of the parameter estimation. Based on a detailed analysis of parameter estimation results, a set of recommendations is formulated to avoid the most problematic pitfalls and increase awareness about the limitations that cannot be circumvented.


title: Resonant and near-resonant internal wave triads for non-uniform stratifications. Part 2: Vertically bounded domain with mild-slope bathymetry

abstract: Weakly nonlinear internal wave-wave interaction is a key mechanism that cascades energy from large to small scales, leading to ocean turbulence and mixing. Oceans typically have a non-uniform density stratification profile; moreover, submarine topography leads to a spatially varying ocean depth ($h$). Under these conditions and assuming mild-slope bathymetry, we employ multiple-scale analysis to derive the wave amplitude equations for triadic- and self-interactions. The waves are assumed to have a slowly (rapidly) varying amplitude (phase) in space and time. For uniform stratifications, the horizontal wavenumber ($k$) condition for waves ($1$,$2$,$3$), given by ${k}_{(1,a)}+{k}_{(2,b)}+{k}_{(3,c)}=0$, is unaffected as $h$ is varied, where $(a,b,c)$ denote the modenumber. Moreover, the nonlinear coupling coefficients (NLC) are proportional to $1/h^2$, implying that triadic waves grow faster while travelling up a seamount. For non-uniform stratifications, triads that do not satisfy the condition $a=b=c$ may not satisfy the horizontal wavenumber condition as $h$ is varied, and unlike uniform stratification, the NLC may not decrease (increase) monotonically with increasing (decreasing) $h$. NLC, and hence wave growth rates for both triads and self-interactions, can also vary rapidly with $h$. The most unstable daughter wave combination of a triad with a mode-1 parent wave can also change for relatively small changes in $h$. We also investigate higher-order self-interactions in the presence of a monochromatic, small amplitude bathymetry; here the bathymetry behaves as a zero frequency wave. We derive the amplitude evolution equations and show that higher-order self-interactions might be a viable mechanism of energy cascade.


title: Every finite graph arises as the singular set of a compact $3$-d calibrated area minimizing surface

abstract: Given any (not necessarily connected) combinatorial finite graph and any compact smooth $6$-manifold $M^6$ with the third Betti number $b_3\not=0$, we construct a calibrated 3-dimensional homologically area minimizing surface on $M$ equipped in a smooth metric $g$, so that the singular set of the surface is precisely an embedding of this finite graph. Moreover, the calibration form near the singular set is a smoothly $GL(6,\mathbb{R})$ twisted special Lagrangian form. The constructions are based on some unpublished ideas of Professor Camillo De Lellis and Professor Robert Bryant.


title: Weighted Hardy Type inequalities with Robin Boundary Conditions

abstract: In this paper, we establish a general weighted Hardy type inequality for the $% p-$Laplace operator with Robin boundary condition. We provide various concrete examples to illustrate our results for different weights. Furthermore, we present some Heisenberg-Pauli-Weyl type inequalities with boundary terms on balls with radius $R$ at the origin in $\mathbb{R}^n$.


title: Thermal properties of nanocrystalline silicon nanobeams

abstract: Controlling thermal energy transfer at the nanoscale has become critically important in many applications and thermal properties since it often limits device performance. In this work, we study the effects on thermal conductivity arising from the nanoscale structure of free-standing nanocrystalline silicon films and the increasing surface-to-volume ratio when fabricated into suspended optomechanical nanobeams. We characterize thermal transport in structures with different grain sizes and elucidate the relative impact of grain size and geometrical dimensions on thermal conductivity. We use a micro-time-domain thermoreflectance method to study the impact of the grain size distribution, from 10 to 400 nm, on the thermal conductivity in free-standing nanocrystalline silicon films considering surface phonon and grain boundary scattering. We find a drastic reduction in the thermal conductivity, down to values of 10 W.m^{-1}.K^{-1} and below, which is just a fraction of the conductivity of single crystalline silicon. Decreasing the grain size further decreases the thermal conductivity. We also observe that this effect is smaller in OM nanostructures than in membranes due to the competition of surface scattering in decreasing thermal conductivity. Finally, we introduce a novel versatile contactless characterization technique that can be adapted to any structure supporting a thermally shifted optical resonance and use it to evaluate the thermal conductivity. This method can be used with optical resonances exhibiting different mode profiles and the data is shown to agrees quantitatively with the thermoreflectance measurements. This work opens the way to a more generalized thermal characterization of optomechanical cavities and to create hot-spots with engineered shapes at desired position in the structures as a means to study thermal transport in coupled photon-phonon structures.

categories: physics.app-ph cond-mat.mes-hall cond-mat.mtrl-sci physics.optics


title: Closed Ziv-Lempel factorization of the $m$-bonacci words

abstract: A word $w$ is said to be closed if it has a proper factor $x$ which occurs exactly twice in $w$, as a prefix and as a suffix of $w$. Based on the concept of Ziv-Lempel factorization, we define the closed $z$-factorization of finite and infinite words. Then we find the closed $z$-factorization of the infinite $m$-bonacci words for all $m \geq 2$. We also classify closed prefixes of the infinite $m$-bonacci words.


title: Propagation of Axial and Polar Gravitational Waves in Kantowski-Sachs Universe

abstract: We apply the Regge-Wheeler formalism to study axial and polar gravitational waves in Kantowski-Sachs universe. The background field equations and the linearised perturbation equations for the modes are derived in presence of matter. To find analytical solutions, we analyze the propagation of waves in vacuum spacetime. The background field equations in absence of matter are solved by assuming the expansion scalar to be proportional to the shear scalar. Using the method of separation of variables, the axial perturbation parameter $ h_0(t, r) $ is obtained from its wave equation. The other perturbation $ h_1(t, r) $ is then determined from $ h_0(t, r) $. The anisotropy of the background spacetime is responsible for the damping of the axial waves. The polar perturbation equations are much more involved compared to their FLRW counterparts, as well as to the axial perturbations in Kantowski-Sachs background. In both the axial and polar cases, the radial and temporal solutions separate out as products. The temporal part of the polar perturbation solutions are plotted against time to obtain an order of magnitude estimate of the frequency of the propagating GWs and lies in the range 1000-2000 Hz. Using standard observational data for the GW strain we have placed constraints on the parameters in the polar perturbation solutions. The perturbation equations in presence of matter show that the axial waves can cause perturbations only in the azimuthal velocity of the fluid without deforming the matter field. But the polar waves must perturb the energy density, the pressure and also the non-azimuthal components of the fluid velocity.


title: Quantum Information and the Mind-Body Problem

abstract: The mind-body problem is reviewed in the context of a non-technical account of quantum information theory. The importance of clearly defining: `what is physical?' is highlighted, since only then can we give meaning to the concept `non-physical'. Physicality is defined in terms of interaction, which is in turn defined to be a correlated exchange of information. This is asserted to be the basis of any meaningful concept of epistemology. Hence, it is argued that a non-physical entity can not `know' anything about the world. Information transfer is then discussed in terms of quantum entanglement and an argument for our perception of time is presented. It is then contended that the notion of `mind' may be meaningfully discussed in the context of a quantum theoretic framework.


title: A response to the Mucino-Okon-Sudarsky's Assessment of Relational Quantum Mechanics

abstract: A recent paper by Mucino, Okon and Sudarsky attempts an assessment of the Relational Interpretation of quantum mechanics. The paper presupposes assumptions that are precisely those questioned in the Relational Interpretation, thus undermining the value of the assessment.


title: Matching N3LO QCD calculations to parton showers

abstract: The search for new interactions and particles in high-energy collider physics relies on precise background predictions. This has led to many advances in combining precise fixed-order cross-section calculations with detailed event generator simulations. In recent years, fixed-order QCD calculations of inclusive cross sections at N3LO precision have emerged, followed by an impressive progress at producing differential results. Once differential results become publicly available, it would be prudent to embed these into event generators to allow the community to leverage these advances. This note offers some concrete thoughts on ME+PS matching at third order in QCD. As a method for testing these thoughts, a toy calculation of $e^+e^-\rightarrow u \bar u$ at $\mathcal{O}(\alpha_s^3)$ is constructed, and combined with an event generator through unitary matching. The toy implementation may serve also as blueprint for high-precision QCD predictions at future lepton colliders. As a byproduct of the N3LO matching formula, a new NNLO+PS formula for processes with "additional" jets is obtained.


title: Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coverage

abstract: This paper studies offline Imitation Learning (IL) where an agent learns to imitate an expert demonstrator without additional online environment interactions. Instead, the learner is presented with a static offline dataset of state-action-next state transition triples from a potentially less proficient behavior policy. We introduce Model-based IL from Offline data (MILO): an algorithmic framework that utilizes the static dataset to solve the offline IL problem efficiently both in theory and in practice. In theory, even if the behavior policy is highly sub-optimal compared to the expert, we show that as long as the data from the behavior policy provides sufficient coverage on the expert state-action traces (and with no necessity for a global coverage over the entire state-action space), MILO can provably combat the covariate shift issue in IL. Complementing our theory results, we also demonstrate that a practical implementation of our approach mitigates covariate shift on benchmark MuJoCo continuous control tasks. We demonstrate that with behavior policies whose performances are less than half of that of the expert, MILO still successfully imitates with an extremely low number of expert state-action pairs while traditional offline IL method such as behavior cloning (BC) fails completely. Source code is provided at https://github.com/jdchang1/milo.


title: Deep Learning-based Type Identification of Volumetric MRI Sequences

abstract: The analysis of Magnetic Resonance Imaging (MRI) sequences enables clinical professionals to monitor the progression of a brain tumor. As the interest for automatizing brain volume MRI analysis increases, it becomes convenient to have each sequence well identified. However, the unstandardized naming of MRI sequences makes their identification difficult for automated systems, as well as makes it difficult for researches to generate or use datasets for machine learning research. In the face of that, we propose a system for identifying types of brain MRI sequences based on deep learning. By training a Convolutional Neural Network (CNN) based on 18-layer ResNet architecture, our system can classify a volumetric brain MRI as a FLAIR, T1, T1c or T2 sequence, or whether it does not belong to any of these classes. The network was evaluated on publicly available datasets comprising both, pre-processed (BraTS dataset) and non-pre-processed (TCGA-GBM dataset), image types with diverse acquisition protocols, requiring only a few slices of the volume for training. Our system can classify among sequence types with an accuracy of 96.81%.


title: Adversarial Classification of the Attacks on Smart Grids Using Game Theory and Deep Learning

abstract: Smart grids are vulnerable to cyber-attacks. This paper proposes a game-theoretic approach to evaluate the variations caused by an attacker on the power measurements. Adversaries can gain financial benefits through the manipulation of the meters of smart grids. On the other hand, there is a defender that tries to maintain the accuracy of the meters. A zero-sum game is used to model the interactions between the attacker and defender. In this paper, two different defenders are used and the effectiveness of each defender in different scenarios is evaluated. Multi-layer perceptrons (MLPs) and traditional state estimators are the two defenders that are studied in this paper. The utility of the defender is also investigated in adversary-aware and adversary-unaware situations. Our simulations suggest that the utility which is gained by the adversary drops significantly when the MLP is used as the defender. It will be shown that the utility of the defender is variant in different scenarios, based on the defender that is being used. In the end, we will show that this zero-sum game does not yield a pure strategy, and the mixed strategy of the game is calculated.


title: Alpha Matte Generation from Single Input for Portrait Matting

abstract: In the portrait matting, the goal is to predict an alpha matte that identifies the effect of each pixel on the foreground subject. Traditional approaches and most of the existing works utilized an additional input, e.g., trimap, background image, to predict alpha matte. However, (1) providing additional input is not always practical, and (2) models are too sensitive to these additional inputs. To address these points, in this paper, we introduce an additional input-free approach to perform portrait matting. We divide the task into two subtasks, segmentation and alpha matte prediction. We first generate a coarse segmentation map from the input image and then predict the alpha matte by utilizing the image and segmentation map. Besides, we present a segmentation encoding block to downsample the coarse segmentation map and provide useful feature representation to the residual block, since using a single encoder causes the vanishing of the segmentation information. We tested our model on four different benchmark datasets. The proposed method outperformed the MODNet and MGMatting methods that also take a single input. Besides, we obtained comparable results with BGM-V2 and FBA methods that require additional input.


title: Distributed Learning and its Application for Time-Series Prediction

abstract: Extreme events are occurrences whose magnitude and potential cause extensive damage on people, infrastructure, and the environment. Motivated by the extreme nature of the current global health landscape, which is plagued by the coronavirus pandemic, we seek to better understand and model extreme events. Modeling extreme events is common in practice and plays an important role in time-series prediction applications. Our goal is to (i) compare and investigate the effect of some common extreme events modeling methods to explore which method can be practical in reality and (ii) accelerate the deep learning training process, which commonly uses deep recurrent neural network (RNN), by implementing the asynchronous local Stochastic Gradient Descent (SGD) framework among multiple compute nodes. In order to verify our distributed extreme events modeling, we evaluate our proposed framework on a stock data set S\&P500, with a standard recurrent neural network. Our intuition is to explore the (best) extreme events modeling method which could work well under the distributed deep learning setting. Moreover, by using asynchronous distributed learning, we aim to significantly reduce the communication cost among the compute nodes and central server, which is the main bottleneck of almost all distributed learning frameworks. We implement our proposed work and evaluate its performance on representative data sets, such as S&P500 stock in $5$-year period. The experimental results validate the correctness of the design principle and show a significant training duration reduction upto $8$x, compared to the baseline single compute node. Our results also show that our proposed work can achieve the same level of test accuracy, compared to the baseline setting.


title: Towards an Understanding of Benign Overfitting in Neural Networks

abstract: Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss; yet surprisingly, they possess near-optimal prediction performance, contradicting classical learning theory. We examine how these benign overfitting phenomena occur in a two-layer neural network setting where sample covariates are corrupted with noise. We address the high dimensional regime, where the data dimension $d$ grows with the number $n$ of data points. Our analysis combines an upper bound on the bias with matching upper and lower bounds on the variance of the interpolator (an estimator that interpolates the data). These results indicate that the excess learning risk of the interpolator decays under mild conditions. We further show that it is possible for the two-layer ReLU network interpolator to achieve a near minimax-optimal learning rate, which to our knowledge is the first generalization result for such networks. Finally, our theory predicts that the excess learning risk starts to increase once the number of parameters $s$ grows beyond $O(n^2)$, matching recent empirical findings.


title: On Local Aggregation in Heterophilic Graphs

abstract: Many recent works have studied the performance of Graph Neural Networks (GNNs) in the context of graph homophily - a label-dependent measure of connectivity. Traditional GNNs generate node embeddings by aggregating information from a node's neighbors in the graph. Recent results in node classification tasks show that this local aggregation approach performs poorly in graphs with low homophily (heterophilic graphs). Several mechanisms have been proposed to improve the accuracy of GNNs on such graphs by increasing the aggregation range of a GNN layer, either through multi-hop aggregation, or through long-range aggregation from distant nodes. In this paper, we show that properly tuned classical GNNs and multi-layer perceptrons match or exceed the accuracy of recent long-range aggregation methods on heterophilic graphs. Thus, our results highlight the need for alternative datasets to benchmark long-range GNN aggregation mechanisms. We also show that homophily is a poor measure of the information in a node's local neighborhood and propose the Neighborhood Information Content(NIC) metric, which is a novel information-theoretic graph metric. We argue that NIC is more relevant for local aggregation methods as used by GNNs. We show that, empirically, it correlates better with GNN accuracy in node classification tasks than homophily.


title: Lower Bounds on Stabilizer Rank

abstract: The stabilizer rank of a quantum state $\psi$ is the minimal $r$ such that $\left| \psi \right \rangle = \sum_{j=1}^r c_j \left|\varphi_j \right\rangle$ for $c_j \in \mathbb{C}$ and stabilizer states $\varphi_j$. The running time of several classical simulation methods for quantum circuits is determined by the stabilizer rank of the $n$-th tensor power of single-qubit magic states. We prove a lower bound of $\Omega(n)$ on the stabilizer rank of such states, improving a previous lower bound of $\Omega(\sqrt{n})$ of Bravyi, Smith and Smolin (arXiv:1506.01396). Further, we prove that for a sufficiently small constant $\delta$, the stabilizer rank of any state which is $\delta$-close to those states is $\Omega(\sqrt{n}/\log n)$. This is the first non-trivial lower bound for approximate stabilizer rank. Our techniques rely on the representation of stabilizer states as quadratic functions over affine subspaces of $\mathbb{F}_2^n$, and we use tools from analysis of boolean functions and complexity theory. The proof of the first result involves a careful analysis of directional derivatives of quadratic polynomials, whereas the proof of the second result uses Razborov-Smolensky low degree polynomial approximations and correlation bounds against the majority function.


title: PreferenceNet: Encoding Human Preferences in Auction Design with Deep Learning

abstract: The design of optimal auctions is a problem of interest in economics, game theory and computer science. Despite decades of effort, strategyproof, revenue-maximizing auction designs are still not known outside of restricted settings. However, recent methods using deep learning have shown some success in approximating optimal auctions, recovering several known solutions and outperforming strong baselines when optimal auctions are not known. In addition to maximizing revenue, auction mechanisms may also seek to encourage socially desirable constraints such as allocation fairness or diversity. However, these philosophical notions neither have standardization nor do they have widely accepted formal definitions. In this paper, we propose PreferenceNet, an extension of existing neural-network-based auction mechanisms to encode constraints using (potentially human-provided) exemplars of desirable allocations. In addition, we introduce a new metric to evaluate an auction allocations' adherence to such socially desirable constraints and demonstrate that our proposed method is competitive with current state-of-the-art neural-network based auction designs. We validate our approach through human subject research and show that we are able to effectively capture real human preferences. Our code is available at https://github.com/neeharperi/PreferenceNet


title: On Memorization in Probabilistic Deep Generative Models

abstract: Recent advances in deep generative models have led to impressive results in a variety of application domains. Motivated by the possibility that deep learning models might memorize part of the input data, there have been increased efforts to understand how memorization arises. In this work, we extend a recently proposed measure of memorization for supervised learning (Feldman, 2019) to the unsupervised density estimation problem and adapt it to be more computationally efficient. Next, we present a study that demonstrates how memorization can occur in probabilistic deep generative models such as variational autoencoders. This reveals that the form of memorization to which these models are susceptible differs fundamentally from mode collapse and overfitting. Furthermore, we show that the proposed memorization score measures a phenomenon that is not captured by commonly-used nearest neighbor tests. Finally, we discuss several strategies that can be used to limit memorization in practice. Our work thus provides a framework for understanding problematic memorization in probabilistic generative models.


title: Spatially homogeneous black hole solutions in $z=4$ Ho\v{r}ava-Lifshitz gravity in $(4+1)$ dimensions with Nil geometry and $H^2\times R$ horizons

abstract: In this paper, we present two new families of spatially homogeneous black hole solution for $z=4$ Ho\v{r}ava-Lifshitz Gravity equations in $(4+1)$ dimensions with general coupling constant $\lambda$ and the especial case $\lambda=1$, considering $\beta=-1/3$. The three-dimensional horizons are considered to have Bianchi types $II$ and $III$ symmetries, and hence the horizons are modeled on two types of Thurston $3$-geometries, namely the Nil geometry and $H^2\times R$. Being foliated by compact 3-manifolds, the horizons are neither spherical, hyperbolic, nor toroidal, and therefore are not of the previously studied topological black hole solutions in Ho\v{r}ava-Lifshitz gravity. Using the Hamiltonian formalism, we establish the conventional thermodynamics of the solutions defining the mass and entropy of the black hole solutions for several classes of solutions. It turned out that for both horizon geometries the area term in the entropy receives two non-logarithmic negative corrections proportional to Ho\v{r}ava-Lifshitz parameters. Also, we show that choosing some proper set of parameters the solutions can exhibit locally stable or unstable behavior.


title: Hypothesis Testing for Hierarchical Structures in Cognitive Diagnosis Models

abstract: Cognitive Diagnosis Models (CDMs) are a special family of discrete latent variable models widely used in educational, psychological and social sciences. In many applications of CDMs, certain hierarchical structures among the latent attributes are assumed by researchers to characterize their dependence structure. Specifically, a directed acyclic graph is used to specify hierarchical constraints on the allowable configurations of the discrete latent attributes. In this paper, we consider the important yet unaddressed problem of testing the existence of latent hierarchical structures in CDMs. We first introduce the concept of testability of hierarchical structures in CDMs and present sufficient conditions. Then we study the asymptotic behaviors of the likelihood ratio test (LRT) statistic, which is widely used for testing nested models. Due to the irregularity of the problem, the asymptotic distribution of LRT becomes nonstandard and tends to provide unsatisfactory finite sample performance under practical conditions. We provide statistical insights on such failures, and propose to use parametric bootstrap to perform the testing. We also demonstrate the effectiveness and superiority of parametric bootstrap for testing the latent hierarchies over non-parametric bootstrap and the na\"ive Chi-squared test through comprehensive simulations and an educational assessment dataset.


title: Experience Report: Writing A Portable GPU Runtime with OpenMP 5.1

abstract: GPU runtimes are historically implemented in CUDA or other vendor specific languages dedicated to GPU programming. In this work we show that OpenMP 5.1, with minor compiler extensions, is capable of replacing existing solutions without a performance penalty. The result is a performant and portable GPU runtime that can be compiled with LLVM/Clang to Nvidia and AMD GPUs without the need for CUDA or HIP during its development and compilation. While we tried to be OpenMP compliant, we identified the need for compiler extensions to achieve the CUDA performance with our OpenMP runtime. We hope that future versions of OpenMP adopt our extensions to make device programming in OpenMP also portable across compilers, not only across execution platforms. The library we ported to OpenMP is the OpenMP device runtime that provides OpenMP functionality on the GPU. This work opens the door for shipping OpenMP offloading with a Linux distribution's LLVM package as the package manager would not need a vendor SDK to build the compiler and runtimes. Furthermore, our OpenMP device runtime can support a new GPU target through the use of a few compiler intrinsics rather than requiring a reimplementation of the entire runtime.


title: PYROBOCOP : Python-based Robotic Control & Optimization Package for Manipulation and Collision Avoidance

abstract: PYROBOCOP is a lightweight Python-based package for control and optimization of robotic systems described by nonlinear Differential Algebraic Equations (DAEs). In particular, the package can handle systems with contacts that are described by complementarity constraints and provides a general framework for specifying obstacle avoidance constraints. The package performs direct transcription of the DAEs into a set of nonlinear equations by performing orthogonal collocation on finite elements. The resulting optimization problem belongs to the class of Mathematical Programs with Complementarity Constraints (MPCCs). MPCCs fail to satisfy commonly assumed constraint qualifications and require special handling of the complementarity constraints in order for NonLinear Program (NLP) solvers to solve them effectively. PYROBOCOP provides automatic reformulation of the complementarity constraints that enables NLP solvers to perform optimization of robotic systems. The package is interfaced with ADOLC for obtaining sparse derivatives by automatic differentiation and IPOPT for performing optimization. We demonstrate the effectiveness of our approach in terms of speed and flexibility. We provide several numerical examples for several robotic systems with collision avoidance as well as contact constraints represented using complementarity constraints. We provide comparisons with other open source optimization packages like CasADi and Pyomo .


title: PAC Best Arm Identification Under a Deadline

abstract: We study $(\epsilon, \delta)$-PAC best arm identification, where a decision-maker must identify an $\epsilon$-optimal arm with probability at least $1 - \delta$, while minimizing the number of arm pulls (samples). Most of the work on this topic is in the sequential setting, where there is no constraint on the time taken to identify such an arm; this allows the decision-maker to pull one arm at a time. In this work, the decision-maker is given a deadline of $T$ rounds, where, on each round, it can adaptively choose which arms to pull and how many times to pull them; this distinguishes the number of decisions made (i.e., time or number of rounds) from the number of samples acquired (cost). Such situations occur in clinical trials, where one may need to identify a promising treatment under a deadline while minimizing the number of test subjects, or in simulation-based studies run on the cloud, where we can elastically scale up or down the number of virtual machines to conduct as many experiments as we wish, but need to pay for the resource-time used. As the decision-maker can only make $T$ decisions, she may need to pull some arms excessively relative to a sequential algorithm in order to perform well on all possible problems. We formalize this added difficulty with two hardness results that indicate that unlike sequential settings, the ability to adapt to the problem difficulty is constrained by the finite deadline. We propose Elastic Batch Racing (EBR), a novel algorithm for this setting and bound its sample complexity, showing that EBR is optimal with respect to both hardness results. We present simulations evaluating EBR in this setting, where it outperforms baselines by several orders of magnitude.


title: Conditionally positive definite unilateral weighted shifts

abstract: In a recent paper [15], Hilbert space operators $T$ with the property that each sequence of the form $\{\|T^n h\|^2\}_{n=0}^{\infty}$ is conditionally positive definite in a semigroup sense were introduced. In the present paper, this line of research is continued in depth in the case of unilateral weighted shifts. The conditional positive definiteness of weighted shifts is characterized in terms of formal moment sequences. The description of the representing triplet, the main object canonically associated with such operators, is provided. The backward extension problem for conditionally positive definite weighted shifts is solved, revealing a new feature that does not appear in the case of other operator classes. Finally, the flatness problem in this context is discussed, with an emphases on unexpected differences from the corresponding problem for subnormal weighted shifts.


title: Meta-learning with implicit gradients in a few-shot setting for medical image segmentation

abstract: Widely used traditional supervised deep learning methods require a large number of training samples but often fail to generalize on unseen datasets. Therefore, a more general application of any trained model is quite limited for medical imaging for clinical practice. Using separately trained models for each unique lesion category or a unique patient population will require sufficiently large curated datasets, which is not practical to use in a real-world clinical set-up. Few-shot learning approaches can not only minimize the need for an enormous number of reliable ground truth labels that are labour-intensive and expensive but can also be used to model on a dataset coming from a new population. To this end, we propose to exploit an optimization-based implicit model agnostic meta-learning (iMAML) algorithm under few-shot settings for medical image segmentation. Our approach can leverage the learned weights from diverse but small training samples to perform analysis on unseen datasets with high accuracy. We show that, unlike classical few-shot learning approaches, our method improves generalization capability. To our knowledge, this is the first work that exploits iMAML for medical image segmentation and explores the strength of the model on scenarios such as meta-training on unique and mixed instances of lesion datasets. Our quantitative results on publicly available skin and polyp datasets show that the proposed method outperforms the naive supervised baseline model and two recent few-shot segmentation approaches by large margins. In addition, our iMAML approach shows an improvement of 2%-4% in dice score compared to its counterpart MAML for most experiments.


title: Hall-Higman type theorems for exceptional groups of Lie type, I

abstract: The paper studies the minimum polynomial degrees of $p$-elements in cross-characteristic representations of simple groups of exceptional Lie type whose BN-pair rank is at most 2. Specifically, we prove that the degree in question equals the order of the element.


title: Efficient Lottery Ticket Finding: Less Data is More

abstract: The lottery ticket hypothesis (LTH) reveals the existence of winning tickets (sparse but critical subnetworks) for dense networks, that can be trained in isolation from random initialization to match the latter's accuracies. However, finding winning tickets requires burdensome computations in the train-prune-retrain process, especially on large-scale datasets (e.g., ImageNet), restricting their practical benefits. This paper explores a new perspective on finding lottery tickets more efficiently, by doing so only with a specially selected subset of data, called Pruning-Aware Critical set (PrAC set), rather than using the full training set. The concept of PrAC set was inspired by the recent observation, that deep networks have samples that are either hard to memorize during training, or easy to forget during pruning. A PrAC set is thus hypothesized to capture those most challenging and informative examples for the dense model. We observe that a high-quality winning ticket can be found with training and pruning the dense network on the very compact PrAC set, which can substantially save training iterations for the ticket finding process. Extensive experiments validate our proposal across diverse datasets and network architectures. Specifically, on CIFAR-10, CIFAR-100, and Tiny ImageNet, we locate effective PrAC sets at 35.32%~78.19% of their training set sizes. On top of them, we can obtain the same competitive winning tickets for the corresponding dense networks, yet saving up to 82.85%~92.77%, 63.54%~74.92%, and 76.14%~86.56% training iterations, respectively. Crucially, we show that a PrAC set found is reusable across different network architectures, which can amortize the extra cost of finding PrAC sets, yielding a practical regime for efficient lottery ticket finding.


title: Minimum cross-entropy distributions on Wasserstein balls and their applications

abstract: Given a prior probability density $p$ on a compact set $K$ we characterize the probability distribution $q_{\delta}^*$ on $K$ contained in a Wasserstein ball $B_{\delta}(\mu)$ centered in a given discrete measure $\mu$ for which the relative-entropy $H(q,p)$ achieves its minimum. This characterization gives us an algorithm for computing such distributions efficiently


title: Neural Tangent Kernel Maximum Mean Discrepancy

abstract: We present a novel neural network Maximum Mean Discrepancy (MMD) statistic by identifying a new connection between neural tangent kernel (NTK) and MMD. This connection enables us to develop a computationally efficient and memory-efficient approach to compute the MMD statistic and perform NTK based two-sample tests towards addressing the long-standing challenge of memory and computational complexity of the MMD statistic, which is essential for online implementation to assimilating new samples. Theoretically, such a connection allows us to understand the NTK test statistic properties, such as the Type-I error and testing power for performing the two-sample test, by adapting existing theories for kernel MMD. Numerical experiments on synthetic and real-world datasets validate the theory and demonstrate the effectiveness of the proposed NTK-MMD statistic.


title: Distributional Reinforcement Learning with Unconstrained Monotonic Neural Networks

abstract: The distributional reinforcement learning (RL) approach advocates for representing the complete probability distribution of the random return instead of only modelling its expectation. A distributional RL algorithm may be characterised by two main components, namely the representation of the distribution together with its parameterisation and the probability metric defining the loss. The present research work considers the unconstrained monotonic neural network (UMNN) architecture, a universal approximator of continuous monotonic functions which is particularly well suited for modelling different representations of a distribution. This property enables the efficient decoupling of the effect of the function approximator class from that of the probability metric. The research paper firstly introduces a methodology for learning different representations of the random return distribution (PDF, CDF and QF). Secondly, a novel distributional RL algorithm named unconstrained monotonic deep Q-network (UMDQN) is presented. To the authors' knowledge, it is the first distributional RL method supporting the learning of three, valid and continuous representations of the random return distribution. Lastly, in light of this new algorithm, an empirical comparison is performed between three probability quasi-metrics, namely the Kullback-Leibler divergence, Cramer distance, and Wasserstein distance. The results highlight the main strengths and weaknesses associated with each probability metric together with an important limitation of the Wasserstein distance.


title: Topological Phase Transition in a Magnetic Weyl Semimetal

abstract: Topological Weyl semimetals (TWSs) are exotic crystals possessing emergent relativistic Weyl fermions connected by unique surface Fermi-arcs (SFAs) in their electronic structures. To realize the TWS state, certain symmetry (such as the inversion or time reversal symmetry) must be broken, leading to a topological phase transition (TPT). Despite the great importance in understanding the formation of TWSs and their unusual properties, direct observation of such a TPT has been challenging. Here, using a recently discovered magnetic TWS Co3Sn2S2, we were able to systematically study its TPT with detailed temperature dependence of the electronic structures by angle-resolved photoemission spectroscopy. The TPT with drastic band structures evolution was clearly observed across the Curie temperature (TC = 177 K), including the disappearance of the characteristic SFAs and the recombination of the spin-split bands that leads to the annihilation of Weyl points with opposite chirality. These results not only reveal important insights on the interplay between the magnetism and band topology in TWSs, but also provide a new method to control their exotic physical properties.


title: Fano-type effect in hydrogen-terminated pure nanodiamond

abstract: Two novel properties, unique for semiconductors: a negative electron affinity [1-2], and a high p-type surface electrical conductivity [3-4], were discovered in diamond at the end of the last century. Both properties appear when the diamond surface is hydrogenated. A natural question arises: is the influence of the surface hydrogen on diamond limited only to the electrical properties? Here, we report the first observation of a transparency peak at 1328 cm-1 in IR absorption of hydrogen-terminated pure (undoped) nanodiamonds. This new optical property is ascribed to Fano-type destructive interference between zone-center phonons and free carriers (holes) appearing in the near-surface layer of hydrogenated nanodiamond. Our work opens the way to exploring the physics of electron-phonon coupling in undoped diamonds and promises the application of the H-terminated nanodiamonds as a new optical material with an induced transparency in IR optical range.


title: Examples of surfaces with canonical map of degree 4

abstract: We give two examples of surfaces with canonical map of degree 4 onto a canonical surface.


title: A Targeted Assessment of Incremental Processing in Neural LanguageModels and Humans

abstract: We present a targeted, scaled-up comparison of incremental processing in humans and neural language models by collecting by-word reaction time data for sixteen different syntactic test suites across a range of structural phenomena. Human reaction time data comes from a novel online experimental paradigm called the Interpolated Maze task. We compare human reaction times to by-word probabilities for four contemporary language models, with different architectures and trained on a range of data set sizes. We find that across many phenomena, both humans and language models show increased processing difficulty in ungrammatical sentence regions with human and model `accuracy' scores (a la Marvin and Linzen(2018)) about equal. However, although language model outputs match humans in direction, we show that models systematically under-predict the difference in magnitude of incremental processing difficulty between grammatical and ungrammatical sentences. Specifically, when models encounter syntactic violations they fail to accurately predict the longer reaction times observed in the human data. These results call into question whether contemporary language models are approaching human-like performance for sensitivity to syntactic violations.


title: A Pre-training Oracle for Predicting Distances in Social Networks

abstract: In this paper, we propose a novel method to make distance predictions in real-world social networks. As predicting missing distances is a difficult problem, we take a two-stage approach. Structural parameters for families of synthetic networks are first estimated from a small set of measurements of a real-world network and these synthetic networks are then used to pre-train the predictive neural networks. Since our model first searches for the most suitable synthetic graph parameters which can be used as an "oracle" to create arbitrarily large training data sets, we call our approach "Oracle Search Pre-training" (OSP). For example, many real-world networks exhibit a Power law structure in their node degree distribution, so a Power law model can provide a foundation for the desired oracle to generate synthetic pre-training networks, if the appropriate Power law graph parameters can be estimated. Accordingly, we conduct experiments on real-world Facebook, Email, and Train Bombing networks and show that OSP outperforms models without pre-training, models pre-trained with inaccurate parameters, and other distance prediction schemes such as Low-rank Matrix Completion. In particular, we achieve a prediction error of less than one hop with only 1% of sampled distances from the social network. OSP can be easily extended to other domains such as random networks by choosing an appropriate model to generate synthetic training data, and therefore promises to impact many different network learning problems.


title: A call for better unit testing for invariant risk minimisation

abstract: In this paper we present a controlled study on the linearized IRM framework (IRMv1) introduced in Arjovsky et al. (2020). We show that IRMv1 (and its variants) framework can be potentially unstable under small changes to the optimal regressor. This can, notably, lead to worse generalisation to new environments, even compared with ERM which converges simply to the global minimum for all training environments mixed up all together. We also highlight the isseus of scaling in the the IRMv1 setup. These observations highlight the importance of rigorous evaluation and importance of unit-testing for measuring progress towards IRM.


title: On the Optimality of Backward Regression: Sparse Recovery and Subset Selection

abstract: Sparse recovery and subset selection are fundamental problems in varied communities, including signal processing, statistics and machine learning. Herein, we focus on an important greedy algorithm for these problems: Backward Stepwise Regression. We present novel guarantees for the algorithm, propose an efficient, numerically stable implementation, and put forth Stepwise Regression with Replacement (SRR), a new family of two-stage algorithms that employs both forward and backward steps for compressed sensing problems. Prior work on the backward algorithm has proven its optimality for the subset selection problem, provided the residual associated with the optimal solution is small enough. However, the existing bounds on the residual magnitude are NP-hard to compute. In contrast, our main theoretical result includes a bound that can be computed in polynomial time, depends chiefly on the smallest singular value of the matrix, and also extends to the method of magnitude pruning. In addition, we report numerical experiments highlighting crucial differences between forward and backward greedy algorithms and compare SRR against popular two-stage algorithms for compressed sensing. Remarkably, SRR algorithms generally maintain good sparse recovery performance on coherent dictionaries. Further, a particular SRR algorithm has an edge over Subspace Pursuit.

categories: math.OC cs.IT cs.NA math.IT math.NA stat.CO


title: Graph2Graph Learning with Conditional Autoregressive Models

abstract: We present a graph neural network model for solving graph-to-graph learning problems. Most deep learning on graphs considers ``simple'' problems such as graph classification or regressing real-valued graph properties. For such tasks, the main requirement for intermediate representations of the data is to maintain the structure needed for output, i.e., keeping classes separated or maintaining the order indicated by the regressor. However, a number of learning tasks, such as regressing graph-valued output, generative models, or graph autoencoders, aim to predict a graph-structured output. In order to successfully do this, the learned representations need to preserve far more structure. We present a conditional auto-regressive model for graph-to-graph learning and illustrate its representational capabilities via experiments on challenging subgraph predictions from graph algorithmics; as a graph autoencoder for reconstruction and visualization; and on pretraining representations that allow graph classification with limited labeled data.


title: Spins of primordial black holes formed in different cosmological scenarios

abstract: Primordial black holes (PBHs) could account for all or part of dark matter, as well as for some LIGO events. We discuss the spins of primordial black holes produced in different cosmological scenarios, with the emphasis on recently discovered possibilities. PBHs produced as a horizon-size collapse of density perturbations are known to have very small spins. In contrast, PBHs resulting from assembly of matterlike objects (particles, Q-balls, oscillons, etc.) can have large or small spins depending on their formation history and the efficiency of radiative cooling. We show that scalar radiation can remove the angular momentum very efficiently, leading to slowly rotating PBHs in those scenarios for which the radiative cooling is important. Gravitational waves astronomy offers an opportunity to determine the spins of black holes, opening a new window on the early Universe if, indeed, some black holes have primordial origin.


title: Mean Field Approximation for solving QUBO problems

abstract: The Quadratic Unconstrained Binary Optimization (QUBO) problems are NP hard; thus, so far, there are no algorithms to solve them efficiently. There are exact methods like the Branch-and-Bound algorithm for smaller problems, and for larger ones, many good approximations like stochastic simulated annealing for discrete variables or the mean field annealing for continuous variables. This paper will show that the statistical physics approach and the quantum mechanical approach in the mean field annealing give the same result. We examined the Ising problem, which is an alternative formulation of the QUBO problem. Our methods consist of a set of simple gradient-based minimizations with continuous variables, thus easy to simulate. We benchmarked our methods with solving the Maximum Cut problem with the G-sets. In many graphs, we could achieve the best-known Cut Value.


title: Constrained Ellipse Fitting for Efficient Parameter Mapping with Phase-cycled bSSFP MRI

abstract: Balanced steady-state free precession (bSSFP) imaging enables high scan efficiency in MRI, but differs from conventional sequences in terms of elevated sensitivity to main field inhomogeneity and nonstandard T2/T1-weighted tissue contrast. To address these limitations, multiple bSSFP images of the same anatomy are commonly acquired with a set of different RF phase-cycling increments. Joint processing of phase-cycled acquisitions serves to mitigate sensitivity to field inhomogeneity. Recently phase-cycled bSSFP acquisitions were also leveraged to estimate relaxation parameters based on explicit signal models. While effective, these model-based methods often involve a large number of acquisitions (N~10-16), degrading scan efficiency. Here, we propose a new constrained ellipse fitting method (CELF) for parameter estimation with improved efficiency and accuracy in phase-cycled bSSFP MRI. CELF is based on the elliptical signal model framework for complex bSSFP signals; and it introduces geometrical constraints on ellipse properties to improve estimation efficiency, and dictionary-based identification to improve estimation accuracy. CELF generates maps of T1, T2, off-resonance and on-resonant bSSFP signal by employing a separate B1 map to mitigate sensitivity to flip angle variations. Our results indicate that CELF can produce accurate off-resonance and banding-free bSSFP maps with as few as N=4 acquisitions, while estimation accuracy for relaxation parameters is notably limited by biases from microstructural sensitivity of bSSFP imaging.


title: A Projection-based Reduced-order Method for Electron Transport Problems with Long-range Interactions

abstract: Long-range interactions play a central role in electron transport. At the same time, they present a challenge for direct computer simulations, since sufficiently large portions of the bath have to be included in the computation to accurately compute the Coulomb potential. This article presents a reduced-order approach, by deriving an open quantum model for the reduced density-matrix. To treat the transient dynamics, the problem is placed in a reduced-order framework. The dynamics, described by the Liouville von Neumann equation, is projected to subspaces using a Petrov-Galerkin projection. In order to recover the global electron density profile as a vehicle to compute the Coulomb potential, we propose a domain decomposition approach, where the computational domain also includes segments of the bath that are selected using logarithmic grids. This approach leads to a multi-component self-energy that enters the effective Hamiltonian. We demonstrate the accuracy of the reduced model using a molecular junction built from a Lithium chains.


title: Using the Swing Lemma and $\mathcal{C}_1$-diagrams for congruences of planar semimodular lattices

abstract: A planar semimodular lattice $K$ is \emph{slim} if $\mathsf{M}_{3}$ is not a sublattice of~$K$. In a recent paper, G. Cz\'edli found four new properties of congruence lattices of slim, planar, semimodular lattices, including the \emph{No Child Property}: \emph{Let~$\mathcal{P}$ be the ordered set of join-irreducible congruences of $K$. Let $x,y,z \in \mathcal{P}$ and let $z$ be a~maximal element of $\mathcal{P}$. If $x \neq y$ and $x, y \prec z$ in $\mathcal{P}$, then there is no element $u$ of $\mathcal{P}$ such that $u \prec x, y$ in $\mathcal{P}$.} We are applying my Swing Lemma, 2015, and a type of standardized diagrams of Cz\'edli's, to verify his four properties.


title: Highlighting the Importance of Reducing Research Bias and Carbon Emissions in CNNs

abstract: Convolutional neural networks (CNNs) have become commonplace in addressing major challenges in computer vision. Researchers are not only coming up with new CNN architectures but are also researching different techniques to improve the performance of existing architectures. However, there is a tendency to over-emphasize performance improvement while neglecting certain important variables such as simplicity, versatility, the fairness of comparisons, and energy efficiency. Overlooking these variables in architectural design and evaluation has led to research bias and a significantly negative environmental impact. Furthermore, this can undermine the positive impact of research in using deep learning models to tackle climate change. Here, we perform an extensive and fair empirical study of a number of proposed techniques to gauge the utility of each technique for segmentation and classification. Our findings restate the importance of favoring simplicity over complexity in model design (Occam's Razor). Furthermore, our results indicate that simple standardized practices can lead to a significant reduction in environmental impact with little drop in performance. We highlight that there is a need to rethink the design and evaluation of CNNs to alleviate the issue of research bias and carbon emissions.


title: Neural Active Learning with Performance Guarantees

abstract: We investigate the problem of active learning in the streaming setting in non-parametric regimes, where the labels are stochastically generated from a class of functions on which we make no assumptions whatsoever. We rely on recently proposed Neural Tangent Kernel (NTK) approximation tools to construct a suitable neural embedding that determines the feature space the algorithm operates on and the learned model computed atop. Since the shape of the label requesting threshold is tightly related to the complexity of the function to be learned, which is a-priori unknown, we also derive a version of the algorithm which is agnostic to any prior knowledge. This algorithm relies on a regret balancing scheme to solve the resulting online model selection problem, and is computationally efficient. We prove joint guarantees on the cumulative regret and number of requested labels which depend on the complexity of the labeling function at hand. In the linear case, these guarantees recover known minimax results of the generalization error as a function of the label complexity in a standard statistical learning setting.


title: Statistical Inference for Cox Proportional Hazards Models with a Diverging Number of Covariates

abstract: For statistical inference on regression models with a diverging number of covariates, the existing literature typically makes sparsity assumptions on the inverse of the Fisher information matrix. Such assumptions, however, are often violated under Cox proportion hazards models, leading to biased estimates with under-coverage confidence intervals. We propose a modified debiased lasso approach, which solves a series of quadratic programming problems to approximate the inverse information matrix without posing sparse matrix assumptions. We establish asymptotic results for the estimated regression coefficients when the dimension of covariates diverges with the sample size. As demonstrated by extensive simulations, our proposed method provides consistent estimates and confidence intervals with nominal coverage probabilities. The utility of the method is further demonstrated by assessing the effects of genetic markers on patients' overall survival with the Boston Lung Cancer Survival Cohort, a large-scale epidemiology study investigating mechanisms underlying the lung cancer.


title: Verification in the Loop: Correct-by-Construction Control Learning with Reach-avoid Guarantees

abstract: In the current control design of safety-critical autonomous systems, formal verification techniques are typically applied after the controller is designed to evaluate whether the required properties (e.g., safety) are satisfied. However, due to the increasing system complexity and the fundamental hardness of designing a controller with formal guarantees, such an open-loop process of design-then-verify often results in many iterations and fails to provide the necessary guarantees. In this paper, we propose a correct-by-construction control learning framework that integrates the verification into the control design process in a closed-loop manner, i.e., design-while-verify. Specifically, we leverage the verification results (computed reachable set of the system state) to construct feedback metrics for control learning, which measure how likely the current design of control parameters can meet the required reach-avoid property for safety and goal-reaching. We formulate an optimization problem based on such metrics for tuning the controller parameters, and develop an approximated gradient descent algorithm with a difference method to solve the optimization problem and learn the controller. The learned controller is formally guaranteed to meet the required reach-avoid property. By treating verifiability as a first-class objective and effectively leveraging the verification results during the control learning process, our approach can significantly improve the chance of finding a control design with formal property guarantees. This is demonstrated via a set of experiments on both linear and non-linear systems that use model-based or neural network based controllers.


title: Extractive Research Slide Generation Using Windowed Labeling Ranking

abstract: Presentation slides describing the content of scientific and technical papers are an efficient and effective way to present that work. However, manually generating presentation slides is labor intensive. We propose a method to automatically generate slides for scientific papers based on a corpus of 5000 paper-slide pairs compiled from conference proceedings websites. The sentence labeling module of our method is based on SummaRuNNer, a neural sequence model for extractive summarization. Instead of ranking sentences based on semantic similarities in the whole document, our algorithm measures importance and novelty of sentences by combining semantic and lexical features within a sentence window. Our method outperforms several baseline methods including SummaRuNNer by a significant margin in terms of ROUGE score.


title: Integral Bases and Invariant Vectors for Weil Representations

abstract: We show that the Weil representation associated with any discriminant form admits a basis in which the action of the representation involves algebraic integers. The action of a general element of $\operatorname{SL}_{2}(\mathbb{Z})$ on many parts of these bases is simple an explicit, a fact that we use for determining the dimension of the space of invariants for some families of discriminant forms.


title: Chaotic Einstein-Podolsky-Rosen pairs, measurements and time reversal

abstract: We consider a situation when evolution of an entangled Einstein-Podolsky-Rosen (EPR) pair takes place in a regime of quantum chaos being chaotic in the classical limit. This situation is studied on an example of chaotic pair dynamics described by the quantum Chirikov standard map. The time evolution is reversible even if a presence of small errors breaks time reversal of classical dynamics due to exponential growth of errors induced by exponential chaos instability. However, the quantum evolution remains reversible since a quantum dynamics instability exists only on a logarithmically short Ehrenfest time scale. We show that due to EPR pair entanglement a measurement of one particle at the moment of time reversal breaks exact time reversal of another particle which demonstrates only an approximate time reversibility. This result is interpreted in the framework of the Schmidt decomposition and Feynman path integral formulation of quantum mechanics. The time reversal in this system has already been realized with cold atoms in kicked optical lattices in absence of entanglement and measurements. On the basis of the obtained results we argue that the experimental investigations of time reversal of chaotic EPR pairs is within reach of present cold atom capabilities.


title: Accurate Self-Configuration of Rectangular Multiport Interferometers

abstract: Multiport interferometers based on integrated beamsplitter meshes are widely used in photonic technologies. While the rectangular mesh is favored for its compactness and uniformity, its geometry resists conventional self-configuration approaches, which are essential to programming large meshes in the presence of fabrication error. Here, we present a new configuration algorithm, related to the $2\times 2$ block decomposition of a unitary matrix, that overcomes this limitation. Our proposed algorithm is robust to errors, requires no prior knowledge of the process variations, and relies only on external sources and detectors. We show that self-configuration using this technique reduces the effect of fabrication errors by the same quadratic factor observed in triangular meshes. This relaxes a significant limit to the size of multiport interferometers, removing a major roadblock to the scaling of optical quantum and machine-learning hardware.


title: Obstructions to the existence of compact Clifford-Klein forms for tangential symmetric spaces

abstract: For a homogeneous space $G/H$ of reductive type, we consider the tangential homogeneous space $G_\theta/H_\theta$. In this paper, we give obstructions to the existence of compact Clifford-Klein forms for such tangential symmetric spaces and obtain new tangential symmetric spaces which do not admit compact Clifford-Klein forms. As a result, in the class of irreducible classical semisimple symmetric spaces, we have only two types of symmetric spaces which are not proved not to admit compact Clifford-Klein forms. The existence problem of compact Clifford-Klein forms for homogeneous spaces of reductive type, which was initiated by T. Kobayashi in 1980s, has been studied by various methods but is not completely solved yet. On the other hand, the one for tangential homogeneous spaces has been studied since 2000s and an analogous criterion was proved by T. Kobayashi and T. Yoshino. In concrete examples, further works are needed to verify Kobayashi-Yoshino's condition by direct calculations. In this paper, some easy-to-check necessary conditions(=obstructions) for the existence of compact quotients in the tangential setting are given, and they are applied to the case of symmetric spaces. The conditions are related to various fields of mathematics such as associated pair of symmetric space, Calabi-Markus phenomenon, trivializability of vector bundle (parallelizability, Pontrjagin class), Hurwitz-Radon number and Pfister's theorem (the existence problem of common zero points of polynomials of odd degree).


title: DyDiff-VAE: A Dynamic Variational Framework for Information Diffusion Prediction

abstract: This paper describes a novel diffusion model, DyDiff-VAE, for information diffusion prediction on social media. Given the initial content and a sequence of forwarding users, DyDiff-VAE aims to estimate the propagation likelihood for other potential users and predict the corresponding user rankings. Inferring user interests from diffusion data lies the foundation of diffusion prediction, because users often forward the information in which they are interested or the information from those who share similar interests. Their interests also evolve over time as the result of the dynamic social influence from neighbors and the time-sensitive information gained inside/outside the social media. Existing works fail to model users' intrinsic interests from the diffusion data and assume user interests remain static along the time. DyDiff-VAE advances the state of the art in two directions: (i) We propose a dynamic encoder to infer the evolution of user interests from observed diffusion data. (ii) We propose a dual attentive decoder to estimate the propagation likelihood by integrating information from both the initial cascade content and the forwarding user sequence. Extensive experiments on four real-world datasets from Twitter and Youtube demonstrate the advantages of the proposed model; we show that it achieves 43.3% relative gains over the best baseline on average. Moreover, it has the lowest run-time compared with recurrent neural network based models.


title: Bayesian graphical modeling for heterogeneous causal effects

abstract: Our motivation stems from current medical research aiming at personalized treatment using a molecular-based approach. The broad goal is to develop a more precise and targeted decision making process, relative to traditional treatments based primarily on clinical diagnoses. Specifically, we consider patients affected by Acute Myeloid Leukemia (AML), an hematological cancer characterized by uncontrolled proliferation of hematopoietic stem cells in the bone marrow. Because AML responds poorly to chemoterapeutic treatments, the development of targeted therapies is essential to improve patients' prospects. In particular, the dataset we analyze contains the levels of proteins involved in cell cycle regulation and linked to the progression of the disease. We analyse treatment effects within a causal framework represented by a Directed Acyclic Graph (DAG) model, whose vertices are the protein levels in the network. A major obstacle in implementing the above program is however represented by individual heterogeneity. We address this issue through a Dirichlet Process (DP) mixture of Gaussian DAG-models where both the graphical structure as well as the allied model parameters are regarded as uncertain. Our procedure determines a clustering structure of the units reflecting the underlying heterogeneity, and produces subject-specific estimates of causal effects based on Bayesian model averaging. With reference to the AML dataset, we identify different effects of protein regulation among individuals; moreover, our method clusters patients into groups that exhibit only mild similarities with traditional categories based on morphological features.


title: Tabular Data: Deep Learning is Not All You Need

abstract: A key element in solving real-life data science problems is selecting the types of models to use. Tree ensemble models (such as XGBoost) are usually recommended for classification and regression problems with tabular data. However, several deep learning models for tabular data have recently been proposed, claiming to outperform XGBoost for some use cases. This paper explores whether these deep models should be a recommended option for tabular data by rigorously comparing the new deep models to XGBoost on various datasets. In addition to systematically comparing their performance, we consider the tuning and computation they require. Our study shows that XGBoost outperforms these deep models across the datasets, including the datasets used in the papers that proposed the deep models. We also demonstrate that XGBoost requires much less tuning. On the positive side, we show that an ensemble of deep models and XGBoost performs better on these datasets than XGBoost alone.


title: Power System Transient Modeling and Simulation using Integrated Circuit

abstract: Transient stability analysis (TSA) plays an important role in power system analysis to investigate the stability of power system. Traditionally, transient stability analysis methods have been developed using time domain simulation by means of numerical integration method. In this paper, a new approach is proposed to model power systems as an integrated circuit and simulate the power system dynamic behavior by integrated circuit simulator. The proposed method modeled power grid, generator, governor, and exciter with high fidelity. The power system dynamic simulation accuracy and efficiency of the proposed approach are verified and demonstrated by case study on an IEEE standard system.


title: GeV muon beams with picometer-class emittance from electron-photon collisions

abstract: One of the challenge of future muon colliders is the production of muon beams carrying high phase space densities. In particular the muon beam normalised transverse emittance is a relevant figure of merit to meet luminosity requests. A typical issue impacting the achieved transverse emittance in muon collider schemes so far considered is the phase space dilution caused by coulomb interaction of primary particles propagating into the target where muons are generated. In this study we present a new scheme for muon beam generation occurring in vacuum by interactions of electron and photon beams. Setting the center of mass energy at about twice the threshold (i.e. around $350$ MeV) the normalised emittance of the muon beam generated via muon pair production reaction ($e^-+\gamma \rightarrow e^-+\mu^+/\mu^-$) is largely independent on the emittance of the colliding electron beam and is set basically by the excess of transverse momentum in the muon pair creation. In absence of any other mechanism for emittance dilution, the resulting muon beam, with energy in the range of few tens of GeV, is characterised by an ultra-low normalised transverse rms emittance of a few nm rad, corresponding to a geometrical emittance below $10$ pm rad. This opens the way to a new muon collider paradigm based on muon sources conceived with primary colliding beams delivered by $100$ GeV-class energy recovery linacs interacting with hard-X ray free electron lasers. The challenge is to achieve the requested luminosity of the muon collider adopting a strategy of low muon fluxes/currents combined to ultra-low emittances, so to largely reduce also the levels of muon beam-induced background.


title: Gravitational lensing and wormhole shadows

abstract: The problem of bending and scattering of light rays passing outside from the entrance to a wormhole with zero gravitational mass is considered. The process of ray capture by a wormhole as well as the process of formation of a shadow when illuminated by a standard screen is investigated. These mechanisms are also compared to the case of motion of light rays in the vicinity of the Schwarzschild black hole.


title: Structured Reordering for Modeling Latent Alignments in Sequence Transduction

abstract: Despite success in many domains, neural models struggle in settings where train and test examples are drawn from different distributions. In particular, in contrast to humans, conventional sequence-to-sequence (seq2seq) models fail to generalize systematically, i.e., interpret sentences representing novel combinations of concepts (e.g., text segments) seen in training. Traditional grammar formalisms excel in such settings by implicitly encoding alignments between input and output segments, but are hard to scale and maintain. Instead of engineering a grammar, we directly model segment-to-segment alignments as discrete structured latent variables within a neural seq2seq model. To efficiently explore the large space of alignments, we introduce a reorder-first align-later framework whose central component is a neural reordering module producing {\it separable} permutations. We present an efficient dynamic programming algorithm performing exact marginal inference of separable permutations, and, thus, enabling end-to-end differentiable training of our model. The resulting seq2seq model exhibits better systematic generalization than standard models on synthetic problems and NLP tasks (i.e., semantic parsing and machine translation).


title: Tuning magnetic states in CrI$_3$ bilayers via Moir\'e patterns

abstract: Commensurable twisted bilayers can drastically change the magnetic properties of chromium trihalide layered compounds, which opens novel opportunities for tuning magnetic states through layer rotations. Here, we introduce a mathematical approach to obtain moir\'e patterns in twisted hexagonal bilayers by performing a certain commensurable rotation $\theta$ over one layer. To test our approach, we apply it to find moir\'e structures with $\theta=21.79^{\circ}$ and $32.20^{\circ}$ in the phases R$\bar{3}$ and C2/m of CrI$_{3}$. For comparison purposes, we also consider a non-shifted CrI$_{3}$ structure. Electronic and magnetic properties of the so-obtained systems are computed by \textit{Ab-Initio} methodologies. Results show the presence of rotation-angle-dependent magnetic configurations and steep modifications of the dispersion bands due to variations in the nearest and next nearest distances among layers of Cr atoms. Modifications obtained from these commensurable rotations are discussed on the basis of competition among different energy contributions due to changes in the atomic arrangements.


title: Understand and Improve Contrastive Learning Methods for Visual Representation: A Review

abstract: Traditional supervised learning methods are hitting a bottleneck because of their dependency on expensive manually labeled data and their weaknesses such as limited generalization ability and vulnerability to adversarial attacks. A promising alternative, self-supervised learning, as a type of unsupervised learning, has gained popularity because of its potential to learn effective data representations without manual labeling. Among self-supervised learning algorithms, contrastive learning has achieved state-of-the-art performance in several fields of research. This literature review aims to provide an up-to-date analysis of the efforts of researchers to understand the key components and the limitations of self-supervised learning.


title: Error estimate of a decoupled numerical scheme for the Cahn-Hilliard-Stokes-Darcy system

abstract: We analyze a fully discrete finite element numerical scheme for the Cahn-Hilliard-Stokes-Darcy system that models two-phase flows in coupled free flow and porous media. To avoid a well-known difficulty associated with the coupling between the Cahn-Hilliard equation and the fluid motion, we make use of the operator-splitting in the numerical scheme, so that these two solvers are decoupled, which in turn would greatly improve the computational efficiency. The unique solvability and the energy stability have been proved in~\cite{CHW2017}. In this work, we carry out a detailed convergence analysis and error estimate for the fully discrete finite element scheme, so that the optimal rate convergence order is established in the energy norm, i.e.,, in the $\ell^\infty (0, T; H^1) \cap \ell^2 (0, T; H^2)$ norm for the phase variables, as well as in the $\ell^\infty (0, T; H^1) \cap \ell^2 (0, T; H^2)$ norm for the velocity variable. Such an energy norm error estimate leads to a cancellation of a nonlinear error term associated with the convection part, which turns out to be a key step to pass through the analysis. In addition, a discrete $\ell^2 (0;T; H^3)$ bound of the numerical solution for the phase variables plays an important role in the error estimate, which is accomplished via a discrete version of Gagliardo-Nirenberg inequality in the finite element setting.