title: Heat kernel estimates for Markov processes of direction-dependent type

abstract: We prove sharp pointwise heat kernel estimates for symmetric Markov processes associated with symmetric Dirichlet forms that are local with respect to some coordinates and nonlocal with respect to the remaining coordinates. The main theorem is a robustness result like the famous estimate for the fundamental solution of second order differential operators, obtained by Donald G. Aronson. Analogous to his result, we show that the corresponding translation-invariant process and the one given by the general Dirichlet form share the same pointwise points.


title: Attention-based Domain Adaptation for Single Stage Detectors

abstract: While domain adaptation has been used to improve the performance of object detectors when the training and test data follow different distributions, previous work has mostly focused on two-stage detectors. This is because their use of region proposals makes it possible to perform local adaptation, which has been shown to significantly improve the adaptation effectiveness. Here, by contrast, we target single-stage architectures, which are better suited to resource-constrained detection than two-stage ones but do not provide region proposals. To nonetheless benefit from the strength of local adaptation, we introduce an attention mechanism that lets us identify the important regions on which adaptation should focus. Our method gradually adapts the features from global, image-level to local, instance-level. Our approach is generic and can be integrated into any single-stage detector. We demonstrate this on standard benchmark datasets by applying it to both SSD and YOLOv5. Furthermore, for equivalent single-stage architectures, our method outperforms the state-of-the-art domain adaptation techniques even though they were designed for specific detectors.


title: Closure relations of Newton strata in Iwahori double cosets

abstract: We consider the Newton stratification on Iwahori double cosets for a connected reductive group. We prove the existence of Newton strata whose closures cannot be expressed as a union of strata, and show how this is implied by the existence of non-equidimensional affine Deligne-Lusztig varieties. We also give an explicit example for a group of type $A_4$.


title: Probing Pre-Trained Language Models for Disease Knowledge

abstract: Pre-trained language models such as ClinicalBERT have achieved impressive results on tasks such as medical Natural Language Inference. At first glance, this may suggest that these models are able to perform medical reasoning tasks, such as mapping symptoms to diseases. However, we find that standard benchmarks such as MedNLI contain relatively few examples that require such forms of reasoning. To better understand the medical reasoning capabilities of existing language models, in this paper we introduce DisKnE, a new benchmark for Disease Knowledge Evaluation. To construct this benchmark, we annotated each positive MedNLI example with the types of medical reasoning that are needed. We then created negative examples by corrupting these positive examples in an adversarial way. Furthermore, we define training-test splits per disease, ensuring that no knowledge about test diseases can be learned from the training data, and we canonicalize the formulation of the hypotheses to avoid the presence of artefacts. This leads to a number of binary classification problems, one for each type of reasoning and each disease. When analysing pre-trained models for the clinical/biomedical domain on the proposed benchmark, we find that their performance drops considerably.


title: TimeLens: Event-based Video Frame Interpolation

abstract: State-of-the-art frame interpolation methods generate intermediate frames by inferring object motions in the image from consecutive key-frames. In the absence of additional information, first-order approximations, i.e. optical flow, must be used, but this choice restricts the types of motions that can be modeled, leading to errors in highly dynamic scenarios. Event cameras are novel sensors that address this limitation by providing auxiliary visual information in the blind-time between frames. They asynchronously measure per-pixel brightness changes and do this with high temporal resolution and low latency. Event-based frame interpolation methods typically adopt a synthesis-based approach, where predicted frame residuals are directly applied to the key-frames. However, while these approaches can capture non-linear motions they suffer from ghosting and perform poorly in low-texture regions with few events. Thus, synthesis-based and flow-based approaches are complementary. In this work, we introduce Time Lens, a novel indicates equal contribution method that leverages the advantages of both. We extensively evaluate our method on three synthetic and two real benchmarks where we show an up to 5.21 dB improvement in terms of PSNR over state-of-the-art frame-based and event-based methods. Finally, we release a new large-scale dataset in highly dynamic scenarios, aimed at pushing the limits of existing methods.


title: Data Science Methodologies: Current Challenges and Future Approaches

abstract: Data science has employed great research efforts in developing advanced analytics, improving data models and cultivating new algorithms. However, not many authors have come across the organizational and socio-technical challenges that arise when executing a data science project: lack of vision and clear objectives, a biased emphasis on technical issues, a low level of maturity for ad-hoc projects and the ambiguity of roles in data science are among these challenges. Few methodologies have been proposed on the literature that tackle these type of challenges, some of them date back to the mid-1990, and consequently they are not updated to the current paradigm and the latest developments in big data and machine learning technologies. In addition, fewer methodologies offer a complete guideline across team, project and data & information management. In this article we would like to explore the necessity of developing a more holistic approach for carrying out data science projects. We first review methodologies that have been presented on the literature to work on data science projects and classify them according to the their focus: project, team, data and information management. Finally, we propose a conceptual framework containing general characteristics that a methodology for managing data science projects with a holistic point of view should have. This framework can be used by other researchers as a roadmap for the design of new data science methodologies or the updating of existing ones.


title: Learning-Aided Heuristics Design for Storage System

abstract: Computer systems such as storage systems normally require transparent white-box algorithms that are interpretable for human experts. In this work, we propose a learning-aided heuristic design method, which automatically generates human-readable strategies from Deep Reinforcement Learning (DRL) agents. This method benefits from the power of deep learning but avoids the shortcoming of its black-box property. Besides the white-box advantage, experiments in our storage productions resource allocation scenario also show that this solution outperforms the systems default settings and the elaborately handcrafted strategy by human experts.


title: Decentralized Personalized Federated Learning for Min-Max Problems

abstract: Personalized Federated Learning (PFL) has witnessed remarkable advancements, enabling the development of innovative machine learning applications that preserve the privacy of training data. However, existing theoretical research in this field has primarily focused on distributed optimization for minimization problems. This paper is the first to study PFL for saddle point problems encompassing a broader range of optimization problems, that require more than just solving minimization problems. In this work, we consider a recently proposed PFL setting with the mixing objective function, an approach combining the learning of a global model together with locally distributed learners. Unlike most previous work, which considered only the centralized setting, we work in a more general and decentralized setup that allows us to design and analyze more practical and federated ways to connect devices to the network. We proposed new algorithms to address this problem and provide a theoretical analysis of the smooth (strongly) convex-(strongly) concave saddle point problems in stochastic and deterministic cases. Numerical experiments for bilinear problems and neural networks with adversarial noise demonstrate the effectiveness of the proposed methods.


title: Excitonic Tonks-Girardeau and charge-density wave phases in monolayer semiconductors

abstract: Excitons in two-dimensional semiconductors provide a novel platform for fundamental studies of many-body interactions. In particular, dipolar interactions between spatially indirect excitons may give rise to strongly correlated phases of matter that so far have been out of reach of experiments. Here, we show that excitonic few-body systems in atomically thin transition-metal dichalcogenides undergo a crossover from a Tonks-Girardeau to a charge-density-wave regime. To this end, we take into account realistic system parameters and predict the effective exciton-exciton interaction potential. We find that the pair correlation function contains key signatures of the many-body crossover already at small exciton numbers and show that photoluminescence spectra provide readily accessible experimental fingerprints of these strongly correlated quantum many-body states.


title: A Multi Polarization Square Patch Antenna with a Reconfigurable Feeding Network

abstract: A multi-polarization square patch antenna with a reconfigurable feeding network is presented in this paper. The reconfigurable feeding network of this antenna is implemented on an FR-4 substrate by a Wilkinson power divider and a branch line coupler which perform amplitude distribution in the feeding network. Besides, two switching circuits which consist of one PIN diode (BAR63-02w) and its DC biasing circuit manage the RF signal flow on this feeding network. These switching circuits control the phase of the RF signal applied to the square patch, so it can provide linear polarization, left-hand and right-hand circular polarization at 2.45 GHz which has many applications in wireless networks. The simulated and measured results are presented which illuminate acceptable axial ratio bandwidth (ARBW) for both right-hand and left-hand circular polarization in (2.38-2.48 GHz) and minimum -10 dB return loss at 2.45 GHz.


title: Topological data analysis identifies emerging adaptive mutations in SARS-CoV-2

abstract: The COVID-19 pandemic has initiated an unprecedented worldwide effort to characterize its evolution through the mapping of mutations of the coronavirus SARS-CoV-2. The early identification of mutations that could confer adaptive advantages to the virus, such as higher infectivity or immune evasion, is of paramount importance. However, the large number of currently available genomes precludes the efficient use of phylogeny-based methods. Here we present CoVtRec, a fast and scalable Topological Data Analysis approach for the surveillance of emerging adaptive mutations in large genomic datasets. Our method overcomes limitations of state-of-the-art phylogeny-based approaches by quantifying the potential adaptiveness of mutations merely by their topological footprint in the genome alignment, without resorting to the reconstruction of a single optimal phylogenetic tree. Analyzing millions of SARS-CoV-2 genomes from GISAID, we find a correlation between topological signals and adaptation to the human host. By leveraging the stratification by time in sequence data, our method enables the high-resolution longitudinal analysis of topological signals of adaptation. We characterize the convergent evolution of the coronavirus throughout the whole pandemic to date, report on emerging potentially adaptive mutations, and pinpoint mutations in Variants of Concern that are likely associated with positive selection. Our approach can improve the surveillance of mutations of concern and guide experimental studies.


title: On the Bieri-Neumann-Strebel-Renz invariants and limit groups over Droms RAAGs

abstract: For a group $G$ that is a limit group over Droms RAAGs such that $G$ has trivial center, we show that $\Sigma^1(G) = \emptyset = \Sigma^1(G, \mathbb{Q})$. For a group $H$ that is a finitely presented residually Droms RAAG we calculate $\Sigma^1(H)$ and $\Sigma^2(H)_{dis}$. In addition, we obtain a necessary condition for $[\chi]$ to belong to $\Sigma^n(H)$.


title: A test of constancy of dark matter halo surface density and radial acceleration relation in relaxed galaxy groups

abstract: The dark matter halo surface density, given by the product of the dark matter core radius ($r_c$) and core density ($\rho_c$) has been shown to be a constant for a wide range of isolated galaxy systems. Here, we carry out a test of this {\em ansatz} using a sample of 17 relaxed galaxy groups observed using Chandra and XMM-Newton, as an extension of our previous analysis with galaxy clusters. We find that $\rho_c \propto r_c^{-1.35^{+0.16}_{-0.17}}$, with an intrinsic scatter of about 27.3%, which is about 1.5 times larger than that seen for galaxy clusters. Our results thereby indicate that the surface density is discrepant with respect to scale invariance by about 2$\sigma$, and its value is about four times greater than that for galaxies. Therefore, the elevated values of the halo surface density for groups and clusters indicate that the surface density cannot be a universal constant for all dark matter dominated systems. Furthermore, we also implement a test of the radial acceleration relation for this group sample. We find that the residual scatter in the radial acceleration relation is about 0.32 dex and a factor of three larger than that obtained using galaxy clusters. The acceleration scale which we obtain is in-between that seen for galaxies and clusters.


title: Transformation formula for the Reduced Bergman kernel and its Application

abstract: In this article, we prove the transformation formula for the reduced Bergman kernels under proper holomorphic correspondences between bounded domains in the complex plane. As a corollary, we obtain the transformation formula for the reduced Bergman kernels under proper holomorphic maps. We also establish the transformation formula for the weighted reduced Bergman kernels under proper holomorphic maps. Finally, we provide an application of this transformation formula.


title: RRULES: An improvement of the RULES rule-based classifier

abstract: RRULES is presented as an improvement and optimization over RULES, a simple inductive learning algorithm for extracting IF-THEN rules from a set of training examples. RRULES optimizes the algorithm by implementing a more effective mechanism to detect irrelevant rules, at the same time that checks the stopping conditions more often. This results in a more compact rule set containing more general rules which prevent overfitting the training set and obtain a higher test accuracy. Moreover, the results show that RRULES outperforms the original algorithm by reducing the coverage rate up to a factor of 7 while running twice or three times faster consistently over several datasets.


title: Node Classification Meets Link Prediction on Knowledge Graphs

abstract: Node classification and link prediction are widely studied in graph representation learning. While both transductive node classification and link prediction operate over a single input graph, they have so far been studied separately. Node classification models take an input graph with node features and incomplete node labels, and implicitly assume that the graph is relationally complete, i.e., no edges are missing. By contrast, link prediction models are solely motivated by relational incompleteness of the input graphs, and do not typically leverage node features or classes. We propose a unifying perspective and study the problems of (i) transductive node classification over incomplete graphs and (ii) link prediction over graphs with node features, introduce a new dataset for this setting, WikiAlumni, and conduct an extensive benchmarking study.


title: Regularity properties of $k$-Brjuno and Wilton functions

abstract: We study functions related to the classical Brjuno function, namely $k$-Brjuno functions and the Wilton function. Both appear in the study of boundary regularity properties of (quasi) modular forms and their integrals. We consider various possible versions of them, based on the $\alpha$-continued fraction developments. We study their BMO regularity properties and their behaviour near rational numbers of their finite truncations.


title: Dynamic Based Estimator for UAVs with Real-time Identification Using DNN and the Modified Relay Feedback Test

abstract: Control performance of Unmanned Aerial Vehicles (UAVs) is directly affected by their ability to estimate their states accurately. With the increasing popularity of autonomous UAV solutions in real world applications, it is imperative to develop robust adaptive estimators that can ameliorate sensor noises in low-cost UAVs. Utilizing the knowledge of UAV dynamics in estimation can provide significant advantages, but remains challenging due to the complex and expensive pre-flight experiments required to obtain UAV dynamic parameters. In this paper, we propose two decoupled dynamic model based Extended Kalman Filters for UAVs, that provide high rate estimates for position, and velocity of rotational and translational states, as well as filtered inertial acceleration. The dynamic model parameters are estimated online using the Deep Neural Network and Modified Relay Feedback Test (DNN-MRFT) framework, without requiring any prior knowledge of the UAV physical parameters. The designed filters with real-time identified process model parameters are tested experimentally and showed two advantages. Firstly, smooth and lag-free estimates of the UAV rotational speed and inertial acceleration are obtained, and used to improve the closed loop system performance, reducing the controller action by over 6 %. Secondly, the proposed approach enabled the UAV to track aggressive trajectories with low rate position measurements, a task usually infeasible under those conditions. The experimental data shows that we achieved estimation performance matching other methods that requires full knowledge of the UAV parameters.


title: Maximin Shares Under Cardinality Constraints

abstract: We study the problem of fair allocation of a set of indivisible items among agents with additive valuations, under cardinality constraints. In this setting, the items are partitioned into categories, each with its own limit on the number of items it may contribute to any bundle. We consider the fairness measure known as the maximin share (MMS) guarantee, and propose a novel polynomial-time algorithm for finding $1/2$-approximate MMS allocations for goods -- an improvement from the previously best available guarantee of $11/30$. For single-category instances, we show that a modified variant of our algorithm is guaranteed to produce $2/3$-approximate MMS allocations. Among various other existence and non-existence results, we show that a $(\sqrt{n}/(2\sqrt{n} - 1))$-approximate MMS allocation always exists for goods. For chores, we show similar results as for goods, with a $2$-approximate algorithm in the general case and a $3/2$-approximate algorithm for single-category instances. We extend the notions and algorithms related to ordered and reduced instances to work with cardinality constraints, and combine these with bag filling style procedures to construct our algorithms.


title: YARARA: Significant improvement of RV precision through post-processing of spectral time-series

abstract: Aims: Even the most-precise radial-velocity instruments gather high-resolution spectra that present systematic errors that a data reduction pipeline cannot identify and correct for efficiently. In this paper, we aim at improving the radial-velocity precision of HARPS measurements by cleaning individual extracted spectra using the wealth of information contained in spectra time-series. Methods: We developed YARARA, a post-processing pipeline designed to clean high-resolution spectra from instrumental systematics and atmospheric contamination. Spectra are corrected for: tellurics, interference pattern, detector stitching, ghosts and fiber B contaminations as well as more advanced spectral line-by-line corrections. YARARA uses Principal Component Analysis on spectra time-series with prior information to disentangle contaminations from real Doppler shifts. We applied YARARA on three systems: HD10700, HD215152 and HD10180 and compared our results to the HARPS standard Data Reduction Software and the SERVAL post-processing pipeline. Results: On HD10700, we obtain radial-velocity measurements that present a rms smaller than 1 m/s over the 13 years of the HARPS observations, which is 20 and 10 % better than the HARPS Data Reduction Software and the SERVAL post-processing pipeline, respectively. We also injected simulated planets on the data of HD10700 and demonstrated that YARARA does not alter pure Doppler shifted signals. On HD215152, we demonstrated that the 1-year signal visible in the periodogram becomes marginal after processing with YARARA and that the signals of the known planets become more significant. Finally, on HD10180, the known six exoplanets are well recovered although different orbitals parameters and planetary masses are provided by the new reduced spectra.


title: Quantum diffusion map for nonlinear dimensionality reduction

abstract: Inspired by random walk on graphs, diffusion map (DM) is a class of unsupervised machine learning that offers automatic identification of low-dimensional data structure hidden in a high-dimensional dataset. In recent years, among its many applications, DM has been successfully applied to discover relevant order parameters in many-body systems, enabling automatic classification of quantum phases of matter. However, classical DM algorithm is computationally prohibitive for a large dataset, and any reduction of the time complexity would be desirable. With a quantum computational speedup in mind, we propose a quantum algorithm for DM, termed quantum diffusion map (qDM). Our qDM takes as an input $N$ classical data vectors, performs an eigen-decomposition of the Markov transition matrix in time $O(\log^3 N)$, and classically constructs the diffusion map via the readout (tomography) of the eigenvectors, giving a total expected runtime proportional to $N^2 \text{polylog}\, N$. Lastly, quantum subroutines in qDM for constructing a Markov transition matrix, and for analyzing its spectral properties can also be useful for other random walk-based algorithms.


title: iNNformant: Boundary Samples as Telltale Watermarks

abstract: Boundary samples are special inputs to artificial neural networks crafted to identify the execution environment used for inference by the resulting output label. The paper presents and evaluates algorithms to generate transparent boundary samples. Transparency refers to a small perceptual distortion of the host signal (i.e., a natural input sample). For two established image classifiers, ResNet on FMNIST and CIFAR10, we show that it is possible to generate sets of boundary samples which can identify any of four tested microarchitectures. These sets can be built to not contain any sample with a worse peak signal-to-noise ratio than 70dB. We analyze the relationship between search complexity and resulting transparency.


title: Bulk reconstruction and Bogoliubov transformations in AdS$_2$

abstract: In the bulk reconstruction program, one constructs boundary representations of bulk fields. We investigate the relation between the global/Poincare and AdS-Rindler representations for AdS$_2$. We obtain the AdS-Rindler smearing function for massive and massless fields and show that the global and AdS-Rindler boundary representations are related by conformal transformations. We also use the boundary representations of creation and annihilation operators to compute the Bogoliubov transformation relating global modes to AdS-Rindler modes for both massive and massless particles.


title: Explicit construction of Atiyah-Singer indices for maximally hypoelliptic operators on contact manifolds

abstract: The Atiyah-Singer index theorem gives a topological formula for the index of an elliptic differential operator. Enlightening from Alain Connes' tangent groupoid proof of the index theorem and van Erp's research for the Heisenberg index theory on contact manifolds, we give an explicit construction of a series of maps, whose induced map in K-theory is the Heisenberg Atiyah-Singer index map on contact manifolds. Our methods derive from Higson's construction for symbol class in K-theory.


title: Constraining Linear-chain CRFs to Regular Languages

abstract: A major challenge in structured prediction is to represent the interdependencies within output structures. When outputs are structured as sequences, linear-chain conditional random fields (CRFs) are a widely used model class which can learn \textit{local} dependencies in the output. However, the CRF's Markov assumption makes it impossible for CRFs to represent distributions with \textit{nonlocal} dependencies, and standard CRFs are unable to respect nonlocal constraints of the data (such as global arity constraints on output labels). We present a generalization of CRFs that can enforce a broad class of constraints, including nonlocal ones, by specifying the space of possible output structures as a regular language $\mathcal{L}$. The resulting regular-constrained CRF (RegCCRF) has the same formal properties as a standard CRF, but assigns zero probability to all label sequences not in $\mathcal{L}$. Notably, RegCCRFs can incorporate their constraints during training, while related models only enforce constraints during decoding. We prove that constrained training is never worse than constrained decoding, and show empirically that it can be substantially better in practice. Additionally, we demonstrate a practical benefit on downstream tasks by incorporating a RegCCRF into a deep neural model for semantic role labeling, exceeding state-of-the-art results on a standard dataset.


title: A Recipe for Social Media Analysis

abstract: The Ubiquitous nature of smartphones has significantly increased the use of social media platforms, such as Facebook, Twitter, TikTok, and LinkedIn, etc., among the public, government, and businesses. Facebook generated ~70 billion USD in 2019 in advertisement revenues alone, a ~27% increase from the previous year. Social media has also played a strong role in outbreaks of social protests responsible for political changes in different countries. As we can see from the above examples, social media plays a big role in business intelligence and international politics. In this paper, we present and discuss a high-level functional intelligence model (recipe) of Social Media Analysis (SMA). This model synthesizes the input data and uses operational intelligence to provide actionable recommendations. In addition, it also matches the synthesized function of the experiences and learning gained from the environment. The SMA model presented is independent of the application domain, and can be applied to different domains, such as Education, Healthcare and Government, etc. Finally, we also present some of the challenges faced by SMA and how the SMA model presented in this paper solves them.


title: Characterising the signatures of star-forming galaxies in the extra-galactic $\gamma$-ray background

abstract: Galaxies experiencing intense star-formation episodes are expected to be rich in energetic cosmic rays (CRs). These CRs undergo hadronic interactions with the interstellar gases of their host to drive $\gamma$-ray emission, which has already been detected from several nearby starbursts. Unresolved $\gamma$-ray emission from more distant star-forming galaxies (SFGs) is expected to contribute to the extra-galactic $\gamma$-ray background (EGB). However, despite the wealth of high-quality all-sky data from the Fermi-LAT $\gamma$-ray space telescope collected over more than a decade of operation, the exact contribution of such SFGs to the EGB remains unsettled. We investigate the high-energy $\gamma$-ray emission from SFGs up to redshift $z=3$ above a GeV, and assess the contribution they can make to the EGB. We show the $\gamma$-ray emission spectrum from a SFG population can be determined from just a small number of key parameters, from which we model a range of possible EGB realisations. We demonstrate that populations of SFGs leave anisotropic signatures in the EGB, and that these can be accessed using the spatial power spectrum. Moreover, we show that such signatures will be accessible with ongoing operation of current $\gamma$-ray instruments, and detection prospects will be greatly improved by the next generation of $\gamma$-ray observatories, in particular the Cherenkov Telescope Array.


title: Quantum oscillations, Magnetic breakdown and thermal Hall effect in Co$_3$Sn$_2$S$_2$

abstract: Co$_3$Sn$_2$S$_2$ is a ferromagnetic semi-metal with Weyl nodes in its band structure and a large anomalous Hall effect below its Curie temperature of 177 K. We present a detailed study of its Fermi surface and examine the relevance of the anomalous transverse Wiedemann Franz law to it. We studied Shubnikov-de Haas oscillations along two orientations in single crystals with a mobility as high as $2.7\times$10$^3$ cm$^2$V$^{-1}$s$^{-1}$ subject to a magnetic field as large as $\sim$ 60 T. The angle dependence of the frequencies is in agreement with density functional theory (DFT) calculations and reveals two types of hole pockets (H1, H2) and two types of electron pockets (E1, E2). An additional unexpected frequency emerges at high magnetic field. We attribute it to magnetic breakdown between the hole pocket H2 and the electron pocket E2, since it is close to the sum of the E2 and H2 fundamental frequencies. By measuring the anomalous thermal and electrical Hall conductivities, we quantified the anomalous transverse Lorenz ratio, which is close to the Sommerfeld ratio ($L_0=\frac{\pi^2}{3}\frac{k_B^2}{e^2}$) below 100 K and deviates downwards at higher temperatures. This finite temperature deviation from the anomalous Wiedemann-Franz law is a source of information on the distance between the sources and sinks of the Berry curvature and the chemical potential.


title: Pixel Sampling for Style Preserving Face Pose Editing

abstract: The existing auto-encoder based face pose editing methods primarily focus on modeling the identity preserving ability during pose synthesis, but are less able to preserve the image style properly, which refers to the color, brightness, saturation, etc. In this paper, we take advantage of the well-known frontal/profile optical illusion and present a novel two-stage approach to solve the aforementioned dilemma, where the task of face pose manipulation is cast into face inpainting. By selectively sampling pixels from the input face and slightly adjust their relative locations with the proposed ``Pixel Attention Sampling" module, the face editing result faithfully keeps the identity information as well as the image style unchanged. By leveraging high-dimensional embedding at the inpainting stage, finer details are generated. Further, with the 3D facial landmarks as guidance, our method is able to manipulate face pose in three degrees of freedom, i.e., yaw, pitch, and roll, resulting in more flexible face pose editing than merely controlling the yaw angle as usually achieved by the current state-of-the-art. Both the qualitative and quantitative evaluations validate the superiority of the proposed approach.


title: Coherent states for a system of an electron moving on plane

abstract: In this paper, we construct the coherent states for a system of an electron moving on plane in uniform external magnetic and electric fields. These coherent states are built in the context of both discrete and continuous spectra and satisfy the Gazeau-Klauder coherent states properties [1].


title: On the paper "Quantum theory cannot consistently describe the use of itself"

abstract: In the paper "Quantum theory cannot consistently describe the use of itself" by D. Frauchiger and R. Renner an atempt is made at proving a "no-go theorem" that states that either quantum theory cannot be universally applied, even to macroscopic systems, or some very intuitive properties concerning recursive reasoning and uniquenes of physical values must be false. In this paper, we give a concise description of the paper's result, and expose a detail in the proof.


title: Smart Gradient -- An Adaptive Technique for Improving Gradient Estimation

abstract: Computing the gradient of a function provides fundamental information about its behavior. This information is essential for several applications and algorithms across various fields. One common application that require gradients are optimization techniques such as stochastic gradient descent, Newton's method and trust region methods. However, these methods usually requires a numerical computation of the gradient at every iteration of the method which is prone to numerical errors. We propose a simple limited-memory technique for improving the accuracy of a numerically computed gradient in this gradient-based optimization framework by exploiting (1) a coordinate transformation of the gradient and (2) the history of previously taken descent directions. The method is verified empirically by extensive experimentation on both test functions and on real data applications. The proposed method is implemented in the R package smartGrad and in C++.


title: Computer Vision Tool for Detection, Mapping and Fault Classification of PV Modules in Aerial IR Videos

abstract: Increasing deployment of photovoltaics (PV) plants demands for cheap and fast inspection. A viable tool for this task is thermographic imaging by unmanned aerial vehicles (UAV). In this work, we develop a computer vision tool for the semi-automatic extraction of PV modules from thermographic UAV videos. We use it to curate a dataset containing 4.3 million IR images of 107842 PV modules from thermographic videos of seven different PV plants. To demonstrate its use for automated PV plant inspection, we train a ResNet-50 to classify ten common module anomalies with more than 90 % test accuracy. Experiments show that our tool generalizes well to different PV plants. It successfully extracts PV modules from 512 out of 561 plant rows. Failures are mostly due to an inappropriate UAV trajectory and erroneous module segmentation. Including all manual steps our tool enables inspection of 3.5 MW p to 9 MW p of PV installations per day, potentially scaling to multi-gigawatt plants due to its parallel nature. While we present an effective method for automated PV plant inspection, we are also confident that our approach helps to meet the growing demand for large thermographic datasets for machine learning tasks, such as power prediction or unsupervised defect identification.


title: A transmission problem for $(p,q)$-Laplacian

abstract: In this paper, we consider a double-phase problem characterised by a transmission that takes place across the zero level "surface" of the minimiser of the functional $$ J(v,\Omega) = \int_\Omega \left( |D v^+|^p + |D v^-|^q \right) dx. $$ We prove that a minimiser exists, and is H\"older continuous, whence using an intrinsic variation we prove a weak formulation of the free boundary condition across the zero level surface, formally represented by $$ (q-1)|D u^-|^q = (p-1) |D u^+|^p, \quad \hbox{on } \partial \{u > 0\}. $$ We show that the free boundary is $C^{1,\alpha}$ a.e. with respect to the measure $\Delta_p u^+$, whose support is of $\sigma$-finite $(n-1)$-dimensional Hausdorff measure.


title: Exploiting Sentence-Level Representations for Passage Ranking

abstract: Recently, pre-trained contextual models, such as BERT, have shown to perform well in language related tasks. We revisit the design decisions that govern the applicability of these models for the passage re-ranking task in open-domain question answering. We find that common approaches in the literature rely on fine-tuning a pre-trained BERT model and using a single, global representation of the input, discarding useful fine-grained relevance signals in token- or sentence-level representations. We argue that these discarded tokens hold useful information that can be leveraged. In this paper, we explicitly model the sentence-level representations by using Dynamic Memory Networks (DMNs) and conduct empirical evaluation to show improvements in passage re-ranking over fine-tuned vanilla BERT models by memory-enhanced explicit sentence modelling on a diverse set of open-domain QA datasets. We further show that freezing the BERT model and only training the DMN layer still comes close to the original performance, while improving training efficiency drastically. This indicates that the usual fine-tuning step mostly helps to aggregate the inherent information in a single output token, as opposed to adapting the whole model to the new task, and only achieves rather small gains.


title: Automated Machine Learning Techniques for Data Streams

abstract: Automated machine learning techniques benefited from tremendous research progress in recently. These developments and the continuous-growing demand for machine learning experts led to the development of numerous AutoML tools. However, these tools assume that the entire training dataset is available upfront and that the underlying distribution does not change over time. These assumptions do not hold in a data stream mining setting where an unbounded stream of data cannot be stored and is likely to manifest concept drift. Industry applications of machine learning on streaming data become more popular due to the increasing adoption of real-time streaming patterns in IoT, microservices architectures, web analytics, and other fields. The research summarized in this paper surveys the state-of-the-art open-source AutoML tools, applies them to data collected from streams, and measures how their performance changes over time. For comparative purposes, batch, batch incremental and instance incremental estimators are applied and compared. Moreover, a meta-learning technique for online algorithm selection based on meta-feature extraction is proposed and compared while model replacement and continual AutoML techniques are discussed. The results show that off-the-shelf AutoML tools can provide satisfactory results but in the presence of concept drift, detection or adaptation techniques have to be applied to maintain the predictive accuracy over time.


title: Multiobjective Bilevel Evolutionary Approach for Off-Grid Direction-of-Arrival Estimation

abstract: The source number identification is an essential step in direction-of-arrival (DOA) estimation. Existing methods may provide a wrong source number due to inferior statistical properties (in low SNR or limited snapshots) or modeling errors (caused by relaxing sparse penalties), especially in impulsive noise. To address this issue, we propose a novel idea of simultaneous source number identification and DOA estimation. We formulate a multiobjective off-grid DOA estimation model to realize this idea, by which the source number can be automatically identified together with DOA estimation. In particular, the source number is properly exploited by the $l_0$ norm of impinging signals without relaxations, guaranteeing accuracy. Furthermore, we design a multiobjective bilevel evolutionary algorithm to solve the proposed model. The source number identification and sparse recovery are simultaneously optimized at the on-grid (lower) level. A forward search strategy is developed to further refine the grid at the off-grid (upper) level. This strategy does not need linear approximations and can eliminate the off-grid gap with low computational complexity. Simulation results demonstrate the outperformance of our method in terms of source number and root mean square error.


title: Coresets for constrained k-median and k-means clustering in low dimensional Euclidean space

abstract: We study (Euclidean) $k$-median and $k$-means with constraints in the streaming model. There have been recent efforts to design unified algorithms to solve constrained $k$-means problems without using knowledge of the specific constraint at hand aside from mild assumptions like the polynomial computability of feasibility under the constraint (compute if a clustering satisfies the constraint) or the presence of an efficient assignment oracle (given a set of centers, produce an optimal assignment of points to the centers which satisfies the constraint). These algorithms have a running time exponential in $k$, but can be applied to a wide range of constraints. We demonstrate that a technique proposed in 2019 for solving a specific constrained streaming $k$-means problem, namely fair $k$-means clustering, actually implies streaming algorithms for all these constraints. These work for low dimensional Euclidean space. [Note that there are more algorithms for streaming fair $k$-means today, in particular they exist for high dimensional spaces now as well.]


title: Classification of Left Invariant Riemannian metrics on Complex hyperbolic space

abstract: It is well known that $\mathbb{C}H^n$ has the structure of solvable Lie group with left invariant metric of constant holomorphic sectional curvature. In this paper we give the full classification of all possible left invariant Riemannian metrics on this Lie group. We prove that all of these metrics are of constant negative scalar curvature and only one of them is Einstein (up to isometry and scaling). Finally, we present the relation between Ricci solitons on Heisenberg group and Einstein metric on $\mathbb{C}H^n$.


title: No Free Lunch: Microservice Practices Reconsidered in Industry

abstract: Microservice architecture advocates a number of technologies and practices such as lightweight container, container orchestration, and DevOps, with the promised benefits of faster delivery, improved scalability, and greater autonomy. However, microservice systems implemented in industry vary a lot in terms of adopted practices and achieved benefits, drastically different from what is advocated in the literature. In this article, we conduct an empirical study, including an online survey with 51 responses and 14 interviews for experienced microservice experts to advance our understanding regarding to microservice practices in industry. As a part of our findings, the empirical study clearly revealed three levels of maturity of microservice systems (from basic to advanced): independent development and deployment, high scalability and availability, and service ecosystem, categorized by the fulfilled benefits of microservices. We also identify 11 practical issues that constrain the microservice capabilities of organizations. For each issue, we summarize the practices that have been explored and adopted in industry, along with the remaining challenges. Our study can help practitioners better position their microservice systems and determine what infrastructures and capabilities are worth investing. Our study can also help researchers better understand industrial microservice practices and identify useful research problems.


title: Phonon-mediated high-temperature superconductivity in ternary borohydride KB$_2$H$_8$ around 12 GPa

abstract: Discovery of high-temperature superconductivity in hydrogen-rich compounds has fuelled the enthusiasm for finding materials with more promising superconducting properties among hydrides. However, the ultrahigh pressure needed to synthesize and maintain high-temperature hydrogen-rich superconductors hinders the experimental investigation of these materials. For practical applications, it is also highly desired to find more hydrogen-rich materials that superconduct at high temperatures but under relatively lower pressures. Based on first-principles density functional theory, we calculate the electronic and phonon band structures for a ternary borohydride formed by intercalating BH$_4$ tetrahedrons into a face-centered-cubic potassium lattice, KB$_2$H$_8$. Remarkably, we find that this material is dynamically stable and one of its $sp^3$-hybridized $\sigma$-bonding bands is metallized (i.e. partially filled) above a moderate high pressure. This metallized $\sigma$-bonding band couples strongly with phonons, giving rise to a strong superconducting pairing potential. By solving the anisotropic Eliashberg equations, we predict that the superconducting transition temperature of this compound is 134-146 K around 12 GPa.


title: Gridless Evolutionary Approach for Line Spectral Estimation with Unknown Model Order

abstract: Gridless methods show great superiority in line spectral estimation. These methods need to solve an atomic $l_0$ norm (i.e., the continuous analog of $l_0$ norm) minimization problem to estimate frequencies and model order. Since this problem is NP-hard to compute, relaxations of atomic $l_0$ norm, such as nuclear norm and reweighted atomic norm, have been employed for promoting sparsity. However, the relaxations give rise to a resolution limit, subsequently leading to biased model order and convergence error. To overcome the above shortcomings of relaxation, we propose a novel idea of simultaneously estimating the frequencies and model order by means of the atomic $l_0$ norm. To accomplish this idea, we build a multiobjective optimization model. The measurment error and the atomic $l_0$ norm are taken as the two optimization objectives. The proposed model directly exploits the model order via the atomic $l_0$ norm, thus breaking the resolution limit. We further design a variable-length evolutionary algorithm to solve the proposed model, which includes two innovations. One is a variable-length coding and search strategy. It flexibly codes and interactively searches diverse solutions with different model orders. These solutions act as steppingstones that help fully exploring the variable and open-ended frequency search space and provide extensive potentials towards the optima. Another innovation is a model order pruning mechanism, which heuristically prunes less contributive frequencies within the solutions, thus significantly enhancing convergence and diversity. Simulation results confirm the superiority of our approach in both frequency estimation and model order selection.


title: Numerical Computations for Bifurcations and Spectral Stability of Solitary Waves in Coupled Nonlinear Schr\"odinger Equations

abstract: We numerically study solitary waves in the coupled nonlinear Schr\"odinger equations. We detect pitchfork bifurcations of the fundamental solitary wave and compute eigenvalues and eigenfunctions of the corresponding eigenvalue problems to determine the spectral stability of solitary waves born at the pitchfork bifurcations. Our numerical results demonstrate the theoretical ones which the authors obtained recently. We also compute generalized eigenfunctions associated with the zero eigenvalue for the bifurcated solitary wave exhibiting a saddle-node bifurcation, and show that it does not change its stability type at the saddle-node bifurcation point.


title: On the influence of rings on orbital velocities

abstract: We analyze the possible effect of rings on orbital velocities in galaxies. The superposition of the central force with the gravitational forces induced by the rings opens up various possibilities of the course of orbital velocities. The orbital velocity depends on the position of the star in the ring. We illustrate this dependence on several models, where we show the course of potential curves and the curves of field strength.


title: qprof: a gprof-inspired quantum profiler

abstract: We introduce qprof, a new and extensible quantum program profiler able to generate profiling reports of various quantum circuits. We describe the internal structure and working of qprof and provide three practical examples on practical quantum circuits with increasing complexity. This tool will allow researchers to visualise their quantum implementation in a different way and reliably localise the bottlenecks for efficient code optimisation.


title: Variational Quanvolutional Neural Networks with enhanced image encoding

abstract: Image classification is an important task in various machine learning applications. In recent years, a number of classification methods based on quantum machine learning and different quantum image encoding techniques have been proposed. In this paper, we study the effect of three different quantum image encoding approaches on the performance of a convolution-inspired hybrid quantum-classical image classification algorithm called quanvolutional neural network (QNN). We furthermore examine the effect of variational - i.e. trainable - quantum circuits on the classification results. Our experiments indicate that some image encodings are better suited for variational circuits. However, our experiments show as well that there is not one best image encoding, but that the choice of the encoding depends on the specific constraints of the application.


title: An energy decomposition theorem for matrices and related questions

abstract: Given $A\subseteq GL_2(\mathbb{F}_q)$, we prove that there exist disjoint subsets $B, C\subseteq A$ such that $A = B \sqcup C$ and their additive and multiplicative energies satisfying \[ \max\{\,E_{+}(B),\, E_{\times}(C)\,\}\ll \frac{|A|^3}{M(|A|)}, \] where \begin{equation*} \label{eqn:MAminBVPolyLSSS} M(|A|) = \min\Bigg\{\,\frac{q^{4/3}}{|A|^{1/3}(\log|A|)^{2/3}},\, \frac{|A|^{4/5}}{q^{13/5}(\log|A|)^{27/10}}\,\Bigg\}. \end{equation*} We also study some related questions on moderate expanders over matrix rings, namely, for $A, B, C\subseteq GL_2(\mathbb{F}_q)$, we have \[|AB+C|, ~|(A+B)C|\gg q^4,\] whenever $|A||B||C|\gg q^{10 + 1/2}$. These improve earlier results due to Karabulut, Koh, Pham, Shen, and Vinh (2019).


title: On-Policy Deep Reinforcement Learning for the Average-Reward Criterion

abstract: We develop theory and algorithms for average-reward on-policy Reinforcement Learning (RL). We first consider bounding the difference of the long-term average reward for two policies. We show that previous work based on the discounted return (Schulman et al., 2015; Achiam et al., 2017) results in a non-meaningful bound in the average-reward setting. By addressing the average-reward criterion directly, we then derive a novel bound which depends on the average divergence between the two policies and Kemeny's constant. Based on this bound, we develop an iterative procedure which produces a sequence of monotonically improved policies for the average reward criterion. This iterative procedure can then be combined with classic DRL (Deep Reinforcement Learning) methods, resulting in practical DRL algorithms that target the long-run average reward criterion. In particular, we demonstrate that Average-Reward TRPO (ATRPO), which adapts the on-policy TRPO algorithm to the average-reward criterion, significantly outperforms TRPO in the most challenging MuJuCo environments.


title: Bi-Lipschitz invariance of planar $BV$- and $W^{1,1}$-extension domains

abstract: We prove that a bi-Lipschitz image of a planar $BV$-extension domain is also a $BV$-extension domain, and that a bi-Lipschitz image of a planar $W^{1,1}$-extension domain is again a $W^{1,1}$-extension domain.


title: Vertex separators, chordality and virtually free groups

abstract: In this paper we consider some results obtained for graphs using minimal vertex separators and generalized chordality and translate them to the context of Geometric Group Theory. Using these new tools, we are able to give two new characterizations for a group to be virtually free. Furthermore, we prove that the Baumslag-Solitar group $BS(1,n)$ is $k$-chordal for some $k$ if and only if $|n|<3$ and we give an application of generalized chordality to the study of the word problem.


title: Leptogenesis, Fermion Masses and Mixings in a SUSY $SU(5)$ GUT with $D_{4}$ Flavor Symmetry

abstract: We propose a model of fermion masses and mixings based on $SU(5)$ grand unified theory (GUT) and a $D_{4}$ flavor symmetry. This is a highly predictive 4D $SU(5)$ GUT with a flavor symmetry that does not contain a triplet irreducible representation. The Yukawa matrices of quarks and charged leptons are obtained after integrating out heavy messenger fields from renormalizable superpotentials while neutrino masses originate from the type I seesaw mechanism. The group theoretical factors from 24- and 45-dimensional Higgs fields lead to ratios between the Yukawa couplings in agreement with data, while the dangerous proton decay operators are highly suppressed. By performing a numerical fit, we find that the model captures accurately the mixing angles, the Yukawa couplings and the $CP$ phase of the quark sector at the GUT scale. The neutrino masses are generated at the renormalizable level with the prediction of trimaximal mixing while an additional effective operator is required to account for the baryon asymmetry of the universe (BAU). The model is remarkably predictive because only the normal neutrino mass ordering and the lower octant of the atmospheric angle are allowed while the $CP$ conserving values of the Dirac neutrino phase $\delta_{CP}$ are excluded. Moreover, the predicted values of the effective Majorana mass $m_{\beta \beta}$ can be tested at future neutrinoless double beta decay experiments. An analytical and a numerical study of the BAU via the leptogenesis mechanism is performed. We focused on the regions of parameter space where leptogenesis from the lightest right-handed neutrino is successfully realized. Strong correlations between the parameters of the neutrino sector and the observed BAU are obtained.


title: Deep Transfer Learning for Brain Magnetic Resonance Image Multi-class Classification

abstract: Magnetic Resonance Imaging (MRI) is a principal diagnostic approach used in the field of radiology to create images of the anatomical and physiological structure of patients. MRI is the prevalent medical imaging practice to find abnormalities in soft tissues. Traditionally they are analyzed by a radiologist to detect abnormalities in soft tissues, especially the brain. The process of interpreting a massive volume of patient's MRI is laborious. Hence, the use of Machine Learning methodologies can aid in detecting abnormalities in soft tissues with considerable accuracy. In this research, we have curated a novel dataset and developed a framework that uses Deep Transfer Learning to perform a multi-classification of tumors in the brain MRI images. In this paper, we adopted the Deep Residual Convolutional Neural Network (ResNet50) architecture for the experiments along with discriminative learning techniques to train the model. Using the novel dataset and two publicly available MRI brain datasets, this proposed approach attained a classification accuracy of 86.40% on the curated dataset, 93.80% on the Harvard Whole Brain Atlas dataset, and 97.05% accuracy on the School of Biomedical Engineering dataset. Results of our experiments significantly demonstrate our proposed framework for transfer learning is a potential and effective method for brain tumor multi-classification tasks.


title: Accretion-modified Stars in Accretion Disks of Active Galactic Nuclei: Gravitational Wave Bursts and Electromagnetic Counterparts from Merging Stellar Black Hole Binaries

abstract: The recent advanced LIGO/Virgo detections of gravitational waves (GWs) from stellar binary black hole (BBH) mergers, in particular GW190521, which is potentially associated with a quasar, have stimulated renewed interest in active galactic nuclei (AGNs) as factories of merging BBHs. Compact objects evolving from massive stars are unavoidably enshrouded by a massive envelope to form accretion-modified stars (AMSs) in the dense gaseous environment of a supermassive black hole (SMBH) accretion disk. We show that most AMSs form binaries due to gravitational interaction with each other during radial migration in the SMBH disk, forming BBHs inside the AMS. When a BBH is born, its orbit is initially governed by the tidal torque of the SMBH. Bondi accretion onto BBH at a hyper-Eddington rate naturally develops and then controls the evolution of its orbits. We find that Bondi accretion leads to efficient removal of orbital angular momentum of the binary, whose final merger produces a GW burst. Meanwhile, the Blandford-Znajek mechanism pumps the spin energy of the merged BH to produce an electromagnetic counterpart (EMC). Moreover, hyper-Eddington accretion onto the BBH develops powerful outflows and triggers a Bondi explosion, which manifests itself as a EMC of the GW burst, depending on the viscosity of the accretion flow. Thermal emission from Bondi sphere appears as one of EMCs. BBHs radiate GWs with frequencies $\sim 10^{2}\,$Hz, which are accessible to LIGO.


title: Quantum Kibble-Zurek mechanism: Kink correlations after a quench in the quantum Ising chain

abstract: The transverse field in the quantum Ising chain is linearly ramped from the para- to the ferromagnetic phase across the quantum critical point at a rate characterized by a quench time $\tau_Q$. We calculate a connected kink-kink correlator in the final state at zero transverse field. The correlator is a sum of two terms: a negative (anti-bunching) Gaussian that depends on the Kibble-Zurek (KZ) correlation length only and a positive term that depends on a second longer scale of length. The second length is made longer by dephasing of the state excited near the critical point during the following ramp across the ferromagnetic phase. This interpretation is corroborated by considering a linear ramp that is halted in the ferromagnetic phase for a finite waiting time and then continued at the same rate as before the halt. The extra time available for dephasing increases the second scale of length that asymptotically grows linearly with the waiting time. The dephasing also suppresses magnitude of the second term making it negligible for waiting times much longer than $\tau_Q$. The same dephasing can be obtained with a smooth ramp that slows down in the ferromagnetic phase. Assuming sufficient dephasing we obtain also higher order kink correlators and the ferromagnetic correlation function.


title: Entropy-Based Proofs of Combinatorial Results on Bipartite Graphs

abstract: This work considers new entropy-based proofs of some known, or otherwise refined, combinatorial bounds for bipartite graphs. These include upper bounds on the number of the independent sets, lower bounds on the minimal number of colors in constrained edge coloring, and lower bounds on the number of walks of a given length in bipartite graphs. The proofs of these combinatorial results rely on basic properties of the Shannon entropy.


title: Speech Disorder Classification Using Extended Factorized Hierarchical Variational Auto-encoders

abstract: Objective speech disorder classification for speakers with communication difficulty is desirable for diagnosis and administering therapy. With the current state of speech technology, it is evident to propose neural networks for this application. But neural network model training is hampered by a lack of labeled disordered speech data. In this research, we apply an extended version of Factorized Hierarchical Variational Auto-encoders (FHVAE) for representation learning on disordered speech. The FHVAE model extracts both content-related and sequence-related latent variables from speech data, and we utilize the extracted variables to explore how disorder type information is represented in the latent variables. For better classification performance, the latent variables are aggregated at the word and sentence level. We show that an extension of the FHVAE model succeeds in the better disentanglement of the content-related and sequence-related related representations, but both representations are still required for best results on disorder type classification.


title: Neighborhood Rough Set based Multi-document Summarization

abstract: This research paper proposes a novel Neighbourhood Rough Set based approach for supervised Multi-document Text Summarization (MDTS) with analysis and impact on the summarization results for MDTS. Here, Rough Set based LERS algorithm is improved using Neighborhood Rough Set which is itself a novel combination called Neighborhood-LERS to be experimented for evaluations of efficacy and efficiency. In this paper, we shall apply and evaluate the proposed Neighborhood-LERS for Multi-document Summarization which here is proved experimentally to be superior to the base LERS technique for MDTS.


title: Kubo's response theory and bosonization with a background gauge field and irrelevant perturbations

abstract: Using conformal field theory calculations of the energy spectrum, within the XXZ model we investigate effects of the flux insertion and the Umklapp term. We discuss two approaches to the evaluation of the Drude weight, the first corresponding to the linear response theory and the second corresponding to the twisted boson theory with the Umklapp term. Divergences obtained in the context of the former contradict the Bethe ansatz results, with the two approaches coinciding for the free fermion point only. The origin of this discrepancy is in the different order in which the Umklapp term and the flux insertion are treated, where the marginal perturbation should be considered before the irrelevant. We calculate the scaling of the conductivity with system size and temperature in the long-wave limit.

categories: cond-mat.stat-mech cond-mat.str-el hep-th math-ph math.MP


title: MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education

abstract: Since the introduction of the original BERT (i.e., BASE BERT), researchers have developed various customized BERT models with improved performance for specific domains and tasks by exploiting the benefits of transfer learning. Due to the nature of mathematical texts, which often use domain specific vocabulary along with equations and math symbols, we posit that the development of a new BERT model for mathematics would be useful for many mathematical downstream tasks. In this resource paper, we introduce our multi-institutional effort (i.e., two learning platforms and three academic institutions in the US) toward this need: MathBERT, a model created by pre-training the BASE BERT model on a large mathematical corpus ranging from pre-kindergarten (pre-k), to high-school, to college graduate level mathematical content. In addition, we select three general NLP tasks that are often used in mathematics education: prediction of knowledge component, auto-grading open-ended Q&A, and knowledge tracing, to demonstrate the superiority of MathBERT over BASE BERT. Our experiments show that MathBERT outperforms prior best methods by 1.2-22% and BASE BERT by 2-8% on these tasks. In addition, we build a mathematics specific vocabulary 'mathVocab' to train with MathBERT. We discover that MathBERT pre-trained with 'mathVocab' outperforms MathBERT trained with the BASE BERT vocabulary (i.e., 'origVocab'). MathBERT is currently being adopted at the participated leaning platforms: Stride, Inc, a commercial educational resource provider, and ASSISTments.org, a free online educational platform. We release MathBERT for public usage at: https://github.com/tbs17/MathBERT.


title: i-Pulse: A NLP based novel approach for employee engagement in logistics organization

abstract: Although most logistics and freight forwarding organizations, in one way or another, claim to have core values. The engagement of employees is a vast structure that affects almost every part of the company's core environmental values. There is little theoretical knowledge about the relationship between firms and the engagement of employees. Based on research literature, this paper aims to provide a novel approach for insight around employee engagement in a logistics organization by implementing deep natural language processing concepts. The artificial intelligence-enabled solution named Intelligent Pulse (I-Pulse) can evaluate hundreds and thousands of pulse survey comments and provides the actionable insights and gist of employee feedback. I-Pulse allows the stakeholders to think in new ways in their organization, helping them to have a powerful influence on employee engagement, retention, and efficiency. This study is of corresponding interest to researchers and practitioners.


title: On the question of the classical analogue of the Fano problem

abstract: A system of interacting classical oscillators is discussed, similar to a quantum mechanical system of a discrete energy level, interacting with the energy quasi-continuum of states considered Fano. The limit of a continuous spectrum is analyzed together with the possible connection of the problem under study with the generation of coherent phonons.


title: Learning to Bridge Metric Spaces: Few-shot Joint Learning of Intent Detection and Slot Filling

abstract: In this paper, we investigate few-shot joint learning for dialogue language understanding. Most existing few-shot models learn a single task each time with only a few examples. However, dialogue language understanding contains two closely related tasks, i.e., intent detection and slot filling, and often benefits from jointly learning the two tasks. This calls for new few-shot learning techniques that are able to capture task relations from only a few examples and jointly learn multiple tasks. To achieve this, we propose a similarity-based few-shot learning scheme, named Contrastive Prototype Merging network (ConProm), that learns to bridge metric spaces of intent and slot on data-rich domains, and then adapt the bridged metric space to the specific few-shot domain. Experiments on two public datasets, Snips and FewJoint, show that our model significantly outperforms the strong baselines in one and five shots settings.


title: Understanding Information Spreading Mechanisms During COVID-19 Pandemic by Analyzing the Impact of Tweet Text and User Features for Retweet Prediction

abstract: COVID-19 has affected the world economy and the daily life routine of almost everyone. It has been a hot topic on social media platforms such as Twitter, Facebook, etc. These social media platforms enable users to share information with other users who can reshare this information, thus causing this information to spread. Twitter's retweet functionality allows users to share the existing content with other users without altering the original content. Analysis of social media platforms can help in detecting emergencies during pandemics that lead to taking preventive measures. One such type of analysis is predicting the number of retweets for a given COVID-19 related tweet. Recently, CIKM organized a retweet prediction challenge for COVID-19 tweets focusing on using numeric features only. However, our hypothesis is, tweet text may play a vital role in an accurate retweet prediction. In this paper, we combine numeric and text features for COVID-19 related retweet predictions. For this purpose, we propose two CNN and RNN based models and evaluate the performance of these models on a publicly available TweetsCOV19 dataset using seven different evaluation metrics. Our evaluation results show that combining tweet text with numeric features improves the performance of retweet prediction significantly.


title: Self-Guided Contrastive Learning for BERT Sentence Representations

abstract: Although BERT and its variants have reshaped the NLP landscape, it still remains unclear how best to derive sentence embeddings from such pre-trained Transformers. In this work, we propose a contrastive learning method that utilizes self-guidance for improving the quality of BERT sentence representations. Our method fine-tunes BERT in a self-supervised fashion, does not rely on data augmentation, and enables the usual [CLS] token embeddings to function as sentence vectors. Moreover, we redesign the contrastive learning objective (NT-Xent) and apply it to sentence representation learning. We demonstrate with extensive experiments that our approach is more effective than competitive baselines on diverse sentence-related tasks. We also show it is efficient at inference and robust to domain shifts.


title: A Query-Driven Topic Model

abstract: Topic modeling is an unsupervised method for revealing the hidden semantic structure of a corpus. It has been increasingly widely adopted as a tool in the social sciences, including political science, digital humanities and sociological research in general. One desirable property of topic models is to allow users to find topics describing a specific aspect of the corpus. A possible solution is to incorporate domain-specific knowledge into topic modeling, but this requires a specification from domain experts. We propose a novel query-driven topic model that allows users to specify a simple query in words or phrases and return query-related topics, thus avoiding tedious work from domain experts. Our proposed approach is particularly attractive when the user-specified query has a low occurrence in a text corpus, making it difficult for traditional topic models built on word cooccurrence patterns to identify relevant topics. Experimental results demonstrate the effectiveness of our model in comparison with both classical topic models and neural topic models.


title: Zipf Matrix Factorization : Matrix Factorization with Matthew Effect Reduction

abstract: Recommender system recommends interesting items to users based on users' past information history. Researchers have been paying attention to improvement of algorithmic performance such as MAE and precision@K. Major techniques such as matrix factorization and learning to rank are optimized based on such evaluation metrics. However, the intrinsic Matthew Effect problem poses great threat to the fairness of the recommender system, and the unfairness problem cannot be resolved by optimization of traditional metrics. In this paper, we propose a novel algorithm that incorporates Matthew Effect reduction with the matrix factorization framework. We demonstrate that our approach can boost the fairness of the algorithm and enhances performance evaluated by traditional metrics.


title: Is it a click bait? Let's predict using Machine Learning

abstract: In this era of digitisation, news reader tend to read news online. This is because, online media instantly provides access to a wide variety of content. Thus, people don't have to wait for tomorrow's newspaper to know what's happening today. Along with these virtues, online news have some vices as well. One such vice is presence of social media posts (tweets) relating to news articles whose sole purpose is to draw attention of the users rather than directing them to read the actual content. Such posts are referred to as clickbaits. The objective of this project is to develop a system which would be capable of predicting how likely are the social media posts (tweets) relating to new articles tend to be clickbait.


title: Using Integrated Gradients and Constituency Parse Trees to explain Linguistic Acceptability learnt by BERT

abstract: Linguistic Acceptability is the task of determining whether a sentence is grammatical or ungrammatical. It has applications in several use cases like Question-Answering, Natural Language Generation, Neural Machine Translation, where grammatical correctness is crucial. In this paper we aim to understand the decision-making process of BERT (Devlin et al., 2019) in distinguishing between Linguistically Acceptable sentences (LA) and Linguistically Unacceptable sentences (LUA). We leverage Layer Integrated Gradients Attribution Scores (LIG) to explain the Linguistic Acceptability criteria that are learnt by BERT on the Corpus of Linguistic Acceptability (CoLA) (Warstadt et al., 2018) benchmark dataset. Our experiments on 5 categories of sentences lead to the following interesting findings: 1) LIG for LA are significantly smaller in comparison to LUA, 2) There are specific subtrees of the Constituency Parse Tree (CPT) for LA and LUA which contribute larger LIG, 3) Across the different categories of sentences we observed around 88% to 100% of the Correctly classified sentences had positive LIG, indicating a strong positive relationship to the prediction confidence of the model, and 4) Around 43% of the Misclassified sentences had negative LIG, which we believe can become correctly classified sentences if the LIG are parameterized in the loss function of the model.


title: THG: Transformer with Hyperbolic Geometry

abstract: Transformer model architectures have become an indispensable staple in deep learning lately for their effectiveness across a range of tasks. Recently, a surge of "X-former" models have been proposed which improve upon the original Transformer architecture. However, most of these variants make changes only around the quadratic time and memory complexity of self-attention, i.e. the dot product between the query and the key. What's more, they are calculate solely in Euclidean space. In this work, we propose a novel Transformer with Hyperbolic Geometry (THG) model, which take the advantage of both Euclidean space and Hyperbolic space. THG makes improvements in linear transformations of self-attention, which are applied on the input sequence to get the query and the key, with the proposed hyperbolic linear. Extensive experiments on sequence labeling task, machine reading comprehension task and classification task demonstrate the effectiveness and generalizability of our model. It also demonstrates THG could alleviate overfitting.


title: From Single Lane to Highways: Analyzing the Adoption of Multipath TCP in the Internet

abstract: Multipath TCP (MPTCP) extends traditional TCP to enable simultaneous use of multiple connection endpoints at the source and destination. MPTCP has been under active development since its standardization in 2013, and more recently in February 2020, MPTCP was upstreamed to the Linux kernel. In this paper, we provide the first broad analysis of MPTCPv0 in the Internet. We probe the entire IPv4 address space and an IPv6 hitlist to detect MPTCP-enabled systems operational on port 80 and 443. Our scans reveal a steady increase in MPTCP-capable IPs, reaching 9k+ on IPv4 and a few dozen on IPv6. We also discover a significant share of seemingly MPTCP-capable hosts, an artifact of middleboxes mirroring TCP options. We conduct targeted HTTP(S) measurements towards select hosts and find that middleboxes can aggressively impact the perceived quality of applications utilizing MPTCP. Finally, we analyze two complementary traffic traces from CAIDA and MAWI to shed light on the real-world usage of MPTCP. We find that while MPTCP usage has increased by a factor of 20 over the past few years, its traffic share is still quite low.


title: MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network

abstract: We present an instance-based nearest neighbor approach to entity linking. In contrast to most prior entity retrieval systems which represent each entity with a single vector, we build a contextualized mention-encoder that learns to place similar mentions of the same entity closer in vector space than mentions of different entities. This approach allows all mentions of an entity to serve as "class prototypes" as inference involves retrieving from the full set of labeled entity mentions in the training set and applying the nearest mention neighbor's entity label. Our model is trained on a large multilingual corpus of mention pairs derived from Wikipedia hyperlinks, and performs nearest neighbor inference on an index of 700 million mentions. It is simpler to train, gives more interpretable predictions, and outperforms all other systems on two multilingual entity linking benchmarks.


title: Posthoc Verification and the Fallibility of the Ground Truth

abstract: Classifiers commonly make use of pre-annotated datasets, wherein a model is evaluated by pre-defined metrics on a held-out test set typically made of human-annotated labels. Metrics used in these evaluations are tied to the availability of well-defined ground truth labels, and these metrics typically do not allow for inexact matches. These noisy ground truth labels and strict evaluation metrics may compromise the validity and realism of evaluation results. In the present work, we discuss these concerns and conduct a systematic posthoc verification experiment on the entity linking (EL) task. Unlike traditional methodologies, which asks annotators to provide free-form annotations, we ask annotators to verify the correctness of annotations after the fact (i.e., posthoc). Compared to pre-annotation evaluation, state-of-the-art EL models performed extremely well according to the posthoc evaluation methodology. Posthoc validation also permits the validation of the ground truth dataset. Surprisingly, we find predictions from EL models had a similar or higher verification rate than the ground truth. We conclude with a discussion on these findings and recommendations for future evaluations.


title: The relationship between the US broad money supply and US GDP for the time period 2001 to 2019 with that of the corresponding time series for US national property and stock market indices, using an information entropy methodology

abstract: The primary objective of this paper was to investigate whether the growth in the major US asset indices could be a function of the US broad money supply and/or US GDP, over the time period 2001 to 2019, using an information entropy methodology. The four US asset indices investigated were: (1) US National Property index; (2) Russell 2000 index; (3) S&P 500 index; and (4) NASDAQ index. Notwithstanding the financial crisis of 2007-2008, US real GDP increased exponentially over the period 2001 to 2019, with an average annual growth rate of approximately 2%. However, over this time period, the average annual rate of growth of US GDP was considerably lower than the average annual rate of growth of the US broad money supply (5.7%). The main determinant of the average growth rate for all four US asset indices studied would appear to be the growth rate in the US broad money supply. In addition, the growth rate in the US Russell 2000 stock index and the NASDAQ index would appear to be a function of the combined positive effects of both the growth rate in the US Broad Money Supply and the growth rate of US GDP.


title: Predicting the imagined contents using brain activation

abstract: Mental imagery refers to percept-like experiences in the absence of sensory input. Brain imaging studies suggest common, modality-specific, neural correlates imagery and perception. We associated abstract visual stimuli with either visually presented or imagined monetary rewards and scrambled pictures. Brain images for a group of 12 participants were collected using functional magnetic resonance imaging. Statistical analysis showed that human midbrain regions were activated irrespective of the monetary rewards being imagined or visually present. A support vector machine trained on the midbrain activation patterns to the visually presented rewards predicted with 75% accuracy whether the participants imagined the monetary reward or the scrambled picture during imagination trials. Training samples were drawn from visually presented trials and classification accuracy was assessed for imagination trials. These results suggest the use of machine learning technique for classification of underlying cognitive states from brain imaging data.


title: Mixture of Virtual-Kernel Experts for Multi-Objective User Profile Modeling

abstract: In industrial applications like online advertising and recommendation systems, diverse and accurate user profiles can greatly help improve personalization. Deep learning is widely applied to mine expressive tags to users from their historical interactions in the system, e.g., click, conversion action in the advertising chain. The usual approach is to take a certain action as the objective, and introduce multiple independent Two-Tower models to predict the possibility of users' action on tags (known as CTR or CVR prediction). The predicted users' high probably attractive tags are to represent their preferences. However, the single-action models cannot learn complementarily and support effective training on data-sparse actions. Besides, limited by the lack of information fusion between the two towers, the model learns insufficiently to represent users' preferences on various tag \textbf{topics} well. This paper introduces a novel multi-task model called Mixture of Virtual-Kernel Experts (MVKE) to learn user preferences on various actions and topics unitedly. In MVKE, we propose a concept of Virtual-Kernel Expert, which focuses on modeling one particular facet of the user's preferences, and all of them learn coordinately. Besides, the gate-based structure used in MVKE builds an information fusion bridge between two towers, improving the model's capability and maintaining high efficiency. We apply the model in Tencent Advertising System, where both online and offline evaluations show that our method has a significant improvement compared with the existing ones and brings about an obvious lift to actual advertising revenue.


title: Conforming and Nonconforming Finite Element Methods for Biharmonic Inverse Source Problem

abstract: This paper deals with the numerical approximation of the biharmonic inverse source problem in an abstract setting in which the measurement data is finite-dimensional. This unified framework in particular covers the conforming and nonconforming finite element methods (FEMs). The inverse problem is analysed through the forward problem. Error estimate for the forward solution is derived in an abstract set-up that applies to conforming and Morley nonconforming FEMs. Since the inverse problem is ill-posed, Tikhonov regularisation is considered to obtain a stable approximate solution. Error estimate is established for the regularised solution for different regularisation schemes. Numerical results that confirm the theoretical results are also presented.


title: Credit spread approximation and improvement using random forest regression

abstract: Credit Default Swap (CDS) levels provide a market appreciation of companies' default risk. These derivatives are not always available, creating a need for CDS approximations. This paper offers a simple, global and transparent CDS structural approximation, which contrasts with more complex and proprietary approximations currently in use. This Equity-to-Credit formula (E2C), inspired by CreditGrades, obtains better CDS approximations, according to empirical analyses based on a large sample spanning 2016-2018. A random forest regression run with this E2C formula and selected additional financial data results in an 87.3% out-of-sample accuracy in CDS approximations. The transparency property of this algorithm confirms the predominance of the E2C estimate, and the impact of companies' debt rating and size, in predicting their CDS.


title: MexPub: Deep Transfer Learning for Metadata Extraction from German Publications

abstract: Extracting metadata from scientific papers can be considered a solved problem in NLP due to the high accuracy of state-of-the-art methods. However, this does not apply to German scientific publications, which have a variety of styles and layouts. In contrast to most of the English scientific publications that follow standard and simple layouts, the order, content, position and size of metadata in German publications vary greatly among publications. This variety makes traditional NLP methods fail to accurately extract metadata from these publications. In this paper, we present a method that extracts metadata from PDF documents with different layouts and styles by viewing the document as an image. We used Mask R-CNN that is trained on COCO dataset and finetuned with PubLayNet dataset that consists of ~200K PDF snapshots with five basic classes (e.g. text, figure, etc). We refine-tuned the model on our proposed synthetic dataset consisting of ~30K article snapshots to extract nine patterns (i.e. author, title, etc). Our synthetic dataset is generated using contents in both languages German and English and a finite set of challenging templates obtained from German publications. Our method achieved an average accuracy of around $90\%$ which validates its capability to accurately extract metadata from a variety of PDF documents with challenging templates.


title: Low-Rank Projections of GCNs Laplacian

abstract: In this work, we study the behavior of standard models for community detection under spectral manipulations. Through various ablation experiments, we evaluate the impact of bandpass filtering on the performance of a GCN: we empirically show that most of the necessary and used information for nodes classification is contained in the low-frequency domain, and thus contrary to images, high frequencies are less crucial to community detection. In particular, it is sometimes possible to obtain accuracies at a state-of-the-art level with simple classifiers that rely only on a few low frequencies.


title: Probabilistic Forecasting of Imbalance Prices in the Belgian Context

abstract: Forecasting imbalance prices is essential for strategic participation in the short-term energy markets. A novel two-step probabilistic approach is proposed, with a particular focus on the Belgian case. The first step consists of computing the net regulation volume state transition probabilities. It is modeled as a matrix computed using historical data. This matrix is then used to infer the imbalance prices since the net regulation volume can be related to the level of reserves activated and the corresponding marginal prices for each activation level are published by the Belgian Transmission System Operator one day before electricity delivery. This approach is compared to a deterministic model, a multi-layer perceptron, and a widely used probabilistic technique, Gaussian Processes.


title: A Numerical Approach to Pricing Exchange Options under Stochastic Volatility and Jump-Diffusion Dynamics

abstract: We consider a method of lines (MOL) approach to determine prices of European and American exchange options when underlying asset prices are modelled with stochastic volatility and jump-diffusion dynamics. As the MOL, as with any other numerical scheme for PDEs, becomes increasingly complex when higher dimensions are involved, we first simplify the problem by transforming the exchange option into a call option written on the ratio of the yield processes of the two assets. This is achieved by taking the second asset yield process as the numeraire. We also characterize the near-maturity behavior of the early exercise boundary of the American exchange option and analyze how model parameters affect this behavior. Using the MOL scheme, we conduct a numerical comparative static analysis of exchange option prices with respect to the model parameters and investigate the impact of stochastic volatility and jumps to option prices. We also consider the effect of boundary conditions at far-but-finite limits of the computational domain on the overall efficiency of the MOL scheme. Toward these objectives, a brief exposition of the MOL and how it can be implemented on computing software are provided.


title: Cognitive-aware Short-text Understanding for Inferring Professions

abstract: Leveraging short-text contents to estimate the occupation of microblog authors has significant gains in many applications. Yet challenges abound. Firstly brief textual contents come with excessive lexical noise that makes the inference problem challenging. Secondly, cognitive-semantics are not evident, and important linguistic features are latent in short-text contents. Thirdly, it is hard to measure the correlation between the cognitive short-text semantics and the features pertaining various occupations. We argue that the multi-aspect cognitive features are needed to correctly associate short-text contents to a particular job and discover suitable people for the careers. To this end, we devise a novel framework that on the one hand, can infer short-text contents and exploit cognitive features, and on the other hand, fuses various adopted novel algorithms, such as curve fitting, support vector, and boosting modules to better predict the occupation of the authors. The final estimation module manufactures the $R^w$-tree via coherence weight to tune the best outcome in the inferring process. We conduct comprehensive experiments on real-life Twitter data. The experimental results show that compared to other rivals, our cognitive multi-aspect model can achieve a higher performance in the career estimation procedure, where it is inevitable to neglect the contextual semantics of users.


title: Meaning Representation of Numeric Fused-Heads in UCCA

abstract: We exhibit that the implicit UCCA parser does not address numeric fused-heads (NFHs) consistently, which could result either from inconsistent annotation, insufficient training data or a modelling limitation. and show which factors are involved. We consider this phenomenon important, as it is pervasive in text and critical for correct inference. Careful design and fine-grained annotation of NFHs in meaning representation frameworks would benefit downstream tasks such as machine translation, natural language inference and question answering, particularly when they require numeric reasoning, as recovering and categorizing them. We are investigating the treatment of this phenomenon by other meaning representations, such as AMR. We encourage researchers in meaning representations, and computational linguistics in general, to address this phenomenon in future research.


title: Spectral emission dependence of tin-vacancy centers in diamond from thermal processing and chemical functionalization

abstract: We report a systematic photoluminescence (PL) investigation of the spectral emission properties of individual optical defects fabricated in diamond upon ion implantation and annealing. Three spectral lines at 620 nm, 631 nm, and 647 nm are identified and attributed to the SnV center due to their occurrence in the PL spectra of the very same single-photon emitting defects. We show that the relative occurrence of the three spectral features can be modified by oxidizing the sample surface following thermal annealing. We finally report the relevant emission properties of each class of individual emitters, including the excited state emission lifetime and the emission intensity saturation parameters.


title: Synthesis and thermal stability of TMD thin films: A comprehensive XPS and Raman study

abstract: Transition metal dichalcogenides (TMDs) have been a core constituent of 2D material research throughout the last decade. Over this time, research focus has progressively shifted from synthesis and fundamental investigations, to exploring their properties for applied research such as electrochemical applications and integration in electrical devices. Due to the rapid pace of development, priority is often given to application-oriented aspects while careful characterisation and analysis of the TMD materials themselves is occasionally neglected. This can be particularly evident for characterisations involving X-ray photoelectron spectroscopy (XPS), where measurement, peak-fitting, and analysis can be complex and nuanced endeavours requiring specific expertise. To improve the availability and accessibility of reference information, here we present a detailed peak-fitted XPS analysis of ten transition metal chalcogenides. The materials were synthesised as large-area thin-films on SiO2 using direct chalcogenisation of pre-deposited metal films. Alongside XPS, the Raman spectra with several excitation wavelengths for each material are also provided. These complementary characterisation methods can provide a more complete understanding of the composition and quality of the material. As material stability is a crucial factor when considering applications, the in-air thermal stability of the TMDs was investigated after several annealing points up to 400 {\deg}C. This delivers a trend of evolving XPS and Raman spectra for each material which improves interpretation of their spectra while also indicating their ambient thermal limits. This provides an accessible library and set of guidelines to characterise, compare, and discuss TMD XPS and Raman spectra.


title: Design of a prototype laser-plasma injector for the DESY-II synchrotron

abstract: The present state of progress in laser wakefield acceleration encourages considering it as a practical alternative to conventional particle accelerators. A promising application would be to use a laser-plasma accelerator as an injector for a synchrotron light source. Yet, the energy spread and jitter of the laser-plasma beam pose a significant difficulty for an efficient injection. In this paper we propose a design of a prototype injector to deliver 500 MeV low-intensity electron bunches to the DESY-II electron synchrotron. The design utilizes presently available conventional accelerator technology, such as a chicane and an X-band radio frequency cavity, to reduce the energy spread and jitter of the electron beam down to a sub-per-mille level.


title: Quality-Aware Network for Face Parsing

abstract: This is a very short technical report, which introduces the solution of the Team BUPT-CASIA for Short-video Face Parsing Track of The 3rd Person in Context (PIC) Workshop and Challenge at CVPR 2021. Face parsing has recently attracted increasing interest due to its numerous application potentials. Generally speaking, it has a lot in common with human parsing, such as task setting, data characteristics, number of categories and so on. Therefore, this work applies state-of-the-art human parsing method to face parsing task to explore the similarities and differences between them. Our submission achieves 86.84% score and wins the 2nd place in the challenge.


title: A Self-Supervised Framework for Function Learning and Extrapolation

abstract: Understanding how agents learn to generalize -- and, in particular, to extrapolate -- in high-dimensional, naturalistic environments remains a challenge for both machine learning and the study of biological agents. One approach to this has been the use of function learning paradigms, which allow peoples' empirical patterns of generalization for smooth scalar functions to be described precisely. However, to date, such work has not succeeded in identifying mechanisms that acquire the kinds of general purpose representations over which function learning can operate to exhibit the patterns of generalization observed in human empirical studies. Here, we present a framework for how a learner may acquire such representations, that then support generalization -- and extrapolation in particular -- in a few-shot fashion. Taking inspiration from a classic theory of visual processing, we construct a self-supervised encoder that implements the basic inductive bias of invariance under topological distortions. We show the resulting representations outperform those from other models for unsupervised time series learning in several downstream function learning tasks, including extrapolation.


title: Ergodicity breaking in an incommensurate system observed by OTOCs and Loschmidt Echoes: From quantum diffusion to sub-diffusion

abstract: The metal-insulator transition (MIT), which includes Anderson localization and Mott insulators as extreme regimes, has received renewed interest as the many-body effects often constitute a limitation for the handling of quantum interference. This resulted in the field dubbed many-body localization (MBL), intensively studied theoretically and experimentally as understanding the appearance of equilibration and thermalization becomes relevant in dealing with finite systems. Here, we propose a new observable to study this transition in a spin chain under the ``disorder'' of a Harper-Hofstadter-Aubry-Andr\'e on-site potential. This quantity, which we call zeroth-order gradient entanglement (ZOGE) is extracted from the fundamental Fourier mode of a family of out-of-time-ordered correlators (OTOCs). These are just Loschmidt Echoes, where a field gradient is applied before the time reversal. In the absence of many-body interactions, the ZOGE coincides with the inverse participation ratio of a Fermionic wave function. By adding an Ising interaction to an XY Hamiltonian, one can explore the MBL phase diagram of the system. Close to the critical region, the excitation dynamics is consistent with a diffusion law. However, for weak disorder, quantum diffusion prevails while for strong disorder the excitation dynamics is sub-diffusive.