title: Annihilation of Magnetic Islands at the Top of Solar Flare Loops

abstract: The dynamics of magnetic reconnection in the solar current sheet (CS) is studied by high-resolution 2.5-dimensional MHD simulation. With the commence of magnetic reconnection, a number of magnetic islands are formed intermittently and move quickly upward and downward along the CS. When colliding with the semi-closed flux of flare loops, the downflow islands cause a second reconnection with a rate even comparable with that in the main CS. Though the time-integrated magnetic energy release is still dominated by the reconnection in main CS, the second reconnection can release substantial magnetic energy, annihilating the main islands and generating secondary islands with various scales at the flare loop top. The distribution function of the flux of the second islands is found to follow a power-law varying from $f\left(\psi\right)\sim\psi^{-1}$ (small scale) to $\psi^{-2}$ (large scale), which seems to be independent with background plasma $\beta$ and if including thermal conduction. However, the spatial scale and the strength of the termination shocks driven by main reconnection outflows or islands decrease if $\beta$ increases or thermal conduction is included. We suggest that the annihilation of magnetic islands at the flare loop top, which is not included in the standard flare model, plays a non-negligible role in releasing magnetic energy to heat flare plasma and accelerate particles.


title: An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models

abstract: Recent work has shown pre-trained language models capture social biases from the large amounts of text they are trained on. This has attracted attention to developing techniques that mitigate such biases. In this work, we perform an empirical survey of five recently proposed bias mitigation techniques: Counterfactual Data Augmentation (CDA), Dropout, Iterative Nullspace Projection, Self-Debias, and SentenceDebias. We quantify the effectiveness of each technique using three intrinsic bias benchmarks while also measuring the impact of these techniques on a model's language modeling ability, as well as its performance on downstream NLU tasks. We experimentally find that: (1) Self-Debias is the strongest debiasing technique, obtaining improved scores on all bias benchmarks; (2) Current debiasing techniques perform less consistently when mitigating non-gender biases; And (3) improvements on bias benchmarks such as StereoSet and CrowS-Pairs by using debiasing strategies are often accompanied by a decrease in language modeling ability, making it difficult to determine whether the bias mitigation was effective.


title: Generalized estimates for thermal expansion of oxide scale in the range from 0C to 1300C with account for movability of phase transitions in its components

abstract: The thermophysical properties of oxide scale, in the general case, are affected by the variation of the temperature of phase transitions (either magnetic or polymorphic) in its components due to impurities, lattice defects, grain sizes, etc. In this case, since the phase transition is usually accompanied by a sharp change in properties, even a small shift of such a critical temperature can lead to large changes in properties in its vicinity. In order to account for this effect, data known from various sources on the true coefficient of linear thermal expansion (CLTE) of w\"ustite , magnetite , hematite , and metallic iron are generalized by approximating functions that include critical temperatures as variable parameters. It is shown that the true CLTE of magnetite at the same rated temperature can differ by up to 30% depending on the position of the Curie point within the limits of its possible "movability". The proposed methods allow to take into account the critical temperatures as adaptation parameters in engineering calculations of thermal expansion of oxide scale. Generalized formulas for each scale component are also given in a particular form for fixed (basic) values of critical temperatures. The dependence of the true CLTE of oxide scale as a whole ({\alpha}sc) on the volume fraction of each component is proposed. It is shown by model computations that there are temperatures at which {\alpha}sc is almost independent of the scale composition, as well as areas of instability, where {\alpha}sc depends significantly on the percentage of components. The results of the work are recommended to be used when mathematical modeling of production and processing of steel products in the presence of oxide scale on their surface.


title: Sharpness-Aware Minimization Improves Language Model Generalization

abstract: The allure of superhuman-level capabilities has led to considerable interest in language models like GPT-3 and T5, wherein the research has, by and large, revolved around new model architectures, training tasks, and loss objectives, along with substantial engineering efforts to scale up model capacity and dataset size. Comparatively little work has been done to improve the generalization of these models through better optimization. In this work, we show that Sharpness-Aware Minimization (SAM), a recently proposed optimization procedure that encourages convergence to flatter minima, can substantially improve the generalization of language models without much computational overhead. We show that SAM is able to boost performance on SuperGLUE, GLUE, Web Questions, Natural Questions, Trivia QA, and TyDiQA, with particularly large gains when training data for these tasks is limited.


title: Rotational controls and uniqueness of constrained viscosity solutions of Hamilton-Jacobi PDE

abstract: The classical inward pointing condition (IPC) for a control system whose state $x$ is constrained in the closure $C:=\bar\Omega$ of an open set $\Omega$ prescribes that at each point of the boundary $x\in \partial \Omega$ the intersection between the dynamics and the interior of the tangent space of $\bar \Omega$ at $x$ is nonempty. Under this hypothesis, for every system trajectory $x(.)$ on a time-interval $[0,T]$, possibly violating the constraint, one can construct a new system trajectory $\hat x(.)$ that satisfies the constraint and whose distance from $x(.)$ is bounded by a quantity proportional to the maximal deviation $d:=\mathrm{dist}(\Omega,x([0,T]))$. When (IPC) is violated, the construction of such a constrained trajectory is not possible in general. However, for a control system of the form $\dot{x}=f_1(x)u_1+f_2(x)u_2$, we prove in this paper that a "higher order" inward pointing condition involving Lie brackets of the dynamics' vector fields allows for a novel construction of a constrained trajectory $\hat x(.)$ whose distance from the reference trajectory $x(.)$ is bounded by a quantity proportional to $\sqrt{d}$. Our method requires a further assumption of non-positiveness of a sort of curvature and is based on the implementation of a suitable "rotating" control strategy. As an application, we establish the continuity up to the boundary of the value function $V$ of a classical optimal control problem, a continuity that allows to regard $V$ as the unique constrained viscosity solution of the corresponding Bellman equation.


title: A theoretical and empirical study of new adaptive algorithms with additional momentum steps and shifted updates for stochastic non-convex optimization

abstract: It is known that adaptive optimization algorithms represent the key pillar behind the rise of the Machine Learning field. In the Optimization literature numerous studies have been devoted to accelerated gradient methods but only recently adaptive iterative techniques were analyzed from a theoretical point of view. In the present paper we introduce new adaptive algorithms endowed with momentum terms for stochastic non-convex optimization problems. Our purpose is to show a deep connection between accelerated methods endowed with different inertial steps and AMSGrad-type momentum methods. Our methodology is based on the framework of stochastic and possibly non-convex objective mappings, along with some assumptions that are often used in the investigation of adaptive algorithms. In addition to discussing the finite-time horizon analysis in relation to a certain final iteration and the almost sure convergence to stationary points, we shall also look at the worst-case iteration complexity. This will be followed by an estimate for the expectation of the squared Euclidean norm of the gradient. Various computational simulations for the training of neural networks are being used to support the theoretical analysis. For future research we emphasize that there are multiple possible extensions to our work, from which we mention the investigation regarding non-smooth objective functions and the theoretical analysis of a more general formulation that encompass our adaptive optimizers in a stochastic framework.


title: Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher

abstract: With ever growing scale of neural models, knowledge distillation (KD) attracts more attention as a prominent tool for neural model compression. However, there are counter intuitive observations in the literature showing some challenging limitations of KD. A case in point is that the best performing checkpoint of the teacher might not necessarily be the best teacher for training the student in KD. Therefore, one important question would be how to find the best checkpoint of the teacher for distillation? Searching through the checkpoints of the teacher would be a very tedious and computationally expensive process, which we refer to as the \textit{checkpoint-search problem}. Moreover, another observation is that larger teachers might not necessarily be better teachers in KD which is referred to as the \textit{capacity-gap} problem. To address these challenging problems, in this work, we introduce our progressive knowledge distillation (Pro-KD) technique which defines a smoother training path for the student by following the training footprints of the teacher instead of solely relying on distilling from a single mature fully-trained teacher. We demonstrate that our technique is quite effective in mitigating the capacity-gap problem and the checkpoint search problem. We evaluate our technique using a comprehensive set of experiments on different tasks such as image classification (CIFAR-10 and CIFAR-100), natural language understanding tasks of the GLUE benchmark, and question answering (SQuAD 1.1 and 2.0) using BERT-based models and consistently got superior results over state-of-the-art techniques.


title: Double sided torus actions and complex geometry on $SU(3)$

abstract: We construct explicit complex structures and transversely K\"ahler holomorphic foliations on $SU(3)$ corresponding to variations of real quadratic equations on a complex quadric in $\mathbb{C}^{6}$ as generalizations of left-invariant complex structures on $SU(3) $ and an invariant K\"ahler structure on the flag variety $SU(3)/T$.Consequently, we obtain orbifold variants of the flag variety $SU(3)/T$ as quotients of double sided torus actions.


title: Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora

abstract: Pretrained language models (PTLMs) are typically learned over a large, static corpus and further fine-tuned for various downstream tasks. However, when deployed in the real world, a PTLM-based model must deal with data distributions that deviate from what the PTLM was initially trained on. In this paper, we study a lifelong language model pretraining challenge where a PTLM is continually updated so as to adapt to emerging data. Over a domain-incremental research paper stream and a chronologically-ordered tweet stream, we incrementally pretrain a PTLM with different continual learning algorithms, and keep track of the downstream task performance (after fine-tuning). We evaluate PTLM's ability to adapt to new corpora while retaining learned knowledge in earlier corpora. Our experiments show distillation-based approaches to be most effective in retaining downstream performance in earlier domains. The algorithms also improve knowledge transfer, allowing models to achieve better downstream performance over the latest data, and improve temporal generalization when distribution gaps exist between training and evaluation because of time. We believe our problem formulation, methods, and analysis will inspire future studies towards continual pretraining of language models.


title: Quantum hashing via single-photon states with orbital angular momentum

abstract: Quantum hashing is a promising generalization of the cryptographic hashing concept on the quantum domain. In this paper, we construct a quantum hash via a sequence of single-photon states and perform a proof-of-principle experiment using orbital angular momentum (OAM) encoding. We experimentally verify the collision resistance of the quantum hash function depending on the number of qubits in use. Based on these results, we conclude that theoretical estimates are confirmed for different bases of OAM states and the proposed technique can be useful in computational and cryptographic scenarios. The possibility of multiplexing different OAM bases can make this approach even more efficient.


title: Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models

abstract: Distilling state-of-the-art transformer models into lightweight student models is an effective way to reduce computation cost at inference time. The student models are typically compact transformers with fewer parameters, while expensive operations such as self-attention persist. Therefore, the improved inference speed may still be unsatisfactory for real-time or high-volume use cases. In this paper, we aim to further push the limit of inference speed by distilling teacher models into bigger, sparser student models -- bigger in that they scale up to billions of parameters; sparser in that most of the model parameters are n-gram embeddings. Our experiments on six single-sentence text classification tasks show that these student models retain 97% of the RoBERTa-Large teacher performance on average, and meanwhile achieve up to 600x speed-up on both GPUs and CPUs at inference time. Further investigation reveals that our pipeline is also helpful for sentence-pair classification tasks, and in domain generalization settings.


title: Verification of MPI programs

abstract: In this paper, we outline an approach to verifying parallel programs. A new mathematical model of parallel programs is introduced. The introduced model is illustrated by the verification of the matrix multiplication MPI program.


title: Substructure Distribution Projection for Zero-Shot Cross-Lingual Dependency Parsing

abstract: We present substructure distribution projection (SubDP), a technique that projects a distribution over structures in one domain to another, by projecting substructure distributions separately. Models for the target domains can be then trained, using the projected distributions as soft silver labels. We evaluate SubDP on zero-shot cross-lingual dependency parsing, taking dependency arcs as substructures: we project the predicted dependency arc distributions in the source language(s) to target language(s), and train a target language parser to fit the resulting distributions. When an English treebank is the only annotation that involves human effort, SubDP achieves better unlabeled attachment score than all prior work on the Universal Dependencies v2.2 (Nivre et al., 2020) test set across eight diverse target languages, as well as the best labeled attachment score on six out of eight languages. In addition, SubDP improves zero-shot cross-lingual dependency parsing with very few (e.g., 50) supervised bitext pairs, across a broader range of target languages.


title: A tangent inequality over primes

abstract: In this paper we introduce a new diophantine inequality with prime numbers. Let $1<c<\frac{10}{9}$. We show that for any fixed $\theta>1$, every sufficiently large positive number $N$ and a small constant $\varepsilon>0$, the tangent inequality \begin{equation*} \big|p^c_1\tan^\theta(\log p_1)+ p^c_2\tan^\theta(\log p_2)+ p^c_3\tan^\theta(\log p_3) -N\big|<\varepsilon \end{equation*} has a solution in prime numbers $p_1,\,p_2,\,p_3$.


title: Quantum Correlations in Jahn-Teller Molecular Systems Simulated with Superconducting Circuits

abstract: We explore quantum correlations, in particular, quantum entanglement, among vibrational phonon modes as well as between electronic and vibrational degrees of freedom in molecular systems, described by Jahn-Teller mechanism. Specifically, to isolate and simplify the phonon-electron interactions in a complex molecular system, the basis of our discussions is taken to be the proposal of simulating two-frequency Jahn-Teller systems using superconducting circuit quantum electrodynamics systems (circuit QED) by Tekin Dereli and co-workers in 2012. We evaluate the quantum correlations, in particular entanglement between the vibrational phonon modes, and present analytical explanations using a single privileged Jahn-Teller mode picture. Furthermore, spin-orbit entanglement or quantum correlations between electronic and vibrational degrees of freedom are examined, too. We conclude by discussing experimental feasibility to detect such quantum correlations, considering the dephasing and decoherence in state-of-the-art superconducting two-level systems (qubits).


title: Deformation of Framed Curves

abstract: We consider curves $\gamma : [0, 1]\to\mathbb{R}^3$ endowed with an adapted orthonormal frame $r : [0, 1]\to SO(3)$. We are interested in the cases where the frame is constrained, in the sense that one of its `curvatures' (i.e., off-diagonal elements of $r'r^T$) is prescribed. One example is the Fr\'enet frame. In order to deform such constrained framed curves without spoiling the constraint, we proceed in two stages. First we deform the frame $r$ in a way that is naturally compatible with the differential constraint, by interpreting it in terms of parallel transport on the sphere. The deformation of the base curve $\gamma$ is achieved in a second step, by means of a suitable reparametrization of the frame. We illustrate this deformation procedure by providing some applications.


title: Hey AI, Can You Solve Complex Tasks by Talking to Agents?

abstract: Training giant models from scratch for each complex task is resource- and data-inefficient. To help develop models that can leverage existing systems, we propose a new challenge: Learning to solve complex tasks by communicating with existing agents (or models) in natural language. We design a synthetic benchmark, CommaQA, with three complex reasoning tasks (explicit, implicit, numeric) designed to be solved by communicating with existing QA agents. For instance, using text and table QA agents to answer questions such as "Who had the longest javelin throw from USA?". We show that black-box models struggle to learn this task from scratch (accuracy under 50\%) even with access to each agent's knowledge and gold facts supervision. In contrast, models that learn to communicate with agents outperform black-box models, reaching scores of 100\% when given gold decomposition supervision. However, we show that the challenge of learning to solve complex tasks by communicating with existing agents \emph{without relying on any auxiliary supervision or data} still remains highly elusive. We release CommaQA, along with a compositional generalization test split, to advance research in this direction. Dataset and Code available at https://github.com/allenai/commaqa.


title: Optimal recovery of operator sequences

abstract: In this paper we consider two recovery problems based on information given with an error. First is the problem of optimal recovery of the class $W^T_q = \{(t_1h_1,t_2h_2,\ldots)\in \ell_q\,:\,\|h\|_q\leqslant 1\}$, where $1\le q < \infty$ and $t_1\geqslant t_2\geqslant \ldots \geqslant 0$, in the space $\ell_q$ when in the capacity of inexact information we know either the first $n\in\mathbb{N}$ elements of a sequence with an error measured in the space of finite sequences $\ell_r^n$, $0 < r \le \infty$, or a sequence itself is known with an error measured in the space $\ell_r$. The second is the problem of optimal recovery of scalar products acting on Cartesian product $W^{T,S}_{p,q}$ of classes $W^T_p$ and $W^S_q$, where $1 < p,q < \infty$, $\frac{1}{p} + \frac{1}{q} = 1$ and $s_1\ge s_2\ge \ldots \ge 0$, when in the capacity of inexact information we know the first $n$ coordinate-wise products $x_1y_1, x_2y_2,\ldots,x_ny_m$ of the element $x\times y \in W^{T,S}_{p,q}$ with an error measured in the space $\ell_r^n$. We find exact solutions to above problems and construct optimal methods of recovery. As an application of our results we consider the problem of optimal recovery of classes in Hilbert spaces by Fourier coefficients known with an error measured in the space $\ell_p$ with $p > 2$.


title: Answering Open-Domain Multi-Answer Questions via a Recall-then-Verify Framework

abstract: Open-domain questions are likely to be open-ended and ambiguous, leading to multiple valid answers. Existing approaches typically adopt the rerank-then-read framework, where a reader reads top-ranking evidence to predict answers. According to our empirical analysis, this framework faces three problems: first, to leverage a large reader under a memory constraint, the reranker should select only a few relevant passages to cover diverse answers, while balancing relevance and diversity is non-trivial; second, the small reading budget prevents the reader from accessing valuable retrieved evidence filtered out by the reranker; third, when using a generative reader to predict answers all at once based on all selected evidence, whether a valid answer will be predicted also pathologically depends on the evidence of some other valid answer(s). To address these issues, we propose to answer open-domain multi-answer questions with a recall-then-verify framework, which separates the reasoning process of each answer so that we can make better use of retrieved evidence while also leveraging large models under the same memory constraint. Our framework achieves state-of-the-art results on two multi-answer datasets, and predicts significantly more gold answers than a rerank-then-read system that uses an oracle reranker.


title: A Unified Speaker Adaptation Approach for ASR

abstract: Transformer models have been used in automatic speech recognition (ASR) successfully and yields state-of-the-art results. However, its performance is still affected by speaker mismatch between training and test data. Further finetuning a trained model with target speaker data is the most natural approach for adaptation, but it takes a lot of compute and may cause catastrophic forgetting to the existing speakers. In this work, we propose a unified speaker adaptation approach consisting of feature adaptation and model adaptation. For feature adaptation, we employ a speaker-aware persistent memory model which generalizes better to unseen test speakers by making use of speaker i-vectors to form a persistent memory. For model adaptation, we use a novel gradual pruning method to adapt to target speakers without changing the model architecture, which to the best of our knowledge, has never been explored in ASR. Specifically, we gradually prune less contributing parameters on model encoder to a certain sparsity level, and use the pruned parameters for adaptation, while freezing the unpruned parameters to keep the original model performance. We conduct experiments on the Librispeech dataset. Our proposed approach brings relative 2.74-6.52% word error rate (WER) reduction on general speaker adaptation. On target speaker adaptation, our method outperforms the baseline with up to 20.58% relative WER reduction, and surpasses the finetuning method by up to relative 2.54%. Besides, with extremely low-resource adaptation data (e.g., 1 utterance), our method could improve the WER by relative 6.53% with only a few epochs of training.


title: Critical Non-Abelian Vortex and Holography for Little String Theory

abstract: It has been shown that non-Abelian vortex string supported in four dimensional (4D) ${\mathcal N}=2$ supersymmetric QCD (SQCD) with the U(2) gauge group and $N_f = 4$ quark flavors becomes a critical superstring. This string propagates in the ten dimensional space formed by a product of the flat 4D space and an internal space given by a Calabi-Yau non-compact threefold, namely, the conifold. The spectrum of closed string states of the associated string theory was obtained using the equivalence between the critical string on the conifold and the non-critical string on the semi-infinite cigar described by SL($2, \mathbb{R}$)/U(1) Wess-Zumino-Novikov-Witten model. This spectrum was identified with the spectrum of hadrons in 4D ${\mathcal N}=2$ SQCD. In order to describe effective interactions of these 4D hadrons in this paper we study correlation functions of normalizable vertex operators localized near the tip of the SL($2, \mathbb{R}$)/U(1) cigar. We also compare our solitonic string approach to the gauge-string duality to the AdS/CFT-type holography for little string theories (LSTs). The latter relates off mass-shell correlation functions on the field theory side to correlation functions of non-normalizable vertex operators on the cigar. We show that in most channels holographic approach works in our theory because normalizable and non-normalizable vertex operators with the same conformal dimension are related due to the reflection from the tip of the cigar. However, we find that holography does not work for lightest hadrons with given baryonic charge.


title: Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation

abstract: This paper demonstrates that multilingual pretraining and multilingual fine-tuning are both critical for facilitating cross-lingual transfer in zero-shot translation, where the neural machine translation (NMT) model is tested on source languages unseen during supervised training. Following this idea, we present SixT+, a strong many-to-English NMT model that supports 100 source languages but is trained with a parallel dataset in only six source languages. SixT+ initializes the decoder embedding and the full encoder with XLM-R large and then trains the encoder and decoder layers with a simple two-stage training strategy. SixT+ achieves impressive performance on many-to-English translation. It significantly outperforms CRISS and m2m-100, two strong multilingual NMT systems, with an average gain of 7.2 and 5.0 BLEU respectively. Additionally, SixT+ offers a set of model parameters that can be further fine-tuned to other unsupervised tasks. We demonstrate that adding SixT+ initialization outperforms state-of-the-art explicitly designed unsupervised NMT models on Si<->En and Ne<->En by over 1.2 average BLEU. When applied to zero-shot cross-lingual abstractive summarization, it produces an average performance gain of 12.3 ROUGE-L over mBART-ft. We conduct detailed analyses to understand the key ingredients of SixT+, including multilinguality of the auxiliary parallel data, positional disentangled encoder, and the cross-lingual transferability of its encoder.


title: Totally nonnegative critical varieties

abstract: We study totally nonnegative parts of critical varieties in the Grassmannian. We show that each totally nonnegative critical variety Crit$^{\ge0}_f$ is the image of an affine poset cyclohedron under a continuous map and use this map to define a boundary stratification of Crit$^{\ge0}_f$. For the case of the top-dimensional positroid cell, we show that the totally nonnegative critical variety Crit$^{\ge0}_{k,n}$ is homeomorphic to the second hypersimplex $\Delta_{2,n}$.


title: Flexibility Framework with Recovery Guarantees for Aggregated Energy Storage Devices

abstract: This paper proposes a framework for the procurement of flexibility reserve from aggregated storage fleets. It allows for arbitrary tree structures of aggregation hierarchy, as well as easily implementable disaggregation via broadcast dispatch. By coupling discharge and recovery modes, the proposed framework enables full-cycle capacity to be procured ahead of real time, with guaranteed recovery and exact accounting for losses. The set of feasible discharging requests is exactly encoded, so that there is no reduction in the ability to meet discharging signals, and recovery capabilities are parametrised as a single virtual battery. Included in this paper is a numerical demonstration of the construction of the constituent curves of the framework and the approach is also benchmarked against relevant alternatives.


title: The enhanced s*-wave component of the superconducting gap in the overdoped HTSC cuprates with orthorhombic distortion

abstract: In this work, the concentration and temperature dependences of the superconducting gap of the HTSC cuprate in the orthorhombic phase are obtained within the framework of the Hubbard model taking into account the exchange mechanism of pairing. The enhanced s*-wave component of the superconducting gap against the background of the prevailing d-wave component in a narrow range of hole concentrations in the overdoped region is found. To elucidate the reasons for such unusual behavior the electronic structure of the low-energy excitations and the structure of the contributions of the pair states to the superconducting gap in momentum space were investigated at different doping. It is shown that the presence of a shallow pocket in the region of the maximum value of the s*-wave component of the gap results in different symmetry of the superconducting gap in different regions of the k-space and an increase in the s*-wave component.


title: HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression

abstract: On many natural language processing tasks, large pre-trained language models (PLMs) have shown overwhelming performances compared with traditional neural network methods. Nevertheless, their huge model size and low inference speed have hindered the deployment on resource-limited devices in practice. In this paper, we target to compress PLMs with knowledge distillation, and propose a hierarchical relational knowledge distillation (HRKD) method to capture both hierarchical and domain relational information. Specifically, to enhance the model capability and transferability, we leverage the idea of meta-learning and set up domain-relational graphs to capture the relational information across different domains. And to dynamically select the most representative prototypes for each domain, we propose a hierarchical compare-aggregate mechanism to capture hierarchical relationships. Extensive experiments on public multi-domain datasets demonstrate the superior performance of our HRKD method as well as its strong few-shot learning ability. For reproducibility, we release the code at https://github.com/cheneydon/hrkd.


title: Virtual Augmentation Supported Contrastive Learning of Sentence Representations

abstract: Despite profound successes, contrastive representation learning relies on carefully designed data augmentations using domain specific knowledge. This challenge is magnified in natural language processing where no general rules exist for data augmentation due to the discrete nature of natural language. We tackle this challenge by presenting a Virtual augmentation Supported Contrastive Learning of sentence representations (VaSCL). Originating from the interpretation that data augmentation essentially constructs the neighborhoods of each training instance, we in turn utilize the neighborhood to generate effective data augmentations. Leveraging the large training batch size of contrastive learning, we approximate the neighborhood of an instance via its K-nearest in-batch neighbors in the representation space. We then define an instance discrimination task regarding this neighborhood and generate the virtual augmentation in an adversarial training manner. We access the performance of VaSCL on a wide range of downstream tasks, and set a new state-of-the-art for unsupervised sentence representation learning.


title: Flows on Metric Graphs with General Boundary Conditions

abstract: In this note we study the generation of $C_0$-semigroups by first order differential operators on $\mathrm{L}^p (\mathbb{R}_+,\mathbb{C}^{\ell})\times \mathrm{L}^p ([0,1],\mathbb{C}^{m})$ with general boundary conditions. In many cases we are able to characterize the generation property in terms of the invertibility of a matrix associated to the boundary conditions. The abstract results are used to study well-posedness of transport equations on non-compact metric graphs.


title: PAGnol: An Extra-Large French Generative Model

abstract: Access to large pre-trained models of varied architectures, in many different languages, is central to the democratization of NLP. We introduce PAGnol, a collection of French GPT models. Using scaling laws, we efficiently train PAGnol-XL (1.5B parameters) with the same computational budget as CamemBERT, a model 13 times smaller. PAGnol-XL is the largest model trained to date for the French language. We plan to train increasingly large and performing versions of PAGnol, exploring the capabilities of French extreme-scale models. For this first release, we focus on the pre-training and scaling calculations underlining PAGnol. We fit a scaling law for compute for the French language, and compare it with its English counterpart. We find the pre-training dataset significantly conditions the quality of the outputs, with common datasets such as OSCAR leading to low-quality offensive text. We evaluate our models on discriminative and generative tasks in French, comparing to other state-of-the-art French and multilingual models, and reaching the state of the art in the abstract summarization task. Our research was conducted on the public GENCI Jean Zay supercomputer, and our models up to the Large are made publicly available.


title: On the Robustness of Reading Comprehension Models to Entity Renaming

abstract: We study the robustness of machine reading comprehension (MRC) models to entity renaming -- do models make more wrong predictions when the same questions are asked about an entity whose name has been changed? Such failures imply that models overly rely on entity information to answer questions, and thus may generalize poorly when facts about the world change or questions are asked about novel entities. To systematically audit this issue, we present a pipeline to automatically generate test examples at scale, by replacing entity names in the original test sample with names from a variety of sources, ranging from names in the same test set, to common names in life, to arbitrary strings. Across five datasets and three pretrained model architectures, MRC models consistently perform worse when entities are renamed, with particularly large accuracy drops on datasets constructed via distant supervision. We also find large differences between models: SpanBERT, which is pretrained with span-level masking, is more robust than RoBERTa, despite having similar accuracy on unperturbed test data. We further experiment with different masking strategies as the continual pretraining objective and find that entity-based masking can improve the robustness of MRC models.


title: Multi-View Stereo Network with attention thin volume

abstract: We propose an efficient multi-view stereo (MVS) network for infering depth value from multiple RGB images. Recent studies have shown that mapping the geometric relationship in real space to neural network is an essential topic of the MVS problem. Specifically, these methods focus on how to express the correspondence between different views by constructing a nice cost volume. In this paper, we propose a more complete cost volume construction approach based on absorbing previous experience. First of all, we introduce the self-attention mechanism to fully aggregate the dominant information from input images and accurately model the long-range dependency, so as to selectively aggregate reference features. Secondly, we introduce the group-wise correlation to feature aggregation, which greatly reduces the memory and calculation burden. Meanwhile, this method enhances the information interaction between different feature channels. With this approach, a more lightweight and efficient cost volume is constructed. Finally we follow the coarse to fine strategy and refine the depth sampling range scale by scale with the help of uncertainty estimation. We further combine the previous steps to get the attention thin volume. Quantitative and qualitative experiments are presented to demonstrate the performance of our model.


title: DPNAS: Neural Architecture Search for Deep Learning with Differential Privacy

abstract: Training deep neural networks (DNNs) for meaningful differential privacy (DP) guarantees severely degrades model utility. In this paper, we demonstrate that the architecture of DNNs has a significant impact on model utility in the context of private deep learning, whereas its effect is largely unexplored in previous studies. In light of this missing, we propose the very first framework that employs neural architecture search to automatic model design for private deep learning, dubbed as DPNAS. To integrate private learning with architecture search, we delicately design a novel search space and propose a DP-aware method for training candidate models. We empirically certify the effectiveness of the proposed framework. The searched model DPNASNet achieves state-of-the-art privacy/utility trade-offs, e.g., for the privacy budget of $(\epsilon, \delta)=(3, 1\times10^{-5})$, our model obtains test accuracy of $98.57\%$ on MNIST, $88.09\%$ on FashionMNIST, and $68.33\%$ on CIFAR-10. Furthermore, by studying the generated architectures, we provide several intriguing findings of designing private-learning-friendly DNNs, which can shed new light on model design for deep learning with differential privacy.


title: Neural Network Pruning Through Constrained Reinforcement Learning

abstract: Network pruning reduces the size of neural networks by removing (pruning) neurons such that the performance drop is minimal. Traditional pruning approaches focus on designing metrics to quantify the usefulness of a neuron which is often quite tedious and sub-optimal. More recent approaches have instead focused on training auxiliary networks to automatically learn how useful each neuron is however, they often do not take computational limitations into account. In this work, we propose a general methodology for pruning neural networks. Our proposed methodology can prune neural networks to respect pre-defined computational budgets on arbitrary, possibly non-differentiable, functions. Furthermore, we only assume the ability to be able to evaluate these functions for different inputs, and hence they do not need to be fully specified beforehand. We achieve this by proposing a novel pruning strategy via constrained reinforcement learning algorithms. We prove the effectiveness of our approach via comparison with state-of-the-art methods on standard image classification datasets. Specifically, we reduce 83-92.90 of total parameters on various variants of VGG while achieving comparable or better performance than that of original networks. We also achieved 75.09 reduction in parameters on ResNet18 without incurring any loss in accuracy.


title: FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metricsfor Automatic Text Generation

abstract: Fast and reliable evaluation metrics are key to R&D progress. While traditional natural language generation metrics are fast, they are not very reliable. Conversely, new metrics based on large pretrained language models are much more reliable, but require significant computational resources. In this paper, we propose FrugalScore, an approach to learn a fixed, low cost version of any expensive NLG metric, while retaining most of its original performance. Experiments with BERTScore and MoverScore on summarization and translation show that FrugalScore is on par with the original metrics (and sometimes better), while having several orders of magnitude less parameters and running several times faster. On average over all learned metrics, tasks, and variants, FrugalScore retains 96.8% of the performance, runs 24 times faster, and has 35 times less parameters than the original metrics. We make our trained metrics publicly available, to benefit the entire NLP community and in particular researchers and practitioners with limited resources.


title: Black holes with Lagrange multiplier and potential in mimetic-like gravitational theory: multi-horizon black holes

abstract: In this paper, we employ the {\bf mimetic-like} field equations coupled with the Lagrange multiplier and %mimetic {\bf potential} to derive non-trivial spherically symmetric black hole (BH) solutions. We divided this study into three cases: The first one in which we take the Lagrange multiplier and %mimetic {\bf the potential} to have vanishing value and derive a BH solution that completely coincides with the BH of the Einstein general relativity despite the non-vanishing value of the {\bf mimetic-like scalar field}. The first case is completely consistent with the previous studies in the literature that mimetic theory coincides with GR \cite{Nashed:2018qag}. In the second case, we derive a solution with a constant value of the {\bf potential} and a dynamical value of the Lagrange multiplier. This solution has no horizon and therefore the obtained spacetime does not correspond to the BH. In this solution, there appears the region of the Euclidian signature where the signature of the diagonal components of the metric is $(+,+,+,+)$ or the region with two times where the signature is $(+,+,-,-)$. Finally, we derive a BH solution with non-vanishing values of the Lagrange multiplier, {\bf potential}, and {\bf mimetic-like scalar field}. This BH shows a soft singularity compared with the Einstein BH solution. The relevant physics of the third case is discussed by showing their behavior of the metric potential at infinity, calculating their energy conditions, and study their thermodynamical quantities. We give a brief discussion on how our third case can generate a BH with three horizons as in the de Sitter-Reissner-Nordstr\"om black hole spacetime, where the largest horizon is the cosmological one and two correspond to the outer and inner horizons of the BH. Even in the third case, there appears the region of the Euclidian signature or the region with two times.


title: Sahlqvist Correspondence Theory for Second-Order Propositional Modal Logic

abstract: Modal logic with propositional quantifiers (i.e. second-order propositional modal logic (SOPML)) has been considered since the early time of modal logic. Its expressive power and complexity are high, and its van-Benthem-Rosen theorem and Goldblatt-Thomason theorem have been proved by ten Cate (2006). However, the Sahlqvist theory of SOPML has not been considered in the literature. In the present paper, we fill in this gap. We develop the Sahlqvist correspondence theory for SOPML, which covers and properly extends existing Sahlqvist formulas in basic modal logic. We define the class of Sahlqvist formulas for SOMPL step by step in a hierarchical way, each formula of which is shown to have a first-order correspondent over Kripke frames effectively computable by an algorithm $ALBA^{SOMPL}$. In addition, we show that certain $\Pi_2$-rules correspond to $\Pi_2$-Sahlqvist formulas in SOMPL, which further correspond to first-order conditions, and that even for very simple SOMPL Sahlqvist formulas, they could already be non-canonical.


title: BNAS v2: Learning Architectures for Binary Networks with Empirical Improvements

abstract: Backbone architectures of most binary networks are well-known floating point (FP) architectures such as the ResNet family. Questioning that the architectures designed for FP networks might not be the best for binary networks, we propose to search architectures for binary networks (BNAS) by defining a new search space for binary architectures and a novel search objective. Specifically, based on the cell based search method, we define the new search space of binary layer types, design a new cell template, and rediscover the utility of and propose to use the Zeroise layer instead of using it as a placeholder. The novel search objective diversifies early search to learn better performing binary architectures. We show that our method searches architectures with stable training curves despite the quantization error inherent in binary networks. Quantitative analyses demonstrate that our searched architectures outperform the architectures used in state-of-the-art binary networks and outperform or perform on par with state-of-the-art binary networks that employ various techniques other than architectural changes. In addition, we further propose improvements to the training scheme of our searched architectures. With the new training scheme for our searched architectures, we achieve the state-of-the-art performance by binary networks by outperforming all previous methods by non-trivial margins.


title: Auction design with ambiguity: Optimality of the first-price and all-pay auctions

abstract: We study the optimal auction design problem when bidders' preferences follow the maxmin expected utility model. We suppose that each bidder's set of priors consists of beliefs close to the seller's belief, where "closeness" is defined by a divergence. For a given allocation rule, we identify a class of optimal transfer candidates, named the win-lose dependent transfers, with the following property: each type of bidder's transfer conditional on winning or losing is independent of the competitor's type report. Our result reduces the infinite-dimensional optimal transfer problem to a two-dimensional optimization problem. By solving the reduced problem, we find that: (i) among efficient mechanisms with no premiums for losers, the first-price auction is optimal; and, (ii) among efficient winner-favored mechanisms where each bidder pays smaller amounts when she wins than loses: the all-pay auction is optimal. Under a simplifying assumption, these two auctions remain optimal under the endogenous allocation rule.


title: Dependence of the initial internal structure of chondrule rim on dust size distribution

abstract: Coarse objects in chondrites such as chondrules and CAIs are mostly coated with fine-grained rims (FGRs). FGRs can be formed on the surface of free floating chondrules in a turbulent nebula, where dust aggregation also occurs. A former study has reported that the morphology of the dust populations accreting onto chondrules affects the initial structures of FGRs. It was revealed that, if monomer grains accrete onto chondrules, the smaller grains tend to accumulate near the surface of chondrules, and FGRs exhibit grain size coarsening from the bottom to the top. However, the study did not consider the effect of temporal growth of dust aggregates on FGRs formation. In this study, we calculate the aggregation of polydisperse monomer grains and their accretion onto chondrules. The following two different stages of dust aggregation can be identified: the monomer-aggregation stage and the BCCA-like stage. In the monomer-aggregation stage, monomer grains are incorporated into aggregates when the average aggregate size reaches the size of the monomer. In the BCCA-like stage, aggregates evolve fractally in a fashion similar to that of single size monomer grains. Based on the results of the previous study, we obtain the requisite conditions for chondrules to acquire monomer-accreting FGRs with grain size coarsening observed in some chondrites. In the case of similar size distribution as that of Inter Stellar Medium (ISM), the maximum grain size of $>$ $1$ $\mu$m is widely ($\alpha$ $<$ $10^{-3}$) required for monomer accretion, while if turbulent intensity in a nebula is extremely weak ($\alpha$ $<$ $10^{-5}$), a maximum grain size $\sim$ $10$ $\mu$m is required. The monomer size distributions having larger mass fraction in the large grains compared to ISM might be necessary for the effective grain size coarsening.


title: Dynamic Graph Echo State Networks

abstract: Dynamic temporal graphs represent evolving relations between entities, e.g. interactions between social network users or infection spreading. We propose an extension of graph echo state networks for the efficient processing of dynamic temporal graphs, with a sufficient condition for their echo state property, and an experimental analysis of reservoir layout impact. Compared to temporal graph kernels that need to hold the entire history of vertex interactions, our model provides a vector encoding for the dynamic graph that is updated at each time-step without requiring training. Experiments show accuracy comparable to approximate temporal graph kernels on twelve dissemination process classification tasks.


title: Enlargeable foliations and the monodromy groupoid

abstract: Let $M$ be a spin manifold, the Dirac operator with coefficient in the universal flat Hilbert $C^\ast \pi_1(M)$-module determines a "Rosenberg index element" which, according to B.Hanke and T.Schick, subsumes the enlargeablility obstruction of positive scalar curvature on $M$. In this note, we generalize this result to the case of spin foliation. More precisely, given a foliation $(M,F)$ with $F$ spin, we shall define a foliation version of "Rosenberg index element" and prove that it is nonzero at the presence of compactly enlargeability of $(M,F)$.


title: Detecting directional forces in the evolution of grammar: A case study of the English perfect with intransitives across EEBO, COHA, and Google Books

abstract: Languages have diverse characteristics that have emerged through evolution. In modern English grammar, the perfect is formed with \textit{have}+PP (past participle), but in earlier English the \textit{be}+PP form also existed. It is widely recognised that the auxiliary verb BE was replaced by HAVE throughout evolution, except for some special cases. However, whether this evolution was caused by natural selection or random drift is still unclear. Here we examined directional forces in the evolution of the English perfect with intransitives by combining three large-scale data sources: EEBO (Early English Books Online), COHA (Corpus of Historical American English), and Google Books. We found that most intransitive verbs exhibited an apparent transition from \textit{be}+PP to \textit{have}+PP, most of which were classified as `selection' by a deep neural network-based model. These results suggest that the English perfect could have evolved through natural selection rather than random drift, and provide insights into the cultural evolution of grammar.


title: ASFormer: Transformer for Action Segmentation

abstract: Algorithms for the action segmentation task typically use temporal models to predict what action is occurring at each frame for a minute-long daily activity. Recent studies have shown the potential of Transformer in modeling the relations among elements in sequential data. However, there are several major concerns when directly applying the Transformer to the action segmentation task, such as the lack of inductive biases with small training sets, the deficit in processing long input sequence, and the limitation of the decoder architecture to utilize temporal relations among multiple action segments to refine the initial predictions. To address these concerns, we design an efficient Transformer-based model for action segmentation task, named ASFormer, with three distinctive characteristics: (i) We explicitly bring in the local connectivity inductive priors because of the high locality of features. It constrains the hypothesis space within a reliable scope, and is beneficial for the action segmentation task to learn a proper target function with small training sets. (ii) We apply a pre-defined hierarchical representation pattern that efficiently handles long input sequences. (iii) We carefully design the decoder to refine the initial predictions from the encoder. Extensive experiments on three public datasets demonstrate that effectiveness of our methods. Code is available at \url{https://github.com/ChinaYi/ASFormer}.


title: Deep Image Debanding

abstract: Banding or false contour is an annoying visual artifact whose impact is even more pronounced in ultra high definition, high dynamic range, and wide colour gamut visual content, which is becoming increasingly popular. Since users associate a heightened expectation of quality with such content and banding leads to deteriorated visual quality-of-experience, the area of banding removal or debanding has taken paramount importance. Existing debanding approaches are mostly knowledge-driven. Despite the widespread success of deep learning in other areas of image processing and computer vision, data-driven debanding approaches remain surprisingly missing. In this work, we make one of the first attempts to develop a deep learning based banding artifact removal method for images and name it deep debanding network (deepDeband). For its training, we construct a large-scale dataset of 51,490 pairs of corresponding pristine and banded image patches. Performance evaluation shows that deepDeband is successful at greatly reducing banding artifacts in images, outperforming existing methods both quantitatively and visually.


title: A Reduced-Bias Weighted least square estimation of the Extreme Value Index

abstract: In this paper, we propose a reduced-bias estimator of the EVI for Pareto-type tails (heavy-tailed) distributions. This is derived using the weighted least squares method. It is shown that the estimator is unbiased, consistent and asymptotically normal under the second-order conditions on the underlying distribution of the data. The finite sample properties of the proposed estimator are studied through a simulation study. The results show that it is competitive to the existing estimators of the extreme value index in terms of bias and Mean Square Error. In addition, it yields estimates of $\gamma>0$ that are less sensitive to the number of top-order statistics, and hence, can be used for selecting an optimal tail fraction. The proposed estimator is further illustrated using practical datasets from pedochemical and insurance.


title: Explore before Moving: A Feasible Path Estimation and Memory Recalling Framework for Embodied Navigation

abstract: An embodied task such as embodied question answering (EmbodiedQA), requires an agent to explore the environment and collect clues to answer a given question that related with specific objects in the scene. The solution of such task usually includes two stages, a navigator and a visual Q&A module. In this paper, we focus on the navigation and solve the problem of existing navigation algorithms lacking experience and common sense, which essentially results in a failure finding target when robot is spawn in unknown environments. Inspired by the human ability to think twice before moving and conceive several feasible paths to seek a goal in unfamiliar scenes, we present a route planning method named Path Estimation and Memory Recalling (PEMR) framework. PEMR includes a "looking ahead" process, \textit{i.e.} a visual feature extractor module that estimates feasible paths for gathering 3D navigational information, which is mimicking the human sense of direction. PEMR contains another process ``looking behind'' process that is a memory recall mechanism aims at fully leveraging past experience collected by the feature extractor. Last but not the least, to encourage the navigator to learn more accurate prior expert experience, we improve the original benchmark dataset and provide a family of evaluation metrics for diagnosing both navigation and question answering modules. We show strong experimental results of PEMR on the EmbodiedQA navigation task.


title: Greedy and Random Broyden's Methods with Explicit Superlinear Convergence Rates in Nonlinear Equations

abstract: In this paper, we propose the greedy and random Broyden's method for solving nonlinear equations. Specifically, the greedy method greedily selects the direction to maximize a certain measure of progress for approximating the current Jacobian matrix, while the random method randomly chooses a direction. We establish explicit (local) superlinear convergence rates of both methods if the initial point and approximate Jacobian are close enough to a solution and corresponding Jacobian. Our two novel variants of Broyden's method enjoy two important advantages that the approximate Jacobians of our algorithms will converge to the exact ones and the convergence rates of our algorithms are asymptotically faster than the original Broyden's method. Our work is the first time to achieve such two advantages theoretically. Our experiments also empirically validate the advantages of our algorithms.


title: Insight from the elliptic flow of identified hadrons measured in relativistic heavy-ion collisions

abstract: In this paper, we have discussed the transverse momentum dependence of elliptic flow measured in relativistic heavy-ion collisions. The transverse momentum dependence of the number of constituent quarks ($n_{q}$) scaled $v_{2}$ is obtained for all the measured identified hadrons. The $n_{q}$ scaled $\phi$ $v_{2}$ is found to be similar to $n_{q}$ scaled $\Omega$ $v_{2}$. This indicates that both $\phi$ and $\Omega$ are produced through quark recombination at RHIC energies. We also find $n_{q}$ scaled proton $v_{2}$ ($v_{2}$ of light quarks) is higher than $n_{q}$ scaled $\phi$ and $\Omega$ $v_{2}$ ($v_{2}$ of strange quarks) at $p_{T}/n_{q}$ $<$ 1.0 GeV/c. This can be explained by considering quark $v_{2}$ is proportional to its transverse kinetic energy considering mass of deconfined light and strange quarks equal to $\sim $4 MeV and $\sim$ 140 MeV, respectively.


title: Nonequilibrium dynamics in pumped Mott insulators

abstract: We use time-evolution techniques for (infinite) matrix-product-states to calculate, directly in the thermodynamic limit, the time-dependent photoemission spectra and dynamic structure factors of the half-filled Hubbard chain after pulse irradiation. These quantities exhibit clear signatures of the photoinduced phase transition from insulator to metal that occurs because of the formation of so-called $\eta$ pairs. In addition, the spin dynamic structure factor loses spectral weight in the whole momentum space, reflecting the suppression of antiferromagnetic correlations due to the buildup of $\eta$-pairing states. The numerical method demonstrated in this work can be readily applied to other one-dimensional models driven out of equilibrium by optical pumping.


title: Cellular resource allocation strategies for cell size and shape control in bacteria

abstract: Bacteria are highly adaptive microorganisms that thrive in a wide range of growth conditions via changes in cell morphologies and macromolecular composition. How bacterial morphologies are regulated in diverse environmental conditions is a longstanding question. Regulation of cell size and shape implies control mechanisms that couple the growth and division of bacteria to their cellular environment and macromolecular composition. In the past decade, simple quantitative laws have emerged that connect cell growth to proteomic composition and the nutrient availability. However, the relationships between cell size, shape and growth physiology remain challenging to disentangle and unifying models are lacking. In this review, we focus on regulatory models of cell size control that reveal the connections between bacterial cell morphology and growth physiology. In particular, we discuss how changes in nutrient conditions and translational perturbations regulate the cell size, growth rate and proteome composition. Integrating quantitative models with experimental data, we identify the physiological principles of bacterial size regulation, and discuss the optimization strategies of cellular resource allocation for size control.


title: Closed-form formulas and properties of coefficients in Maclaurin's series expansion of Wilf's function composited by inverse tangent, square root, and exponential functions

abstract: In the paper, with the aid of the Fa\`a di Bruno formula, be virtue of several identities for the Bell polynomials of the second kind, with the help of two combinatorial identities, by means of the (logarithmically) complete monotonicity of generating functions of several integer sequences, and in the light of Wronski's theorem, the authors establish Taylor's series expansions of several functions involving the inverse (hyperbolic) tangent function, find out Maclaurin's series expansion of a complex function composited by inverse tangent, square root, and exponential functions and posed by Herbert S. Wilf, and analyze some properties, including generating functions, limits, positivity, monotonicity, and logarithmic convexity, of the coefficients in Maclaurin's series expansion of Wilf's function. These coefficients in Maclaurin's series expansion of Wilf's function are closed-form expressions in terms of the Stirling numbers of the second kind. The authors also derive a closed-form formula for a sequence of special values of Gauss' hypergeometric function, discover a closed-form formula for a sequence of special values of the Bell polynomials of the second kind, present several infinite series representations of the circular constant Pi and other sequences, recover an asymptotic rational approximation to the circular constant Pi, and connect several integer sequences by determinants.


title: Nys-Newton: Nystr\"om-Approximated Curvature for Stochastic Optimization

abstract: Second-order optimization methods are among the most widely used optimization approaches for convex optimization problems, and have recently been used to optimize non-convex optimization problems such as deep learning models. The widely used second-order optimization methods such as quasi-Newton methods generally provide curvature information by approximating the Hessian using the secant equation. However, the secant equation becomes insipid in approximating the Newton step owing to its use of the first-order derivatives. In this study, we propose an approximate Newton sketch-based stochastic optimization algorithm for large-scale empirical risk minimization. Specifically, we compute a partial column Hessian of size ($d\times m$) with $m\ll d$ randomly selected variables, then use the \emph{Nystr\"om method} to better approximate the full Hessian matrix. To further reduce the computational complexity per iteration, we directly compute the update step ($\Delta\boldsymbol{w}$) without computing and storing the full Hessian or its inverse. We then integrate our approximated Hessian with stochastic gradient descent and stochastic variance-reduced gradient methods. The results of numerical experiments on both convex and non-convex functions show that the proposed approach was able to obtain a better approximation of Newton\textquotesingle s method, exhibiting performance competitive with that of state-of-the-art first-order and stochastic quasi-Newton methods. Furthermore, we provide a theoretical convergence analysis for convex functions.


title: Visual-aware Attention Dual-stream Decoder for Video Captioning

abstract: Video captioning is a challenging task that captures different visual parts and describes them in sentences, for it requires visual and linguistic coherence. The attention mechanism in the current video captioning method learns to assign weight to each frame, promoting the decoder dynamically. This may not explicitly model the correlation and the temporal coherence of the visual features extracted in the sequence frames.To generate semantically coherent sentences, we propose a new Visual-aware Attention (VA) model, which concatenates dynamic changes of temporal sequence frames with the words at the previous moment, as the input of attention mechanism to extract sequence features.In addition, the prevalent approaches widely use the teacher-forcing (TF) learning during training, where the next token is generated conditioned on the previous ground-truth tokens. The semantic information in the previously generated tokens is lost. Therefore, we design a self-forcing (SF) stream that takes the semantic information in the probability distribution of the previous token as input to enhance the current token.The Dual-stream Decoder (DD) architecture unifies the TF and SF streams, generating sentences to promote the annotated captioning for both streams.Meanwhile, with the Dual-stream Decoder utilized, the exposure bias problem is alleviated, caused by the discrepancy between the training and testing in the TF learning.The effectiveness of the proposed Visual-aware Attention Dual-stream Decoder (VADD) is demonstrated through the result of experimental studies on Microsoft video description (MSVD) corpus and MSR-Video to text (MSR-VTT) datasets.


title: Stochastic Processes in Networks of Queues with Exponential Service Times and only one Class of Customers

abstract: Two networks of queues models, presented initially by Jackson, in the open case, and Gordon and Newell, in the closed case, stochastic processes are presented and studied in some of their details and problems. The service times are exponentially distributed and there is only one class of customers.


title: Intelligent Video Editing: Incorporating Modern Talking Face Generation Algorithms in a Video Editor

abstract: This paper proposes a video editor based on OpenShot with several state-of-the-art facial video editing algorithms as added functionalities. Our editor provides an easy-to-use interface to apply modern lip-syncing algorithms interactively. Apart from lip-syncing, the editor also uses audio and facial re-enactment to generate expressive talking faces. The manual control improves the overall experience of video editing without missing out on the benefits of modern synthetic video generation algorithms. This control enables us to lip-sync complex dubbed movie scenes, interviews, television shows, and other visual content. Furthermore, our editor provides features that automatically translate lectures from spoken content, lip-sync of the professor, and background content like slides. While doing so, we also tackle the critical aspect of synchronizing background content with the translated speech. We qualitatively evaluate the usefulness of the proposed editor by conducting human evaluations. Our evaluations show a clear improvement in the efficiency of using human editors and an improved video generation quality. We attach demo videos with the supplementary material clearly explaining the tool and also showcasing multiple results.


title: A Tutorial on Terahertz-Band Localization for 6G Communication Systems

abstract: Terahertz (THz) communications are celebrated as key enablers for converged localization and sensing in future sixth-generation (6G) wireless communication systems and beyond. Instead of being a byproduct of the communication system, localization in 6G is indispensable for location-aware communications. Towards this end, we aim to identify the prospects, challenges, and requirements of THz localization techniques. We first review the history and trends of localization methods and discuss their objectives, constraints, and applications in contemporary communication systems. We then detail the latest advances in THz communications and introduce the THz-specific channel and system models. Afterward, we formulate THz-band localization as a 3D position/orientation estimation problem, detailing geometry-based localization techniques and describing potential THz localization and sensing extensions. We further formulate the offline design and online optimization of THz localization systems, provide numerical simulation results, and conclude by providing lessons learned and future research directions. Preliminary results illustrate that under the same transmission power and array footprint, THz-based localization outperforms millimeter wave-based localization. In other words, the same level of localization performance can be achieved at THz-band with less transmission power or a smaller footprint.


title: Bifurcation analysis of a fractional-order Pinsky-Rinzel model

abstract: Abstract The present work describes a new fractional-order system of a two-compartment CA3 hippocampal pyramidal cell, which is known as Pinsky-Rinzel model with Caputo fractional derivative. Firstly, The transient of the solutions is investigated. Then based on the bifurcation diagrams, we study the general behavior of the system. In this case, fractional derivative order and currents injection, are taken as bifurcation parameters. Chaotic regions are obtained for different values of the fractional derivative order and different injection currents. Finally, a numerical approach is introduced to study the stability of the system under certain conditions. The obtained results can be considered as help to control relevant diseases caused by maximal injection currents abnormality.


title: ASR4REAL: An extended benchmark for speech models

abstract: Popular ASR benchmarks such as Librispeech and Switchboard are limited in the diversity of settings and speakers they represent. We introduce a set of benchmarks matching real-life conditions, aimed at spotting possible biases and weaknesses in models. We have found out that even though recent models do not seem to exhibit a gender bias, they usually show important performance discrepancies by accent, and even more important ones depending on the socio-economic status of the speakers. Finally, all tested models show a strong performance drop when tested on conversational speech, and in this precise context even a language model trained on a dataset as big as Common Crawl does not seem to have significant positive effect which reiterates the importance of developing conversational language models


title: Surveying Wonderland for many more literature visualization techniques

abstract: There are still many potential literature visualizations to be discovered. By focusing on a single text, the author surveys many existing visualizations across research domains, in the wild, and creates new visualizations. 58 techniques are indicated, suggesting a wider variety of visualizations beyond research disciplines.


title: Configurational Entropy and braneworlds in f(T,B) gravity

abstract: The thick brane scenario built on the f(T,B) teleparallel gravity theory was considered for the study of phase transitions, internal structures, and new classes of solutions in a model. In this theory, T denotes the torsion scalar, and B is a boundary term. An interesting result was observed when brane splitting occurs, i. e., internal structures in the model arise as a consequence of the appearance of new domain walls in the theory. In fact, this preliminary result influences the profile of the matter field (from kink to multi-kink) so that for appropriate values of the parameters $k_{1,2}$ multiple phase transitions are identified. To perform this analysis, the Differential Configurational Entropy (DCE) which has the ability to predict the existence of phase transitions through critical points was used. Furthermore, the DCE is able to select the most stable solutions since it gives us details about the informational content of the field settings.


title: Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments

abstract: Autonomous driving is a complex task, which has been tackled since the first self-driving car ALVINN in 1989, with a supervised learning approach, or behavioral cloning (BC). In BC, a neural network is trained with state-action pairs that constitute the training set made by an expert, i.e., a human driver. However, this type of imitation learning does not take into account the temporal dependencies that might exist between actions taken in different moments of a navigation trajectory. These type of tasks are better handled by reinforcement learning (RL) algorithms, which need to define a reward function. On the other hand, more recent approaches to imitation learning, such as Generative Adversarial Imitation Learning (GAIL), can train policies without explicitly requiring to define a reward function, allowing an agent to learn by trial and error directly on a training set of expert trajectories. In this work, we propose two variations of GAIL for autonomous navigation of a vehicle in the realistic CARLA simulation environment for urban scenarios. Both of them use the same network architecture, which process high dimensional image input from three frontal cameras, and other nine continuous inputs representing the velocity, the next point from the sparse trajectory and a high-level driving command. We show that both of them are capable of imitating the expert trajectory from start to end after training ends, but the GAIL loss function that is augmented with BC outperforms the former in terms of convergence time and training stability.


title: Extension of Laguerre polynomials with negative arguments II

abstract: For $n \geq 3$ and $s \leq 92$, it is proved in \cite{ShSi} that, except for finitely many pairs $(n, s), G_1(x) = G_1(x, n, s) $ is either irreducible or linear factor times an irreducible polynomial. If $s \leq 30$, we determine here explicitely the set of pairs $(n, s)$ in the above assertion. This implies a new proof of the result of Nair and Shorey \cite{NaSh1} that $G_1(x)$ is irreducible for $s \leq 22$.


title: Preproduction Deploys: Cloud-Native Integration Testing

abstract: The microservice architecture for cloud-based systems is extended to not only require each loosely coupled component to be independently deployable, but also to provide independent routing for each component. This supports canary deployments, green/blue deployments and roll-back. Both ad hoc and system integration test traffic can be directed to components before they are released to production traffic. Front-end code is included in this architecture by using server-side rendering of JS bundles. Environments for integration testing are created with preproduction deploys side by side with production deploys using appropriate levels of isolation. After a successful integration test run, preproduction components are known to work with production precisely as it is. For isolation, test traffic uses staging databases that are copied daily from the production databases, omitting sensitive data. Safety and security concerns are dealt with in a targeted fashion, not monolithically. This architecture scales well with organization size; is more effective for integration testing; and is better aligned with agile business practices than traditional approaches.


title: Pseudo-label refinement using superpixels for semi-supervised brain tumour segmentation

abstract: Training neural networks using limited annotations is an important problem in the medical domain. Deep Neural Networks (DNNs) typically require large, annotated datasets to achieve acceptable performance which, in the medical domain, are especially difficult to obtain as they require significant time from expert radiologists. Semi-supervised learning aims to overcome this problem by learning segmentations with very little annotated data, whilst exploiting large amounts of unlabelled data. However, the best-known technique, which utilises inferred pseudo-labels, is vulnerable to inaccurate pseudo-labels degrading the performance. We propose a framework based on superpixels - meaningful clusters of adjacent pixels - to improve the accuracy of the pseudo labels and address this issue. Our framework combines superpixels with semi-supervised learning, refining the pseudo-labels during training using the features and edges of the superpixel maps. This method is evaluated on a multimodal magnetic resonance imaging (MRI) dataset for the task of brain tumour region segmentation. Our method demonstrates improved performance over the standard semi-supervised pseudo-labelling baseline when there is a reduced annotator burden and only 5 annotated patients are available. We report DSC=0.824 and DSC=0.707 for the test set whole tumour and tumour core regions respectively.


title: Automated Remote Sensing Forest Inventory Using Satellite Imagery

abstract: For many countries like Russia, Canada, or the USA, a robust and detailed tree species inventory is essential to manage their forests sustainably. Since one can not apply unmanned aerial vehicle (UAV) imagery-based approaches to large-scale forest inventory applications, the utilization of machine learning algorithms on satellite imagery is a rising topic of research. Although satellite imagery quality is relatively low, additional spectral channels provide a sufficient amount of information for tree crown classification tasks. Assuming that tree crowns are detected already, we use embeddings of tree crowns generated by Autoencoders as a data set to train classical Machine Learning algorithms. We compare our Autoencoder (AE) based approach to traditional convolutional neural networks (CNN) end-to-end classifiers.


title: n-stage Latent Dirichlet Allocation: A Novel Approach for LDA

abstract: Nowadays, data analysis has become a problem as the amount of data is constantly increasing. In order to overcome this problem in textual data, many models and methods are used in natural language processing. The topic modeling field is one of these methods. Topic modeling allows determining the semantic structure of a text document. Latent Dirichlet Allocation (LDA) is the most common method among topic modeling methods. In this article, the proposed n-stage LDA method, which can enable the LDA method to be used more effectively, is explained in detail. The positive effect of the method has been demonstrated by the applied English and Turkish studies. Since the method focuses on reducing the word count in the dictionary, it can be used language-independently. You can access the open-source code of the method and the example: https://github.com/anil1055/n-stage_LDA


title: Self-stabilizing Byzantine- and Intrusion-tolerant Consensus

abstract: One of the most celebrated problems of fault-tolerant distributed computing is the consensus problem. It was shown to abstract a myriad of problems in which processes have to agree on a single value. Consensus applications include fundamental services for the environments of the Cloud or Blockchain. In such challenging environments, malicious behavior is often modeled as adversarial Byzantine faults. At OPODIS 2010, Moste}faoui and Raynal, in short, MR, presented a Byzantine- and intrusion-tolerant solution to consensus in which the decided value cannot be a value proposed only by Byzantine processes. In addition to this validity property, MR has optimal resilience since it can deal with up to t < n/3 Byzantine processes, where n is the number of processes. We note that MR provides this multivalued consensus object (which accepts proposals taken from a set with a finite number of values) assuming the availability of a single Binary consensus object (which accepts proposals taken from the set {0,1}). This work, which focuses on multivalued consensus, aims at the design of an even more robust solution than MR. Our proposal expands MR's fault-model with self-stabilization, a vigorous notion of fault-tolerance. In addition to tolerating Byzantine and communication failures, self-stabilizing systems can automatically recover after the occurrence of arbitrary transient-faults. These faults represent any violation of the assumptions according to which the system was designed to operate (provided that the algorithm code remains intact). To the best of our knowledge, we propose the first self-stabilizing solution for intrusion-tolerant multivalued consensus for asynchronous message-passing systems prone to Byzantine failures.


title: First principles design of 2 dimensional Nickel dichalcogenide Janus materials NiXY (X,Y=S,Se,Te)

abstract: In this work, we propose novel two-dimensional (2D) Janus Ni dichalcogenide materials and explore their feasibility, stability and evaluate their electronic and optical properties with ab-initio calculations. Three unique Janus materials, namely NiSSe, NiSTe and NiSeTe, based on the 2H hexagonal polytype of NiS2, NiSe2 and NiTe2 were proposed. Density functional theory (DFT) calculations, show that among the three proposed NiXY Janus 2D materials, NiSSe had the best energetic and dynamical stability. GGA PBE calculations showed NiSSe to have a semi-metallic bandstructure with the Ni-Se interaction having a dominant role in the band profile near the Fermi energy. Electron localization function (ELF) and total potential plots show a distinguishable asymmetry in terms of valence electron localization and distribution between the S and Se atoms in 2D NiSSe. The presence of large amount of electron gas like feature in the ELF around the chalcogen atoms also indicates their importance in the conduction properties. Optical properties calculated with random phase approximation (RPA) show the 2D NiSSe to have broad spectrum optical response with significant peaks lying in each of the infra-red, visible and the ultraviolet range of the spectra.


title: Higgs scalar potential coupled to gravity in the exponential parametrization in arbitrary gauge

abstract: We study the parametrization and gauge dependences in the Higgs field coupled to gravity in the context of asymptotic safety. We use the exponential parametrization to derive the fixed points for the cosmological constant, Planck mass, Higgs mass and its coupling, keeping arbitrary gauge parameters $\alpha$ and $\beta$, and compare the results with the linear split. We find that the beta functions for the Higgs potential are expressed in terms of redefined Planck mass such that the apparent gauge dependence is absent. Only the trace mode of the gravity fluctuations couples to the Higgs potential and it tends to decouple in the large $\beta$ limit, but the anomalous dimension becomes large, invalidating the local potential approximation. This gives the limitation of the exponential parametrization. There are also singularities for some values of the gauge parameters but well away from these, we find rather stable fixed points and critical exponents. We thus find that there are regions for the gauge parameters to give stable fixed points and critical exponents against the change of gauge parameters. The Higgs coupling is confirmed to be irrelevant for the reasonable choice of gauge parameters.


title: A MIMO Radar-based Few-Shot Learning Approach for Human-ID

abstract: Radar for deep learning-based human identification has become a research area of increasing interest. It has been shown that micro-Doppler ($\mu$-D) can reflect the walking behavior through capturing the periodic limbs' micro-motions. One of the main aspects is maximizing the number of included classes while considering the real-time and training dataset size constraints. In this paper, a multiple-input-multiple-output (MIMO) radar is used to formulate micro-motion spectrograms of the elevation angular velocity ($\mu$-$\omega$). The effectiveness of concatenating this newly-formulated spectrogram with the commonly used $\mu$-D is investigated. To accommodate for non-constrained real walking motion, an adaptive cycle segmentation framework is utilized and a metric learning network is trained on half gait cycles ($\approx$ 0.5 s). Studies on the effects of various numbers of classes (5--20), different dataset sizes, and varying observation time windows 1--2 s are conducted. A non-constrained walking dataset of 22 subjects is collected with different aspect angles with respect to the radar. The proposed few-shot learning (FSL) approach achieves a classification error of 11.3 % with only 2 min of training data per subject.


title: On Hofstadter butterfly spectrum: Chern-Simons theory, subband gap mapping, IQHE and FQHE labelling

abstract: The magnetic field affects the Bloch band structure in a couple of ways. First it breaks the Bloch band into magnetic subbands or the Landau levels are broadened into magnetic Bloch bands. The resulting group of subbands in the central portion of the energy scale is associated with the integer quantum Hall effect (IQHE). Then at high fields it changes the integrated density of states of the remaining lowest and topmost subband, respectively, which can be associated with fractional quantum Hall effect (FQHE). Here, we employ the Maxwell Chern-Simons gauge theory to formulate the subband-gap mapping algorithm and to construct the butterfly profile of the Hofstadter spectrum. The two regions in the spectrum responsible for the IQHE are identified. At very high magnetic fields the highest and lowest subband are affected by magnetic-field induced restructuring of the integrated density of states in each subband, respectively. The resulting transformation of each of the two subband is responsible for the FQHE. Thus, in the central regions of the energy scale, the principal group of subbands defined by the gap mapping is responsible for the IQHE. The fine structure of the topmost and lowest subband, which convey an iterative nature of the magnetic spectrum is a result of a hierarchical scaling and restructuring by the magnetic fields on the integrated density of states in each respective subband and is responsible for the FQHE.


title: HD207897 b: A dense sub-Neptune transiting a nearby and bright K-type star

abstract: We present the discovery and characterization of a transiting sub-Neptune orbiting with a 16.20 day period around a nearby (28 pc) and bright(V=8.37) K0V star HD207897 (TOI-1611). This discovery is based on photometric measurements from the Transiting Exoplanet Survey Satellite(TESS) mission and radial velocity (RV) observations from the SOPHIE, Automated Planet Finder (APF) and HIRES high precision spectrographs. We used EXOFASTv2 for simultaneously modeling the parameters of the planet and its host star, combining photometric and RV data to determine the planetary system parameters. We show that the planet has a radius of 2.50+/-0.08 RE and a mass of either 14.4+/-1.6 ME or 15.9+/-1.6 ME with nearly equal probability; the two solutions correspond to two possibilities for the stellar activity period. Hence, the density is either 5.1+/-0.7 g cm^-3 or 5.5^{+0.8}_{-0.7} g cm^-3, making it one of the relatively rare dense sub-Neptunes. The existence of such a dense planet at only 0.12 AU from its host star is unusual in the currently observed sub-Neptune (2<RE<4) population. The most likely scenario is that this planet has migrated to its current position.


title: A Variational Bayesian Approach to Learning Latent Variables for Acoustic Knowledge Transfer

abstract: We propose a variational Bayesian (VB) approach to learning distributions of latent variables in deep neural network (DNN) models for cross-domain knowledge transfer, to address acoustic mismatches between training and testing conditions. Instead of carrying out point estimation in conventional maximum a posteriori estimation with a risk of having a curse of dimensionality in estimating a huge number of model parameters, we focus our attention on estimating a manageable number of latent variables of DNNs via a VB inference framework. To accomplish model transfer, knowledge learnt from a source domain is encoded in prior distributions of latent variables and optimally combined, in a Bayesian sense, with a small set of adaptation data from a target domain to approximate the corresponding posterior distributions. Experimental results on device adaptation in acoustic scene classification show that our proposed VB approach can obtain good improvements on target devices, and consistently outperforms 13 state-of-the-art knowledge transfer algorithms.


title: Mapping illegal waste dumping sites with neural-network classification of satellite imagery

abstract: Public health and habitat quality are crucial goals of urban planning. In recent years, the severe social and environmental impact of illegal waste dumping sites has made them one of the most serious problems faced by cities in the Global South, in a context of scarce information available for decision making. To help identify the location of dumping sites and track their evolution over time we adopt a data-driven model from the machine learning domain, analyzing satellite images. This allows us to take advantage of the increasing availability of geo-spatial open-data, high-resolution satellite imagery, and open source tools to train machine learning algorithms with a small set of known waste dumping sites in Buenos Aires, and then predict the location of other sites over vast areas at high speed and low cost. This case study shows the results of a collaboration between Dymaxion Labs and Fundaci\'on Bunge y Born to harness this technique in order to create a comprehensive map of potential locations of illegal waste dumping sites in the region.


title: PDMM: A novel Primal-Dual Majorization-Minimization algorithm for Poisson Phase-Retrieval problem

abstract: In this paper, we introduce a novel iterative algorithm for the problem of phase-retrieval where the measurements consist of only the magnitude of linear function of the unknown signal, and the noise in the measurements follow Poisson distribution. The proposed algorithm is based on the principle of majorization-minimization (MM); however, the application of MM here is very novel and distinct from the way MM has been usually used to solve optimization problems in the literature. More precisely, we reformulate the original minimization problem into a saddle point problem by invoking Fenchel dual representation of the log (.) term in the Poisson likelihood function. We then propose tighter surrogate functions over both primal and dual variables resulting in a double-loop MM algorithm, which we have named as Primal-Dual Majorization-Minimization (PDMM) algorithm. The iterative steps of the resulting algorithm are simple to implement and involve only computing matrix vector products. We also extend our algorithm to handle various L1 regularized Poisson phase-retrieval problems (which exploit sparsity). The proposed algorithm is compared with previously proposed algorithms such as wirtinger flow (WF), MM (conventional), and alternating direction methods of multipliers (ADMM) for the Poisson data model. The simulation results under different experimental settings show that PDMM is faster than the competing methods, and its performance in recovering the original signal is at par with the state-of-the-art algorithms.


title: Magnetic-field effect in high-order above-threshold ionization

abstract: We experimentally and theoretically investigate the influence of the magnetic component of an electromagnetic field on high-order above-threshold ionization of xenon atoms driven by ultrashort femtosecond laser pulses. The nondipole shift of the electron momentum distribution along the light-propagation direction for high energy electrons beyond the classical cutoff is found to be vastly different from that below the cutoff. A V-shape structure in the momentum dependence of the nondipole shift above the cutoff is identified for the first time. With the help of classical and quantum-orbit analysis, we show that large-angle rescattering of the electrons strongly alters the partitioning of the photon momentum between electron and ion. The sensitivity of the observed nondipole shift to the electronic structure of the target atom is confirmed by three-dimensional time-dependent Schr\"odinger equation simulations for different model potentials.


title: Measure of distance and overlap between two arbitrary ellipses on a sphere

abstract: Various packing problems and simulations of hard and soft interacting particles, such as microscopic models of nematic liquid crystals, reduce to calculations of intersections and pair interactions between ellipsoids. When constrained to a spherical surface, curvature and compactness lead to nontrivial behavior that finds uses in physics, computer science and geometry. A well-known idealized isotropic example is the Tammes problem of finding optimal non-intersecting packings of equal hard disks. The anisotropic case of elliptic particles remains, on the other hand, comparatively unexplored. We develop an algorithm to detect collisions between ellipses constrained to the two-dimensional surface of a sphere based on a solution of an eigenvalue problem. We investigate and discuss topologically distinct ways two ellipses may touch or intersect on a sphere, and define a contact function that can be used for construction of short- and long-range pair potentials.


title: Networks of Queues Models with Several Classes of Customers and Exponential Service Times

abstract: The main target of this paper is to present the Markov chain C that, not giving explicitly the queue lengths stationary probabilities, has the necessary information to its determination for open networks of queues with several classes of customers and exponential service times, allowing to overcome ingeniously this problem. The situation for closed networks, in the same conditions, much easier is also presented.


title: LSA: Modeling Aspect Sentiment Coherency via Local Sentiment Aggregation

abstract: Aspect sentiment coherency is an intriguing yet underexplored topic in the field of aspect-based sentiment classification. This concept reflects the common pattern where adjacent aspects often share similar sentiments. Despite its prevalence, current studies have not fully recognized the potential of modeling aspect sentiment coherency, including its implications in adversarial defense. To model aspect sentiment coherency, we propose a novel local sentiment aggregation (LSA) paradigm based on constructing a differential-weighted sentiment aggregation window. We have rigorously evaluated our model through experiments, and the results affirm the proficiency of LSA in terms of aspect coherency prediction and aspect sentiment classification. For instance, it outperforms existing models and achieves state-of-the-art sentiment classification performance across five public datasets. Furthermore, we demonstrate the promising ability of LSA in ABSC adversarial defense, thanks to its sentiment coherency modeling. To encourage further exploration and application of this concept, we have made our code publicly accessible. This will provide researchers with a valuable tool to delve into sentiment coherency modeling in future research.


title: Statistics in everyone's backyard: an impact study via citation network analysis

abstract: The increasing availability of curated citation data provides a wealth of resources for analyzing and understanding the intellectual influence of scientific publications. In the field of statistics, current studies of citation data have mostly focused on the interactions between statistical journals and papers, limiting the measure of influence to mainly within statistics itself. In this paper, we take the first step towards understanding the impact statistics has made on other scientific fields in the era of Big Data. By collecting comprehensive bibliometric data from the Web of Science database for selected statistical journals, we investigate the citation trends and compositions of citing fields over time to show that their diversity has been increasing. Furthermore, we use the local clustering technique involving personalized PageRank with conductance for size selection to find the most relevant statistical research area for a given external topic of interest. We provide theoretical guarantees for the procedure and, through a number of case studies, show the results from our citation data align well with our knowledge and intuition about these external topics. Overall, we have found that the statistical theory and methods recently invented by the statistics community have made increasing impact on other scientific fields.


title: Lattices of t-structures and thick subcategories for discrete cluster categories

abstract: We classify t-structures and thick subcategories in discrete cluster categories $\mathcal{C}(\mathcal{Z})$ of Dynkin type $A$, and show that the set of all t-structures on $\mathcal{C}(\mathcal{Z})$ is a lattice under inclusion of aisles, with meet given by their intersection. We show that both the lattice of t-structures on $\mathcal{C}(\mathcal{Z})$ obtained in this way and the lattice of thick subcategories of $\mathcal{C}(\mathcal{Z})$ are intimately related to the lattice of non-crossing partitions of type $A$. In particular, the lattice of equivalence classes of non-degenerate t-structures on such a category is isomorphic to the lattice of non-crossing partitions of a finite linearly ordered set.


title: Physics-guided Deep Markov Models for Learning Nonlinear Dynamical Systems with Uncertainty

abstract: In this paper, we propose a probabilistic physics-guided framework, termed Physics-guided Deep Markov Model (PgDMM). The framework targets the inference of the characteristics and latent structure of nonlinear dynamical systems from measurement data, where exact inference of latent variables is typically intractable. A recently surfaced option pertains to leveraging variational inference to perform approximate inference. In such a scheme, transition and emission functions of the system are parameterized via feed-forward neural networks (deep generative models). However, due to the generalized and highly versatile formulation of neural network functions, the learned latent space often lacks physical interpretation and structured representation. To address this, we bridge physics-based state space models with Deep Markov Models, thus delivering a hybrid modeling framework for unsupervised learning and identification of nonlinear dynamical systems. The proposed framework takes advantage of the expressive power of deep learning, while retaining the driving physics of the dynamical system by imposing physics-driven restrictions on the side of the latent space. We demonstrate the benefits of such a fusion in terms of achieving improved performance on illustrative simulation examples and experimental case studies of nonlinear systems. Our results indicate that the physics-based models involved in the employed transition and emission functions essentially enforce a more structured and physically interpretable latent space, which is essential for enhancing and generalizing the predictive capabilities of deep learning-based models.