title: A Late Multi-Modal Fusion Model for Detecting Hybrid Spam E-mail

abstract: In recent years, spammers are now trying to obfuscate their intents by introducing hybrid spam e-mail combining both image and text parts, which is more challenging to detect in comparison to e-mails containing text or image only. The motivation behind this research is to design an effective approach filtering out hybrid spam e-mails to avoid situations where traditional text-based or image-baesd only filters fail to detect hybrid spam e-mails. To the best of our knowledge, a few studies have been conducted with the goal of detecting hybrid spam e-mails. Ordinarily, Optical Character Recognition (OCR) technology is used to eliminate the image parts of spam by transforming images into text. However, the research questions are that although OCR scanning is a very successful technique in processing text-and-image hybrid spam, it is not an effective solution for dealing with huge quantities due to the CPU power required and the execution time it takes to scan e-mail files. And the OCR techniques are not always reliable in the transformation processes. To address such problems, we propose new late multi-modal fusion training frameworks for a text-and-image hybrid spam e-mail filtering system compared to the classical early fusion detection frameworks based on the OCR method. Convolutional Neural Network (CNN) and Continuous Bag of Words were implemented to extract features from image and text parts of hybrid spam respectively, whereas generated features were fed to sigmoid layer and Machine Learning based classifiers including Random Forest (RF), Decision Tree (DT), Naive Bayes (NB) and Support Vector Machine (SVM) to determine the e-mail ham or spam.


title: A new thin layer model for viscous flow between two nearby non-static surfaces

abstract: We propose a two-dimensional flow model of a viscous fluid between two close moving surfaces. We show, using a formal asymptotic expansion of the solution, that its asymptotic behavior, when the distance between the two surfaces tends to zero, is the same as that of the the Navier-Stokes equations. The leading term of the formal asymptotic expansions of the solutions to the new model and Navier-Stokes equations are solution of the same limit problem, and the type of the limit problem depends on the boundary conditions. If slip velocity boundary conditions are imposed on the upper and lower bound surfaces, the limit is a solution of a lubrication model, but if the tractions and friction forces are known on both bound surfaces, the limit is a solution of a thin fluid layer model. The model proposed has been obtained to be a valuable tool for computing viscous fluid flow between two nearby moving surfaces, without the need to decide a priori whether the flow is typical of a lubrication or a thin fluid layer problem, and without the enormous computational effort that would be required to solve the Navier-Stokes equations in such a thin domain.


title: SemFormer: Semantic Guided Activation Transformer for Weakly Supervised Semantic Segmentation

abstract: Recent mainstream weakly supervised semantic segmentation (WSSS) approaches are mainly based on Class Activation Map (CAM) generated by a CNN (Convolutional Neural Network) based image classifier. In this paper, we propose a novel transformer-based framework, named Semantic Guided Activation Transformer (SemFormer), for WSSS. We design a transformer-based Class-Aware AutoEncoder (CAAE) to extract the class embeddings for the input image and learn class semantics for all classes of the dataset. The class embeddings and learned class semantics are then used to guide the generation of activation maps with four losses, i.e., class-foreground, class-background, activation suppression, and activation complementation loss. Experimental results show that our SemFormer achieves \textbf{74.3}\% mIoU and surpasses many recent mainstream WSSS approaches by a large margin on PASCAL VOC 2012 dataset. Code will be available at \url{https://github.com/JLChen-C/SemFormer}.


title: Environment-Aware AUV Trajectory Design and Resource Management for Multi-Tier Underwater Computing

abstract: The Internet of underwater things (IoUT) is envisioned to be an essential part of maritime activities. Given the IoUT devices' wide-area distribution and constrained transmit power, autonomous underwater vehicles (AUVs) have been widely adopted for collecting and forwarding the data sensed by IoUT devices to the surface-stations. In order to accommodate the diverse requirements of IoUT applications, it is imperative to conceive a multi-tier underwater computing (MTUC) framework by carefully harnessing both the computing and the communications as well as the storage resources of both the surface-station and of the AUVs as well as of the IoUT devices. Furthermore, to meet the stringent energy constraints of the IoUT devices and to reduce the operating cost of the MTUC framework, a joint environment-aware AUV trajectory design and resource management problem is formulated, which is a high-dimensional NP-hard problem. To tackle this challenge, we first transform the problem into a Markov decision process (MDP) and solve it with the aid of the asynchronous advantage actor-critic (A3C) algorithm. Our simulation results demonstrate the superiority of our scheme.


title: Nucleation and kinematics of vortices in stirred Bose Einstein condensates

abstract: We apply the Halperin-Mazenco formalism within the Gross-Pitaevskii theory to characterise the kinematics and nucleation of quantum vortices in a two-dimensional stirred Bose Einstein condensate. We introduce a smooth defect density field measuring the superfluid vorticity and is a topologically conserved quantity. We use this defect density field and its associated current density to study the precursory pattern formations that occur inside the repulsive potential of an obstacle and determine the onset of vortex nucleation and shedding. We demonstrate that phase slips form inside hard potentials even in the absence of vortex nucleation, whereas for soft potentials they occur only above a critical stirring velocity leading to vortex nucleation. The Halperin-Mazenco formalism provides an elegant and accurate method of deriving the point vortex dynamic directly from the Gross-Pitaevskii equation.


title: A new band selection approach based on information theory and support vector machine for hyperspectral images reduction and classification

abstract: The high dimensionality of hyperspectral images consisting of several bands often imposes a big computational challenge for image processing. Therefore, spectral band selection is an essential step for removing the irrelevant, noisy and redundant bands. Consequently increasing the classification accuracy. However, identification of useful bands from hundreds or even thousands of related bands is a nontrivial task. This paper aims at identifying a small set of highly discriminative bands, for improving computational speed and prediction accuracy. Hence, we proposed a new strategy based on joint mutual information to measure the statistical dependence and correlation between the selected bands and evaluate the relative utility of each one to classification. The proposed filter approach is compared to an effective reproduced filters based on mutual information. Simulations results on the hyperpectral image HSI AVIRIS 92AV3C using the SVM classifier have shown that the effective proposed algorithm outperforms the reproduced filters strategy performance. Keywords-Hyperspectral images, Classification, band Selection, Joint Mutual Information, dimensionality reduction ,correlation, SVM.


title: DEMIS: A Threat Model for Selectively Encrypted Visual Surveillance Data

abstract: The monitoring of individuals/objects has become increasingly possible in recent years due to the convenience of integrated cameras in many devices. Due to the important moments or activities of people captured by these devices, it has made it a great asset for attackers to launch attacks against by exploiting the weaknesses in these devices. Different studies proposed na\"ive/selective encryption of the captured visual data for safety but despite the encryption, an attacker can still access or manipulate such data. This paper proposed a novel threat model, DEMIS which helps analyse the threats against such encrypted videos. The paper also examines the attack vectors that can be used for threats and the mitigation that will reduce or prevent the attack. For experiments, firstly the data set is generated by applying selective encryption on the Regions-of-interests (ROI) of the tested videos using the image segmentation technique and Chacha20 cipher. Secondly, different types of attacks, such as inverse, lowercase, uppercase, random insertion, and malleability attacks were simulated in experiments to show the effects of the attacks, the risk matrix, and the severity of these attacks. Our developed data set with the original, selective encrypted, and attacked videos are available on git-repository(https://github.com/Ifeoluwapoo/video-datasets) for future researchers.


title: K3 surfaces with two involutions and low Picard number

abstract: Let $X$ be a complex algebraic K3 surface of degree $2d$ and with Picard number $\rho$. Assume that $X$ admits two commuting involutions: one holomorphic and one anti-holomorphic. In that case, $\rho \geq 1$ when $d=1$ and $\rho \geq 2$ when $d \geq 2$. For $d=1$, the first example defined over $\mathbb{Q}$ with $\rho=1$ was produced already in 2008 by Elsenhans and Jahnel. A K3 surface provided by Kond\={o}, also defined over $\mathbb{Q}$, can be used to realise the minimum $\rho=2$ for all $d\geq 2$. In these notes we construct new explicit examples of K3 surfaces over the rational numbers realising the minimum $\rho=2$ for $d=2,3,4$. We also show that a nodal quartic surface can be used to realise the minimum $\rho=2$ for infinitely many different values of $d$. Finally, we strengthen a result of Morrison by showing that for any even lattice $N$ of rank $1\leq r \leq 10$ and signature $(1,r-1)$ there exists a K3 surface $Y$ defined over $\mathbb{R}$ such that $\textrm{Pic} Y_\mathbb{C}=\textrm{Pic} Y \cong N$.


title: RapidAI4EO: Mono- and Multi-temporal Deep Learning models for Updating the CORINE Land Cover Product

abstract: In the remote sensing community, Land Use Land Cover (LULC) classification with satellite imagery is a main focus of current research activities. Accurate and appropriate LULC classification, however, continues to be a challenging task. In this paper, we evaluate the performance of multi-temporal (monthly time series) compared to mono-temporal (single time step) satellite images for multi-label classification using supervised learning on the RapidAI4EO dataset. As a first step, we trained our CNN model on images at a single time step for multi-label classification, i.e. mono-temporal. We incorporated time-series images using a LSTM model to assess whether or not multi-temporal signals from satellites improves CLC classification. The results demonstrate an improvement of approximately 0.89% in classifying satellite imagery on 15 classes using a multi-temporal approach on monthly time series images compared to the mono-temporal approach. Using features from multi-temporal or mono-temporal images, this work is a step towards an efficient change detection and land monitoring approach.


title: Explanatory Depth in Primordial Cosmology: A Comparative Study of Inflationary and Bouncing Paradigms

abstract: We develop and apply a multi-dimensional account of explanatory depth towards a comparative analysis of inflationary and bouncing paradigms in primordial cosmology. Our analysis builds on earlier work due to Azhar and Loeb (2021) that establishes initial conditions fine-tuning as a dimension of explanatory depth relevant to debates in contemporary cosmology. We propose dynamical fine-tuning and autonomy as two further dimensions of depth in the context of problems with instability and trans-Planckian modes that afflict bouncing and inflationary approaches respectively. In the context of the latter issue, we argue that the recently formulated trans-Planckian censorship conjecture leads to a trade-off for inflationary models between dynamical fine-tuning and autonomy. We conclude with the suggestion that explanatory preference with regard to the different dimensions of depth is best understood in terms of differing attitudes towards heuristics for future model building.


title: Local derivations on the Lie algebra $W(2,2)$

abstract: The present paper is devoted to studying local derivations on the Lie algebra $W(2,2)$ which has some outer derivations. Using some linear algebra methods in \cite{CZZ} and a key construction for $W(2,2)$ we prove that every local derivation on $W(2, 2)$ is a derivation. As an application, we determine all local derivations on the deformed $\mathfrak{bms}_3$ algebra.


title: Channel-Aware Ordered Successive Relaying with Finite-Blocklength Coding

abstract: Successive relaying can improve the transmission rate by allowing the source and relays to transmit messages simultaneously, but it may cause severe inter-relay interference (IRI). IRI cancellation schemes have been proposed to mitigate IRI. However, interference cancellation methods have a high risk of error propagation, resulting in a severe transmission rate loss in finite blocklength regimes. Thus, jointly decoding for successive relaying with finite-blocklength coding (FBC) remains a challenge. In this paper, we present an optimized channel-aware ordered successive relaying protocol with finite-blocklength coding (CAO-SIR-FBC), which can recover the rate loss by carefully adapting the relay transmission order and rate. We analyze the average throughput of the CAO-SIR-FBC method, based on which a closed-form expression in a high signal-to-noise regime (SNR) is presented. Average throughput analysis and simulations show that CAO-SIR-FBC outperforms conventional two-timeslot half-duplex relaying in terms of spectral efficiency.


title: Provable Sample-Efficient Sparse Phase Retrieval Initialized by Truncated Power Method

abstract: We study the sparse phase retrieval problem, recovering an $s$-sparse length-$n$ signal from $m$ magnitude-only measurements. Two-stage non-convex approaches have drawn much attention in recent studies for this problem. Despite non-convexity, many two-stage algorithms provably converge to the underlying solution linearly when appropriately initialized. However, in terms of sample complexity, the bottleneck of those algorithms often comes from the initialization stage. Although the refinement stage usually needs only $m=\Omega(s\log n)$ measurements, the widely used spectral initialization in the initialization stage requires $m=\Omega(s^2\log n)$ measurements to produce a desired initial guess, which causes the total sample complexity order-wisely more than necessary. To reduce the number of measurements, we propose a truncated power method to replace the spectral initialization for non-convex sparse phase retrieval algorithms. We prove that $m=\Omega(\bar{s} s\log n)$ measurements, where $\bar{s}$ is the stable sparsity of the underlying signal, are sufficient to produce a desired initial guess. When the underlying signal contains only very few significant components, the sample complexity of the proposed algorithm is $m=\Omega(s\log n)$ and optimal. Numerical experiments illustrate that the proposed method is more sample-efficient than state-of-the-art algorithms.


title: Highly unbreakable graph with a fixed excluded minor are almost rigid

abstract: A set $X \subseteq V(G)$ in a graph $G$ is $(q,k)$-unbreakable if every separation $(A,B)$ of order at most $k$ in $G$ satisfies $|A \cap X| \leq q$ or $|B \cap X| \leq q$. In this paper, we prove the following result: If a graph $G$ excludes a fixed complete graph $K_h$ as a minor and satisfies certain unbreakability guarantees, then $G$ is almost rigid in the following sense: the vertices of $G$ can be partitioned in an isomorphism-invariant way into a part inducing a graph of bounded treewidth and a part that admits a small isomorphism-invariant family of labelings. This result is the key ingredient in the fixed-parameter algorithm for Graph Isomorphism parameterized by the Hadwiger number of the graph, which is presented in a companion paper.


title: Orders On Free Metabelian Groups

abstract: A bi-order on a group $G$ is a total, bi-multiplication invariant order. A subset $S$ in an ordered group $(G,\leqslant)$ is convex if for all $f\leqslant g$ in $S$, every element $h\in G$ satisfying $f\leqslant h \leqslant g$ belongs to $S$. In this paper, we show that the derived subgroup of the free metabelian group of rank 2 is convex with respect to any bi-order. Moreover, we study the convex hull of the derived subgroup of a free metabelian group of higher rank. As an application, we prove that the space of bi-order of non-abelian free metabelian group of finite rank is homeomorphic to the Cantor set. In addition, we show that no bi-order for these groups can be recognised by a regular language.


title: The Influence of Payment Method: Do Consumers Pay More with Mobile Payment?

abstract: The introduction of new payment methods has resulted in one of the most significant changes in the way we consume goods and services. In this paper, I present results of a field and a laboratory experiment designed to determine the effect of payment method (cash vs. mobile payment) on spending, and a meta-analysis of previous literature about payment method effect. In the field experiment, I collected cashier receipts from Chinese supermarkets. Compared to cash payment, mobile payments significantly increased the amount purchased and the average amount spent on each item. This effect was found to be particularly large for high price elasticity goods. In the laboratory experiment, participants were randomly assigned to one of four groups that varied with respect to the kind of payment and the kind of incentives, eliminating the potential endogeneity problem from the field experiment. I found that compared to cash, mobile payments lead to a significantly higher willingness to pay (WTP) for consumption. In contrast to others, I found that pain of paying does not moderate the payment method effect; however, other psychological factors were found to work as potential mechanisms for affecting WTP.


title: Cover Reproducible Steganography via Deep Generative Models

abstract: Whereas cryptography easily arouses attacks by means of encrypting a secret message into a suspicious form, steganography is advantageous for its resilience to attacks by concealing the message in an innocent-looking cover signal. Minimal distortion steganography, one of the mainstream steganography frameworks, embeds messages while minimizing the distortion caused by the modification on the cover elements. Due to the unavailability of the original cover signal for the receiver, message embedding is realized by finding the coset leader of the syndrome function of steganographic codes migrated from channel coding, which is complex and has limited performance. Fortunately, deep generative models and the robust semantic of generated data make it possible for the receiver to perfectly reproduce the cover signal from the stego signal. With this advantage, we propose cover-reproducible steganography where the source coding, e.g., arithmetic coding, serves as the steganographic code. Specifically, the decoding process of arithmetic coding is used for message embedding and its encoding process is regarded as message extraction. Taking text-to-speech and text-to-image synthesis tasks as two examples, we illustrate the feasibility of cover-reproducible steganography. Steganalysis experiments and theoretical analysis are conducted to demonstrate that the proposed methods outperform the existing methods in most cases.


title: Graph Filter Transfer via Probability Density Ratio Weighting

abstract: The problem of recovering graph signals is one of the main topics in graph signal processing. A representative approach to this problem is the graph Wiener filter, which utilizes the statistical information of the target signal computed from historical data to construct an effective estimator. However, we often encounter situations where the current graph differs from that of historical data due to topology changes, leading to performance degradation of the estimator. This paper proposes a graph filter transfer method, which learns an effective estimator from historical data under topology changes. The proposed method leverages the probability density ratio of the current and historical observations and constructs an estimator that minimizes the reconstruction error in the current graph domain. The experiment on synthetic data demonstrates that the proposed method outperforms other methods.


title: Evolution of pits at the surface of 67P/Churyumov-Gerasimenko

abstract: The observation of pits at the surface of comets offers the opportunity to take a glimpse into the properties and the mechanisms that shape a nucleus through cometary activity. If the origin of these pits is still a matter of debate, multiple studies have recently suggested that known phase transitions alone could not have carved these morphological features on the surface of 67P/C-G. We want to understand how the progressive modification of 67P's surface due to cometary activity might have affected the characteristics of pits. In particular, we aim to understand whether signatures of the formation mechanism of these morphological features can still be identified. To quantify the amount of erosion sustained at the surface of 67P since it arrived on its currently observed orbit, we selected 380 facets of a medium-resolution shape model of the nucleus, sampling 30 pits across the surface. We computed the surface energy balance with a high temporal resolution, including shadowing and self-heating. We then applied a thermal evolution model to assess the amount of erosion sustained after ten orbital revolutions under current illumination conditions. We find that the maximum erosion sustained after ten orbital revolutions is on the order of 80 m, for facets located in the southern hemisphere. We thus confirm that progressive erosion cannot form pits and alcoves, as local erosion is much lower than their observed depth and diameter. We find that plateaus tend to erode more than bottoms, especially for the deepest depressions, and that some differential erosion can affect their morphology. As a general rule, our results suggest that sharp morphological features tend to be erased by progressive erosion. This study supports the assumption that deep circular pits, such as Seth1, are the least processed morphological features at the surface of 67P, or the best preserved since their formation.


title: Dramatic Plasmon Response to the Charge-Density-Wave Gap Development in $1\textit{T}-{\mathrm{TiSe}}_{2}$

abstract: 1T-TiSe2 is one of the most studied charge density wave (CDW) systems, not only because of its peculiar properties related to the CDW transition, but also due to its status as a promising candidate of exciton insulator signaled by the proposed plasmon softening at the CDW wave vector. Using high-resolution electron energy loss spectroscopy, we report a systematic study of the temperature-dependent plasmon behaviors of 1T-TiSe2. We unambiguously resolve the plasmon from phonon modes, revealing the existence of Landau damping to the plasmon at finite momentums, which does not support the plasmon softening picture for exciton condensation. Moreover, we discover that the plasmon lifetime at zero momentum responds dramatically to the bandgap evolution associated with the CDW transition. The interband transitions near the Fermi energy in the normal phase is demonstrated serving as a strong damping channel of plasmons, while such a channel in the CDW phase is suppressed due to the CDW gap opening, which results in the dramatic tunability of the plasmon in semimetals or small-gap semiconductors.


title: Fast Yet Effective Speech Emotion Recognition with Self-distillation

abstract: Speech emotion recognition (SER) is the task of recognising human's emotional states from speech. SER is extremely prevalent in helping dialogue systems to truly understand our emotions and become a trustworthy human conversational partner. Due to the lengthy nature of speech, SER also suffers from the lack of abundant labelled data for powerful models like deep neural networks. Pre-trained complex models on large-scale speech datasets have been successfully applied to SER via transfer learning. However, fine-tuning complex models still requires large memory space and results in low inference efficiency. In this paper, we argue achieving a fast yet effective SER is possible with self-distillation, a method of simultaneously fine-tuning a pretrained model and training shallower versions of itself. The benefits of our self-distillation framework are threefold: (1) the adoption of self-distillation method upon the acoustic modality breaks through the limited ground-truth of speech data, and outperforms the existing models' performance on an SER dataset; (2) executing powerful models at different depth can achieve adaptive accuracy-efficiency trade-offs on resource-limited edge devices; (3) a new fine-tuning process rather than training from scratch for self-distillation leads to faster learning time and the state-of-the-art accuracy on data with small quantities of label information.


title: Far Ultra-Violet Insights Into NGC 1399's Globular Cluster Population

abstract: We investigate archival Hubble Space Telescope ACS/SBC F140LP observations of NGC~1399 to search for evidence of multiple stellar populations in extragalactic globular clusters. Enhanced FUV populations are thought to be indicators of He-enhanced second generation populations in globular clusters, specifically extreme/blue horizontal branch stars. Out of 149 globular clusters in the field of view, 58 have far ultraviolet (FUV) counterparts with magnitudes brighter than 28.5. Six of these FUV-deteced globular clusters are also detected in X-rays, including one ultraluminous X-ray source ($L_X > 10^{39}$ erg/s). While optically bright clusters corresponded to brighter FUV counterparts, we observe FUV emission from both metal-rich and metal-poor clusters, which implies that the FUV excess is not dependent on optical colour. We also find no evidence that the cluster size influences the FUV emission. The clusters with X-ray emission are not unusually FUV bright, which suggests that even the ultraluminous X-ray source does not provide significant FUV contributions. NGC 1399 is only the fourth galaxy to have its globular cluster system probed for evidence of FUV-enhanced populations, and we compare these clusters to previous studies of the Milky Way, M31, M87, and the brightest cluster in M81. These sources indicate that many globular clusters likely host extreme HB stars and/or second generation stars, and highlight the need for more complete FUV observations of extragalactic globular cluster systems.


title: Fixed-parameter tractability of Graph Isomorphism in graphs with an excluded minor

abstract: We prove that Graph Isomorphism and Canonization in graphs excluding a fixed graph $H$ as a minor can be solved by an algorithm working in time $f(H)\cdot n^{O(1)}$, where $f$ is some function. In other words, we show that these problems are fixed-parameter tractable when parameterized by the size of the excluded minor, with the caveat that the bound on the running time is not necessarily computable. The underlying approach is based on decomposing the graph in a canonical way into unbreakable (intuitively, well-connected) parts, which essentially provides a reduction to the case where the given $H$-minor-free graph is unbreakable itself. This is complemented by an analysis of unbreakable $H$-minor-free graphs, performed in a second subordinate manuscript, which reveals that every such graph can be canonically decomposed into a part that admits few automorphisms and a part that has bounded treewidth.


title: Stau searches and measurements with the ILD concept at the International Linear Collider

abstract: One of the most interesting channels to search for SUSY is the direct pair-production of the $\tau$-lepton superpartner, $\widetilde{\tau}$. The $\widetilde{\tau}$ is with high probability the lightest of the scalar leptons, so one of the first SUSY particles that can be observerd, and the signature of $\widetilde{\tau}$ pair production signal events is one of the most difficult ones, yielding to the ``worst'' and so most global scenario for the searches. Analysis performed at LEP set the current model-independent $\widetilde{\tau}$ limits, suffering from the low energy of this facility. Only under strong model assumptions, these limits are extended to higher masses by LHC studies. In this contribution we show the capability of the ILC, a future electron-positron collider with energy up to 1 TeV, for determining $\widetilde{\tau}$ exclusion/discovery limits in a model-independent way, including an overview of the current state-of-the-art. The determination of the ``worst'' scenario for $\widetilde{\tau}$ exclusion/discovery, taking into account the effect of the $\widetilde{\tau}$ mixing on $\widetilde{\tau}$ production cross-section and efficiency, is also presented. For selected benchmarks, the prospect for measuring masses and polarised cross-sections will be shown. The studies were done studying events passed through the full detector simulation and reconstruction procedures of the International Large Detector (ILD) concept at the ILC. The simulation included all SM backgrounds, as well as the machine induced ones.


title: HSVI can solve zero-sum Partially Observable Stochastic Games

abstract: State-of-the-art methods for solving 2-player zero-sum imperfect information games rely on linear programming or regret minimization, though not on dynamic programming (DP) or heuristic search (HS), while the latter are often at the core of state-of-the-art solvers for other sequential decision-making problems. In partially observable or collaborative settings (e.g., POMDPs and Dec- POMDPs), DP and HS require introducing an appropriate statistic that induces a fully observable problem as well as bounding (convex) approximators of the optimal value function. This approach has succeeded in some subclasses of 2-player zero-sum partially observable stochastic games (zs- POSGs) as well, but how to apply it in the general case still remains an open question. We answer it by (i) rigorously defining an equivalent game to work with, (ii) proving mathematical properties of the optimal value function that allow deriving bounds that come with solution strategies, (iii) proposing for the first time an HSVI-like solver that provably converges to an $\epsilon$-optimal solution in finite time, and (iv) empirically analyzing it. This opens the door to a novel family of promising approaches complementing those relying on linear programming or iterative methods.


title: The intrinsic electrostatic dielectric behaviour of graphite anodes in Li-ion batteries -- across the entire functional range of charge

abstract: Lithium-graphite intercalation compounds (Li-GICs) are the most common anode material for modern Li-ion batteries. However, the dielectric response of this material in the electrostatic limit (and its variation with the state of charge (SOC)) has not been investigated to a satisfactory degree, especially not for the higher SOC range. Nevertheless, said dielectric behaviour is a highly desired property, particularly as an input parameter for charged kinetic Monte Carlo simulations, one of the most promising modeling techniques for energy materials. In this work, we employ our recent DFTB parametrization for Li-GICs based on a machine-learned repulsive potential in order to overcome the computational hurdles of sampling the long-ranged Coulomb interactions within this material, as experienced by the charge carriers within. This approach is rather novel due to computational cost, but best suited for investigating our specific property of interest. For the first time, we discover a mostly linear dependency of the relative permittivity on the SOC, from ca. 7 at SOC 0% to ca. 25 at SOC 100%. In doing so, we also present a straightforward approach that can be used in future research for other intercalation compounds -- once sufficiently fast and long-ranged computational methods, e.g. linear-scaling DFT, a good DFTB parametrization, or atomic potentials with inbuilt electrostatics, become available. However, while the presented qualitative behaviour is robust and our results compare favourably with the few experimental studies available, we stress that quantitative results are strongly dependent on our estimation of the partial charge transfer from the intercalated Li-ion to the carbon host structure, and need to be verified by further experiments and calculations. Yet, our research shows that in principle, two measurements -- one at low and one at high SOC -- should suffice for that purpose.


title: Exploring Data-Driven Chemical SMILES Tokenization Approaches to Identify Key Protein-Ligand Binding Moieties

abstract: Machine learning models have found numerous successful applications in computational drug discovery. A large body of these models represents molecules as sequences since molecular sequences are easily available, simple, and informative. The sequence-based models often segment molecular sequences into pieces called chemical words (analogous to the words that make up sentences in human languages) and then apply advanced natural language processing techniques for tasks such as $\textit{de novo}$ drug design, property prediction, and binding affinity prediction. However, the chemical characteristics and significance of these building blocks, chemical words, remain unexplored. This study aims to investigate the chemical vocabularies generated by popular subword tokenization algorithms, namely Byte Pair Encoding (BPE), WordPiece, and Unigram, and identify key chemical words associated with protein-ligand binding. To this end, we build a language-inspired pipeline that treats high affinity ligands of protein targets as documents and selects key chemical words making up those ligands based on tf-idf weighting. Further, we conduct case studies on a number of protein families to analyze the impact of key chemical words on binding. Through our analysis, we find that these key chemical words are specific to protein targets and correspond to known pharmacophores and functional groups. Our findings will help shed light on the chemistry captured by the chemical words, and by machine learning models for drug discovery at large.


title: Generic Properties of First Order Mean Field Games

abstract: We consider a class of deterministic mean field games, where the state associated with each player evolves according to an ODE which is linear w.r.t. the control. Existence, uniqueness, and stability of solutions are studied from the point of view of generic theory. Within a suitable topological space of dynamics and cost functionals, we prove that, for nearly all mean field games(in the Baire category sense) the best reply map is single valued for a.e. player. As a consequence, the mean field game admits a strong (not randomized) solution. Examples are given of open sets of games admitting a single solution, and other open sets admitting multiple solutions. Further examples show the existence of an open set of MFG having a unique solution which is asymptotically stable w.r.t. the best reply map, and another open set of MFG having a unique solution which is unstable. We conclude with an example of a MFG with terminal constraints which does not have any solution, not even in the mild sense with randomized strategies.


title: Speaker Diarization Based on Multi-channel Microphone Array in Small-scale Meeting

abstract: In the task of speaker diarization, the number of small-scale meetings accounts for a large proportion. When microphone arrays are employed as a recording device, its spatial information is usually ignored by most researchers. In this paper, inspired by the clustering method combining d-vector and microphone array spatial vector, we proposed a diarization method which using multi-channel microphone arrays for a meeting with no more than 4 speakers. We utilize speech enhancement to preprocess the audio from the microphone array. The Steered-Response Power Phase Transform (SRP-PHAT) algorithm are employed to get more accurate speakers, and apply the number of speakers to recluster the speech segments to achieve better performance. Finally, we fuse our system by DOVER-LAP to get the best result. We evaluated our system on the AMI corpus. Compared with the best experimental results so far, our system has achieved largely improvement in the diarization error rate (DER).


title: Super-Resolution Based Patch-Free 3D Image Segmentation with High-Frequency Guidance

abstract: High resolution (HR) 3D images are widely used nowadays, such as medical images like Magnetic Resonance Imaging (MRI) and Computed Tomography (CT). However, segmentation of these 3D images remains a challenge due to their high spatial resolution and dimensionality in contrast to currently limited GPU memory. Therefore, most existing 3D image segmentation methods use patch-based models, which have low inference efficiency and ignore global contextual information. To address these problems, we propose a super-resolution (SR) based patch-free 3D image segmentation framework that can realize HR segmentation from a global-wise low-resolution (LR) input. The framework contains two sub-tasks, of which semantic segmentation is the main task and super resolution is an auxiliary task aiding in rebuilding the high frequency information from the LR input. To furthermore balance the information loss with the LR input, we propose a High-Frequency Guidance Module (HGM), and design an efficient selective cropping algorithm to crop an HR patch from the original image as restoration guidance for it. In addition, we also propose a Task-Fusion Module (TFM) to exploit the inter connections between segmentation and SR task, realizing joint optimization of the two tasks. When predicting, only the main segmentation task is needed, while other modules can be removed for acceleration. The experimental results on two different datasets show that our framework has a four times higher inference speed compared to traditional patch-based methods, while its performance also surpasses other patch-based and patch-free models.


title: Singularity-free Formation Path Following of Underactuated AUVs: Extended Version

abstract: This paper proposes a method for formation path following control of a fleet of underactuated autonomous underwater vehicles. The proposed method combines several hierarchic tasks in a null space-based behavioral algorithm to safely guide the vehicles. Compared to the existing literature, the algorithm includes both inter-vehicle and obstacle collision avoidance, and employs a scheme that keeps the vehicles within given operation limits. The algorithm is applied to a six degree-of-freedom model, using rotation matrices to describe the attitude to avoid singularities. Using the results of cascaded systems theory, we prove that the closed-loop system is uniformly semiglobally exponentially stable. We use numerical simulations to validate the results.


title: Energetics of a Microscopic Feynman Ratchet

abstract: A general formalism is derived describing both dynamical and energetic properties of a microscopic Feynman ratchet. Work and heat flows are given as a series expansion in the thermodynamic forces, obtaining analytical expressions for the (non)linear response coefficients. Our results extend previously obtained expressions in the context of a chiral heat pump.


title: Masked Modeling Duo: Learning Representations by Encouraging Both Networks to Model the Input

abstract: Masked Autoencoders is a simple yet powerful self-supervised learning method. However, it learns representations indirectly by reconstructing masked input patches. Several methods learn representations directly by predicting representations of masked patches; however, we think using all patches to encode training signal representations is suboptimal. We propose a new method, Masked Modeling Duo (M2D), that learns representations directly while obtaining training signals using only masked patches. In the M2D, the online network encodes visible patches and predicts masked patch representations, and the target network, a momentum encoder, encodes masked patches. To better predict target representations, the online network should model the input well, while the target network should also model it well to agree with online predictions. Then the learned representations should better model the input. We validated the M2D by learning general-purpose audio representations, and M2D set new state-of-the-art performance on tasks such as UrbanSound8K, VoxCeleb1, AudioSet20K, GTZAN, and SpeechCommandsV2. We additionally validate the effectiveness of M2D for images using ImageNet-1K in the appendix.


title: Higher-Order MSL Horn Constraints

abstract: The monadic shallow linear (MSL) class is a decidable fragment of first-order Horn clauses that was discovered and rediscovered around the turn of the century, with applications in static analysis and verification. We propose a new class of higher-order Horn constraints which extend MSL to higher-order logic and develop a resolution-based decision procedure. Higher-order MSL Horn constraints can quite naturally capture the complex patterns of call and return that are possible in higher-order programs, which make them well suited to higher-order program verification. In fact, we show that the higher-order MSL satisfiability problem and the HORS model checking problem are interreducible, so that higher-order MSL can be seen as a constraint-based approach to higher-order model checking. Finally, we describe an implementation of our decision procedure and its application to verified socket programming.


title: MOCHA: A Multi-Task Training Approach for Coherent Text Generation from Cognitive Perspective

abstract: Teaching neural models to generate narrative coherent texts is a critical problem. Recent pre-trained language models have achieved promising results, but there is still a gap between human written texts and machine-generated outputs. In this work, we propose a novel multi-task training strategy for coherent text generation grounded on the cognitive theory of writing, which empowers the model to learn essential subskills needed for writing including planning and reviewing besides end-to-end generation. We extensively evaluate our model on three open-ended generation tasks including story generation, news article writing and argument generation. Experiments show that our model achieves better results on both few-shot and fully-supervised settings than strong baselines, and human evaluations confirm that our model can generate more coherent outputs.


title: Superior damage tolerance of fish skins

abstract: Skin is the largest organ of many animals. Its protective function against hostile environments and predatorial attack makes high mechanical strength a vital characteristic. Here, we measured the mechanical properties of bass fish skins and found that fish skins are highly ductile with a rupture strain of up to 30-40% and a rupture strength of 10-15 MPa. The fish skins exhibit a strain-stiffening behavior. Stretching can effectively eliminate the stress concentrations near the pre-existing holes and edge notches, suggesting that the skins are highly damage tolerant. Our measurement determined a flaw-insensitivity length of several millimeters, which exceeds that of most engineering materials. The strain-stiffening and damage tolerance of fish skins are explained by an agent-based model of collagen network in which the load-bearing collagen microfibers assembled from nanofibrils undergo straightening and reorientation upon stretching. Our study inspires development of artificial skins that are thin, flexible, but highly fracture-resistant and widely applicable in soft robots.


title: Time evolution and the Schr\"odinger equation on time dependent quantum graphs

abstract: The purpose of the present paper is to discuss the time dependent Schr\"odinger equation on a metric graph with time-dependent edge lengths, and the proper way to pose the problem so that the corresponding time evolution is unitary. We show that the well posedness of the Schr\"odinger equation can be guaranteed by replacing the standard Kirchhoff Laplacian with a magnetic Schr\"odinger operator with a harmonic potential. We then generalize the result to time dependent families of vertex conditions. We also apply the theory to show the existence of a geometric phase associated with a slowly changing quantum graph.


title: TSUP Speaker Diarization System for Conversational Short-phrase Speaker Diarization Challenge

abstract: This paper describes the TSUP team's submission to the ISCSLP 2022 conversational short-phrase speaker diarization (CSSD) challenge which particularly focuses on short-phrase conversations with a new evaluation metric called conversational diarization error rate (CDER). In this challenge, we explore three kinds of typical speaker diarization systems, which are spectral clustering(SC) based diarization, target-speaker voice activity detection(TS-VAD) and end-to-end neural diarization(EEND) respectively. Our major findings are summarized as follows. First, the SC approach is more favored over the other two approaches under the new CDER metric. Second, tuning on hyperparameters is essential to CDER for all three types of speaker diarization systems. Specifically, CDER becomes smaller when the length of sub-segments setting longer. Finally, multi-system fusion through DOVER-LAP will worsen the CDER metric on the challenge data. Our submitted SC system eventually ranks the third place in the challenge.


title: Solvability of the heat equation on a half-space with a dynamical boundary condition and unbounded initial data

abstract: We study the linear heat equation on a halfspace with a linear dynamical boundary condition. We are interested in an appropriate choice of the function space of initial functions such that the problem possesses a solution. It was known before that bounded initial data guarantee solvability. Here we extend that result by showing that data from a weighted Lebesgue space will also do so.


title: Model of work motivation based on happiness: pandemic related study

abstract: This study aims to enrich the current literature by providing a new approach to motivating Generation Z employees in Poland. Employees need to be motivated in order to be efficient at doing a particular task at the workplace. As young people born between 1995 and 2004 called Generation Z, enter the labour market it, is essential to consider how employees' motivation might be affected. Traditionally identified motivators are known, but many reports indicate that the motivation continues decreasing. This situation causes some perturbations in business and fluctuations of staff. In order to prevent this situation, the employers are looking for new solutions to motivate the employees. A quantitative approach was used to collect new evidence from 200 Polish respondents completing an online survey. The research were conducted before and during pandemic time. We report and analyse the survey results conducted in Poland among representatives of Generation Z, who were employed for at least 6 months. We developed and validated a new approach to motivation using methodologies called Factor Analysis. Based on empirical verification, we found a new tool that connects employee motivation and selected areas of the Hygge concept called Hygge star model, which has the same semantics before and during Covid-19 pandemic.


title: Searching for shell stars in LAMOST DR4 by probing the Fe 42 multiplet lines

abstract: Shell stars, in particular the cooler ones, often do not show conspicuous Balmer-line emission and may consequently be missed in surveys that specifically search for emission signatures in the Halpha line. The present work is aimed at identifying stars with shell-signatures via a search for strong FeII multiplet 42 lines at 4924, 5018, 5169A in archival LAMOST spectra. Candidates were selected by probing the FeII 42 lines in the spectra of a sample of colour-preselected early-type stars using a modified version of the MKCLASS code and then categorised by visual inspection of their spectra. We identified 75 stars showing conspicuous shell features, 43 Am/CP1 stars, 12 Ap/CP2 stars, and three objects with composite spectra. Spectral types and equivalent width measurements of the FeII 42 lines are presented for the sample of shell stars. Except for three objects, all shell stars appear significantly removed from the ZAMS in the colour-magnitude diagram, which is likely due to extinction by circumstellar material. We find a correlation between the equivalent width of the 5169A line and the distance to the locus of the main-sequence stars (the larger the IR-excess, the stronger the 5169A line) and studied the variability of the shell star sample using TESS data, identifying a very high proportion of double stars. All but 14 shell stars are new discoveries, which highlights the efficiency of the here presented novel approach to identify stars with subtle shell features. This study may be used as a blueprint for discovering these objects in massive spectral databases.


title: Multi-Objective Hardware-Mapping Co-Optimisation for Multi-Tenant DNN Accelerators

abstract: To meet the ever-increasing computation demand from emerging workloads, a scalable design paradigm combines multiple Deep Neural Network (DNN) accelerators to build a large multi-accelerator system. They are mainly proposed for data centers, where workload varies across vision, language, recommendation, etc. Existing works independently explore their hardware configuration and mapping strategies due to the extremely large cross-coupled design space. However, hardware and mapping are interdependent and, if not explored together, may lead to sub-optimal performance when workload changes. Moreover, even though a data center accelerator has multiple objectives, almost all the existing works prefer aggregating them into one (mono-objective). But aggregation does not help if the objectives are conflicting, as improving one will worsen the other. This work proposes MOHaM, a multi-objective hardware-mapping co-optimisation framework for multi-tenant DNN accelerators. Specifically, given an application model and a library of heterogeneous, parameterised and reconfigurable sub-accelerator templates, MOHaM returns a Pareto-optimal set of multi-accelerator systems with an optimal schedule for each one of them to minimise the overall system latency, energy and area. MOHaM is evaluated for diverse workload scenarios with state-of-the-art sub-accelerators. The Pareto-optimal set of competitive design choices enables selecting the best one as per the requirement.


title: RKKY interactions mediated by topological states in transition metal doped bismuthene

abstract: We have investigated magnetic interactions between transition metal ions in bismuthene topological insulator with protected edge states. We find that these topological states have a crucial role on the magnetic interactions in 2D topological insulators. Using first-principles and model Hamiltonian we make a comparative study of transition metal doped bulk and nanoribbon bismuthene. While direct overlap between the transition metal prevails in gapped bulk bismuthene, at the borders of nanoribbons a long-range magnetism is present. The exchange interactions are well described by a RKKY-like Hamiltonian mediated by topological states. Our results show a dominance of antiferromagnetism promoted by the topological states, preserving the spin-locked Dirac crossing states due to a global time-reversal symmetry preservation. This extended magnetic interactions mediated by massless electrons can increase the spin diffusion length being promising for fast dissipationless spintronic devices.


title: Maximal estimates for the bilinear Riesz means on Heisenberg groups

abstract: In this article, we investigate the maximal bilinear Riesz means $S^{\alpha }_{*}$ associated to the sublaplacian on the Heisenberg group. We prove that the operator $S^{\alpha }_{*}$ is bounded from $L^{p_{1}}\times L^{p_{2}}$ into $% L^{p}$ for $2\leq p_{1}, p_{2}\leq \infty $ and $1/p=1/p_{1}+1/p_{2}$ when $% \alpha $ is large than a suitable smoothness index $\alpha (p_{1},p_{2})$. For obtaining a lower index $\alpha (p_{1},p_{2})$, we define two important auxiliary operators and investigate their $L^{p}$ estimates,which play a key role in our proof.


title: Simulated Bifurcation Algorithm for MIMO Detection

abstract: We study the performance of the simulated bifurcation (SB) algorithm for signal detection in multiple-input multiple-output (MIMO) system, a problem of key interest in modern wireless communication systems. Our results show that SB algorithm can achieve significant performance improvement over the widely used linear minimum-mean square error decoder in terms of the bit error rate versus the signal-to-noise ratio, as well as performance improvement over the coherent Ising machine based MIMO detection method.


title: Full-band General Audio Synthesis with Score-based Diffusion

abstract: Recent works have shown the capability of deep generative models to tackle general audio synthesis from a single label, producing a variety of impulsive, tonal, and environmental sounds. Such models operate on band-limited signals and, as a result of an autoregressive approach, they are typically conformed by pre-trained latent encoders and/or several cascaded modules. In this work, we propose a diffusion-based generative model for general audio synthesis, named DAG, which deals with full-band signals end-to-end in the waveform domain. Results show the superiority of DAG over existing label-conditioned generators in terms of both quality and diversity. More specifically, when compared to the state of the art, the band-limited and full-band versions of DAG achieve relative improvements that go up to 40 and 65%, respectively. We believe DAG is flexible enough to accommodate different conditioning schemas while providing good quality synthesis.


title: Parametrized quantum circuit for weight-adjustable quantum loop gas

abstract: Motivated by the recent success of realizing the topologically ordered ground state of the exactly solvable toric code model by a quantum circuit on the real quantum device [K. J. Satzinger {\it et al}., Science \textbf{374}, 1237 (2021)], here we propose a parametrized quantum circuit (PQC) with the same real-device-performable optimal structure to represent quantum loop gas states with adjustably weighted loop configurations. Combining such a PQC with the variational quantum eigensolver, we obtain the accurate quantum circuit representation for the toric code model in an external magnetic field with any field strength, where the system is not exactly solvable. The topological quantum phase transition in this system is further observed in the optimized circuits by measuring the magnetization and topological entanglement entropy.


title: Model reduction for molecular diffusion in nanoporous media

abstract: Porous materials are widely used for applications in gas storage and separation. The diffusive properties of a variety of gases in porous media can be modeled using molecular dynamics simulations that can be computationally demanding depending on the pore geometry, complexity and amount of gas adsorbed. We explore a dimensionality reduction approach for estimating the self-diffusion coefficient of gases in simple pores using Langevin dynamics, such that the three-dimensional (3D) atomistic interactions that determine the diffusion properties of realistic systems can be reduced to an effective one-dimensional (1D) diffusion problem along the pore axis. We demonstrate the approach by modeling the transport of nitrogen molecules in single-walled carbon nanotubes of different radii, showing that 1D Langevin models can be parametrized with a few single-particle 3D atomistic simulations. The reduced 1D model predicts accurate diffusion coefficients over a broad range of temperatures and gas densities. Our work paves the way for studying the diffusion process of more general porous materials as zeolites or metal-organics frameworks with effective models of reduced complexity.


title: Coresets for Vertical Federated Learning: Regularized Linear Regression and $K$-Means Clustering

abstract: Vertical federated learning (VFL), where data features are stored in multiple parties distributively, is an important area in machine learning. However, the communication complexity for VFL is typically very high. In this paper, we propose a unified framework by constructing coresets in a distributed fashion for communication-efficient VFL. We study two important learning tasks in the VFL setting: regularized linear regression and $k$-means clustering, and apply our coreset framework to both problems. We theoretically show that using coresets can drastically alleviate the communication complexity, while nearly maintain the solution quality. Numerical experiments are conducted to corroborate our theoretical findings.


title: Desiderata for next generation of ML model serving

abstract: Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale.


title: Xiaoicesing 2: A High-Fidelity Singing Voice Synthesizer Based on Generative Adversarial Network

abstract: XiaoiceSing is a singing voice synthesis (SVS) system that aims at generating 48kHz singing voices. However, the mel-spectrogram generated by it is over-smoothing in middle- and high-frequency areas due to no special design for modeling the details of these parts. In this paper, we propose XiaoiceSing2, which can generate the details of middle- and high-frequency parts to better construct the full-band mel-spectrogram. Specifically, in order to alleviate this problem, XiaoiceSing2 adopts a generative adversarial network (GAN), which consists of a FastSpeech-based generator and a multi-band discriminator. We improve the feed-forward Transformer (FFT) block by adding multiple residual convolutional blocks in parallel with the self-attention block to balance the local and global features. The multi-band discriminator contains three sub-discriminators responsible for low-, middle-, and high-frequency parts of the mel-spectrogram, respectively. Each sub-discriminator is composed of several segment discriminators (SD) and detail discriminators (DD) to distinguish the audio from different aspects. The experiment on our internal 48kHz singing voice dataset shows XiaoiceSing2 significantly improves the quality of the singing voice over XiaoiceSing.


title: A Bilingual Parallel Corpus with Discourse Annotations

abstract: Machine translation (MT) has almost achieved human parity at sentence-level translation. In response, the MT community has, in part, shifted its focus to document-level translation. However, the development of document-level MT systems is hampered by the lack of parallel document corpora. This paper describes BWB, a large parallel corpus first introduced in Jiang et al. (2022), along with an annotated test set. The BWB corpus consists of Chinese novels translated by experts into English, and the annotated test set is designed to probe the ability of machine translation systems to model various discourse phenomena. Our resource is freely available, and we hope it will serve as a guide and inspiration for more work in document-level machine translation.


title: Why would reduced Kronecker coefficients be log-concave?

abstract: We discuss a log-concavity conjecture on the reduced/stable Kronecker coefficients, which is a certain generalization of Okounkov's conjecture on the log-concavity of the Littlewood-Richardson coefficients and the Schur log-concavity theorem of Lam-Postnikov-Pylyavskyy. We prove the conjecture in a special case. We also consider some implications of this concavity. These are informal notes akin to Okounkov's famous paper "Why would multiplicities be log-concave?"


title: Monte Carlo simulation method of polarization effects in Laser Compton Scattering on relativistic electrons

abstract: Quasi-monochromatic, high energy and highly polarized $\gamma$-ray beam sources based on Compton scattering of laser photons (LCS) on relativistic electrons have developed for the last few decades as established instruments for nuclear physics studies. Following an extensive photoneutron experimental campaign at the LCS $\gamma$-ray beam line of the NewSUBARU synchrotron radiation facility at SPring8, Japan, a dedicated simulation code was developed for characterizing the incident $\gamma$-ray beams. The eliLaBr code is implemented using Geant4 and is available on the GitHub repository, at https://github.com/dan-mihai-filipescu/eliLaBr . The present work describes step-by-step the Monte Carlo algorithm with focus on modeling the polarization properties of the scattered photon. The polarization is treated independently both in the Stokes parameters and in the polarization vector formalisms. An intervalidation between the two methods is given. Based on polarization state description requirements of different Geant4 physics classes, user recommendations are given on which of the two methods to be employed. The spatial and energy distributions for the LCS $\gamma$-ray beam and its Stokes parameters are obtained for head-on laser - relativistic electron collisions, where several incident laser polarization states were considered: linear, unpolarized, circular and mixed linear and circular polarization. The influence of variable incident angle between photon and electron beam was also investigated. We show that the degree of polarization transfer from the incident photon to the scattered photon increases with the collision angle. However, we show that, for $\gamma$-ray sources based on Compton scattering of laser photons on relativistic electrons, the polarization degree of the incident photon is almost completely transferred to the scattered photon for any incident angle.


title: Boosting Semi-Supervised Semantic Segmentation with Probabilistic Representations

abstract: Recent breakthroughs in semi-supervised semantic segmentation have been developed through contrastive learning. In prevalent pixel-wise contrastive learning solutions, the model maps pixels to deterministic representations and regularizes them in the latent space. However, there exist inaccurate pseudo-labels which map the ambiguous representations of pixels to the wrong classes due to the limited cognitive ability of the model. In this paper, we define pixel-wise representations from a new perspective of probability theory and propose a Probabilistic Representation Contrastive Learning (PRCL) framework that improves representation quality by taking its probability into consideration. Through modelling the mapping from pixels to representations as the probability via multivariate Gaussian distributions, we can tune the contribution of the ambiguous representations to tolerate the risk of inaccurate pseudo-labels. Furthermore, we define prototypes in the form of distributions, which indicates the confidence of a class, while the point prototype cannot. Moreover, we propose to regularize the distribution variance to enhance the reliability of representations. Taking advantage of these benefits, high-quality feature representations can be derived in the latent space, thereby the performance of semantic segmentation can be further improved. We conduct sufficient experiment to evaluate PRCL on Pascal VOC and CityScapes to demonstrate its superiority. The code is available at https://github.com/Haoyu-Xie/PRCL.


title: Bures-Wasserstein Barycenters and Low-Rank Matrix Recovery

abstract: We revisit the problem of recovering a low-rank positive semidefinite matrix from rank-one projections using tools from optimal transport. More specifically, we show that a variational formulation of this problem is equivalent to computing a Wasserstein barycenter. In turn, this new perspective enables the development of new geometric first-order methods with strong convergence guarantees in Bures-Wasserstein distance. Experiments on simulated data demonstrate the advantages of our new methodology over existing methods.


title: Sparsity in Continuous-Depth Neural Networks

abstract: Neural Ordinary Differential Equations (NODEs) have proven successful in learning dynamical systems in terms of accurately recovering the observed trajectories. While different types of sparsity have been proposed to improve robustness, the generalization properties of NODEs for dynamical systems beyond the observed data are underexplored. We systematically study the influence of weight and feature sparsity on forecasting as well as on identifying the underlying dynamical laws. Besides assessing existing methods, we propose a regularization technique to sparsify "input-output connections" and extract relevant features during training. Moreover, we curate real-world datasets consisting of human motion capture and human hematopoiesis single-cell RNA-seq data to realistically analyze different levels of out-of-distribution (OOD) generalization in forecasting and dynamics identification respectively. Our extensive empirical evaluation on these challenging benchmarks suggests that weight sparsity improves generalization in the presence of noise or irregular sampling. However, it does not prevent learning spurious feature dependencies in the inferred dynamics, rendering them impractical for predictions under interventions, or for inferring the true underlying dynamics. Instead, feature sparsity can indeed help with recovering sparse ground-truth dynamics compared to unregularized NODEs.


title: Riesz means and bilinear Riesz means on M\'{e}tivier groups

abstract: In this paper, we investigate the $L^{p}$-boundedness of the Riesz means and the $L^{p_{1}}\times L^{p_{2}}\rightarrow L^{p}$ boundedness of the bilinear Riesz means on M\'{e}tivier groups. M\'{e}tivier groups are generalization of Heisenberg groups and general H-type groups. Because general M\'{e}tivier groups only satisfy the non-degeneracy condition and have high-dimensional centre, we have to use different methods and techniques from those on Heisenberg groups and H-type groups.


title: Controlled remote implementation of operations via graph states

abstract: We propose protocols for controlled remote implementation of operations with convincing control power. Sharing a $(2N+1)$-partite graph state, $2N$ participants collaborate to prepare the stator and realize the operation $\otimes_{j=1}^N\exp{[i\alpha_j\sigma_{n_{O_j}}]}$ on $N$ unknown states for distributed systems $O_j$, with the permission of a controller. All the implementation requirements of our protocol can be satisfied by means of local operations and classical communications, and the experimental feasibility is presented according to current techniques. We characterize the entanglement requirement of our protocol in terms of geometric measure of entanglement. It turns out to be economic to realize the control function from the perspective of entanglement cost. Further we show that the control power of our protocol is reliable by positive operator valued measurement.


title: Comparison of neural closure models for discretised PDEs

abstract: Neural closure models have recently been proposed as a method for efficiently approximating small scales in multiscale systems with neural networks. The choice of loss function and associated training procedure has a large effect on the accuracy and stability of the resulting neural closure model. In this work, we systematically compare three distinct procedures: "derivative fitting", "trajectory fitting" with discretise-then-optimise, and "trajectory fitting" with optimise-then-discretise. Derivative fitting is conceptually the simplest and computationally the most efficient approach and is found to perform reasonably well on one of the test problems (Kuramoto-Sivashinsky) but poorly on the other (Burgers). Trajectory fitting is computationally more expensive but is more robust and is therefore the preferred approach. Of the two trajectory fitting procedures, the discretise-then-optimise approach produces more accurate models than the optimise-then-discretise approach. While the optimise-then-discretise approach can still produce accurate models, care must be taken in choosing the length of the trajectories used for training, in order to train the models on long-term behaviour while still producing reasonably accurate gradients during training. Two existing theorems are interpreted in a novel way that gives insight into the long-term accuracy of a neural closure model based on how accurate it is in the short term.


title: A remark on post-critically finite compositions of polynomials

abstract: The second author proved that the set of post-critically finite polynomials of given degree is a set of bounded height, up to change of variables. Motivated by an observation about unicritical polynomials, we complement this by proving that the set of monic polynomials g(z) of given degree with the property that there exists a d > 1 such that g(z^d) is post-critically finite, is also a set of bounded height. Moreover, we establish a lower bound on the critical height of g(z^d).


title: How precise are performance estimates for typical medical image segmentation tasks?

abstract: An important issue in medical image processing is to be able to estimate not only the performances of algorithms but also the precision of the estimation of these performances. Reporting precision typically amounts to reporting standard-error of the mean (SEM) or equivalently confidence intervals. However, this is rarely done in medical image segmentation studies. In this paper, we aim to estimate what is the typical confidence that can be expected in such studies. To that end, we first perform experiments for Dice metric estimation using a standard deep learning model (U-net) and a classical task from the Medical Segmentation Decathlon. We extensively study precision estimation using both Gaussian assumption and bootstrapping (which does not require any assumption on the distribution). We then perform simulations for other test set sizes and performance spreads. Overall, our work shows that small test sets lead to wide confidence intervals (e.g. $\sim$8 points of Dice for 20 samples with $\sigma \simeq 10$).


title: Investigating the Role of Centering Theory in the Context of Neural Coreference Resolution Systems

abstract: Centering theory (CT; Grosz et al., 1995) provides a linguistic analysis of the structure of discourse. According to the theory, local coherence of discourse arises from the manner and extent to which successive utterances make reference to the same entities. In this paper, we investigate the connection between centering theory and modern coreference resolution systems. We provide an operationalization of centering and systematically investigate if neural coreference resolvers adhere to the rules of centering theory by defining various discourse metrics and developing a search-based methodology. Our information-theoretic analysis reveals a positive dependence between coreference and centering; but also shows that high-quality neural coreference resolvers may not benefit much from explicitly modeling centering ideas. Our analysis further shows that contextualized embeddings contain much of the coherence information, which helps explain why CT can only provide little gains to modern neural coreference resolvers which make use of pretrained representations. Finally, we discuss factors that contribute to coreference which are not modeled by CT such as world knowledge and recency bias. We formulate a version of CT that also models recency and show that it captures coreference information better compared to vanilla CT.


title: On the epidemic threshold of a network

abstract: The graph invariant examined in this paper is the largest eigenvalue of the adjacency matrix of a graph. Previous work demonstrates the tight relationship between this invariant, the birth and death rate of a contagion spreading on the graph, and the trajectory of the contagion over time. We begin by conducting a simulation confirming this and explore bounds on the birth and death rate in terms of well-known graph invariants. As a result, the change in the largest eigenvalue resulting from removal of a vertex in the network is the best measure of effectiveness of interventions that slow the spread of a contagion. We define the spread centrality of a vertex $v$ in a graph $G$ as the difference between the largest eigenvalues of $G$ and $G-v$. While the spread centrality is a distinct centrality measure and serves as another graph invariant for distinguishing graphs, we found experimental evidence that vertices ranked by the spread centrality and those ranked by eigenvector centrality are strongly correlated. Since eigenvector centrality is easier to compute than the spread centrality, this justifies using eigenvector centrality as a measure of spread, especially in large networks with unknown portions. We also examine two strategies for selecting members of a population to vaccinate.


title: An adaptive finite element/finite difference domain decomposition method for applications in microwave imaging

abstract: A new domain decomposition method for Maxwell's equations in conductive media is presented. Using this method reconstruction algorithms are developed for determination of dielectric permittivity function using time-dependent scattered data of electric field. All reconstruction algorithms are based on optimization approach to find stationary point of the Lagrangian. Adaptive reconstruction algorithms and space mesh refinement indicators are also presented. Our computational tests show qualitative reconstruction of dielectric permittivity function using anatomically realistic breast phantom.


title: Theory of strong down-conversion in multi-mode cavity and circuit QED

abstract: We revisit the superstrong coupling regime of multi-mode cavity quantum electrodynamics (QED), defined to occur when the frequency of vacuum Rabi oscillations between the qubit and the nearest cavity mode exceeds the cavity's free spectral range. A novel prediction is made that the cavity's linear spectrum, measured in the vanishing power limit, can acquire an intricate fine structure associated with the qubit-induced cascades of coherent single-photon down-conversion processes. This many-body effect is hard to capture by a brute-force numerics and it is sensitive to the light-matter coupling parameters both in the infra-red and the ultra-violet limits. We focused at the example case of a superconducting fluxonium qubit coupled to a long transmission line section. The conversion rate in such a circuit QED setup can readily exceed a few MHz, which is plenty to overcome the usual decoherence processes. Analytical calculations were made possible by an unconventional gauge choice, in which the qubit circuit interacts with radiation via the flux/charge variable in the low-/high-frequency limits, respectively. Our prediction of the fine spectral structure lays the foundation for the "strong down-conversion" regime in quantum optics, in which a single photon excited in a non-linear medium spontaneously down-converts faster than it is absorbed.


title: In search of strong embedding extractors for speaker diarisation

abstract: Speaker embedding extractors (EEs), which map input audio to a speaker discriminant latent space, are of paramount importance in speaker diarisation. However, there are several challenges when adopting EEs for diarisation, from which we tackle two key problems. First, the evaluation is not straightforward because the features required for better performance differ between speaker verification and diarisation. We show that better performance on widely adopted speaker verification evaluation protocols does not lead to better diarisation performance. Second, embedding extractors have not seen utterances in which multiple speakers exist. These inputs are inevitably present in speaker diarisation because of overlapped speech and speaker changes; they degrade the performance. To mitigate the first problem, we generate speaker verification evaluation protocols that mimic the diarisation scenario better. We propose two data augmentation techniques to alleviate the second problem, making embedding extractors aware of overlapped speech or speaker change input. One technique generates overlapped speech segments, and the other generates segments where two speakers utter sequentially. Extensive experimental results using three state-of-the-art speaker embedding extractors demonstrate that both proposed approaches are effective.


title: Unconventional magneto-resistance, and electronic transition in Mn$_3$Ge Weyl semimetal

abstract: Weyl semimetals are well known for their anomalous transport effects caused by a large fictitious magnetic field generated by the non-zero Berry curvature. We performed the analysis of the electrical transport measurements of the magnetic Weyl semimetal Mn$_{3}$Ge in the a-b and a-c plane. We have observed negative longitudinal magneto-resistance (LMR) at a low magnetic field ($B<1.5$ T) along all the axes. The high field LMR shows different behavior along x and z axes. A similar trend has been observed in the case of planar Hall effect (PHE) measurements as well. The nature of high field LMR along the x axis changes near 200 K. Dominant carrier concentration type, and metallic to semimetallic transition also occur near 200 K. These observations suggest two main conclusions: (i) The high field LMR in Mn$_3$Ge is driven by the metallic - semimetallic nature of the compound. (ii) Mn$_3$Ge compound goes through an electronic band topological transition near 200 K. Single crystal neutron diffraction does not show any change in the magnetic structure below 300 K. However, the in-plane lattice parameter (a) shows a maximum near 230 K. Therefore, it is possible that the change in electronic band structure near 200 K is driven by the a lattice parameter of the compound.


title: Nonlinear System Identification: Learning while respecting physical models using a sequential Monte Carlo method

abstract: Identification of nonlinear systems is a challenging problem. Physical knowledge of the system can be used in the identification process to significantly improve the predictive performance by restricting the space of possible mappings from the input to the output. Typically, the physical models contain unknown parameters that must be learned from data. Classical methods often restrict the possible models or have to resort to approximations of the model that introduce biases. Sequential Monte Carlo methods enable learning without introducing any bias for a more general class of models. In addition, they can also be used to approximate a posterior distribution of the model parameters in a Bayesian setting. This article provides a general introduction to sequential Monte Carlo and shows how it naturally fits in system identification by giving examples of specific algorithms. The methods are illustrated on two systems: a system with two cascaded water tanks with possible overflow in both tanks and a compartmental model for the spreading of a disease.


title: Leveraging Demonstrations with Latent Space Priors

abstract: Demonstrations provide insight into relevant state or action space regions, bearing great potential to boost the efficiency and practicality of reinforcement learning agents. In this work, we propose to leverage demonstration datasets by combining skill learning and sequence modeling. Starting with a learned joint latent space, we separately train a generative model of demonstration sequences and an accompanying low-level policy. The sequence model forms a latent space prior over plausible demonstration behaviors to accelerate learning of high-level policies. We show how to acquire such priors from state-only motion capture demonstrations and explore several methods for integrating them into policy learning on transfer tasks. Our experimental results confirm that latent space priors provide significant gains in learning speed and final performance. We benchmark our approach on a set of challenging sparse-reward environments with a complex, simulated humanoid, and on offline RL benchmarks for navigation and object manipulation. Videos, source code and pre-trained models are available at the corresponding project website at https://facebookresearch.github.io/latent-space-priors .


title: Anomalous DNA hybridisation kinetics on gold nanorods revealed via a dual single-molecule imaging and optoplasmonic sensing platform

abstract: Observing the hybridisation kinetics of DNA probes immobilised on plasmonic nanoparticles is key in plamon enhanced fluorescence detection from weak emitting species, and refractive index based single-molecule detection on optoplasmonic sensors. The role of the local field in providing plasmonic signal enhancements for single-molecule detection has been studied in great detail. Nevertheless, few studies have compared the experimental results in both techniques for single-molecule studies. Here we developed the first optical setup that integrates optoplasmonic and DNA-PAINT based detection of oligonucleotides to compare these sub-platforms and provide complementary insights into single-molecule processes. We record the fluorescence and optoplasmonic sensor signals for individual, transient hybridisation events. The hybridisation events are observed in the same sample cell and over a prolonged time (i.e. towards high binding site occupancies). A decrease in the association rate over the measurement duration is reported. Our dual optoplasmonic sensing and imaging platform offers insight into the observed phenomenon, revealing that irreversible hybridisation events accumulate over detected step signals in optoplasmonic sensing. Our results point to novel physicochemical mechanisms that result in the stabilisation of DNA hybridisation on optically-excited plasmonic nanoparticles.


title: Which is the best model for my data?

abstract: In this paper, we tackle the problem of selecting the optimal model for a given structured pattern classification dataset. In this context, a model can be understood as a classifier and a hyperparameter configuration. The proposed meta-learning approach purely relies on machine learning and involves four major steps. Firstly, we present a concise collection of 62 meta-features that address the problem of information cancellation when aggregation measure values involving positive and negative measurements. Secondly, we describe two different approaches for synthetic data generation intending to enlarge the training data. Thirdly, we fit a set of pre-defined classification models for each classification problem while optimizing their hyperparameters using grid search. The goal is to create a meta-dataset such that each row denotes a multilabel instance describing a specific problem. The features of these meta-instances denote the statistical properties of the generated datasets, while the labels encode the grid search results as binary vectors such that best-performing models are positively labeled. Finally, we tackle the model selection problem with several multilabel classifiers, including a Convolutional Neural Network designed to handle tabular data. The simulation results show that our meta-learning approach can correctly predict an optimal model for 91% of the synthetic datasets and for 87% of the real-world datasets. Furthermore, we noticed that most meta-classifiers produced better results when using our meta-features. Overall, our proposal differs from other meta-learning approaches since it tackles the algorithm selection and hyperparameter tuning problems in a single step. Toward the end, we perform a feature importance analysis to determine which statistical features drive the model selection mechanism.


title: Scaling Law Analysis for Covariance Based Activity Detection in Cooperative Multi-Cell Massive MIMO

abstract: This paper studies the covariance based activity detection problem in a multi-cell massive multiple-input multiple-output (MIMO) system, where the active devices transmit their signature sequences to multiple base stations (BSs), and the BSs cooperatively detect the active devices based on the received signals. The scaling law of covariance based activity detection in the single-cell scenario has been thoroughly analyzed in the literature. This paper aims to analyze the scaling law of covariance based activity detection in the multi-cell massive MIMO system. In particular, this paper shows a quadratic scaling law in the multi-cell system under the assumption that the exponent in the classical path-loss model is greater than 2, which demonstrates that in the multi-cell MIMO system the maximum number of active devices that can be correctly detected in each cell increases quadratically with the length of the signature sequence and decreases logarithmically with the number of cells (as the number of antennas tends to infinity). This paper also characterizes the distribution of the estimation error in the multi-cell scenario.


title: Non-abelian simple groups which occur as the type of a Hopf-Galois structure on a solvable extension

abstract: We determine the finite non-abelian simple groups which occur as the type of a Hopf-Galois structure on a solvable extension. In the language of skew braces, our result gives a complete list of finite non-abelian simple groups which occur as the additive group of a skew brace with solvable multiplicative group.


title: Simulation of the response of a diamond-based radiation detector to ultra-short and intense high-energy electron pulses

abstract: Single-crystal synthetic diamond sensors have been widely used in radiation dosimetry and beam diagnostics. The foreseen harsh radiation environment in electron-positron colliders at the luminosity frontier requires a thorough investigation of diamond's response to large radiation burst, in particular, to intense high-energy electron pulses. In this article, a two-step numerical simulation approach (Sentaurus~+~LTspice) is proposed to explore this topic. Time response of the diamond detector is simulated via TCAD-Sentaurus while the transmission effect of the electronic circuit is taken into account using LTspice. Good agreement is observed between results of the numerical simulation and preliminary experimental data from detector's exposure to high-energy sub-picosecond electron pulses, on both the amplitude and the shape of the induced signals. This simulation combination is a novel approach to designing and optimising diamond detectors for radiation and beam loss monitoring in particle physics experiments.


title: Pronunciation Generation for Foreign Language Words in Intra-Sentential Code-Switching Speech Recognition

abstract: Code-Switching refers to the phenomenon of switching languages within a sentence or discourse. However, limited code-switching , different language phoneme-sets and high rebuilding costs throw a challenge to make the specialized acoustic model for code-switching speech recognition. In this paper, we make use of limited code-switching data as driving materials and explore a shortcut to quickly develop intra-sentential code-switching recognition skill on the commissioned native language acoustic model, where we propose a data-driven method to make the seed lexicon which is used to train grapheme-to-phoneme model to predict mapping pronunciations for foreign language word in code-switching sentences. The core work of the data-driven technology in this paper consists of a phonetic decoding method and different selection methods. And for imbalanced word-level driving materials problem, we have an internal assistance inspiration that learning the good pronunciation rules in the words that possess sufficient materials using the grapheme-to-phoneme model to help the scarce. Our experiments show that the Mixed Error Rate in intra-sentential Chinese-English code-switching recognition reduced from 29.15\%, acquired on the pure Chinese recognizer, to 12.13\% by adding foreign language words' pronunciation through our data-driven approach, and finally get the best result 11.14\% with the combination of different selection methods and internal assistance tactic.


title: Identifying Threats, Cybercrime and Digital Forensic Opportunities in Smart City Infrastructure via Threat Modeling

abstract: Technological advances have enabled multiple countries to consider implementing Smart City Infrastructure to provide in-depth insights into different data points and enhance the lives of citizens. Unfortunately, these new technological implementations also entice adversaries and cybercriminals to execute cyber-attacks and commit criminal acts on these modern infrastructures. Given the borderless nature of cyber attacks, varying levels of understanding of smart city infrastructure and ongoing investigation workloads, law enforcement agencies and investigators would be hard-pressed to respond to these kinds of cybercrime. Without an investigative capability by investigators, these smart infrastructures could become new targets favored by cybercriminals. To address the challenges faced by investigators, we propose a common definition of smart city infrastructure. Based on the definition, we utilize the STRIDE threat modeling methodology and the Microsoft Threat Modeling Tool to identify threats present in the infrastructure and create a threat model which can be further customized or extended by interested parties. Next, we map offences, possible evidence sources and types of threats identified to help investigators understand what crimes could have been committed and what evidence would be required in their investigation work. Finally, noting that Smart City Infrastructure investigations would be a global multi-faceted challenge, we discuss technical and legal opportunities in digital forensics on Smart City Infrastructure.


title: On the positivity of a certain function related with the Digamma function

abstract: It is proved that $$\left(\frac{x^n}{1-e^{-x}}\right)^{(n)}>0$$ for all $x\in (\log 2, \infty)$ and $n\in \mathbb{N}$, which improves the result of [Al-Musallam and Bustoz in Ramanujan J. 11 (2006) 399-402].


title: Nearly critical Galton--Watson processes

abstract: We investigate Galton--Watson processes in varying environment, for which $\bar f_n \uparrow 1$ and $\sum_{n=1}^\infty (1-\bar f_n) = \infty$, where $\bar f_n$ stands for the offspring mean in generation $n$. Since the process dies out almost surely, to obtain nontrivial limit we consider two scenarios: conditioning on non-extinction, or adding immigration. In both cases we show that the process converges in distribution without normalization to a nondegenerate compound-Poisson limit law. The proofs rely on the shape function technique, worked out by Kersting (2020).


title: Information Shift Dynamics Described by Tsallis $q=3$ Entropy on a Compact Phase Space

abstract: Recent mathematical investigations have shown that under very general conditions exponential mixing implies the Bernoulli property. As a concrete example of a statistical mechanics which is exponentially mixing we consider a Bernoulli shift dynamics by Chebyshev maps of arbitrary order $N\geq 2$, which maximizes Tsallis $q=3$ entropy rather than the ordinary $q=1$ Boltzmann-Gibbs entropy. Such an information shift dynamics may be relevant in a pre-universe before ordinary space-time is created. We discuss symmetry properties of the coupled Chebyshev systems, which are different for even and odd $N$. We show that the value of the fine structure constant $\alpha_{el}=1/137$ is distinguished as a coupling constant in this context, leading to uncorrelated behaviour in the spatial direction of the corresponding coupled map lattice for $N=3$.

categories: cond-mat.stat-mech math-ph math.DS math.MP nlin.CD


title: Quantum Monte Carlo Study of Semiconductor Artificial Graphene Nanostructures

abstract: Semiconductor artificial graphene nanostructures where Hubbard model parameter $U/t$ can be of the order of 100, provide a highly controllable platform to study strongly correlated quantum many-particle phases. We use accurate variational and diffusion Monte Carlo methods to demonstrate a transition from antiferromagnetic to metallic phases for experimentally accessible lattice constant $a=50$ nm in terms of lattice site radius $\rho$, for finite sized artificial honeycomb structures nanopatterned on GaAs quantum wells containing up to 114 electrons. By analysing spin-spin correlation functions for hexagonal flakes with armchair edges and triangular flakes with zigzag edges, we show that edge type, geometry and charge nonuniformity affect the steepness and the crossover $\rho$ value of the phase transition. For triangular structures, the metal-insulator transition is accompanied with a smoother edge polarization transition.


title: Multi-element metamaterial's design through the relaxed micromorphic model

abstract: Exploring the dynamical response of mechanical metamaterials has gathered increasing attention in the last decades, enabling the design of microstructures exotically interacting with elastic waves (focusing, channeling, band-gaps, negative refraction, cloaking, and many more). Yet, the application and use of such metamaterials in engineering practice is still deficient due to the lack of effective models unveiling metamaterials' interactions with more classical materials at finite scales. In this paper, we show that the relaxed micromorphic model can bring an answer to this open problem and can be effectively used to explore and optimize metamaterials' structures consisting of metamaterials' and classical materials' bricks of finite size. We investigate two examples, namely a double-shield structure that can be used to widen the frequency range for which the internal region can be protected and a multiple-shield structure that optimizes both the screening of the regions internal to the single shields and of the zones exterior to the shields themselves. The exploration of these complex meta-structures has been enabled by the finite element implementation of the relaxed micromorphic model that predicts their response at a fraction of the computational cost when compared to classical simulations.


title: Autoregressive Structured Prediction with Language Models

abstract: Recent years have seen a paradigm shift in NLP towards using pretrained language models ({PLM}) for a wide range of tasks. However, there are many difficult design decisions to represent structures (e.g. tagged text, coreference chains) in a way such that they can be captured by PLMs. Prior work on structured prediction with PLMs typically flattens the structured output into a sequence, which limits the quality of structural information being learned and leads to inferior performance compared to classic discriminative models. In this work, we describe an approach to model structures as sequences of actions in an autoregressive manner with PLMs, allowing in-structure dependencies to be learned without any loss. Our approach achieves the new state-of-the-art on all the structured prediction tasks we looked at, namely, named entity recognition, end-to-end relation extraction, and coreference resolution.


title: Piloting Copilot and Codex: Hot Temperature, Cold Prompts, or Black Magic?

abstract: Language models are promising solutions for tackling increasing complex problems. In software engineering, they recently attracted attention in code assistants, with programs automatically written in a given programming language from a programming task description in natural language. They have the potential to save time and effort when writing code. However, these systems are currently poorly understood, preventing them from being used optimally. In this paper, we investigate the various input parameters of two language models, and conduct a study to understand if variations of these input parameters (e.g. programming task description and the surrounding context, creativity of the language model, number of generated solutions) can have a significant impact on the quality of the generated programs. We design specific operators for varying input parameters and apply them over two code assistants (Copilot and Codex) and two benchmarks representing algorithmic problems (HumanEval and LeetCode). Our results showed that varying the input parameters can significantly improve the performance of language models. However, there is a tight dependency when varying the temperature, the prompt and the number of generated solutions, making potentially hard for developers to properly control the parameters to obtain an optimal result. This work opens opportunities to propose (automated) strategies for improving performance.