title: Light Robust Monocular Depth Estimation For Outdoor Environment Via Monochrome And Color Camera Fusion

abstract: Depth estimation plays a important role in SLAM, odometry, and autonomous driving. Especially, monocular depth estimation is profitable technology because of its low cost, memory, and computation. However, it is not a sufficiently predicting depth map due to a camera often failing to get a clean image because of light conditions. To solve this problem, various sensor fusion method has been proposed. Even though it is a powerful method, sensor fusion requires expensive sensors, additional memory, and high computational performance. In this paper, we present color image and monochrome image pixel-level fusion and stereo matching with partially enhanced correlation coefficient maximization. Our methods not only outperform the state-of-the-art works across all metrics but also efficient in terms of cost, memory, and computation. We also validate the effectiveness of our design with an ablation study.


title: Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction

abstract: In this paper, we propose an effective yet efficient model PAIE for both sentence-level and document-level Event Argument Extraction (EAE), which also generalizes well when there is a lack of training data. On the one hand, PAIE utilizes prompt tuning for extractive objectives to take the best advantages of Pre-trained Language Models (PLMs). It introduces two span selectors based on the prompt to select start/end tokens among input texts for each role. On the other hand, it captures argument interactions via multi-role prompts and conducts joint optimization with optimal span assignments via a bipartite matching loss. Also, with a flexible prompt design, PAIE can extract multiple arguments with the same role instead of conventional heuristic threshold tuning. We have conducted extensive experiments on three benchmarks, including both sentence- and document-level EAE. The results present promising improvements from PAIE (3.5\% and 2.3\% F1 gains in average on three benchmarks, for PAIE-base and PAIE-large respectively). Further analysis demonstrates the efficiency, generalization to few-shot settings, and effectiveness of different extractive prompt tuning strategies. Our code is available at https://github.com/mayubo2333/PAIE.


title: Four-fold non-Hermitian phase transitions and non-reciprocal coupled resonator optical waveguides

abstract: Non-Hermitian systems can exhibit extraordinary sensitivity to boundary conditions. Given that topological boundary modes and non-Hermitian skin effects can either coexist or individually appear in non-Hermitian systems, it is of great value to present a comprehensive non-Hermitian phase diagram, for further flexible control in realistic non-Hermitian systems. Here, we reveal four-fold non-Hermitian phase transitions at a mathematically level, where phase I exhibits only topological boundary modes, phase II displays both topological boundary modes and skin modes, phase III exhibits only skin modes, and phase IV cannot manifest any boundary modes. By deriving non-Hermitian winding numbers, the existence or non-existence condition of topological boundary modes are analytically expressed, consistent with the numerical results obtained through the iterative Green's function method. Combining with the study on non-Hermitian skin effects, we rigorously establish the four-fold phase diagram. We also design an array of coupled resonator optical waveguides. The introduction of non-Hermiticity in the photonic structure induces a phenomenon similar to band inversion in topological insulators, indicating the presence of topological boundary modes in the photonic bands.


title: An Algorithm for Computing the Covering Radius of a Linear Code Based on Vilenkin-Chrestenson Transform

abstract: We present a generalization of Walsh-Hadamard transform that is suitable for applications in Coding Theory, especially for computation of the weight distribution and the covering radius of a linear code over a finite field. The transform used in our research, is a modification of Vilenkin-Chrestenson transform. Instead of using all the vectors in the considered space, we take a maximal set of nonproportional vectors, which reduces the computational complexity.


title: Composition Operators on Dirichlet Spaces over the Half-plane

abstract: As continuation of the study of polynomial approximation and composition operators on Dirichlet spaces of unit disk, which has settled a problem posed by Cima in 1976, the present paper aims to consider the case of the unbounded domains, such as the half-plane. Specifically, we may obtain the rational approximations in the Dirichlet spaces and characterize the composition operators which has dense range on the Dirichlet spaces over the half-plane. Moreover, this paper also considers the relationship between the Dirichlet spaces and Hardy spaces on half-plane.


title: Semiseparable functors

abstract: In this paper we introduce and investigate the notion of semiseparable functor. One of its first features is that it allows a novel description of separable and naturally full functors in terms of faithful and full functors, respectively. To any semiseparable functor we attach an invariant, given by an idempotent natural transformation, which controls when the functor is separable and yields a characterization of separable functors in terms of (dual) Maschke and conservative functors. We prove that any semiseparable functor admits a canonical factorization as a naturally full functor followed by a separable functor. Here the main tool is the construction of the coidentifier category attached to the associated idempotent natural transformation. Then we move our attention to the semiseparability of functors that have an adjoint. First we obtain a Rafael-type Theorem. Next we characterize the semiseparability of adjoint functors in terms of the (co)separability of the associated (co)monads and the natural fullness of the corresponding (co)comparison functor. We also focus on functors that are part of an adjoint triple. In particular, we describe bireflections as semiseparable (co)reflections, or equivalently, as either Frobenius or naturally full (co)reflections. As an application of our results, we study the semiseparability of functors traditionally attached to ring homomorphisms, coalgebra maps, corings and bimodules, introducing the notions of semicosplit coring and semiseparability relative to a bimodule which extend those of cosplit coring and Sugano's separability relative to a bimodule, respectively.


title: Faster Born probability estimation via gate merging and frame optimisation

abstract: Outcome probability estimation via classical methods is an important task for validating quantum computing devices. Outcome probabilities of any quantum circuit can be estimated using Monte Carlo sampling, where the amount of negativity present in the circuit frame representation quantifies the overhead on the number of samples required to achieve a certain precision. In this paper, we propose two classical sub-routines: circuit gate merging and frame optimisation, which optimise the circuit representation to reduce the sampling overhead. We show that the runtimes of both sub-routines scale polynomially in circuit size and gate depth. Our methods are applicable to general circuits, regardless of generating gate sets, qudit dimensions and the chosen frame representations for the circuit components. We numerically demonstrate that our methods provide improved scaling in the negativity overhead for all tested cases of random circuits with Clifford+$T$ and Haar-random gates, and that the performance of our methods compares favourably with prior quasi-probability simulators as the number of non-Clifford gates increases.


title: Methodology for estimating the magnetic Prandtl number and application to solar surface small-scale dynamo simulations

abstract: Context. A crucial step in the numerical investigation of small-scale dynamos in the solar atmosphere consists of an accurate determination of the magnetic Prandtl number, Prm, stemming from radiative magneto-hydrodynamic (MHD) simulations. Aims. The aims are to provide a reliable methodology for estimating the effective Reynolds and magnetic Reynolds numbers, Re and Rem, and their ratio Prm=Rem/Re (the magnetic Prandlt number), that characterise MHD simulations and to categorise small-scale dynamo simulations in terms of these dimensionless parameters. Methods. The methodology proposed for computing Re and Rem is based on the method of projection on proper elements and it relies on a post-processing step carried out using higher order accurate numerical operators than the ones in the simulation code. A number of radiative MHD simulations with different effective viscosities and plasma resistivities were carried out with the CO5BOLD code, and the resulting growth rate of the magnetic energy and saturated magnetic field strengths were characterised in terms of Re and Rem. Results. Overall, the proposed methodology provides a solid estimate of the dissipation coefficients affecting the momentum and induction equations of MHD simulation codes, and consequently also a reliable evaluation of the magnetic Prandtl number characterising the numerical results. Additionally, it is found that small-scale dynamos are active and can amplify a small seed magnetic field up to significant values in CO5BOLD simulations with a grid spacing smaller than h=12 km, even at Prm=0.65. However, it is also evident that it is difficult to categorise dynamo simulations in terms of Prm alone, because it is not only important to estimate the amplitude of the dissipation coefficients, but also at which scales energy dissipation takes place.


title: Motion-driven Visual Tempo Learning for Video-based Action Recognition

abstract: Action visual tempo characterizes the dynamics and the temporal scale of an action, which is helpful to distinguish human actions that share high similarities in visual dynamics and appearance. Previous methods capture the visual tempo either by sampling raw videos with multiple rates, which require a costly multi-layer network to handle each rate, or by hierarchically sampling backbone features, which rely heavily on high-level features that miss fine-grained temporal dynamics. In this work, we propose a Temporal Correlation Module (TCM), which can be easily embedded into the current action recognition backbones in a plug-in-and-play manner, to extract action visual tempo from low-level backbone features at single-layer remarkably. Specifically, our TCM contains two main components: a Multi-scale Temporal Dynamics Module (MTDM) and a Temporal Attention Module (TAM). MTDM applies a correlation operation to learn pixel-wise fine-grained temporal dynamics for both fast-tempo and slow-tempo. TAM adaptively emphasizes expressive features and suppresses inessential ones via analyzing the global information across various tempos. Extensive experiments conducted on several action recognition benchmarks, e.g. Something-Something V1 $\&$ V2, Kinetics-400, UCF-101, and HMDB-51, have demonstrated that the proposed TCM is effective to promote the performance of the existing video-based action recognition models for a large margin. The source code is publicly released at https://github.com/yzfly/TCM.


title: Universal framework for the long-time position distribution of free active particles

abstract: Active particles self-propel themselves with a stochastically evolving velocity, generating a persistent motion leading to a non-diffusive behavior of the position distribution. Nevertheless, an effective diffusive behavior emerges at times much larger than the persistence time. Here we develop a general framework for studying the long-time behaviour for a class of active particle dynamics and illustrate it using the examples of run-and-tumble particle, active Ornstein-Uhlenbeck particle, active Brownian particle, and direction reversing active Brownian particle. Treating the ratio of the persistence-time to the observation time as the small parameter, we show that the position distribution generically satisfies the diffusion equation at the leading order. We further show that the sub-leading contributions, at each order, satisfies an inhomogeneous diffusion equation, where the source term depends on the previous order solutions. We explicitly obtain a few sub-leading contributions to the Gaussian position distribution. As a part of our framework, we also prescribe a way to find the position moments recursively and compute the first few explicitly for each model.


title: Gap probability and full counting statistics in the one dimensional one-component plasma

abstract: We consider the $1d$ one-component plasma (OCP) in thermal equilibrium, consisting of $N$ equally charged particles on a line, with pairwise Coulomb repulsion and confined by an external harmonic potential. We study two observables: (i) the distribution of the gap between two consecutive particles in the bulk and (ii) the distribution of the number of particles $N_I$ in a fixed interval $I=[-L,+L]$ inside the bulk, the so-called full-counting-statistics (FCS). For both observables, we compute, for large $N$, the distribution of the typical as well as atypical large fluctuations. We show that the distribution of the typical fluctuations of the gap are described by the scaling form ${\cal P}_{\rm gap, bulk}(g,N) \sim N H_\alpha(g\,N)$, where $\alpha$ is the interaction coupling and the scaling function $H_\alpha(z)$ is computed explicitly. It has a faster than Gaussian tail for large $z$: $H_\alpha(z) \sim e^{-z^3/(96 \alpha)}$ as $z \to \infty$. Similarly, for the FCS, we show that the distribution of the typical fluctuations of $N_I$ is described by the scaling form ${\cal P}_{\rm FCS}(N_I,N) \sim 2\alpha \, U_\alpha[2 \alpha(N_I - \bar{N}_I)]$, where $\bar{N}_I = L\,N/(2 \alpha)$ is the average value of $N_I$ and the scaling function $U_\alpha(z)$ is obtained explicitly. For both observables, we show that the probability of large fluctuations are described by large deviations forms with respective rate functions that we compute explicitly. Our numerical Monte-Carlo simulations are in good agreement with our analytical predictions.

categories: cond-mat.stat-mech math-ph math.MP math.PR


title: Optimal Convergence Rates of Deep Convolutional Neural Networks: Additive Ridge Functions

abstract: Convolutional neural networks have shown impressive abilities in many applications, especially those related to the classification tasks. However, for the regression problem, the abilities of convolutional structures have not been fully understood, and further investigation is needed. In this paper, we consider the mean squared error analysis for deep convolutional neural networks. We show that, for additive ridge functions, convolutional neural networks followed by one fully connected layer with ReLU activation functions can reach optimal mini-max rates (up to a log factor). The input dimension only appears in the constant of convergence rates. This work shows the statistical optimality of convolutional neural networks and may shed light on why convolutional neural networks are able to behave well for high dimensional input.


title: Temporal Convolution Domain Adaptation Learning for Crops Growth Prediction

abstract: Existing Deep Neural Nets on crops growth prediction mostly rely on availability of a large amount of data. In practice, it is difficult to collect enough high-quality data to utilize the full potential of these deep learning models. In this paper, we construct an innovative network architecture based on domain adaptation learning to predict crops growth curves with limited available crop data. This network architecture overcomes the challenge of data availability by incorporating generated data from the developed crops simulation model. We are the first to use the temporal convolution filters as the backbone to construct a domain adaptation network architecture which is suitable for deep learning regression models with very limited training data of the target domain. We conduct experiments to test the performance of the network and compare our proposed architecture with other state-of-the-art methods, including a recent LSTM-based domain adaptation network architecture. The results show that the proposed temporal convolution-based network architecture outperforms all benchmarks not only in accuracy but also in model size and convergence rate.


title: Modeling and Predicting Spatio-temporal Dynamics of PM$_{2.5}$ Concentrations Through Time-evolving Covariance Models

abstract: Fine particulate matter (PM$_{2.5}$) has become a great concern worldwide due to its adverse health effects. PM$_{2.5}$ concentrations typically exhibit complex spatio-temporal variations. Both the mean and the spatio-temporal dependence evolve with time due to seasonality, which makes the statistical analysis of PM$_{2.5}$ challenging. In geostatistics, Gaussian process is a powerful tool for characterizing and predicting such spatio-temporal dynamics, for which the specification of a spatio-temporal covariance function is the key. While the extant literature offers a wide range of choices for flexible stationary spatio-temporal covariance models, the temporally evolving spatio-temporal dependence has received scant attention only. To this end, we propose a time-varying spatio-temporal covariance model for describing the time-evolving spatio-temporal dependence in PM$_{2.5}$ concentrations. For estimation, we develop a composite likelihood-based procedure to handle large spatio-temporal datasets.The proposed model is shown to outperform traditionally used models through simulation studies in terms of predictions. We apply our model to analyze the PM$_{2.5}$ data in the state of Oregon, US. Therein, we show that the spatial scale and smoothness exhibit periodicity. The proposed model is also shown to be beneficial over traditionally used models on this dataset for predictions.


title: Demonstrating BrainScaleS-2 Inter-Chip Pulse-Communication using EXTOLL

abstract: The BrainScaleS-2 (BSS-2) Neuromorphic Computing System currently consists of multiple single-chip setups, which are connected to a compute cluster via Gigabit-Ethernet network technology. This is convenient for small experiments, where the neural networks fit into a single chip. When modeling networks of larger size, neurons have to be connected across chip boundaries. We implement these connections for BSS-2 using the EXTOLL networking technology. This provides high bandwidths and low latencies, as well as high message rates. Here, we describe the targeted pulse-routing implementation and required extensions to the BSS-2 software stack. We as well demonstrate feed-forward pulse-routing on BSS-2 using a scaled-down version without temporal merging.


title: An Information-theoretical Approach to Semi-supervised Learning under Covariate-shift

abstract: A common assumption in semi-supervised learning is that the labeled, unlabeled, and test data are drawn from the same distribution. However, this assumption is not satisfied in many applications. In many scenarios, the data is collected sequentially (e.g., healthcare) and the distribution of the data may change over time often exhibiting so-called covariate shifts. In this paper, we propose an approach for semi-supervised learning algorithms that is capable of addressing this issue. Our framework also recovers some popular methods, including entropy minimization and pseudo-labeling. We provide new information-theoretical based generalization error upper bounds inspired by our novel framework. Our bounds are applicable to both general semi-supervised learning and the covariate-shift scenario. Finally, we show numerically that our method outperforms previous approaches proposed for semi-supervised learning under the covariate shift.


title: Projected gradient-tracking in multi-cluster games and its application to power management

abstract: We are concerned with a distributed approach to solve multi-cluster games arising in multi-agent systems. In such games, agents are separated into distinct clusters. The agents belonging to the same cluster cooperate with each other to achieve a common cluster goal while a non-cooperative game is played between the clusters. To be able to deal with the sparsity of information, as each agent only knows a specific part of the problem, we combine gradient-tracking and consensus methods for information distribution into an algorithm that can solve both the cooperative and non-cooperative problem in a single run. The constraints of the problem are taken into account by the corresponding projection operators and linear convergence is proven given an appropriate constant step size. The algorithm is applied to a day-ahead power management problem, posed as a multi-cluster game, and its efficiency is demonstrated by simulations.


title: Extremal problems for trinomials with fold symmetry

abstract: The famous T. Suffridge polynomials have many extremal properties: the maximality of coefficients when the leading coefficient is maximal; the zeros of the derivative are located on the unit circle; the maximum radius of stretching the unit disk with the schlicht normalization $F(0)=0$, $F'(0)=1$; the maximum size of the unit disk contraction in the direction of the real axis for univalent polynomials with the normalization $F(0)=0$, $F(1)=1.$ However, under the standard symmetrization method $\sqrt[T]{F(z^T)}$, these polynomials go to functions, which are not polynomials. How can we construct the polynomials with fold symmetry that have properties similar to those of the Suffridge polynomial? What values will the corresponding extremal quantities take in the above-mentioned extremal problems? The paper is devoted to solving these questions for the case of the trinomials $F(z)=z+az^{1+T}+bz^{1+2T}$. Also, there are suggested hypotheses for the general case in the work.


title: Zeros of derivatives of $L$-functions in the Selberg class on $\Re(s)<1/2$

abstract: In this article, we show that the Riemann hypothesis for an $L$-function $F$ belonging to the Selberg class implies that all the derivatives of $F$ can have at most finitely many zeros on the left of the critical line with imaginary part greater than a certain constant. This was shown for the Riemann zeta function by Levinson and Montgomery in 1974.


title: Robust random walk-like Metropolis-Hastings algorithms for concentrating posteriors

abstract: Motivated by Bayesian inference with highly informative data we analyze the performance of random walk-like Metropolis-Hastings algorithms for approximate sampling of increasingly concentrating target distributions. We focus on Gaussian proposals which use a Hessian-based approximation of the target covariance. By means of pushforward transition kernels we show that for Gaussian target measures the spectral gap of the corresponding Metropolis-Hastings algorithm is independent of the concentration of the posterior, i.e., the noise level in the observational data that is used for Bayesian inference. Moreover, by exploiting the convergence of the concentrating posteriors to their Laplace approximation we extend the analysis to non-Gaussian target measures which either concentrate around a single point or along a linear manifold. In particular, in that setting we show that the average acceptance rate as well as the expected squared jump distance of suitable Metropolis-Hastings Markov chains do not deteriorate as the target concentrates.

categories: stat.CO cs.NA math.NA math.PR math.ST stat.TH


title: After-sales services during an asset's lifetime: collaborative planning of system upgrades

abstract: We consider a physical asset consisting of complex systems, where the systems may require upgrades during the lifetime of the asset. In practice, the asset owner and system supplier can make the upgrade decisions together, requiring a decision support model that can be jointly used to optimize the total benefit for both parties. Motivated by a real-life use case including an asset owner and a system supplier, we build a continuous-time model to optimize the upgrade decisions of a system during the fixed lifetime of the asset. In our model, we capture the key critical factors that drive the upgrade decisions: increasing functionality requirements due to evolving technology, age-dependent maintenance costs, a predetermined overhaul plan of the asset, and the lifetime of the asset. A system upgrade is less costly if it is executed jointly with an asset overhaul. We first analyze the case with no additional cost of upgrading outside an overhaul. We analytically characterize the structure of the optimal upgrade policy under various realistic assumptions that lead to different types of cost functions. We then use these results as a building block to characterize the optimal policy for a generalized cost function. When there is a penalty for upgrading outside an overhaul moment, we propose a dynamic programming approach that efficiently determines the optimal upgrade policy by using our analytical results. We also prove that as this penalty increases, the optimal policy can only change to one where the number of upgrades not jointly executed with overhauls is reduced. However, the optimal number of upgrades is a non-increasing function of this penalty. Also surprisingly, more overhauls may lead to a smaller number of upgrades under the optimal policy.


title: Symmetries at Causal Boundaries in 2D and 3D Gravity

abstract: We study 2d and 3d gravity theories on spacetimes with causal (timelike or null) codimension one boundaries while allowing for variations in the position of the boundary. We construct the corresponding solution phase space and specify boundary degrees freedom by analysing boundary (surface) charges labelling them. We discuss Y and W freedoms and change of slicing in the solution space. For D dimensional case we find D+1 surface charges, which are generic functions over the causal boundary. We show that there exist solution space slicings in which the charges are integrable. For the 3d case there exists an integrable slicing where charge algebra takes the form of Heisenberg \oplus\ {\cal A}_3 where {\cal A}_3 is two copies of Virasoro at Brown-Henneaux central charge for AdS_3 gravity and BMS_3 for the 3d flat space gravity.


title: Lipschitz stable determination of polyhedral conductivity inclusions from local boundary measurements

abstract: We consider the problem of determining a polyhedral conductivity inclusion embedded in a homogeneous isotropic medium from boundary measurements. We prove global Lipschitz stability for the polyhedral inclusion from the local Dirichlet-to-Neumann map extending in a highly nontrivial way the results obtained in [18] and [19] in the two-dimensional case to the three-dimensional setting.


title: The Shortest Path with Increasing Chords in a Simple Polygon

abstract: We study the problem of finding the shortest path with increasing chords in a simple polygon. A path has increasing chords if and only if for any points a, b, c, and d that lie on the path in that order, |ad| >= |bc|. In this paper we show that the shortest path with increasing chords is unique and present an algorithm to construct it.


title: "splink" is happy and "phrouth" is scary: Emotion Intensity Analysis for Nonsense Words

abstract: People associate affective meanings to words - "death" is scary and sad while "party" is connotated with surprise and joy. This raises the question if the association is purely a product of the learned affective imports inherent to semantic meanings, or is also an effect of other features of words, e.g., morphological and phonological patterns. We approach this question with an annotation-based analysis leveraging nonsense words. Specifically, we conduct a best-worst scaling crowdsourcing study in which participants assign intensity scores for joy, sadness, anger, disgust, fear, and surprise to 272 non-sense words and, for comparison of the results to previous work, to 68 real words. Based on this resource, we develop character-level and phonology-based intensity regressors. We evaluate them on both nonsense words and real words (making use of the NRC emotion intensity lexicon of 7493 words), across six emotion categories. The analysis of our data reveals that some phonetic patterns show clear differences between emotion intensities. For instance, s as a first phoneme contributes to joy, sh to surprise, p as last phoneme more to disgust than to anger and fear. In the modelling experiments, a regressor trained on real words from the NRC emotion intensity lexicon shows a higher performance (r = 0.17) than regressors that aim at learning the emotion connotation purely from nonsense words. We conclude that humans do associate affective meaning to words based on surface patterns, but also based on similarities to existing words ("juy" to "joy", or "flike" to "like").


title: Conformally curved initial data for charged, spinning black hole binaries on arbitrary orbits

abstract: We present a method to construct conformally curved initial data for charged black hole binaries with spin on arbitrary orbits. We generalize the superposed Kerr-Schild, extended conformal thin sandwich construction from [Lovelace et al., Phys. Rev. D {78}, 084017 (2008)] to use Kerr-Newman metrics for the superposed black holes and to solve the electromagnetic constraint equations. We implement the construction in the pseudospectral code SGRID. The code thus provides a complementary and completely independent excision-based construction, compared to the existing charged black hole initial data constructed using the puncture method [Bozzola and Paschalidis, Phys. Rev. D {99}, 104044 (2019)]. It also provides an independent implementation (with some small changes) of the Lovelace et al. vacuum construction. We construct initial data for different configurations of orbiting binaries, e.g., with black holes that are highly charged or rapidly spinning (90 and 80 percent of the extremal values, respectively, for this initial test, though the code should be able to produce data with even higher values of these parameters using higher resolutions), as well as for generic spinning, charged black holes. We carry out exploratory evolutions with the finite difference, moving punctures codes BAM (in the vacuum case) and HAD (for head-on collisions including charge), filling inside the excision surfaces. In the charged case, evolutions of these initial data provide a proxy for binary black hole waveforms in modified theories of gravity. Moreover, the generalization of the construction to Einstein-Maxwell-dilaton theory should be straightforward.


title: Measurements of differential cross-sections in top-quark pair events with a high transverse momentum top quark and limits on beyond the Standard Model contributions to top-quark pair production with the ATLAS detector at $\sqrt{s}=13$ TeV

abstract: Cross-section measurements of top-quark pair production where the hadronically decaying top quark has transverse momentum greater than $355$ GeV and the other top quark decays into $\ell \nu b$ are presented using 139 fb$^{-1}$ of data collected by the ATLAS experiment during proton-proton collisions at the LHC. The fiducial cross-section at $\sqrt{s}=13$ TeV is measured to be $\sigma = 1.267 \pm 0.005 \pm 0.053$ pb, where the uncertainties reflect the limited number of data events and the systematic uncertainties, giving a total uncertainty of $4.2\%$. The cross-section is measured differentially as a function of variables characterising the $t\bar{t}$ system and additional radiation in the events. The results are compared with various Monte Carlo generators, including comparisons where the generators are reweighted to match a parton-level calculation at next-to-next-to-leading order. The reweighting improves the agreement between data and theory. The measured distribution of the top-quark transverse momentum is used to set limits on the Wilson coefficients of the dimension-six operators $O_{tG}$ and $O_{tq}^{(8)}$ in the effective field theory framework.


title: McKay correspondence and orbifold equivalence

abstract: We prove that a pair of singularities related by a transformation arising from the McKay correspondence are orbifold equivalent. From this we deduce a new proof of a McKay type equivalence for the matrix factorization categories.


title: A phase-field approach for detecting cavities via a Kohn-Vogelius type functional

abstract: We deal with the geometrical inverse problem of the shape reconstruction of cavities in a bounded linear isotropic medium by means of boundary data. The problem is addressed from the point of view of optimal control: the goal is to minimize in the class of Lipschitz domains a Kohn-Vogelius type functional with a perimeter regularization term which penalizes the perimeter of the cavity to be reconstructed. To solve numerically the optimization problem, we use a phase-field approach, approximating the perimeter functional with a Modica-Mortola relaxation and modeling the cavity as an inclusion with a very small elastic tensor. We provide a detailed analysis showing the robustness of the algorithm through some numerical experiments.


title: From Zero-Intelligence to Queue-Reactive: Limit Order Book modeling for high-frequency volatility estimation and optimal execution

abstract: The estimation of the volatility with high-frequency data is plagued by the presence of microstructure noise, which leads to biased measures. Alternative estimators have been developed and tested either on specific structures of the noise or by the speed of convergence to their asymptotic distributions. Gatheral and Oomen (2010) proposed to use the Zero-Intelligence model of the limit order book to test the finite-sample performance of several estimators of the integrated variance. Building on this approach, in this paper we introduce three main innovations: (i) we use as data-generating process the Queue-Reactive model of the limit order book (Huang et al. (2015)), which - compared to the Zero-Intelligence model - generates more realistic microstructure dynamics, as shown here by using an Hausman test; (ii) we consider not only estimators of the integrated volatility but also of the spot volatility; (iii) we show the relevance of the estimator in the prediction of the variance of the cost of a simulated VWAP execution. Overall we find that, for the integrated volatility, the pre-averaging estimator optimizes the estimation bias, while the unified and the alternation estimator lead to optimal mean squared error values. Instead, in the case of the spot volatility, the Fourier estimator yields the optimal accuracy, both in terms of bias and mean squared error. The latter estimator leads also to the optimal prediction of the cost variance of a VWAP execution.


title: How reparametrization trick broke differentially-private text representation learning

abstract: As privacy gains traction in the NLP community, researchers have started adopting various approaches to privacy-preserving methods. One of the favorite privacy frameworks, differential privacy (DP), is perhaps the most compelling thanks to its fundamental theoretical guarantees. Despite the apparent simplicity of the general concept of differential privacy, it seems non-trivial to get it right when applying it to NLP. In this short paper, we formally analyze several recent NLP papers proposing text representation learning using DPText (Beigi et al., 2019a,b; Alnasser et al., 2021; Beigi et al., 2021) and reveal their false claims of being differentially private. Furthermore, we also show a simple yet general empirical sanity check to determine whether a given implementation of a DP mechanism almost certainly violates the privacy loss guarantees. Our main goal is to raise awareness and help the community understand potential pitfalls of applying differential privacy to text representation learning.


title: Testing Deep Learning Models: A First Comparative Study of Multiple Testing Techniques

abstract: Deep Learning (DL) has revolutionized the capabilities of vision-based systems (VBS) in critical applications such as autonomous driving, robotic surgery, critical infrastructure surveillance, air and maritime traffic control, etc. By analyzing images, voice, videos, or any type of complex signals, DL has considerably increased the situation awareness of these systems. At the same time, while relying more and more on trained DL models, the reliability and robustness of VBS have been challenged and it has become crucial to test thoroughly these models to assess their capabilities and potential errors. To discover faults in DL models, existing software testing methods have been adapted and refined accordingly. In this article, we provide an overview of these software testing methods, namely differential, metamorphic, mutation, and combinatorial testing, as well as adversarial perturbation testing and review some challenges in their deployment for boosting perception systems used in VBS. We also provide a first experimental comparative study on a classical benchmark used in VBS and discuss its results.


title: Studying Aspects of the Early Universe with Primordial Black Holes

abstract: This thesis by publication is devoted to the study of aspects of the early universe in the context of primordial black hole (PBH) physics. Firstly, we review the fundamentals of the early universe cosmology and we recap the basics of the PBHs physics. In particular, we propose a refinement in the determination of the PBH formation threshold, a fundamental quantity in PBH physics, in the context of a time-dependent equation-of-state parameter. Afterwards, we briefly present the theory of inflationary perturbations, which is the theoretical framework within which PBHs are studied in this thesis. Then, in the second part of the thesis, we review the core of the research conducted within my PhD, in which aspects of the early universe and the gravitational wave physics are combined with the physics of PBHs. Moreover, aspects of the PBH gravitational collapse process are studied in the presence of anisotropies. Specifically, we study PBHs produced from the preheating instability in the context of single-field inflation. Interestingly, we find that PBHs produced during preheating can potentially dominate the universe's content and drive reheating through their evaporation. Then, we focus on the scalar induced second-order stochastic gravitational wave background (SGWB) induced from Poisson energy density fluctuations of ultralight PBHs. By taking then into account gravitational wave backreaction effects we set model-independent constraints on the initial abundance of ultralight PBHs as a function of their mass. Afterwards, we study in a covariant way the anisotropic spherical gravitational collapse of PBHs during a radiation-dominated era in which one can compute the PBH formation threshold as a function of the anisotropy. Finally, we summarize our research results by discussing future prospects opened up as a result of the research work conducted within this thesis.


title: Bilateral series and Ramanujan's radial limits

abstract: Ramanujan's last letter to Hardy explored the asymptotic properties of modular forms, as well as those of certain interesting $q$-series which he called \emph{mock theta functions}. For his mock theta function $f(q)$, he claimed that as $q$ approaches an even order $2k$ root of unity $\zeta$, \[\lim_{q\to \zeta} \big(f(q) - (-1)^k (1-q)(1-q^3)(1-q^5)\cdots (1-2q + 2q^4 - \cdots)\big) = O(1),\] and hinted at the existence of similar statements for his other mock theta functions. Recent work of Folsom-Ono-Rhoades provides a closed formula for the implied constant in this radial limit of $f(q)$. Here, by different methods, we prove similar results for all of Ramanujan's 5th order mock theta functions. Namely, we show that each 5th order mock theta function may be related to a modular bilateral series, and exploit this connection to obtain our results. We further explore other mock theta functions to which this method can be applied.


title: Pretraining without Wordpieces: Learning Over a Vocabulary of Millions of Words

abstract: The standard BERT adopts subword-based tokenization, which may break a word into two or more wordpieces (e.g., converting "lossless" to "loss" and "less"). This will bring inconvenience in following situations: (1) what is the best way to obtain the contextual vector of a word that is divided into multiple wordpieces? (2) how to predict a word via cloze test without knowing the number of wordpieces in advance? In this work, we explore the possibility of developing BERT-style pretrained model over a vocabulary of words instead of wordpieces. We call such word-level BERT model as WordBERT. We train models with different vocabulary sizes, initialization configurations and languages. Results show that, compared to standard wordpiece-based BERT, WordBERT makes significant improvements on cloze test and machine reading comprehension. On many other natural language understanding tasks, including POS tagging, chunking and NER, WordBERT consistently performs better than BERT. Model analysis indicates that the major advantage of WordBERT over BERT lies in the understanding for low-frequency words and rare words. Furthermore, since the pipeline is language-independent, we train WordBERT for Chinese language and obtain significant gains on five natural language understanding datasets. Lastly, the analyse on inference speed illustrates WordBERT has comparable time cost to BERT in natural language understanding tasks.


title: Flexural deformations and collapse of bilayer two-dimensional crystals by interlayer exciton

abstract: We develop a consistent theory of the interlayer exciton-polaron formed in atomically-thin bilayers. Coulomb attraction between an electron and a hole situated in the different layers results in their flexural deformation and provides an efficient mechanism of the exciton coupling with flexural phonons. We study the effect of layers tension on the polaron binding energy and effective mass leading to suppression of polaron formation by the tension both in the weak and strong coupling regimes. We also consider the role of the nonlinearity related to the interaction between the out- and in-plane lattice displacements and obtain the criterion of the layer sticking, where the exciton collapses, due to the Coulomb attraction between the charge carriers.


title: A note on the minimal tensor product and the C*-envelope of operator systems

abstract: In this article, we show that the $C^*$-envelope of the minimal tensor product of two operator systems is isomorphic to the minimal tensor product of their $C^*$-envelopes. We do this by identifying the Silov boundary ideal of the minimal tensor product of two operator systems. Finally, as an application of this result, we show that the propagation number of the minimal tensor product of operator systems is the maximum of the propagation numbers of the factors.


title: Scalable single-mode Berkeley Surface Emitting Lasers

abstract: The scaling of electromagnetic apertures is a long-standing question that has been investigated for at least six decades but has still not been resolved [1-5]. The size of single aperture cavities, bounded by the existence of higher-order transverse modes, fundamentally limits the power emitted by single-mode lasers or the brightness of quantum light sources. The free-spectral range of existing electromagnetic apertures goes to zero when the size of the aperture increases, and the demonstration of scale-invariant apertures or lasers has remained elusive. Here, we report open-Dirac electromagnetic apertures that exploit a subtle cavity-mode-dependent scaling of losses in reciprocal space. For cavities with a quadratic dispersion, the complex frequencies of modes converge towards each other with the size of cavities, making cavities invariably multimode. Surprisingly, for a class of cavities with linear dispersion, we discover that, while the convergence of the real parts of cavity modes towards each other is delayed (it still quickly goes to zero), the normalized complex free-spectral range converge towards a constant governed by the loss rate of distinct Bloch bands. We show that this unconventional scaling of the complex frequency of cavity modes in open-Dirac electromagnetic apertures guarantees single-mode operation of large cavities. We experimentally demonstrate that single-mode lasing of such cavities is maintained when the cavity is scale up in size. We name such sources Berkeley Surface Emitting Lasers (BerkSELs). We further show that the far-field emitted by the proposed single-mode BerkSELs corresponds to a topological singularity of charge two, in full agreement with our theory. Open-Dirac apertures open new avenues for light-matter interaction and basic science in open-wave systems with applications in classical and quantum communications, sensing, and imaging.


title: Ultrahigh reversible hydrogen storage in K and Ca decorated 4-6-8 biphenylene sheet

abstract: By applying density functional theory (DFT) and ab-initio molecular dynamics (AIMD) simulations, we predict the ultrahigh hydrogen storage capacity of K and Ca decorated single-layer biphenylene sheet (BPS). We have kept various alkali and alkali earth metals, including Na, Be, Mg, K, Ca, at different sites of BPS and found that K and Ca atoms prefer to bind individually on the BPS instead of forming clusters. It was found that 2x2x1 supercell of biphenylene sheet can adsorb eight K, or eight Ca atoms, and each K or Ca atom can adsorb 5 H$_2$, leading to 11.90 % or 11.63 % of hydrogen uptake, respectively, which is significantly higher than the DOE-US demands of 6.5 %. The average adsorption energy of H$_2$ for K and Ca decorated BPS is -0.24 eV and -0.33 eV, respectively, in the suitable range for reversible H$_2$ storage. Hydrogen molecules get polarized in the vicinity of ionized metal atoms hence get attached to the metal atoms through electrostatic and van der Waals interactions. We have estimated the desorption temperatures of H$_2$ and found that the adsorbed H$_2$ can be utilized for reversible use. We have found that a sufficient energy barrier of 2.52 eV exists for the movement of Ca atoms, calculated using the climbing-image nudged elastic band (CI-NEB) method. This energy barrier can prevent the clustering issue of Ca atoms. The solidity of K and Ca decorated BPS structures were investigated using AIMD simulations.


title: Lattice Boltzmann simulations of two linear microswimmers using the immersed boundary method

abstract: The performance of a single or the collection of microswimmers strongly depends on the hydrodynamic coupling among their constituents and themselves. We present a numerical study for a single and a pair of microswimmers based on lattice Boltzmann method (LBM) simulations. Our numerical algorithm consists of two separable parts. Lagrange polynomials provide a discretization of the microswimmers and the lattice Boltzmann method captures the dynamics of the surrounding fluid. The two components couple via an immersed boundary method. We present data for a single swimmer system and our data also show the onset of collective effects and, in particular, an overall velocity increment of clusters of swimmers.


title: A novel unsupervised covid lung lesion segmentation based on the lung tissue identification

abstract: This study aimed to evaluate the performance of a novel unsupervised deep learning-based framework for automated infections lesion segmentation from CT images of Covid patients. In the first step, two residual networks were independently trained to identify the lung tissue for normal and Covid patients in a supervised manner. These two models, referred to as DL-Covid and DL-Norm for Covid-19 and normal patients, respectively, generate the voxel-wise probability maps for lung tissue identification. To detect Covid lesions, the CT image of the Covid patient is processed by the DL-Covid and DL-Norm models to obtain two lung probability maps. Since the DL-Norm model is not familiar with Covid infections within the lung, this model would assign lower probabilities to the lesions than the DL-Covid. Hence, the probability maps of the Covid infections could be generated through the subtraction of the two lung probability maps obtained from the DL-Covid and DL-Norm models. Manual lesion segmentation of 50 Covid-19 CT images was used to assess the accuracy of the unsupervised lesion segmentation approach. The Dice coefficients of 0.985 and 0.978 were achieved for the lung segmentation of normal and Covid patients in the external validation dataset, respectively. Quantitative results of infection segmentation by the proposed unsupervised method showed the Dice coefficient and Jaccard index of 0.67 and 0.60, respectively. Quantitative evaluation of the proposed unsupervised approach for Covid-19 infectious lesion segmentation showed relatively satisfactory results. Since this framework does not require any annotated dataset, it could be used to generate very large training samples for the supervised machine learning algorithms dedicated to noisy and/or weakly annotated datasets.


title: Phonon thermal Hall effect in a metallic spin ice

abstract: It has become common knowledge that phonons can generate thermal Hall effect in a wide variety of materials, although the underlying mechanism is still controversial. We study longitudinal $\kappa_{xx}$ and transverse $\kappa_{xy}$ thermal conductivity in Pr$_2$Ir$_2$O$_7$, which is a metallic analogue of spin ice. Despite the presence of mobile charge carriers, we find that both $\kappa_{xx}$ and $\kappa_{xy}$ are dominated by phonons. A $T/H$ scaling of $\kappa_{xx}$ unambiguously reveals that longitudinal heat current is substantially impeded by resonant scattering of phonons on paramagnetic spins. Upon cooling, the resonant scattering is strongly affected by a development of spin ice correlation and $\kappa_{xx}$ deviates from the scaling in an anisotropic way with respect to field directions. Strikingly, a set of the $\kappa_{xx}$ and $\kappa_{xy}$ data clearly shows that $\kappa_{xy}$ correlates with $\kappa_{xx}$ in its response to magnetic field including a success of the $T/H$ scaling and its failure at low temperature. This remarkable correlation provides solid evidence that an indispensable role is played by spin-phonon scattering not only for hindering the longitudinal heat conduction, but also for generating the transverse response.


title: Tighter Expected Generalization Error Bounds via Convexity of Information Measures

abstract: Generalization error bounds are essential to understanding machine learning algorithms. This paper presents novel expected generalization error upper bounds based on the average joint distribution between the output hypothesis and each input training sample. Multiple generalization error upper bounds based on different information measures are provided, including Wasserstein distance, total variation distance, KL divergence, and Jensen-Shannon divergence. Due to the convexity of the information measures, the proposed bounds in terms of Wasserstein distance and total variation distance are shown to be tighter than their counterparts based on individual samples in the literature. An example is provided to demonstrate the tightness of the proposed generalization error bounds.


title: Magnon corner states in twisted bilayer honeycomb magnets

abstract: Search for higher-order topological insulators, characterized by topologically protected gapless boundary states of codimension higher than one, in bosonic systems has attracted growing interest. Here, we establish twisted bilayer honeycomb magnets as a new platform for hosting second-order topological magnon insulators (SOTMIs) without fine-tuning. We employ a simple, minimal Heisenberg spin model to describe misaligned bilayer sheets of honeycomb ferromagnetic magnets with a large commensurate twist angle. We found that the higher-order topology in this bilayer system shows a significant dependence on the interlayer exchange coupling. The SOTMI, featuring topologically protected magnon corner states, appears for ferromagnetic interlayer couplings, while the twisted bilayer exhibits a nodal phase in the case of antiferromagnetic interlayer coupling.


title: Discrete approximation of the Griffith functional by adaptive finite elements

abstract: This paper is devoted to show a discrete adaptive finite element approximation result for the isotropic two-dimensional Griffith energy arising in fracture mechanics. The problem is addressed in the geometric measure theoretic framework of generalized special functions of bounded deformation which corresponds to the natural energy space for this functional. It is proved to be approximated in the sense of $\Gamma$-convergence by a sequence of discrete integral functionals defined on continuous piecewise affine functions. The main feature of this result is that the mesh is part of the unknown of the problem, and it gives enough flexibility to recover isotropic surface energies.


title: Estimates of Lebesgue Constants for Lagrange Interpolation Processes by Rational Functions under Mild Restrictions to their Fixed Poles

abstract: We estimate the Lebesgue constants for Lagrange interpolation processes on one or several intervals by rational functions with fixed poles. We admit that the poles have accumulation points on the intervals. To prove it we use an analog of the inverse polynomial image method for rational functions with fixed poles.


title: Towards Effective and Robust Neural Trojan Defenses via Input Filtering

abstract: Trojan attacks on deep neural networks are both dangerous and surreptitious. Over the past few years, Trojan attacks have advanced from using only a single input-agnostic trigger and targeting only one class to using multiple, input-specific triggers and targeting multiple classes. However, Trojan defenses have not caught up with this development. Most defense methods still make inadequate assumptions about Trojan triggers and target classes, thus, can be easily circumvented by modern Trojan attacks. To deal with this problem, we propose two novel "filtering" defenses called Variational Input Filtering (VIF) and Adversarial Input Filtering (AIF) which leverage lossy data compression and adversarial learning respectively to effectively purify potential Trojan triggers in the input at run time without making assumptions about the number of triggers/target classes or the input dependence property of triggers. In addition, we introduce a new defense mechanism called "Filtering-then-Contrasting" (FtC) which helps avoid the drop in classification accuracy on clean data caused by "filtering", and combine it with VIF/AIF to derive new defenses of this kind. Extensive experimental results and ablation studies show that our proposed defenses significantly outperform well-known baseline defenses in mitigating five advanced Trojan attacks including two recent state-of-the-art while being quite robust to small amounts of training data and large-norm triggers.


title: Cyclicity of Rigid Centers on Center Manifolds of Three-dimensional systems

abstract: We work with polynomial three-dimensional rigid differential systems. Using the Lyapunov constants, we obtain lower bounds for the cyclicity of the known rigid centers on their center manifolds. Moreover, we obtain an example of a quadratic rigid center from which is possible to bifurcate 13 limit cycles, which is a new lower bound for three-dimensional quadratic systems.


title: A Monte Carlo Method for Rayleigh Scattering in Liquid Detectors

abstract: A new Monte Carlo method has been implemented to describe the angular and polarization distributions of anisotropic liquids, like water and linear alkylbenzene, by considering orientational fluctuations of polarizability tensors. The scattered light of anisotropic liquids is depolarized with an angular distribution of $1+(1-\rho_v)/(1+3\rho_v)\cos^2\theta$, which is modified by the depolarization ratio $\rho_v$. A standalone experiment has validated the simulation results of LAB. The new method can provide more accurate knowledge on light propagation in large liquid detectors, which is beneficial to the developments of reconstruction for detectors.


title: Gluon Fusion Production at NLO: Merging the Transverse Momentum and the High-Energy Expansions

abstract: The virtual corrections to $gg\to HH$ and $gg\to ZH$ are analytically evaluated combining an expansion in the small transverse momentum of the final particles with an expansion valid at high energies. The two expansion methods describe complementary regions of the phase space and we merge their results, extending the range of validity of both expansions using Pad\'e approximants. We show that this approach can reproduce the available numerical results retaining the exact top quark mass dependence with an accuracy well below the 1% level. Our results allow a fast and flexible evaluation of the virtual corrections of the considered processes. Furthermore, they are available in different renormalisation schemes of the top quark mass.


title: Tube Stochastic Optimal Control for Nonlinear Constrained Trajectory Optimization Problems

abstract: Recent low-thrust space missions have highlighted the importance of designing trajectories that are robust against uncertainties. In its complete form, this process is formulated as a nonlinear constrained stochastic optimal control problem. This problem is among the most complex in control theory, and no practically applicable method to low-thrust trajectory optimization problems has been proposed to date. This paper presents a new algorithm to solve stochastic optimal control problems with nonlinear systems and constraints. The proposed algorithm uses the unscented transform to convert a stochastic optimal control problem into a deterministic problem, which is then solved by trajectory optimization methods such as differential dynamic programming. Two numerical examples, one of which applies the proposed method to low-thrust trajectory design, illustrate that it automatically introduces margins that improve robustness. Finally, Monte Carlo simulations are used to evaluate the robustness and optimality of the solution.


title: An NLP Solution to Foster the Use of Information in Electronic Health Records for Efficiency in Decision-Making in Hospital Care

abstract: The project aimed to define the rules and develop a technological solution to automatically identify a set of attributes within free-text clinical records written in Portuguese. The first application developed and implemented on this basis was a structured summary of a patient's clinical history, including previous diagnoses and procedures, usual medication, and relevant characteristics or conditions for clinical decisions, such as allergies, being under anticoagulant therapy, etc. The project's goal was achieved by a multidisciplinary team that included clinicians, epidemiologists, computational linguists, machine learning researchers and software engineers, bringing together the expertise and perspectives of a public hospital, the university and the private sector. Relevant benefits to users and patients are related with facilitated access to the patient's history, which translates into exhaustiveness in apprehending the patient's clinical past and efficiency due to time saving.


title: Self-Attention for Incomplete Utterance Rewriting

abstract: Incomplete utterance rewriting (IUR) has recently become an essential task in NLP, aiming to complement the incomplete utterance with sufficient context information for comprehension. In this paper, we propose a novel method by directly extracting the coreference and omission relationship from the self-attention weight matrix of the transformer instead of word embeddings and edit the original text accordingly to generate the complete utterance. Benefiting from the rich information in the self-attention weight matrix, our method achieved competitive results on public IUR datasets.


title: Line-field Coherent Sensing with LED Illumination

abstract: We describe a method of low-coherence interferometry based optical profilometry using standard light-emitting diode (LED) illumination and complementary metal-oxide-semiconductor (CMOS) image sensors. A line-field illumination strategy allows for the simultaneous measurement of many points in space. Micron scale accuracy and resolution are achieved and demonstrated using a variety of targets.


title: Measuring CLEVRness: Blackbox testing of Visual Reasoning Models

abstract: How can we measure the reasoning capabilities of intelligence systems? Visual question answering provides a convenient framework for testing the model's abilities by interrogating the model through questions about the scene. However, despite scores of various visual QA datasets and architectures, which sometimes yield even a super-human performance, the question of whether those architectures can actually reason remains open to debate. To answer this, we extend the visual question answering framework and propose the following behavioral test in the form of a two-player game. We consider black-box neural models of CLEVR. These models are trained on a diagnostic dataset benchmarking reasoning. Next, we train an adversarial player that re-configures the scene to fool the CLEVR model. We show that CLEVR models, which otherwise could perform at a human level, can easily be fooled by our agent. Our results put in doubt whether data-driven approaches can do reasoning without exploiting the numerous biases that are often present in those datasets. Finally, we also propose a controlled experiment measuring the efficiency of such models to learn and perform reasoning.


title: Attentive Temporal Pooling for Conformer-based Streaming Language Identification in Long-form Speech

abstract: In this paper, we introduce a novel language identification system based on conformer layers. We propose an attentive temporal pooling mechanism to allow the model to carry information in long-form audio via a recurrent form, such that the inference can be performed in a streaming fashion. Additionally, we investigate two domain adaptation approaches to allow adapting an existing language identification model without retraining the model parameters for a new domain. We perform a comparative study of different model topologies under different constraints of model size, and find that conformer-based models significantly outperform LSTM and transformer based models. Our experiments also show that attentive temporal pooling and domain adaptation improve model accuracy.


title: The Dunkl-Laplace transform and Macdonald's hypergeometric series

abstract: We continue a program generalizing classical results from the analysis on symmetric cones to the Dunkl setting for root systems of type A. In particular, we prove a Dunkl-Laplace transform identity for Heckman-Opdam hypergeometric functions of type A and more generally, for the associated Cherednik kernel. This is achieved by analytic continuation from a Laplace transform identity for non-symmetric Jack polynomials which was stated, for the symmetric case, as a key conjecture in an unpublished manuscript of Macdonald (2013). Our proof for the Jack polynomials is based on Dunkl operator techniques and the raising operator of Knop and Sahi. Moreover, we use these results to establish Laplace transform identities between hypergeometric series in terms of Jack polynomials. Finally, we conclude with a Post-Widder inversion formula for the Dunkl-Laplace transform.


title: Transformers in Medical Image Analysis: A Review

abstract: Transformers have dominated the field of natural language processing, and recently impacted the computer vision area. In the field of medical image analysis, Transformers have also been successfully applied to full-stack clinical applications, including image synthesis/reconstruction, registration, segmentation, detection, and diagnosis. Our paper aims to promote awareness and application of Transformers in the field of medical image analysis. Specifically, we first overview the core concepts of the attention mechanism built into Transformers and other basic components. Second, we review various Transformer architectures tailored for medical image applications and discuss their limitations. Within this review, we investigate key challenges revolving around the use of Transformers in different learning paradigms, improving the model efficiency, and their coupling with other techniques. We hope this review can give a comprehensive picture of Transformers to the readers in the field of medical image analysis.


title: Attention Enables Zero Approximation Error

abstract: Deep learning models have been widely applied in various aspects of daily life. Many variant models based on deep learning structures have achieved even better performances. Attention-based architectures have become almost ubiquitous in deep learning structures. Especially, the transformer model has now defeated the convolutional neural network in image classification tasks to become the most widely used tool. However, the theoretical properties of attention-based models are seldom considered. In this work, we show that with suitable adaptations, the single-head self-attention transformer with a fixed number of transformer encoder blocks and free parameters is able to generate any desired polynomial of the input with no error. The number of transformer encoder blocks is the same as the degree of the target polynomial. Even more exciting, we find that these transformer encoder blocks in this model do not need to be trained. As a direct consequence, we show that the single-head self-attention transformer with increasing numbers of free parameters is universal. These surprising theoretical results clearly explain the outstanding performances of the transformer model and may shed light on future modifications in real applications. We also provide some experiments to verify our theoretical result.


title: Role of surfactant-induced Marangoni stresses in retracting liquid sheets

abstract: In this work, we study the effect of insoluble surfactants on the three-dimensional rim-driven retraction dynamics of thin water sheets in air. We employ an interface-tracking/level-set method to ensure the full coupling between the surfactant-induced Marangoni-stresses, interfacial diffusion, and inertia. Our findings are contrasted with the (Newtonian) dynamics of a liquid sheet edge finding that the surfactant concentration can prevent, or delay, the breakup of the rim. Our simulations use the fastest growing Rayleigh-Plateau instability to drive droplet detachment from the fluid sheet (rim). The results of this work unravel the significant role of Marangoni stresses in the retracting sheet dynamics at large elasticity numbers. We study the sensitivity of the dynamics to the elasticity number and the rigidification of the interface.


title: Entropies in $\mu$-framework of canonical metrics and K-stability, II -- Non-archimedean aspect: non-archimedean $\mu$-entropy and $\mu$K-semistability

abstract: This is the second in a series of two papers studying $\mu$-cscK metrics and $\mu$K-stability from a new perspective, inspired by observations on $\mu$-character in arXiv:2004.06393 and on Perelman's $W$-entropy in the first paper arXiv:2101.11197. This second paper is devoted to studying a non-archimedean counterpart of Perelman's $\mu$-entropy. The concept originally appeared as $\mu$-character of polarized family in the previous research arXiv:2004.06393, where we used it to introduce an analogue of CM line bundle adapted to $\mu$K-stability. We firstly show some differential of the characteristic $\mu$-entropy $\mathbf{\check{\mu}}^\lambda$ is the minus of $\mu^\lambda$-Futaki invariant, which connects $\mu^\lambda$K-semistability to the maximization of characteristic $\mu^\lambda$-entropy. It in particular provides us a criterion for $\mu^\lambda$K-semistability working without detecting the vector $\xi$ involved in the $\mu^\lambda_\xi$-Futaki invariant. In the latter part, we propose a non-archimedean pluripotential approach to the maximization problem. In order to adjust the characteristic $\mu$-entropy $\mathbf{\check{\mu}}^\lambda$ to Boucksom--Jonsson's non-archimedean framework, we introduce a natural modification $\mathbf{\check{\mu}}^\lambda_{\mathrm{NA}}$ which we call non-archimedean $\mu$-entropy. We extend the non-archimedean $\mu$-entropy from the set of test configurations to a space $\mathcal{E}^{\exp} (X, L)$ of non-archimedean psh metrics on the Berkovich space $X^{\mathrm{NA}}$, which is endowed with a complete metric structure. We introduce a measure $\int \chi \mathcal{D}_\varphi$ on Berkovich space called moment measure for this sake, which can be considered as a hybrid of Monge--Amp\`ere measure and Duistermaat--Heckman measure.


title: Closing the Gap between Single-User and Multi-User VoiceFilter-Lite

abstract: VoiceFilter-Lite is a speaker-conditioned voice separation model that plays a crucial role in improving speech recognition and speaker verification by suppressing overlapping speech from non-target speakers. However, one limitation of VoiceFilter-Lite, and other speaker-conditioned speech models in general, is that these models are usually limited to a single target speaker. This is undesirable as most smart home devices now support multiple enrolled users. In order to extend the benefits of personalization to multiple users, we previously developed an attention-based speaker selection mechanism and applied it to VoiceFilter-Lite. However, the original multi-user VoiceFilter-Lite model suffers from significant performance degradation compared with single-user models. In this paper, we devised a series of experiments to improve the multi-user VoiceFilter-Lite model. By incorporating a dual learning rate schedule and by using feature-wise linear modulation (FiLM) to condition the model with the attended speaker embedding, we successfully closed the performance gap between multi-user and single-user VoiceFilter-Lite models on single-speaker evaluations. At the same time, the new model can also be easily extended to support any number of users, and significantly outperforms our previously published model on multi-speaker evaluations.


title: A pathway to peptides in space through the condensation of atomic carbon

abstract: Organic molecules are widely present in the dense interstellar medium, and many have been synthesized in the laboratory on Earth under the conditions typical for an interstellar environment. Until now, however, only relatively small molecules of biological interest have been demonstrated to form experimentally under typical space conditions. Here we prove experimentally that the condensation of carbon atoms on the surface of cold solid particles (cosmic dust) leads to the formation of isomeric polyglycine monomers (aminoketene molecules). Following encounters between aminoketene molecules, they polymerize to produce peptides of different lengths. The chemistry involves three of the most abundant species (CO, C and NH$_3$) present in star-forming molecular clouds, and proceeds via a novel pathway that skips the stage of amino acid formation in protein synthesis. The process is efficient, even at low temperatures, without irradiation or the presence of water. The delivery of biopolymers formed by this chemistry to rocky planets in the habitable zone might be an important element in the origins of life.


title: Exact mediation analysis for ordinal outcome and binary mediator

abstract: With reference to a single mediator context, this brief report presents a model-based strategy to estimate counterfactual direct and indirect effects when the response variable is ordinal and the mediator is binary. Postulating a logistic regression model for the mediator and a cumulative logit model for the outcome, the exact parametric formulation of the causal effects is presented, thereby extending previous work that only contained approximated results. The identification conditions are equivalent to the ones already established in the literature. The effects can be estimated by making use of standard statistical software and standard errors can be computed via a bootstrap algorithm. To make the methodology accessible, routines to implement the proposal in R are presented in the Appendix. A natural effect model coherent with the postulated data generating mechanism is also derived.


title: Overcoming a Theoretical Limitation of Self-Attention

abstract: Although transformers are remarkably effective for many tasks, there are some surprisingly easy-looking regular languages that they struggle with. Hahn shows that for languages where acceptance depends on a single input symbol, a transformer's classification decisions become less and less confident (that is, with cross-entropy approaching 1 bit per string) as input strings get longer and longer. We examine this limitation using two languages: PARITY, the language of bit strings with an odd number of 1s, and FIRST, the language of bit strings starting with a 1. We demonstrate three ways of overcoming the limitation suggested by Hahn's lemma. First, we settle an open question by constructing a transformer that recognizes PARITY with perfect accuracy, and similarly for FIRST. Second, we use layer normalization to bring the cross-entropy of both models arbitrarily close to zero. Third, when transformers need to focus on a single position, as for FIRST, we find that they can fail to generalize to longer strings; we offer a simple remedy to this problem that also improves length generalization in machine translation.


title: The Impact of Selfish Behavior in Load Balancing Games

abstract: To what extent does the structure of the players' strategy space influence the efficiency of decentralized solutions in congestion games? In this work, we investigate whether better performance are possible when restricting to load balancing games in which players can only choose among single resources. We consider three different solutions concepts, namely, approximate pure Nash equilibria, approximate one-round walks generated by selfish players aiming at minimizing their personal cost and approximate one-round walks generated by cooperative players aiming at minimizing the marginal increase in the sum of the players' personal costs. The last two concepts can be interpreted as solutions of greedy online algorithms for the related resource selection problem. We show that, under fairly general latency functions on the resources, better bounds cannot be achieved if players are either weighted or asymmetric. On the positive side, we prove that, under mild assumptions on the latency functions, improvements on the performance of approximate pure Nash equilibria are possible for load balancing games with weighted and symmetric players in the case of identical resources. We also design lower bounds on the performance of one-round walks in load balancing games with unweighted players and identical resources.


title: Collaborative Training of Heterogeneous Reinforcement Learning Agents in Environments with Sparse Rewards: What and When to Share?

abstract: In the early stages of human life, babies develop their skills by exploring different scenarios motivated by their inherent satisfaction rather than by extrinsic rewards from the environment. This behavior, referred to as intrinsic motivation, has emerged as one solution to address the exploration challenge derived from reinforcement learning environments with sparse rewards. Diverse exploration approaches have been proposed to accelerate the learning process over single- and multi-agent problems with homogeneous agents. However, scarce studies have elaborated on collaborative learning frameworks between heterogeneous agents deployed into the same environment, but interacting with different instances of the latter without any prior knowledge. Beyond the heterogeneity, each agent's characteristics grant access only to a subset of the full state space, which may hide different exploration strategies and optimal solutions. In this work we combine ideas from intrinsic motivation and transfer learning. Specifically, we focus on sharing parameters in actor-critic model architectures and on combining information obtained through intrinsic motivation with the aim of having a more efficient exploration and faster learning. We test our strategies through experiments performed over a modified ViZDooM's My Way Home scenario, which is more challenging than its original version and allows evaluating the heterogeneity between agents. Our results reveal different ways in which a collaborative framework with little additional computational cost can outperform an independent learning process without knowledge sharing. Additionally, we depict the need for modulating correctly the importance between the extrinsic and intrinsic rewards to avoid undesired agent behaviors.


title: Removing Popular Faces in Curve Arrangements

abstract: A face in a curve arrangement is called popular if it is bounded by the same curve multiple times. Motivated by the automatic generation of curved nonogram puzzles, we investigate possibilities to eliminate the popular faces in an arrangement by inserting a single additional curve. This turns out to be NP-hard; however, it becomes tractable when the number of popular faces is small: We present a probabilistic FPT-approach in the number of popular faces.


title: Clarifying MCMC-based training of modern EBMs : Contrastive Divergence versus Maximum Likelihood

abstract: The Energy-Based Model (EBM) framework is a very general approach to generative modeling that tries to learn and exploit probability distributions only defined though unnormalized scores. It has risen in popularity recently thanks to the impressive results obtained in image generation by parameterizing the distribution with Convolutional Neural Networks (CNN). However, the motivation and theoretical foundations behind modern EBMs are often absent from recent papers and this sometimes results in some confusion. In particular, the theoretical justifications behind the popular MCMC-based learning algorithm Contrastive Divergence (CD) are often glossed over and we find that this leads to theoretical errors in recent influential papers (Du & Mordatch, 2019; Du et al., 2020). After offering a first-principles introduction of MCMC-based training, we argue that the learning algorithm they use can in fact not be described as CD and reinterpret theirs methods in light of a new interpretation. Finally, we discuss the implications of our new interpretation and provide some illustrative experiments.


title: Bubble Planner: Planning High-speed Smooth Quadrotor Trajectories using Receding Corridors

abstract: Quadrotors are agile platforms. With human experts, they can perform extremely high-speed flights in cluttered environments. However, fully autonomous flight at high speed remains a significant challenge. In this work, we propose a motion planning algorithm based on the corridor-constrained minimum control effort trajectory optimization (MINCO) framework. Specifically, we use a series of overlapping spheres to represent the free space of the environment and propose two novel designs that enable the algorithm to plan high-speed quadrotor trajectories in real-time. One is a sampling-based corridor generation method that generates spheres with large overlapped areas (hence overall corridor size) between two neighboring spheres. The second is a Receding Horizon Corridors (RHC) strategy, where part of the previously generated corridor is reused in each replan. Together, these two designs enlarge the corridor spaces in accordance with the quadrotor's current state and hence allow the quadrotor to maneuver at high speeds. We benchmark our algorithm against other state-of-the-art planning methods to show its superiority in simulation. Comprehensive ablation studies are also conducted to show the necessity of the two designs. The proposed method is finally evaluated on an autonomous LiDAR-navigated quadrotor UAV in woods environments, achieving flight speeds over 13.7 m/s without any prior map of the environment or external localization facility.


title: Nonunitary Superconductivity in Complex Quantum Materials

abstract: We present a comprehensive discussion on nonunitary superconductivity in complex quantum materials. Starting with a brief review of the notion of nonunitary superconductivity, we discuss its spectral signatures in simple models with only the spin as an internal degree of freedom. In complex materials with multiple internal degrees of freedom, there are many more possibilities for the development of nonunitary order parameters. We provide examples focusing on d-electron systems with two orbitals, applicable to a variety of materials. We discuss the consequences for the superconducting spectra, highlighting that gap openings of band crossings at finite energies can be attributed to a nonunitary order parameter if this is associated with a finite superconducting fitness matrix $\hat{F}_C(\mathbf{k})$. We speculate that nonunitary superconductivity in complex quantum materials is in fact very common and can be associated with multiple cases of time-reversal symmetry breaking superconductors.


title: Cybersecurity Challenges in the Offshore Oil and Gas Industry: An Industrial Cyber-Physical Systems (ICPS) Perspective

abstract: The offshore oil and gas industry has recently been going through a digitalisation drive, with use of `smart' equipment using technologies like the Industrial Internet of Things (IIoT) and Industrial Cyber-Physical Systems (ICPS). There has also been a corresponding increase in cyber attacks targeted at oil and gas companies. Oil production offshore is usually in remote locations, requiring remote access and control. This is achieved by integrating ICPS, Supervisory, Control and Data Acquisition (SCADA) systems, and IIoT technologies. A successful cyber attack against an oil and gas offshore asset could have a devastating impact on the environment, marine ecosystem and safety of personnel. Any disruption to the world's supply of oil and gas (O\&G) can also have an effect on oil prices and in turn, the global economy. This makes it important to secure the industry against cyber threats. We describe the potential cyberattack surface within the oil and gas industry, discussing emerging trends in the offshore sub-sector, and provide a timeline of known cyberattacks. We also present a case study of a subsea control system architecture typically used in offshore oil and gas operations and highlight potential vulnerabilities affecting the components of the system. This study is the first to provide a detailed analysis on the attack vectors in a subsea control system and is crucial to understanding key vulnerabilities, primarily to implement efficient mitigation methods that safeguard the safety of personnel and the environment when using such systems.


title: Quantum Deep Reinforcement Learning for Robot Navigation Tasks

abstract: We utilize hybrid quantum deep reinforcement learning to learn navigation tasks for a simple, wheeled robot in simulated environments of increasing complexity. For this, we train parameterized quantum circuits (PQCs) with two different encoding strategies in a hybrid quantum-classical setup as well as a classical neural network baseline with the double deep Q network (DDQN) reinforcement learning algorithm. Quantum deep reinforcement learning (QDRL) has previously been studied in several relatively simple benchmark environments, mainly from the OpenAI gym suite. However, scaling behavior and applicability of QDRL to more demanding tasks closer to real-world problems e. g., from the robotics domain, have not been studied previously. Here, we show that quantum circuits in hybrid quantum-classic reinforcement learning setups are capable of learning optimal policies in multiple robotic navigation scenarios with notably fewer trainable parameters compared to a classical baseline. Across a large number of experimental configurations, we find that the employed quantum circuits outperform the classical neural network baselines when equating for the number of trainable parameters. Yet, the classical neural network consistently showed better results concerning training times and stability, with at least one order of magnitude of trainable parameters more than the best-performing quantum circuits. However, validating the robustness of the learning methods in a large and dynamic environment, we find that the classical baseline produces more stable and better performing policies overall.


title: FreeSOLO: Learning to Segment Objects without Annotations

abstract: Instance segmentation is a fundamental vision task that aims to recognize and segment each object in an image. However, it requires costly annotations such as bounding boxes and segmentation masks for learning. In this work, we propose a fully unsupervised learning method that learns class-agnostic instance segmentation without any annotations. We present FreeSOLO, a self-supervised instance segmentation framework built on top of the simple instance segmentation method SOLO. Our method also presents a novel localization-aware pre-training framework, where objects can be discovered from complicated scenes in an unsupervised manner. FreeSOLO achieves 9.8% AP_{50} on the challenging COCO dataset, which even outperforms several segmentation proposal methods that use manual annotations. For the first time, we demonstrate unsupervised class-agnostic instance segmentation successfully. FreeSOLO's box localization significantly outperforms state-of-the-art unsupervised object detection/discovery methods, with about 100% relative improvements in COCO AP. FreeSOLO further demonstrates superiority as a strong pre-training method, outperforming state-of-the-art self-supervised pre-training methods by +9.8% AP when fine-tuning instance segmentation with only 5% COCO masks. Code is available at: github.com/NVlabs/FreeSOLO


title: Cavity-induced bifurcation in classical rate theory

abstract: We show how coupling an ensemble of bistable systems to a common cavity field affects the collective stochastic behavior of this ensemble. In particular, the cavity provides an effective interaction between the systems, and parametrically modifies the transition rates between the metastable states. We predict that the cavity induces a collective phase transition at a critical temperature which depends linearly on the number of systems. It shows up as a spontaneous symmetry breaking where the stationary states of the bistable system bifurcate. We observe that the transition rates slow down independently of the phase transition, but the rate modification vanishes for alternating signs of the system-cavity couplings, corresponding to a disordered ensemble of dipoles. Our results are of particular relevance in polaritonic chemistry where the presence of a cavity has been suggested to affect chemical reactions.


title: Large-scale Stochastic Optimization of NDCG Surrogates for Deep Learning with Provable Convergence

abstract: NDCG, namely Normalized Discounted Cumulative Gain, is a widely used ranking metric in information retrieval and machine learning. However, efficient and provable stochastic methods for maximizing NDCG are still lacking, especially for deep models. In this paper, we propose a principled approach to optimize NDCG and its top-$K$ variant. First, we formulate a novel compositional optimization problem for optimizing the NDCG surrogate, and a novel bilevel compositional optimization problem for optimizing the top-$K$ NDCG surrogate. Then, we develop efficient stochastic algorithms with provable convergence guarantees for the non-convex objectives. Different from existing NDCG optimization methods, the per-iteration complexity of our algorithms scales with the mini-batch size instead of the number of total items. To improve the effectiveness for deep learning, we further propose practical strategies by using initial warm-up and stop gradient operator. Experimental results on multiple datasets demonstrate that our methods outperform prior ranking approaches in terms of NDCG. To the best of our knowledge, this is the first time that stochastic algorithms are proposed to optimize NDCG with a provable convergence guarantee. Our proposed methods are implemented in the LibAUC library at https://libauc.org/.


title: Consistent data fusion with Parker

abstract: When combining data from multiple sources, inconsistent data complicates the production of a coherent result. In this paper, we introduce a new type of constraints called edit rules under a partial key (EPKs). These constraints can model inconsistencies both within and between sources, but in a loosely-coupled matter. We show that we can adapt the well-known set cover methodology to the setting of EPKs and this yields an efficient algorithm to find minimal cost repairs of sources. This algorithm is implemented in a repair engine called Parker. Empirical results show that Parker is several orders of magnitude faster than state-of-the-art repair tools. At the same time, the quality of the repairs in terms of $F_1$-score ranges from comparable to better compared to these tools.


title: Competition among alliances of different sizes

abstract: To understand the biodiversity of an ecosystem cannot be understood by solely analyzing the pair relations of competing species. Instead, we should consider multi-point interactions because the presence of a third party could change the original microscopic outcome significantly. In this way an alliance may emerge where species, who may have biased relations otherwise, can protect each other from an external invader. Such an alliance can be formed by two, three or even more species. By introducing a minimal model where six species compete for space we here study how the size of an alliance determines the vitality of a formation. We show that in the majority of parameter space the group of the smallest size prevails and other solutions can only be observed in a limited parameter range. These phases are separated by discontinuous phase transitions which can only be identified by intensive numerical efforts due to serious finite size effects and long relaxation processes.

categories: q-bio.PE cond-mat.stat-mech physics.bio-ph physics.soc-ph


title: Sequential asset ranking in nonstationary time series

abstract: We create a ranking algorithm, the naive Bayes asset ranker. Our algorithm computes the posterior probability that individual assets will be ranked higher than other portfolio constituents. Unlike earlier algorithms, such as the weighted majority, our algorithm allows poor-performing experts to have increased weight when they start performing well. We outperform the long-only holding of the S&P 500 index and a regress-then-rank baseline.


title: SonOpt: Sonifying Bi-objective Population-Based Optimization Algorithms

abstract: We propose SonOpt, the first (open source) data sonification application for monitoring the progress of bi-objective population-based optimization algorithms during search, to facilitate algorithm understanding. SonOpt provides insights into convergence/stagnation of search, the evolution of the approximation set shape, location of recurring points in the approximation set, and population diversity. The benefits of data sonification have been shown for various non-optimization related monitoring tasks. However, very few attempts have been made in the context of optimization and their focus has been exclusively on single-objective problems. In comparison, SonOpt is designed for bi-objective optimization problems, relies on objective function values of non-dominated solutions only, and is designed with the user (listener) in mind; avoiding convolution of multiple sounds and prioritising ease of familiarizing with the system. This is achieved using two sonification paths relying on the concepts of wavetable and additive synthesis. This paper motivates and describes the architecture of SonOpt, and then validates SonOpt for two popular multi-objective optimization algorithms (NSGA-II and MOEA/D). Experience SonOpt yourself via https://github.com/tasos-a/SonOpt-1.0 .


title: Euler integral as a source of chaos in the three-body problem

abstract: In this paper we address, from a purely numerical point of view, the question, raised in [20, 21], and partly considered in [22, 9, 3], whether a certain function, referred to as "Euler Integral", is a quasi-integral along the trajectories of the three-body problem. Differently from our previous investigations, here we focus on the region of the "unperturbed separatrix", which turns to be complicated by a collision singularity. Concretely, we reduce the Hamiltonian to two degrees of freedom and, after fixing some energy level, we discuss in detail the resulting three-dimensional phase space around an elliptic and an hyperbolic periodic orbit. After measuring the strength of variation of the Euler Integral (which are in fact small), we detect the existence of chaos closely to the unperturbed separatrix. The latter result is obtained through a careful use of the machinery of covering relations, developed in [13, 24, 23].


title: Finite-size effects in the diffusion dynamics of a glass-forming binary mixture with large size ratio

abstract: Extensive molecular dynamics (MD) computer simulations of an equimolar glass-forming AB mixture with large size ratio are presented. While the large A particles show a glass transition around the critical density of mode coupling theory $\rho_c$, the small B particles remain mobile with a relatively weak decrease of their self diffusion coefficient $D_{\rm B}$ with increasing density. Surprisingly, around $\rho_c$, the self-diffusion coefficient of the A particles, $D_{\rm A}$, also starts to follow a rather weak dependence on density. We show that this is due to finite-size effects that can be understood from the analysis of the collective interdiffusion dynamics.


title: Improved Hausdorff dimension estimate of the singular set of the supercritical surface quasigeostrophic equation

abstract: We prove that the spacetime singular set of any suitable Leray-Hopf solution of the surface quasigeostrophic equation with fractional dissipation of order $0< \alpha < \frac{1}{2}$ has Hausdorff dimension at most $\frac{1}{2\alpha^2}\,.$ This result improves previously known dimension estimate established in [6] and builds on the excess decay result and the control on the particle flow already developed there. The improvement lies in the initial iteration of the local energy inequality in analogy with the celebrated result of Caffarelli-Kohn-Nirenberg [2] for the Navier-Stokes equations.


title: Finding Inverse Document Frequency Information in BERT

abstract: For many decades, BM25 and its variants have been the dominant document retrieval approach, where their two underlying features are Term Frequency (TF) and Inverse Document Frequency (IDF). The traditional approach, however, is being rapidly replaced by Neural Ranking Models (NRMs) that can exploit semantic features. In this work, we consider BERT-based NRMs and study if IDF information is present in the NRMs. This simple question is interesting because IDF has been indispensable for the traditional lexical matching, but global features like IDF are not explicitly learned by neural language models including BERT. We adopt linear probing as the main analysis tool because typical BERT based NRMs utilize linear or inner-product based score aggregators. We analyze input embeddings, representations of all BERT layers, and the self-attention weights of CLS. By studying MS-MARCO dataset with three BERT-based models, we show that all of them contain information that is strongly dependent on IDF.


title: A decreasing upper bound of energy for time-fractional phase-field equations

abstract: In this article, we study the energy dissipation property of time-fractional Allen-Cahn equation. We propose a decreasing upper bound of energy that decreases with respect to time and coincides with the original energy at $t = 0$ and as $t$ tends to $\infty$. This upper bound can also be viewed as a nonlocal-in-time modified energy, the summation of the original energy and an accumulation term due to the memory effect of time fractional derivative. In particular, this indicates that the original energy indeed decays w.r.t. time in a small neighborhood at $t=0$. We illustrate the theory mainly with the time-fractional Allen-Cahn equation, but it could be applied to other time-fractional phase-field models such as the Cahn-Hilliard equation. On the discrete level, the first-order L1 and second-order L2 schemes for time-fractional Allen-Cahn equation have similar decreasing modified energies, so that the stability can be established. Some numerical results are provided to illustrate the behavior of this modified energy and to verify our theoretical results.


title: Decomposition-Based Synthesis for Applying Divide-and-Conquer-Like Algorithmic Paradigms

abstract: Algorithmic paradigms such as divide-and-conquer (D&C) are proposed to guide developers in designing efficient algorithms, but it can still be difficult to apply algorithmic paradigms to practical tasks. To ease the usage of paradigms, many research efforts have been devoted to the automatic application of algorithmic paradigms. However, most existing approaches to this problem rely on syntax-based program transformations and thus put significant restrictions on the original program. In this paper, we study the automatic application of D&C and several similar paradigms, denoted as D&C-like algorithmic paradigms, and aim to remove the restrictions from syntax-based transformations. To achieve this goal, we propose an efficient synthesizer, named AutoLifter, which does not depend on syntax-based transformations. Specifically, the main challenge of applying algorithmic paradigms is from the large scale of the synthesized programs, and AutoLifter addresses this challenge by applying two novel decomposition methods that do not depend on the syntax of the input program, component elimination and variable elimination, to soundly divide the whole problem into simpler subtasks, each synthesizing a sub-program of the final program and being tractable with existing synthesizers. We evaluate AutoLifter on 96 programming tasks related to 6 different algorithmic paradigms. AutoLifter solves 82/96 tasks with an average time cost of 20.17 seconds, significantly outperforming existing approaches.