title: Overstable rotating convection in presence of vertical magnetic field

abstract: We present the results of our investigation on nonlinear overstable rotating magnetoconvection (RMC) in presence of vertical external magnetic field. We focus on the dynamics appearing near the onset of convection by varying the system control parameters, namely, the Taylor number ($\mathrm{Ta}$), the Chandrasekhar number ($\mathrm{Q}$) and the Prandtl number ($\mathrm{Pr}$) in the ranges $750\leq\mathrm{Ta}\leq10^6$, $0 < \mathrm{Q} \leq 10^3$ and $0 < \mathrm{Pr} \leq 0.5$. Three dimensional (3D) direct numerical simulations (DNS) of the governing equations and low-dimensional modeling of the system are performed for this purpose. Extensive DNS in the specified parameter space shows two qualitatively different onsets depending on $\mathrm{Ta}$, $\mathrm{Q}$ and $\mathrm{Pr}$. In the first one, bistability appears at the onset, where both subcritical and supercritical convection coexist, while only supercritical convection is observed in the second one. Analysis of the low-dimensional model reveals that a supercritical Hopf bifurcation is responsible for the supercritical onset and a subcritical pitchfork bifurcation is responsible for the subcritical onset. It is also observed that appearance of subcritical convection at the onset has strong dependence on all three control parameters $\mathrm{Ta}$, $\mathrm{Q}$ and $\mathrm{Pr}$. The scenario of subcritical convection is found to disappear as $\mathrm{Pr}$ is increased for fixed $\mathrm{Ta}$ and $\mathrm{Q}$. However, most striking findings of the investigation is that the increment in $\mathrm{Ta}$ for fixed $\mathrm{Q}$ and $\mathrm{Pr}$ opposes the subcritical convection, whereas the increment in $\mathrm{Q}$ for fixed $\mathrm{Ta}$ and $\mathrm{Pr}$ favors it. This is in sharp contrast with the earlier results reported in RMC.


title: Preprocessing of vector magnetograms for magnetohydrostatic extrapolations

abstract: Context. Understanding the 3D magnetic field as well as the plasma in the chromosphere and transition region is important. One way is to extrapolate the magnetic field and plasma from the routinely measured vector magnetogram on the photosphere based on the assumption of the magnetohydrostatic (MHS) state. However, photospheric data may be inconsistent with the MHS assumption. Therefore, we must study the restriction on the photospheric magnetic field, which is required by the MHS system. Moreover, the data should be transformed accordingly before MHS extrapolations can be applied. Aims. We aim to obtain a set of surface integrals as criteria for the MHS system and use this set of integrals to preprocess a vector magnetogram. Methods. By applying Gauss' theorem and assuming an isolated active region on the Sun, we related the magnetic energy and forces in the volume to the surface integral on the photosphere. The same method was applied to obtain restrictions on the photospheric magnetic field as necessary criteria for a MHS system. We used an optimization method to preprocess the data to minimize the deviation from the criteria as well as the measured value. Results. By applying the virial theorem to the active region, we find the boundary integral that is used to compute the energy of a force-free field usually underestimates the magnetic energy of a large active region. We also find that the MHS assumption only requires the x-, y-component of net Lorentz force and the z-component of net torque to be zero. These zero components are part of Aly's criteria for a force-free field. However, other components of net force and torque can be non-zero values. According to new criteria, we preprocess the magnetogram to make it more consistent with the MHS system and, at the same time close, to the original data.


title: Neural Gaussian Mirror for Controlled Feature Selection in Neural Networks

abstract: Deep neural networks (DNNs) have become increasingly popular and achieved outstanding performance in predictive tasks. However, the DNN framework itself cannot inform the user which features are more or less relevant for making the prediction, which limits its applicability in many scientific fields. We introduce neural Gaussian mirrors (NGMs), in which mirrored features are created, via a structured perturbation based on a kernel-based conditional dependence measure, to help evaluate feature importance. We design two modifications of the DNN architecture for incorporating mirrored features and providing mirror statistics to measure feature importance. As shown in simulated and real data examples, the proposed method controls the feature selection error rate at a predefined level and maintains a high selection power even with the presence of highly correlated features.


title: ISTA-NAS: Efficient and Consistent Neural Architecture Search by Sparse Coding

abstract: Neural architecture search (NAS) aims to produce the optimal sparse solution from a high-dimensional space spanned by all candidate connections. Current gradient-based NAS methods commonly ignore the constraint of sparsity in the search phase, but project the optimized solution onto a sparse one by post-processing. As a result, the dense super-net for search is inefficient to train and has a gap with the projected architecture for evaluation. In this paper, we formulate neural architecture search as a sparse coding problem. We perform the differentiable search on a compressed lower-dimensional space that has the same validation loss as the original sparse solution space, and recover an architecture by solving the sparse coding problem. The differentiable search and architecture recovery are optimized in an alternate manner. By doing so, our network for search at each update satisfies the sparsity constraint and is efficient to train. In order to also eliminate the depth and width gap between the network in search and the target-net in evaluation, we further propose a method to search and evaluate in one stage under the target-net settings. When training finishes, architecture variables are absorbed into network weights. Thus we get the searched architecture and optimized parameters in a single run. In experiments, our two-stage method on CIFAR-10 requires only 0.05 GPU-day for search. Our one-stage method produces state-of-the-art performances on both CIFAR-10 and ImageNet at the cost of only evaluation time.


title: COVID-19 Imaging Data Privacy by Federated Learning Design: A Theoretical Framework

abstract: To address COVID-19 healthcare challenges, we need frequent sharing of health data, knowledge and resources at a global scale. However, in this digital age, data privacy is a big concern that requires the secure embedding of privacy assurance into the design of all technological solutions that use health data. In this paper, we introduce differential privacy by design (dPbD) framework and discuss its embedding into the federated machine learning system. To limit the scope of our paper, we focus on the problem scenario of COVID-19 imaging data privacy for disease diagnosis by computer vision and deep learning approaches. We discuss the evaluation of the proposed design of federated machine learning systems and discuss how differential privacy by design (dPbD) framework can enhance data privacy in federated learning systems with scalability and robustness. We argue that scalable differentially private federated learning design is a promising solution for building a secure, private and collaborative machine learning model such as required to combat COVID19 challenge.


title: Differential radial basis function network for sequence modelling

abstract: We propose a differential radial basis function (RBF) network termed RBF-DiffNet -- whose hidden layer blocks are partial differential equations (PDEs) linear in terms of the RBF -- to make the baseline RBF network robust to noise in sequential data. Assuming that the sequential data derives from the discretisation of the solution to an underlying PDE, the differential RBF network learns constant linear coefficients of the PDE, consequently regularising the RBF network by following modified backward-Euler updates. We experimentally validate the differential RBF network on the logistic map chaotic timeseries as well as on 30 real-world timeseries provided by Walmart in the M5 forecasting competition. The proposed model is compared with the normalised and unnormalised RBF networks, ARIMA, and ensembles of multilayer perceptrons (MLPs) and recurrent networks with long short-term memory (LSTM) blocks. From the experimental results, RBF-DiffNet consistently shows a marked reduction over the baseline RBF network in terms of the prediction error (e.g., 26% reduction in the root mean squared scaled error on the M5 dataset); RBF-DiffNet also shows a comparable performance to the LSTM ensemble at less than one-sixteenth the LSTM computational time. Our proposed network consequently enables more accurate predictions -- in the presence of observational noise -- in sequence modelling tasks such as timeseries forecasting that leverage the model interpretability, fast training, and function approximation properties of the RBF network.


title: Towards a self-consistent analysis of the anisotropic galaxy two- and three-point correlation functions on large scales: application to mock galaxy catalogues

abstract: We establish a practical method for the joint analysis of anisotropic galaxy two- and three-point correlation functions (2PCF and 3PCF) on the basis of the decomposition formalism of the 3PCF using tri-polar spherical harmonics. We perform such an analysis with MultiDark Patchy mock catalogues to demonstrate and understand the benefit of the anisotropic 3PCF. We focus on scales above $80 h^{-1}\,{\rm Mpc}$, and use information from the shape and the baryon acoustic oscillation (BAO) signals of the 2PCF and 3PCF. We also apply density field reconstruction to increase the signal-noise ratio of BAO in the 2PCF measurement, but not in the 3PCF measurement. In particular, we study in detail the constraints on the angular diameter distance and the Hubble parameter. We build a model of the bispectrum or 3PCF that includes the nonlinear damping of the BAO signal in redshift space. We carefully account for various uncertainties in our analysis including theoretical models of the 3PCF, window function corrections, biases in estimated parameters from the fiducial values, the number of mock realizations to estimate the covariance matrix, and bin size. The joint analysis of the 2PCF and 3PCF monopole and quadrupole components shows a $30\%$ a nd $20\%$ improvement in Hubble parameter constraints before and after reconstruction of the 2PCF measurements, respectively, compared to the 2PCF analysis alone. This study clearly shows that the anisotropic 3PCF increases cosmological information from galaxy surveys and encourages further development of the modeling of the 3PCF on smaller scales than we consider.


title: Selective Pumping of Localized States in a Disordered Active Medium

abstract: Light scattering and localization in strongly scattering disordered systems is governed by the nature of the underlying eigenmodes, specially their spatial extension within the system. One of the main challenges in studying experimentally Anderson like localized states resides in the difficulty to excite these states independently and observe them individually anywhere in the sample. The modes mutual interaction and their coupling to the sample boundaries makes it extremely challenging to isolate them spectrally and image them alone. In the presence of gain, random lasing occurs above threshold and lasing modes are excited. It was shown recently that shaping the intensity profile of the optical pump is a very effective way to control and force random lasing in singlemode regime at any desired emission wavelength. Active random media therefore offer a unique platform to select modes individually. By using pump shaping technique in a strongly scattering random laser, we successfully select localized lasing modes individually. Direct imaging of the light distribution within the random laser confirms the confined nature of the modes and enables us to elucidate long standing questions on the role of non-Hermiticity in active scattering media and the influence of nonlinearities on Anderson wave localization.


title: An odd analog of Plamenevskaya's invariant of transverse knots

abstract: Plamenevskaya defined an invariant of transverse links as a distinguished class in the even Khovanov homology of a link. We define an analog of Plamenevskaya's invariant in the odd Khovanov homology of Ozsv\'ath, Rasmussen, and Szab\'o. We show that the analog is also an invariant of transverse links and has similar properties to Plamenevskaya's invariant. We also show that the analog invariant can be identified with an equivalent invariant in the reduced odd Khovanov homology. We demonstrate computations of the invariant on various transverse knot pairs with the same topological knot type and self-linking number.


title: Magnetic response of Majorana Kramers pairs with an order-two symmetry

abstract: We study an intrinsic relation between the topology of bulk electronic states and magnetic responses of Majorana Kramers pairs, Kramers pairs of Majorana fermions, on a surface of time-reversal-invariant topological superconductors. Majorana Kramers pairs respond to an applied magnetic field anisotropically due to the interplay between time-reversal and crystal symmetries. In this paper, we propose a systematic procedure to determine such surface magnetic responses in systems with an order-two symmetry. From the analysis of topological invariants associated with an order-two symmetry, it is found that magnetic responses are classified into four types, which are attributed to different topological invariants and exhibit distinguishable, characteristic magnetic responses. For a Kramers pair of Majorana fermions protected by $\mathbb{Z}_2$ topological invariants, we clarify that types of magnetic responses are determined only from Fermi--surface topology and symmetry of Cooper pairs. Finally, we apply our theory to the topological nonsymmorphic crystalline superconducting state in UCoGe, which exhibits a biaxially anisotropic magnetic response.


title: Axion Dark Matter eXperiment: Run 1B Analysis Details

abstract: Searching for axion dark matter, the ADMX collaboration acquired data from January to October 2018, over the mass range 2.81--3.31 $\mu$eV, corresponding to the frequency range 680--790 MHz. Using an axion haloscope consisting of a microwave cavity in a strong magnetic field, the ADMX experiment excluded Dine-Fischler-Srednicki-Zhitnisky (DFSZ) axions at 100% dark matter density over this entire frequency range, except for a few gaps due to mode crossings. This paper explains the full ADMX analysis for Run 1B, motivating analysis choices informed by details specific to this run.


title: Low scale leptogenesis in a model with promising $CP$ structure

abstract: We consider a simple extension of the standard model, which could give a solution to its $CP$ issues through both the Peccei-Quinn mechanism and the Nelson-Barr mechanism. Its low energy effective model coincides with the scotogenic model in the leptonic sector. Although leptogenesis is known not to work well at lower reheating temperature than $10^9$ GeV in simple seesaw and scotogenic frameworks, such low reheating temperature could be consistent with both neutrino mass generation and thermal leptogenesis via newly introduced fields without referring to the resonance effect. An alternative dark matter candidate to axion is prepared as an indispensable ingredient of the model.


title: The workweek is the best time to start a family -- A Study of GPT-2 Based Claim Generation

abstract: Argument generation is a challenging task whose research is timely considering its potential impact on social media and the dissemination of information. Here we suggest a pipeline based on GPT-2 for generating coherent claims, and explore the types of claims that it produces, and their veracity, using an array of manual and automatic assessments. In addition, we explore the interplay between this task and the task of Claim Retrieval, showing how they can complement one another.


title: Correlation-wise Smoothing: Lightweight Knowledge Extraction for HPC Monitoring Data

abstract: Modern High-Performance Computing (HPC) and data center operators rely more and more on data analytics techniques to improve the efficiency and reliability of their operations. They employ models that ingest time-series monitoring sensor data and transform it into actionable knowledge for system tuning: a process known as Operational Data Analytics (ODA). However, monitoring data has a high dimensionality, is hardware-dependent and difficult to interpret. This, coupled with the strict requirements of ODA, makes most traditional data mining methods impractical and in turn renders this type of data cumbersome to process. Most current ODA solutions use ad-hoc processing methods that are not generic, are sensible to the sensors' features and are not fit for visualization. In this paper we propose a novel method, called Correlation-wise Smoothing (CS), to extract descriptive signatures from time-series monitoring data in a generic and lightweight way. Our CS method exploits correlations between data dimensions to form groups and produces image-like signatures that can be easily manipulated, visualized and compared. We evaluate the CS method on HPC-ODA, a collection of datasets that we release with this work, and show that it leads to the same performance as most state-of-the-art methods while producing signatures that are up to ten times smaller and up to ten times faster, while gaining visualizability, portability across systems and clear scaling properties.


title: Deep Reinforcement Learning and Transportation Research: A Comprehensive Review

abstract: Deep reinforcement learning (DRL) is an emerging methodology that is transforming the way many complicated transportation decision-making problems are tackled. Researchers have been increasingly turning to this powerful learning-based methodology to solve challenging problems across transportation fields. While many promising applications have been reported in the literature, there remains a lack of comprehensive synthesis of the many DRL algorithms and their uses and adaptations. The objective of this paper is to fill this gap by conducting a comprehensive, synthesized review of DRL applications in transportation. We start by offering an overview of the DRL mathematical background, popular and promising DRL algorithms, and some highly effective DRL extensions. Building on this overview, a systematic investigation of about 150 DRL studies that have appeared in the transportation literature, divided into seven different categories, is performed. Building on this review, we continue to examine the applicability, strengths, shortcomings, and common and application-specific issues of DRL techniques with regard to their applications in transportation. In the end, we recommend directions for future research and present available resources for actually implementing DRL.


title: When Wireless Communications Meet Computer Vision in Beyond 5G

abstract: This article articulates the emerging paradigm, sitting at the confluence of computer vision and wireless communication, to enable beyond-5G/6G mission-critical applications (autonomous/remote-controlled vehicles, visuo-haptic VR, and other cyber-physical applications). First, drawing on recent advances in machine learning and the availability of non-RF data, vision-aided wireless networks are shown to significantly enhance the reliability of wireless communication without sacrificing spectral efficiency. In particular, we demonstrate how computer vision enables {look-ahead} prediction in a millimeter-wave channel blockage scenario, before the blockage actually happens. From a computer vision perspective, we highlight how radio frequency (RF) based sensing and imaging are instrumental in robustifying computer vision applications against occlusion and failure. This is corroborated via an RF-based image reconstruction use case, showcasing a receiver-side image failure correction resulting in reduced retransmission and latency. Taken together, this article sheds light on the much-needed convergence of RF and non-RF modalities to enable ultra-reliable communication and truly intelligent 6G networks.


title: X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models

abstract: Language models (LMs) have proven surprisingly successful at capturing factual knowledge by completing cloze-style fill-in-the-blank questions such as "Punta Cana is located in _." However, while knowledge is both written and queried in many languages, studies on LMs' factual representation ability have almost invariably been performed on English. To assess factual knowledge retrieval in LMs in different languages, we create a multilingual benchmark of cloze-style probes for 23 typologically diverse languages. To properly handle language variations, we expand probing methods from single- to multi-word entities, and develop several decoding algorithms to generate multi-token predictions. Extensive experimental results provide insights about how well (or poorly) current state-of-the-art LMs perform at this task in languages with more or fewer available resources. We further propose a code-switching-based method to improve the ability of multilingual LMs to access knowledge, and verify its effectiveness on several benchmark languages. Benchmark data and code have been released at https://x-factr.github.io.


title: Path-Dependent Hamilton--Jacobi Equations: The Minimax Solutions Revised

abstract: Motivated by optimal control problems and differential games for functional differential equations of retarded type, the paper deals with a Cauchy problem for a path-dependent Hamilton--Jacobi equation with a right-end boundary condition. Minimax solutions of this problem are studied. The existence and uniqueness result is obtained under assumptions that are weaker than those considered earlier. In contrast to previous works, on the one hand, we do not require any properties concerning positive homogeneity of the Hamiltonian in the impulse variable, and on the other hand, we suppose that the Hamiltonian satisfies a Lipshitz continuity condition with respect to the path variable in the uniform (supremum) norm. The progress is related to the fact that a suitable Lyapunov--Krasovskii functional is built that allows to prove a comparison principle. This functional is in some sense equivalent to the square of the uniform norm of the path variable and, at the same time, it possesses appropriate smoothness properties. In addition, the paper provides non-local and infinitesimal criteria of minimax solutions, their stability with respect to perturbations of the Hamiltonian and the boundary functional, as well as consistency of the approach with the non-path-dependent case. Connection of the problem statement under consideration with some other possible statements (regarding the choice of path spaces and derivatives used) known in the theory of path-dependent Hamilton--Jacobi equations is discussed. Some remarks concerning viscosity solutions of the studied Cauchy problem are given.


title: Large band splitting with tunable spin polarization in two-dimensional ferroelectric GaXY (X= Se, Te; Y= Cl, Br, I) family

abstract: It has been generally accepted that the spin-orbit coupling effect in noncentrosymmetric materials leads to the band splitting and non-trivial spin polarization in the momentum space. However, in some cases, zero net spin polarization in the split bands may occurs, dubbed as the band splitting with vanishing spin polarization (BSVSP) effect, protected by non-pseudo-polar point group symmetry of the wave vector in the first Brillouin zone [Liu et. al., Nat. Commun. \textbf{10}, 5144 (2019)]. In this paper, by using first-principles calculations, we show that the BSVSP effect emerges in two-dimensional (2D) nonsymmorphic Ga$XY$ ($X$= Se, Te; $Y$= Cl, Br, I) family, a new class of 2D materials having in-plane ferroelectricity. Taking the GaTeCl monolayer as a representative example, we observe the BSVSP effect in the split bands along the $X-M$ line located in the proximity of the conduction band minimum. By using $\vec{k}\cdot\vec{p}$ Hamiltonian derived based on the symmetry analysis, we clarify that such effect is originated from the cancellation of the local spin polarization, enforced by non-pseudo-polar $C_{2v}$ point group symmetry of the wave vector along the $X-M$ line. Importantly, we find that the spin polarization can be effectively induced by applying an external out-of-plane electric field, indicating that an electrically tunable spin polarization for spintronic applications is plausible.


title: Revisiting BFloat16 Training

abstract: State-of-the-art generic low-precision training algorithms use a mix of 16-bit and 32-bit precision, creating the folklore that 16-bit hardware compute units alone are not enough to maximize model accuracy. As a result, deep learning accelerators are forced to support both 16-bit and 32-bit floating-point units (FPUs), which is more costly than only using 16-bit FPUs for hardware design. We ask: can we train deep learning models only with 16-bit floating-point units, while still matching the model accuracy attained by 32-bit training? Towards this end, we study 16-bit-FPU training on the widely adopted BFloat16 unit. While these units conventionally use nearest rounding to cast output to 16-bit precision, we show that nearest rounding for model weight updates often cancels small updates, which degrades the convergence and model accuracy. Motivated by this, we study two simple techniques well-established in numerical analysis, stochastic rounding and Kahan summation, to remedy the model accuracy degradation in 16-bit-FPU training. We demonstrate that these two techniques can enable up to 7% absolute validation accuracy gain in 16-bit-FPU training. This leads to 0.1% lower to 0.2% higher validation accuracy compared to 32-bit training across seven deep learning applications.


title: Induced gravitational waves as a cosmological probe of the sound speed during the QCD phase transition

abstract: The standard model of particle physics is known to be intriguingly successful. However their rich phenomena represented by the phase transitions (PTs) have not been completely understood yet, including the possibility of the existence of unknown dark sectors. In this Letter, we investigate the measurement of the equation of state parameter $w$ and the sound speed $c_{\rm s}$ of the PT plasma with use of the gravitational waves (GWs) of the universe. Though the propagation of GW is insensitive to $c_{\rm s}$ in itself, the sound speed value affects the dynamics of primordial density (or scalar curvature) perturbations and the induced GW by their horizon reentry can then be an indirect probe both $w$ and $c_{\rm s}$. We numerically reveal the concrete spectrum of the predicted induced GW with two simple examples of the scalar perturbation spectrum: the monochromatic and scale-invariant spectra. In the monochromatic case, we see that the resonant amplification and cancellation scales of the induced GW depend on the $c_{\rm s}$ values at different time respectively. The scale-invariant case gives a more realistic spectrum and its specific shape will be compared with observations. In particular, the QCD phase transition corresponds with the frequency range of the pulsar timing array (PTA) observations. If the amplitude of primordial scalar power is in the range of $10^{-4}\lesssim A_\zeta\lesssim10^{-2}$, the induced GW is consistent with current observational constraints and detectable in the future observation in Square Kilometer Array. Futhermore the recent possible detection of stochastic GWs by NANOGrav 12.5 yr analysis~[1] can be explained by the induced GW if $A_\zeta\sim\sqrt{7}\times10^{-3}$.


title: Labeling the Phrases of a Conversational Agent with a Unique Personalized Vocabulary

abstract: Mapping spoken text to gestures is an important research topic for robots with conversation capabilities. According to studies on human co-speech gestures, a reasonable solution for mapping is using a concept-based approach in which a text is first mapped to a semantic cluster (i.e., a concept) containing texts with similar meanings. Subsequently, each concept is mapped to a predefined gesture. By using a concept-based approach, this paper discusses the practical issue of obtaining concepts for a unique vocabulary personalized for a conversational agent. Using Microsoft Rinna as an agent, we qualitatively compare concepts obtained automatically through a natural language processing (NLP) approach to those obtained manually through a sociological approach. We then identify three limitations of the NLP approach: at the semantic level with emojis and symbols; at the semantic level with slang, new words, and buzzwords; and at the pragmatic level. We attribute these limitations to the personalized vocabulary of Rinna. A follow-up experiment demonstrates that robot gestures selected using a concept-based approach leave a better impression than randomly selected gestures for the Rinna vocabulary, suggesting the usefulness of a concept-based gesture generation system for personalized vocabularies. This study provides insights into the development of gesture generation systems for conversational agents with personalized vocabularies.


title: Learning to Cache: Distributed Coded Caching in a Cellular Network With Correlated Demands

abstract: Design of distributed caching mechanisms is considered as an active area of research due to its promising solution in reducing data load in the backhaul link of a cellular network. In this paper, the problem of distributed content caching in a small-cell Base Stations (sBSs) wireless network that maximizes the cache hit performance is considered. Most of the existing works focus on static demands, however, here, data at each sBS is considered to be correlated across time and sBSs. The caching strategy is assumed to be a weighted combination of past caching strategies. A high probability generalization guarantees on the performance of the proposed caching strategy is derived. The theoretical guarantee provides following insights on obtaining the caching strategy: (i) run regret minimization at each sBS to obtain a sequence of caching strategies across time, and (ii) maximize an estimate of the bound to obtain a set of weights for the caching strategy which depends on the discrepancy. Also, theoretical guarantee on the performance of the LRFU caching strategy is derived. Further, federated learning based heuristic caching algorithm is also proposed. Finally, it is shown through simulations using Movie Lens dataset that the proposed algorithm significantly outperforms LRFU algorithm.


title: Mathematical Word Problem Generation from Commonsense Knowledge Graph and Equations

abstract: There is an increasing interest in the use of mathematical word problem (MWP) generation in educational assessment. Different from standard natural question generation, MWP generation needs to maintain the underlying mathematical operations between quantities and variables, while at the same time ensuring the relevance between the output and the given topic. To address above problem, we develop an end-to-end neural model to generate diverse MWPs in real-world scenarios from commonsense knowledge graph and equations. The proposed model (1) learns both representations from edge-enhanced Levi graphs of symbolic equations and commonsense knowledge; (2) automatically fuses equation and commonsense knowledge information via a self-planning module when generating the MWPs. Experiments on an educational gold-standard set and a large-scale generated MWP set show that our approach is superior on the MWP generation task, and it outperforms the SOTA models in terms of both automatic evaluation metrics, i.e., BLEU-4, ROUGE-L, Self-BLEU, and human evaluation metrics, i.e., equation relevance, topic relevance, and language coherence. To encourage reproducible results, we make our code and MWP dataset public available at \url{https://github.com/tal-ai/MaKE_EMNLP2021}.


title: Context-Aware Drive-thru Recommendation Service at Fast Food Restaurants

abstract: Drive-thru is a popular sales channel in the fast food industry where consumers can make food purchases without leaving their cars. Drive-thru recommendation systems allow restaurants to display food recommendations on the digital menu board as guests are making their orders. Popular recommendation models in eCommerce scenarios rely on user attributes (such as user profiles or purchase history) to generate recommendations, while such information is hard to obtain in the drive-thru use case. Thus, in this paper, we propose a new recommendation model Transformer Cross Transformer (TxT), which exploits the guest order behavior and contextual features (such as location, time, and weather) using Transformer encoders for drive-thru recommendations. Empirical results show that our TxT model achieves superior results in Burger King's drive-thru production environment compared with existing recommendation solutions. In addition, we implement a unified system to run end-to-end big data analytics and deep learning workloads on the same cluster. We find that in practice, maintaining a single big data cluster for the entire pipeline is more efficient and cost-saving. Our recommendation system is not only beneficial for drive-thru scenarios, and it can also be generalized to other customer interaction channels.


title: Visual Security Evaluation of Learnable Image Encryption Methods against Ciphertext-only Attacks

abstract: Various visual information protection methods have been proposed for privacy-preserving deep neural networks (DNNs). In contrast, attack methods on such protection methods have been studied simultaneously. In this paper, we evaluate state-of-the-art visual protection methods for privacy-preserving DNNs in terms of visual security against ciphertext-only attacks (COAs). We focus on brute-force attack, feature reconstruction attack (FR-Attack), inverse transformation attack (ITN-Attack), and GAN-based attack (GAN-Attack), which have been proposed to reconstruct visual information on plain images from the visually-protected images. The detail of various attack is first summarized, and then visual security of the protection methods is evaluated. Experimental results demonstrate that most of protection methods, including pixel-wise encryption, have not enough robustness against GAN-Attack, while a few protection methods are robust enough against GAN-Attack.


title: Toeplitz Momentary Symbols: definition, results, and limitations in the spectral analysis of Structured Matrices

abstract: A powerful tool for analyzing and approximating the singular values and eigenvalues of structured matrices is the theory of GLT sequences. By the GLT theory one can derive a function, which describes the singular value or the eigenvalue distribution of the sequence, the latter under precise assumptions. However, for small values of the matrix size of the considered sequence, the approximations may not be as good as it is desirable, since in the construction of the GLT symbol one disregards small norm and low-rank perturbations. On the other hand, LFA can be used to construct polynomial symbols in a similar manner for discretizations, where the geometric information is present, but the small norm perturbations are retained. The main focus of this paper is the introduction of the concept of sequence of "Toeplitz momentary symbols", associated with a given sequence of truncated Toeplitz-like matrices. We construct the symbol in the same way as in the GLT theory, but we keep the information of the small norm contributions. The low-rank contributions are still disregarded, and we give an idea on the reason why this is negligible in certain cases and why it is not in other cases, being aware that in presence of high nonnormality the same low-rank perturbation can produce a dramatic change in the eigenvalue distribution. Moreover, a difference with respect to the LFA symbols is that GLT symbols and Toeplitz momentary symbols are more general and are applicable to a larger class of matrices. We show the applicability of the approach which leads to higher accuracy in some cases when compared with the GLT symbol. Finally, since for many applications and their analysis it is often necessary to consider non-square Toeplitz matrices, we formalize and provide some useful definitions, applicable for non-square Toeplitz momentary symbols.


title: End-to-end Triplet Loss based Emotion Embedding System for Speech Emotion Recognition

abstract: In this paper, an end-to-end neural embedding system based on triplet loss and residual learning has been proposed for speech emotion recognition. The proposed system learns the embeddings from the emotional information of the speech utterances. The learned embeddings are used to recognize the emotions portrayed by given speech samples of various lengths. The proposed system implements Residual Neural Network architecture. It is trained using softmax pre-training and triplet loss function. The weights between the fully connected and embedding layers of the trained network are used to calculate the embedding values. The embedding representations of various emotions are mapped onto a hyperplane, and the angles among them are computed using the cosine similarity. These angles are utilized to classify a new speech sample into its appropriate emotion class. The proposed system has demonstrated 91.67% and 64.44% accuracy while recognizing emotions for RAVDESS and IEMOCAP dataset, respectively.


title: Experimental Quantum Generative Adversarial Networks for Image Generation

abstract: Quantum machine learning is expected to be one of the first practical applications of near-term quantum devices. Pioneer theoretical works suggest that quantum generative adversarial networks (GANs) may exhibit a potential exponential advantage over classical GANs, thus attracting widespread attention. However, it remains elusive whether quantum GANs implemented on near-term quantum devices can actually solve real-world learning tasks. Here, we devise a flexible quantum GAN scheme to narrow this knowledge gap, which could accomplish image generation with arbitrarily high-dimensional features, and could also take advantage of quantum superposition to train multiple examples in parallel. For the first time, we experimentally achieve the learning and generation of real-world hand-written digit images on a superconducting quantum processor. Moreover, we utilize a gray-scale bar dataset to exhibit the competitive performance between quantum GANs and the classical GANs based on multilayer perceptron and convolutional neural network architectures, respectively, benchmarked by the Fr\'echet Distance score. Our work provides guidance for developing advanced quantum generative models on near-term quantum devices and opens up an avenue for exploring quantum advantages in various GAN-related learning tasks.


title: Empirical relationships for heavy-ion equilibrated charges and charge-changing cross-sections in rarefied hydrogen and their application

abstract: Highly ionized heavy evaporation residues (ERs) resulting from heavy ion (HI) fusion-evaporation reactions are knocked out from solid targets to rarefied gas of gas-filled recoil separators. In gas, these ERs undergo charge-changing collisions on their way to a detection system. An equilibrium between the loss of charge (electron capture) and the gain of charge (electron loss) for ionized ERs allows one to use equilibrated charge state distributions in trajectory calculations for ERs moving through a magnetic field. The distance from a target to the point where the ER charge state distributions become to be the equilibrated one is essential for such calculations. This distance can be estimated in simulations based on electron capture and loss cross-sections for energetic HIs. Proper approximations of available experimental data have provided these cross-sections, which were applied to the simulation of the ionic charge evolution for ionized heavy ERs.


title: Mitigating Gender Bias in Machine Translation with Target Gender Annotations

abstract: When translating "The secretary asked for details." to a language with grammatical gender, it might be necessary to determine the gender of the subject "secretary". If the sentence does not contain the necessary information, it is not always possible to disambiguate. In such cases, machine translation systems select the most common translation option, which often corresponds to the stereotypical translations, thus potentially exacerbating prejudice and marginalisation of certain groups and people. We argue that the information necessary for an adequate translation can not always be deduced from the sentence being translated or even might depend on external knowledge. Therefore, in this work, we propose to decouple the task of acquiring the necessary information from the task of learning to translate correctly when such information is available. To that end, we present a method for training machine translation systems to use word-level annotations containing information about subject's gender. To prepare training data, we annotate regular source language words with grammatical gender information of the corresponding target language words. Using such data to train machine translation systems reduces their reliance on gender stereotypes when information about the subject's gender is available. Our experiments on five language pairs show that this allows improving accuracy on the WinoMT test set by up to 25.8 percentage points.


title: Mirror energy differences in T=1/2 f7/2-shell nuclei within isospin-dependent DFT

abstract: Background: Small asymmetry between neutrons and protons, caused by the differences in masses and charges of the up and down constituent quarks leads to the isospin symmetry breaking. The isospin non-conservation affects broad range of observables from superallowed Fermi weak interaction to isospin-forbidden electromagnetic rates. Its most profound and cleanest manifestation are systematic shifts in masses and excitation energies of mirror atomic nuclei. Purpose: Recently, we constructed the charge-dependent DFT that includes class II and III local interactions and demonstrated that the model allows for very accurate reproduction of Mirror and Triplet Displacement energies in a very broad range of masses. The aim of this work is to further test the charge-dependent functional by studying Mirror Energy Differences (MEDs) in function of angular momentum $I$. Methods: To compute MEDs we use DFT-rooted no core configuration interaction model. This post mean-field method restores rotational symmetry and takes into account configuration mixing within a space that includes relevant (multi)particle-(multi)hole Slater determinants. Results: We applied the model to $f_{7/2}$-shell mirror pairs of $A=43$, $45$, $47$, and $49$ focusing on MEDs in low-spin part (below band crossing) what allowed us to limit the model space to seniority one and three (one broken pair) configurations. Conclusions: We demonstrate that, for spins $I\leq 15/2$ being subject of the present study, our model reproduces well experimental MEDs which vary strongly in function of $I$ and $A$. The quality of model's predictions is comparable to the nuclear shell-model results by Bentley et al. Phys. Rev. C 92, 024310 (2015).


title: The observation of the Crab Nebula with LHAASO-KM2A for the performance study

abstract: As a sub-array of the Large High Altitude Air Shower Observatory (LHAASO), KM2A is mainly designed to cover a large fraction of the northern sky to hunt for gamma-ray sources at energies above 10 TeV. Even though the detector construction is still underway, a half of the KM2A array has been operating stably since the end of 2019. In this paper, we present the pipeline of KM2A data analysis and the first observation on the Crab Nebula, a standard candle in very high energy gamma-ray astronomy. We detect gamma-ray signals from the Crab Nebula in both energy ranges of 10$-$100 TeV and $>$100 TeV with high significance, by analyzing the KM2A data of 136 live days between December 2019 and May 2020. With the observations, we test the detector performance including angular resolution, pointing accuracy and cosmic ray background rejection power. The energy spectrum of the Crab Nebula in the energy range 10-250 TeV fits well with a single power-law function dN/dE =(1.13$\pm$0.05$_{stat}$$\pm$0.08$_{sys}$)$\times$10$^{-14}$$\cdot$(E/20TeV)$^{-3.09\pm0.06_{stat}\pm0.02_{sys}}$ cm$^{-2}$ s$^{-1}$ TeV$^{-1}$. It is consistent with previous measurements by other experiments. This opens a new window of gamma-ray astronomy above 0.1 PeV through which ultrahigh-energy gamma-ray new phenomena, such as cosmic PeVatrons, might be discovered.


title: Butson full propelinear codes

abstract: In this paper we study Butson Hadamard matrices, and codes over finite rings coming from these matrices in logarithmic form, called BH-codes. We introduce a new morphism of Butson Hadamard matrices through a generalized Gray map on the matrices in logarithmic form, which is comparable to the morphism given in a recent note of \'{O} Cath\'{a}in and Swartz. That is, we show how, if given a Butson Hadamard matrix over the $k^{\rm th}$ roots of unity, we can construct a larger Butson matrix over the $\ell^{\rm th}$ roots of unity for any $\ell$ dividing $k$, provided that any prime $p$ dividing $k$ also divides $\ell$. We prove that a $\mathbb{Z}_{p^s}$-additive code with $p$ a prime number is isomorphic as a group to a BH-code over $\mathbb{Z}_{p^s}$ and the image of this BH-code under the Gray map is a BH-code over $\mathbb{Z}_p$ (binary Hadamard code for $p=2$). Further, we investigate the inherent propelinear structure of these codes (and their images) when the Butson matrix is cocyclic. Some structural properties of these codes are studied and examples are provided.


title: Harmonic functions of polynomial growth on infinite penny graphs

abstract: For an infinite penny graph, we study the finite-dimensional property for the space of harmonic functions, or ancient solutions of the heat equation, of polynomial growth. We prove the asymptotically sharp dimensional estimate for the above spaces.


title: DoFE: Domain-oriented Feature Embedding for Generalizable Fundus Image Segmentation on Unseen Datasets

abstract: Deep convolutional neural networks have significantly boosted the performance of fundus image segmentation when test datasets have the same distribution as the training datasets. However, in clinical practice, medical images often exhibit variations in appearance for various reasons, e.g., different scanner vendors and image quality. These distribution discrepancies could lead the deep networks to over-fit on the training datasets and lack generalization ability on the unseen test datasets. To alleviate this issue, we present a novel Domain-oriented Feature Embedding (DoFE) framework to improve the generalization ability of CNNs on unseen target domains by exploring the knowledge from multiple source domains. Our DoFE framework dynamically enriches the image features with additional domain prior knowledge learned from multi-source domains to make the semantic features more discriminative. Specifically, we introduce a Domain Knowledge Pool to learn and memorize the prior information extracted from multi-source domains. Then the original image features are augmented with domain-oriented aggregated features, which are induced from the knowledge pool based on the similarity between the input image and multi-source domain images. We further design a novel domain code prediction branch to infer this similarity and employ an attention-guided mechanism to dynamically combine the aggregated features with the semantic features. We comprehensively evaluate our DoFE framework on two fundus image segmentation tasks, including the optic cup and disc segmentation and vessel segmentation. Our DoFE framework generates satisfying segmentation results on unseen datasets and surpasses other domain generalization and network regularization methods.


title: Deep Reservoir Networks with Learned Hidden Reservoir Weights using Direct Feedback Alignment

abstract: Deep Reservoir Computing has emerged as a new paradigm for deep learning, which is based around the reservoir computing principle of maintaining random pools of neurons combined with hierarchical deep learning. The reservoir paradigm reflects and respects the high degree of recurrence in biological brains, and the role that neuronal dynamics play in learning. However, one issue hampering deep reservoir network development is that one cannot backpropagate through the reservoir layers. Recent deep reservoir architectures do not learn hidden or hierarchical representations in the same manner as deep artificial neural networks, but rather concatenate all hidden reservoirs together to perform traditional regression. Here we present a novel Deep Reservoir Network for time series prediction and classification that learns through the non-differentiable hidden reservoir layers using a biologically-inspired backpropagation alternative called Direct Feedback Alignment, which resembles global dopamine signal broadcasting in the brain. We demonstrate its efficacy on two real world multidimensional time series datasets.


title: A note on the Turing universality of homogeneous potential wells and geodesible flows

abstract: We explore some properties of flows with strongly adapted 1-forms, originally discovered in (Tao 2017), which can be used to embed Turing machines into dynamical systems. In particular, we discuss some relations to geodesible flows, and show that even a slight modification of the dynamical system, such as homogeneity, can lead to an intermediate class of flows between adapted flows and geodesible flows, while still retaining Turing universality.


title: Detection of simplest amino acid glycine in the atmosphere of Venus

abstract: Amino acids are considered to be prime ingredients in chemistry, leading to life. Glycine is the simplest amino acid and most commonly found in animal proteins. It is a glucogenic and non-essential amino acid that is produced naturally by living bodies and plays a key role in the creation of several other important bio-compounds and proteins. We report the first spectroscopic detection of the rotational absorption lines of the simplest amino acid glycine (NH$_{2}$CH$_{2}$COOH) with confirmer I and II in the atmosphere of Venus using the archival data from the Atacama Large Millimeter/Submillimeter Array (ALMA). We detect the eleven rotational absorption lines of NH$_{2}$CH$_{2}$COOH between the frequency range of $\nu$ = 245$-$262 GHz with $\geq$3$\sigma$ statistical significance. We calculate the total column density of glycine in the atmosphere of Venus is $N$(NH$_{2}$CH$_{2}$COOH) $\sim$ 5$\times$10$^{14}$ cm$^{-2}$. Using the column density information of glycine, we calculate the abundance of glycine is $\sim$ 1.6$\times$10$^{-9}$ in the atmosphere of Venus. The detection of glycine in the atmosphere of Venus might be one of the keys to an understanding of the formation mechanisms of prebiotic molecules in the atmosphere of Venus. The detection of glycine indicates that the upper atmosphere of Venus may be going through nearly the same biological method as Earth billions of years ago.


title: S3ML: A Secure Serving System for Machine Learning Inference

abstract: We present S3ML, a secure serving system for machine learning inference in this paper. S3ML runs machine learning models in Intel SGX enclaves to protect users' privacy. S3ML designs a secure key management service to construct flexible privacy-preserving server clusters and proposes novel SGX-aware load balancing and scaling methods to satisfy users' Service-Level Objectives. We have implemented S3ML based on Kubernetes as a low-overhead, high-available, and scalable system. We demonstrate the system performance and effectiveness of S3ML through extensive experiments on a series of widely-used models.


title: KLearn: Background Knowledge Inference from Summarization Data

abstract: The goal of text summarization is to compress documents to the relevant information while excluding background information already known to the receiver. So far, summarization researchers have given considerably more attention to relevance than to background knowledge. In contrast, this work puts background knowledge in the foreground. Building on the realization that the choices made by human summarizers and annotators contain implicit information about their background knowledge, we develop and compare techniques for inferring background knowledge from summarization data. Based on this framework, we define summary scoring functions that explicitly model background knowledge, and show that these scoring functions fit human judgments significantly better than baselines. We illustrate some of the many potential applications of our framework. First, we provide insights into human information importance priors. Second, we demonstrate that averaging the background knowledge of multiple, potentially biased annotators or corpora greatly improves summary-scoring performance. Finally, we discuss potential applications of our framework beyond summarization.


title: Proposal and Proof-of-Principle Demonstration of Fast-Switching Broadband Frequency-Shifting for a Frequency-Multiplexed Quantum Repeater

abstract: A proposal for fast-switching broadband frequency-shifting technology making use of frequency conversion in a nonlinear crystal is set forth, whereby the shifting is imparted to the converted photons by creating a bank of frequency-displaced pump modes that can be selected by a photonic switch and directed to the nonlinear crystal. Proof-of-principle results show that the expected frequency-shifting operation can be achieved. Even though the dimensions of the currently employed crystal and significant excess loss in the experimental setup prevented conversion of single-photon-level inputs, thorough experimental and theoretical analysis of the noise contribution allowed for the estimation of the system performance in an optimized scenario, where the expected signal-to-noise ratio for single-photon conversion and frequency-shifting can reach up to 25 dB with proper narrow-band filtering and state-of-the-art devices. The proposed frequency-shifting solution figures as a promising candidate for applications in frequency-multiplexed quantum repeater architectures with 25 dB output SNR (with 20\% conversion efficiency) and capacity for 16 channels spread around a 100 GHz spectral region.


title: Few-shot Action Recognition with Implicit Temporal Alignment and Pair Similarity Optimization

abstract: Few-shot learning aims to recognize instances from novel classes with few labeled samples, which has great value in research and application. Although there has been a lot of work in this area recently, most of the existing work is based on image classification tasks. Video-based few-shot action recognition has not been explored well and remains challenging: 1) the differences of implementation details among different papers make a fair comparison difficult; 2) the wide variations and misalignment of temporal sequences make the video-level similarity comparison difficult; 3) the scarcity of labeled data makes the optimization difficult. To solve these problems, this paper presents 1) a specific setting to evaluate the performance of few-shot action recognition algorithms; 2) an implicit sequence-alignment algorithm for better video-level similarity comparison; 3) an advanced loss for few-shot learning to optimize pair similarity with limited data. Specifically, we propose a novel few-shot action recognition framework that uses long short-term memory following 3D convolutional layers for sequence modeling and alignment. Circle loss is introduced to maximize the within-class similarity and minimize the between-class similarity flexibly towards a more definite convergence target. Instead of using random or ambiguous experimental settings, we set a concrete criterion analogous to the standard image-based few-shot learning setting for few-shot action recognition evaluation. Extensive experiments on two datasets demonstrate the effectiveness of our proposed method.


title: Resolution as Intersection Subtyping via Modus Ponens

abstract: Resolution and subtyping are two common mechanisms in programming languages. Resolution is used by features such as type classes or Scala-style implicits to synthesize values automatically from contextual type information. Subtyping is commonly used to automatically convert the type of a value into another compatible type. So far the two mechanisms have been considered independently of each other. This paper shows that, with a small extension, subtyping with intersection types can subsume resolution. This has three main consequences. Firstly, resolution does not need to be implemented as a separate mechanism. Secondly, the interaction between resolution and subtyping becomes apparent. Finally, the integration of resolution into subtyping enables first-class (implicit) environments. The extension that recovers the power of resolution via subtyping is the modus ponens rule of propositional logic. While it is easily added to declarative subtyping, significant care needs to be taken to retain desirable properties, such as transitivity and decidability of algorithmic subtyping, and coherence. To materialize these ideas we develop $\lambda_i^{\mathsf{MP}}$, a calculus that extends a iprevious calculus with disjoint intersection types, and develop its metatheory in the Coq theorem prover.


title: TM-NET: Deep Generative Networks for Textured Meshes

abstract: We introduce TM-NET, a novel deep generative model for synthesizing textured meshes in a part-aware manner. Once trained, the network can generate novel textured meshes from scratch or predict textures for a given 3D mesh, without image guidance. Plausible and diverse textures can be generated for the same mesh part, while texture compatibility between parts in the same shape is achieved via conditional generation. Specifically, our method produces texture maps for individual shape parts, each as a deformable box, leading to a natural UV map with minimal distortion. The network separately embeds part geometry (via a PartVAE) and part texture (via a TextureVAE) into their respective latent spaces, so as to facilitate learning texture probability distributions conditioned on geometry. We introduce a conditional autoregressive model for texture generation, which can be conditioned on both part geometry and textures already generated for other parts to achieve texture compatibility. To produce high-frequency texture details, our TextureVAE operates in a high-dimensional latent space via dictionary-based vector quantization. We also exploit transparencies in the texture as an effective means to model complex shape structures including topological details. Extensive experiments demonstrate the plausibility, quality, and diversity of the textures and geometries generated by our network, while avoiding inconsistency issues that are common to novel view synthesis methods.


title: Self-Supervised Multi-View Synchronization Learning for 3D Pose Estimation

abstract: Current state-of-the-art methods cast monocular 3D human pose estimation as a learning problem by training neural networks on large data sets of images and corresponding skeleton poses. In contrast, we propose an approach that can exploit small annotated data sets by fine-tuning networks pre-trained via self-supervised learning on (large) unlabeled data sets. To drive such networks towards supporting 3D pose estimation during the pre-training step, we introduce a novel self-supervised feature learning task designed to focus on the 3D structure in an image. We exploit images extracted from videos captured with a multi-view camera system. The task is to classify whether two images depict two views of the same scene up to a rigid transformation. In a multi-view data set, where objects deform in a non-rigid manner, a rigid transformation occurs only between two views taken at the exact same time, i.e., when they are synchronized. We demonstrate the effectiveness of the synchronization task on the Human3.6M data set and achieve state-of-the-art results in 3D human pose estimation.


title: Investigating the Scalability and Biological Plausibility of the Activation Relaxation Algorithm

abstract: The recently proposed Activation Relaxation (AR) algorithm provides a simple and robust approach for approximating the backpropagation of error algorithm using only local learning rules. Unlike competing schemes, it converges to the exact backpropagation gradients, and utilises only a single type of computational unit and a single backwards relaxation phase. We have previously shown that the algorithm can be further simplified and made more biologically plausible by (i) introducing a learnable set of backwards weights, which overcomes the weight-transport problem, and (ii) avoiding the computation of nonlinear derivatives at each neuron. However, tthe efficacy of these simplifications has, so far, only been tested on simple multi-layer-perceptron (MLP) networks. Here, we show that these simplifications still maintain performance using more complex CNN architectures and challenging datasets, which have proven difficult for other biologically-plausible schemes to scale to. We also investigate whether another biologically implausible assumption of the original AR algorithm -- the frozen feedforward pass -- can be relaxed without damaging performance.


title: Gaussian Closure Scheme in the Quasi-Linkage Equilibrium Regime of Evolving Genome Populations

abstract: Describing the evolution of a population of genomes evolving in a complex fitness landscape is generally very hard. We here introduce an approximate Gaussian closure scheme to characterize analytically the statistics of a genomic population in the so-called Quasi--Linkage Equilibrium (QLE) regime, applicable to generic values of the rates of mutation or recombination and fitness functions. The Gaussian approximation is illustrated on a short-range fitness landscape with two far away and competing maxima. It unveils the existence of a phase transition from a broad to a polarized distribution of genomes as the strength of epistatic couplings is increased, characterized by slow coarsening dynamics of competing allele domains. Results of the closure scheme are corroborated by numerical simulations.


title: 2-Cluster Fixed-Point Analysis of Mean-Coupled Stuart-Landau Oscillators in the Center Manifold

abstract: We reduce the dynamics of an ensemble of mean-coupled Stuart-Landau oscillators close to the synchronized solution. In particular, we map the system onto the center manifold of the Benjamin-Feir instability, the bifurcation destabilizing the synchronized oscillation. Using symmetry arguments, we describe the structure of the dynamics on this center manifold up to cubic order, and derive expressions for its parameters. This allows us to investigate phenomena described by the Stuart-Landau ensemble, such as clustering and cluster singularities, in the lower-dimensional center manifold, providing further insights into the symmetry-broken dynamics of coupled oscillators. We show that cluster singularities in the Stuart-Landau ensemble correspond to vanishing quadratic terms in the center manifold dynamics. In addition, they act as organizing centers for the saddle-node bifurcations creating unbalanced cluster states as well for the transverse bifurcations altering the cluster stability. Furthermore, we show that bistability of different solutions with the same cluster-size distribution can only occur when either cluster contains at least $1/3$ of the oscillators, independent of the system parameters.


title: Free group representations from vector-valued multiplicative functions, III

abstract: Let $\pi$ be an irreducible unitary representation of a finitely generated nonabelian free group $\Gamma$; suppose $\pi$ is weakly contained in the regular representation. In 2001 the first and third authors conjectured that such a representation must be either odd or monotonous or duplicitous. In 2004 they introduced the class of multiplicative representations: this is a large class of representations obtained by looking at the action of $\Gamma$ on its Cayley graph. In the second paper of this series we showed that some of the multiplicative representations were monotonous. Here we show that all the other multiplicative representations are either odd or duplicitous. The conjecture is therefore established for multiplicative representations.


title: Direct Federated Neural Architecture Search

abstract: Neural Architecture Search (NAS) is a collection of methods to craft the way neural networks are built. We apply this idea to Federated Learning (FL), wherein predefined neural network models are trained on the client/device data. This approach is not optimal as the model developers can't observe the local data, and hence, are unable to build highly accurate and efficient models. NAS is promising for FL which can search for global and personalized models automatically for the non-IID data. Most NAS methods are computationally expensive and require fine-tuning after the search, making it a two-stage complex process with possible human intervention. Thus there is a need for end-to-end NAS which can run on the heterogeneous data and resource distribution typically seen in the FL scenario. In this paper, we present an effective approach for direct federated NAS which is hardware agnostic, computationally lightweight, and a one-stage method to search for ready-to-deploy neural network models. Our results show an order of magnitude reduction in resource consumption while edging out prior art in accuracy. This opens up a window of opportunity to create optimized and computationally efficient federated learning systems.


title: Two-Stream Compare and Contrast Network for Vertebral Compression Fracture Diagnosis

abstract: Differentiating Vertebral Compression Fractures (VCFs) associated with trauma and osteoporosis (benign VCFs) or those caused by metastatic cancer (malignant VCFs) are critically important for treatment decisions. So far, automatic VCFs diagnosis is solved in a two-step manner, i.e. first identify VCFs and then classify it into benign or malignant. In this paper, we explore to model VCFs diagnosis as a three-class classification problem, i.e. normal vertebrae, benign VCFs, and malignant VCFs. However, VCFs recognition and classification require very different features, and both tasks are characterized by high intra-class variation and high inter-class similarity. Moreover, the dataset is extremely class-imbalanced. To address the above challenges, we propose a novel Two-Stream Compare and Contrast Network (TSCCN) for VCFs diagnosis. This network consists of two streams, a recognition stream which learns to identify VCFs through comparing and contrasting between adjacent vertebra, and a classification stream which compares and contrasts between intra-class and inter-class to learn features for fine-grained classification. The two streams are integrated via a learnable weight control module which adaptively sets their contribution. The TSCCN is evaluated on a dataset consisting of 239 VCFs patients and achieves the average sensitivity and specificity of 92.56\% and 96.29\%, respectively.


title: Transcranial Bipolar Direct Current Stimulation of the Frontoparietal Cortex Reduces Ketamine-Induced Oscillopathies: A Pilot Study in the Sedated Rat

abstract: Running title: Frontoparietal anodal tDCS reduces ketamine-induced oscillopathies.Abstract: During the prodromal phase of schizophrenia with its complex and insidious clinical picture, electroencephalographic recordings detect widespread oscillation disturbances (or oscillopathies). Neural oscillations are electro-biomarkers of the connectivity state within systems. A single systemic administration of ketamine, a non-competitive NMDA glutamate receptor antagonist, transiently reproduces the oscillopathies with a clinical picture reminiscent of the psychosis prodrome. This acute pharmacological model may help the research and development of innovative treatments against the psychotic transition. Transcranial electrical stimulation is recognized as an appropriate non-invasive therapeutic modality since it can increase cognitive performance and modulate neural oscillations with little or no side effects. Therefore, our objective was to set up, in the sedated adult rat, a stimulation method able to normalize the ketamine-induced oscillopathies. Unilateral transcranial frontoparietal anodal stimulation by direct current (<+1 mA) was applied in ketamine-treated rats. A concomitant electroencephalographic recording of the parietal cortex measured the stimulation effects on its spontaneously-occurring oscillations. A 5-min bipolar anodal tDCS immediately and quickly reduced, significantly with an intensity-effect relationship, the ketamine-induced oscillopathies at least in the bilateral parietal cortex. A duration effect was also recorded. These preliminary neurophysiological findings are promising for developing a therapeutic proof-of-concept against neuropsychiatric disorders.


title: New Formulas of Feedback Capacity for AGN Channels with Memory: A Time-Domain Sufficient Statistic Approach

abstract: In the recent paper [1] it is shown, via an application example, that the Cover and Pombra [2] "characterization of the $n-$block or transmission" feedback capacity formula, of additive Gaussian noise (AGN) channels, is the subject of much confusion in the literature, with redundant incorrect results. The main objective of this paper is to clarify the main points of confusion and remove any further ambiguity. The first part of the paper applies time-domain methods, to derive for a first time, equivalent sequential characterizations of the Cover and Pombra characterization of feedback capacity of AGN channels driven by nonstationary and nonergodic Gaussian noise. The optimal channel input processes of the new equivalent sequential characterizations are expressed as functionals of a sufficient statistic and a Gaussian orthogonal innovations process. From the new representations follows that the Cover and Pombra $n-$block capacity formula is expressed as a functional of two generalized matrix difference Riccati equations (DRE) of filtering theory of Gaussian systems, contrary to results that appeared in the literature. In the second part of the paper the existence of the asymptotic limit of the $n-$block feedback capacity formula is shown to be equivalent to the convergence properties of solutions of the two generalized DREs. Further, necessary and or sufficient conditions are identified for existence of the asymptotic limits, for stable and unstable Gaussian noise, when the optimal input distributions are time-invariant, but not necessarily stationary. The paper contains an in depth analysis, with examples, of the specific technical issues, which are overlooked in past literature [3-7], that studied the AGN channel of [2], for stationary noises.


title: Distributional Modeling and Forecasting of Natural Gas Prices

abstract: We examine the problem of modeling and forecasting European Day-Ahead and Month-Ahead natural gas prices. For this, we propose two distinct probabilistic models that can be utilized in risk- and portfolio management. We use daily pricing data ranging from 2011 to 2020. Extensive descriptive data analysis shows that both time series feature heavy tails, conditional heteroscedasticity, and show asymmetric behavior in their differences. We propose state-space time series models under skewed, heavy-tailed distributions to capture all stylized facts of the data. They include the impact of autocorrelation, seasonality, risk premia, temperature, storage levels, the price of European Emission Allowances, and related fuel prices of oil, coal, and electricity. We provide rigorous model diagnostics and interpret all model components in detail. Additionally, we conduct a probabilistic forecasting study with significance tests and compare the predictive performance against literature benchmarks. The proposed Day-Ahead (Month-Ahead) model leads to a 13% (9%) reduction in out-of-sample continuous ranked probability score (CRPS) compared to the best performing benchmark model, mainly due to adequate modeling of the volatility and heavy tails.


title: A Chebyshev-Tau spectral method for normal modes of underwater sound propagation with a layered marine environment

abstract: The normal mode model is one of the most popular approaches for solving underwater sound propagation problems. Among other methods, the finite difference method is widely used in classic normal mode programs. In many recent studies, the spectral method has been used for discretization. It is generally more accurate than the finite difference method. However, the spectral method requires that the variables to be solved are continuous in space, and the traditional spectral method is powerless for a layered marine environment. A Chebyshev-Tau spectral method based on domain decomposition is applied to the construction of underwater acoustic normal modes in this paper. In this method, the differential equation is projected onto spectral space from the original physical space with the help of an orthogonal basis of Chebyshev polynomials. A complex matrix eigenvalue / eigenvector problem is thus formed, from which the solution of horizontal wavenumbers and modal functions can be solved. The validity of the acoustic field calculation is tested in comparison with classic programs. The results of analysis and tests show that compared with the classic finite difference method, the proposed Chebyshev-Tau spectral method has the advantage of high computational accuracy. In addition, in terms of running time, our method is faster than the Legendre-Galerkin spectral method.


title: Amplitude- and phase-resolved nano-imaging and nano-spectroscopy of polaritons in liquid environment

abstract: Localized and propagating polaritons allow for highly sensitive analysis of (bio)chemical substances and processes. Nanoimaging of the polaritons evanescent fields allows for critically important experimental mode identification and for studying field confinement. Here we describe two setups for polariton nanoimaging and spectroscopy in liquid, which is an indispensable environment for (bio)chemical samples. We first demonstrate antenna mapping with a transflection infrared scattering-type scanning near-field optical microscope (s-SNOM), where the tip acts as a near-field scattering probe. We then demonstrate a total internal reflection (TIR) based setup, where the tip is both launching and probing ultra-confined polaritons in van der Waals materials, here phonon polaritons in hexagonal boron nitride (h-BN) flakes. This work lays the foundation for s-SNOM based polariton interferometry in liquid, which has wide application potential for in-situ studies of chemical reactions at the bare or functionalized surface of polaritonic materials, including (bio)chemical recognition analogous to the classical surface plasmon resonance spectroscopy.


title: A variational autoencoder for music generation controlled by tonal tension

abstract: Many of the music generation systems based on neural networks are fully autonomous and do not offer control over the generation process. In this research, we present a controllable music generation system in terms of tonal tension. We incorporate two tonal tension measures based on the Spiral Array Tension theory into a variational autoencoder model. This allows us to control the direction of the tonal tension throughout the generated piece, as well as the overall level of tonal tension. Given a seed musical fragment, stemming from either the user input or from directly sampling from the latent space, the model can generate variations of this original seed fragment with altered tonal tension. This altered music still resembles the seed music rhythmically, but the pitch of the notes are changed to match the desired tonal tension as conditioned by the user.


title: Slow chaos in surface flows

abstract: This is a survey of recent advances in the study of chaotic and spectral properties of smooth area-preserving flows on surfaces, written in occasion of a plenary talk given at the XXI Congress of the Italian Mathematical Union, in Pavia in September 2020.


title: Landau Instability and soliton formations

abstract: Consider at a finite temperature T a superfluid moving with a velocity v relative to the thermal bath or its normal component. From Landau's argument there exists a critical v_c (T) beyond which excitations can be spontaneously generated and the system becomes unstable. Identifying the final state induced by such an instability has been an outstanding open question. Using holographic duality we perform dynamical simulations of evolutions from initial unstable states, and find that the system settles to a homogenous superfluid state with a final velocity below the critical velocity. The dynamical evolution process appears to be highly chaotic, exhibiting transient turbulence. Nevertheless we are able to identify from the simulations a universal physical mechanism for the reduction of superfluid velocity, in terms of spontaneous nucleation of solitons. We also derive a simple analytic formula which relates the final velocity to the number of solitons nucleated during the evolution.

categories: hep-th cond-mat.quant-gas cond-mat.str-el gr-qc


title: Artist-driven layering and user's behaviour impact on recommendations in a playlist continuation scenario

abstract: In this paper we provide an overview of the approach we used as team Creamy Fireflies for the ACM RecSys Challenge 2018. The competition, organized by Spotify, focuses on the problem of playlist continuation, that is suggesting which tracks the user may add to an existing playlist. The challenge addresses this issue in many use cases, from playlist cold start to playlists already composed by up to a hundred tracks. Our team proposes a solution based on a few well known models both content based and collaborative, whose predictions are aggregated via an ensembling step. Moreover by analyzing the underlying structure of the data, we propose a series of boosts to be applied on top of the final predictions and improve the recommendation quality. The proposed approach leverages well-known algorithms and is able to offer a high recommendation quality while requiring a limited amount of computational resources.


title: The maximum energy of shock-accelerated electrons in a microturbulent magnetic field

abstract: Relativistic shocks propagating into a medium with low magnetization are generated and sustained by small-scale but very strong magnetic field turbulence. This so-called "microturbulence" modifies the typical shock acceleration process, and in particular that of electrons. In this work we perform Monte Carlo (MC) simulations of electrons encountering shocks with microturbulent fields. The simulations cover a three-dimensional parameter space in shock speed, acceleration efficiency, and peak magnetic field strength. From these, a Markov Chain Monte Carlo (MCMC) method was employed to estimate the maximum electron momentum from the MC-simulated electron spectra. Having estimated this quantity at many points well-distributed over an astrophysically relevant parameter space, an MCMC method was again used to estimate the parameters of an empirical formula that computes the maximum momentum of a Fermi-accelerated electron population anywhere in this parameter space. The maximum energy is well-approximated as a broken power-law in shock speed, with the break occurring when the shock decelerates to the point where electrons can begin to escape upstream from the shock.


title: Robust Two-Stream Multi-Feature Network for Driver Drowsiness Detection

abstract: Drowsiness driving is a major cause of traffic accidents and thus numerous previous researches have focused on driver drowsiness detection. Many drive relevant factors have been taken into consideration for fatigue detection and can lead to high precision, but there are still several serious constraints, such as most existing models are environmentally susceptible. In this paper, fatigue detection is considered as temporal action detection problem instead of image classification. The proposed detection system can be divided into four parts: (1) Localize the key patches of the detected driver picture which are critical for fatigue detection and calculate the corresponding optical flow. (2) Contrast Limited Adaptive Histogram Equalization (CLAHE) is used in our system to reduce the impact of different light conditions. (3) Three individual two-stream networks combined with attention mechanism are designed for each feature to extract temporal information. (4) The outputs of the three sub-networks will be concatenated and sent to the fully-connected network, which judges the status of the driver. The drowsiness detection system is trained and evaluated on the famous Nation Tsing Hua University Driver Drowsiness Detection (NTHU-DDD) dataset and we obtain an accuracy of 94.46%, which outperforms most existing fatigue detection models.


title: Average Cost Optimal Control of Stochastic Systems Using Reinforcement Learning

abstract: This paper addresses the average cost minimization problem for discrete-time systems with multiplicative and additive noises via reinforcement learning. By using Q-function, we propose an online learning scheme to estimate the kernel matrix of Q-function and to update the control gain using the data along the system trajectories. The obtained control gain and kernel matrix are proved to converge to the optimal ones. To implement the proposed learning scheme, an online model-free reinforcement learning algorithm is given, where recursive least squares method is used to estimate the kernel matrix of Q-function. A numerical example is presented to illustrate the proposed approach.


title: The Anosov-Katok method and pseudo-rotations in symplectic dynamics

abstract: We prove that toric symplectic manifolds admit Hamiltonian pseudo-rotations with a finite, and in a sense minimal, number of ergodic measures. The set of ergodic measures of these pseudo-rotations consists of the measure induced by the symplectic volume form and the Dirac measures supported at the fixed points of the torus action. Our construction relies on the conjugation method of Anosov and Katok.


title: Massive MIMO for Cellular-Connected UAV: Challenges and Promising Solutions

abstract: Massive multiple-input multiple-output (MIMO) is a promising technology for enabling cellular-connected unmanned aerial vehicle (UAV) communications in the future. Equipped with full-dimensional large arrays, ground base stations (GBSs) can apply adaptive fine-grained three-dimensional (3D) beamforming to mitigate the strong interference between high-altitude UAVs and low-altitude terrestrial users, thus significantly enhancing the network spectral efficiency. However, the performance gain of massive MIMO critically depends on the accurate channel state information (CSI) of both UAVs and terrestrial users at the GBSs, which is practically difficult to achieve due to UAV-induced pilot contamination and UAV's high mobility in 3D. Moreover, the increasingly popular applications relying on a large group of coordinated UAVs or UAV swarm as well as the practical hybrid GBS beamforming architecture for massive MIMO further complicate the pilot contamination and channel/beam tracking problems. In this article, we provide an overview of the above challenging issues, propose new solutions to cope with them, and discuss about promising directions for future research. Preliminary simulation results are also provided to validate the effectiveness of proposed solutions.


title: A method-of-lines formulation for a model of reactive settling in tanks with varying cross-sectional area

abstract: Reactive settling denotes the process of sedimentation of small solid particles dispersed in a viscous fluid with simultaneous reactions between the components that constitute the solid and liquid phases. This process is of particular importance for the simulation and control of secondary settling tanks (SSTs) in water resource recovery facilities (WRRFs), formerly known as wastewater treatment plants. A spatially one-dimensional model of reactive settling in an SST is formulated by combining a mechanistic model of sedimentation with compression with a model of biokinetic reactions. In addition, the cross-sectional area of the tank is allowed to vary as a function of height. The final model is a system of strongly degenerate parabolic, nonlinear partial differential equations (PDEs)that include discontinuous coefficients to describe the feed, underflow and overflow mechanisms, as well as singular source terms that model the feed mechanism. A finite difference scheme for the final model is derived by first deriving a method-of-lines formulation (discrete in space, continuous in time), and then passing to a fully discrete scheme by a time discretization. The advantage of this formulation is its compatibility with common practice in development of software for WRRFs. The main mathematical result is an invariant-region property, which implies that physically relevant numerical solutions are produced. Simulations of denitrification in SSTs in wastewater treatment illustrate the model and its discretization.


title: Semilinear equations for non-local operators: beyond the fractional Laplacian

abstract: We study semilinear problems in general bounded open sets for non-local operators with exterior and boundary conditions. The operators are more general than the fractional Laplacian. We also give results in case of bounded $C^{1,1}$ open sets.


title: Double-peaked Lyman-$\alpha$ emission at z=6.803: a reionisation-era galaxy self-ionising its local HII bubble

abstract: We report the discovery of a double-peaked Lyman-alpha profile in a galaxy at $z=6.803$, A370p\_z1, in the parallel Frontier Field of Abell 370. The velocity separation between the blue and red peaks of the Lyman-$\alpha$ profile ($\Delta v=101_{-19}^{+38} (\pm48)\,\text{km s}^{-1}$) suggests an extremely high escape fraction of ionising photons $> 59(51)\% (2\sigma)$. The spectral energy distribution indicates a young ($50$ Myr), star-forming ($12\pm 6 \, \rm{M}_\odot \rm{yr}^{-1}$) galaxy with an IRAC excess implying strong [OIII]+H$\beta$ emission. On the basis of the high escape fraction measured, we demonstrate that A370p\_z1 was solely capable of creating an ionised bubble sufficiently large to account for the blue component of its Lyman-alpha profile. We discuss whether A370p\_z1 may be representative of a larger population of luminous $z\simeq7$ double-peaked Lyman-alpha emitting sources with high escape fractions that self-ionised their surroundings without contributions from associated UV-fainter sources.


title: Measuring entanglement of a rank-2 mixed state prepared on a quantum computer

abstract: We study the entanglement between a certain qubit and the remaining system in rank- 2 mixed states prepared on the quantum computer. The protocol, which we propose for this purpose, is based on the relation of geometric measure of entanglement with correlations between qubits. As a special case, we consider a two-qubit rank-2 mixed state and find the relation of concurrence with the geometric measure of entanglement. On the ibmq-melbourne quantum computer we measure the geometric measure of entanglement in the cases of 2- and 4-qubit mixed quantum states which consist of Schr\"odinger cat states. We study the dependence of the value of entanglement on the parameter which defines the weight of pure states. Finally, we determine the concurrence of 2-qubit mixed state.


title: Magnetism of Sigma-Phase Fe68V32 Compound

abstract: The magnetic phase diagram in the H-T coordinates has been determined for {\sigma}-Fe68V32 from the ZFC/FC magnetization measurements. The re-entrant character of magnetism, going from paramagnetic through ferromagnetic to spin-glass (SG) states, has been evidenced. The SG phase is magnetically heterogeneous, because two sub phases can be identified i.e. with the strong (SG-SI) and the weak (SG-WI) irreversibility. The ireversibility, T_irr and the crossover, T_cros, temperatures were quantitatively analysed using the mean-field theory and {\phi}_irr=1.6(2) and {\phi}_cros=0.91(9) values were obtained. A qualitative agreement with the Gabay-Toulouse model was reached. The isothermal magnetization measurements point to a soft magnetic behaviour of the studied sample. The {\gamma} critical exponent was determined with the Kouvel-Fisher approach yielding the value of {\gamma}=1.0(1) in line with the mean-field theory.


title: Transient chaos in time-delayed systems subjected to parameter drift

abstract: External and internal factors may cause a system's parameter to vary with time before it stabilizes. This drift induces a regime shift when the parameter crosses a bifurcation. Here, we study the case of an infinite dimensional system: a time-delayed oscillator whose time delay varies at a small but non-negligible rate. Our research shows that due to this parameter drift, trajectories from a chaotic attractor tip to other states with a certain probability. This causes the appearance of the phenomenon of transient chaos. By using an ensemble approach, we find a gamma distribution of transient lifetimes, unlike in other non-delayed systems where normal distributions have been found to govern the process. Furthermore, we analyze how the parameter change rate influences the tipping probability, and we derive a scaling law relating the parameter value for which the tipping takes place and the lifetime of the transient chaos with the parameter change rate.


title: Automation of Hemocompatibility Analysis Using Image Segmentation and a Random Forest

abstract: The hemocompatibility of blood-contacting medical devices remains one of the major challenges in biomedical engineering and makes research in the field of new and improved materials inevitable. However, current in-vitro test and analysis methods are still lacking standardization and comparability, which impedes advances in material design. For example, the optical platelet analysis of material in-vitro hemocompatibility tests is carried out manually or semi-manually by each research group individually. As a step towards standardization, this paper proposes an automation approach for the optical platelet count and analysis. To this end, fluorescence images are segmented using Zach's convexification of the multiphase-phase piecewise constant Mumford--Shah model. The resulting connected components of the non-background segments then need to be classified as platelet or no platelet. Therefore, a supervised random forest is applied to feature vectors derived from the components using features like area, perimeter and circularity. With an overall high accuracy and low error rates, the random forest achieves reliable results. This is supported by high areas under the receiver-operator and the prediction-recall curve, respectively. We developed a new method for a fast, user-independent and reproducible analysis of material hemocompatibility tests, which is therefore a unique and powerful tool for advances in biomaterial research.


title: Variable impedance control and learning -- A review

abstract: Robots that physically interact with their surroundings, in order to accomplish some tasks or assist humans in their activities, require to exploit contact forces in a safe and proficient manner. Impedance control is considered as a prominent approach in robotics to avoid large impact forces while operating in unstructured environments. In such environments, the conditions under which the interaction occurs may significantly vary during the task execution. This demands robots to be endowed with on-line adaptation capabilities to cope with sudden and unexpected changes in the environment. In this context, variable impedance control arises as a powerful tool to modulate the robot's behavior in response to variations in its surroundings. In this survey, we present the state-of-the-art of approaches devoted to variable impedance control from control and learning perspectives (separately and jointly). Moreover, we propose a new taxonomy for mechanical impedance based on variability, learning, and control. The objective of this survey is to put together the concepts and efforts that have been done so far in this field, and to describe advantages and disadvantages of each approach. The survey concludes with open issues in the field and an envisioned framework that may potentially solve them.


title: Unfolding recurrence by Green's functions for optimized reservoir computing

abstract: Cortical networks are strongly recurrent, and neurons have intrinsic temporal dynamics. This sets them apart from deep feed-forward networks. Despite the tremendous progress in the application of feed-forward networks and their theoretical understanding, it remains unclear how the interplay of recurrence and non-linearities in recurrent cortical networks contributes to their function. The purpose of this work is to present a solvable recurrent network model that links to feed forward networks. By perturbative methods we transform the time-continuous, recurrent dynamics into an effective feed-forward structure of linear and non-linear temporal kernels. The resulting analytical expressions allow us to build optimal time-series classifiers from random reservoir networks. Firstly, this allows us to optimize not only the readout vectors, but also the input projection, demonstrating a strong potential performance gain. Secondly, the analysis exposes how the second order stimulus statistics is a crucial element that interacts with the non-linearity of the dynamics and boosts performance.


title: Exploring Universal Speech Attributes for Speaker Verification with an Improved Cross-stitch Network

abstract: The universal speech attributes for x-vector based speaker verification (SV) are addressed in this paper. The manner and place of articulation form the fundamental speech attribute unit (SAU), and then new speech attribute (NSA) units for acoustic modeling are generated by tied tri-SAU states. An improved cross-stitch network is adopted as a multitask learning (MTL) framework for integrating these universal speech attributes into the x-vector network training process. Experiments are conducted on common condition 5 (CC5) of the core-core and the 10 s-10 s tests of the NIST SRE10 evaluation set, and the proposed algorithm can achieve consistent improvements over the baseline x-vector on both these tasks.


title: Emergent excitability in adaptive networks of non-excitable units

abstract: Population bursts in a large ensemble of coupled elements result from the interplay between the local excitable properties of the nodes and the global network topology. Here collective excitability and self-sustained bursting oscillations are shown to spontaneously emerge in adaptive networks of globally coupled non-excitable units. The ingredients to observe collective excitability are the coexistence of states with different degree of synchronizaton joined to a global feedback acting, on a slow timescale, against the synchronization (desynchronization) of the oscillators. These regimes are illustrated for two paradigmatic classes of coupled rotators: namely, the Kuramoto model with and without inertia. For the bimodal Kuramoto model we analytically show that the macroscopic evolution originates from the existence of a critical manifold organizing the fast collective dynamics on a slow timescale. Our results provide evidence that adaptation can induce excitability by maintaining a network permanently out-of-equilibrium.


title: Regret minimization in stochastic non-convex learning via a proximal-gradient approach

abstract: Motivated by applications in machine learning and operations research, we study regret minimization with stochastic first-order oracle feedback in online constrained, and possibly non-smooth, non-convex problems. In this setting, the minimization of external regret is beyond reach for first-order methods, so we focus on a local regret measure defined via a proximal-gradient mapping. To achieve no (local) regret in this setting, we develop a prox-grad method based on stochastic first-order feedback, and a simpler method for when access to a perfect first-order oracle is possible. Both methods are min-max order-optimal, and we also establish a bound on the number of prox-grad queries these methods require. As an important application of our results, we also obtain a link between online and offline non-convex stochastic optimization manifested as a new prox-grad scheme with complexity guarantees matching those obtained via variance reduction techniques.


title: Annotationsaurus: A Searchable Directory of Annotation Tools

abstract: Manual annotation of textual documents is a necessary task when constructing benchmark corpora for training and evaluating machine learning algorithms. We created a comprehensive directory of annotation tools that currently includes 93 tools. We analyzed the tools over a set of 31 features and implemented simple scripts and a Web application that filters the tools based on chosen criteria. We present two use cases using the directory and propose ideas for its maintenance. The directory, source codes for scripts, and link to the Web application are available at: https://github.com/mariananeves/annotation-tools