title: Online Learning Robust Control of Nonlinear Dynamical Systems

abstract: In this work we address the problem of the online robust control of nonlinear dynamical systems perturbed by disturbance. We study the problem of attenuation of the total cost over a duration $T$ in response to the disturbances. We consider the setting where the cost function (at a particular time) is a general continuous function and adversarial, the disturbance is adversarial and bounded at any point of time. Our goal is to design a controller that can learn and adapt to achieve a certain level of attenuation. We analyse two cases (i) when the system is known and (ii) when the system is unknown. We measure the performance of the controller by the deviation of the controller's cost for a sequence of cost functions with respect to an attenuation $\gamma$, $R^p_t$. We propose an online controller and present guarantees for the metric $R^p_t$ when the maximum possible attenuation is given by $\overline{\gamma}$, which is a system constant. We show that when the controller has preview of the cost functions and the disturbances for a short duration of time and the system is known $R^p_T(\gamma) = O(1)$ when $\gamma \geq \gamma_c$, where $\gamma_c = \mathcal{O}(\overline{\gamma})$. We then show that when the system is unknown the proposed controller with a preview of the cost functions and the disturbances for a short horizon achieves $R^p_T(\gamma) = \mathcal{O}(N) + \mathcal{O}(1) + \mathcal{O}((T-N)g(N))$, when $\gamma \geq \gamma_c$, where $g(N)$ is the accuracy of a given nonlinear estimator and $N$ is the duration of the initial estimation period. We also characterize the lower bound on the required prediction horizon for these guarantees to hold in terms of the system constants.


title: Large eddy simulation of a supersonic lifted hydrogen flame with sparse-Lagrangian multiple mapping conditioning approach

abstract: The Multiple Mapping Conditioning / Large Eddy Simulation (MMC-LES) approach is used to simulate a supersonic lifted hydrogen jet flame, which features shock-induced autoignition, shock-flame interaction, lifted flame stabilization, and finite-rate chemistry effects. The shocks and expansion waves, shock-reaction interactions and overall flame characteristics are accurately reproduced by the model. Predictions are compared with the detailed experimental data for the mean axial velocity, mean and root-mean-square temperature, species mole fractions, and mixture fraction at various locations. The predicted and experimentally observed flame structures are compared through scatter plots of species mole fractions and temperature against mixture fraction. Unlike most past MMC-LES which has been applied to low-Mach flames, in this supersonic flame case pressure work and viscous heating are included in the stochastic FDF equations. Analysis indicates that the pressure work plays an important role in autoignition induction and flame stabilization, whereas viscous heating is only significant in shear layers (but still negligibly small compared to the pressure work). The evolutions of particle information subject to local gas dynamics are extracted through trajectory analysis on representative fuel and oxidizer particles. The particles intermittently enter the extinction region and may be deviated from the full burning or mixing lines under the effects of shocks, expansion waves and viscous heating. The chemical explosive mode analysis performed on the Lagrangian particles shows that temperature, the H and OH radicals contribute dominantly to CEM respectively in the central fuel jet, fuel-rich and fuel-lean sides. The pronounced particle Damk\"ohler numbers first occur in the fuel jet / coflow shear layer, enhanced at the first shock intersection point and peak around the flame stabilization point.


title: Game-Theoretic Model Predictive Control with Data-Driven Identification of Vehicle Model for Head-to-Head Autonomous Racing

abstract: Resolving edge-cases in autonomous driving, head-to-head autonomous racing is getting a lot of attention from the industry and academia. In this study, we propose a game-theoretic model predictive control (MPC) approach for head-to-head autonomous racing and data-driven model identification method. For the practical estimation of nonlinear model parameters, we adopted the hyperband algorithm, which is used for neural model training in machine learning. The proposed controller comprises three modules: 1) game-based opponents' trajectory predictor, 2) high-level race strategy planner, and 3) MPC-based low-level controller. The game-based predictor was designed to predict the future trajectories of competitors. Based on the prediction results, the high-level race strategy planner plans several behaviors to respond to various race circumstances. Finally, the MPC-based controller computes the optimal control commands to follow the trajectories. The proposed approach was validated under various racing circumstances in an official simulator of the Indy Autonomous Challenge. The experimental results show that the proposed method can effectively overtake competitors, while driving through the track as quickly as possible without collisions.


title: Diverse Part Discovery: Occluded Person Re-identification with Part-Aware Transformer

abstract: Occluded person re-identification (Re-ID) is a challenging task as persons are frequently occluded by various obstacles or other persons, especially in the crowd scenario. To address these issues, we propose a novel end-to-end Part-Aware Transformer (PAT) for occluded person Re-ID through diverse part discovery via a transformer encoderdecoder architecture, including a pixel context based transformer encoder and a part prototype based transformer decoder. The proposed PAT model enjoys several merits. First, to the best of our knowledge, this is the first work to exploit the transformer encoder-decoder architecture for occluded person Re-ID in a unified deep model. Second, to learn part prototypes well with only identity labels, we design two effective mechanisms including part diversity and part discriminability. Consequently, we can achieve diverse part discovery for occluded person Re-ID in a weakly supervised manner. Extensive experimental results on six challenging benchmarks for three tasks (occluded, partial and holistic Re-ID) demonstrate that our proposed PAT performs favorably against stat-of-the-art methods.


title: Linear Convergence of Entropy-Regularized Natural Policy Gradient with Linear Function Approximation

abstract: Natural policy gradient (NPG) methods with entropy regularization achieve impressive empirical success in reinforcement learning problems with large state-action spaces. However, their convergence properties and the impact of entropy regularization remain elusive in the function approximation regime. In this paper, we establish finite-time convergence analyses of entropy-regularized NPG with linear function approximation under softmax parameterization. In particular, we prove that entropy-regularized NPG with averaging satisfies the \emph{persistence of excitation} condition, and achieves a fast convergence rate of $\tilde{O}(1/T)$ up to a function approximation error in regularized Markov decision processes. This convergence result does not require any a priori assumptions on the policies. Furthermore, under mild regularity conditions on the concentrability coefficient and basis vectors, we prove that entropy-regularized NPG exhibits \emph{linear convergence} up to a function approximation error.


title: A Sequence Selection Bound for the Capacity of the Nonlinear Fiber Channel

abstract: A novel technique to optimize the input distribution and compute a lower bound for the capacity of the nonlinear optical fiber channel is proposed. The technique improves previous bounds obtained with the additive white Gaussian noise decoding metric.


title: Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model

abstract: Recently, there is an effort to extend fine-grained entity typing by using a richer and ultra-fine set of types, and labeling noun phrases including pronouns and nominal nouns instead of just named entity mentions. A key challenge for this ultra-fine entity typing task is that human annotated data are extremely scarce, and the annotation ability of existing distant or weak supervision approaches is very limited. To remedy this problem, in this paper, we propose to obtain training data for ultra-fine entity typing by using a BERT Masked Language Model (MLM). Given a mention in a sentence, our approach constructs an input for the BERT MLM so that it predicts context dependent hypernyms of the mention, which can be used as type labels. Experimental results demonstrate that, with the help of these automatically generated labels, the performance of an ultra-fine entity typing model can be improved substantially. We also show that our approach can be applied to improve traditional fine-grained entity typing after performing simple type mapping.


title: Probabilistic Scan Matching: Bayesian Pose Estimation from Point Clouds

abstract: Estimating position and orientation change of a mobile platform from two consecutive point clouds provided by a high-resolution sensor is a key problem in autonomous navigation. In particular, scan matching algorithms aim to find the translation and rotation of the platform such that the two point clouds coincide. The association of measurements in point cloud one with measurements in point cloud two is a problem inherent to scan matching. Existing methods perform non-probabilistic data association, i.e., they assume a single association hypothesis. This leads to overconfident pose estimates and reduced estimation accuracy in ambiguous environments. Our probabilistic scan matching approach addresses this issue by considering all association hypotheses with their respective likelihoods. We formulate a holistic Bayesian estimation problem for both data association and pose estimation and present the corresponding joint factor graph. Near-optimum maximum a posteriori (MAP) estimates of the sensor pose are computed by performing iterative message passing on the factor graph. Our numerical study shows performance improvements compared to non-probabilistic scan matching methods that are based on the normal distributions transform (NDT) and implicit moving least squares (IMLS).


title: A Simulation Study of Ultra-relativistic Jets -- II. Structures and Dynamics of FR-II Jets

abstract: We study the structures of ultra-relativistic jets injected into the intracluster medium (ICM) and the associated flow dynamics, such as shocks, velocity shear, and turbulence, through three-dimensional relativistic hydrodynamic (RHD) simulations. To that end, we have developed a high-order accurate RHD code, equipped with a weighted essentially non-oscillatory (WENO) scheme and a realistic equation of state (Seo et al. 2021, Paper I). Using the code, we explore a set of jet models with the parameters relevant to FR-II radio galaxies. We confirm that the overall jet morphology is primarily determined by the jet power, and the jet-to-background density and pressure ratios play secondary roles. Jets with higher powers propagate faster, resulting in more elongated structures, while those with lower powers produce more extended cocoons. Shear interfaces in the jet are dynamically unstable, and hence, chaotic structures with shocks and turbulence develop. We find that the fraction of the jet-injected energy dissipated through shocks and turbulence is greater in less powerful jets, although the actual amount of the dissipated energy is larger in more powerful jets. In lower power jets, the backflow is dominant in the energy dissipation owing to the broad cocoon filled with shocks and turbulence. In higher power jets, by contrast, both the backflow and jet spine flow are important for the energy dissipation. Our results imply that different mechanisms, such as diffusive shock acceleration, shear acceleration, and stochastic turbulent acceleration, may be involved in the production of ultra-high energy cosmic rays in FR-II radio galaxies.


title: A Simulation Study of Ultra-Relativistic Jets -- I. A New Code for Relativistic Hydrodynamics

abstract: In an attempt to investigate the structures of ultra-relativistic jets injected into the intracluster medium (ICM) and the associated flow dynamics, such as shocks, velocity shear, and turbulence, we have developed a new special relativistic hydrodynamic (RHD) code in the Cartesian coordinates, based on the weighted essentially non-oscillatory (WENO) scheme. It is a finite difference scheme of high spatial accuracy, which has been widely employed for solving hyperbolic systems of conservation equations. The code is equipped with different WENO versions, such as the 5th-order accurate WENO-JS (Jiang & Shu 1996), WENO-Z, and WENO-ZA, and different time integration methods, such as the 4th-order accurate Runge-Kutta (RK4) and strong stability preserving RK (SSPRK), as well as the implementation of the equations of state (EOSs) that closely approximate the EOS of the single-component perfect gas in relativistic regime. In addition, it incorporates a high-order accurate averaging of fluxes along the transverse directions to enhance the accuracy of multi-dimensional problems, and a modification of eigenvalues for the acoustic modes to effectively control the carbuncle instability. Through extensive numerical tests, we assess the accuracy and robustness of the code, and choose WENO-Z, SSPRK, and the EOS suggested in Ryu et al. (2006) as the fiducial setup for simulations of ultra-relativistic jets. The results of our study of ultra-relativistic jets using the code is reported in an accompanying paper (Seo et al. 2021, Paper II).


title: Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality

abstract: We release a new benchmark for lexical substitution, the task of finding appropriate substitutes for a target word in a context. To assist humans with writing, lexical substitution systems can suggest words that humans cannot easily think of. However, existing benchmarks depend on human recall as the only source of data, and therefore lack coverage of the substitutes that would be most helpful to humans. Furthermore, annotators often provide substitutes of low quality, which are not actually appropriate in the given context. We collect higher-coverage and higher-quality data by framing lexical substitution as a classification problem, guided by the intuition that it is easier for humans to judge the appropriateness of candidate substitutes than conjure them from memory. To this end, we use a context-free thesaurus to produce candidates and rely on human judgement to determine contextual appropriateness. Compared to the previous largest benchmark, our Swords benchmark has 4.1x more substitutes per target word for the same level of quality, and its substitutes are 1.5x more appropriate (based on human judgement) for the same number of substitutes.


title: Revisiting the variational two-particle reduced density matrix for nuclear systems

abstract: In most nuclear many-body methods, observables are calculated using many-body wave functions explicitly. The variational two-particle reduced density matrix method is one of the few exceptions to the rule. Ground-state energies of both closed-shell and open-shell nuclear systems can indeed be evaluated by minimizing a constrained linear functional of the two-particle reduced density matrix. However, it has virtually never been used in nuclear theory, because nuclear ground states were found to be well overbound, contrary to those of atoms and molecules. Consequently, we introduced new constraints in the nuclear variational two-particle reduced density matrix method, developed recently for atomic and molecular systems. Our calculations then show that this approach can provide a proper description of nuclear systems where only valence neutrons are included. For the nuclear systems where both neutrons and protons are active, however, the energies obtained with the variational two-particle reduced density matrix method are still overbound. The possible reasons for the noticed discrepancies and solutions to this problem will be discussed.


title: Design of Low-Artifact Interpolation Kernels by Means of Computer Algebra

abstract: We present a number of new piecewise-polynomial kernels for image interpolation. The kernels are constructed by optimizing a measure of interpolation quality based on the magnitude of anisotropic artifacts. The kernel design process is performed symbolically using Mathematica computer algebra system. Experimental evaluation involving 14 image quality assessment methods demonstrates that our results compare favorably with the existing linear interpolators.


title: Entropic Independence I: Modified Log-Sobolev Inequalities for Fractionally Log-Concave Distributions and High-Temperature Ising Models

abstract: We introduce a notion called entropic independence that is an entropic analog of spectral notions of high-dimensional expansion. Informally, entropic independence of a background distribution $\mu$ on $k$-sized subsets of a ground set of elements says that for any (possibly randomly chosen) set $S$, the relative entropy of a single element of $S$ drawn uniformly at random carries at most $O(1/k)$ fraction of the relative entropy of $S$. Entropic independence is the analog of the notion of spectral independence, if one replaces variance by entropy. We use entropic independence to derive tight mixing time bounds, overcoming the lossy nature of spectral analysis of Markov chains on exponential-sized state spaces. In our main technical result, we show a general way of deriving entropy contraction, a.k.a. modified log-Sobolev inequalities, for down-up random walks from spectral notions. We show that spectral independence of a distribution under arbitrary external fields automatically implies entropic independence. To derive our results, we relate entropic independence to properties of polynomials: $\mu$ is entropically independent exactly when a transformed version of the generating polynomial of $\mu$ is upper bounded by its linear tangent; this property is implied by concavity of the said transformation, which was shown by prior work to be locally equivalent to spectral independence. We apply our results to obtain tight modified log-Sobolev inequalities and mixing times for multi-step down-up walks on fractionally log-concave distributions. As our flagship application, we establish the tight mixing time of $O(n\log n)$ for Glauber dynamics on Ising models whose interaction matrix has eigenspectrum lying within an interval of length smaller than $1$, improving upon the prior quadratic dependence on $n$.


title: A Regression-based Approach to Robust Estimation and Inference for Genetic Covariance

abstract: Genome-wide association studies (GWAS) have identified thousands of genetic variants associated with complex traits, and some variants are shown to be associated with multiple complex traits. Genetic covariance between two traits is defined as the underlying covariance of genetic effects and can be used to measure the shared genetic architecture. The data used to estimate such a genetic covariance can be from the same group or different groups of individuals, and the traits can be of different types or collected based on different study designs. This paper proposes a unified regression-based approach to robust estimation and inference for genetic covariance of general traits that may be associated with genetic variants nonlinearly. The asymptotic properties of the proposed estimator are provided and are shown to be robust under certain model mis-specification. Our method under linear working models provides a robust inference for the narrow-sense genetic covariance, even when both linear models are mis-specified. Numerical experiments are performed to support the theoretical results. Our method is applied to an outbred mice GWAS data set to study the overlapping genetic effects between the behavioral and physiological phenotypes. The real data results reveal interesting genetic covariance among different mice developmental traits.


title: X-ray burst ignition location on the surface of accreting X-ray pulsars: Can bursts preferentially ignite at the hotspot?

abstract: Hotspots on the surface of accreting neutron stars have been directly observed via pulsations in the lightcurves of X-ray pulsars. They are thought to occur due to magnetic channelling of the accreted fuel to the neutron star magnetic poles. Some X-ray pulsars exhibit burst oscillations during Type I thermonuclear X-ray bursts which are thought to be caused by asymmetries in the burning. In rapidly rotating neutron stars, it has been shown that the lower gravity at the equator can lead to preferential ignition of X-ray bursts at this location. These models, however, do not include the effect of accretion hotspots at the neutron star surface. There are two accreting neutron star sources in which burst oscillations have been observed to track exactly the neutron star spin period. We analyse whether this could be due to the X-ray bursts igniting at the magnetic pole of the neutron star, because of heating in the accreted layers under the hotspot causing ignition conditions to be reached earlier. We investigate heat transport in the accreted layers using a 2D model and study the prevalence of heating down to the ignition depth of X-ray bursts for different hotspot temperatures and sizes. We perform calculations for accretion at the pole and at the equator, and infer that ignition could occur away from the equator at the magnetic pole for hotspots with temperatures greater than $1\times10^8$ K. However, current observations have not identified such high temperatures in accreting X-ray pulsars.


title: Fully Transformer Networks for Semantic Image Segmentation

abstract: Transformers have shown impressive performance in various natural language processing and computer vision tasks, due to the capability of modeling long-range dependencies. Recent progress has demonstrated that combining such Transformers with CNN-based semantic image segmentation models is very promising. However, it is not well studied yet on how well a pure Transformer based approach can achieve for image segmentation. In this work, we explore a novel framework for semantic image segmentation, which is encoder-decoder based Fully Transformer Networks (FTN). Specifically, we first propose a Pyramid Group Transformer (PGT) as the encoder for progressively learning hierarchical features, meanwhile reducing the computation complexity of the standard Visual Transformer (ViT). Then, we propose a Feature Pyramid Transformer (FPT) to fuse semantic-level and spatial-level information from multiple levels of the PGT encoder for semantic image segmentation. Surprisingly, this simple baseline can achieve better results on multiple challenging semantic segmentation and face parsing benchmarks, including PASCAL Context, ADE20K, COCOStuff, and CelebAMask-HQ. The source code will be released on https://github.com/BR-IDL/PaddleViT.


title: Some estimations of the {\L}ojasiewicz exponent

abstract: We strengthen some estimations of the local and global {\L}ojasiewicz exponent for polynomial mappings on closed semialgebraic sets obtained by K.Kurdyka, S.Spodzieja and A.Szlachci\'nska.


title: A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNs

abstract: Deep neural networks (DNNs) in the infinite width/channel limit have received much attention recently, as they provide a clear analytical window to deep learning via mappings to Gaussian Processes (GPs). Despite its theoretical appeal, this viewpoint lacks a crucial ingredient of deep learning in finite DNNs, laying at the heart of their success -- feature learning. Here we consider DNNs trained with noisy gradient descent on a large training set and derive a self consistent Gaussian Process theory accounting for strong finite-DNN and feature learning effects. Applying this to a toy model of a two-layer linear convolutional neural network (CNN) shows good agreement with experiments. We further identify, both analytical and numerically, a sharp transition between a feature learning regime and a lazy learning regime in this model. Strong finite-DNN effects are also derived for a non-linear two-layer fully connected network. Our self consistent theory provides a rich and versatile analytical framework for studying feature learning and other non-lazy effects in finite DNNs.


title: TIMES I: a Systematic Observation in Multiple Molecular Lines Toward the Orion A and Ophiuchus Clouds

abstract: We have used the Taeduk Radio Astronomy Observatory to observe the Orion A and Ophiuchus clouds in the $J=$1$-$0 lines of $^{13}$CO, C$^{18}$O, HCN, HCO$^+$, and N$_2$H$^+$ and the $J=$2$-$1 line of CS. The fully sampled maps with uniform noise levels are used to create moment maps. The variations of the line intensity and velocity dispersion with total column density, derived from dust emission maps, are presented and compared to previous work. The CS line traces dust column density over more than one order of magnitude, and the N$_2$H$^+$ line best traces the highest column density regime ($\log(N_\mathrm{H_2}$) $>$ 22.8). Line luminosities, integrated over the cloud, are compared to those seen in other galaxies. The HCO$^+$-to-HCN luminosity ratio in the Orion A cloud is similar to that of starburst galaxies, while that in the Ophiuchus cloud is in between those of active galactic nuclei and starburst galaxies.


title: Harnessing Unrecognizable Faces for Improving Face Recognition

abstract: The common implementation of face recognition systems as a cascade of a detection stage and a recognition or verification stage can cause problems beyond failures of the detector. When the detector succeeds, it can detect faces that cannot be recognized, no matter how capable the recognition system. Recognizability, a latent variable, should therefore be factored into the design and implementation of face recognition systems. We propose a measure of recognizability of a face image that leverages a key empirical observation: an embedding of face images, implemented by a deep neural network trained using mostly recognizable identities, induces a partition of the hypersphere whereby unrecognizable identities cluster together. This occurs regardless of the phenomenon that causes a face to be unrecognizable, it be optical or motion blur, partial occlusion, spatial quantization, poor illumination. Therefore, we use the distance from such an "unrecognizable identity" as a measure of recognizability, and incorporate it in the design of the over-all system. We show that accounting for recognizability reduces error rate of single-image face recognition by 58% at FAR=1e-5 on the IJB-C Covariate Verification benchmark, and reduces verification error rate by 24% at FAR=1e-5 in set-based recognition on the IJB-C benchmark.


title: Self-supervised Graph-level Representation Learning with Local and Global Structure

abstract: This paper studies unsupervised/self-supervised whole-graph representation learning, which is critical in many tasks such as molecule properties prediction in drug and material discovery. Existing methods mainly focus on preserving the local similarity structure between different graph instances but fail to discover the global semantic structure of the entire data set. In this paper, we propose a unified framework called Local-instance and Global-semantic Learning (GraphLoG) for self-supervised whole-graph representation learning. Specifically, besides preserving the local similarities, GraphLoG introduces the hierarchical prototypes to capture the global semantic clusters. An efficient online expectation-maximization (EM) algorithm is further developed for learning the model. We evaluate GraphLoG by pre-training it on massive unlabeled graphs followed by fine-tuning on downstream tasks. Extensive experiments on both chemical and biological benchmark data sets demonstrate the effectiveness of the proposed approach.


title: Theoretically Motivated Data Augmentation and Regularization for Portfolio Construction

abstract: The task we consider is portfolio construction in a speculative market, a fundamental problem in modern finance. While various empirical works now exist to explore deep learning in finance, the theory side is almost non-existent. In this work, we focus on developing a theoretical framework for understanding the use of data augmentation for deep-learning-based approaches to quantitative finance. The proposed theory clarifies the role and necessity of data augmentation for finance; moreover, our theory implies that a simple algorithm of injecting a random noise of strength $\sqrt{|r_{t-1}|}$ to the observed return $r_{t}$ is better than not injecting any noise and a few other financially irrelevant data augmentation techniques.


title: Measurement and Modeling of Proton-Induced Reactions on Arsenic from 35 to 200 MeV

abstract: $^{72}$As is a promising positron emitter for diagnostic imaging that can be employed locally using a $^{72}$Se generator. However, current reaction pathways to $^{72}$Se have insufficient nuclear data for efficient production using regional 100-200 MeV high-intensity proton accelerators. In order to address this deficiency, stacked-target irradiations were performed at LBNL, LANL, and BNL to measure the production of the $^{72}$Se/$^{72}$As PET generator system via $^{75}$As(p,x) between 35 and 200 MeV. This work provides the most well-characterized excitation function for $^{75}$As(p,4n)$^{72}$Se starting from threshold. Additional focus was given to report the first measurements of $^{75}$As(p,x)$^{68}$Ge and bolster an already robust production capability for the highly valuable $^{68}$Ge/$^{68}$Ga PET generator. Thick target yield comparisons with prior established formation routes to both generators are made. In total, high-energy proton-induced cross sections are reported for 55 measured residual products from $^{75}$As, Cu, and Ti targets, where the latter two materials were present as monitor foils. These results were compared with literature data as well as the default theoretical calculations of the nuclear model codes TALYS, CoH, EMPIRE, and ALICE. Reaction modeling at these energies is typically unsatisfactory due to few prior published data and many interacting physics models. Therefore, a detailed assessment of the TALYS code was performed with simultaneous parameter adjustments applied according to a standardized procedure. Particular attention was paid to the formulation of the two-component exciton model in the transition between the compound and pre-equilibrium regions, with a linked investigation of level density models for nuclei off of stability and their impact on modeling predictive power.


title: Discrete-to-Continuous Extensions: piecewise multilinear extension, min-max theory and spectral theory

abstract: We introduce the homogeneous and piecewise multilinear extensions and the eigenvalue problem for locally Lipschitz function pairs, in order to develop a systematic framework for relating discrete and continuous min-max problems. This also enables us to investigate spectral properties for pairs of $p$-homogeneous functions and to propose a critical point theory for zero-homogeneous functions. The main contributions are: (1) We provide several min-max relations between an original discrete formulation and its piecewise multilinear extension. We introduce the concept of perfect domain pairs to view comonotonicity on vectors as an extension of inclusion chains on sets. The piecewise multi-linear extension is (slice-)rank preserving, which closely relates to Tao's lemma on diagonal tensors. More discrete-to-continuous equalities are obtained, including a general form involving log-concave polynomials. And by employing these fundamental correspondences, we get further results and applications on tensors, Tur\'an's problem, signed (hyper-)graphs, etc. (2) We derive the mountain pass characterization, linking theorems, nodal domain inequalities, inertia bounds, duality theorems and distribution of eigenvalues for pairs of $p$-homogeneous functions. We establish a new property on the subderivative of a convex function which relates to the Gauss map of the graph of the convex function. Based on these fundamental results, we can analyze the structure of eigenspaces in depth. For example, we show a simple one-to-one correspondence between the nonzero eigenvalues of the vertex p-Laplacian and the edge $p^*$-Laplacian of a graph. We also apply the theory to Cheeger inequalities and $p$-Laplacians on oriented hypergraphs and simplicial complexes. Also, the first nonlinear analog of Huang's approach for hypergraphs is provided.


title: The best of both worlds: stochastic and adversarial episodic MDPs with unknown transition

abstract: We consider the best-of-both-worlds problem for learning an episodic Markov Decision Process through $T$ episodes, with the goal of achieving $\widetilde{\mathcal{O}}(\sqrt{T})$ regret when the losses are adversarial and simultaneously $\mathcal{O}(\text{polylog}(T))$ regret when the losses are (almost) stochastic. Recent work by [Jin and Luo, 2020] achieves this goal when the fixed transition is known, and leaves the case of unknown transition as a major open question. In this work, we resolve this open problem by using the same Follow-the-Regularized-Leader ($\text{FTRL}$) framework together with a set of new techniques. Specifically, we first propose a loss-shifting trick in the $\text{FTRL}$ analysis, which greatly simplifies the approach of [Jin and Luo, 2020] and already improves their results for the known transition case. Then, we extend this idea to the unknown transition case and develop a novel analysis which upper bounds the transition estimation error by (a fraction of) the regret itself in the stochastic setting, a key property to ensure $\mathcal{O}(\text{polylog}(T))$ regret.


title: Searching for consistent associations with a multi-environment knockoff filter

abstract: This paper develops a method based on model-X knockoffs to find conditional associations that are consistent across diverse environments, controlling the false discovery rate. The motivation for this problem is that large data sets may contain numerous associations that are statistically significant and yet misleading, as they are induced by confounders or sampling imperfections. However, associations consistently replicated under different conditions may be more interesting. In fact, consistency sometimes provably leads to valid causal inferences even if conditional associations do not. While the proposed method is flexible and can be deployed in a wide range of applications, this paper highlights its relevance to genome-wide association studies, in which consistency across populations with diverse ancestries mitigates confounding due to unmeasured variants. The effectiveness of this approach is demonstrated by simulations and applications to the UK Biobank data.


title: LaserShark: Establishing Fast, Bidirectional Communication into Air-Gapped Systems

abstract: Physical isolation, so called air-gapping, is an effective method for protecting security-critical computers and networks. While it might be possible to introduce malicious code through the supply chain, insider attacks, or social engineering, communicating with the outside world is prevented. Different approaches to breach this essential line of defense have been developed based on electromagnetic, acoustic, and optical communication channels. However, all of these approaches are limited in either data rate or distance, and frequently offer only exfiltration of data. We present a novel approach to infiltrate data to and exfiltrate data from air-gapped systems without any additional hardware on-site. By aiming lasers at already built-in LEDs and recording their response, we are the first to enable a long-distance (25m), bidirectional, and fast (18.2kbps in & 100kbps out) covert communication channel. The approach can be used against any office device that operates LEDs at the CPU's GPIO interface.


title: Throughput Analysis of UAV-assisted CellularNetworks by Mat\'{e}rn Hardcore Point Process

abstract: Unmanned aerial vehicles (UAVs) are expected to coexist with conventional terrestrial cellular networks and become an important component to support high rate transmissions. This paper presents an analytical framework for evaluating the throughput performance of a downlink two-tier heterogeneous network. Considering the minimum distance constraint among UAVs, Mat\'{e}rn hardcore point process (MHP) is utilized to model the locations of UAVs. The locations of terrestrial base stations (BSs) are modeled by Poisson point process (PPP). Tools of stochastic geometry are invoked to derive tractable expressions for average data rates of users. With the analytical results, we discuss the optimal combinations of UAVs' height and power control factor. The result shows that an appropriate power control factor can effectively maximize UAV users' average data rate as well as guaranteeing the BS users' performance under our proposed model.


title: Multi-dataset Pretraining: A Unified Model for Semantic Segmentation

abstract: Collecting annotated data for semantic segmentation is time-consuming and hard to scale up. In this paper, we for the first time propose a unified framework, termed as Multi-Dataset Pretraining, to take full advantage of the fragmented annotations of different datasets. The highlight is that the annotations from different domains can be efficiently reused and consistently boost performance for each specific domain. This is achieved by first pretraining the network via the proposed pixel-to-prototype contrastive loss over multiple datasets regardless of their taxonomy labels, and followed by fine-tuning the pretrained model over specific dataset as usual. In order to better model the relationship among images and classes from different datasets, we extend the pixel level embeddings via cross dataset mixing and propose a pixel-to-class sparse coding strategy that explicitly models the pixel-class similarity over the manifold embedding space. In this way, we are able to increase intra-class compactness and inter-class separability, as well as considering inter-class similarity across different datasets for better transferability. Experiments conducted on several benchmarks demonstrate its superior performance. Notably, MDP consistently outperforms the pretrained models over ImageNet by a considerable margin, while only using less than 10% samples for pretraining.


title: CloudChain: A Cloud Blockchain Using Shared Memory Consensus and RDMA

abstract: Blockchain technologies can enable secure computing environments among mistrusting parties. Permissioned blockchains are particularly enlightened by companies, enterprises, and government agencies due to their efficiency, customizability, and governance-friendly features. Obviously, seamlessly fusing blockchain and cloud computing can significantly benefit permissioned blockchains; nevertheless, most blockchains implemented on clouds are originally designed for loosely-coupled networks where nodes communicate asynchronously, failing to take advantages of the closely-coupled nature of cloud servers. In this paper, we propose an innovative cloud-oriented blockchain -- CloudChain, which is a modularized three-layer system composed of the network layer, consensus layer, and blockchain layer. CloudChain is based on a shared-memory model where nodes communicate synchronously by direct memory accesses. We realize the shared-memory model with the Remote Direct Memory Access technology, based on which we propose a shared-memory consensus algorithm to ensure presistence and liveness, the two crucial blockchain security properties countering Byzantine nodes. We also implement a CloudChain prototype based on a RoCEv2-based testbed to experimentally validate our design, and the results verify the feasibility and efficiency of CloudChain.


title: Embodied negation and levels of concreteness: A TMS Study on German and Italian language processing

abstract: According to the embodied cognition perspective, linguistic negation may block the motor simulations induced by language processing. Transcranial magnetic stimulation (TMS) was applied to the left primary motor cortex (hand area) of monolingual Italian and German healthy participants during a rapid serial visual presentation of sentences from their own language. In these languages, the negative particle is located at the beginning and at the end of the sentence, respectively. The study investigated whether the interruption of the motor simulation processes, accounted for by reduced motor evoked potentials (MEPs), takes place similarly in two languages differing on the position of the negative marker. Different levels of sentence concreteness were also manipulated to investigate if negation exerts generalized effects or if it is affected by the semantic features of the sentence. Our findings indicate that negation acts as a block on motor representations, but independently from the language and words concreteness level.


title: Dark matter and low scale leptogenesis in a flavor symmetric neutrino two Higgs doublet model($\nu$2HDM)

abstract: We have studied dark matter (DM) phenomenology, neutrinoless double beta decay (NDBD) and realized low scale leptogenesis in an extension of Standard Model(SM) with three neutral fermions, a scalar doublet and a dark sector incorporating a singlet scalar and a Dirac singlet fermion. A generic model based on $A_4\times Z_4$ flavor symmetry including five flavons is used to explain both normal and inverted hierarchy mass patterns of neutrino and also to accommodate the dark matter mass. In this extension of the $\nu$2HDM, the effective neutrino mass observed in $0\nu\beta\beta$ is well within the experimental limit provided by KamLAND-ZEN. In order to validate DM within this model, we have checked relic abundance and free streaming length of the dark sector component, i.e. a Dirac singlet fermion constraining its mass in keV range. More importantly we have also realised low scale leptogenesis simultaneously within this framework and also the Dirac CP phase gets constrained with the results. Bound from LFV is also incorporated in order to constrain the Yukawa couplings. More importantly, we have analyzed the dependence of various phenomenology with decay parameter for different choice of arbitrary complex angles.


title: On the uniqueness theorems for transmissions problems related to models of elasticity, diffusion and electrocardiography

abstract: We consider a generalization of the inverse problem of the electrocardiography in the framework of the theory of elliptic and parabolic differential operators. More precisely, starting with the standard bidomain mathematical model related to the problem of the reconstruction of the transmembrane potential in the myocardium from known body surface potentials we formulate a more general transmission problem for elliptic and parabolic equations in the Sobolev type spaces and describe conditions, providing uniqueness theorems for its solutions. Next, the new transmission problem is interpreted in the framework of the elasticity theory applied to composite media. Finally, we prove a uniqueness theorem for an evolutionary transmission problem that can be easily adopted to many models involving the diffusion type equations.


title: Fractional Schr\"odinger equations with singular potentials of higher order. II: Hypoelliptic case

abstract: In this paper we consider the space-fractional Schr\"odinger equation with a singular potential for a wide class of fractional hypoelliptic operators. Such analysis can be conveniently realised in the setting of graded Lie groups. The paper is a continuation and extension of a previous one where the classical Schr\"odinger equation on $\mathbb R^n$ with singular potentials was considered.


title: Left Ventricle Contouring in Cardiac Images Based on Deep Reinforcement Learning

abstract: Medical image segmentation is one of the important tasks of computer-aided diagnosis in medical image analysis. Since most medical images have the characteristics of blurred boundaries and uneven intensity distribution, through existing segmentation methods, the discontinuity within the target area and the discontinuity of the target boundary are likely to lead to rough or even erroneous boundary delineation. In this paper, we propose a new iterative refined interactive segmentation method for medical images based on agent reinforcement learning, which focuses on the problem of target segmentation boundaries. We model the dynamic process of drawing the target contour in a certain order as a Markov Decision Process (MDP) based on a deep reinforcement learning method. In the dynamic process of continuous interaction between the agent and the image, the agent tracks the boundary point by point in order within a limited length range until the contour of the target is completely drawn. In this process, the agent can quickly improve the segmentation performance by exploring an interactive policy in the image. The method we proposed is simple and effective. At the same time, we evaluate our method on the cardiac MRI scan data set. Experimental results show that our method has a better segmentation effect on the left ventricle in a small number of medical image data sets, especially in terms of segmentation boundaries, this method is better than existing methods. Based on our proposed method, the dynamic generation process of the predicted contour trajectory of the left ventricle will be displayed online at https://github.com/H1997ym/LV-contour-trajectory.


title: Conversational Fashion Image Retrieval via Multiturn Natural Language Feedback

abstract: We study the task of conversational fashion image retrieval via multiturn natural language feedback. Most previous studies are based on single-turn settings. Existing models on multiturn conversational fashion image retrieval have limitations, such as employing traditional models, and leading to ineffective performance. We propose a novel framework that can effectively handle conversational fashion image retrieval with multiturn natural language feedback texts. One characteristic of the framework is that it searches for candidate images based on exploitation of the encoded reference image and feedback text information together with the conversation history. Furthermore, the image fashion attribute information is leveraged via a mutual attention strategy. Since there is no existing fashion dataset suitable for the multiturn setting of our task, we derive a large-scale multiturn fashion dataset via additional manual annotation efforts on an existing single-turn dataset. The experiments show that our proposed model significantly outperforms existing state-of-the-art methods.


title: Personalized PercepNet: Real-time, Low-complexity Target Voice Separation and Enhancement

abstract: The presence of multiple talkers in the surrounding environment poses a difficult challenge for real-time speech communication systems considering the constraints on network size and complexity. In this paper, we present Personalized PercepNet, a real-time speech enhancement model that separates a target speaker from a noisy multi-talker mixture without compromising on complexity of the recently proposed PercepNet. To enable speaker-dependent speech enhancement, we first show how we can train a perceptually motivated speaker embedder network to produce a representative embedding vector for the given speaker. Personalized PercepNet uses the target speaker embedding as additional information to pick out and enhance only the target speaker while suppressing all other competing sounds. Our experiments show that the proposed model significantly outperforms PercepNet and other baselines, both in terms of objective speech enhancement metrics and human opinion scores.


title: EnMcGAN: Adversarial Ensemble Learning for 3D Complete Renal Structures Segmentation

abstract: 3D complete renal structures(CRS) segmentation targets on segmenting the kidneys, tumors, renal arteries and veins in one inference. Once successful, it will provide preoperative plans and intraoperative guidance for laparoscopic partial nephrectomy(LPN), playing a key role in the renal cancer treatment. However, no success has been reported in 3D CRS segmentation due to the complex shapes of renal structures, low contrast and large anatomical variation. In this study, we utilize the adversarial ensemble learning and propose Ensemble Multi-condition GAN(EnMcGAN) for 3D CRS segmentation for the first time. Its contribution is three-fold. 1)Inspired by windowing, we propose the multi-windowing committee which divides CTA image into multiple narrow windows with different window centers and widths enhancing the contrast for salient boundaries and soft tissues. And then, it builds an ensemble segmentation model on these narrow windows to fuse the segmentation superiorities and improve whole segmentation quality. 2)We propose the multi-condition GAN which equips the segmentation model with multiple discriminators to encourage the segmented structures meeting their real shape conditions, thus improving the shape feature extraction ability. 3)We propose the adversarial weighted ensemble module which uses the trained discriminators to evaluate the quality of segmented structures, and normalizes these evaluation scores for the ensemble weights directed at the input image, thus enhancing the ensemble results. 122 patients are enrolled in this study and the mean Dice coefficient of the renal structures achieves 84.6%. Extensive experiments with promising results on renal structures reveal powerful segmentation accuracy and great clinical significance in renal cancer treatment.


title: Soliton generation and drift wave turbulence spreading via geodesic acoustic mode excitation

abstract: The two-field equations governing fully nonlinear dynamics of the drift wave (DW) and geodesic acoustic mode (GAM) in the toroidal geometry are derived in nonlinear gyrokinetic framework. Two stages with distinctive features are identified and analyzed. In the linear growth stage, the set of nonlinear equations can be reduced to the intensively studied parametric decay instability (PDI), accounting for the spontaneous resonant excitation of GAM by DW. The main results of previous works on spontaneous GAM excitation, e.g., the much enhanced GAM group velocity and the nonlinear growth rate of GAM, are reproduced from numerical solution of the two-field equations. In the fully nonlinear stage, soliton structures are observed to form due to the balancing of the self-trapping effect by the spontaneously excited GAM and kinetic dispersiveness of DW. The soliton structures enhance turbulence spreading from DW linearly unstable to stable region, exhibiting convective propagation instead of typical linear dispersive process, and is thus, expected to induce core-edge interaction and nonlocal transport.


title: Galois correspondence for augmented monads

abstract: We establish a Galois connection between sub-monads of an augmented monad and sub-functors of the forgetful functor from its Eilenberg-Moore category. This connection is given in terms of invariants and stabilizers defined through universal properties. An explicit procedure for the computation of invariants is given assuming the existence of suitable right adjoints. Additionally, in the context of monoidal closed categories, a characterization of stabilizers is made in terms of Tannakian reconstruction.


title: Efficient Speech Emotion Recognition Using Multi-Scale CNN and Attention

abstract: Emotion recognition from speech is a challenging task. Re-cent advances in deep learning have led bi-directional recur-rent neural network (Bi-RNN) and attention mechanism as astandard method for speech emotion recognition, extractingand attending multi-modal features - audio and text, and thenfusing them for downstream emotion classification tasks. Inthis paper, we propose a simple yet efficient neural networkarchitecture to exploit both acoustic and lexical informationfrom speech. The proposed framework using multi-scale con-volutional layers (MSCNN) to obtain both audio and text hid-den representations. Then, a statistical pooling unit (SPU)is used to further extract the features in each modality. Be-sides, an attention module can be built on top of the MSCNN-SPU (audio) and MSCNN (text) to further improve the perfor-mance. Extensive experiments show that the proposed modeloutperforms previous state-of-the-art methods on IEMOCAPdataset with four emotion categories (i.e., angry, happy, sadand neutral) in both weighted accuracy (WA) and unweightedaccuracy (UA), with an improvement of 5.0% and 5.2% respectively under the ASR setting.


title: Cheap and Good? Simple and Effective Data Augmentation for Low Resource Machine Reading

abstract: We propose a simple and effective strategy for data augmentation for low-resource machine reading comprehension (MRC). Our approach first pretrains the answer extraction components of a MRC system on the augmented data that contains approximate context of the correct answers, before training it on the exact answer spans. The approximate context helps the QA method components in narrowing the location of the answers. We demonstrate that our simple strategy substantially improves both document retrieval and answer extraction performance by providing larger context of the answers and additional training data. In particular, our method significantly improves the performance of BERT based retriever (15.12\%), and answer extractor (4.33\% F1) on TechQA, a complex, low-resource MRC task. Further, our data augmentation strategy yields significant improvements of up to 3.9\% exact match (EM) and 2.7\% F1 for answer extraction on PolicyQA, another practical but moderate sized QA dataset that also contains long answer spans.


title: Regularity of the free boundary for a parabolic cooperative system

abstract: In this paper we study the following parabolic system \begin{equation*} \Delta \u -\partial_t \u =|\u|^{q-1}\u\,\chi_{\{ |\u|>0 \}}, \qquad \u = (u^1, \cdots , u^m) \ , \end{equation*} with free boundary $\partial \{|\u | >0\}$. For $0\leq q<1$, we prove optimal growth rate for solutions $\u $ to the above system near free boundary points, and show that in a uniform neighbourhood of any a priori well-behaved free boundary point the free boundary is $C^{1, \alpha}$ in space directions and half-Lipschitz in the time direction.


title: Unrestricted Quantum Moduli Algebras, II: Noetherianity and Simple Fraction Rings at Roots of 1

abstract: We prove that the quantum graph algebra and the quantum moduli algebra associated to a punctured sphere and complex semisimple Lie algebra $\mathfrak{g}$ are Noetherian rings and finitely generated rings over $\mathbb{C}(q)$. Moreover, we show that these two properties still hold on $\mathbb{C}\big[q,q^{-1}\big]$ for the integral version of the quantum graph algebra. We also study the specializations $\mathcal{L}_{0,n}^\epsilon$ of the quantum graph algebra at a root of unity $\epsilon$ of odd order, and show that $\mathcal{L}_{0,n}^\epsilon$ and its invariant algebra under the quantum group $U_\epsilon(\mathfrak{g})$ have classical fraction algebras which are central simple algebras of PI degrees that we compute.


title: Theoretical study of reflection spectroscopy for superconducting quantum parametrons

abstract: Superconducting parametrons in the single-photon Kerr regime, also called KPOs, have been attracting increasing attention in terms of their applications to quantum annealing and universal quantum computation. It is of practical importance to obtain information of superconducting parametrons operating under an oscillating pump field. Spectroscopy can provide information of a superconducting parametron under examination, such as energy level structure, and also useful information for calibration of the pump field. We theoretically study the reflection spectroscopy of superconducting parametrons, and develop a method to obtain the reflection coefficient. We present formulae of the reflection coefficient, the nominal external and the internal decay rates, and examine the obtained spectra. It is shown that the difference of the populations of energy levels manifests itself as a dip or peak in the amplitude of the reflection coefficient, and one can directly extract the coupling strength between the energy levels by measuring the nominal decay rates when the pump field is sufficiently large.


title: Interaction-free imaging of multi-pixel objects

abstract: Quantum imaging, one of the pillars of quantum technologies, is well-suited to study sensitive samples which require low-light conditions, like biological tissues. In this context, interaction-free measurements (IFM) allow us infer the presence of an opaque object without the photon interacting with the sample. Current IFM schemes are designed for single-pixel objects, while real-life samples are structured, multi-pixel objects. Here we extend the IFM imaging schemes to multi-pixel, semi-transparent objects, by encoding the information about the pixels into an internal degree of freedom, namely orbital angular momentum (OAM). This allows us to image the pixels in parallel. Our solution exhibits a better theoretical efficiency than the single-pixel case. Our scheme can be extended to other degrees of freedom, like the photon radial quantum number, in order to image 1D and 2D objects.


title: Image Deformation Estimation via Multi-Objective Optimization

abstract: The free-form deformation model can represent a wide range of non-rigid deformations by manipulating a control point lattice over the image. However, due to a large number of parameters, it is challenging to fit the free-form deformation model directly to the deformed image for deformation estimation because of the complexity of the fitness landscape. In this paper, we cast the registration task as a multi-objective optimization problem (MOP) according to the fact that regions affected by each control point overlap with each other. Specifically, by partitioning the template image into several regions and measuring the similarity of each region independently, multiple objectives are built and deformation estimation can thus be realized by solving the MOP with off-the-shelf multi-objective evolutionary algorithms (MOEAs). In addition, a coarse-to-fine strategy is realized by image pyramid combined with control point mesh subdivision. Specifically, the optimized candidate solutions of the current image level are inherited by the next level, which increases the ability to deal with large deformation. Also, a post-processing procedure is proposed to generate a single output utilizing the Pareto optimal solutions. Comparative experiments on both synthetic and real-world images show the effectiveness and usefulness of our deformation estimation method.


title: Broadcasted Residual Learning for Efficient Keyword Spotting

abstract: Keyword spotting is an important research field because it plays a key role in device wake-up and user interaction on smart devices. However, it is challenging to minimize errors while operating efficiently in devices with limited resources such as mobile phones. We present a broadcasted residual learning method to achieve high accuracy with small model size and computational load. Our method configures most of the residual functions as 1D temporal convolution while still allows 2D convolution together using a broadcasted-residual connection that expands temporal output to frequency-temporal dimension. This residual mapping enables the network to effectively represent useful audio features with much less computation than conventional convolutional neural networks. We also propose a novel network architecture, Broadcasting-residual network (BC-ResNet), based on broadcasted residual learning and describe how to scale up the model according to the target device's resources. BC-ResNets achieve state-of-the-art 98.0% and 98.7% top-1 accuracy on Google speech command datasets v1 and v2, respectively, and consistently outperform previous approaches, using fewer computations and parameters. Code is available at https://github.com/Qualcomm-AI-research/bcresnet.


title: Maximal momentum GUP leads to quadratic gravity

abstract: Quantum theories of gravity predict interesting phenomenological features such as a minimum measurable length and maximum momentum. We use the Generalized Uncertainty Principle (GUP), which is an extension of the standard Heisenberg Uncertainty Principle motivated by Quantum Gravity, to model the above features. In particular, we use a GUP with modelling maximum momentum to establish a correspondence between the GUP-modified dynamics of a massless spin-2 field and quadratic (referred to as Stelle) gravity. In other words, Stelle gravity can be regarded as the classical manifestation of a maximum momentum and the related GUP. We explore the applications of Stelle gravity to cosmology and specifically show that Stelle gravity applied to a homogeneous and isotropic background leads to inflation with an exit. Using the above, we obtain strong bounds on the GUP parameter from CMB observations. Unlike previous works, which fixed only upper bounds for GUP parameters, we obtain both \emph{lower and upper bounds} on the GUP parameter.


title: Efficient Estimation For The Joint Model of Survival and Longitudinal Data

abstract: In survival studies it is important to record the values of key longitudinal covariates until the occurrence of event of a subject. For this reason, it is essential to study the association between longitudinal and time-to-event outcomes using the joint model. In this paper, profile likelihood approach has been used to estimate the cumulative hazard and regression parameters from the joint model of survival and longitudinal data. Moreover, we show the asymptotic normality of the profile likelihood estimator via asymptotic expansion of the profile likelihood and obtain the explicit form of the variance estimator. The estimators of the regression parameters from the joint model are shown to be semi-parametric efficient.


title: Probing Magnetic Moment Operators in $H \gamma$ Production and $H \to \tau^+ \tau^- \gamma$ Rare Decay

abstract: The magnetic moment ($a_\gamma$) and weak magnetic moment ($a_W$) of charged leptons and quarks are sensitive to quantum effects of new physics heavy resonances. In effective field theory $a_\gamma$ and $a_W$ are induced by two independent operators, therefore, one has to measure both the $a_\gamma$ and $a_W$ to shed lights on new physics. The $a_W$'s of the SM fermions are measured at the LEP. In this work, we analyze the contributions from magnetic and weak magnetic moment operators in the processes of $pp\to H \gamma$ and $gg\to H \to \tau^+ \tau^- \gamma$ at the High-Luminosity Large Hadron Collider. We demonstrate that the two processes could cover most of the parameter space that cannot be probed at the LEP.


title: Adversarial Semantic Hallucination for Domain Generalized Semantic Segmentation

abstract: Convolutional neural networks typically perform poorly when the test (target domain) and training (source domain) data have significantly different distributions. While this problem can be mitigated by using the target domain data to align the source and target domain feature representations, the target domain data may be unavailable due to privacy concerns. Consequently, there is a need for methods that generalize well despite restricted access to target domain data during training. In this work, we propose an adversarial semantic hallucination approach (ASH), which combines a class-conditioned hallucination module and a semantic segmentation module. Since the segmentation performance varies across different classes, we design a semantic-conditioned style hallucination module to generate affine transformation parameters from semantic information in the segmentation probability maps of the source domain image. Unlike previous adaptation approaches, which treat all classes equally, ASH considers the class-wise differences. The segmentation module and the hallucination module compete adversarially, with the hallucination module generating increasingly "difficult" stylized images to challenge the segmentation module. In response, the segmentation module improves as it is trained with generated samples at an appropriate class-wise difficulty level. Our results on the Cityscapes and Mapillary benchmark datasets show that our method is competitive with state of the art work. Code is made available at https://github.com/gabriel-tjio/ASH.


title: Unbalanced Optimal Transport through Non-negative Penalized Linear Regression

abstract: This paper addresses the problem of Unbalanced Optimal Transport (UOT) in which the marginal conditions are relaxed (using weighted penalties in lieu of equality) and no additional regularization is enforced on the OT plan. In this context, we show that the corresponding optimization problem can be reformulated as a non-negative penalized linear regression problem. This reformulation allows us to propose novel algorithms inspired from inverse problems and nonnegative matrix factorization. In particular, we consider majorization-minimization which leads in our setting to efficient multiplicative updates for a variety of penalties. Furthermore, we derive for the first time an efficient algorithm to compute the regularization path of UOT with quadratic penalties. The proposed algorithm provides a continuity of piece-wise linear OT plans converging to the solution of balanced OT (corresponding to infinite penalty weights). We perform several numerical experiments on simulated and real data illustrating the new algorithms, and provide a detailed discussion about more sophisticated optimization tools that can further be used to solve OT problems thanks to our reformulation.


title: Risk Ranked Recall: Collision Safety Metric for Object Detection Systems in Autonomous Vehicles

abstract: Commonly used metrics for evaluation of object detection systems (precision, recall, mAP) do not give complete information about their suitability of use in safety critical tasks, like obstacle detection for collision avoidance in Autonomous Vehicles (AV). This work introduces the Risk Ranked Recall ($R^3$) metrics for object detection systems. The $R^3$ metrics categorize objects within three ranks. Ranks are assigned based on an objective cyber-physical model for the risk of collision. Recall is measured for each rank.


title: Importance of dxy orbital and electron correlation in iron-based superconductors revealed by phase diagram for 1111-system

abstract: The structural flexibility at three substitution sites in LaFeAsO enabled investigation of the relation between superconductivity and structural parameters over a wide range of crystal compositions. Substitutions of Nd for La, Sb or P for As, and F or H for O were performed. All these substitutions modify the local structural parameters, while the F/H-substitution also changes band filling. It was found that the superconducting transition temperature $T_{c}$ is strongly affected by the pnictogen height $h_{Pn}$ from the Fe-plane that controls the electron correlation strength and the size of the $d_{xy}$ hole Fermi surface (FS). With increasing $h_{Pn}$, weak coupling superconductivity switches to the strong coupling one where the $d_{xy}$ hole FS is crucially important.


title: RECOWNs: Probabilistic Circuits for Trustworthy Time Series Forecasting

abstract: Time series forecasting is a relevant task that is performed in several real-world scenarios such as product sales analysis and prediction of energy demand. Given their accuracy performance, currently, Recurrent Neural Networks (RNNs) are the models of choice for this task. Despite their success in time series forecasting, less attention has been paid to make the RNNs trustworthy. For example, RNNs can not naturally provide an uncertainty measure to their predictions. This could be extremely useful in practice in several cases e.g. to detect when a prediction might be completely wrong due to an unusual pattern in the time series. Whittle Sum-Product Networks (WSPNs), prominent deep tractable probabilistic circuits (PCs) for time series, can assist an RNN with providing meaningful probabilities as uncertainty measure. With this aim, we propose RECOWN, a novel architecture that employs RNNs and a discriminant variant of WSPNs called Conditional WSPNs (CWSPNs). We also formulate a Log-Likelihood Ratio Score as better estimation of uncertainty that is tailored to time series and Whittle likelihoods. In our experiments, we show that RECOWNs are accurate and trustworthy time series predictors, able to "know when they do not know".


title: To Smooth or Not? When Label Smoothing Meets Noisy Labels

abstract: Label smoothing (LS) is an arising learning paradigm that uses the positively weighted average of both the hard training labels and uniformly distributed soft labels. It was shown that LS serves as a regularizer for training data with hard labels and therefore improves the generalization of the model. Later it was reported LS even helps with improving robustness when learning with noisy labels. However, we observed that the advantage of LS vanishes when we operate in a high label noise regime. Intuitively speaking, this is due to the increased entropy of $\mathbb{P}(\text{noisy label}|X)$ when the noise rate is high, in which case, further applying LS tends to "over-smooth" the estimated posterior. We proceeded to discover that several learning-with-noisy-labels solutions in the literature instead relate more closely to negative/not label smoothing (NLS), which acts counter to LS and defines as using a negative weight to combine the hard and soft labels! We provide understandings for the properties of LS and NLS when learning with noisy labels. Among other established properties, we theoretically show NLS is considered more beneficial when the label noise rates are high. We provide extensive experimental results on multiple benchmarks to support our findings too. Code is publicly available at https://github.com/UCSC-REAL/negative-label-smoothing.


title: Few-Shot Action Localization without Knowing Boundaries

abstract: Learning to localize actions in long, cluttered, and untrimmed videos is a hard task, that in the literature has typically been addressed assuming the availability of large amounts of annotated training samples for each class -- either in a fully-supervised setting, where action boundaries are known, or in a weakly-supervised setting, where only class labels are known for each video. In this paper, we go a step further and show that it is possible to learn to localize actions in untrimmed videos when a) only one/few trimmed examples of the target action are available at test time, and b) when a large collection of videos with only class label annotation (some trimmed and some weakly annotated untrimmed ones) are available for training; with no overlap between the classes used during training and testing. To do so, we propose a network that learns to estimate Temporal Similarity Matrices (TSMs) that model a fine-grained similarity pattern between pairs of videos (trimmed or untrimmed), and uses them to generate Temporal Class Activation Maps (TCAMs) for seen or unseen classes. The TCAMs serve as temporal attention mechanisms to extract video-level representations of untrimmed videos, and to temporally localize actions at test time. To the best of our knowledge, we are the first to propose a weakly-supervised, one/few-shot action localization network that can be trained in an end-to-end fashion. Experimental results on THUMOS14 and ActivityNet1.2 datasets, show that our method achieves performance comparable or better to state-of-the-art fully-supervised, few-shot learning methods.


title: Cross-Domain Gradient Discrepancy Minimization for Unsupervised Domain Adaptation

abstract: Unsupervised Domain Adaptation (UDA) aims to generalize the knowledge learned from a well-labeled source domain to an unlabeled target domain. Recently, adversarial domain adaptation with two distinct classifiers (bi-classifier) has been introduced into UDA which is effective to align distributions between different domains. Previous bi-classifier adversarial learning methods only focus on the similarity between the outputs of two distinct classifiers. However, the similarity of the outputs cannot guarantee the accuracy of target samples, i.e., target samples may match to wrong categories even if the discrepancy between two classifiers is small. To challenge this issue, in this paper, we propose a cross-domain gradient discrepancy minimization (CGDM) method which explicitly minimizes the discrepancy of gradients generated by source samples and target samples. Specifically, the gradient gives a cue for the semantic information of target samples so it can be used as a good supervision to improve the accuracy of target samples. In order to compute the gradient signal of target samples, we further obtain target pseudo labels through a clustering-based self-supervised learning. Extensive experiments on three widely used UDA datasets show that our method surpasses many previous state-of-the-arts. Codes are available at https://github.com/lijin118/CGDM.


title: PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learning

abstract: Learning good feature representations is important for deep reinforcement learning (RL). However, with limited experience, RL often suffers from data inefficiency for training. For un-experienced or less-experienced trajectories (i.e., state-action sequences), the lack of data limits the use of them for better feature learning. In this work, we propose a novel method, dubbed PlayVirtual, which augments cycle-consistent virtual trajectories to enhance the data efficiency for RL feature representation learning. Specifically, PlayVirtual predicts future states in the latent space based on the current state and action by a dynamics model and then predicts the previous states by a backward dynamics model, which forms a trajectory cycle. Based on this, we augment the actions to generate a large amount of virtual state-action trajectories. Being free of groudtruth state supervision, we enforce a trajectory to meet the cycle consistency constraint, which can significantly enhance the data efficiency. We validate the effectiveness of our designs on the Atari and DeepMind Control Suite benchmarks. Our method achieves the state-of-the-art performance on both benchmarks.


title: Closed-form expressions for the magnetic field of permanent magnets in three dimensions

abstract: We derive a closed-form expression of the magnetic field of a finite-size current sheet and use it to calculate the field of permanent magnets, which are modeled through their surface current densities. We illustrate the method by determining the multipoles and the effective length due to fringe fields of a finite-length dipole constructed of magnetic cubes.


title: Observation of the out-of-plane polarized spin current from CVD grown WTe2

abstract: Weyl semimetal Td-phase WTe2 possesses the spin-resolved band structure with strong spin-orbit coupling, holding promises as a useful spin source material. The noncentrosymmetric crystalline structure of Td-WTe2 endows the generation of the out-of-plane polarized spin, which is of great interest in magnetic memory applications. Previously, WTe2 was explored in spin devices based on mechanically exfoliated single crystal flakes with a size of micrometers. For practical spintronics applications, it is highly desirable to implement wafer-scale thin films. In this work, we utilize centimeter-scale chemical vapor deposition (CVD)-grown Td-WTe2 thin films and study the spin current generation by the spin torque ferromagnetic resonance technique. We find the in-plane and out-of-plane spin conductivities of 7.36 x 10^3 (h/2e) (ohm-m)^-1 and 1.76 x 10^3 (h/2e) (ohm-m)^-1, respectively, in CVD-growth 5 nm-WTe2. We further demonstrate the current-induced magnetization switching in WTe2/NiFe at room temperature in the domain wall motion regime, which may invigorate potential spintronic device innovations based on Weyl semimetals.


title: Review Polarity-wise Recommender

abstract: Utilizing review information to enhance recommendation, the de facto review-involved recommender systems, have received increasing interests over the past few years. Thereinto, one advanced branch is to extract salient aspects from textual reviews (i.e., the item attributes that users express) and combine them with the matrix factorization technique. However, existing approaches all ignore the fact that semantically different reviews often include opposite aspect information. In particular, positive reviews usually express aspects that users prefer, while negative ones describe aspects that users reject. As a result, it may mislead the recommender systems into making incorrect decisions pertaining to user preference modeling. Towards this end, in this paper, we propose a Review Polarity-wise Recommender model, dubbed as RPR, to discriminately treat reviews with different polarities. To be specific, in this model, positive and negative reviews are separately gathered and utilized to model the user-preferred and user-rejected aspects, respectively. Besides, in order to overcome the imbalance problem of semantically different reviews, we also develop an aspect-aware importance weighting approach to align the aspect importance for these two kinds of reviews. Extensive experiments conducted on eight benchmark datasets have demonstrated the superiority of our model as compared to a series of state-of-the-art review-involved baselines. Moreover, our method can provide certain explanations to the real-world rating prediction scenarios.


title: Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss

abstract: Recent works in self-supervised learning have advanced the state-of-the-art by relying on the contrastive learning paradigm, which learns representations by pushing positive pairs, or similar examples from the same class, closer together while keeping negative pairs far apart. Despite the empirical successes, theoretical foundations are limited -- prior analyses assume conditional independence of the positive pairs given the same class label, but recent empirical applications use heavily correlated positive pairs (i.e., data augmentations of the same image). Our work analyzes contrastive learning without assuming conditional independence of positive pairs using a novel concept of the augmentation graph on data. Edges in this graph connect augmentations of the same data, and ground-truth classes naturally form connected sub-graphs. We propose a loss that performs spectral decomposition on the population augmentation graph and can be succinctly written as a contrastive learning objective on neural net representations. Minimizing this objective leads to features with provable accuracy guarantees under linear probe evaluation. By standard generalization bounds, these accuracy guarantees also hold when minimizing the training contrastive loss. Empirically, the features learned by our objective can match or outperform several strong baselines on benchmark vision datasets. In all, this work provides the first provable analysis for contrastive learning where guarantees for linear probe evaluation can apply to realistic empirical settings.


title: Spatio-temporal coarse-graining decomposition of the global ocean geostrophic kinetic energy

abstract: We expand on a recent determination of the first global energy spectrum of the ocean's surface geostrophic circulation (Storer et al., 2022) using a coarse-graining (CG) method. We compare spectra from CG to those from spherical harmonics by treating land in a manner consistent with the boundary conditions. While the two methods yield qualitatively consistent domain-averaged results, spherical harmonics spectra are too noisy at gyre-scales ($>1000~$km). More importantly, spherical harmonics are inherently global and cannot provide local information connecting scales with currents geographically. CG shows that the extra-tropics mesoscales (100-500~km) have a root-mean-square (rms) velocity of $\sim15~$cm/s, which increases to $\sim30$-40~cm/s locally in the Gulf Stream and Kuroshio and to $\sim16$-28~cm/s in the ACC. There is notable hemispheric asymmetry in mesoscale energy-per-area, which is higher in the north due to continental boundaries. We estimate that $\approx25$-50\% of total geostrophic energy is at scales smaller than 100~km, and is un(der)-resolved by pre-SWOT satellite products. Spectra of the time-mean component show that most of its energy (up to $70\%$) resides in stationary mesoscales ($<500~$km), highlighting the preponderance of `standing' small-scale structures in the global ocean. By coarse-graining in space and time, we compute the first spatio-temporal global spectrum of geostrophic circulation from AVISO and NEMO. These spectra show that every length-scale evolves over a wide range of time-scales with a consistent peak at $\approx200$ km and $\approx2$-3~weeks.


title: Vertical structure of Galactic disk kinematics from LAMOST K giants

abstract: We examine the vertical structure of Galactic disk kinematics over a Galactocentric radial distance range of $R=5-15$ $\rm{kpc}$ and up to $3$ $\rm{kpc}$ away from the Galactic plane, using the K-type giants surveyed by LAMOST. Based on robust measurements of three-dimensional velocity moments, a wobbly disk is detected in a phenomenological sense. An outflow dominates the radial motion of the inner disk, while in the outer disk there exist alternate outward and inward flows. The vertical bulk velocities is a combination of breathing and bending modes. A contraction-like breathing mode with amplitudes increasing with the distance to the plane and an upward bending mode dominate the vertical motion outside $R_0$, and there are reversed breathing mode and bending mode at $R<R_0$, with amplitudes much smaller than those outside $R_0$. The mean azimuthal velocity decreases with the increasing distance to the plane, with gradients shallower for larger $R$. Stars in the south disk are rotating faster than stars in the north. The velocity ellipsoid orientation differs between different $R$: in the range of $5<R<9$ $\rm{kpc}$, the gradient of the tilt angle with respect to $\arctan(Z/R)$ decreases from $\sim0.83$ for the inner disk to $\sim0.52$ for the outer disk; within $9<R<15$ $\rm{kpc}$, the tilt of velocity ellipsoid deviates from vertical antisymmetry. A clear flaring signature is found for both north and south disks based on the observed vertical structures of velocity ellipsoid.


title: Fast Federated Learning in the Presence of Arbitrary Device Unavailability

abstract: Federated Learning (FL) coordinates with numerous heterogeneous devices to collaboratively train a shared model while preserving user privacy. Despite its multiple advantages, FL faces new challenges. One challenge arises when devices drop out of the training process beyond the control of the central server. In this case, the convergence of popular FL algorithms such as FedAvg is severely influenced by the straggling devices. To tackle this challenge, we study federated learning algorithms under arbitrary device unavailability and propose an algorithm named Memory-augmented Impatient Federated Averaging (MIFA). Our algorithm efficiently avoids excessive latency induced by inactive devices, and corrects the gradient bias using the memorized latest updates from the devices. We prove that MIFA achieves minimax optimal convergence rates on non-i.i.d. data for both strongly convex and non-convex smooth functions. We also provide an explicit characterization of the improvement over baseline algorithms through a case study, and validate the results by numerical experiments on real-world datasets.


title: Implementation of discrete positive operator valued measures on linear optical systems using cosine-sine decomposition

abstract: Positive operator valued measurements (POVMs) play an important role in efficient quantum communication and computation. While optical systems are one of the strongest candidates for long distance quantum communication and information processing, efficient methods to implement POVMs in these systems are scarce. Here we propose an all-optical scheme to implement an arbitrary POVM using linear optical components on m-dimensional Hilbert space of internal degrees of freedom. Linear optical nature of the proposed scheme makes it efficient and robust. We show how the scheme can be applied for state tomography and for preparing arbitrary mixed states.


title: Characterization of soft X-ray echo-enabled harmonic generation free-electron laser pulses in the presence of incoherent energy modulations

abstract: Echo-enabled harmonic generation free-electron lasers (EEHG FELs) are promising candidates to produce fully coherent soft x-ray pulses by virtue of efficient high harmonic frequency up-conversion from UV lasers. The ultimate spectral limit of EEHG, however, remains unclear, because of the broadening and distortions induced in the output spectrum by residual broadband energy modulations in the electron beam. We present a mathematical description of the impact of incoherent (broadband) energy modulations on the bunching spectrum produced by the microbunching instability through both the accelerator and the EEHG line. The model is in agreement with a systematic experimental characterization of the FERMI EEHG FEL in the photon energy range $130-210$ eV. We find that amplification of electron beam energy distortions primarily in the EEHG dispersive sections explains an observed reduction of the FEL spectral brightness that is proportional to the EEHG harmonic number. Local maxima of the FEL spectral brightness and of the spectral stability are found for a suitable balance of the dispersive sections' strength and the first seed laser pulse energy. Such characterization provides a benchmark for user experiments and future EEHG implementations designed to reach shorter wavelengths.


title: Electron-boson spectral density functions of cuprates obtained from optical spectra via machine learning

abstract: The electron-boson spectral density (or glue) function can be obtained from measured optical scattering rate by solving a generalized Allen formula, which relates the two quantities with an integral equation and is an inversion problem. Thus far, numerical approaches, such as the maximum entropy method (MEM) and the least squares fitting method, have been applied for solving the generalized Allen formula. Here, we developed a new method to obtain the glue functions from the optical scattering rate using a machine learning approach (MLA). We found that the MLA is more robust against random noise compared with the MEM. We applied the new developed MLA to experimentally measured optical scattering rates and obtained reliable glue functions in terms of their shapes including the amplitudes. We expect that the MLA can be a useful and rapid method for solving other inversion problems, which may contain random noise.


title: Superconducting microresonators for electron spin resonance, the good, the bad, and the future

abstract: The field of electron spin resonance is in constant need to improve its capabilities. Among other things, this means having better resonators which would provide improved spin sensitivity, as well as enable larger microwave magnetic field power conversion factors. Surface micro resonators, made of small metallic patches on a dielectric substrate, provide very good absolute spin sensitivity and high conversion factors due to their very small mode volume. However, such resonators suffer from having a relatively low quality factor, which offsets some of their significant potential advantages. The use of superconducting patches to replace the metallic layer seems like a reasonable and straightforward solution to the quality factor issue, at least for measurements carried out at cryogenic temperatures. Nevertheless, superconducting materials are not easily incorporated into setups requiring high magnetic fields, due to electric current vortices generated in the latter's surface. This makes the transition from normal conducing materials to superconductors highly nontrivial. Here we present the design, fabrication, and testing results of surface micro resonators made of yttrium barium copper oxide (YBCO) superconducting material. We show that with a unique experimental setup, these resonators can be made to operate well even at high fields of about 1.2 T. Furthermore, we analyze the effect of current vortices on the ESR signal and the spins' coherence times. Finally, we provide a head to head comparison of YBCO vs copper resonators of the same dimensions, which clearly shows their pros and cons and directs us to future potential developments and improvements in this field.


title: Superradiant many-qubit absorption refrigerator

abstract: We show that the lower levels of a large-spin network with a collective anti-ferromagnetic interaction and collective couplings to three reservoirs may function as a quantum absorption refrigerator. In appropriate regimes, the steady-state cooling current of this refrigerator scales quadratically with the size of the working medium, i.e., the number of spins. The same scaling is observed for the noise and the entropy production rate.


title: Neural Hybrid Automata: Learning Dynamics with Multiple Modes and Stochastic Transitions

abstract: Effective control and prediction of dynamical systems often require appropriate handling of continuous-time and discrete, event-triggered processes. Stochastic hybrid systems (SHSs), common across engineering domains, provide a formalism for dynamical systems subject to discrete, possibly stochastic, state jumps and multi-modal continuous-time flows. Despite the versatility and importance of SHSs across applications, a general procedure for the explicit learning of both discrete events and multi-mode continuous dynamics remains an open problem. This work introduces Neural Hybrid Automata (NHAs), a recipe for learning SHS dynamics without a priori knowledge on the number of modes and inter-modal transition dynamics. NHAs provide a systematic inference method based on normalizing flows, neural differential equations and self-supervision. We showcase NHAs on several tasks, including mode recovery and flow learning in systems with stochastic transitions, and end-to-end learning of hierarchical robot controllers.


title: Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics

abstract: Learning dynamics governed by differential equations is crucial for predicting and controlling the systems in science and engineering. Neural Ordinary Differential Equation (NODE), a deep learning model integrated with differential equations, is popular in learning dynamics recently due to its robustness to irregular samples and its flexibility to high-dimensional input. However, the training of NODE is sensitive to the precision of the numerical solver, which makes the convergence of NODE unstable, especially for ill-conditioned dynamical systems. In this paper, to reduce the reliance on the numerical solver, we propose to enhance the supervised signal in the training of NODE. Specifically, we pre-train a neural differential operator (NDO) to output an estimation of the derivatives to serve as an additional supervised signal. The NDO is pre-trained on a class of basis functions and learns the mapping between the trajectory samples of these functions to their derivatives. To leverage both the trajectory signal and the estimated derivatives from NDO, we propose an algorithm called NDO-NODE, in which the loss function contains two terms: the fitness on the true trajectory samples and the fitness on the estimated derivatives that are outputted by the pre-trained NDO. Experiments on various kinds of dynamics show that our proposed NDO-NODE can consistently improve the forecasting accuracy with one pre-trained NDO. Especially for the stiff ODEs, we observe that NDO-NODE can capture the transitions in the dynamics more accurately compared with other regularization methods.


title: Limit behaviour of Weyl coefficients

abstract: We study the sets of radial or nontangential limit points towards $i\infty$ of a Nevanlinna function q. Given a nonempty, closed, and connected subset L of $C_+$ , we explicitly construct a Hamiltonian H such that the radial- and outer angular cluster sets towards $i\infty$ of the Weyl coefficient q H are both equal to L. Our method is based on a study of the continuous group action of rescaling operators on the set of all Hamiltonians.


title: Schur expansion of random-matrix reproducing kernels

abstract: We give expansions of reproducing kernels of the Christoffel-Darboux type in terms of Schur polynomials. For this, we use evaluations of averages of characteristic polynomials and Schur polynomials in random matrix ensembles. We explicitly compute new Schur averages, such as the Schur average in a $q$-Laguerre ensemble, and the ensuing expansions of random matrix kernels. In addition to classical and $q$-deformed cases on the real line, we use extensions of Dotsenko-Fateev integrals to obtain expressions for kernels on the complex plane. Moreover, a known interplay between Wronskians of Laguerre polynomials, Painlev\'e tau functions and conformal block expansions is discussed in relationship to the Schur expansion obtained.


title: On Improving Adversarial Transferability of Vision Transformers

abstract: Vision transformers (ViTs) process input images as sequences of patches via self-attention; a radically different architecture than convolutional neural networks (CNNs). This makes it interesting to study the adversarial feature space of ViT models and their transferability. In particular, we observe that adversarial patterns found via conventional adversarial attacks show very \emph{low} black-box transferability even for large ViT models. We show that this phenomenon is only due to the sub-optimal attack procedures that do not leverage the true representation potential of ViTs. A deep ViT is composed of multiple blocks, with a consistent architecture comprising of self-attention and feed-forward layers, where each block is capable of independently producing a class token. Formulating an attack using only the last class token (conventional approach) does not directly leverage the discriminative information stored in the earlier tokens, leading to poor adversarial transferability of ViTs. Using the compositional nature of ViT models, we enhance transferability of existing attacks by introducing two novel strategies specific to the architecture of ViT models. (i) Self-Ensemble: We propose a method to find multiple discriminative pathways by dissecting a single ViT model into an ensemble of networks. This allows explicitly utilizing class-specific information at each ViT block. (ii) Token Refinement: We then propose to refine the tokens to further enhance the discriminative capacity at each block of ViT. Our token refinement systematically combines the class tokens with structural information preserved within the patch tokens.


title: Scalable conditional deep inverse Rosenblatt transports using tensor-trains and gradient-based dimension reduction

abstract: We present a novel offline-online method to mitigate the computational burden of the characterization of posterior random variables in statistical learning. In the offline phase, the proposed method learns the joint law of the parameter random variables and the observable random variables in the tensor-train (TT) format. In the online phase, the resulting order-preserving conditional transport can characterize the posterior random variables given newly observed data in real time. Compared with the state-of-the-art normalizing flow techniques, the proposed method relies on function approximation and is equipped with a thorough performance analysis. The function approximation perspective also allows us to further extend the capability of transport maps in challenging problems with high-dimensional observations and high-dimensional parameters. On the one hand, we present novel heuristics to reorder and/or reparametrize the variables to enhance the approximation power of TT. On the other hand, we integrate the TT-based transport maps and the parameter reordering/reparametrization into layered compositions to further improve the performance of the resulting transport maps. We demonstrate the efficiency of the proposed method on various statistical learning tasks in ordinary differential equations (ODEs) and partial differential equations (PDEs).


title: Optimization of selective two-photon absorption in cavity polaritons

abstract: We investigate optimal states of photon pairs to excite a target transition in a multilevel quantum system. With the help of coherent control theory for two-photon absorption with quantum light, we infer the maximal population achievable by optimal entangled vs. separable states of light. Interference between excitation pathways, as well as the presence of nearby states, may hamper the selective excitation of a particular target state, but we show that quantum correlations can help to overcome this problem, and enhance the achievable "selectivity" between two energy levels, i.e. the relative difference in population transferred into each of them. We find that the added value of optimal entangled states of light increases with broadening linewidths of the target states.