title: Privacy-Preserving Eye-tracking Using Deep Learning

abstract: The expanding usage of complex machine learning methods like deep learning has led to an explosion in human activity recognition, particularly applied to health. In particular, as part of a larger body sensor network system, face and full-body analysis is becoming increasingly common for evaluating health status. However, complex models which handle private and sometimes protected data, raise concerns about the potential leak of identifiable data. In this work, we focus on the case of a deep network model trained on images of individual faces. Full-face video recordings taken from 493 individuals undergoing an eye-tracking based evaluation of neurological function were used. Outputs, gradients, intermediate layer outputs, loss, and labels were used as inputs for a deep network with an added support vector machine emission layer to recognize membership in the training data. The inference attack method and associated mathematical analysis indicate that there is a low likelihood of unintended memorization of facial features in the deep learning model. In this study, it is showed that the named model preserves the integrity of training data with reasonable confidence. The same process can be implemented in similar conditions for different models.


title: Extracting Different Levels of Speech Information from EEG Using an LSTM-Based Model

abstract: Decoding the speech signal that a person is listening to from the human brain via electroencephalography (EEG) can help us understand how our auditory system works. Linear models have been used to reconstruct the EEG from speech or vice versa. Recently, Artificial Neural Networks (ANNs) such as Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) based architectures have outperformed linear models in modeling the relation between EEG and speech. Before attempting to use these models in real-world applications such as hearing tests or (second) language comprehension assessment we need to know what level of speech information is being utilized by these models. In this study, we aim to analyze the performance of an LSTM-based model using different levels of speech features. The task of the model is to determine which of two given speech segments is matched with the recorded EEG. We used low- and high-level speech features including: envelope, mel spectrogram, voice activity, phoneme identity, and word embedding. Our results suggest that the model exploits information about silences, intensity, and broad phonetic classes from the EEG. Furthermore, the mel spectrogram, which contains all this information, yields the highest accuracy (84%) among all the features.


title: Towards Explainable Student Group Collaboration Assessment Models Using Temporal Representations of Individual Student Roles

abstract: Collaboration is identified as a required and necessary skill for students to be successful in the fields of Science, Technology, Engineering and Mathematics (STEM). However, due to growing student population and limited teaching staff it is difficult for teachers to provide constructive feedback and instill collaborative skills using instructional methods. Development of simple and easily explainable machine-learning-based automated systems can help address this problem. Improving upon our previous work, in this paper we propose using simple temporal-CNN deep-learning models to assess student group collaboration that take in temporal representations of individual student roles as input. We check the applicability of dynamically changing feature representations for student group collaboration assessment and how they impact the overall performance. We also use Grad-CAM visualizations to better understand and interpret the important temporal indices that led to the deep-learning model's decision.


title: Probabilistic Stability Assessment for Active Distribution Grids

abstract: This paper demonstrates the concept of probabilistic stability assessment on large-signal stability in the use case of short circuits in an active distribution grid. Here, the concept of survivability is applied, which extends classical stability assessments by evaluating the stability and operational limits during transients for a wide range of operating points and failures. For this purpose, a free, open-source, and computationally efficient environment (Julia) for dynamic simulation of power grids is used to demonstrate its capabilities. The model implementation is validated against established commercial software and deviations are minimal with respect to power flow and dynamic simulations.The results of a large-scale survivability analysis reveal i) a broad field of application for probabilistic stability analysis and ii) that new non-intuitive stability correlations can be obtained. Hence,the proposed method shows strong potential to efficiently conduct power system stability analysis in active distribution grids.


title: Poisson-Dirichlet asymptotics in condensing particle systems

abstract: We study measures on random partitions, arising from condensing stochastic particle systems with stationary product distributions. We provide fairly general conditions on the stationary weights, which lead to Poisson-Dirichlet statistics of the condensed phase in the thermodynamic limit. The Poisson-Dirichlet distribution is known to be the unique reversible measure of split-merge dynamics for random partitions, which we use to characterize the limit law. We also establish concentration results for the macroscopic phase, using size-biased sampling techniques and the equivalence of ensembles to characterize the bulk distribution of the system.

categories: math.PR cond-mat.stat-mech math-ph math.MP


title: ALMA discovery of a dual dense probably rotating outflow from a massive young stellar object G18.88MME

abstract: We report the discovery of a very dense jet-like fast molecular outflow surrounded by a wide-angle wind in a massive young stellar object (MYSO) G18.88MME (stellar mass $\sim$8 M$_{\odot}$) powering an Extended Green Object G18.89$-$0.47. Four cores MM1-4 are identified in the Atacama Large Millimeter/submillimeter Array (ALMA) 1.3 mm continuum map (resolution $\sim$0.$"$8) toward G18.88MME, and are seen at the center of the emission structure (extent $\sim$0.3 pc $\times$ 0.2 pc) detected in the ALMA map. G18.88MME is embedded in the core MM1 (mass $\sim$13-18 M$_{\odot}$), where no radio continuum emission is detected. The molecular outflow centered at MM1 is investigated in the SiO(5-4), HC$_{3}$N(24-23) and $^{13}$CO(2-1) lines. The detection of HC$_{3}$N in the outflow is rare in MYSOs and indicates its very high density. The position-velocity diagrams display a fast narrow outflow (extent $\sim$28000 AU) and a slower wide-angle more extended outflow toward MM1, and both of these components show a transverse velocity gradient indicative of a possible rotation. All these observed features together make G18.88MME as a unique object for studying the unification of the jet-driven and wind-driven scenarios of molecular outflows in MYSOs.


title: Towards Prevention of Sportsmen Burnout: Formal Analysis of Sub-Optimal Tournament Scheduling

abstract: Scheduling a sports tournament is a complex optimization problem, which requires a large number of hard constraints to satisfy. Despite the availability of several such constraints in the literature, there remains a gap since most of the new sports events pose their own unique set of requirements, and demand novel constraints. Specifically talking of the strictly time bound events, ensuring fairness between the different teams in terms of their rest days, traveling, and the number of successive games they play, becomes a difficult task to resolve, and demands attention. In this work, we present a similar situation with a recently played sports event, where a suboptimal schedule favored some of the sides more than the others. We introduce various competitive parameters to draw a fairness comparison between the sides and propose a weighting criterion to point out the sides that enjoyed this schedule more than the others. Furthermore, we use root mean squared error between an ideal schedule and the actual ones for each side to determine unfairness in the distribution of rest days across their entire schedules. The latter is crucial, since successively playing a large number of games may lead to sportsmen burnout, which must be prevented.


title: Application of CNN to a fine segmented scintillator detector for a single particle and neutrino-nucleon event

abstract: This paper presents studies on application of convolutional neural network (CNN) to GEANT4 optical simulation data generated with a scintillator detector subdivided into 1 cubic cm, which is designed for the long-baseline neutrino experiment. Classification of interaction, regression of momentum, and segmentation of hits are demonstrated for single particle and neutrino-nucleon interaction events with well established CNN architectures by feeding reconstructed 2D projection images. In the study it is shown that the application of CNN to the 1 cm subdivided scintillator detector can provide a factor about 2 better momentum resolution compared to a standard method, as well as a classification capability of about 94% for the single particle and 70% for the neutrino-nucleon interaction events. Cross-section analyses with CNN is also shown to be feasible.


title: Relating entropies of quantum channels

abstract: In this work, we study two different approaches to defining the entropy of a quantum channel. One of these is based on the von Neumann entropy of the corresponding Choi-Jamio{\l}kowski state. The second one is based on the relative entropy of the output of the extended channel relative to the output of the extended completely depolarizing channel. This entropy then needs to be optimized over all possible input states. Our results first show that the former entropy provides an upper bound on the latter. Next, we show that for unital qubit channels, this bound is saturated. Finally, we conjecture and provide numerical intuitions that the bound can also be saturated for random channels as their dimension tends to infinity.


title: Lensing of Gravitational Waves as a Novel Probe of Graviton Mass

abstract: The diffraction patterns of lensed gravitational waves encode information about their propagation speeds. If gravitons have mass, the dispersion relation and speed of gravitational waves will be affected in a frequency-dependent manner, which would leave traces in the diffraction pattern if the waves are lensed. In this paper, we study how the alternative dispersion relation induced by massive gravitons affects gravitational waves lensed by point-mass lenses, such as intermediate-mass black holes. We find that the waveform morphology of lensed dispersive gravitational waves depends on the graviton mass more sensitively than their unlensed counterpart. Together with lensing amplification, the waveform-morphology modifications due to lensing can improve the measurement accuracy of the graviton mass. A single lensed gravitational-wave signal enables us to measure the graviton mass with an accuracy comparable with the combined measurement across $\mathcal{O}(10^3)$ unlensed signals. Our method allows us to incorporate lensed gravitational-wave signals into existing graviton-mass measurements. Our method can also be generalized to other lens types, gravitational-wave sources, and detector networks, preparing ourselves for thoroughly understanding the nature of gravitational waves in the era of lensed gravitational-wave astronomy.


title: Computational Optimization of MnBi to Enhance Energy Product

abstract: High energy density magnets are preferred over induction magnets for many applications, including electric motors used in flying rovers, electric vehicles, and wind turbines. However, several issues related to cost and supply with state-of-the-art rare-earth-based magnet necessities development of high-flux magnets containing low cost, earth-abundant materials. Here, we demonstrate the possibility of tuning magnetization and magnetocrystalline anisotropy of one of the candidate materials, MnBi, by alloying it with foreign elements. By using the density functional theory in the high-throughput fashion, we consider the possibility of alloying MnBi with all possible metal and non-metal elements in the periodic table and found that MnBi-based alloys with Pd, Pt, Rh, Li, and O are stable against decomposition to constituent elements and have larger magnetization, energy product compared and magnetic anisotropy compared to MnBi We consider the possibility of these elements occupying half and all of the available empty sites. Combined with other favorable properties of MnBi, such as high Curie temperature and earth abundancy of constituents elements, we envision the possibility of MnBi-based high-energy-density magnets.


title: Large-Scale Multiple Testing for Matrix-Valued Data under Double Dependency

abstract: High-dimensional inference based on matrix-valued data has drawn increasing attention in modern statistical research, yet not much progress has been made in large-scale multiple testing specifically designed for analysing such data sets. Motivated by this, we consider in this article an electroencephalography (EEG) experiment that produces matrix-valued data and presents a scope of developing novel matrix-valued data based multiple testing methods controlling false discoveries for hypotheses that are of importance in such an experiment. The row-column cross-dependency of observations appearing in a matrix form, referred to as double-dependency, is one of the main challenges in the development of such methods. We address it by assuming matrix normal distribution for the observations at each of the independent matrix data-points. This allows us to fully capture the underlying double-dependency informed through the row- and column-covariance matrices and develop methods that are potentially more powerful than the corresponding one (e.g., Fan and Han (2017)) obtained by vectorizing each data point and thus ignoring the double-dependency. We propose two methods to approximate the false discovery proportion with statistical accuracy. While one of these methods is a general approach under double-dependency, the other one provides more computational efficiency for higher dimensionality. Extensive numerical studies illustrate the superior performance of the proposed methods over the principal factor approximation method of Fan and Han (2017). The proposed methods have been further applied to the aforementioned EEG data.


title: Optimal Relevant Subset Designs in Nonlinear Models

abstract: Fisher (1934) argued that certain ancillary statistics form a relevant subset, a subset of the sample space on which inference should be restricted, and showed that conditioning on their observed value reduces the dimension of the data without a loss of information. The use of ancillary statistics in post-data inference has received significant attention; however, their role in the design of the experiment has not been well characterized. Ancillary statistics are unknown prior to data collection and as a result cannot be incorporated into the design a priori. However, if the data are observed sequentially then the ancillary statistics based on the data from the preceding observations can be used to determine the design assignment for the current observation. The main results of this work describe the benefits of incorporating ancillary statistics, specifically, the ancillary statistic that constitutes a relevant subset, into an adaptive design.


title: An Endless Optical Phase Delay for Phase Synchronization in High-Capacity DCIs

abstract: In this work, we propose and demonstrate a module to linearly add an arbitrary amount of continuous (reset-free) phase delay to an optical signal. The proposed endless optical phase delay (EOPD) uses an optical IQ modulator and control electronics (CE) to add the desired amount of phase delay that can continuously increase with time. In order to adjust for the bias voltages and control voltage amplitudes in the EOPD, some of which may be time varying, a multivariate gradient descent algorithm is used. The EOPD has been demonstrated experimentally, and its use in a high-capacity data center interconnect (DCI) application has been outlined in this letter. The EOPD may find its use in many other applications that require precise phase/frequency adjustments in real-time.


title: Quantum error as an emergent magnetic field

abstract: We investigate the effect of quantum errors on a monitored Brownian Sachdev-Ye-Kitaev (SYK) model featuring a measurement-induced phase transition that can be understood as a symmetry-breaking transition of an effective $Z_4$ magnet in the replica space. The errors describe the loss of information about the measurement outcomes and are applied during the non-unitary evolution or at the end of the evolution. In the former case, we find that this error can be mapped to an emergent magnetic field in the $Z_4$ magnet, and as a consequence, the symmetry is explicitly broken independent of the measurement rate. R\'enyi entropies computed by twisting boundary conditions now generate domain walls even in the would-be symmetric phase at a high measurement rate. The entropy is therefore volume-law irrespective of the measurement rate. In the latter case, the error-induced magnetic field only exists near the boundary of the magnet. Varying the magnetic field leads to a pinning transition of domain walls, corresponding to error threshold of the quantum code prepared by the non-unitary SYK dynamics.

categories: quant-ph cond-mat.stat-mech cond-mat.str-el hep-th


title: Multi-Modal Prototype Learning for Interpretable Multivariable Time Series Classification

abstract: Multivariable time series classification problems are increasing in prevalence and complexity in a variety of domains, such as biology and finance. While deep learning methods are an effective tool for these problems, they often lack interpretability. In this work, we propose a novel modular prototype learning framework for multivariable time series classification. In the first stage of our framework, encoders extract features from each variable independently. Prototype layers identify single-variable prototypes in the resulting feature spaces. The next stage of our framework represents the multivariable time series sample points in terms of their similarity to these single-variable prototypes. This results in an inherently interpretable representation of multivariable patterns, on which prototype learning is applied to extract representative examples i.e. multivariable prototypes. Our framework is thus able to explicitly identify both informative patterns in the individual variables, as well as the relationships between the variables. We validate our framework on a simulated dataset with embedded patterns, as well as a real human activity recognition problem. Our framework attains comparable or superior classification performance to existing time series classification methods on these tasks. On the simulated dataset, we find that our model returns interpretations consistent with the embedded patterns. Moreover, the interpretations learned on the activity recognition dataset align with domain knowledge.


title: AttDLNet: Attention-based DL Network for 3D LiDAR Place Recognition

abstract: LiDAR-based place recognition is one of the key components of SLAM and global localization in autonomous vehicles and robotics applications. With the success of DL approaches in learning useful information from 3D LiDARs, place recognition has also benefited from this modality, which has led to higher re-localization and loop-closure detection performance, particularly, in environments with significant changing conditions. Despite the progress in this field, the extraction of proper and efficient descriptors from 3D LiDAR data that are invariant to changing conditions and orientation is still an unsolved challenge. To address this problem, this work proposes a novel 3D LiDAR-based deep learning network (named AttDLNet) that uses a range-based proxy representation for point clouds and an attention network with stacked attention layers to selectively focus on long-range context and inter-feature relationships. The proposed network is trained and validated on the KITTI dataset and an ablation study is presented to assess the novel attention network. Results show that adding attention to the network improves performance, leading to efficient loop closures, and outperforming an established 3D LiDAR-based place recognition approach. From the ablation study, results indicate that the middle encoder layers have the highest mean performance, while deeper layers are more robust to orientation change. The code is publicly available at https://github.com/Cybonic/AttDLNet


title: Improved Majority Identification by the Coarsened Majority Automaton

abstract: The initial majority identification task is a fundamental test problem in cellular automaton research. To pass the test, an automaton must evolve to a uniform configuration consisting of the state that was in the majority for any initial configuration, employing only its internal, local dynamics. It is known that no two-state automaton can perform the majority task perfectly. Thus, it is a matter of continuing interest to identify and analyze new automata with improved majority identification capability. Here, we show that a coarsened version of one of the best majority identifiers can out-perform its parent automaton while simultaneously reducing the associated computational costs.


title: Implementing Permutations in the Brain and SVO Frequencies of Languages

abstract: The subject-verb-object (SVO) word order prevalent in English is shared by about $42\%$ of world languages. Another $45\%$ of all languages follow the SOV order, $9\%$ the VSO order, and fewer languages use the three remaining permutations. None of the many extant explanations of this phenomenon take into account the difficulty of implementing these permutations in the brain. We propose a plausible model of sentence generation inspired by the recently proposed Assembly Calculus framework of brain function. Our model results in a natural explanation of the uneven frequencies. Estimating the parameters of this model yields predictions of the relative difficulty of dis-inhibiting one brain area from another. Our model is based on the standard syntax tree, a simple binary tree with three leaves. Each leaf corresponds to one of the three parts of a basic sentence. The leaves can be activated through lock and unlock operations and the sequence of activation of the leaves implements a specific word order. More generally, we also formulate and algorithmically solve the problems of implementing a permutation of the leaves of any binary tree, and of selecting the permutation that is easiest to implement on a given binary tree.


title: Microgrid Resilience: A Holistic and Context-Aware Resilience Metric

abstract: Microgrids present an effective solution for the coordinated deployment of various distributed energy resources and furthermore provide myriad additional benefits such as resilience, decreased carbon footprint, and reliability to energy consumers and the energy system as a whole. Boosting the resilience of distribution systems is another major benefit of microgrids. This is because they can also serve as a backup power source when the utility grid operations are interrupted due to either high-probability low-impact events like a component failure or low-probability high-impact events - be it a natural disaster or a planned cyberattack. However, the degree to which any particular system can defend, adapt, and restore normal operation depends on various factors including the type and severity of events to which a microgrid is subjected. These factors, in turn, are dependent on the geographical location of the deployed microgrid as well as the cyber risk profile of the site where the microgrid is operating. Therefore, in this work, we attempt to capture this multi-dimensional interplay of various factors in quantifying the ability of the microgrid to be resilient in these varying aspects. This paper, thus, proposes a customized site-specific quantification of the resilience strength for the individual microgrid capability to absorb, restore, and adapt to the changing circumstances for sustaining the critical load when a low-probability high-impact event occurs - termed as - context-aware resilience metric. We also present a case study to illustrate the key elements of our integrated analytical approach.


title: Local non-periodic order and diam-mean equicontinuity on cellular automata

abstract: Diam-mean equicontinuity is a dynamical property that has been of use in the study of non-periodic order. Using some type of "local" skew product between a shift and an odometer looking cellular automaton (CA), we will show there exists an almost diam-mean equicontinuous CA that is not almost equicontinuous, (and hence not almost locally periodic). As an application we show that Kurka's dichotomy does not hold for diam-mean versions of sensitivity and equicontinuity.


title: Full counting statistics of Schwinger pair production and annihilation

abstract: We study the probability distribution of the number of particle and antiparticle pairs produced via the Schwinger effect when a uniform but time-dependent electric field is applied to noninteracting scalars or spinors initially at a thermodynamic equilibrium. We derive the formula for the characteristic function by employing techniques in mesoscopic physics, reflecting a close analogy between the Schwinger effect and mesoscopic tunneling transports. In particular, we find that the pair production in a medium is enhanced (suppressed) for scalars (spinors) due to the Bose stimulation (Pauli blocking). Furthermore, in addition to the production of accelerated pairs by the electric field, the annihilation of decelerated pairs is found to take place in a medium. Our formula allows us to extract the probability distributions in various situations, such as those obeying the generalized trinomial statistics for spin-momentum resolved counting and the bidirectional Poisson statistics for spin-momentum unresolved counting.


title: MetaBalance: High-Performance Neural Networks for Class-Imbalanced Data

abstract: Class-imbalanced data, in which some classes contain far more samples than others, is ubiquitous in real-world applications. Standard techniques for handling class-imbalance usually work by training on a re-weighted loss or on re-balanced data. Unfortunately, training overparameterized neural networks on such objectives causes rapid memorization of minority class data. To avoid this trap, we harness meta-learning, which uses both an ''outer-loop'' and an ''inner-loop'' loss, each of which may be balanced using different strategies. We evaluate our method, MetaBalance, on image classification, credit-card fraud detection, loan default prediction, and facial recognition tasks with severely imbalanced data, and we find that MetaBalance outperforms a wide array of popular re-sampling strategies.


title: Complexity Analysis of Dynamical Cylinder in Massive Brans-Dicke Gravity

abstract: In this paper, a complexity factor is devised for a non-static cylindrical system in the framework of massive Brans-Dicke theory. The definition of complexity is developed by taking into account the essential physical characteristics (such as anisotropy, inhomogeneity, etc.) of the system. In order to determine the complexity factor of the self-gravitating object, we acquire structure scalars from the orthogonal splitting of the Riemann tensor. Moreover, we discuss two patterns of evolution and choose the homologous mode as the simplest pattern under the influence of massive scalar field. We derive solutions in the absence as well as presence of heat dissipation for a specific form of the scalar field. The factors that induce complexity in an initially complexity-free system are also examined. It is concluded that the massive scalar field as well as heat dissipation contribute to the complexity of the celestial system. Thus, a dynamical cylinder is more complex as compared to its static counterpart.


title: Prototypical Graph Contrastive Learning

abstract: Graph-level representations are critical in various real-world applications, such as predicting the properties of molecules. But in practice, precise graph annotations are generally very expensive and time-consuming. To address this issue, graph contrastive learning constructs instance discrimination task which pulls together positive pairs (augmentation pairs of the same graph) and pushes away negative pairs (augmentation pairs of different graphs) for unsupervised representation learning. However, since for a query, its negatives are uniformly sampled from all graphs, existing methods suffer from the critical sampling bias issue, i.e., the negatives likely having the same semantic structure with the query, leading to performance degradation. To mitigate this sampling bias issue, in this paper, we propose a Prototypical Graph Contrastive Learning (PGCL) approach. Specifically, PGCL models the underlying semantic structure of the graph data via clustering semantically similar graphs into the same group, and simultaneously encourages the clustering consistency for different augmentations of the same graph. Then given a query, it performs negative sampling via drawing the graphs from those clusters that differ from the cluster of query, which ensures the semantic difference between query and its negative samples. Moreover, for a query, PGCL further reweights its negative samples based on the distance between their prototypes (cluster centroids) and the query prototype such that those negatives having moderate prototype distance enjoy relatively large weights. This reweighting strategy is proved to be more effective than uniform sampling. Experimental results on various graph benchmarks testify the advantages of our PGCL over state-of-the-art methods. Code is publicly available at https://github.com/ha-lins/PGCL.


title: Impurity-induced increase in the thermal quantum correlations and teleportation in an Ising-XXZ diamond chain

abstract: In this work we analyze the quantum correlations in a spin-1/2 Ising-XXZ diamond chain with one plaquette distorted impurity. We have shown that the introduction of impurity into the chain can significantly increase entanglement as well as quantum correlations compared to the original model, without impurity. Due to the great flexibility in the choice of impurity parameters, the model presented is very general and this fact can be very useful for future experimental measurements. In addition to entanglement and quantum coherence, we studied quantum teleportation through a quantum channel composed by a coupled of Heisenberg dimers with distorted impurity in an Ising-XXZ diamond chain, as well as fidelity in teleportation. Our analysis shows that the appropriate choice of parameters can greatly increase all the measures analyzed. For comparison purposes, we present all our results together with the results of the measurements made for the original model, without impurity, studied in previous works.


title: Deep Learning Through the Lens of Example Difficulty

abstract: Existing work on understanding deep learning often employs measures that compress all data-dependent information into a few numbers. In this work, we adopt a perspective based on the role of individual examples. We introduce a measure of the computational difficulty of making a prediction for a given input: the (effective) prediction depth. Our extensive investigation reveals surprising yet simple relationships between the prediction depth of a given input and the model's uncertainty, confidence, accuracy and speed of learning for that data point. We further categorize difficult examples into three interpretable groups, demonstrate how these groups are processed differently inside deep models and showcase how this understanding allows us to improve prediction accuracy. Insights from our study lead to a coherent view of a number of separately reported phenomena in the literature: early layers generalize while later layers memorize; early layers converge faster and networks learn easy data and simple functions first.


title: Band topology of pseudo-Hermitian phases through tensor Berry connections and quantum metric

abstract: Among non-Hermitian systems, pseudo-Hermitian phases represent a special class of physical models characterized by real energy spectra and by the absence of non-Hermitian skin effects. Here, we show that several pseudo-Hermitian phases in two and three dimensions can be built by employing $q$-deformed matrices, which are related to the representation of deformed algebras. Through this algebraic approach we present and study the pseudo-Hermitian version of well known Hermitian topological phases, raging from two-dimensional Chern insulators and time-reversal-invariant topological insulators to three-dimensional Weyl semimetals and chiral topological insulators. We analyze their topological bulk states through non-Hermitian generalizations of Abelian and non-Abelian tensor Berry connections and quantum metric. Although our pseudo-Hermitian models and their Hermitian counterparts share the same topological invariants, their band geometries are different. We indeed show that some of our pseudo-Hermitian phases naturally support nearly-flat topological bands, opening the route to the study of pseudo-Hermitian strongly-interacting systems. Finally, we provide an experimental protocol to realize our models and measure the full non-Hermitian quantum geometric tensor in synthetic matter.


title: Influence of the bus waveguide on the linear and nonlinear response of a taiji microresonator

abstract: We study the linear and nonlinear response of a unidirectional reflector where a nonlinear breaking of the Lorentz reciprocity is observed. The device under test consists of a racetrack microresonator, with an embedded S-shaped waveguide, coupled to an external bus waveguide (BW). This geometry of the microresonator, known as "taiji" microresonator (TJMR), allows to selectively couple counter-propagating modes depending on the propagation direction of the incident light and, at the nonlinear level, leads to an effective breaking of Lorentz reciprocity. Here, we show that a full description of the device needs to consider also the role of the BW, which introduces (i) Fabry-Perot oscillations (FPOs) due to reflections at its facets, and (ii) asymmetric losses, which depend on the actual position of the TJMR. At sufficiently low powers the asymmetric loss does not affect the unidirectional behavior, but the FP interference fringes can cancel the effect of the S-shaped waveguide. However, at high input power, both the asymmetric loss and the FPOs contribute to the redistribution of the energy between the clockwise and counterclockwise modes within the TJMR. This strongly modifies the nonlinear response, giving rise to counter-intuitive features where, due to the FP effect and the asymmetric losses, the BW properties can determine the violation of the Lorentz reciprocity and, in particular, the difference between the transmittance in the two directions of excitation. The experimental results are explained by using an analytical model based on the transfer matrix approach, a numerical finite-element model and exploiting intuitive interference diagrams.


title: Multi-head or Single-head? An Empirical Comparison for Transformer Training

abstract: Multi-head attention plays a crucial role in the recent success of Transformer models, which leads to consistent performance improvements over conventional attention in various applications. The popular belief is that this effectiveness stems from the ability of jointly attending multiple positions. In this paper, we first demonstrate that jointly attending multiple positions is not a unique feature of multi-head attention, as multi-layer single-head attention also attends multiple positions and is more effective. Then, we suggest the main advantage of the multi-head attention is the training stability, since it has less number of layers than the single-head attention, when attending the same number of positions. For example, 24-layer 16-head Transformer (BERT-large) and 384-layer single-head Transformer has the same total attention head number and roughly the same model size, while the multi-head one is significantly shallower. Meanwhile, we show that, with recent advances in deep learning, we can successfully stabilize the training of the 384-layer Transformer. As the training difficulty is no longer a bottleneck, substantially deeper single-head Transformer achieves consistent performance improvements without tuning hyper-parameters.


title: Gate-tunable Veselago Interference in a Bipolar Graphene Microcavity

abstract: The relativistic charge carriers in monolayer graphene can be manipulated in manners akin to conventional optics (electron-optics): angle-dependent Klein tunneling collimates an electron beam (analogous to a laser), while a Veselago refraction process focuses it (analogous to an optical lens). Both processes have been previously investigated, but the collimation and focusing efficiency have been reported to be relatively low even in state-of-the-art ballistic pn-junction devices. These limitations prevented the realization of more advanced quantum devices based on electron-optical interference, while understanding of the underlying physics remains elusive. Here, we present a novel device architecture of a graphene microcavity defined by carefully-engineered local strain and electrostatic fields. We create a controlled electron-optic interference process at zero magnetic field as a consequence of consecutive Veselago refractions in the microcavity and provide direct experimental evidence through low-temperature electrical transport measurements. The experimentally observed first-, second-, and third-order interference peaks agree quantitatively with the Veselago physics in a microcavity. In addition, we demonstrate decoherence of the interference by an external magnetic field, as the cyclotron radius becomes comparable to the interference length scale. For its application in electron-optics, we utilize Veselago interference to localize uncollimated electrons and characterize its contribution in further improving collimation efficiency. Our work sheds new light on relativistic single-particle physics and provides important technical improvements toward next-generation quantum devices based on the coherent manipulation of electron momentum and trajectory.


title: Detection of the cyanomidyl radical (HNCN): a new interstellar species with the NCN backbone

abstract: We report here the first detection in the interstellar medium of the cyanomidyl radical (HNCN). Using the Yebes 40m and the IRAM 30m telescopes, we have targeted the doublets of the $N$=2$-$1, 4$-$3, 5$-$4, 6$-$5, and 7$-$6 transitions of HNCN toward the molecular cloud G+0.693-0.027. We have detected three unblended lines of HNCN, these are the $N$=6$-$5 doublet and one line of the $N$=4$-$3 transition. Additionally we present one line of the $N$=5$-$4 transition partially blended with emission from other species. The Local Thermodynamic Equilibrium best fit to the data gives a molecular abundance of (0.91$\pm$0.05)$\times$10$^{-10}$ with respect to H$_2$. The relatively low abundance of this species in G+0.693-0.027, and its high reactivity, suggest that HNCN is possibly produced by gas-phase chemistry. Our work shows that this highly reactive molecule is present in interstellar space, and thus it represents a plausible precursor of larger prebiotic molecules with the NCN backbone such as cyanamide (NH$_2$CN), carbodiimide (HNCNH) and formamidine (NH$_2$CHNH).


title: Work Generation from Thermal Noise by Quantum Phase-Sensitive Observation

abstract: We put forward the concept of work extraction from thermal noise by phase-sensitive (homodyne) measurements of the noisy input followed by (outcome-dependent) unitary manipulations of the post-measured state. For optimized measurements, noise input with more than one quantum on average is shown to yield heat-to-work conversion with efficiency and power that grow with the mean number of input quanta, detector efficiency and its inverse temperature. This protocol is shown to be advantageous compared to common models of information and heat engines.


title: Quasi-solid-state sodium-ion hybrid capacitors enabled by UiO-66@PVDF-HFP multifunctional separators: selective charge transfer and high safety

abstract: The practical application of sodium-ion hybrid capacitors is limited by their low energy densities resulted from the kinetics mismatch between cathodes and anodes, and the fire safety related to the flammable electrolyte-separator system. Hence, we report a rational design of metal-organic frameworks (MOFs, UiO-66) modified PVDF-HFP separator. High tensile strength and dimensional thermal stability of the separator reduce the risk of electrode short circuit caused by the separator deformation. MCC test demonstrates a reduction of 75% in peak heat release rate (pHRR), indicating an enhanced fire-resistant property of the separator. This is due to the transformation of UiO-66 into ZrO2 accompanied by the consumption of oxygen and the formation of the barrier char that suppresses further heat release. Quasi-solid-state electrolyte prepared based on this separator presents an enhanced ionic conductivity of 2.44 mS*cm-1 and Na-ion transference number of 0.55, which are related to the high porosity ( >70%) and electrolyte uptake (~ 320%) of the separator. Moreover, the open metal sites of UiO-66 can capture PF6- and consequently liberate the Na+ for faster migration, thus reducing the kinetics mismatch between cathodes and anodes. Such multifunctional separator enables the quasi-solid-state Na-ion hybrid capacitor to achieve high energy density (182 Wh*kg-1 @31 W*kg-1) and power density (5280 W*kg-1 @22 Wh*kg-1), as well as excellent cyclic stability (10000 cycles @1000 mA*g-1). Keywords: Quasi-solid-state; PVDF-HFP; Metal-organic frameworks; Dimensional thermal stability; Fire safety; Selective charge transfer


title: A geometrical description of the universal $1 \to 2$ asymmetric quantum cloning region

abstract: We consider the problem of determining the achievable region of parameters for universal $1 \to 2$ asymmetric quantum cloning. Measuring the cloning performance with the figure of merit of singlet fraction, we show that the physical region is a union of ellipses in the plane. Equivalently, we characterize the parameter region of quantum state compatibility of two possibly different isotropic states, considering, for the first time, negative singlet fractions.


title: Beam-based aperture measurements with movable collimator jaws as performance booster of the CERN Large Hadron Collider

abstract: The beam aperture of a particle accelerator defines the clearance available for the circulating beams and is a parameter of paramount importance for the accelerator performance. At the CERN Large Hadron Collider (LHC), the knowledge and control of the available aperture is crucial because the nominal proton beams carry an energy of 362 MJ stored in a superconducting environment. Even a tiny fraction of beam losses could quench the superconducting magnets or cause severe material damage. Furthermore, in a circular collider, the performance in terms of peak luminosity depends to a large extent on the aperture of the inner triplet quadrupoles, which are used to focus the beams at the interaction points. In the LHC, this aperture represents the smallest aperture at top-energy with squeezed beams and determines the maximum potential reach of the peak luminosity. Beam-based aperture measurements in these conditions are difficult and challenging. In this paper, we present different methods that have been developed over the years for precise beam-based aperture measurements in the LHC, highlighting applications and results that contributed to boost the operational LHC performance in Run 1 (2010-2013) and Run 2 (2015-2018).


title: Eikonal quasinormal modes of black holes beyond general relativity III: scalar Gauss-Bonnet gravity

abstract: In a recent series of papers we have shown how the eikonal/geometrical optics approximation can be used to calculate analytically the fundamental quasinormal mode frequencies associated with coupled systems of wave equations, which arise, for instance, in the study of perturbations of black holes in gravity theories beyond General Relativity. As a continuation to this series, we here focus on the quasinormal modes of nonrotating black holes in scalar Gauss-Bonnet gravity assuming a small-coupling expansion. We show that the axial perturbations are purely tensorial and are described by a modified Regge-Wheeler equation, while the polar perturbations are of mixed scalar-tensor character and are described by a system of two coupled wave equations. When applied to these equations, the eikonal machinery leads to axial modes that deviate from the general relativistic results at quadratic order in the Gauss-Bonnet coupling constant. We show that this result is in agreement with an analysis of unstable circular null orbits around blackholes in this theory, allowing us to establish the geometrical optics-null geodesic correspondence for the axial modes. For the polar modes the small-coupling approximation forces us to consider the ordering between eikonal and small-coupling perturbative parameters; one of which we show, by explicit comparison against numerical data, yields the correct identification of the quasinormal modes of the scalar-tensor coupled system of wave equations. These corrections lift the general relativistic degeneracy between scalar and tensorial eikonal quasinormal modes at quadratic order in Gauss-Bonnet coupling in a way reminiscent of the Zeeman effect. In general, our analytic, eikonal quasinormal mode frequencies (normalized to the General Relativity ones) agree with numerical results with an error of $\sim 10\%$ in the regime of small coupling constant. (abridged)


title: Non-intrusive Nonlinear Model Reduction via Machine Learning Approximations to Low-dimensional Operators

abstract: Although projection-based reduced-order models (ROMs) for parameterized nonlinear dynamical systems have demonstrated exciting results across a range of applications, their broad adoption has been limited by their intrusivity: implementing such a reduced-order model typically requires significant modifications to the underlying simulation code. To address this, we propose a method that enables traditionally intrusive reduced-order models to be accurately approximated in a non-intrusive manner. Specifically, the approach approximates the low-dimensional operators associated with projection-based reduced-order models (ROMs) using modern machine-learning regression techniques. The only requirement of the simulation code is the ability to export the velocity given the state and parameters as this functionality is used to train the approximated low-dimensional operators. In addition to enabling nonintrusivity, we demonstrate that the approach also leads to very low computational complexity, achieving up to $1000\times$ reduction in run time. We demonstrate the effectiveness of the proposed technique on two types of PDEs.


title: Robustness and Consistency in Linear Quadratic Control with Untrusted Predictions

abstract: We study the problem of learning-augmented predictive linear quadratic control. Our goal is to design a controller that balances \textit{"consistency"}, which measures the competitive ratio when predictions are accurate, and \textit{"robustness"}, which bounds the competitive ratio when predictions are inaccurate. We propose a novel $\lambda$-confident policy and provide a competitive ratio upper bound that depends on a trust parameter $\lambda\in [0,1]$ set based on the confidence in the predictions and some prediction error $\varepsilon$. Motivated by online learning methods, we design a self-tuning policy that adaptively learns the trust parameter $\lambda$ with a competitive ratio that depends on $\varepsilon$ and the variation of system perturbations and predictions. We show that its competitive ratio is bounded from above by $ 1+{O(\varepsilon)}/({{\Theta(1)+\Theta(\varepsilon)}})+O(\mu_{\mathsf{Var}})$ where $\mu_\mathsf{Var}$ measures the variation of perturbations and predictions. It implies that when the variations of perturbations and predictions are small, by automatically adjusting the trust parameter online, the self-tuning scheme ensures a competitive ratio that does not scale up with the prediction error $\varepsilon$.


title: WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis

abstract: This paper introduces WaveGrad 2, a non-autoregressive generative model for text-to-speech synthesis. WaveGrad 2 is trained to estimate the gradient of the log conditional density of the waveform given a phoneme sequence. The model takes an input phoneme sequence, and through an iterative refinement process, generates an audio waveform. This contrasts to the original WaveGrad vocoder which conditions on mel-spectrogram features, generated by a separate model. The iterative refinement process starts from Gaussian noise, and through a series of refinement steps (e.g., 50 steps), progressively recovers the audio sequence. WaveGrad 2 offers a natural way to trade-off between inference speed and sample quality, through adjusting the number of refinement steps. Experiments show that the model can generate high fidelity audio, approaching the performance of a state-of-the-art neural TTS system. We also report various ablation studies over different model configurations. Audio samples are available at https://wavegrad.github.io/v2.


title: A Fubini-type theorem for Hausdorff dimension

abstract: It is well known that a classical Fubini theorem for Hausdorff dimension cannot hold; that is, the dimension of the intersections of a fixed set with a parallel family of planes do not determine the dimension of the set. Here we prove that a Fubini theorem for Hausdorff dimension does hold modulo sets that are small on all Lipschitz graphs. We say that $G\subset \mathbb{R}^k\times \mathbb{R}^n$ is $\Gamma_k$-null if for every Lipschitz function $f:\mathbb{R}^k\to \mathbb{R}^n$ the set $\{t\in\mathbb{R}^k\,:\,(t,f(t))\in G\}$ has measure zero. We show that for every Borel set $E\subset \mathbb{R}^k\times \mathbb{R}^n$ with $\dim (\text{proj}_{\mathbb{R}^k} E)=k$ there is a $\Gamma_k$-null subset $G\subset E$ such that $$\dim (E\setminus G) = k+\text{ess-}\sup(\dim E_t)$$ where $\text{ess-}\sup(\dim E_t)$ is the essential supremum of the Hausdorff dimension of the vertical sections $\{E_t\}_{t\in \mathbb{R}^k}$ of $E$. In addition, we show that, provided that $E$ is not $\Gamma_k$-null, there is a $\Gamma_k$-null subset $G\subset E$ such that for $F=E \setminus G$, the Fubini-property holds, that is, $\dim (F) = k+\text{ess-}\sup(\dim F_t)$. We also obtain more general results by replacing $\mathbb{R}^k$ by an Ahlfors-David regular set. Applications of our results include Fubini-type results for unions of affine subspaces, connection to the Kakeya conjecture and projection theorems.


title: Automatic Segmentation of the Prostate on 3D Trans-rectal Ultrasound Images using Statistical Shape Models and Convolutional Neural Networks

abstract: In this work we propose to segment the prostate on a challenging dataset of trans-rectal ultrasound (TRUS) images using convolutional neural networks (CNNs) and statistical shape models (SSMs). TRUS is commonly used for a number of image-guided interventions on the prostate. Fast and accurate segmentation on the organ in these images is crucial to planning and fusion with other modalities such as magnetic resonance images (MRIs) . However, TRUS has limited soft tissue contrast and signal to noise ratio which makes the task of segmenting the prostate challenging and subject to inter-observer and intra-observer variability. This is especially problematic at the base and apex where the gland boundary is hard to define. In this paper, we aim to tackle this problem by taking advantage of shape priors learnt on an MR dataset which has higher soft tissue contrast allowing the prostate to be contoured more accurately. We use this shape prior in combination with a prostate tissue probability map computed by a CNN for segmentation.


title: A Short Note of PAGE: Optimal Convergence Rates for Nonconvex Optimization

abstract: In this note, we first recall the nonconvex problem setting and introduce the optimal PAGE algorithm (Li et al., ICML'21). Then we provide a simple and clean convergence analysis of PAGE for achieving optimal convergence rates. Moreover, PAGE and its analysis can be easily adopted and generalized to other works. We hope that this note provides the insights and is helpful for future works.


title: Design and Analysis of Robust Deep Learning Models for Stock Price Prediction

abstract: Building predictive models for robust and accurate prediction of stock prices and stock price movement is a challenging research problem to solve. The well-known efficient market hypothesis believes in the impossibility of accurate prediction of future stock prices in an efficient stock market as the stock prices are assumed to be purely stochastic. However, numerous works proposed by researchers have demonstrated that it is possible to predict future stock prices with a high level of precision using sophisticated algorithms, model architectures, and the selection of appropriate variables in the models. This chapter proposes a collection of predictive regression models built on deep learning architecture for robust and precise prediction of the future prices of a stock listed in the diversified sectors in the National Stock Exchange (NSE) of India. The Metastock tool is used to download the historical stock prices over a period of two years (2013- 2014) at 5 minutes intervals. While the records for the first year are used to train the models, the testing is carried out using the remaining records. The design approaches of all the models and their performance results are presented in detail. The models are also compared based on their execution time and accuracy of prediction.


title: Understanding the Effectiveness of Reviews in E-commerce Top-N Recommendation

abstract: Modern E-commerce websites contain heterogeneous sources of information, such as numerical ratings, textual reviews and images. These information can be utilized to assist recommendation. Through textual reviews, a user explicitly express her affinity towards the item. Previous researchers found that by using the information extracted from these reviews, we can better profile the users' explicit preferences as well as the item features, leading to the improvement of recommendation performance. However, most of the previous algorithms were only utilizing the review information for explicit-feedback problem i.e. rating prediction, and when it comes to implicit-feedback ranking problem such as top-N recommendation, the usage of review information has not been fully explored. Seeing this gap, in this work, we investigate the effectiveness of textual review information for top-N recommendation under E-commerce settings. We adapt several SOTA review-based rating prediction models for top-N recommendation tasks and compare them to existing top-N recommendation models from both performance and efficiency. We find that models utilizing only review information can not achieve better performances than vanilla implicit-feedback matrix factorization method. When utilizing review information as a regularizer or auxiliary information, the performance of implicit-feedback matrix factorization method can be further improved. However, the optimal model structure to utilize textual reviews for E-commerce top-N recommendation is yet to be determined.


title: Cyclic spacetimes through singularity scattering maps. Plane-symmetric gravitational collisions

abstract: We study the plane-symmetric collision of two gravitational waves and describe the global spacetime geometry generated by this collision. To this end, we formulate the characteristic initial value problem for the Einstein equations, when Goursat data describing the incoming waves are prescribed on two null hypersurfaces. We then construct a global solution representing a cyclic spacetime based on junction conditions associated with a prescribed singularity scattering map (a notion recently introduced by the authors). From a mathematical analysis standpoint, this amounts to a detailed analysis of the Goursat and Fuchsian initial value problems associated with singular hyperbolic equations, when junction conditions at interfaces are prescribed. In our construction, we introduce a partition into monotonicity diamonds (that is, suitable spacetime domains) and we construct the solution by concatenating domains across interfaces of timelike, null, or spacelike type.


title: Poisoning and Backdooring Contrastive Learning

abstract: Multimodal contrastive learning methods like CLIP train on noisy and uncurated training datasets. This is cheaper than labeling datasets manually, and even improves out-of-distribution robustness. We show that this practice makes backdoor and poisoning attacks a significant threat. By poisoning just 0.01% of a dataset (e.g., just 300 images of the 3 million-example Conceptual Captions dataset), we can cause the model to misclassify test images by overlaying a small patch. Targeted poisoning attacks, whereby the model misclassifies a particular test input with an adversarially-desired label, are even easier requiring control of 0.0001% of the dataset (e.g., just three out of the 3 million images). Our attacks call into question whether training on noisy and uncurated Internet scrapes is desirable.


title: Multi-modal fusion with gating using audio, lexical and disfluency features for Alzheimer's Dementia recognition from spontaneous speech

abstract: This paper is a submission to the Alzheimer's Dementia Recognition through Spontaneous Speech (ADReSS) challenge, which aims to develop methods that can assist in the automated prediction of severity of Alzheimer's Disease from speech data. We focus on acoustic and natural language features for cognitive impairment detection in spontaneous speech in the context of Alzheimer's Disease Diagnosis and the mini-mental state examination (MMSE) score prediction. We proposed a model that obtains unimodal decisions from different LSTMs, one for each modality of text and audio, and then combines them using a gating mechanism for the final prediction. We focused on sequential modelling of text and audio and investigated whether the disfluencies present in individuals' speech relate to the extent of their cognitive impairment. Our results show that the proposed classification and regression schemes obtain very promising results on both development and test sets. This suggests Alzheimer's Disease can be detected successfully with sequence modeling of the speech data of medical sessions.


title: Improving On-Screen Sound Separation for Open-Domain Videos with Audio-Visual Self-Attention

abstract: We introduce a state-of-the-art audio-visual on-screen sound separation system which is capable of learning to separate sounds and associate them with on-screen objects by looking at in-the-wild videos. We identify limitations of previous work on audio-visual on-screen sound separation, including the simplicity and coarse resolution of spatio-temporal attention, and poor convergence of the audio separation model. Our proposed model addresses these issues using cross-modal and self-attention modules that capture audio-visual dependencies at a finer resolution over time, and by unsupervised pre-training of audio separation model. These improvements allow the model to generalize to a much wider set of unseen videos. We also show a robust way to further improve the generalization capability of our models by calibrating the probabilities of our audio-visual on-screen classifier, using only a small amount of in-domain videos labeled for their on-screen presence. For evaluation and semi-supervised training, we collected human annotations of on-screen audio from a large database of in-the-wild videos (YFCC100m). Our results show marked improvements in on-screen separation performance, in more general conditions than previous methods.


title: Indian Masked Faces in the Wild Dataset

abstract: Due to the COVID-19 pandemic, wearing face masks has become a mandate in public places worldwide. Face masks occlude a significant portion of the facial region. Additionally, people wear different types of masks, from simple ones to ones with graphics and prints. These pose new challenges to face recognition algorithms. Researchers have recently proposed a few masked face datasets for designing algorithms to overcome the challenges of masked face recognition. However, existing datasets lack the cultural diversity and collection in the unrestricted settings. Country like India with attire diversity, people are not limited to wearing traditional masks but also clothing like a thin cotton printed towel (locally called as ``gamcha''), ``stoles'', and ``handkerchiefs'' to cover their faces. In this paper, we present a novel \textbf{Indian Masked Faces in the Wild (IMFW)} dataset which contains images with variations in pose, illumination, resolution, and the variety of masks worn by the subjects. We have also benchmarked the performance of existing face recognition models on the proposed IMFW dataset. Experimental results demonstrate the limitations of existing algorithms in presence of diverse conditions.


title: Pseudo-Euclidean Attract-Repel Embeddings for Undirected Graphs

abstract: Dot product embeddings take a graph and construct vectors for nodes such that dot products between two vectors give the strength of the edge. Dot products make a strong transitivity assumption, however, many important forces generating graphs in the real world lead to non-transitive relationships. We remove the transitivity assumption by embedding nodes into a pseudo-Euclidean space - giving each node an attract and a repel vector. The inner product between two nodes is defined by taking the dot product in attract vectors and subtracting the dot product in repel vectors. Pseudo-Euclidean embeddings can compress networks efficiently, allow for multiple notions of nearest neighbors each with their own interpretation, and can be `slotted' into existing models such as exponential family embeddings or graph neural networks for better link prediction.


title: The 2021 Image Similarity Dataset and Challenge

abstract: This paper introduces a new benchmark for large-scale image similarity detection. This benchmark is used for the Image Similarity Challenge at NeurIPS'21 (ISC2021). The goal is to determine whether a query image is a modified copy of any image in a reference corpus of size 1~million. The benchmark features a variety of image transformations such as automated transformations, hand-crafted image edits and machine-learning based manipulations. This mimics real-life cases appearing in social media, for example for integrity-related problems dealing with misinformation and objectionable content. The strength of the image manipulations, and therefore the difficulty of the benchmark, is calibrated according to the performance of a set of baseline approaches. Both the query and reference set contain a majority of "distractor" images that do not match, which corresponds to a real-life needle-in-haystack setting, and the evaluation metric reflects that. We expect the DISC21 benchmark to promote image copy detection as an important and challenging computer vision task and refresh the state of the art. Code and data are available at https://github.com/facebookresearch/isc2021


title: Equivariant maps to subshifts whose points have small stabilizers

abstract: Let $\Gamma$ be a countably infinite group. Given $k \in \mathbb{N}$, we use $\mathrm{Free}(k^\Gamma)$ to denote the free part of the Bernoulli shift action of $\Gamma$ on $k^\Gamma$. Seward and Tucker-Drob showed that there exists a free subshift $\mathcal{S} \subseteq \mathrm{Free}(2^\Gamma)$ such that every free Borel action of $\Gamma$ on a Polish space admits a Borel $\Gamma$-equivariant map to $\mathcal{S}$. Here we generalize this result as follows. Let $\mathcal{S}$ be a subshift of finite type (for example, $\mathcal{S}$ could be the set of all proper colorings of the Cayley graph of $\Gamma$ with some finite number of colors). Suppose that $\pi \colon \mathrm{Free}(k^\Gamma) \to \mathcal{S}$ is a continuous $\Gamma$-equivariant map and let $\mathrm{Stab}(\pi)$ be the set of all group elements that fix every point in the image of $\pi$. Unless $\pi$ is constant, $\mathrm{Stab}(\pi)$ is a finite normal subgroup of $\Gamma$. We prove that there exists a subshift $\mathcal{S}' \subseteq \mathcal{S}$ such that the stabilizer of every point in $\mathcal{S}'$ is $\mathrm{Stab}(\pi)$ and every free Borel action of $\Gamma$ on a Polish space admits a Borel $\Gamma$-equivariant map to $\mathcal{S}'$. In particular, if the shift action of $\Gamma$ on the image of $\pi$ is faithful (i.e., if $\mathrm{Stab}(\pi)$ is trivial), then the subshift $\mathcal{S}'$ is free. As an application of this general result, we deduce that if $F$ is a finite symmetric subset of $\Gamma \setminus \{\mathbf{1}\}$ of size $|F| = d \geq 1$ and $\mathrm{Col}(F, d + 1) \subseteq (d+1)^\Gamma$ is the set of all proper $(d+1)$-colorings of the Cayley graph of $\Gamma$ corresponding to $F$, then there is a free subshift $\mathcal{S} \subseteq \mathrm{Col}(F, d+1)$ such that every free Borel action of $\Gamma$ on a Polish space admits a Borel $\Gamma$-equivariant map to $\mathcal{S}$.


title: The Breaking of Geometric Constraint of Classical Dimers on the Square Lattice

abstract: We study a model of two-dimensional classical dimers on the square lattice with strong geometric constraints (there is exactly one bond with the nearest point for every point in the lattice). This model corresponds to the quantum dimer model suggested by D.S. Rokhsar and S.A. Kivelson (1988). We use the directed-loop algorithm to show the system undergoes a Berezinskii-Kostelitz Thousless transition (BKT transition) in finite temperatures. After that, if we destroy the geometric constraint of dimers, the topological transition will transfer to a quasi one-order transition. For the dimer updates, we also introduce a new cluster updating algorithm called the edged cluster algorithm. By this method, we succeed in rapidly traversing the winding (topological) sections uniformly and widening the effective matrical ensemble to include more topological sections.


title: Gone Fishing: Neural Active Learning with Fisher Embeddings

abstract: There is an increasing need for effective active learning algorithms that are compatible with deep neural networks. This paper motivates and revisits a classic, Fisher-based active selection objective, and proposes BAIT, a practical, tractable, and high-performing algorithm that makes it viable for use with neural models. BAIT draws inspiration from the theoretical analysis of maximum likelihood estimators (MLE) for parametric models. It selects batches of samples by optimizing a bound on the MLE error in terms of the Fisher information, which we show can be implemented efficiently at scale by exploiting linear-algebraic structure especially amenable to execution on modern hardware. Our experiments demonstrate that BAIT outperforms the previous state of the art on both classification and regression problems, and is flexible enough to be used with a variety of model architectures.


title: On ECP-Groups

abstract: According to T. Foguel a subgroup $H$ of a group $G$ is called conjugate-permutable if $ HH^x=H^xH$ for every $x\in G$. Mingyao Xu and Qinhai Zhang studied finite groups with every subgroup conjugate-permutable (ECP-groups) and asked three questions about them. We gave the answers on these questions. In particular, every group of exponent 3 is ECP-group, there exist non-regular ECP-$3$-groups and the class of all finite ECP-groups is neither formation nor variety.


title: Adaptive Low-Rank Regularization with Damping Sequences to Restrict Lazy Weights in Deep Networks

abstract: Overfitting is one of the critical problems in deep neural networks. Many regularization schemes try to prevent overfitting blindly. However, they decrease the convergence speed of training algorithms. Adaptive regularization schemes can solve overfitting more intelligently. They usually do not affect the entire network weights. This paper detects a subset of the weighting layers that cause overfitting. The overfitting recognizes by matrix and tensor condition numbers. An adaptive regularization scheme entitled Adaptive Low-Rank (ALR) is proposed that converges a subset of the weighting layers to their Low-Rank Factorization (LRF). It happens by minimizing a new Tikhonov-based loss function. ALR also encourages lazy weights to contribute to the regularization when epochs grow up. It uses a damping sequence to increment layer selection likelihood in the last generations. Thus before falling the training accuracy, ALR reduces the lazy weights and regularizes the network substantially. The experimental results show that ALR regularizes the deep networks well with high training speed and low resource usage.


title: SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies

abstract: Generalization has been a long-standing challenge for reinforcement learning (RL). Visual RL, in particular, can be easily distracted by irrelevant factors in high-dimensional observation space. In this work, we consider robust policy learning which targets zero-shot generalization to unseen visual environments with large distributional shift. We propose SECANT, a novel self-expert cloning technique that leverages image augmentation in two stages to decouple robust representation learning from policy optimization. Specifically, an expert policy is first trained by RL from scratch with weak augmentations. A student network then learns to mimic the expert policy by supervised learning with strong augmentations, making its representation more robust against visual variations compared to the expert. Extensive experiments demonstrate that SECANT significantly advances the state of the art in zero-shot generalization across 4 challenging domains. Our average reward improvements over prior SOTAs are: DeepMind Control (+26.5%), robotic manipulation (+337.8%), vision-based autonomous driving (+47.7%), and indoor object navigation (+15.8%). Code release and video are available at https://linxifan.github.io/secant-site/.


title: JOKR: Joint Keypoint Representation for Unsupervised Cross-Domain Motion Retargeting

abstract: The task of unsupervised motion retargeting in videos has seen substantial advancements through the use of deep neural networks. While early works concentrated on specific object priors such as a human face or body, recent work considered the unsupervised case. When the source and target videos, however, are of different shapes, current methods fail. To alleviate this problem, we introduce JOKR - a JOint Keypoint Representation that captures the motion common to both the source and target videos, without requiring any object prior or data collection. By employing a domain confusion term, we enforce the unsupervised keypoint representations of both videos to be indistinguishable. This encourages disentanglement between the parts of the motion that are common to the two domains, and their distinctive appearance and motion, enabling the generation of videos that capture the motion of the one while depicting the style of the other. To enable cases where the objects are of different proportions or orientations, we apply a learned affine transformation between the JOKRs. This augments the representation to be affine invariant, and in practice broadens the variety of possible retargeting pairs. This geometry-driven representation enables further intuitive control, such as temporal coherence and manual editing. Through comprehensive experimentation, we demonstrate the applicability of our method to different challenging cross-domain video pairs. We evaluate our method both qualitatively and quantitatively, and demonstrate that our method handles various cross-domain scenarios, such as different animals, different flowers, and humans. We also demonstrate superior temporal coherency and visual quality compared to state-of-the-art alternatives, through statistical metrics and a user study. Source code and videos can be found at https://rmokady.github.io/JOKR/ .


title: Accuracy, Interpretability, and Differential Privacy via Explainable Boosting

abstract: We show that adding differential privacy to Explainable Boosting Machines (EBMs), a recent method for training interpretable ML models, yields state-of-the-art accuracy while protecting privacy. Our experiments on multiple classification and regression datasets show that DP-EBM models suffer surprisingly little accuracy loss even with strong differential privacy guarantees. In addition to high accuracy, two other benefits of applying DP to EBMs are: a) trained models provide exact global and local interpretability, which is often important in settings where differential privacy is needed; and b) the models can be edited after training without loss of privacy to correct errors which DP noise may have introduced.


title: XCiT: Cross-Covariance Image Transformers

abstract: Following their success in natural language processing, transformers have recently shown much promise for computer vision. The self-attention operation underlying transformers yields global interactions between all tokens ,i.e. words or image patches, and enables flexible modelling of image data beyond the local interactions of convolutions. This flexibility, however, comes with a quadratic complexity in time and memory, hindering application to long sequences and high-resolution images. We propose a "transposed" version of self-attention that operates across feature channels rather than tokens, where the interactions are based on the cross-covariance matrix between keys and queries. The resulting cross-covariance attention (XCA) has linear complexity in the number of tokens, and allows efficient processing of high-resolution images. Our cross-covariance image transformer (XCiT) is built upon XCA. It combines the accuracy of conventional transformers with the scalability of convolutional architectures. We validate the effectiveness and generality of XCiT by reporting excellent results on multiple vision benchmarks, including image classification and self-supervised feature learning on ImageNet-1k, object detection and instance segmentation on COCO, and semantic segmentation on ADE20k.


title: On the chaoticity of derivatives

abstract: We show that the $n$th derivative with maximal domain is a chaotic operator in the spaces $C[a,b]$ and $L_p(a,b)$ ($-\infty<a<b<\infty$, $1\le p<\infty$) for each $n\in \mathbb{N}$.


title: PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classes

abstract: We give a novel, unified derivation of conditional PAC-Bayesian and mutual information (MI) generalization bounds. We derive conditional MI bounds as an instance, with special choice of prior, of conditional MAC-Bayesian (Mean Approximately Correct) bounds, itself derived from conditional PAC-Bayesian bounds, where `conditional' means that one can use priors conditioned on a joint training and ghost sample. This allows us to get nontrivial PAC-Bayes and MI-style bounds for general VC classes, something recently shown to be impossible with standard PAC-Bayesian/MI bounds. Second, it allows us to get faster rates of order $O \left(({\text{KL}}/n)^{\gamma}\right)$ for $\gamma > 1/2$ if a Bernstein condition holds and for exp-concave losses (with $\gamma=1$), which is impossible with both standard PAC-Bayes generalization and MI bounds. Our work extends the recent work by Steinke and Zakynthinou [2020] who handle MI with VC but neither PAC-Bayes nor fast rates, the recent work of Hellstr\"om and Durisi [2020] who extend the latter to the PAC-Bayes setting via a unifying exponential inequality, and Mhammedi et al. [2019] who initiated fast rate PAC-Bayes generalization error bounds but handle neither MI nor general VC classes.


title: LIV effects on the quantum stochastic motion in an acoustic FRW-geometry

abstract: It is well known in the literature that vacuum fluctuations can induce a random motion of particles which is sometimes called quantum Brownian motion or quantum stochastic motion. In this paper, we consider Lorentz Invariance Violation (LIV) in an acoustic spatially flat Friedman-Robertson-Walker (FRW) geometry. In particular, we are looking for the LIV effects in the stochastic motion of scalar and massive test particles. This motion is induced by a massless quantized scalar field on this geometry, which in turn is derived from an Abelian Higgs model with LIV. Deviations in the velocity dispersion of the particles proportional to the LIV parameter are found.


title: LoRA: Low-Rank Adaptation of Large Language Models

abstract: An important paradigm of natural language processing consists of large-scale pre-training on general domain data and adaptation to particular tasks or domains. As we pre-train larger models, full fine-tuning, which retrains all model parameters, becomes less feasible. Using GPT-3 175B as an example -- deploying independent instances of fine-tuned models, each with 175B parameters, is prohibitively expensive. We propose Low-Rank Adaptation, or LoRA, which freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks. Compared to GPT-3 175B fine-tuned with Adam, LoRA can reduce the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times. LoRA performs on-par or better than fine-tuning in model quality on RoBERTa, DeBERTa, GPT-2, and GPT-3, despite having fewer trainable parameters, a higher training throughput, and, unlike adapters, no additional inference latency. We also provide an empirical investigation into rank-deficiency in language model adaptation, which sheds light on the efficacy of LoRA. We release a package that facilitates the integration of LoRA with PyTorch models and provide our implementations and model checkpoints for RoBERTa, DeBERTa, and GPT-2 at https://github.com/microsoft/LoRA.


title: How Low Can We Go: Trading Memory for Error in Low-Precision Training

abstract: Low-precision arithmetic trains deep learning models using less energy, less memory and less time. However, we pay a price for the savings: lower precision may yield larger round-off error and hence larger prediction error. As applications proliferate, users must choose which precision to use to train a new model, and chip manufacturers must decide which precisions to manufacture. We view these precision choices as a hyperparameter tuning problem, and borrow ideas from meta-learning to learn the tradeoff between memory and error. In this paper, we introduce Pareto Estimation to Pick the Perfect Precision (PEPPP). We use matrix factorization to find non-dominated configurations (the Pareto frontier) with a limited number of network evaluations. For any given memory budget, the precision that minimizes error is a point on this frontier. Practitioners can use the frontier to trade memory for error and choose the best precision for their goals.


title: Super-Acceleration with Cyclical Step-sizes

abstract: We develop a convergence-rate analysis of momentum with cyclical step-sizes. We show that under some assumption on the spectral gap of Hessians in machine learning, cyclical step-sizes are provably faster than constant step-sizes. More precisely, we develop a convergence rate analysis for quadratic objectives that provides optimal parameters and shows that cyclical learning rates can improve upon traditional lower complexity bounds. We further propose a systematic approach to design optimal first order methods for quadratic minimization with a given spectral structure. Finally, we provide a local convergence rate analysis beyond quadratic minimization for the proposed methods and illustrate our findings through benchmarks on least squares and logistic regression problems.


title: A Ramsey-Tur\'an theory for tilings in graphs

abstract: For a $k$-vertex graph $F$ and an $n$-vertex graph $G$, an $F$-tiling in $G$ is a collection of vertex-disjoint copies of $F$ in $G$. For $r\in \mathbb{N}$, the $r$-independence number of $G$, denoted $\alpha_r(G)$ is the largest size of a $K_r$-free set of vertices in $G$. In this paper, we discuss Ramsey--Tur\'an-type theorems for tilings where one is interested in minimum degree and independence number conditions (and the interaction between the two) that guarantee the existence of optimal $F$-tilings. For cliques, we show that for any $k\geq 3$ and $\eta>0$, any graph $G$ on $n$ vertices with $\delta(G)\geq \eta n$ and $\alpha_k(G)=o(n)$ has a $K_k$-tiling covering all but $\lfloor\tfrac{1}{\eta}\rfloor(k-1)$ vertices. All conditions in this result are tight; the number of vertices left uncovered can not be improved and for $\eta<\tfrac{1}{k}$, a condition of $\alpha_{k-1}(G)=o(n)$ would not suffice. When $\eta>\tfrac{1}{k}$, we then show that $\alpha_{k-1}(G)=o(n)$ does suffice, but not $\alpha_{k-2}(G)=o(n)$. These results unify and generalise previous results of Balogh-Molla-Sharifzadeh, Nenadov-Pehova and Balogh-McDowell-Molla-Mycroft on the subject. We further explore the picture when $F$ is a tree or a cycle and discuss the effect of replacing the independence number condition with $\alpha^*(G)=o(n)$ (meaning that any pair of disjoint linear sized sets induce an edge between them) where one can force perfect $F$-tilings covering all the vertices. Finally we discuss the consequences of these results in the randomly perturbed setting.


title: Statistical Query Lower Bounds for List-Decodable Linear Regression

abstract: We study the problem of list-decodable linear regression, where an adversary can corrupt a majority of the examples. Specifically, we are given a set $T$ of labeled examples $(x, y) \in \mathbb{R}^d \times \mathbb{R}$ and a parameter $0< \alpha <1/2$ such that an $\alpha$-fraction of the points in $T$ are i.i.d. samples from a linear regression model with Gaussian covariates, and the remaining $(1-\alpha)$-fraction of the points are drawn from an arbitrary noise distribution. The goal is to output a small list of hypothesis vectors such that at least one of them is close to the target regression vector. Our main result is a Statistical Query (SQ) lower bound of $d^{\mathrm{poly}(1/\alpha)}$ for this problem. Our SQ lower bound qualitatively matches the performance of previously developed algorithms, providing evidence that current upper bounds for this task are nearly best possible.


title: Observing $t\bar{t}Z$ spin correlations at the LHC

abstract: Spin correlations in the production of top-antitop quark ($t\bar{t}$) pairs at the Large Hadron Collider (LHC) are an experimentally verified prediction of the Standard Model. In this paper, we compute the full spin density matrix for $t\bar{t}Z$ production at next-to-leading order precision in QCD, for center-of-mass energies of 13 and 14 TeV. We find that the additional emission of a $Z$ boson leads to significantly different spin correlations with respect to the $t\bar{t}$ case, and induces small longitudinal polarisations of the top quarks. We further propose an analysis strategy that could lead to the observation of spin correlations in $t\bar{t}Z$ events at the end of Run 3 of the LHC, or possibly earlier by combining the ATLAS and CMS datasets. In addition, we show that the pure angular information contained in the spin density matrix provides novel constraints on the dimension-6 effective field theory (EFT) operators relevant to the $t$-$Z$ interaction, without any reference to the total production rates.


title: Interactive Change Point Detection using optimisation approach and Bayesian statistics applied to real world applications

abstract: Change point detection becomes more and more important as datasets increase in size, where unsupervised detection algorithms can help users process data. To detect change points, a number of unsupervised algorithms have been developed which are based on different principles. One approach is to define an optimisation problem and minimise a cost function along with a penalty function. In the optimisation approach, the choice of the cost function affects the predictions made by the algorithm. In extension to the existing studies, a new type of cost function using Tikhonov regularisation is introduced. Another approach uses Bayesian statistics to calculate the posterior probability distribution of a specific point being a change point. It uses a priori knowledge on the distance between consecutive change points and a likelihood function with information about the segments. The optimisation and Bayesian approaches for offline change point detection are studied and applied to simulated datasets as well as a real world multi-phase dataset. The approaches have previously been studied separately and a novelty lies in comparing the predictions made by the two approaches in a specific setting, consisting of simulated datasets and a real world example. The study has found that the performance of the change point detection algorithms are affected by the features in the data.


title: Hi-Phy: A Benchmark for Hierarchical Physical Reasoning

abstract: Reasoning about the behaviour of physical objects is a key capability of agents operating in physical worlds. Humans are very experienced in physical reasoning while it remains a major challenge for AI. To facilitate research addressing this problem, several benchmarks have been proposed recently. However, these benchmarks do not enable us to measure an agent's granular physical reasoning capabilities when solving a complex reasoning task. In this paper, we propose a new benchmark for physical reasoning that allows us to test individual physical reasoning capabilities. Inspired by how humans acquire these capabilities, we propose a general hierarchy of physical reasoning capabilities with increasing complexity. Our benchmark tests capabilities according to this hierarchy through generated physical reasoning tasks in the video game Angry Birds. This benchmark enables us to conduct a comprehensive agent evaluation by measuring the agent's granular physical reasoning capabilities. We conduct an evaluation with human players, learning agents, and heuristic agents and determine their capabilities. Our evaluation shows that learning agents, with good local generalization ability, still struggle to learn the underlying physical reasoning capabilities and perform worse than current state-of-the-art heuristic agents and humans. We believe that this benchmark will encourage researchers to develop intelligent agents with advanced, human-like physical reasoning capabilities. URL: https://github.com/Cheng-Xue/Hi-Phy


title: Orthogonal-Pad\'e Activation Functions: Trainable Activation functions for smooth and faster convergence in deep networks

abstract: We have proposed orthogonal-Pad\'e activation functions, which are trainable activation functions and show that they have faster learning capability and improves the accuracy in standard deep learning datasets and models. Based on our experiments, we have found two best candidates out of six orthogonal-Pad\'e activations, which we call safe Hermite-Pade (HP) activation functions, namely HP-1 and HP-2. When compared to ReLU, HP-1 and HP-2 has an increment in top-1 accuracy by 5.06% and 4.63% respectively in PreActResNet-34, by 3.02% and 2.75% respectively in MobileNet V2 model on CIFAR100 dataset while on CIFAR10 dataset top-1 accuracy increases by 2.02% and 1.78% respectively in PreActResNet-34, by 2.24% and 2.06% respectively in LeNet, by 2.15% and 2.03% respectively in Efficientnet B0.


title: Simulation study on the fleet performance of shared autonomous bicycles

abstract: Rethinking cities is now more imperative than ever, as society faces global challenges such as population growth and climate change. The design of cities can not be abstracted from the design of its mobility system, and, therefore, efficient solutions must be found to transport people and goods throughout the city in an ecological way. An autonomous bicycle-sharing system would combine the most relevant benefits of vehicle sharing, electrification, autonomy, and micro-mobility, increasing the efficiency and convenience of bicycle-sharing systems and incentivizing more people to bike and enjoy their cities in an environmentally friendly way. Due to the uniqueness and radical novelty of introducing autonomous driving technology into bicycle-sharing systems and the inherent complexity of these systems, there is a need to quantify the potential impact of autonomy on fleet performance and user experience. This paper presents an ad-hoc agent-based simulator that provides an in-depth understanding of the fleet behavior of autonomous bicycle-sharing systems in realistic scenarios, including a rebalancing system based on demand prediction. In addition, this work describes the impact of different parameters on system efficiency and service quality and quantifies the extent to which an autonomous system would outperform current bicycle-sharing schemes. The obtained results show that with a fleet size three and a half times smaller than a station-based system and eight times smaller than a dockless system, an autonomous system can provide overall improved performance and user experience even with no rebalancing. These findings indicate that the remarkable efficiency of an autonomous bicycle-sharing system could compensate for the additional cost of autonomous bicycles.


title: Search for the Decay $B_s^0 \rightarrow \eta^\prime \eta$

abstract: We report the results of the first search for the decay $B_s^0 \rightarrow \eta^\prime \eta$ using $121.4~\textrm{fb}^{-1}$ of data collected at the $\Upsilon(5S)$ resonance with the Belle detector at the KEKB asymmetric-energy $e^+e^-$ collider. We observe no significant signal and set a 90\% confidence-level upper limit of %$7.1 \times 10^{-5}$ $6.5 \times 10^{-5}$ on the branching fraction of this decay.


title: BABEL: Bodies, Action and Behavior with English Labels

abstract: Understanding the semantics of human movement -- the what, how and why of the movement -- is an important problem that requires datasets of human actions with semantic labels. Existing datasets take one of two approaches. Large-scale video datasets contain many action labels but do not contain ground-truth 3D human motion. Alternatively, motion-capture (mocap) datasets have precise body motions but are limited to a small number of actions. To address this, we present BABEL, a large dataset with language labels describing the actions being performed in mocap sequences. BABEL consists of action labels for about 43 hours of mocap sequences from AMASS. Action labels are at two levels of abstraction -- sequence labels describe the overall action in the sequence, and frame labels describe all actions in every frame of the sequence. Each frame label is precisely aligned with the duration of the corresponding action in the mocap sequence, and multiple actions can overlap. There are over 28k sequence labels, and 63k frame labels in BABEL, which belong to over 250 unique action categories. Labels from BABEL can be leveraged for tasks like action recognition, temporal action localization, motion synthesis, etc. To demonstrate the value of BABEL as a benchmark, we evaluate the performance of models on 3D action recognition. We demonstrate that BABEL poses interesting learning challenges that are applicable to real-world scenarios, and can serve as a useful benchmark of progress in 3D action recognition. The dataset, baseline method, and evaluation code is made available, and supported for academic research purposes at https://babel.is.tue.mpg.de/.


title: The non-resonant bilinear Hilbert--Carleson operator

abstract: In this paper we introduce the class of bilinear Hilbert--Carleson operators $\{BC^a\}_{a>0}$ defined by $$ BC^{a}(f,g)(x):= \sup_{\lambda\in {\mathbb R}} \Big|\int f(x-t)\, g(x+t)\, e^{i\lambda t^a} \, \frac{dt}{t} \Big| $$ and show that in the non-resonant case $a\in (0,\infty)\setminus\{1,2\}$ the operator $BC^a$ extends continuously from $L^p({\mathbb R})\times L^q({\mathbb R})$ into $L^r({\mathbb R})$ whenever $\frac{1}{p}+\frac{1}{q}=\frac{1}{r}$ with $1<p,\,q\leq\infty$ and $\frac{2}{3}<r<\infty$. A key novel feature of these operators is that -- in the non-resonant case -- $BC^{a}$ has a \emph{hybrid} nature enjoying both (1) ``zero curvature'' features inherited from the modulation invariance property of the classical bilinear Hilbert transform (BHT), and (2) ``non-zero curvature'' features arising from the Carleson-type operator with nonlinear phase $\lambda t^a$.


title: Interaction-driven breakdown of dynamical localization in a kicked quantum gas

abstract: Quantum interference can terminate energy growth in a continually kicked system, via a single-particle ergodicity-breaking mechanism known as dynamical localization. The effect of many-body interactions on dynamically localized states, while important to a fundamental understanding of quantum decoherence, has remained unexplored despite a quarter-century of experimental studies. We report the experimental realization of a tunably-interacting kicked quantum rotor ensemble using a Bose-Einstein condensate in a pulsed optical lattice. We observe signatures of a prethermal localized plateau, followed for interacting samples by interaction-induced anomalous diffusion with an exponent near one half. Echo-type time reversal experiments establish the role of interactions in destroying reversibility. These results quantitatively elucidate the dynamical transition to many-body quantum chaos, advance our understanding of quantum anomalous diffusion, and delimit some possibilities for protecting quantum information in interacting driven systems.


title: The Kinematic Properties of Milky Way Stellar Halo Populations

abstract: In the $Gaia$ era stellar kinematics are extensively used to study Galactic halo stellar populations, to search for halo structures, and to characterize the interface between the halo and hot disc populations. We use distribution function-based models of modern datasets with 6D phase space data to qualitatively describe a variety of kinematic spaces commonly used in the study of the Galactic halo. Furthermore, we quantitatively assess how well each kinematic space can separate radially anisotropic from isotropic halo populations. We find that scaled action space (the ``action diamond'') is superior to other commonly used kinematic spaces at this task. We present a new, easy to implement selection criterion for members of the radially-anisotropic $Gaia$-Enceladus merger remnant. Assuming a 1:1 ratio of $Gaia$-Enceladus stars to more isotropic halo, we find our selection achieves a sample purity of 82 per cent in our models with respect to contamination from the more isotropic halo. We compare this criterion to literature criteria, finding that it produces the highest purity in the resulting samples, at the expense of a modest reduction in completeness. We also show that selection biases that underlie nearly all contemporary spectroscopic datasets can noticeably impact the $E-L_{z}$ distribution of samples in a manner that may be confused for real substructure. We conclude by providing recommendations for how authors should use stellar kinematics in the future to study the Galactic stellar halo.


title: Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study

abstract: Biomedical knowledge graphs (KGs) hold rich information on entities such as diseases, drugs, and genes. Predicting missing links in these graphs can boost many important applications, such as drug design and repurposing. Recent work has shown that general-domain language models (LMs) can serve as "soft" KGs, and that they can be fine-tuned for the task of KG completion. In this work, we study scientific LMs for KG completion, exploring whether we can tap into their latent knowledge to enhance biomedical link prediction. We evaluate several domain-specific LMs, fine-tuning them on datasets centered on drugs and diseases that we represent as KGs and enrich with textual entity descriptions. We integrate the LM-based models with KG embedding models, using a router method that learns to assign each input example to either type of model and provides a substantial boost in performance. Finally, we demonstrate the advantage of LM models in the inductive setting with novel scientific entities. Our datasets and code are made publicly available.


title: Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning

abstract: Modern computer vision applications suffer from catastrophic forgetting when incrementally learning new concepts over time. The most successful approaches to alleviate this forgetting require extensive replay of previously seen data, which is problematic when memory constraints or data legality concerns exist. In this work, we consider the high-impact problem of Data-Free Class-Incremental Learning (DFCIL), where an incremental learning agent must learn new concepts over time without storing generators or training data from past tasks. One approach for DFCIL is to replay synthetic images produced by inverting a frozen copy of the learner's classification model, but we show this approach fails for common class-incremental benchmarks when using standard distillation strategies. We diagnose the cause of this failure and propose a novel incremental distillation strategy for DFCIL, contributing a modified cross-entropy training and importance-weighted feature distillation, and show that our method results in up to a 25.1% increase in final task accuracy (absolute difference) compared to SOTA DFCIL methods for common class-incremental benchmarks. Our method even outperforms several standard replay based methods which store a coreset of images.