title: Future Artificial Intelligence tools and perspectives in medicine

abstract: Purpose of review: Artificial intelligence (AI) has become popular in medical applications, specifically as a clinical support tool for computer-aided diagnosis. These tools are typically employed on medical data (i.e., image, molecular data, clinical variables, etc.) and used the statistical and machine learning methods to measure the model performance. In this review, we summarized and discussed the most recent radiomic pipeline used for clinical analysis. Recent findings:Currently, limited management of cancers benefits from artificial intelligence, mostly related to a computer-aided diagnosis that avoids a biopsy analysis that presents additional risks and costs. Most AI tools are based on imaging features, known as radiomic analysis that can be refined into predictive models in non-invasively acquired imaging data. This review explores the progress of AI-based radiomic tools for clinical applications with a brief description of necessary technical steps. Explaining new radiomic approaches based on deep learning techniques will explain how the new radiomic models (deep radiomic analysis) can benefit from deep convolutional neural networks and be applied on limited data sets. Summary: To consider the radiomic algorithms, further investigations are recommended to involve deep learning in radiomic models with additional validation steps on various cancer types.


title: Inerter-based Elastic Metamaterials for Band Gap at Extremely Low Frequency

abstract: We reveal the unique and fundamental advantage of inerter-based elastic metamaterials by a comparative study among different configurations. When the embedded inerter is connected to the matrix material on both ends, the metamaterial shows definite superiority in forming a band gap in the ultra-low frequency -- equivalently the ultra-long wavelength -- regime, where the unit cell size can be four or more orders of magnitude smaller than the operating wavelength. In addition, our parametric studies in both one and two dimensions pave the way towards designing next-generation metamaterials for structural vibration mitigation.


title: GAAF: Searching Activation Functions for Binary Neural Networks through Genetic Algorithm

abstract: Binary neural networks (BNNs) show promising utilization in cost and power-restricted domains such as edge devices and mobile systems. This is due to its significantly less computation and storage demand, but at the cost of degraded performance. To close the accuracy gap, in this paper we propose to add a complementary activation function (AF) ahead of the sign based binarization, and rely on the genetic algorithm (GA) to automatically search for the ideal AFs. These AFs can help extract extra information from the input data in the forward pass, while allowing improved gradient approximation in the backward pass. Fifteen novel AFs are identified through our GA-based search, while most of them show improved performance (up to 2.54% on ImageNet) when testing on different datasets and network models. Our method offers a novel approach for designing general and application-specific BNN architecture. Our code is available at http://github.com/flying-Yan/GAAF.


title: Learning-Based Motion Planning with Mixture Density Networks

abstract: The trade-off between computation time and path optimality is a key consideration in motion planning algorithms. While classical sampling based algorithms fall short of computational efficiency in high dimensional planning, learning based methods have shown great potential in achieving time efficient and optimal motion planning. The SOTA learning based motion planning algorithms utilize paths generated by sampling based methods as expert supervision data and train networks via regression techniques. However, these methods often overlook the important multimodal property of the optimal paths in the training set, making them incapable of finding good paths in some scenarios. In this paper, we propose a Multimodal Neuron Planner (MNP) based on the mixture density networks that explicitly takes into account the multimodality of the training data and simultaneously achieves time efficiency and path optimality. For environments represented by a point cloud, MNP first efficiently compresses the point cloud into a latent vector by encoding networks that are suitable for processing point clouds. We then design multimodal planning networks which enables MNP to learn and predict multiple optimal solutions. Simulation results show that our method outperforms SOTA learning based method MPNet and advanced sampling based methods IRRT* and BIT*.


title: Joint Manifold Learning and Density Estimation Using Normalizing Flows

abstract: Based on the manifold hypothesis, real-world data often lie on a low-dimensional manifold, while normalizing flows as a likelihood-based generative model are incapable of finding this manifold due to their structural constraints. So, one interesting question arises: $\textit{"Can we find sub-manifold(s) of data in normalizing flows and estimate the density of the data on the sub-manifold(s)?"}$. In this paper, we introduce two approaches, namely per-pixel penalized log-likelihood and hierarchical training, to answer the mentioned question. We propose a single-step method for joint manifold learning and density estimation by disentangling the transformed space obtained by normalizing flows to manifold and off-manifold parts. This is done by a per-pixel penalized likelihood function for learning a sub-manifold of the data. Normalizing flows assume the transformed data is Gaussianizationed, but this imposed assumption is not necessarily true, especially in high dimensions. To tackle this problem, a hierarchical training approach is employed to improve the density estimation on the sub-manifold. The results validate the superiority of the proposed methods in simultaneous manifold learning and density estimation using normalizing flows in terms of generated image quality and likelihood.


title: A diagrammatic calculus for categorical quantum protocols

abstract: As shown by Abramsky and Coecke, quantum mechanics can be studied in terms of dagger compact closed categories with biproducts. Within this structure, many well-known quantum protocols can be described and their validity can be shown by establishing the commutativity of certain diagrams in that category. In this paper, we propose an explicit realisation of a category with enough structure to check the validity of a certain class of quantum protocols. In order to do this, we construct a category based on 1-dimensional cobordisms with attached elements of a certain group freely generated by a finite set. We use this category as a graphical language, and we show that it is dagger compact closed with biproducts. Then, relying on the coherence result for compact closed categories, proved by Kelly and Laplaza, we show the coherence result, which enables us to check the validity of quantum protocols just by drawing diagrams. In particular, we show the validity of quantum teleportation, entanglement swapping (as formulated in the work of Abramsky and Coecke) and superdense coding protocol.


title: Singularities of normal quartic surfaces III (char=2, non-supersingular)

abstract: We show that the maximal number of singular points of a normal quartic surface $X \subset \mathbb{P}^3_K$ defined over an algebraically closed field $K$ of characteristic 2 is at most 12, if the minimal resolution of $X$ is not a supersingular K3 surface. We also provide a family of explicit examples, valid in any characteristic.


title: Multi-wavelength Variation Phenomena of PKS 0735+178 on Diverse Timescale

abstract: BL Lac object PKS 0735+178 showed some complex multi-wavelength variation phenomena in the previous studies, especially for its color behavior. Bluer-when-brighter, redder-when-brighter and achromatic behavior were all found to be possible long-term trends of PKS 0735+178. In this work, we collected the long-term multi-wavelength data of PKS 0735+178, and also performed a multi-color optical monitoring on intraday timescale. The intraday variability was detected on one night. On the long timescale, a possible 22-day time lag was found between the $R$ and $\gamma$-ray bands. The results of the cross-correlation analysis exhibited strong correlations between various optical bands on both intraday and long timescale. However, only a mild correlation was found between the long-term $\gamma$-ray and $R$-band light curves, which could be interpreted by the different emission mechanisms of the $\gamma$-ray and optical emissions. PKS 0735+178 showed a significant harder-when-brighter in the $\gamma$-ray band, which is consistent with the observed optical bluer-when-brighter trend on both long-term and intraday timescales. We found that the HWB and BWB trends will be enhanced during the active states, especially for the historical low state. Such phenomenon indicates a special activity-dependent color behavior of PKS 0735+178, and it could be well interpreted by the jet emission model.


title: Random singlet-like phase of disordered Hubbard chains

abstract: Local moment formation is ubiquitous in disordered semiconductors such as Si:P, where it is observed both in the metallic and the insulating regimes. Here, we focus on local moment behavior in disordered insulators, which arises from short-ranged, repulsive electron-electron interactions. Using density matrix renormalization group and strong-disorder renormalization group methods, we study paradigmatic models of interacting insulators: one dimensional Hubbard chains with quenched randomness. In chains with either random fermion hoppings or random chemical potentials, both at and away from half-filling, we find exponential decay of charge and fermion 2-point correlations but power-law decay of spin correlations that are indicative of the random singlet phase. The numerical results can be understood qualitatively by appealing to the large-interaction limit of the Hubbard chain, in which a remarkably simple picture emerges.


title: Spatiotemporal Pricing and Fleet Management of Autonomous Mobility-on-Demand Networks: A Decomposition and Dynamic Programming Approach with Bounded Optimality Gap

abstract: This paper studies spatiotemporal pricing and fleet management for autonomous mobility-on-demand (AMoD) systems while taking elastic demand into account. We consider a platform that offers ride-hailing services using a fleet of autonomous vehicles and makes pricing, rebalancing, and fleet sizing decisions in response to demand fluctuations. A network flow model is developed to characterize the evolution of system states over space and time, which captures the vehicle-passenger matching process and demand elasticity with respect to price and waiting time. The platform's objective of maximizing profit is formulated as a constrained optimal control problem, which is highly nonconvex due to the nonlinear demand model and complex supply-demand interdependence. To address this challenge, an integrated decomposition and dynamic programming approach is proposed, where we first relax the problem through a change of variable, then separate the relaxed problem into a few small-scale subproblems via dual decomposition, and finally solve each subproblem using dynamic programming. Despite the nonconvexity, our approach establishes a theoretical upper bound to evaluate the solution optimality. The proposed model and methodology are validated in numerical studies for Manhattan. We find that compared to the benchmark case, the proposed upper bound is significantly tighter. We also find that compared to pricing alone, joint pricing and fleet rebalancing can only offer a minor profit improvement when demand can be accurately predicted. However, during unanticipated demand surges, joint pricing and rebalancing can lead to substantially improved profits, and the impacts of demand shocks, despite being more widespread, can dissipate faster.


title: Generalization Error Bounds for Deep Neural Networks Trained by SGD

abstract: Generalization error bounds for deep neural networks trained by stochastic gradient descent (SGD) are derived by combining a dynamical control of an appropriate parameter norm and the Rademacher complexity estimate based on parameter norms. The bounds explicitly depend on the loss along the training trajectory, and work for a wide range of network architectures including multilayer perceptron (MLP) and convolutional neural networks (CNN). Compared with other algorithm-depending generalization estimates such as uniform stability-based bounds, our bounds do not require $L$-smoothness of the nonconvex loss function, and apply directly to SGD instead of Stochastic Langevin gradient descent (SGLD). Numerical results show that our bounds are non-vacuous and robust with the change of optimizer and network hyperparameters.


title: Optimizing spectral stacking for 21-cm observations of galaxies: accuracy assessment and symmetrized stacking

abstract: We present an assessment of the accuracy of common operations performed in $21$-cm spectral line stacking experiments. To this end, we generate mock interferometric data surveying the 21-cm emission at frequency $1310<\nu<1420$ MHz ($0.005<z<0.084$) and covering an area $\sim 6$ deg$^2$ of the sky, mimicking the observational characteristics of real MeerKAT observations. We find that the primary beam correction accounts for just few per cent ($\sim8\%$ at 0 primary beam power, $\sim 3\%$ at 0.6 primary beam power) deviations from the true $M_{\rm HI}$ signal, and that weighting schemes based on noise properties provide unbiased results. On the contrary, weighting schemes based on distance can account for significant systematic mass differences when applied to a flux-limited sample ($\Delta M_{\rm HI}\sim 40-50\%$ in the studied case). We find no significant difference in the final $\braket{M_{\rm HI}}$ obtained when spectroscopic redshift uncertainties are accounted for in the stacking procedure ($ \Delta z\sim 0.00035$, i.e. $\Delta v \sim 100\,{\rm km\, s}^{-1}$). We also present a novel technique to increase the effective size of the galaxy sample by exploiting the geometric symmetries of galaxy cubelets, potentially enhancing the SNR by a factor $\sim\sqrt{2}$ when analyzing the final stacked spectrum (a factor 4 in a cubelet). This procedure is found to be robustly unbiased, while efficiently increasing the SNR, as expected. We argue that an appropriate framework employing detailed and realistic simulations is required to exploit upcoming datasets from SKA pathfinders in an accurate and reliable manner.


title: Recent Advances in Bayesian Optimization

abstract: Bayesian optimization has emerged at the forefront of expensive black-box optimization due to its data efficiency. Recent years have witnessed a proliferation of studies on the development of new Bayesian optimization algorithms and their applications. Hence, this paper attempts to provide a comprehensive and updated survey of recent advances in Bayesian optimization and identify interesting open problems. We categorize the existing work on Bayesian optimization into nine main groups according to the motivations and focus of the proposed algorithms. For each category, we present the main advances with respect to the construction of surrogate models and adaptation of the acquisition functions. Finally, we discuss the open questions and suggest promising future research directions, in particular with regard to heterogeneity, privacy preservation, and fairness in distributed and federated optimization systems.


title: Analytic solution and Noether symmetries for the hyperbolic inflationary model in the Jordan frame

abstract: The Noether symmetry analysis is applied for the study of a multifield cosmological model in a spatially flat FLRW background geometry. The gravitational Action Integral consists of two scalar fields, the Brans-Dicke field and a second scalar field minimally coupled to gravity. However, the two scalar fields interact in the kinetic terms. This multifield has been found to describe the equivalent of hyperbolic inflation in the Jordan frame. The application of Noether's theorems constrain the free parameters of the model that conservation laws exist. We find that the field equations form an integrable dynamical system and the analytic solution is derived.


title: Magnetar Engines in Fast Blue Optical Transients and Their Connections with SLSNe, SNe Ic-BL, and lGRBs

abstract: We fit the multi-band lightcurves of 40 fast blue optical transients (FBOTs) with the magnetar engine model. The mass of the FBOT ejecta, the initial spin period and polar magnetic field of the FBOT magnetars are respectively constrained to $M_{\rm{ej}}=0.18^{+0.52}_{-0.13}\,M_\odot$, $P_{\rm{i}}=9.4^{+8.1}_{-3.9}\,{\rm{ms}}$, and $B_{\rm p}=7^{+16}_{-5}\times10^{14}\,{\rm{G}}$. The wide distribution of the value of $B_{\rm p}$ spreads the parameter ranges of the magnetars from superluminous supernovae (SLSNe) to broad-line Type Ic supernovae (SNe Ic-BL; some are observed to be associated with long-duration gamma-ray bursts), which are also suggested to be driven by magnetars. Combining FBOTs with the other transients, we find a strong universal anti-correlation as $P_{\rm{i}}\propto{M_{\rm{ej}}^{-0.45}}$, indicating them could share a common origin. To be specific, it is suspected that all of these transients originate from collapse of extreme-stripped stars in close binary systems, but with different progenitor masses. As a result, FBOTs distinct themselves by their small ejecta masses with an upper limit of ${\sim}1\,M_\odot$, which leads to an observational separation in the rise time of the lightcurves $\sim12\,{\rm d}$. In addition, the FBOTs together with SLSNe can be separated from SNe Ic-BL by an empirical line in the $M_{\rm peak}-t_{\rm rise}$ plane corresponding to an energy requirement of a mass of $^{56}$Ni of $\sim0.3M_{\rm ej}$, where $M_{\rm peak}$ is the peak absolute magnitude of the transients and $t_{\rm rise}$ is the rise time.


title: On the balance between the training time and interpretability of neural ODE for time series modelling

abstract: Most machine learning methods are used as a black box for modelling. We may try to extract some knowledge from physics-based training methods, such as neural ODE (ordinary differential equation). Neural ODE has advantages like a possibly higher class of represented functions, the extended interpretability compared to black-box machine learning models, ability to describe both trend and local behaviour. Such advantages are especially critical for time series with complicated trends. However, the known drawback is the high training time compared to the autoregressive models and long-short term memory (LSTM) networks widely used for data-driven time series modelling. Therefore, we should be able to balance interpretability and training time to apply neural ODE in practice. The paper shows that modern neural ODE cannot be reduced to simpler models for time-series modelling applications. The complexity of neural ODE is compared to or exceeds the conventional time-series modelling tools. The only interpretation that could be extracted is the eigenspace of the operator, which is an ill-posed problem for a large system. Spectra could be extracted using different classical analysis methods that do not have the drawback of extended time. Consequently, we reduce the neural ODE to a simpler linear form and propose a new view on time-series modelling using combined neural networks and an ODE system approach.


title: Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking

abstract: Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. The model needs to capture the system behavior in multiple flight regimes and operating conditions, including those producing highly nonlinear effects such as aerodynamic forces and torques, rotor interactions, or possible system configuration modifications. Classical approaches rely on handcrafted models and struggle to generalize and scale to capture these effects. In this paper, we present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions. In addition, physics constraints are embedded in the training process to facilitate the network's generalization capabilities to data outside the training distribution. Finally, we design a model predictive control approach that incorporates the learned dynamics for accurate closed-loop trajectory tracking fully exploiting the learned model predictions in a receding horizon fashion. Experimental results demonstrate that our approach accurately extracts the structure of the quadrotor's dynamics from data, capturing effects that would remain hidden to classical approaches. To the best of our knowledge, this is the first time physics-inspired deep learning is successfully applied to temporal convolutional networks and to the system identification task, while concurrently enabling predictive control.


title: Household and individual economic responses to different health shocks: The role of medical innovations

abstract: This study provides new evidence regarding the extent to which medical care mitigates the economic consequences of various health shocks for the individual and a wider family. To obtain causal effects, I focus on the role of medical scientific discoveries and leverage the longitudinal dimension of unique administrative data for Sweden. The results indicate that medical innovations strongly mitigate the negative economic consequences of a health shock for the individual and create spillovers to relatives. Such mitigating effects are highly heterogeneous across prognoses. These results suggest that medical innovation substantially reduces the burden of welfare costs yet produces income inequalities.


title: Anomalous transport in driven periodic systems: distribution of the absolute negative mobility effect in the parameter space

abstract: Absolute negative mobility is one of the most paradoxical forms of anomalous transport behaviour. At the first glance it contradicts the superposition principle and the second law of thermodynamics, however, its fascinating nature bridges nonlinearity and nonequlibrium in which these fundamental rules are no longer valid. We consider a paradigmatic model of the nonlinear Brownian motion in a driven periodic system which exhibits the absolute negative mobility. So far research on this anomalous transport feature has been limited mostly to the single case studies due to the fact that this model possesses the complex multidimensional parameter space. In contrast, here we harvest GPU supercomputers to analyze the distribution of negative mobility in the parameter space. We consider nearly $10^9$ parameter regimes to discuss how the emergence of negative mobility depends on the system parameters as well as provide the optimal ones for which it occurs most frequently.


title: Chip-integrated van der Waals PN heterojunction photodetector with low dark current and high responsivity

abstract: Two-dimensional materials are attractive for constructing high-performance photonic chip-integrated photodetectors because of their remarkable electronic and optical properties and dangling-bond-free surfaces. However, the reported chip-integrated two-dimensional material photodetectors were mainly implemented with the configuration of metal-semiconductor-metal, suffering from high dark currents and low responsivities at high operation speed. Here, we report a van der Waals PN heterojunction photodetector, composed of p-type black phosphorous and n-type molybdenum telluride, integrated on a silicon nitride waveguide. The built-in electric field of the PN heterojunction significantly suppresses the dark current and improves the responsivity. Under a bias of 1 V pointing from n-type molybdenum telluride to p-type black phosphorous, the dark current is lower than 7 nA, which is more than two orders of magnitude lower than those reported in other waveguide-integrated black phosphorus photodetectors. An intrinsic responsivity up to 577 mA/W is obtained. Remarkably, the van der Waals PN heterojunction is tunable by the electrostatic doping to further engineer its rectification and improve the photodetection, enabling an increased responsivity of 709 mA/W. Besides, the heterojunction photodetector exhibits a response bandwidth of ~1.0 GHz and a uniform photodetection over a wide spectral range, as experimentally measured from 1500 to 1630 nm. The demonstrated chip-integrated van der Waals PN heterojunction photodetector with low dark current, high responsivity and fast response has great potentials to develop high-performance on-chip photodetectors for various photonic integrated circuits based on silicon, lithium niobate, polymer, etc.


title: Removing proteins or bacteria on a tilted surface using air bubbles

abstract: Cleaning surfaces with bubbles has been a topic of discussion in recent years due to the growing interest in sustainable methods for cleaning. Specifically, a method of using air bubbles to sanitize agricultural produce has been proposed as an eco-friendly alternative to current methods. In this study, we conduct experiments to test the cleaning efficacy at different angles of inclination of a contaminated surface. We use two different types of surface coated with either a protein solution or a bacterial biofilm. Our experimental results indicate that bubbles exhibit the best cleaning efficacy at the surface angle of $\theta \approx 20^{\rm{o}}$ for polydisperse bubbles in the range of 0.3-2 mm and with an average radius of 0.6 mm in radius. To gain a better understanding of the underlying mechanism, we perform a numerical analysis of a single air bubble impacting surfaces with different angles. Our numerical and theoretical results show that the shear stress, which is proportional to the sliding speed but inversely proportional to the thickness of the film, results in the maximum shear force occurring at $\theta\approx22^{\rm{o}} \approx \pi/8$ which agrees well with the experiments.


title: PyTSK: A Python Toolbox for TSK Fuzzy Systems

abstract: This paper presents PyTSK, a Python toolbox for developing Takagi-Sugeno-Kang (TSK) fuzzy systems. Based on scikit-learn and PyTorch, PyTSK allows users to optimize TSK fuzzy systems using fuzzy clustering or mini-batch gradient descent (MBGD) based algorithms. Several state-of-the-art MBGD-based optimization algorithms are implemented in the toolbox, which can improve the generalization performance of TSK fuzzy systems, especially for big data applications. PyTSK can also be easily extended and customized for more complicated algorithms, such as modifying the structure of TSK fuzzy systems, developing more sophisticated training algorithms, and combining TSK fuzzy systems with neural networks. The code of PyTSK can be found at https://github.com/YuqiCui/pytsk.


title: Theorizing Information Sources for Hope: Belief, Desire, Imagination, and Metacognition

abstract: Introduction. Hope is a positive attitude oriented toward a possible (yet uncertain), desired outcome. Though hope is a virtue, hopelessness is widespread and seems related not only to current events but also to information about current events. This paper examines how hope can be sparked through information. Method. This study uses the philosophical methods of conceptual analysis and design to advance a theoretical argument. Analysis. First, a conceptualization of hope is offered, drawing on work primarily in virtue ethics. Then, four types of information sources for hope are theorized, building on and synthesizing work from philosophy and psychology. Results. Four categories of information source conducive to hopefulness are identified: information for forming beliefs about the past or future; information for engaging the moral imagination regarding possibilities for the future; information for sparking desire for particular moral outcomes; and information for metacognition, or about how we become informed with respect to hope. Conclusions. Hope is, in many cases, responsive to information. This suggests a moral opportunity for information professionals and scholars to work toward connecting people with information for hope, particularly in difficult times. Avenues for further research, particularly in information behavior and practices, are suggested.


title: Neuro-Nav: A Library for Neurally-Plausible Reinforcement Learning

abstract: In this work we propose Neuro-Nav, an open-source library for neurally plausible reinforcement learning (RL). RL is among the most common modeling frameworks for studying decision making, learning, and navigation in biological organisms. In utilizing RL, cognitive scientists often handcraft environments and agents to meet the needs of their particular studies. On the other hand, artificial intelligence researchers often struggle to find benchmarks for neurally and biologically plausible representation and behavior (e.g., in decision making or navigation). In order to streamline this process across both fields with transparency and reproducibility, Neuro-Nav offers a set of standardized environments and RL algorithms drawn from canonical behavioral and neural studies in rodents and humans. We demonstrate that the toolkit replicates relevant findings from a number of studies across both cognitive science and RL literatures. We furthermore describe ways in which the library can be extended with novel algorithms (including deep RL) and environments to address future research needs of the field.


title: Quantum oblivious transfer: a short review

abstract: Quantum cryptography is the field of cryptography that explores the quantum properties of matter. Its aim is to develop primitives beyond the reach of classical cryptography or to improve on existing classical implementations. Although much of the work in this field is dedicated to quantum key distribution (QKD), some important steps were made towards the study and development of quantum oblivious transfer (QOT). It is possible to draw a comparison between the application structure of both QKD and QOT primitives. Just as QKD protocols allow quantum-safe communication, QOT protocols allow quantum-safe computation. However, the conditions under which QOT is actually quantum-safe have been subject to a great amount of scrutiny and study. In this review article, we survey the work developed around the concept of oblivious transfer in the area of theoretical quantum cryptography, with an emphasis on some proposed protocols and their security requirements. We review the impossibility results that daunt this primitive and discuss several quantum security models under which it is possible to prove QOT security.


title: Integrating Random Effects in Deep Neural Networks

abstract: Modern approaches to supervised learning like deep neural networks (DNNs) typically implicitly assume that observed responses are statistically independent. In contrast, correlated data are prevalent in real-life large-scale applications, with typical sources of correlation including spatial, temporal and clustering structures. These correlations are either ignored by DNNs, or ad-hoc solutions are developed for specific use cases. We propose to use the mixed models framework to handle correlated data in DNNs. By treating the effects underlying the correlation structure as random effects, mixed models are able to avoid overfitted parameter estimates and ultimately yield better predictive performance. The key to combining mixed models and DNNs is using the Gaussian negative log-likelihood (NLL) as a natural loss function that is minimized with DNN machinery including stochastic gradient descent (SGD). Since NLL does not decompose like standard DNN loss functions, the use of SGD with NLL presents some theoretical and implementation challenges, which we address. Our approach which we call LMMNN is demonstrated to improve performance over natural competitors in various correlation scenarios on diverse simulated and real datasets. Our focus is on a regression setting and tabular datasets, but we also show some results for classification. Our code is available at https://github.com/gsimchoni/lmmnn.


title: Neural Network Decoders for Permutation Codes Correcting Different Errors

abstract: Permutation codes were extensively studied in order to correct different types of errors for the applications on power line communication and rank modulation for flash memory. In this paper, we introduce the neural network decoders for permutation codes to correct these errors with one-shot decoding, which treat the decoding as $n$ classification tasks for non-binary symbols for a code of length $n$. These are actually the first general decoders introduced to deal with any error type for these two applications. The performance of the decoders is evaluated by simulations with different error models.


title: WISDOM project -- XI. Star Formation Efficiency in the Bulge of the AGN-host Galaxy NGC 3169 with SITELLE and ALMA

abstract: The star formation efficiency (SFE) has been shown to vary across different environments, particularly within galactic starbursts and deep within the bulges of galaxies. Various quenching mechanisms may be responsible, ranging from galactic dynamics to feedback from active galactic nuclei (AGN). Here, we use spatially-resolved observations of warm ionised gas emission lines from the imaging Fourier transform spectrograph SITELLE at the Canada-France-Hawaii Telescope (CFHT) and cold molecular gas (CO(2-1)) from the Atacama Large Millimeter/sub-millimeter Array (ALMA) to study the SFE in the bulge of the AGN-host galaxy NGC 3169. After distinguishing star-forming regions from AGN-ionised regions using emission-line ratio diagnostics, we measure spatially-resolved molecular gas depletion times (\tau_dep = 1/SFE) with a spatial resolution of \approx 100 pc within a galactocentric radius of 1.8 kpc. We identify a star-forming ring located at radii 1.25 \pm 0.6 kpc with an average \tau_dep of 0.3 Gyr. At radii < 0.9 kpc, however, the molecular gas surface densities and depletion times increase with decreasing radius, the latter reaching approximately 2.3 Gyr at a radius \approx 500 pc. Based on analyses of the gas kinematics and comparisons with simulations, we identify AGN feedback, bulge morphology and dynamics as the possible causes of the radial profile of SFE observed in the central region of NGC 3169.


title: Subject Membership Inference Attacks in Federated Learning

abstract: Privacy attacks on Machine Learning (ML) models often focus on inferring the existence of particular data points in the training data. However, what the adversary really wants to know is if a particular individual's (subject's) data was included during training. In such scenarios, the adversary is more likely to have access to the distribution of a particular subject than actual records. Furthermore, in settings like cross-silo Federated Learning (FL), a subject's data can be embodied by multiple data records that are spread across multiple organizations. Nearly all of the existing private FL literature is dedicated to studying privacy at two granularities -- item-level (individual data records), and user-level (participating user in the federation), neither of which apply to data subjects in cross-silo FL. This insight motivates us to shift our attention from the privacy of data records to the privacy of data subjects, also known as subject-level privacy. We propose two novel black-box attacks for subject membership inference, of which one assumes access to a model after each training round. Using these attacks, we estimate subject membership inference risk on real-world data for single-party models as well as FL scenarios. We find our attacks to be extremely potent, even without access to exact training records, and using the knowledge of membership for a handful of subjects. To better understand the various factors that may influence subject privacy risk in cross-silo FL settings, we systematically generate several hundred synthetic federation configurations, varying properties of the data, model design and training, and the federation itself. Finally, we investigate the effectiveness of Differential Privacy in mitigating this threat.


title: LegoNN: Building Modular Encoder-Decoder Models

abstract: State-of-the-art encoder-decoder models (e.g. for machine translation (MT) or automatic speech recognition (ASR)) are constructed and trained end-to-end as an atomic unit. No component of the model can be (re-)used without the others, making it impossible to share parts, e.g. a high resourced decoder, across tasks. We describe LegoNN, a procedure for building encoder-decoder architectures in a way so that its parts can be applied to other tasks without the need for any fine-tuning. To achieve this reusability, the interface between encoder and decoder modules is grounded to a sequence of marginal distributions over a pre-defined discrete vocabulary. We present two approaches for ingesting these marginals; one is differentiable, allowing the flow of gradients across the entire network, and the other is gradient-isolating. To enable the portability of decoder modules between MT tasks for different source languages and across other tasks like ASR, we introduce a modality agnostic encoder which consists of a length control mechanism to dynamically adapt encoders' output lengths in order to match the expected input length range of pre-trained decoders. We present several experiments to demonstrate the effectiveness of LegoNN models: a trained language generation LegoNN decoder module from German-English (De-En) MT task can be reused without any fine-tuning for the Europarl English ASR and the Romanian-English (Ro-En) MT tasks, matching or beating the performance of baseline. After fine-tuning, LegoNN models improve the Ro-En MT task by 1.5 BLEU points and achieve 12.5% relative WER reduction on the Europarl ASR task. To show how the approach generalizes, we compose a LegoNN ASR model from three modules -- each has been learned within different end-to-end trained models on three different datasets -- achieving an overall WER reduction of 19.5%.


title: A Differentially Private Linear-Time fPTAS for the Minimum Enclosing Ball Problem

abstract: The Minimum Enclosing Ball (MEB) problem is one of the most fundamental problems in clustering, with applications in operations research, statistics and computational geometry. In this works, we give the first differentially private (DP) fPTAS for the Minimum Enclosing Ball problem, improving both on the runtime and the utility bound of the best known DP-PTAS for the problem, of Ghazi et al. (2020). Given $n$ points in $\R^d$ that are covered by the ball $B(\theta_{opt},r_{opt})$, our simple iterative DP-algorithm returns a ball $B(\theta,r)$ where $r\leq (1+\gamma)r_{opt}$ and which leaves at most $\tilde O(\frac{\sqrt d}{\gamma\epsilon})$ points uncovered in $\tilde O(\nicefrac n {\gamma^2})$-time. We also give a local-model version of our algorithm, that leaves at most $\tilde O(\frac{\sqrt {nd}}{\gamma\epsilon})$ points uncovered, improving on the $n^{0.67}$-bound of Nissim and Stemmer (2018) (at the expense of other parameters). In addition, we test our algorithm empirically and discuss future open problems.


title: Two incompatible types of invariants in the octonion spaces

abstract: The paper aims to study some invariants and conservation laws relevant to electromagnetic and gravitational fields, by means of the rotational transformations of octonion coordinate systems. The scholars utilize the octonions to analyze the electromagnetic and gravitational fields simultaneously, including the octonion field potential, field strength, field source, linear momentum, angular momentum, torque and force. When the octonion coordinate system transforms rotationally, the vector part of one octonion may alter, while the scalar part of the octonion will remain unchanged. This property allows one to deduce a few invariants, such as the scalar part of octonion radius vector, speed of light, and norm of octonion radius vector. These invariants are the basic postulates for the Galilean transformation and Lorentz transformation. Further, from the rotation transform of octonion coordinate systems, it is capable of deducing several invariants, including the mass, energy and power relevant to gravitational fields, in one octonion space $\mathbb{O}$. And the term relevant to the electric charge will transform with the rotation of coordinate systems. In another octonion space $\mathbb{O}_u$, it is capable of inferring a few invariants, including the electric charge related to electromagnetic fields. And the terms relevant to the mass and energy will vary with the rotation of octonion coordinate systems. So the invariants are divided into two different groups. In particular, the mass conservation law and energy conservation law can be effective simultaneously. But the charge conservation law and mass conservation law are unable to be valid simultaneously, in the strict sense. It is beneficial to further understand the laws of conservation.


title: Early Abnormal Detection of Sewage Pipe Network: Bagging of Various Abnormal Detection Algorithms

abstract: Abnormalities of the sewage pipe network will affect the normal operation of the whole city. Therefore, it is important to detect the abnormalities early. This paper propose an early abnormal-detection method. The abnormalities are detected by using the conventional algorithms, such as isolation forest algorithm, two innovations are given: (1) The current and historical data measured by the sensors placed in the sewage pipe network (such as ultrasonic Doppler flowmeter) are taken as the overall dataset, and then the general dataset is detected by using the conventional anomaly detection method to diagnose the anomaly of the data. The anomaly refers to the sample different from the others samples in the whole dataset. Because the definition of anomaly is not through the algorithm, but the whole dataset, the construction of the whole dataset is the key to propose the early abnormal-detection algorithms. (2) A bagging strategy for a variety of conventional anomaly detection algorithms is proposed to achieve the early detection of anomalies with the high precision and recall. The results show that this method can achieve the early anomaly detection with the highest precision of 98.21%, the recall rate 63.58% and F1-score of 0.774.


title: Deep Learning-based Finite Element Analysis (FEA) surrogate for sub-sea pressure vessel

abstract: During the design process of an autonomous underwater vehicle (AUV), the pressure vessel has a critical role. The pressure vessel contains dry electronics, power sources, and other sensors that can not be flooded. A traditional design approach for a pressure vessel design involves running multiple Finite Element Analysis (FEA) based simulations and optimizing the design to find the best suitable design which meets the requirement. Running these FEAs are computationally very costly for any optimization process and it becomes difficult to run even hundreds of evaluation. In such a case, a better approach is the surrogate design with the goal of replacing FEA-based prediction with some learning-based regressor. Once the surrogate is trained for a class of problem, then the learned response surface can be used to analyze the stress effect without running the FEA for that class of problem. The challenge of creating a surrogate for a class of problems is data generation. Since the process is computationally costly, it is not possible to densely sample the design space and the learning response surface on sparse data set becomes difficult. During experimentation, we observed that a Deep Learning-based surrogate outperforms other regression models on such sparse data. In the present work, we are utilizing the Deep Learning-based model to replace the costly finite element analysis-based simulation process. By creating the surrogate we speed up the prediction on the other design much faster than direct Finite element Analysis. We also compared our DL-based surrogate with other classical Machine Learning (ML) based regression models( random forest and Gradient Boost regressor). We observed on the sparser data, the DL-based surrogate performs much better than other regression models.


title: Fully reducible simple Venn diagrams

abstract: We generalize Venn diagrams in spaces of arbitrary dimension $\geq 2$ and study simple Venn diagrams with the reducing property. Three equivalent conditions for a simple Venn diagram to reduce it completely and a classification of those diagrams is discussed. For example, we show that a simple $m$-dimensional $n$-Venn diagram is fully reducible if $n\leq m+1$ and conjecture that the converse is also true. An application of the generalized Venn diagrams and some more open problems are given.


title: Efficient decentralized multi-agent learning in asymmetric bipartite queueing systems

abstract: We study decentralized multi-agent learning in bipartite queueing systems, a standard model for service systems. In particular, N agents request service from K servers in a fully decentralized way, i.e, by running the same algorithm without communication. Previous decentralized algorithms are restricted to symmetric systems, have performance that is degrading exponentially in the number of servers, require communication through shared randomness and unique agent identities, and are computationally demanding. In contrast, we provide a simple learning algorithm that, when run decentrally by each agent, leads the queueing system to have efficient performance in general asymmetric bipartite queueing systems while also having additional robustness properties. Along the way, we provide the first provably efficient UCB-based algorithm for the centralized case of the problem.


title: Searching Similarity Measure for Binarized Neural Networks

abstract: Being a promising model to be deployed in resource-limited devices, Binarized Neural Networks (BNNs) have drawn extensive attention from both academic and industry. However, comparing to the full-precision deep neural networks (DNNs), BNNs suffer from non-trivial accuracy degradation, limiting its applicability in various domains. This is partially because existing network components, such as the similarity measure, are specially designed for DNNs, and might be sub-optimal for BNNs. In this work, we focus on the key component of BNNs -- the similarity measure, which quantifies the distance between input feature maps and filters, and propose an automatic searching method, based on genetic algorithm, for BNN-tailored similarity measure. Evaluation results on Cifar10 and Cifar100 using ResNet, NIN and VGG show that most of the identified similarty measure can achieve considerable accuracy improvement (up to 3.39%) over the commonly-used cross-correlation approach.


title: Compilation and Optimizations for Efficient Machine Learning on Embedded Systems

abstract: Deep Neural Networks (DNNs) have achieved great success in a variety of machine learning (ML) applications, delivering high-quality inferencing solutions in computer vision, natural language processing, and virtual reality, etc. However, DNN-based ML applications also bring much increased computational and storage requirements, which are particularly challenging for embedded systems with limited compute/storage resources, tight power budgets, and small form factors. Challenges also come from the diverse application-specific requirements, including real-time responses, high-throughput performance, and reliable inference accuracy. To address these challenges, we introduce a series of effective design methodologies, including efficient ML model designs, customized hardware accelerator designs, and hardware/software co-design strategies to enable efficient ML applications on embedded systems.


title: The Yang-Mills-Higgs functional on complex line bundles: $\Gamma$-convergence and the London equation

abstract: We consider the Abelian Yang-Mills-Higgs functional, in the non-self dual scaling, on a complex line bundle over a closed Riemannian manifold of dimension $n\geq 3$. This functional is the natural generalisation of the Ginzburg-Landau model for superconductivity to the non-Euclidean setting. We prove a $\Gamma$-convergence result, in the strongly repulsive limit, on the functional rescaled by the logarithm of the coupling parameter. As a corollary, we prove that the energy of minimisers concentrates on an area-minimising surface of dimension $n-2$, while the curvature of minimisers converges to a solution of the London equation.


title: Concentration bounds for SSP Q-learning for average cost MDPs

abstract: We derive a concentration bound for a Q-learning algorithm for average cost Markov decision processes based on an equivalent shortest path problem, and compare it numerically with the alternative scheme based on relative value iteration.


title: Concentration analysis of multivariate elliptic diffusion processes

abstract: We prove concentration inequalities and associated PAC bounds for continuous- and discrete-time additive functionals for possibly unbounded functions of multivariate, nonreversible diffusion processes. Our analysis relies on an approach via the Poisson equation allowing us to consider a very broad class of subexponentially ergodic processes. These results add to existing concentration inequalities for additive functionals of diffusion processes which have so far been only available for either bounded functions or for unbounded functions of processes from a significantly smaller class. We demonstrate the power of these exponential inequalities by two examples of very different areas. Considering a possibly high-dimensional parametric nonlinear drift model under sparsity constraints, we apply the continuous-time concentration results to validate the restricted eigenvalue condition for Lasso estimation, which is fundamental for the derivation of oracle inequalities. The results for discrete additive functionals are used to investigate the unadjusted Langevin MCMC algorithm for sampling of moderately heavy-tailed densities $\pi$. In particular, we provide PAC bounds for the sample Monte Carlo estimator of integrals $\pi(f)$ for polynomially growing functions $f$ that quantify sufficient sample and step sizes for approximation within a prescribed margin with high probability.


title: EEG-based Emotion Recognition with Spatial and Functional Brain Mapping of CNS and PNS Signals

abstract: Emotion plays a significant role in our daily life. Recognition of emotion is wide-spread in the field of health care and human-computer interaction. Emotion is the result of the coordinated activities of cortical and subcortical neural processes, which correlate to specific physiological responses. However, the existing emotion recognition techniques failed to combine various physiological signals as one integrated feature representation. Meanwhile, many researchers ignored the problem of over-fitting model with high accuracy, which was actually false high accuracy caused by improper pre-processing. In this paper, sigmoid baseline filtering is conducted to solve the over-fitting problem from source. To construct a physiological-based algorithm, a 3D spatial and functional brain mapping is proposed based on human physiological mechanism and international electrode system, which combines the signals of the central and peripheral nervous system together. By combining the baseline filtering, 3D brain mapping, and simple 4D-CNN, a novel emotion recognition model is finally proposed. Experiment results demonstrate that the performance of the proposed model is comparable to the state of art algorithms.


title: Improving the Diagnosis of Psychiatric Disorders with Self-Supervised Graph State Space Models

abstract: Single subject prediction of brain disorders from neuroimaging data has gained increasing attention in recent years. Yet, for some heterogeneous disorders such as major depression disorder (MDD) and autism spectrum disorder (ASD), the performance of prediction models on large-scale multi-site datasets remains poor. We present a two-stage framework to improve the diagnosis of heterogeneous psychiatric disorders from resting-state functional magnetic resonance imaging (rs-fMRI). First, we propose a self-supervised mask prediction task on data from healthy individuals that can exploit differences between healthy controls and patients in clinical datasets. Next, we train a supervised classifier on the learned discriminative representations. To model rs-fMRI data, we develop Graph-S4; an extension to the recently proposed state-space model S4 to graph settings where the underlying graph structure is not known in advance. We show that combining the framework and Graph-S4 can significantly improve the diagnostic performance of neuroimaging-based single subject prediction models of MDD and ASD on three open-source multi-center rs-fMRI clinical datasets.


title: Residual amplitude modulation at the $10^{-7}$ level for ultra-stable lasers

abstract: The stabilization of lasers on ultra-stable optical cavities by the Pound-Drever-Hall (PDH) technique is a widely used method. The PDH method relies on the phase-modulation of the laser, which is usually performed by an electro-optic modulator (EOM). When approaching the $10^{-16}$ level, this technology requires an active control of the residual amplitude modulation (RAM) generated by the EOM in order to bring the frequency stability of the laser down to the thermal noise limit of the ultra-stable cavity. In this article, we report on the development of an active system of RAM reduction based on a free space EOM, which is used to perform PDH-stabilization of a laser on a cryogenic silicon cavity. A RAM stability of $1.4 \times 10^{-7}$ is obtained by employing a digital servo that stabilizes the EOM DC electric field, the crystal temperature and the laser power. Considering an ultra-stable cavity with a finesse of $2.5\times 10^5$, this RAM level would contribute to the fractional frequency instability at the level of about $5\times 10^{-19}$, well below the state of the art thermal noise limit of a few $ 10^{-17}$.


title: Assessing Project-Level Fine-Tuning of ML4SE Models

abstract: Machine Learning for Software Engineering (ML4SE) is an actively growing research area that focuses on methods that help programmers in their work. In order to apply the developed methods in practice, they need to achieve reasonable quality in order to help rather than distract developers. While the development of new approaches to code representation and data collection improves the overall quality of the models, it does not take into account the information that we can get from the project at hand. In this work, we investigate how the model's quality can be improved if we target a specific project. We develop a framework to assess quality improvements that models can get after fine-tuning for the method name prediction task on a particular project. We evaluate three models of different complexity and compare their quality in three settings: trained on a large dataset of Java projects, further fine-tuned on the data from a particular project, and trained from scratch on this data. We show that per-project fine-tuning can greatly improve the models' quality as they capture the project's domain and naming conventions. We open-source the tool we used for data collection, as well as the code to run the experiments: https://zenodo.org/record/6040745.


title: Correlations of network trajectories

abstract: Temporal networks model how the interaction between elements in a complex system evolve over time. Just like complex systems display collective dynamics, here we interpret temporal networks as trajectories performing a collective motion in graph space, following a latent graph dynamical system. Under this paradigm, we propose a way to measure how the network pulsates and collectively fluctuates over time and space. To this aim, we extend the notion of linear correlations function to the case of sequences of network snapshots, i.e. a network trajectory. We construct stochastic and deterministic graph dynamical systems and show that the emergent collective correlations are well captured by simple measures, and illustrate how these patterns are revealed in empirical networks arising in different domains.


title: Spin and charge transport in ferromagnet-superconductor-ferromagnet heterostructures: Stoner versus spin mass mismatch mechanism

abstract: We study transport phenomena through a ballistic ferromagnet-superconductor-ferromagnet (F/S/F) junction, comparing the case in which the ferromagnetic order in the two F layers is of the standard Stoner type with the case where it is driven by a spin mass mismatch (SMM). It is shown that the two mechanisms lead to a different behavior in the charge and the spin conductances, especially when compared to the corresponding non-superconducting ferromagnet-normal-ferromagnet (F/N/F) junctions. In particular, when the injected current is perpendicular to the barrier, for high barrier transparency and large magnetization of the F layers, the large mass mismatch gives rise to an enhancement of both low-bias charge and spin conductances of the F/S/F junction, which is not observed in the equal-mass case. When all the allowed injection directions are considered, the low bias enhancement of the charge conductance for SMM leads still holds for high barrier transparency and large magnetization of the F layers. However, in the case of non-transparent interfaces, spin transport with SMM ferromagnets exhibits an opposite sign response with respect to the Stoner case at high biases for all magnetization values, also manifesting a significant amplification induced by superconductivity at the gap edge. The above mentioned differences can be exploited to probe the nature of the electronic mechanism underlying the establishment of the ferromagnetic order in a given material.


title: Parotid Gland MRI Segmentation Based on Swin-Unet and Multimodal Images

abstract: Background and objective: Parotid gland tumors account for approximately 2% to 10% of head and neck tumors. Preoperative tumor localization, differential diagnosis, and subsequent selection of appropriate treatment for parotid gland tumors are critical. However, the relative rarity of these tumors and the highly dispersed tissue types have left an unmet need for a subtle differential diagnosis of such neoplastic lesions based on preoperative radiomics. Recently, deep learning methods have developed rapidly, especially Transformer beats the traditional convolutional neural network in computer vision. Many new Transformer-based networks have been proposed for computer vision tasks. Methods: In this study, multicenter multimodal parotid gland MR images were collected. The Swin-Unet which was based on Transformer was used. MR images of short time inversion recovery, T1-weighted and T2-weighted modalities were combined into three-channel data to train the network. We achieved segmentation of the region of interest for parotid gland and tumor. Results: The Dice-Similarity Coefficient of the model on the test set was 88.63%, Mean Pixel Accuracy was 99.31%, Mean Intersection over Union was 83.99%, and Hausdorff Distance was 3.04. Then a series of comparison experiments were designed in this paper to further validate the segmentation performance of the algorithm. Conclusions: Experimental results showed that our method has good results for parotid gland and tumor segmentation. The Transformer-based network outperforms the traditional convolutional neural network in the field of medical images.


title: Behaviour of solutions to $p$-Laplacian with Robin boundary conditions as $p$ goes to $1$

abstract: We study the asymptotic behaviour, as $p\to 1^{+}$, of the solutions of the following inhomogeneous Robin boundary value problem: \begin{equation} \label{pbabstract} \tag{P} \left\{\begin{array}{ll} \displaystyle -\Delta_p u_p = f & \text{in }\Omega, \displaystyle |\nabla u_p|^{p-2}\nabla u_p\cdot \nu +\lambda |u_p|^{p-2}u_p = g& \text{on } \partial\Omega, \end{array}\right. \end{equation} where $\Omega$ is a bounded domain in $\mathbb R^{N}$ with sufficiently smooth boundary, $\nu$ is its unit outward normal vector and $\Delta_p v$ is the $p$-Laplacian operator with $p>1$. The data $f\in L^{N,\infty}(\Omega)$ (which denotes the Marcinkiewicz space) and $\lambda,g$ are bounded functions defined on $\partial\Omega$ with $\lambda\ge0$. We find the threshold below which the family of $p$--solutions goes to 0 and above which this family blows up. As a second interest we deal with the $1$-Laplacian problem formally arising by taking $p\to 1^+$ in \eqref{pbabstract}.


title: Extended minimal theories of massive gravity

abstract: In this work, we introduce a class of extended Minimal Theories of Massive Gravity (eMTMG), without requiring a priori that the theory should admit the same homogeneous and isotropic cosmological solutions as the de Rham-Gabadadze-Tolley massive gravity. The theory is constructed as to have only two degrees of freedom in the gravity sector. In order to perform this step we first introduce a precursor theory endowed with a general graviton mass term, to which, at the level of the Hamiltonian, we add two extra constraints as to remove the unwanted degrees of freedom, which otherwise would typically lead to ghosts and/or instabilities. On analyzing the number of independent constraints and the properties of tensor mode perturbations, we see that the gravitational waves are the only propagating gravitational degrees of freedom which do acquire a non-trivial mass, as expected. In order to understand how the effective gravitational force works for this theory we then investigate cosmological scalar perturbations in the presence of a pressureless fluid. We then restrict the whole class of models by imposing the following conditions at all times: 1) it is possible to define an effective gravitational constant, $G_{{\rm eff}}$; 2) the value $G_{\text{eff}}/G_{N}$ is always finite but not always equal to unity (as to allow some non-trivial modifications of gravity, besides the massive tensorial modes); and 3) the square of mass of the graviton is always positive. These constraints automatically make also the ISW-effect contributions finite at all times. Finally we focus on a simple subclass of such theories, and show they already can give a rich and interesting phenomenology.


title: A spectral Erd\H{o}s-S\'os theorem

abstract: The famous Erd\H{o}s-S\'os conjecture states that every graph of average degree more than $t-1$ must contain every tree on $t+1$ vertices. In this paper, we study a spectral version of this conjecture. For $n>k$, let $S_{n,k}$ be the join of a clique on $k$ vertices with an independent set of $n-k$ vertices and denote by $S_{n,k}^+$ the graph obtained from $S_{n,k}$ by adding one edge. We show that for fixed $k\geq 2$ and sufficiently large $n$, if a graph on $n$ vertices has adjacency spectral radius at least as large as $S_{n,k}$ and is not isomorphic to $S_{n,k}$, then it contains all trees on $2k+2$ vertices. Similarly, if a sufficiently large graph has spectral radius at least as large as $S_{n,k}^+$, then it either contains all trees on $2k+3$ vertices or is isomorphic to $S_{n,k}^+$. This answers a two-part conjecture of Nikiforov affirmatively.


title: A 4D IIB Flux Vacuum and Supersymmetry Breaking. I. Fermionic Spectrum

abstract: We consider the type-IIB supergravity vacua that include an internal $T^5$, depend on a single coordinate $r$ and respect a four-dimensional Poincar\'e symmetry, with the aim of highlighting low-energy spectra with broken supersymmetry and a bounded string coupling. These vacua are characterized by the flux $\Phi$ of the self-dual five form in the internal torus, the length $\ell$ of the interval described by the coordinate $r$, a dilaton profile that is inevitably constant and a strictly positive dimensionless parameter $h$. As $\ell\rightarrow\infty$ while retaining finite values for $\Phi$ and $\ell \,h^{-\,\frac{5}{4}}$, half of the original ten-dimensional supersymmetry is recovered, while finite values of $\ell$ break it completely. In the large-$\ell$ limit one boundary disappears but the other is still present, and is felt as a BPS orientifold by a probe brane. In this paper we focus on the fermionic zero modes and show that, although supersymmetry is broken for finite values of $\ell$, they are surprisingly those of four-dimensional $N=4$ supergravity coupled to five vector multiplets. The gravitini can acquire masses via radiative corrections, absorbing four of the massless spin-$\frac{1}{2}$ modes.


title: Introducing 4D Geometric Shell Shaping for Mitigating Nonlinear Interference Noise

abstract: Four dimensional geometric shell shaping (4D-GSS) is introduced as an approach for closing the nonlinearity-caused shaping gap. This format is designed at the spectral efficiency of 8 b/4D-sym and is compared against polarization-multiplexed 16QAM (PM-16QAM) and probabilistically shaped PM-16QAM (PS-PM-16QAM) in a 400ZR-compatible transmission setup with high amount of nonlinearities. Reach increase and nonlinearity tolerance are evaluated in terms of achievable information rates and post-FEC bit-error rate. Numerical simulations for a single-span, single-channel show that 4D-GSS achieves increased nonlinear tolerance and reach increase against PM-16QAM and PS-PM-16QAM when optimized for bit-metric decoding (RBMD). In terms of RBMD, gains are small with a reach increase of 1.7% compared to PM-16QAM. When optimizing for mutual information, a larger reach increase of 3% is achieved compared to PM-16QAM. Moreover, the introduced GSS scheme provides a scalable framework for designing well-structured 4D modulation formats with low complexity.


title: Group Properties of Polar Codes for Automorphism Ensemble Decoding

abstract: In this paper, we propose an analysis of the automorphism group of polar codes, with the scope of designing codes tailored for automorphism ensemble (AE) decoding. We prove the equivalence between the notion of decreasing monomial codes and the universal partial order (UPO) framework for the description of polar codes. Then, we analyze the algebraic properties of the affine automorphisms group of polar codes, providing a novel description of its structure and proposing a classification of automorphisms providing the same results under permutation decoding. Finally, we propose a method to list all the automorphisms that may lead to different candidates under AE decoding; by introducing the concept of redundant automorphisms, we find the maximum number of permutations providing possibly different codeword candidates under AE-SC, proposing a method to list all of them. A numerical analysis of the error correction performance of AE algorithm for the decoding of polar codes concludes the paper.


title: Closing in on critical net-baryon fluctuations at LHC energies: cumulants up to third order in Pb$-$Pb collisions

abstract: Fluctuation measurements are important sources of information on the mechanism of particle production at LHC energies. This article reports the first experimental results on third-order cumulants of the net-proton distributions in Pb$-$Pb collisions at a center-of-mass energy $\sqrt{s_{\rm NN}} = 5.02$ TeV recorded by the ALICE detector. The results on the second-order cumulants of net-proton distributions at $\sqrt{s_{\rm NN}} = 2.76$ and $5.02$ TeV are also discussed in view of effects due to the global and local baryon number conservation. The results demonstrate the presence of long-range rapidity correlations between protons and antiprotons. Such correlations originate from the early phase of the collision. The experimental results are compared with HIJING and EPOS model calculations, and the dependence of the fluctuation measurements on the phase-space coverage is examined in the context of lattice quantum chromodynamics (LQCD) and hadron resonance gas (HRG) model estimations. The measured third-order cumulants are consistent with zero within experimental uncertainties of about 4% and are described well by LQCD and HRG predictions.


title: Towards the understanding of the genuine three-body interaction for p$-$p$-$p and p$-$p$-\Lambda$

abstract: Three-body nuclear forces play an important role in the structure of nuclei and hypernuclei and are also incorporated in models to describe the dynamics of dense baryonic matter, such as in neutron stars. So far, only indirect measurements anchored to the binding energies of nuclei can be used to constrain the three-nucleon force, and if hyperons are considered, the scarce data on hypernuclei impose only weak constraints on the three-body forces. In this work, we present the first direct measurement of the p$-$p$-$p and p$-$p$-\Lambda$ systems in terms of three-particle correlation functions carried out for pp collisions at $\sqrt{s} = 13$ TeV. Three-particle cumulants are extracted from the correlation functions by applying the Kubo formalism, where the three-particle interaction contribution to these correlations can be isolated after subtracting the known two-body interaction terms. A negative cumulant is found for the p$-$p$-$p system, hinting to the presence of a residual three-body effect while for p$-$p$-\Lambda$ the cumulant is consistent with zero. This measurement demonstrates the accessibility of three-baryon correlations at the LHC.


title: Preconditioned Gradient Descent for Overparameterized Nonconvex Burer--Monteiro Factorization with Global Optimality Certification

abstract: We consider using gradient descent to minimize the nonconvex function $f(X)=\phi(XX^{T})$ over an $n\times r$ factor matrix $X$, in which $\phi$ is an underlying smooth convex cost function defined over $n\times n$ matrices. While only a second-order stationary point $X$ can be provably found in reasonable time, if $X$ is additionally rank deficient, then its rank deficiency certifies it as being globally optimal. This way of certifying global optimality necessarily requires the search rank $r$ of the current iterate $X$ to be overparameterized with respect to the rank $r^{\star}$ of the global minimizer $X^{\star}$. Unfortunately, overparameterization significantly slows down the convergence of gradient descent, from a linear rate with $r=r^{\star}$ to a sublinear rate when $r>r^{\star}$, even when $\phi$ is strongly convex. In this paper, we propose an inexpensive preconditioner that restores the convergence rate of gradient descent back to linear in the overparameterized case, while also making it agnostic to possible ill-conditioning in the global minimizer $X^{\star}$.


title: Excitation & Excavation of the Claws of the Southern Crab

abstract: We show that the Southern Crab (aka Hen2-104) presents an auspicious opportunity to study the form and speed of the invisible winds that excavate and shock the lobes of various types of bipolar nebulae associated with close and highly evolved binary stars. A deep three-color image overlay of Hen2-104 reveals that the ionization state of its lobe edges, or "claws", increases steadily from singly to doubly ionized values with increasing wall latitude. This "reverse" ionization pattern is unique among planetary nebulae (and similar objects) and incompatible with UV photoionization from a central source. We show that the most self-consistent explanation for the ionization pattern is shock ionization by a fast (~600 km s^-1) "tapered" stellar wind in which the speed and momentum flux of the wind increase with equatorial latitude. We present a hydrodynamic simulation that places the latitude-dependent form, the knotty walls, and the reverse ionization of the outer lobes of Hen2-104 into a unified context.


title: Convergence rate of general entropic optimal transport costs

abstract: We investigate the convergence rate of the optimal entropic cost $v_\varepsilon$ to the optimal transport cost as the noise parameter $\varepsilon \downarrow 0$. We show that for a large class of cost functions $c$ on $\mathbb{R}^d\times \mathbb{R}^d$ (for which optimal plans are not necessarily unique or induced by a transport map) and compactly supported and $L^{\infty}$ marginals, one has $v_\varepsilon-v_0= \frac{d}{2} \varepsilon \log(1/\varepsilon)+ O(\varepsilon)$. Upper bounds are obtained by a block approximation strategy and an integral variant of Alexandrov's theorem. Under an infinitesimal twist condition on $c$, i.e. invertibility of $\nabla_{xy}^2 c(x,y)$, we get the lower bound by establishing a quadratic detachment of the duality gap in $d$ dimensions thanks to Minty's trick.


title: Specification-Guided Learning of Nash Equilibria with High Social Welfare

abstract: Reinforcement learning has been shown to be an effective strategy for automatically training policies for challenging control problems. Focusing on non-cooperative multi-agent systems, we propose a novel reinforcement learning framework for training joint policies that form a Nash equilibrium. In our approach, rather than providing low-level reward functions, the user provides high-level specifications that encode the objective of each agent. Then, guided by the structure of the specifications, our algorithm searches over policies to identify one that provably forms an $\epsilon$-Nash equilibrium (with high probability). Importantly, it prioritizes policies in a way that maximizes social welfare across all agents. Our empirical evaluation demonstrates that our algorithm computes equilibrium policies with high social welfare, whereas state-of-the-art baselines either fail to compute Nash equilibria or compute ones with comparatively lower social welfare.


title: Semiclassical quantization conditions in strained moir\'e lattices

abstract: In this article we generalize the Bohr-Sommerfeld rule for scalar symbols at a potential well to matrix-valued symbols having eigenvalues that may coalesce precisely at the bottom of the well. As an application, we study the existence of approximately flat bands in moir\'e heterostructures such as strained two-dimensional honeycomb lattices in a model recently introduced by Timmel and Mele.

categories: math.AP cond-mat.str-el math-ph math.MP math.SP quant-ph


title: Shifted moments of the Riemann zeta function

abstract: In this article, we prove that the Riemann hypothesis implies a conjecture of Chandee on shifted moments of the Riemann zeta function. The proof is based on ideas of Harper concerning sharp upper bounds for the $2k$-th moments of the Riemann zeta function on the critical line.


title: AS2T: Arbitrary Source-To-Target Adversarial Attack on Speaker Recognition Systems

abstract: Recent work has illuminated the vulnerability of speaker recognition systems (SRSs) against adversarial attacks, raising significant security concerns in deploying SRSs. However, they considered only a few settings (e.g., some combinations of source and target speakers), leaving many interesting and important settings in real-world attack scenarios alone. In this work, we present AS2T, the first attack in this domain which covers all the settings, thus allows the adversary to craft adversarial voices using arbitrary source and target speakers for any of three main recognition tasks. Since none of the existing loss functions can be applied to all the settings, we explore many candidate loss functions for each setting including the existing and newly designed ones. We thoroughly evaluate their efficacy and find that some existing loss functions are suboptimal. Then, to improve the robustness of AS2T towards practical over-the-air attack, we study the possible distortions occurred in over-the-air transmission, utilize different transformation functions with different parameters to model those distortions, and incorporate them into the generation of adversarial voices. Our simulated over-the-air evaluation validates the effectiveness of our solution in producing robust adversarial voices which remain effective under various hardware devices and various acoustic environments with different reverberation, ambient noises, and noise levels. Finally, we leverage AS2T to perform thus far the largest-scale evaluation to understand transferability among 14 diverse SRSs. The transferability analysis provides many interesting and useful insights which challenge several findings and conclusion drawn in previous works in the image domain. Our study also sheds light on future directions of adversarial attacks in the speaker recognition domain.


title: Searching for Optimal Subword Tokenization in Cross-domain NER

abstract: Input distribution shift is one of the vital problems in unsupervised domain adaptation (UDA). The most popular UDA approaches focus on domain-invariant representation learning, trying to align the features from different domains into similar feature distributions. However, these approaches ignore the direct alignment of input word distributions between domains, which is a vital factor in word-level classification tasks such as cross-domain NER. In this work, we shed new light on cross-domain NER by introducing a subword-level solution, X-Piece, for input word-level distribution shift in NER. Specifically, we re-tokenize the input words of the source domain to approach the target subword distribution, which is formulated and solved as an optimal transport problem. As this approach focuses on the input level, it can also be combined with previous DIRL methods for further improvement. Experimental results show the effectiveness of the proposed method based on BERT-tagger on four benchmark NER datasets. Also, the proposed method is proved to benefit DIRL methods such as DANN.


title: Improving Adversarial Robustness by Putting More Regularizations on Less Robust Samples

abstract: Adversarial training, which is to enhance robustness against adversarial attacks, has received much attention because it is easy to generate human-imperceptible perturbations of data to deceive a given deep neural network. In this paper, we propose a new adversarial training algorithm that is theoretically well motivated and empirically superior to other existing algorithms. A novel feature of the proposed algorithm is to apply more regularization to data vulnerable to adversarial attacks than other existing regularization algorithms do. Theoretically, we show that our algorithm can be understood as an algorithm of minimizing the regularized empirical risk motivated from a newly derived upper bound of the robust risk. Numerical experiments illustrate that our proposed algorithm improves the generalization (accuracy on examples) and robustness (accuracy on adversarial attacks) simultaneously to achieve the state-of-the-art performance.


title: cViL: Cross-Lingual Training of Vision-Language Models using Knowledge Distillation

abstract: Vision-and-language tasks are gaining popularity in the research community, but the focus is still mainly on English. We propose a pipeline that utilizes English-only vision-language models to train a monolingual model for a target language. We propose to extend OSCAR+, a model which leverages object tags as anchor points for learning image-text alignments, to train on visual question answering datasets in different languages. We propose a novel approach to knowledge distillation to train the model in other languages using parallel sentences. Compared to other models that use the target language in the pretraining corpora, we can leverage an existing English model to transfer the knowledge to the target language using significantly lesser resources. We also release a large-scale visual question answering dataset in Japanese and Hindi language. Though we restrict our work to visual question answering, our model can be extended to any sequence-level classification task, and it can be extended to other languages as well. This paper focuses on two languages for the visual question answering task - Japanese and Hindi. Our pipeline outperforms the current state-of-the-art models by a relative increase of 4.4% and 13.4% respectively in accuracy.


title: Molecular dynamics simulation of ion dynamics within SPEEK polymer electrolyte of PEM Fuel Cells

abstract: Proton transport property is studied by modelling the intermolecular pair correlation functions of the proton ion with the electrode and the electrolyte of a polymer electrolyte fuel cell (PEMFC) by using Materials-Studio and then applying molecular dynamics simulation. A stable structure of the novel electrode design is obtained using density functional theory. When the polymer electrolyte is assumed as anhydrous, the efficiency of the proton transport increases. Analysis of the proton coordination numbers shows that more protons are found in the region of oxygen than sulfur atoms of the Sulfonic acid Ether Ester Ketone~(SEEK) electrolyte. The proton conductivity values are increased with including interaction effects from electrode compared to without. At a temperature of 333 K, these values of ion conductivity are $7.69 \times 10^{5} \hspace{1mm}\mathrm{S \hspace{1mm}cm^{-1}}$ and $4.28 \times 10^{5} \hspace{1mm}\mathrm{S \hspace{1mm}cm^{-1}}$, respectively, with and without. The network of the hydrogen bond is the path to transport of protons via the processes of hydrogen bond creating and breaking. The values of highest peaks in the radial distribution function ($g(r)$) calculations appear to fall in the hydrogen bond formation region. Thus, it looks that a combination of Pd$\rm_{3}$Ag as an electrode and unhydrated SEEK as an electrolyte could make up for a cost effective components of new design PEMFC.


title: Position Paper: Online Modeling for Offline Planning

abstract: The definition and representation of planning problems is at the heart of AI planning research. A key part is the representation of action models. Decades of advances improving declarative action model representations resulted in numerous theoretical advances, and capable, working, domain-independent planners. However, despite the maturity of the field, AI planning technology is still rarely used outside the research community, suggesting that current representations fail to capture real-world requirements, such as utilizing complex mathematical functions and models learned from data. We argue that this is because the modeling process is assumed to have taken place and completed prior to the planning process, i.e., offline modeling for offline planning. There are several challenges inherent to this approach, including: limited expressiveness of declarative modeling languages; early commitment to modeling choices and computation, that preclude using the most appropriate resolution for each action model -- which can only be known during planning; and difficulty in reliably using non-declarative, learned, models. We therefore suggest to change the AI planning process, such that is carries out online modeling in offline planning, i.e., the use of action models that are computed or even generated as part of the planning process, as they are accessed. This generalizes the existing approach (offline modeling). The proposed definition admits novel planning processes, and we suggest one concrete implementation, demonstrating the approach. We sketch initial results that were obtained as part of a first attempt to follow this approach by planning with action cost estimators. We conclude by discussing open challenges.


title: Evidence for surface spin structures from first order reversal curves in Co3Sn2S2 and Fe3GeTe2 magnetic topological semimetals

abstract: We study magnetization reversal and first order reversal curves for two different magnetic topological semimetals, Co3Sn2S2 and Fe3GeTe2, in a wide temperature range. For the magnetization reversal, we observe strong temperature dependence of the initial (low-temperature) step-like magnetization switchings, so the inverted hysteresis appears at high temperatures. Usually, the inverted hysteresis is a fingerprint of material with two independent magnetic phases, the inversion reflects the phase interaction. First order reversal curve analysis confirms the two-phase behavior even at the lowest temperatures of the experiment. While the bulk ferromagnetic magnetization shows strong temperature dependence, one of the observed phases demonstrates perfect stability below the Curie temperature. The obtained hysteresis loops are of the bow-tie type, which is usually ascribed to appearance of the skyrmionic phase. The described two-phase behavior is mostly identical for Co3Sn2S2 and Fe3GeTe2 magnetic topological semimetals, only the characteristic temperatures differ for these materials. The specifics of our experiment is the excellent temperature stability of the second phase, while the skyrmions are usually observed near the Curie temperature. On the other hand, temperature stability can be expected for surface-state induced spin textures due to the topological protection of surface states in topological semimetals. This also explains the universal behavior of the second phase for two different topological semimetals Co3Sn2S2 and FGT. Both these materials have strongly different bulk properties, the only similarity is the presence of the topological surface states. Thus, we can ascribe the second, temperature-stable magnetic phase to the surface states in topological semimetals.


title: Towards a Coq formalization of a quantified modal logic

abstract: We present a Coq formalization of the Quantified Reflection Calculus with one modality, or $\mathsf{QRC}_1$. This is a decidable, strictly positive, and quantified modal logic previously studied for its applications in proof theory. The highlights are a deep embedding of $\mathsf{QRC}_1$ in the Coq proof assistant, a mechanization of the notion of Kripke model with varying domains and a formalization of the soundness theorem. We focus on the design decisions inherent to the formalization and the insights that led to new and simplified proofs.


title: An efficient semi-supervised quality control system trained using physics-based MRI-artefact generators and adversarial training

abstract: Large medical imaging data sets are becoming increasingly available, but ensuring sample quality without significant artefacts is challenging. Existing methods for identifying imperfections in medical imaging rely on data-intensive approaches, compounded by a scarcity of artefact-rich scans for training machine learning models in clinical research. To tackle this problem, we propose a framework with four main components: 1) artefact generators inspired by magnetic resonance physics to corrupt brain MRI scans and augment a training dataset, 2) abstract and engineered features to represent images compactly, 3) a feature selection process depending on the artefact class to improve classification, and 4) SVM classifiers to identify artefacts. Our contributions are threefold: first, physics-based artefact generators produce synthetic brain MRI scans with controlled artefacts for data augmentation. This will avoid the labour-intensive collection and labelling process of scans with rare artefacts. Second, we propose a pool of abstract and engineered image features to identify 9 different artefacts for structural MRI. Finally, we use an artefact-based feature selection block that, for each class of artefacts, finds the set of features providing the best classification performance. We performed validation experiments on a large data set of scans with artificially-generated artefacts, and in a multiple sclerosis clinical trial where real artefacts were identified by experts, showing that the proposed pipeline outperforms traditional methods. In particular, our data augmentation increases performance by up to 12.5 percentage points on accuracy, precision, and recall. The computational efficiency of our pipeline enables potential real-time deployment, promising high-throughput clinical applications through automated image-processing pipelines driven by quality control systems.


title: Towards Explainable Social Agent Authoring tools: A case study on FAtiMA-Toolkit

abstract: The deployment of Socially Intelligent Agents (SIAs) in learning environments has proven to have several advantages in different areas of application. Social Agent Authoring Tools allow scenario designers to create tailored experiences with high control over SIAs behaviour, however, on the flip side, this comes at a cost as the complexity of the scenarios and its authoring can become overbearing. In this paper we introduce the concept of Explainable Social Agent Authoring Tools with the goal of analysing if authoring tools for social agents are understandable and interpretable. To this end we examine whether an authoring tool, FAtiMA-Toolkit, is understandable and its authoring steps interpretable, from the point-of-view of the author. We conducted two user studies to quantitatively assess the Interpretability, Comprehensibility and Transparency of FAtiMA-Toolkit from the perspective of a scenario designer. One of the key findings is the fact that FAtiMA-Toolkit's conceptual model is, in general, understandable, however the emotional-based concepts were not as easily understood and used by the authors. Although there are some positive aspects regarding the explainability of FAtiMA-Toolkit, there is still progress to be made to achieve a fully explainable social agent authoring tool. We provide a set of key concepts and possible solutions that can guide developers to build such tools.


title: Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution

abstract: As a highly ill-posed issue, single image super-resolution (SISR) has been widely investigated in recent years. The main task of SISR is to recover the information loss caused by the degradation procedure. According to the Nyquist sampling theory, the degradation leads to aliasing effect and makes it hard to restore the correct textures from low-resolution (LR) images. In practice, there are correlations and self-similarities among the adjacent patches in the natural images. This paper considers the self-similarity and proposes a hierarchical image super-resolution network (HSRNet) to suppress the influence of aliasing. We consider the SISR issue in the optimization perspective, and propose an iterative solution pattern based on the half-quadratic splitting (HQS) method. To explore the texture with local image prior, we design a hierarchical exploration block (HEB) and progressive increase the receptive field. Furthermore, multi-level spatial attention (MSA) is devised to obtain the relations of adjacent feature and enhance the high-frequency information, which acts as a crucial role for visual experience. Experimental result shows HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.


title: Building Robust Ensembles via Margin Boosting

abstract: In the context of adversarial robustness, a single model does not usually have enough power to defend against all possible adversarial attacks, and as a result, has sub-optimal robustness. Consequently, an emerging line of work has focused on learning an ensemble of neural networks to defend against adversarial attacks. In this work, we take a principled approach towards building robust ensembles. We view this problem from the perspective of margin-boosting and develop an algorithm for learning an ensemble with maximum margin. Through extensive empirical evaluation on benchmark datasets, we show that our algorithm not only outperforms existing ensembling techniques, but also large models trained in an end-to-end fashion. An important byproduct of our work is a margin-maximizing cross-entropy (MCE) loss, which is a better alternative to the standard cross-entropy (CE) loss. Empirically, we show that replacing the CE loss in state-of-the-art adversarial training techniques with our MCE loss leads to significant performance improvement.


title: Random-bond antiferromagnetic Ising model in a field

abstract: Using combinatorial optimisation techniques we study the critical properties of the two- and the three-dimensional Ising model with uniformly distributed random antiferromagnetic couplings $(1 \le J_i \le 2)$ in the presence of a homogeneous longitudinal field, $h$, at zero temperature. In finite systems of linear size, $L$, we measure the average correlation function, $C_L(\ell,h)$, when the sites are either on the same sub-lattice, or they belong to different sub-lattices. The phase transition, which is of first-order in the pure system, turns to mixed order in two dimensions with critical exponents $1/\nu \approx 0.5$ and $\eta \approx 0.7$. In three dimensions we obtain $1/\nu \approx 0.7$, which is compatible with the value of the random-field Ising model, but we cannot discriminate between second-order and mixed-order transitions.


title: KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Prediction

abstract: Designing accurate deep learning models for molecular property prediction plays an increasingly essential role in drug and material discovery. Recently, due to the scarcity of labeled molecules, self-supervised learning methods for learning generalizable and transferable representations of molecular graphs have attracted lots of attention. In this paper, we argue that there exist two major issues hindering current self-supervised learning methods from obtaining desired performance on molecular property prediction, that is, the ill-defined pre-training tasks and the limited model capacity. To this end, we introduce Knowledge-guided Pre-training of Graph Transformer (KPGT), a novel self-supervised learning framework for molecular graph representation learning, to alleviate the aforementioned issues and improve the performance on the downstream molecular property prediction tasks. More specifically, we first introduce a high-capacity model, named Line Graph Transformer (LiGhT), which emphasizes the importance of chemical bonds and is mainly designed to model the structural information of molecular graphs. Then, a knowledge-guided pre-training strategy is proposed to exploit the additional knowledge of molecules to guide the model to capture the abundant structural and semantic information from large-scale unlabeled molecular graphs. Extensive computational tests demonstrated that KPGT can offer superior performance over current state-of-the-art methods on several molecular property prediction tasks.


title: DeepOPF-AL: Augmented Learning for Solving AC-OPF Problems with Multiple Load-Solution Mappings

abstract: The existence of multiple load-solution mappings of non-convex AC-OPF problems poses a fundamental challenge to deep neural network (DNN) schemes. As the training dataset may contain a mixture of data points corresponding to different load-solution mappings, the DNN can fail to learn a legitimate mapping and generate inferior solutions. We propose DeepOPF-AL as an augmented-learning approach to tackle this issue. The idea is to train a DNN to learn a unique mapping from an augmented input, i.e., (load, initial point), to the solution generated by an iterative OPF solver with the load and initial point as intake. We then apply the learned augmented mapping to solve AC-OPF problems much faster than conventional solvers. Simulation results over IEEE test cases show that DeepOPF-AL achieves noticeably better optimality and similar feasibility and speedup performance, as compared to a recent DNN scheme, with the same DNN size yet elevated training complexity.


title: Evolution of circuit complexity in a harmonic chain under multiple quenches

abstract: We study Nielsen's circuit complexity in a periodic harmonic oscillator chain, under single and multiple quenches. In a multiple quench scenario, it is shown that the complexity shows remarkably different behaviour compared to the other information theoretic measures, such as the entanglement entropy. In particular, after two successive quenches, when the frequency returns to its initial value, there is a lower limit of complexity, which cannot be made to approach zero. Further, we show that by applying a large number of successive quenches, the complexity of the time evolved state can be increased to a high value, which is not possible by applying a single quench. This model also exhibits the interesting phenomenon of crossover of complexities between two successive quenches performed at different times.


title: Localizing Semantic Patches for Accelerating Image Classification

abstract: Existing works often focus on reducing the architecture redundancy for accelerating image classification but ignore the spatial redundancy of the input image. This paper proposes an efficient image classification pipeline to solve this problem. We first pinpoint task-aware regions over the input image by a lightweight patch proposal network called AnchorNet. We then feed these localized semantic patches with much smaller spatial redundancy into a general classification network. Unlike the popular design of deep CNN, we aim to carefully design the Receptive Field of AnchorNet without intermediate convolutional paddings. This ensures the exact mapping from a high-level spatial location to the specific input image patch. The contribution of each patch is interpretable. Moreover, AnchorNet is compatible with any downstream architecture. Experimental results on ImageNet show that our method outperforms SOTA dynamic inference methods with fewer inference costs. Our code is available at https://github.com/winycg/AnchorNet.