title: Multi-Camera Vehicle Counting Using Edge-AI

abstract: This paper presents a novel solution to automatically count vehicles in a parking lot using images captured by smart cameras. Unlike most of the literature on this task, which focuses on the analysis of single images, this paper proposes the use of multiple visual sources to monitor a wider parking area from different perspectives. The proposed multi-camera system is capable of automatically estimate the number of cars present in the entire parking lot directly on board the edge devices. It comprises an on-device deep learning-based detector that locates and counts the vehicles from the captured images and a decentralized geometric-based approach that can analyze the inter-camera shared areas and merge the data acquired by all the devices. We conduct the experimental evaluation on an extended version of the CNRPark-EXT dataset, a collection of images taken from the parking lot on the campus of the National Research Council (CNR) in Pisa, Italy. We show that our system is robust and takes advantage of the redundant information deriving from the different cameras, improving the overall performance without requiring any extra geometrical information of the monitored scene.


title: Local well-posedness of Dirac equations with nonlinearity derived from honeycomb structure in 2 dimensions

abstract: The aim of this paper is to show the local well-posedness of 2 dimensional Dirac equations with power type and Hartree type nonlinearity derived from honeycomb structure in $H^s$ for $s>\frac78$ and $s>\frac38$, respectively. We also provide the smoothness failure of flows of Dirac equations.


title: Hierarchy and robustness of multilevel two-time temporal quantum correlations

abstract: Quantum steering refers to correlations that can be classified as intermediate between entanglement and Bell nonlocality. Every state exhibiting Bell nonlocality exhibits also quantum steering and every state exhibiting quantum steering is also entangled. In low dimensional cases similar hierarchical relations have been observed between the temporal counterparts of these correlations. Here, we study the hierarchy of such temporal correlations for a general multilevel quantum system. We demonstrate that the same hierarchy holds for two definitions of state over time. In order to compare different types of temporal correlations, we show that temporal counterparts of Bell nonlocality and entanglement can be quantified with a temporal nonlocality robustness and temporal entanglement robustness. Our numerical result reveal that in contrast to temporal steering, for temporal nonlocality to manifest itself we require the initial state not to be in a completely mixed state.


title: Semi-Supervised Domain Adaptation via Adaptive and Progressive Feature Alignment

abstract: Contemporary domain adaptive semantic segmentation aims to address data annotation challenges by assuming that target domains are completely unannotated. However, annotating a few target samples is usually very manageable and worthwhile especially if it improves the adaptation performance substantially. This paper presents SSDAS, a Semi-Supervised Domain Adaptive image Segmentation network that employs a few labeled target samples as anchors for adaptive and progressive feature alignment between labeled source samples and unlabeled target samples. We position the few labeled target samples as references that gauge the similarity between source and target features and guide adaptive inter-domain alignment for learning more similar source features. In addition, we replace the dissimilar source features by high-confidence target features continuously during the iterative training process, which achieves progressive intra-domain alignment between confident and unconfident target features. Extensive experiments show the proposed SSDAS greatly outperforms a number of baselines, i.e., UDA-based semantic segmentation and SSDA-based image classification. In addition, SSDAS is complementary and can be easily incorporated into UDA-based methods with consistent improvements in domain adaptive semantic segmentation.


title: Probing heavy charged fermions at $e^+e^-$ collider using the Optimal Observable Technique

abstract: In this work we study the production of color-neutral and singly-charged heavy leptons at the proposed International Linear Collider. We use the optimal observable technique to determine the statistical accuracy to which the coupling of such fermions to the $Z$ gauge boson (vector, axial or chiral) can be measured. We also consider a UV-complete model that contains these particles as well as a dark matter candidate, and consider some observable effects involving both; the correspondence to chargino production in supersymmetric models with heavy sleptons is briefly discussed.


title: Navigating to the Best Policy in Markov Decision Processes

abstract: We investigate the classical active pure exploration problem in Markov Decision Processes, where the agent sequentially selects actions and, from the resulting system trajectory, aims at identifying the best policy as fast as possible. We propose a problem-dependent lower bound on the average number of steps required before a correct answer can be given with probability at least $1-\delta$. We further provide the first algorithm with an instance-specific sample complexity in this setting. This algorithm addresses the general case of communicating MDPs; we also propose a variant with a reduced exploration rate (and hence faster convergence) under an additional ergodicity assumption. This work extends previous results relative to the \emph{generative setting}~\cite{pmlr-v139-marjani21a}, where the agent could at each step query the random outcome of any (state, action) pair. In contrast, we show here how to deal with the \emph{navigation constraints}, induced by the \emph{online setting}. Our analysis relies on an ergodic theorem for non-homogeneous Markov chains which we consider of wide interest in the analysis of Markov Decision Processes.


title: Numerical Composition of Differential Privacy

abstract: We give a fast algorithm to optimally compose privacy guarantees of differentially private (DP) algorithms to arbitrary accuracy. Our method is based on the notion of privacy loss random variables to quantify the privacy loss of DP algorithms. The running time and memory needed for our algorithm to approximate the privacy curve of a DP algorithm composed with itself $k$ times is $\tilde{O}(\sqrt{k})$. This improves over the best prior method by Koskela et al. (2020) which requires $\tilde{\Omega}(k^{1.5})$ running time. We demonstrate the utility of our algorithm by accurately computing the privacy loss of DP-SGD algorithm of Abadi et al. (2016) and showing that our algorithm speeds up the privacy computations by a few orders of magnitude compared to prior work, while maintaining similar accuracy.


title: Extraordinary transverse spin: Hidden vorticity of the energy flow and momentum distributions in propagating light fields

abstract: Spatially inhomogeneous fields of electromagnetic guided modes exhibit a complex of extraordinary dynamical properties such as the polarization-dependent transverse momentum, helicity-independent transverse spin, spin-associated non-reciprocity and unidirectional propagation, etc. Recently, the remarkable relationship has been established between the spin and propagation features of such fields, expressed through the spin-momentum equations [Proc. Natl. Acad. Sci. 118 (2021) e2018816118] connecting the wave spin with the curl of momentum. Here, the meaning, limitations and specific forms of this correspondence are further investigated, involving the physically transparent and consistent examples of paraxial light fields, plane-wave superpositions and evanescent waves. The conclusion is inferred that the spin-momentum equation is an attribute of guided waves with well defined direction of propagation, and it unites the helicity-independent "extraordinary" transverse spin with the spatially-inhomogeneous longitudinal field momentum (energy flow) density. Physical analogies with the layered hydrodynamic flows and possible generalizations for other wave fields are discussed. The results can be useful in optical trapping, manipulation and the data processing techniques.


title: Tetrad: Actively Secure 4PC for Secure Training and Inference

abstract: Mixing arithmetic and boolean circuits to perform privacy-preserving machine learning has become increasingly popular. Towards this, we propose a framework for the case of four parties with at most one active corruption called Tetrad. Tetrad works over rings and supports two levels of security, fairness and robustness. The fair multiplication protocol costs 5 ring elements, improving over the state-of-the-art Trident (Chaudhari et al. NDSS'20). A key feature of Tetrad is that robustness comes for free over fair protocols. Other highlights across the two variants include (a) probabilistic truncation without overhead, (b) multi-input multiplication protocols, and (c) conversion protocols to switch between the computational domains, along with a tailor-made garbled circuit approach. Benchmarking of Tetrad for both training and inference is conducted over deep neural networks such as LeNet and VGG16. We found that Tetrad is up to 4 times faster in ML training and up to 5 times faster in ML inference. Tetrad is also lightweight in terms of deployment cost, costing up to 6 times less than Trident.


title: SURPRISE! and When to Schedule It

abstract: Information flow measures, over the duration of a game, the audience's belief of who will win, and thus can reflect the amount of surprise in a game. To quantify the relationship between information flow and audiences' perceived quality, we conduct a case study where subjects watch one of the world's biggest esports events, LOL S10. In addition to eliciting information flow, we also ask subjects to report their rating for each game. We find that the amount of surprise in the end of the game plays a dominant role in predicting the rating. This suggests the importance of incorporating when the surprise occurs, in addition to the amount of surprise, in perceived quality models. For content providers, it implies that everything else being equal, it is better for twists to be more likely to happen toward the end of a show rather than uniformly throughout.


title: Patch Slimming for Efficient Vision Transformers

abstract: This paper studies the efficiency problem for visual transformers by excavating redundant calculation in given networks. The recent transformer architecture has demonstrated its effectiveness for achieving excellent performance on a series of computer vision tasks. However, similar to that of convolutional neural networks, the huge computational cost of vision transformers is still a severe issue. Considering that the attention mechanism aggregates different patches layer-by-layer, we present a novel patch slimming approach that discards useless patches in a top-down paradigm. We first identify the effective patches in the last layer and then use them to guide the patch selection process of previous layers. For each layer, the impact of a patch on the final output feature is approximated and patches with less impact will be removed. Experimental results on benchmark datasets demonstrate that the proposed method can significantly reduce the computational costs of vision transformers without affecting their performances. For example, over 45% FLOPs of the ViT-Ti model can be reduced with only 0.2% top-1 accuracy drop on the ImageNet dataset.


title: Region-aware Adaptive Instance Normalization for Image Harmonization

abstract: Image composition plays a common but important role in photo editing. To acquire photo-realistic composite images, one must adjust the appearance and visual style of the foreground to be compatible with the background. Existing deep learning methods for harmonizing composite images directly learn an image mapping network from the composite to the real one, without explicit exploration on visual style consistency between the background and the foreground images. To ensure the visual style consistency between the foreground and the background, in this paper, we treat image harmonization as a style transfer problem. In particular, we propose a simple yet effective Region-aware Adaptive Instance Normalization (RAIN) module, which explicitly formulates the visual style from the background and adaptively applies them to the foreground. With our settings, our RAIN module can be used as a drop-in module for existing image harmonization networks and is able to bring significant improvements. Extensive experiments on the existing image harmonization benchmark datasets show the superior capability of the proposed method. Code is available at {https://github.com/junleen/RainNet}.


title: Orders of strong and weak averaging principle for multiscale SPDEs driven by $\alpha$-stable process

abstract: In this paper, the averaging principle is studied for a class of multiscale stochastic partial differential equations driven by $\alpha$-stable process, where $\alpha\in(1,2)$. Using the technique of Poisson equation, the orders of strong and weak convergence are given $1-1/\alpha$ and $1-r$ for any $r\in (0,1)$ respectively. The main results extend Wiener noise considered by Br\'{e}hier in [6] and Ge et al. in [17] to $\alpha$-stable process, and the finite dimensional case considered by Sun et al. in [39] to the infinite dimensional case.


title: Multi-armed Bandit Algorithms on System-on-Chip: Go Frequentist or Bayesian?

abstract: Multi-armed Bandit (MAB) algorithms identify the best arm among multiple arms via exploration-exploitation trade-off without prior knowledge of arm statistics. Their usefulness in wireless radio, IoT, and robotics demand deployment on edge devices, and hence, a mapping on system-on-chip (SoC) is desired. Theoretically, the Bayesian approach-based Thompson Sampling (TS) algorithm offers better performance than the frequentist approach-based Upper Confidence Bound (UCB) algorithm. However, TS is not synthesizable due to Beta function. We address this problem by approximating it via a pseudo-random number generator-based approach and efficiently realize the TS algorithm on Zynq SoC. In practice, the type of arms distribution (e.g., Bernoulli, Gaussian, etc.) is unknown and hence, a single algorithm may not be optimal. We propose a reconfigurable and intelligent MAB (RI-MAB) framework. Here, intelligence enables the identification of appropriate MAB algorithms for a given environment, and reconfigurability allows on-the-fly switching between algorithms on the SoC. This eliminates the need for parallel implementation of algorithms resulting in huge savings in resources and power consumption. We analyze the functional correctness, area, power, and execution time of the proposed and existing architectures for various arm distributions, word-length, and hardware-software co-design approaches. We demonstrate the superiority of the RI-MAB over TS and UCB only architectures.


title: Reinforcement Learning for Assignment Problem with Time Constraints

abstract: We present an end-to-end framework for the Assignment Problem with multiple tasks mapped to a group of workers, using reinforcement learning while preserving many constraints. Tasks and workers have time constraints and there is a cost associated with assigning a worker to a task. Each worker can perform multiple tasks until it exhausts its allowed time units (capacity). We train a reinforcement learning agent to find near optimal solutions to the problem by minimizing total cost associated with the assignments while maintaining hard constraints. We use proximal policy optimization to optimize model parameters. The model generates a sequence of actions in real-time which correspond to task assignment to workers, without having to retrain for changes in the dynamic state of the environment. In our problem setting reward is computed as negative of the assignment cost. We also demonstrate our results on bin packing and capacitated vehicle routing problem, using the same framework. Our results outperform Google OR-Tools using MIP and CP-SAT solvers with large problem instances, in terms of solution quality and computation time.


title: Neutrino-Nucleus scattering in the SuSA model

abstract: The Super-Scaling Approach (SuSA) model, based on the analogies between electron and neutrino interactions with nuclei, is reviewed and its application to the description of neutrino-nucleus scattering is presented. The contribution of both one- and two-body relativistic currents is considered. A selection of results is presented where theoretical predictions are compared with cross section measurements from the main ongoing neutrino oscillation experiments.


title: A controllability method for Maxwell's equations

abstract: We propose a controllability method for the numerical solution of time-harmonic Maxwell's equations in their first-order formulation. By minimizing a quadratic cost functional, which measures the deviation from periodicity, the controllability method determines iteratively a periodic solution in the time domain. At each conjugate gradient iteration, the gradient of the cost functional is simply computed by running any time-dependent simulation code forward and backward for one period, thus leading to a non-intrusive implementation easily integrated into existing software. Moreover, the proposed algorithm automatically inherits the parallelism, scalability, and low memory footprint of the underlying time-domain solver. Since the time-periodic solution obtained by minimization is not necessarily unique, we apply a cheap post-processing filtering procedure which recovers the time-harmonic solution from any minimizer. Finally, we present a series of numerical examples which show that our algorithm greatly speeds up the convergence towards the desired time-harmonic solution when compared to simply running the time-marching code until the time-harmonic regime is eventually reached.


title: Convolutional Neural Networks with Gated Recurrent Connections

abstract: The convolutional neural network (CNN) has become a basic model for solving many computer vision problems. In recent years, a new class of CNNs, recurrent convolution neural network (RCNN), inspired by abundant recurrent connections in the visual systems of animals, was proposed. The critical element of RCNN is the recurrent convolutional layer (RCL), which incorporates recurrent connections between neurons in the standard convolutional layer. With increasing number of recurrent computations, the receptive fields (RFs) of neurons in RCL expand unboundedly, which is inconsistent with biological facts. We propose to modulate the RFs of neurons by introducing gates to the recurrent connections. The gates control the amount of context information inputting to the neurons and the neurons' RFs therefore become adaptive. The resulting layer is called gated recurrent convolution layer (GRCL). Multiple GRCLs constitute a deep model called gated RCNN (GRCNN). The GRCNN was evaluated on several computer vision tasks including object recognition, scene text recognition and object detection, and obtained much better results than the RCNN. In addition, when combined with other adaptive RF techniques, the GRCNN demonstrated competitive performance to the state-of-the-art models on benchmark datasets for these tasks. The codes are released at \href{https://github.com/Jianf-Wang/GRCNN}{https://github.com/Jianf-Wang/GRCNN}.


title: Expected number of zeros of random power series with finitely dependent Gaussian coefficients

abstract: We are concerned with zeros of random power series with coefficients being a stationary, centered, complex Gaussian process. We show that the expected number of zeros in every smooth domain in the disk of convergence is less than that of the hyperbolic GAF with i.i.d. coefficients. When coefficients are finitely dependent, i.e., the spectral density is a trigonometric polynomial, we derive precise asymptotics of the expected number of zeros inside the disk of radius $r$ centered at the origin as $r$ tends to the radius of convergence, in the proof of which we clarify that the negative contribution to the number of zeros stems from the zeros of the spectral density.


title: Optimal Taxation of Assets

abstract: The optimal taxation of assets requires attention to two concerns: 1) the elasticity of the supply of assets and 2) the impact of taxing assets on distributional objectives. The most efficient way to attend to these two concerns is to tax assets of different types separately, rather than having one tax on all assets. When assets are created by specialized effort rather than by saving, as with innovations, discoveries of mineral deposits and development of unregulated natural monopolies, it is interesting to consider a regime in which the government awards a prize for the creation of the asset and then collects the remaining value of the asset in taxes. Analytically, the prize is like a wage after taxes. In this perspective, prizes are awarded based on a variation on optimal taxation theory, while assets of different types are taxed in divergent ways, depending on their characteristics. Some categories of assets are abolished.


title: Antenna Array Diagnosis for Millimeter-Wave MIMO Systems

abstract: The densely packed antennas of millimeter-Wave (mmWave) MIMO systems are often blocked by the rain, snow, dust and even by fingers, which will change the channel's characteristics and degrades the system's performance. In order to solve this problem, we propose a cross-entropy inspired antenna array diagnosis detection (CE-AAD) technique by exploiting the correlations of adjacent antennas, when blockages occur at the transmitter. Then, we extend the proposed CE-AAD algorithm to the case, where blockages occur at transmitter and receiver simultaneously. Our simulation results show that the proposed CE-AAD algorithm outperforms its traditional counterparts.


title: Primordial black holes formation in the inflationary model with field-dependent kinetic term for quartic and natural potentials

abstract: Within the framework of inflationary model with field-dependent kinetic term for quartic and natural potentials, we investigate generation of the primordial black holes (PBHs) and induced gravitational waves (GWs). In this setup, we consider a kinetic function as $G(\phi)=g_I(\phi)\big(1+g_{II}(\phi)\big)$ and show that in the presence of first term $g_I(\phi)$ both quartic and natural potentials, in contrast to the standard model of inflation, can be consistent, with the 68\% CL of Planck observations. Besides, the second term $g_{II}(\phi)$ can cause a significant enhancement in the primordial curvature perturbations at the small scales which results the PBHs formation. For the both potentials, we obtain an enhancement in the scalar power spectrum at the scales $k\sim10^{12}~\rm Mpc^{-1}$, $10^{8}~\rm Mpc^{-1}$, and $10^{5}~\rm Mpc^{-1}$, which causes PBHs production in mass scales around $10^{-13}M_{\odot}$, $10^{-5}M_{\odot}$, and $10 M_{\odot}$, respectively. Observational constraints confirm that PBHs with a mass scale of $10^{-13}M_{\odot}$ can constitute the total of dark matter in the universe. Furthermore, we estimate the energy density parameter of induced GWs which can be examined by the observation. Also we conclude that it can be parametrized as a power-law function $\Omega_{\rm GW}\sim (f/f_c)^n$, where the power index equals $n=3-2/\ln(f_c/f)$ in the infrared limit $f\ll f_{c}$.


title: An End-to-End Breast Tumour Classification Model Using Context-Based Patch Modelling- A BiLSTM Approach for Image Classification

abstract: Researchers working on computational analysis of Whole Slide Images (WSIs) in histopathology have primarily resorted to patch-based modelling due to large resolution of each WSI. The large resolution makes WSIs infeasible to be fed directly into the machine learning models due to computational constraints. However, due to patch-based analysis, most of the current methods fail to exploit the underlying spatial relationship among the patches. In our work, we have tried to integrate this relationship along with feature-based correlation among the extracted patches from the particular tumorous region. For the given task of classification, we have used BiLSTMs to model both forward and backward contextual relationship. RNN based models eliminate the limitation of sequence size by allowing the modelling of variable size images within a deep learning model. We have also incorporated the effect of spatial continuity by exploring different scanning techniques used to sample patches. To establish the efficiency of our approach, we trained and tested our model on two datasets, microscopy images and WSI tumour regions. After comparing with contemporary literature we achieved the better performance with accuracy of 90% for microscopy image dataset. For WSI tumour region dataset, we compared the classification results with deep learning networks such as ResNet, DenseNet, and InceptionV3 using maximum voting technique. We achieved the highest performance accuracy of 84%. We found out that BiLSTMs with CNN features have performed much better in modelling patches into an end-to-end Image classification network. Additionally, the variable dimensions of WSI tumour regions were used for classification without the need for resizing. This suggests that our method is independent of tumour image size and can process large dimensional images without losing the resolution details.


title: Consensus Analysis over Clustered Networks of Multi-Agent Systems under External Disturbances

abstract: This paper studies a consensus problem of multi-agent systems subjected to external disturbances over the clustered network. It considers that the agents are divided into several clusters. They are almost all the time isolated one from another, which has a directed spanning tree. The goal of agents achieves a common value. To support interaction between clusters with a minimum exchange of information, we consider that each cluster has an agent, who can exchange information to any agents outside of its cluster at some discrete instants of time. Our main contribution proposes a consensus protocol, which takes into account the continuous-time communications among agents inside the clusters and discrete-time communication information across clusters. Accordingly, the consensus and the robust $\mathcal{H}_{\infty}$ consensus over the clustered network are respectively analyzed. Thanks to results from matrix theory and algebraic graph theory, we show that the proposed control protocols can solve the problems mentioned above. Finally, a numerical example is given to show the effectiveness of the proposed theoretical results.


title: Conditional Contrastive Learning for Improving Fairness in Self-Supervised Learning

abstract: Contrastive self-supervised learning (SSL) learns an embedding space that maps similar data pairs closer and dissimilar data pairs farther apart. Despite its success, one issue has been overlooked: the fairness aspect of representations learned using contrastive SSL. Without mitigation, contrastive SSL techniques can incorporate sensitive information such as gender or race and cause potentially unfair predictions on downstream tasks. In this paper, we propose a Conditional Contrastive Learning (CCL) approach to improve the fairness of contrastive SSL methods. Our approach samples positive and negative pairs from distributions conditioning on the sensitive attribute, or empirically speaking, sampling positive and negative pairs from the same gender or the same race. We show that our approach provably maximizes the conditional mutual information between the learned representations of the positive pairs, and reduces the effect of the sensitive attribute by taking it as the conditional variable. On seven fairness and vision datasets, we empirically demonstrate that the proposed approach achieves state-of-the-art downstream performances compared to unsupervised baselines and significantly improves the fairness of contrastive SSL models on multiple fairness metrics.


title: Ensemble Defense with Data Diversity: Weak Correlation Implies Strong Robustness

abstract: In this paper, we propose a framework of filter-based ensemble of deep neuralnetworks (DNNs) to defend against adversarial attacks. The framework builds an ensemble of sub-models -- DNNs with differentiated preprocessing filters. From the theoretical perspective of DNN robustness, we argue that under the assumption of high quality of the filters, the weaker the correlations of the sensitivity of the filters are, the more robust the ensemble model tends to be, and this is corroborated by the experiments of transfer-based attacks. Correspondingly, we propose a principle that chooses the specific filters with smaller Pearson correlation coefficients, which ensures the diversity of the inputs received by DNNs, as well as the effectiveness of the entire framework against attacks. Our ensemble models are more robust than those constructed by previous defense methods like adversarial training, and even competitive with the classical ensemble of adversarial trained DNNs under adversarial attacks when the attacking radius is large.


title: Qualitative properties for the 1-D Impulsive Wave Equation: Controllability and Observability

abstract: In this paper, we establish some important results for the impulsive wave equation. We begin by proving the existence of a solution. Then, we study the impulse approximate controllability where the control function acts on a subdomain $\omega$ and at one instant of time $\tau \in (0, T)$. Afterward, we study impulse observability


title: Integrating Auxiliary Information in Self-supervised Learning

abstract: This paper presents to integrate the auxiliary information (e.g., additional attributes for data such as the hashtags for Instagram images) in the self-supervised learning process. We first observe that the auxiliary information may bring us useful information about data structures: for instance, the Instagram images with the same hashtags can be semantically similar. Hence, to leverage the structural information from the auxiliary information, we present to construct data clusters according to the auxiliary information. Then, we introduce the Clustering InfoNCE (Cl-InfoNCE) objective that learns similar representations for augmented variants of data from the same cluster and dissimilar representations for data from different clusters. Our approach contributes as follows: 1) Comparing to conventional self-supervised representations, the auxiliary-information-infused self-supervised representations bring the performance closer to the supervised representations; 2) The presented Cl-InfoNCE can also work with unsupervised constructed clusters (e.g., k-means clusters) and outperform strong clustering-based self-supervised learning approaches, such as the Prototypical Contrastive Learning (PCL) method; 3) We show that Cl-InfoNCE may be a better approach to leverage the data clustering information, by comparing it to the baseline approach - learning to predict the clustering assignments with cross-entropy loss. For analysis, we connect the goodness of the learned representations with the statistical relationships: i) the mutual information between the labels and the clusters and ii) the conditional entropy of the clusters given the labels.


title: Bidirectional Distillation for Top-K Recommender System

abstract: Recommender systems (RS) have started to employ knowledge distillation, which is a model compression technique training a compact model (student) with the knowledge transferred from a cumbersome model (teacher). The state-of-the-art methods rely on unidirectional distillation transferring the knowledge only from the teacher to the student, with an underlying assumption that the teacher is always superior to the student. However, we demonstrate that the student performs better than the teacher on a significant proportion of the test set, especially for RS. Based on this observation, we propose Bidirectional Distillation (BD) framework whereby both the teacher and the student collaboratively improve with each other. Specifically, each model is trained with the distillation loss that makes to follow the other's prediction along with its original loss function. For effective bidirectional distillation, we propose rank discrepancy-aware sampling scheme to distill only the informative knowledge that can fully enhance each other. The proposed scheme is designed to effectively cope with a large performance gap between the teacher and the student. Trained in the bidirectional way, it turns out that both the teacher and the student are significantly improved compared to when being trained separately. Our extensive experiments on real-world datasets show that our proposed framework consistently outperforms the state-of-the-art competitors. We also provide analyses for an in-depth understanding of BD and ablation studies to verify the effectiveness of each proposed component.


title: Discrete Frechet distance for closed curves

abstract: The paper presents a discrete variation of the Frechet distance between closed curves, which can be seen as an approximation of the continuous measure. A rather straightforward approach to compute the discrete Frechet distance between two closed sequences of m and n points using binary search takes O(mn log mn) time. We present an algorithm that takes O(mn log* mn) time, where log* is the iterated logarithm.


title: Frequency analysis of OGLE-IV photometry for classical Cepheids in Galactic fields: non-radial modes and modulations

abstract: We analyse photometry of $\sim$2000 Galactic Cepheids available in the OGLE Collection of Variable Stars. We analyse both Galactic disk and Galactic bulge fields; stars classified both as single- and multi-periodic. Our goal was to search for additional low-amplitude variability. We extend the sample of multi-mode radial pulsators by identifying ten new candidates for double-mode and six new candidates for triple-mode pulsation. In the first overtone OGLE sample, we found twelve Cepheids with additional periodicity having period ratio $P_{\rm x}/P_{\rm 1O}\in (0.60,\, 0.65)$. These periodicities do not correspond to any other radial mode. While such variables are abundant in the Magellanic Clouds, only one Cepheid of this class was known in the Galaxy before our analysis. Comparing our sample with the Magellanic Cloud Cepheids we note a systematic shift towards longer pulsation periods for more metal rich Galactic stars. Moreover in eleven stars we find one more type of additional variability, with characteristic frequencies close to half of that reported in the group with (0.60,\, 0.65) period ratios. Two out of the above inventory show simultaneous presence of both signals. Most likely, origin of these signals is connected to excitation of non-radial pulsation modes. We report three Cepheids with low-amplitude periodic modulation of pulsation: two stars are single-mode fundamental and first overtone Cepheids and one is a double-mode Cepheid pulsating simultaneously in fundamental and in first overtone modes. Only the former mode is modulated. It is a first detection of periodic modulation of pulsation in this type of double-mode Cepheids.


title: Spherical adjunctions of stable $\infty$-categories and the relative S-construction

abstract: We develop the theory of semi-orthogonal decompositions and spherical functors in the framework of stable $\infty$-categories. Building on this, we study the relative Waldhausen S-construction $S_\bullet(F)$ of a spherical functor $F$ and equip it with a natural paracyclic structure (``rotational symmetry''). This fulfills a part of the general program to provide a rigorous account of perverse schobers which are (thus far conjectural) categorifications of perverse sheaves. Namely, in terms of our previous identification of perverse sheaves on Riemann surfaces with Milnor sheaves, the relative $S$-construction with its paracyclic symmetry amounts to a categorification of the stalks of a Milnor sheaf at a singularity of the corresponding perverse sheaf. The action of the paracyclic rotation is a categorical analog of the monodromy on the vanishing cycles of a perverse sheaf. Having this local categorification in mind, we may view the S-construction of a spherical functor as defining a schober locally at a singularity. Each component $S_n(F)$ can be interpreted as a partially wrapped Fukaya category of the disk with coefficients in the schober and with $n+1$ stops at the boundary.


title: RDA: Robust Domain Adaptation via Fourier Adversarial Attacking

abstract: Unsupervised domain adaptation (UDA) involves a supervised loss in a labeled source domain and an unsupervised loss in an unlabeled target domain, which often faces more severe overfitting (than classical supervised learning) as the supervised source loss has clear domain gap and the unsupervised target loss is often noisy due to the lack of annotations. This paper presents RDA, a robust domain adaptation technique that introduces adversarial attacking to mitigate overfitting in UDA. We achieve robust domain adaptation by a novel Fourier adversarial attacking (FAA) method that allows large magnitude of perturbation noises but has minimal modification of image semantics, the former is critical to the effectiveness of its generated adversarial samples due to the existence of 'domain gaps'. Specifically, FAA decomposes images into multiple frequency components (FCs) and generates adversarial samples by just perturbating certain FCs that capture little semantic information. With FAA-generated samples, the training can continue the 'random walk' and drift into an area with a flat loss landscape, leading to more robust domain adaptation. Extensive experiments over multiple domain adaptation tasks show that RDA can work with different computer vision tasks with superior performance.


title: Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progression

abstract: Modeling a system's temporal behaviour in reaction to external stimuli is a fundamental problem in many areas. Pure Machine Learning (ML) approaches often fail in the small sample regime and cannot provide actionable insights beyond predictions. A promising modification has been to incorporate expert domain knowledge into ML models. The application we consider is predicting the progression of disease under medications, where a plethora of domain knowledge is available from pharmacology. Pharmacological models describe the dynamics of carefully-chosen medically meaningful variables in terms of systems of Ordinary Differential Equations (ODEs). However, these models only describe a limited collection of variables, and these variables are often not observable in clinical environments. To close this gap, we propose the latent hybridisation model (LHM) that integrates a system of expert-designed ODEs with machine-learned Neural ODEs to fully describe the dynamics of the system and to link the expert and latent variables to observable quantities. We evaluated LHM on synthetic data as well as real-world intensive care data of COVID-19 patients. LHM consistently outperforms previous works, especially when few training samples are available such as at the beginning of the pandemic.


title: Hyperspectral imaging with Raman scattered photons: A new paradigm in Raman analysis

abstract: Surface enhanced Raman spectroscopy, is a technique of fundamental importance to analytical science and technology where the amplified Raman spectrum of analytes is used for chemical fingerprinting. Here, we showcase an engineered hierarchical substrate in which the plasmonically active regions are restricted to a micron scale, 2D hexagonal pattern. The Raman signal enhancement of any analyte uniformly coating the substrate is consequently bears a high registry with the 2D pattern. This spatially segregated enhancement allows optical imaging of the 2D pattern solely using the Raman scattered photons from the analyte. While pattern brightness and contrast determine analyte identification and detection sensitivity, the spectrally selective contrast allows for tuning specificity. Conceptual proof of the technique is demonstrated via the acquisition of Raman images with rhodamine and fluorescein and finally applied to detect glucose in 40 mM concentration. The large area imaging and the inherent requirement of spatial uniformity for positive detection implemented using a machine learning based automated pattern recognition protocol increases the statistical confidence of analyte detection. This simultaneously multisite signal detection sacrifices continuous spectral information at the cost of speed, reproducibility and human error via automation of detection of the hyperspectral imaging technique presented here.


title: $G$-deviations of polygons and their applications in Electric Power Engineering

abstract: For any metric space $X$ endowed with the action of a group $G$, and two $n$-gons $\vec x=(x_1,\dots,x_n)\in X^n$ and $\vec y=(y_1,\dots,y_n)\in X^n$ in $X$, we introduce the $G$-deviation $d(G\vec x,\vec y\,)$ of $\vec x$ from $\vec y$ as the distance in $X^n$ from $\vec y$ to the $G$-orbit $G\vec x$ of $\vec x$ in the $n$-th power $X^n$ of $X$. For some groups $G$ of affine transformations of the complex plane, we deduce simple-to-apply formulas for calculating the $G$-derviation between $n$-gons on the complex plane. We apply these formulas for defining new measures of asymmetry of triangles. These new measures can be applied in Electric Power Engineering for evaluating the quality of 3-phase electric power. One of such measures, namely the affine deviation, is espressible via the unbalance degree, which is a standard characteristic of quality of three-phase electric power.


title: A Generative Node-attribute Network Model for Detecting Generalized Structure

abstract: Exploring meaningful structural regularities embedded in networks is a key to understanding and analyzing the structure and function of a network. The node-attribute information can help improve such understanding and analysis. However, most of the existing methods focus on detecting traditional communities, i.e., groupings of nodes with dense internal connections and sparse external ones. In this paper, based on the connectivity behavior of nodes and homogeneity of attributes, we propose a principle model (named GNAN), which can generate both topology information and attribute information. The new model can detect not only community structure, but also a range of other types of structure in networks, such as bipartite structure, core-periphery structure, and their mixture structure, which are collectively referred to as generalized structure. The proposed model that combines topological information and node-attribute information can detect communities more accurately than the model that only uses topology information. The dependency between attributes and communities can be automatically learned by our model and thus we can ignore the attributes that do not contain useful information. The model parameters are inferred by using the expectation-maximization algorithm. And a case study is provided to show the ability of our model in the semantic interpretability of communities. Experiments on both synthetic and real-world networks show that the new model is competitive with other state-of-the-art models.


title: Global well-posedness to stochastic reaction-diffusion equations on the real line $\mathbb{R}$ with superlinear drifts driven by multiplicative space-time white noise

abstract: Consider the stochastic reaction-diffusion equation with logarithmic nonlinearity driven by space-time white noise: \begin{align}\label{1.a} \left\{ \begin{aligned} & \mathrm{d}u(t,x) = \frac{1}{2}\Delta u(t,x)\,\mathrm{d}t+ b(u(t,x)) \,\mathrm{d}t \nonumber\\ & ~~~~~~~~~~~~~~~~ + \sigma(u(t,x)) \,W(\mathrm{d}t,\mathrm{d}x), \ t>0, x\in I , \\ & u(0,x)=u_0(x), \quad x\in I .\nonumber \end{aligned} \right. \end{align} When $I$ is a compact interval, say $I=[0,1]$, the well-posedness of the above equation was established in [DKZ] (Ann. Prob. 47:1,2019). The case where $I=\mathbb{R}$ was left open. The essential obstacle is caused by the explosion of the supremum norm of the solution, $\sup_{x\in\mathbb{R}}|u(t,x)|=\infty$, making the usual truncation procedure invalid. In this paper, we prove that there exists a unique global solution to the stochastic reaction-diffusion equation on the whole real line $\mathbb{R}$ with logarithmic nonlinearity. Because of the nature of the nonlinearity, to get the uniqueness, we are forced to work with the first order moment of the solutions on the space $C_{tem}(\mathbb{R})$ with a specially designed norm $$\sup_{t\leq T, x\in\mathbb{R}}\left(|u(t,x)|e^{-\lambda |x|e^{\beta t}}\right),$$ where, unlike the usual norm in $C_{tem}(\mathbb{R})$, the exponent also depends on time $t$ in a particular way. Our approach depends heavily on the new, precise lower order moment estimates of the stochastic convolution and a new type of Gronwall's inequalities we obtained, which are of interest on their own right.


title: Right-Most Position of a Last Progeny Modified Branching Random Walk

abstract: In this work, we consider a modification of the usual Branching Random Walk (BRW), where we give certain independent and identically distributed (i.i.d.) displacements to all the particles at the $n$-th generation, which may be different from the driving increment distribution. We call this process \emph{last progeny modified branching random walk (LPM-BRW)}. Depending on the value of a parameter, $\theta$, we classify the model in three distinct cases, namely, the boundary case, below the boundary case, and above the boundary case. Under very minimal assumptions on the underlying point process of the increments, we show that at the \emph{boundary case}, $\theta=\theta_0$, where $\theta_0$ is a parameter value associated with the displacement point process, the maximum displacement converges to a limit after only an appropriate centering, which is of the form $c_1 n - c_2 \log n$. We give an explicit formula for the constants $c_1$ and $c_2$ and show that $c_1$ is exactly the same, while $c_2$ is $1/3$ of the corresponding constants of the usual BRW Aidekon (2013). We also characterize the limiting distribution. We further show that below the boundary, $\theta < \theta_0$, the logarithmic correction term is absent. For above the boundary case, $\theta > \theta_0$, the logarithmic correction term is exactly the same as that of the classical BRW. For $\theta \leq \theta_0$, we further derive Brunet-Derrida -type results of point process convergence of our LPM-BRW to a Poisson point process. Our proofs are based on a novel method of coupling the maximum displacement with a \emph{linear statistic} associated with a more well-studied process in statistics, known as the \emph{smoothing transformation}.


title: Graph Infomax Adversarial Learning for Treatment Effect Estimation with Networked Observational Data

abstract: Treatment effect estimation from observational data is a critical research topic across many domains. The foremost challenge in treatment effect estimation is how to capture hidden confounders. Recently, the growing availability of networked observational data offers a new opportunity to deal with the issue of hidden confounders. Unlike networked data in traditional graph learning tasks, such as node classification and link detection, the networked data under the causal inference problem has its particularity, i.e., imbalanced network structure. In this paper, we propose a Graph Infomax Adversarial Learning (GIAL) model for treatment effect estimation, which makes full use of the network structure to capture more information by recognizing the imbalance in network structure. We evaluate the performance of our GIAL model on two benchmark datasets, and the results demonstrate superiority over the state-of-the-art methods.


title: Integrality of the LMOV invariants for framed unknot

abstract: The Labastida-Marin\~o-Ooguri-Vafa (LMOV) invariants are the open string BPS invariants which are expected to be integers based on the string duality conjecture from M-theory. Several explicit formulae of LMOV invariants for framed unknot have been obtained in the literature. In this paper, we present a unified method to deal with the integrality of such explicit formulae. Furthermore, we also prove the integrality of certain LMOV invariants for framed unknot in higher genera.


title: Wideband Channel Estimation for IRS-Aided Systems in the Face of Beam Squint

abstract: Intelligent reflecting surfaces (IRSs) improve both the bandwidth and energy efficiency of wideband communication systems by using low-cost passive elements for reflecting the impinging signals with adjustable phase shifts. To realize the full potential of IRS-aided systems, having accurate channel state information (CSI) is indispensable, but it is challenging to acquire, since these passive devices cannot carry out transmit/receive signal processing. The existing channel estimation methods conceived for wideband IRS-aided communication systems only consider the channel's frequency selectivity, but ignore the effect of beam squint, despite its severe performance degradation. Hence we fill this gap and conceive wideband channel estimation for IRS-aided communication systems by explicitly taking the effect of beam squint into consideration. We demonstrate that the mutual correlation function between the spatial steering vectors and the cascaded two-hop channel reflected by the IRS has two peaks, which leads to a pair of estimated angles for a single propagation path, due to the effect of beam squint. One of these two estimated angles is the frequency-independent `actual angle', while the other one is the frequency-dependent `false angle'. To reduce the influence of false angles on channel estimation, we propose a twin-stage orthogonal matching pursuit (TS-OMP) algorithm.


title: A Deep Variational Bayesian Framework for Blind Image Deblurring

abstract: Blind image deblurring is an important yet very challenging problem in low-level vision. Traditional optimization based methods generally formulate this task as a maximum-a-posteriori estimation or variational inference problem, whose performance highly relies on the handcraft priors for both the latent image and the blur kernel. In contrast, recent deep learning methods generally learn, from a large collection of training images, deep neural networks (DNNs) directly mapping the blurry image to the clean one or to the blur kernel, paying less attention to the physical degradation process of the blurry image. In this paper, we present a deep variational Bayesian framework for blind image deblurring. Under this framework, the posterior of the latent clean image and blur kernel can be jointly estimated in an amortized inference fashion with DNNs, and the involved inference DNNs can be trained by fully considering the physical blur model, together with the supervision of data driven priors for the clean image and blur kernel, which is naturally led to by the evidence lower bound objective. Comprehensive experiments are conducted to substantiate the effectiveness of the proposed framework. The results show that it can not only achieve a promising performance with relatively simple networks, but also enhance the performance of existing DNNs for deblurring.


title: Category Contrast for Unsupervised Domain Adaptation in Visual Tasks

abstract: Instance contrast for unsupervised representation learning has achieved great success in recent years. In this work, we explore the idea of instance contrastive learning in unsupervised domain adaptation (UDA) and propose a novel Category Contrast technique (CaCo) that introduces semantic priors on top of instance discrimination for visual UDA tasks. By considering instance contrastive learning as a dictionary look-up operation, we construct a semantics-aware dictionary with samples from both source and target domains where each target sample is assigned a (pseudo) category label based on the category priors of source samples. This allows category contrastive learning (between target queries and the category-level dictionary) for category-discriminative yet domain-invariant feature representations: samples of the same category (from either source or target domain) are pulled closer while those of different categories are pushed apart simultaneously. Extensive UDA experiments in multiple visual tasks (e.g., segmentation, classification and detection) show that CaCo achieves superior performance as compared with state-of-the-art methods. The experiments also demonstrate that CaCo is complementary to existing UDA methods and generalizable to other learning setups such as unsupervised model adaptation, open-/partial-set adaptation etc.


title: Context-Aware Sparse Deep Coordination Graphs

abstract: Learning sparse coordination graphs adaptive to the coordination dynamics among agents is a long-standing problem in cooperative multi-agent learning. This paper studies this problem and proposes a novel method using the variance of payoff functions to construct context-aware sparse coordination topologies. We theoretically consolidate our method by proving that the smaller the variance of payoff functions is, the less likely action selection will change after removing the corresponding edge. Moreover, we propose to learn action representations to effectively reduce the influence of payoff functions' estimation errors on graph construction. To empirically evaluate our method, we present the Multi-Agent COordination (MACO) benchmark by collecting classic coordination problems in the literature, increasing their difficulty, and classifying them into different types. We carry out a case study and experiments on the MACO and StarCraft II micromanagement benchmark to demonstrate the dynamics of sparse graph learning, the influence of graph sparseness, and the learning performance of our method. (The MACO benchmark and codes are publicly available at https://github.com/TonghanWang/CASEC-MACO-benchmark.)


title: An Empirical Study on Tensor Shape Faults in Deep Learning Systems

abstract: Software developers frequently adopt deep learning (DL) libraries to incorporate learning solutions into software systems. However, misuses of these libraries can cause various DL faults. Among them, tensor shape faults are most prevalent. Tensor shape faults occur when restriction conditions of operations are not met, leading to many system crashes. To support efficient detection and fixing of these faults, we conduct an empirical study to obtain a deep insight. We construct SFData, a set of 146 buggy programs with crashing tensor shape faults (i.e., those causing programs to crash). By analyzing the faults in SFData, we categorize them into four types and get some valuable observations.


title: Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization

abstract: Many popular learning-rate schedules for deep neural networks combine a decaying trend with local perturbations that attempt to escape saddle points and bad local minima. We derive convergence guarantees for bandwidth-based step-sizes, a general class of learning rates that are allowed to vary in a banded region. This framework includes many popular cyclic and non-monotonic step-sizes for which no theoretical guarantees were previously known. We provide worst-case guarantees for SGD on smooth non-convex problems under several bandwidth-based step sizes, including stagewise $1/\sqrt{t}$ and the popular step-decay (constant and then drop by a constant), which is also shown to be optimal. Moreover, we show that its momentum variant converges as fast as SGD with the bandwidth-based step-decay step-size. Finally, we propose novel step-size schemes in the bandwidth-based family and verify their efficiency on several deep neural network training tasks.


title: A flexible control system for atomic, molecular and optical physics experiments

abstract: We have implemented a control system for experiments in atomic, molecular and optical physics based on a commercial low-cost board, featuring a field-programmable gate array as part of a system-on-a-chip on which a Linux operating system is running. The board features Gigabit Ethernet, allowing for fast data transmission and operation of remote experimental systems. A single board can control a set of devices generating digital, analog and radio frequency signals with a precise timing given either by an external or internal clock. Contiguous output and input sampling rates of up to 40 MHz are achievable. Several boards can run synchronously with a timing error approaching 1 ns. For this purpose, a novel auto-synchronization scheme is demonstrated, with possible application in complex distributed experimental setups with demanding timing requests.


title: Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?

abstract: Can models with particular structure avoid being biased towards spurious correlation in out-of-distribution (OOD) generalization? Peters et al. (2016) provides a positive answer for linear cases. In this paper, we use a functional modular probing method to analyze deep model structures under OOD setting. We demonstrate that even in biased models (which focus on spurious correlation) there still exist unbiased functional subnetworks. Furthermore, we articulate and demonstrate the functional lottery ticket hypothesis: full network contains a subnetwork that can achieve better OOD performance. We then propose Modular Risk Minimization to solve the subnetwork selection problem. Our algorithm learns the subnetwork structure from a given dataset, and can be combined with any other OOD regularization methods. Experiments on various OOD generalization tasks corroborate the effectiveness of our method.


title: Heat rectification by two qubits coupled with Dzyaloshinskii-Moriya interaction

abstract: We investigate heat rectification in a two-qubit system coupled via the Dzyaloshinskii-Moriya (DM) interaction. We derive analytical expressions for heat currents and thermal rectification and provide possible physical mechanisms behind the observed results. We show that the anisotropy of DM interaction in itself is insufficient for heat rectification, and some other form of asymmetry is needed. We employ off-resonant qubits as the source of this asymmetry. We find the regime of parameters for higher rectification factors by examining the analytical expressions of rectification obtained from a global master equation solution. In addition, it is shown that the direction and quality of rectification can be controlled via various system parameters. Furthermore, we compare the influence of different orientations of the DM field anisotropy on the performance of heat rectification. Finally, we investigate the possible interplay between quantum correlations and the performance of the quantum thermal rectifier. We find that asymmetry in the coherences is a fundamental resource for the performance of the quantum thermal rectifier.


title: Training Robust Graph Neural Networks with Topology Adaptive Edge Dropping

abstract: Graph neural networks (GNNs) are processing architectures that exploit graph structural information to model representations from network data. Despite their success, GNNs suffer from sub-optimal generalization performance given limited training data, referred to as over-fitting. This paper proposes Topology Adaptive Edge Dropping (TADropEdge) method as an adaptive data augmentation technique to improve generalization performance and learn robust GNN models. We start by explicitly analyzing how random edge dropping increases the data diversity during training, while indicating i.i.d. edge dropping does not account for graph structural information and could result in noisy augmented data degrading performance. To overcome this issue, we consider graph connectivity as the key property that captures graph topology. TADropEdge incorporates this factor into random edge dropping such that the edge-dropped subgraphs maintain similar topology as the underlying graph, yielding more satisfactory data augmentation. In particular, TADropEdge first leverages the graph spectrum to assign proper weights to graph edges, which represent their criticality for establishing the graph connectivity. It then normalizes the edge weights and drops graph edges adaptively based on their normalized weights. Besides improving generalization performance, TADropEdge reduces variance for efficient training and can be applied as a generic method modular to different GNN models. Intensive experiments on real-life and synthetic datasets corroborate theory and verify the effectiveness of the proposed method.


title: Stern and Diffuse Layer Interactions During Ionic Strength Cycling

abstract: Second harmonic generation amplitude and phase measurements are acquired in real time from fused silica:water interfaces that are subjected to ionic strength transitions conducted at pH 5.8. In conjunction with atomistic modeling, we identify correlations between structure in the Stern layer, encoded in the total second-order nonlinear susceptibility, chi(2)tot, and in the diffuse layer, encoded in the product of chi(2)tot and the total interfacial potential, phi(0)tot. chi(2)tot:phi(0)tot correlation plots indicate that the dynamics in the Stern and diffuse layers are decoupled from one another under some conditions (large change in ionic strength), while they change in lockstep under others (smaller change in ionic strength) as the ionic strength in the aqueous bulk solution varies. The quantitative structural and electrostatic information obtained also informs on the molecular origin of hysteresis in ionic strength cycling over fused silica. Atomistic simulations suggest a prominent role of contact ion pairs (as opposed to solvent-separated ion pairs) in the Stern layer. Those simulations also indicate that net water alignment is limited to the first 2 nm from the interface, even at 0 M ionic strength, highlighting water's polarization as an important contributor to nonlinear optical signal generation.


title: MoleHD: Ultra-Low-Cost Drug Discovery using Hyperdimensional Computing

abstract: Modern drug discovery is often time-consuming, complex and cost-ineffective due to the large volume of molecular data and complicated molecular properties. Recently, machine learning algorithms have shown promising results in virtual screening of automated drug discovery by predicting molecular properties. While emerging learning methods such as graph neural networks and recurrent neural networks exhibit high accuracy, they are also notoriously computation-intensive and memory-intensive with operations such as feature embeddings or deep convolutions. In this paper, we propose a viable alternative to existing learning methods by presenting MoleHD, a method based on brain-inspired hyperdimensional computing (HDC) for molecular property prediction. We develop HDC encoders to project SMILES representation of a molecule into high-dimensional vectors that are used for HDC training and inference. We perform an extensive evaluation using 29 classification tasks from 3 widely-used molecule datasets (Clintox, BBBP, SIDER) under three splits methods (random, scaffold, and stratified). By an comprehensive comparison with 8 existing learning models including SOTA graph/recurrent neural networks, we show that MoleHD is able to achieve highest ROC-AUC score on random and scaffold splits on average across 3 datasets and achieve second-highest on stratified split. Importantly, MoleHD achieves such performance with significantly reduced computing cost and training efforts. To the best of our knowledge, this is the first HDC-based method for drug discovery. The promising results presented in this paper can potentially lead to a novel path in drug discovery research.


title: On length of the period of the continued fraction of $n\sqrt{d}$

abstract: For a given quadratic irrational $\alpha$, let us denote by $D(\alpha)$ the length of the periodic part of the continued fraction expansion of $\alpha$. We prove that for a positive integer $d$, which is not a perfect square, the sequence $(D(n\sqrt{d}))_{n=1}^{\infty}$ has infinitely many limit points.


title: Human Listening and Live Captioning: Multi-Task Training for Speech Enhancement

abstract: With the surge of online meetings, it has become more critical than ever to provide high-quality speech audio and live captioning under various noise conditions. However, most monaural speech enhancement (SE) models introduce processing artifacts and thus degrade the performance of downstream tasks, including automatic speech recognition (ASR). This paper proposes a multi-task training framework to make the SE models unharmful to ASR. Because most ASR training samples do not have corresponding clean signal references, we alternately perform two model update steps called SE-step and ASR-step. The SE-step uses clean and noisy signal pairs and a signal-based loss function. The ASR-step applies a pre-trained ASR model to training signals enhanced with the SE model. A cross-entropy loss between the ASR output and reference transcriptions is calculated to update the SE model parameters. Experimental results with realistic large-scale settings using ASR models trained on 75,000-hour data show that the proposed framework improves the word error rate for the SE output by 11.82% with little compromise in the SE quality. Performance analysis is also carried out by changing the ASR model, the data used for the ASR-step, and the schedule of the two update steps.


title: The basic distributional theory for the product of zero mean correlated normal random variables

abstract: The product of two zero mean correlated normal random variables, and more generally the sum of independent copies of such random variables, has received much attention in the statistics literature and appears in many application areas. However, many important distributional properties are yet to be recorded. This review paper fills this gap by providing the basic distributional theory for the sum of independent copies of the product of two zero mean correlated normal random variables. Properties covered include probability and cumulative distribution functions, generating functions, moments and cumulants, mode and median, Stein characterisations, representations in terms of other random variables, and a list of related distributions. We also review how the product of two zero mean correlated normal random variables arises naturally as a limiting distribution, with an example given for the distributional approximation of double Wiener-It\^{o} integrals.


title: Dynamic Resolution Network

abstract: Deep convolutional neural networks (CNNs) are often of sophisticated design with numerous learnable parameters for the accuracy reason. To alleviate the expensive costs of deploying them on mobile devices, recent works have made huge efforts for excavating redundancy in pre-defined architectures. Nevertheless, the redundancy on the input resolution of modern CNNs has not been fully investigated, i.e., the resolution of input image is fixed. In this paper, we observe that the smallest resolution for accurately predicting the given image is different using the same neural network. To this end, we propose a novel dynamic-resolution network (DRNet) in which the input resolution is determined dynamically based on each input sample. Wherein, a resolution predictor with negligible computational costs is explored and optimized jointly with the desired network. Specifically, the predictor learns the smallest resolution that can retain and even exceed the original recognition accuracy for each image. During the inference, each input image will be resized to its predicted resolution for minimizing the overall computation burden. We then conduct extensive experiments on several benchmark networks and datasets. The results show that our DRNet can be embedded in any off-the-shelf network architecture to obtain a considerable reduction in computational complexity. For instance, DR-ResNet-50 achieves similar performance with an about 34% computation reduction, while gaining 1.4% accuracy increase with 10% computation reduction compared to the original ResNet-50 on ImageNet.


title: $L^\infty$-estimates in optimal transport for non quadratic costs

abstract: For cost functions $c(x,y)=h(x-y)$ with $h\in C^2$ homogeneous of degree $p\geq 2$, we show $L^\infty$-estimates of $Tx-x$ on balls, where $T$ is an $h$-monotone map. Estimates for the interpolating mappings $T_t=t(T-I)+I$ are deduced from this.


title: Differentially Private Multi-Armed Bandits in the Shuffle Model

abstract: We give an $(\varepsilon,\delta)$-differentially private algorithm for the multi-armed bandit (MAB) problem in the shuffle model with a distribution-dependent regret of $O\left(\left(\sum_{a\in [k]:\Delta_a>0}\frac{\log T}{\Delta_a}\right)+\frac{k\sqrt{\log\frac{1}{\delta}}\log T}{\varepsilon}\right)$, and a distribution-independent regret of $O\left(\sqrt{kT\log T}+\frac{k\sqrt{\log\frac{1}{\delta}}\log T}{\varepsilon}\right)$, where $T$ is the number of rounds, $\Delta_a$ is the suboptimality gap of the arm $a$, and $k$ is the total number of arms. Our upper bound almost matches the regret of the best known algorithms for the centralized model, and significantly outperforms the best known algorithm in the local model.


title: Hierarchical Temperature Imaging Using Pseudo-Inversed Convolutional Neural Network Aided TDLAS Tomography

abstract: As an in situ combustion diagnostic tool, Tunable Diode Laser Absorption Spectroscopy (TDLAS) tomography has been widely used for imaging of two-dimensional temperature distributions in reactive flows. Compared with the computational tomographic algorithms, Convolutional Neural Networks (CNNs) have been proofed to be more robust and accurate for image reconstruction, particularly in case of limited access of laser beams in the Region of Interest (RoI). In practice, flame in the RoI that requires to be reconstructed with good spatial resolution is commonly surrounded by low-temperature background. Although the background is not of high interest, spectroscopic absorption still exists due to heat dissipation and gas convection. Therefore, we propose a Pseudo-Inversed CNN (PI-CNN) for hierarchical temperature imaging that (a) uses efficiently the training and learning resources for temperature imaging in the RoI with good spatial resolution, and (b) reconstructs the less spatially resolved background temperature by adequately addressing the integrity of the spectroscopic absorption model. In comparison with the traditional CNN, the newly introduced pseudo inversion of the RoI sensitivity matrix is more penetrating for revealing the inherent correlation between the projection data and the RoI to be reconstructed, thus prioritising the temperature imaging in the RoI with high accuracy and high computational efficiency. In this paper, the proposed algorithm was validated by both numerical simulation and lab-scale experiment, indicating good agreement between the phantoms and the high-fidelity reconstructions.


title: BERTnesia: Investigating the capture and forgetting of knowledge in BERT

abstract: Probing complex language models has recently revealed several insights into linguistic and semantic patterns found in the learned representations. In this article, we probe BERT specifically to understand and measure the relational knowledge it captures in its parametric memory. While probing for linguistic understanding is commonly applied to all layers of BERT as well as fine-tuned models, this has not been done for factual knowledge. We utilize existing knowledge base completion tasks (LAMA) to probe every layer of pre-trained as well as fine-tuned BERT models(ranking, question answering, NER). Our findings show that knowledge is not just contained in BERT's final layers. Intermediate layers contribute a significant amount (17-60%) to the total knowledge found. Probing intermediate layers also reveals how different types of knowledge emerge at varying rates. When BERT is fine-tuned, relational knowledge is forgotten. The extent of forgetting is impacted by the fine-tuning objective and the training data. We found that ranking models forget the least and retain more knowledge in their final layer compared to masked language modeling and question-answering. However, masked language modeling performed the best at acquiring new knowledge from the training data. When it comes to learning facts, we found that capacity and fact density are key factors. We hope this initial work will spur further research into understanding the parametric memory of language models and the effect of training objectives on factual knowledge. The code to repeat the experiments is publicly available on GitHub.


title: Large barocaloric effect with high pressure-driving efficiency in hexagonal MnNi0.77Fe0.23Ge alloy

abstract: The hydrostatic pressure is expected to be an effective knob to tune the magnetostructural phase transitions of hexagonal MMX alloy. In this study, magnetization measurements under hydrostatic pressure were performed on a MMX martensitic MnNi0.77Fe0.23Ge alloy. The magnetostructural transition temperature can be efficiently tuned to lower temperatures by applying moderate pressures, with a giant shift rate of -151 K GPa-1. A temperature span of 30 K is obtained under the pressure, within which a large magnetic entropy change of -23 J kg-1 K-1 in a field change of 5 T is induced by the mechanical energy gain due to the large volume change. Meanwhile, a decoupling of structural and magnetic transitions is observed at low temperatures when the martensitic transition temperature is lower than the Curie temperature. These results show a multi-parameter tunable caloric effect that benefits the solid-state cooling.


title: Magnetic-field modulation of topological electronic states and emergent magneto-transport in a magnetic Weyl semimetal

abstract: The modulation of topological electronic states by an external magnetic field is highly desired for condensed matter physics. Schemes to achieve this have been proposed theoretically, but few can be realized experimentally. Here, combining transverse transport, theoretical calculations, and scanning tunneling microscopy/spectroscopy (STM/S) investigations, we provide an observation that the topological electronic states, accompanied by an emergent magneto-transport phenomenon, were modulated by applying magnetic fields through induced non-collinear magnetism in the magnetic Weyl semimetal EuB6. A giant unconventional anomalous Hall effect (UAHE) is found during the magnetisation re-orientation from easy axis to hard ones in magnetic fields, with a UAHE peak around the low field of 5 kOe. Under the reasonable spin-canting effect, the folding of the topological anti-crossing bands occurs, generating a strong Berry curvature that accounts for the observed UAHE. Field-dependent STM/S reveals a highly synchronous evolution of electronic density of states, with a dI/dV peak around the same field of 5 kOe, which provides evidence to the folded bands and excited UAHE by external magnetic fields. This finding elucidates the connection between the real-space non-collinear magnetism and the k-space topological electronic states and establishes a novel manner to engineer the magneto-transport behaviours of correlated electrons for future topological spintronics.


title: On the number of heterochromatic trees in nice and beautiful colourings of complete graphs

abstract: We introduce classes of edge-colourings of the complete graph -- that we call nice and beautiful -- and study how many heterochromatic spanning trees appear under such colourings. We prove that if the colouring is nice, there is at least a quadratic number of different heterochromatic trees; and if the colouring is beautiful there is an exponential number of different such trees.


title: On the anomalous low-resistance state and exceptional Hall component in hard-magnetic Weyl nanoflakes

abstract: Magnetic topological materials, which combine magnetism and topology, are expected to host emerging topological states and exotic quantum phenomena. In this study, with the aid of greatly enhanced coercive fields in high-quality nanoflakes of the magnetic Weyl semimetal Co3Sn2S2, we investigate anomalous electronic transport properties that are difficult to reveal in bulk Co3Sn2S2 or other magnetic materials. When the magnetization is antiparallel to the applied magnetic field, the low longitudinal resistance state occurs, which is in sharp contrast to the high resistance state for the parallel case. Meanwhile, an exceptional Hall component that can be up to three times larger than conventional anomalous Hall resistivity is also observed for transverse transport. These anomalous transport behaviors can be further understood by considering nonlinear magnetic textures and the chiral magnetic field associated with Weyl fermions, extending the longitudinal and transverse transport physics and providing novel degrees of freedom in the spintronic applications of emerging topological magnets.


title: The use of Boosted Decision Trees for Energy Reconstruction in JUNO experiment

abstract: The Jiangmen Underground Neutrino Observatory (JUNO) is a neutrino experiment with a broad physical program. The main goals of JUNO are the determination of the neutrino mass ordering and high precision investigation of neutrino oscillation properties. The precise reconstruction of the event energy is crucial for the success of the experiment. JUNO is equiped with 17612 + 25600 PMT channels of two kind which provide both charge and hit time information. In this work we present a fast Boosted Decision Trees model using small set of aggregated features. The model predicts event energy deposition. We describe the motivation and the details of our feature engineering and feature selection procedures. We demonstrate that the proposed aggregated approach can achieve a reconstruction quality that is competitive with the quality of much more complex models like Convolution Neural Networks (ResNet, VGG and GNN).


title: Single-file dynamics of colloids in circular channels: time scales, scaling laws and their universality

abstract: In colloidal systems, Brownian motion emerges from the massive separation of time and length scales associated to characteristic dynamics of the solute and solvent constituents. This separation of scales produces several temporal regimes in the colloidal dynamics when combined with the effects of the interaction between the particles, confinement conditions, and state variables, such as density and temperature. Some examples are the short- and long-time regimes in two- and three-dimensional open systems and the diffusive and sub-diffusive regimes observed in the single-file dynamics along a straight line. This work studies the way in which a confining geometry induces new time scales. We report on the dynamics of interacting colloidal particles moving along a circle by combining a heuristic theoretical analysis of the involved scales, Brownian Dynamics computer simulations, and video-microscopy experiments with paramagnetic colloids confined to lithographic circular channels subjected to an external magnetic field. The systems display four temporal regimes in this order: one-dimensional free diffusion, single-file sub-diffusion, free-cluster rotational diffusion, and the expected saturation due to the confinement. We also report analytical expressions for the mean-square angular displacement and crossover times obtained from scaling arguments, which accurately reproduce both experiments and simulations. Our generic approach can be used to predict the long-time dynamics of many other confined physical systems.


title: Parameter Estimation for Grouped Data Using EM and MCEM Algorithms

abstract: Nowadays, the confidentiality of data and information is of great importance for many companies and organizations. For this reason, they may prefer not to release exact data, but instead to grant researchers access to approximate data. For example, rather than providing the exact measurements of their clients, they may only provide researchers with grouped data, that is, the number of clients falling in each of a set of non-overlapping measurement intervals. The challenge is to estimate the mean and variance structure of the hidden ungrouped data based on the observed grouped data. To tackle this problem, this work considers the exact observed data likelihood and applies the Expectation-Maximization (EM) and Monte-Carlo EM (MCEM) algorithms for cases where the hidden data follow a univariate, bivariate, or multivariate normal distribution. Simulation studies are conducted to evaluate the performance of the proposed EM and MCEM algorithms. The well-known Galton data set is considered as an application example.


title: The electron acoustic waves in plasmas with two kappa-distributed electrons at the same temperatures and immobile ions

abstract: The linear electron acoustic waves propagating in plasmas with two kappa-distributed electrons and stationary ions are investigated. The temperatures of the two electrons are assumed to be the same, but the kappa indices are not. It shows that if one kappa index is small enough and the other one is large enough, a weak damping regime of the electron acoustic waves exists. The dispersions and damping rates are numerically studied. The parameter spaces for the weakly damped electron acoustic waves are analyzed. Moreover, the electron acoustic waves in the present model are compared with those in other models, especially the plasmas with two-temperature electrons. At last, we perform Vlasov-Poisson simulations to verify the theory.


title: Global dynamics of a parabolic type equation arising from the curvature flow

abstract: This paper studies a type of degenerate parabolic problem with nonlocal term \begin{equation*} \begin{cases} u_t=u^p(u_{xx}+u-\bar{u}) & 0<t<T_{{\max}},\ 0<x<a, u_x(0,t)=u_x(a,t)=0 & 0<t<T_{{\max}}, u(x,0)=u_0(x) & 0<x<a, \end{cases} \end{equation*} where $p>1$, $a>0$. In this paper, the classification of the finite-time blowup/global existence phenomena based on the associated energy functional and explicit expression of all nonnegative steady states are demonstrated. Moreover, we combine the applications of Lojasiewicz-Simon inequality and energy estimates to derive that any bounded solution with positive initial data converges to some steady state as $t\rightarrow +\infty$.


title: Explosive synchronization and chimera in interpinned multilayer networks

abstract: This Letter investigates the nature of synchronization in multilayered and multiplexed populations in which the interlayer interactions are randomly pinned. First, we show that a multilayer network constructed by setting up all-to-all interlayer connections between the two populations leads to explosive synchronization in the two populations successively, leading to the coexistence of coherent and incoherent populations forming chimera states. Second, a multiplex formation of the two populations in which only the mirror nodes are interconnected espouses explosive transitions in the two populations concurrently. The occurrence of both explosive synchronization and chimera are substantiated with rigorous theoretical mean-field analysis. The random pinning in the interlayer interactions concerns the practical problems where the impact of dynamics of one network on that of other interconnected networks remains elusive, as is the case for many real-world systems.


title: Projective collineations of decomposable spacetimes generated by the Lie point symmetries of geodesic equations

abstract: We investigate the relation of the Lie point symmetries for the geodesic equations with the collineations of decomposable spacetimes. We review previous results in the literature on the Lie point symmetries of the geodesic equations and we follow a previous proposed geometric construction approach for the symmetries of differential equations. In this study we prove that the projective collineations of a $\left( n+1\right) \,$% -dimensional decomposable Riemannian space are the Lie point symmetries for geodesic equations of the $n$-dimensional subspace. We demonstrate the application of our results with the presentation of applications.


title: Feature Flow Regularization: Improving Structured Sparsity in Deep Neural Networks

abstract: Pruning is a model compression method that removes redundant parameters in deep neural networks (DNNs) while maintaining accuracy. Most available filter pruning methods require complex treatments such as iterative pruning, features statistics/ranking, or additional optimization designs in the training process. In this paper, we propose a simple and effective regularization strategy from a new perspective of evolution of features, which we call feature flow regularization (FFR), for improving structured sparsity and filter pruning in DNNs. Specifically, FFR imposes controls on the gradient and curvature of feature flow along the neural network, which implicitly increases the sparsity of the parameters. The principle behind FFR is that coherent and smooth evolution of features will lead to an efficient network that avoids redundant parameters. The high structured sparsity obtained from FFR enables us to prune filters effectively. Experiments with VGGNets, ResNets on CIFAR-10/100, and Tiny ImageNet datasets demonstrate that FFR can significantly improve both unstructured and structured sparsity. Our pruning results in terms of reduction of parameters and FLOPs are comparable to or even better than those of state-of-the-art pruning methods.


title: Solution Method For Higher Order System

abstract: Consider a higher order state space system and the aim of this paper is to linearize the system preserving system characteristics. That is, linearization preserving system characteristics(e.g, controllability, observability, various zeros and transfer function) for analysis of higher order systems gives the solution for higher order system. We study recovery of zero directions of higher order state space system from those of the linearizations. That is, the zero directions of the transfer functions associated to higher order state space system are recovered from the eigenvectors of the Fiedler pencils without performing any arithmetic operations


title: 3D Tensor-based Deep Learning Models for Predicting Option Price

abstract: Option pricing is a significant problem for option risk management and trading. In this article, we utilize a framework to present financial data from different sources. The data is processed and represented in a form of 2D tensors in three channels. Furthermore, we propose two deep learning models that can deal with 3D tensor data. Experiments performed on the Chinese market option dataset prove the practicability of the proposed strategies over commonly used ways, including B-S model and vector-based LSTM.


title: On the Stratification of Product Portfolios

abstract: Stratifying commercial product portfolios into multiple classes of decreasing priority, ABCD analysis, is a common supply chain tool. Key planning parameters that drive strategic and execution priorities are tied to the resulting segmentation. These priorities in turn drive supply chain performance. For large product assortments, manual segmentation is infeasible so an automated algorithm is needed. We therefore advocate that careful attention be paid to the design of such an ABCD algorithm and present three key features that can be incorporated into such a calculation to improve its quality and commercial utility.


title: Radio afterglow of magnetars' giant flares

abstract: We develop a model for the radio afterglow of the giant flare of SGR 1806-20 arising due to the interaction of magnetically-dominated cloud, an analogue of Solar Coronal Mass Ejections (CMEs), with the interstellar medium (ISM). The CME is modeled as a spheromak-like configuration. The CME is first advected with the magnetar's wind and later interacts with the ISM, creating a strong forward shock and complicated backwards exhaust flow. Using three-dimensional magnetohydrodynamic simulations, we study various relative configurations of the magnetic field of the CME with respect to the ISM's magnetic field. We show that the dynamics of the forward shock mostly follows the Sedov-Taylor blastwave, while the internal structure of the shocked medium is considerably modified by the back flow, creating a multiple shock configuration. We calculate synthetic synchrotron emissivity maps and light curves using two assumptions: (i) magnetic field compression; (ii) amplification of the magnetic field at the shock.We find that models with magnetic field amplification account better for the observed radio emission.


title: Solving the dimer problem of the vertex-edge graph of a cubic graph

abstract: Let $G$ be a graph with vertex set $V(G)$ and edge set $E(G)$, and $L(G)$ be the line graph of $G$, which has vertex set $E(G)$ and two vertices $e$ and $f$ of $L(G)$ is adjacent if $e$ and $f$ is incident in $G$. The vertex-edge graph $M(G)$ of $G$ has vertex set $V(G)\cup E(G)$ and edge set $E(L(G))\cup \{ue,ve|\ \forall\ e=uv\in E(G)\}$. In this paper, by a combinatorial technique, we show that if $G$ is a connected cubic graph with an even number of edges, then the number of dimer coverings of $M(G)$ equals $2^{|V(G)|/2+1}3^{|V(G)|/4}$. As an application, we obtain the exact solution of the dimer problem of the weighted solicate network obtained from the hexagonal lattice in the context of statistical physics.


title: A novel way of constraining the $\alpha$-attractor chaotic inflation through Planck data

abstract: Defining a scale of $k$-modes of the quantum fluctuations during inflation through the dynamical horizon crossing condition $k = aH$ we go from the physical $t$ variable to $k$ variable and solve the equations of cosmological first-order perturbations self consistently, with the chaotic $\alpha$-attractor type potentials. This enables us to study the behaviour of $n_{s}$, $r$, $n_{t}$ and $N$ in the $k$-space. Comparison of our results in the low-$k$ regime with the Planck data puts constraints on the values of the $\alpha$ parameter through microscopic calculations. Recent studies had already put model-dependent constraints on the values of $\alpha$ through the hyperbolic geometry of a Poincar\'{e} disk: consistent with both the maximal supergravity model $\mathcal{N}=8$ and the minimal supergravity model $\mathcal{N}=1$, the constraints on the values of $\alpha$ are $\frac{1}{3}$, $\frac{2}{3}$, 1, $\frac{4}{3}$, $\frac{5}{3}$, 2, $\frac{7}{3}$. The minimal $\mathcal{N}=1$ supersymmetric cosmological models with $B$-mode targets, derived from these supergravity models, predicted the values of $r$ between $10^{-2}$ and $10^{-3}$. Both in the $E$-model and the $T$-model potentials, we have obtained, in our calculations, the values of $r$ in this range for all the constrained values of $\alpha$ stated above, within $68\%$ CL. Moreover, we have calculated $r$ for some other possible values of $\alpha$ both in low-$\alpha$ limit, using the formula $r=\frac{12\alpha}{N^{2}}$, and in the high-$\alpha$ limit, using the formula $r=\frac{4n}{N}$, for $n=2$ and $4$. With all such values of $\alpha$, our calculated results match with the Planck-2018 data with $68\%$ or near $95\%$ CL.


title: Empirically Evaluating Creative Arc Negotiation for Improvisational Decision-making

abstract: Action selection from many options with few constraints is crucial for improvisation and co-creativity. Our previous work proposed creative arc negotiation to solve this problem, i.e., selecting actions to follow an author-defined `creative arc' or trajectory over estimates of novelty, unexpectedness, and quality for potential actions. The CARNIVAL agent architecture demonstrated this approach for playing the Props game from improv theatre in the Robot Improv Circus installation. This article evaluates the creative arc negotiation experience with CARNIVAL through two crowdsourced observer studies and one improviser laboratory study. The studies focus on subjects' ability to identify creative arcs in performance and their preference for creative arc negotiation compared to a random selection baseline. Our results show empirically that observers successfully identified creative arcs in performances. Both groups also preferred creative arc negotiation in agent creativity and logical coherence, while observers enjoyed it more too.


title: A finite element method for two-phase flow with material viscous interface

abstract: This paper studies a model of two-phase flow with an immersed material viscous interface and a finite element method for numerical solution of the resulting system of PDEs. The interaction between the bulk and surface media is characterized by no-penetration and slip with friction interface conditions. The system is shown to be dissipative and a model stationary problem is proved to be well-posed. The finite element method applied in this paper belongs to a family of unfitted discretizations. The performance of the method when model and discretization parameters vary is assessed. Moreover, an iterative procedure based on the splitting of the system into bulk and surface problems is introduced and studied numerically.


title: Local Disentanglement in Variational Auto-Encoders Using Jacobian $L_1$ Regularization

abstract: There have been many recent advances in representation learning; however, unsupervised representation learning can still struggle with model identification issues related to rotations of the latent space. Variational Auto-Encoders (VAEs) and their extensions such as $\beta$-VAEs have been shown to improve local alignment of latent variables with PCA directions, which can help to improve model disentanglement under some conditions. Borrowing inspiration from Independent Component Analysis (ICA) and sparse coding, we propose applying an $L_1$ loss to the VAE's generative Jacobian during training to encourage local latent variable alignment with independent factors of variation in images of multiple objects or images with multiple parts. We demonstrate our results on a variety of datasets, giving qualitative and quantitative results using information theoretic and modularity measures that show our added $L_1$ cost encourages local axis alignment of the latent representation with individual factors of variation.


title: A Cauchy-Davenport theorem for locally compact groups

abstract: We generalize the Cauchy-Davenport theorem to locally compact groups.


title: Tensor Normal Training for Deep Learning Models

abstract: Despite the predominant use of first-order methods for training deep learning models, second-order methods, and in particular, natural gradient methods, remain of interest because of their potential for accelerating training through the use of curvature information. Several methods with non-diagonal preconditioning matrices, including KFAC, Shampoo, and K-BFGS, have been proposed and shown to be effective. Based on the so-called tensor normal (TN) distribution, we propose and analyze a brand new approximate natural gradient method, Tensor Normal Training (TNT), which like Shampoo, only requires knowledge of the shape of the training parameters. By approximating the probabilistically based Fisher matrix, as opposed to the empirical Fisher matrix, our method uses the block-wise covariance of the sampling based gradient as the pre-conditioning matrix. Moreover, the assumption that the sampling-based (tensor) gradient follows a TN distribution, ensures that its covariance has a Kronecker separable structure, which leads to a tractable approximation to the Fisher matrix. Consequently, TNT's memory requirements and per-iteration computational costs are only slightly higher than those for first-order methods. In our experiments, TNT exhibited superior optimization performance to state-of-the-art first-order methods, and comparable optimization performance to the state-of-the-art second-order methods KFAC and Shampoo. Moreover, TNT demonstrated its ability to generalize as well as first-order methods, while using fewer epochs.


title: IM-META: Influence Maximization Using Node Metadata in Networks With Unknown Topology

abstract: Since the structure of complex networks is often unknown, we may identify the most influential seed nodes by exploring only a part of the underlying network, given a small budget for node queries. We propose IM-META, a solution to influence maximization (IM) in networks with unknown topology by retrieving information from queries and node metadata. Since using such metadata is not without risk due to the noisy nature of metadata and uncertainties in connectivity inference, we formulate a new IM problem that aims to find both seed nodes and queried nodes. In IM-META, we develop an effective method that iteratively performs three steps: 1) we learn the relationship between collected metadata and edges via a Siamese neural network, 2) we select a number of inferred confident edges to construct a reinforced graph, and 3) we identify the next node to query by maximizing the inferred influence spread using our topology-aware ranking strategy. Through experimental evaluation of IM-META on four real-world datasets, we demonstrate a) the speed of network exploration via node queries, b) the effectiveness of each module, c) the superiority over benchmark methods, d) the robustness to more difficult settings, e) the hyperparameter sensitivity, and f) the scalability.


title: A Generalized Framework for Joint Dynamic Optimal RF Interface Setting and Next-Hop Selection in IoT networks with Similar Requests

abstract: Many applications which run on the Internet-of-Things machines require similar bandwidths. These machines are equipped with multiple Radio Frequency (RF) interfaces for machine-to-machine or machine-to-Base Station(BS) communications. Joint dynamic optimal RF interface setting and next-hop selection in data transmission can maximize the network data rate. For this purpose, we propose a generalized framework for simultaneous selection of these components according to the required bandwidth of machines. This framework is suitable for networks with multiple BSs and sources with similar bandwidth requests. Simulation results show that using the proposed framework, the average data rate of network sources increases up to 117%.