title: Securing Large-Scale D2D Networks Using Covert Communication and Friendly Jamming

abstract: We exploit both covert communication and friendly jamming to propose a friendly jamming-assisted covert communication and use it to doubly secure a large-scale device-to-device (D2D) network against eavesdroppers (i.e., wardens). The D2D transmitters defend against the wardens by: 1) hiding their transmissions with enhanced covert communication, and 2) leveraging friendly jamming to ensure information secrecy even if the D2D transmissions are detected. We model the combat between the wardens and the D2D network (the transmitters and the friendly jammers) as a two-stage Stackelberg game. Therein, the wardens are the followers at the lower stage aiming to minimize their detection errors, and the D2D network is the leader at the upper stage aiming to maximize its utility (in terms of link reliability and communication security) subject to the constraint on communication covertness. We apply stochastic geometry to model the network spatial configuration so as to conduct a system-level study. We develop a bi-level optimization algorithm to search for the equilibrium of the proposed Stackelberg game based on the successive convex approximation (SCA) method and Rosenbrock method. Numerical results reveal interesting insights. We observe that without the assistance from the jammers, it is difficult to achieve covert communication on D2D transmission. Moreover, we illustrate the advantages of the proposed friendly jamming-assisted covert communication by comparing it with the information-theoretical secrecy approach in terms of the secure communication probability and network utility.


title: State-specific protein-ligand complex structure prediction with a multi-scale deep generative model

abstract: The binding complexes formed by proteins and small molecule ligands are ubiquitous and critical to life. Despite recent advancements in protein structure prediction, existing algorithms are so far unable to systematically predict the binding ligand structures along with their regulatory effects on protein folding. To address this discrepancy, we present NeuralPLexer, a computational approach that can directly predict protein-ligand complex structures solely using protein sequence and ligand molecular graph inputs. NeuralPLexer adopts a deep generative model to sample the 3D structures of the binding complex and their conformational changes at an atomistic resolution. The model is based on a diffusion process that incorporates essential biophysical constraints and a multi-scale geometric deep learning system to iteratively sample residue-level contact maps and all heavy-atom coordinates in a hierarchical manner. NeuralPLexer achieves state-of-the-art performance compared to all existing methods on benchmarks for both protein-ligand blind docking and flexible binding site structure recovery. Moreover, owing to its specificity in sampling both ligand-free-state and ligand-bound-state ensembles, NeuralPLexer consistently outperforms AlphaFold2 in terms of global protein structure accuracy on both representative structure pairs with large conformational changes (average TM-score=0.93) and recently determined ligand-binding proteins (average TM-score=0.89). Case studies reveal that the predicted conformational variations are consistent with structure determination experiments for important targets, including human KRAS$^\textrm{G12C}$, ketol-acid reductoisomerase, and purine GPCRs. Our study suggests that a data-driven approach can capture the structural cooperativity between proteins and small molecules, showing promise in accelerating the design of enzymes, drug molecules, and beyond.


title: Understanding Pure CLIP Guidance for Voxel Grid NeRF Models

abstract: We explore the task of text to 3D object generation using CLIP. Specifically, we use CLIP for guidance without access to any datasets, a setting we refer to as pure CLIP guidance. While prior work has adopted this setting, there is no systematic study of mechanics for preventing adversarial generations within CLIP. We illustrate how different image-based augmentations prevent the adversarial generation problem, and how the generated results are impacted. We test different CLIP model architectures and show that ensembling different models for guidance can prevent adversarial generations within bigger models and generate sharper results. Furthermore, we implement an implicit voxel grid model to show how neural networks provide an additional layer of regularization, resulting in better geometrical structure and coherency of generated objects. Compared to prior work, we achieve more coherent results with higher memory efficiency and faster training speeds.


title: Construction of the Radio Map with Defective GPS Position Information

abstract: The basic idea of RSS-based indoor positioning is to estimate the receiver location by matching the measured received signal strength indicator (RSSI) with preestablished RSSI collections with corresponding locations, known as the radio map. Hence, constructing an accurate radio map directly relates to accurate positioning performance in RSS-based indoor positioning. RSS-based indoor positioning can be easily conducted with a radio map that surveys every location, but a complete radio map cannot be constructed when the map area includes locations that are physically impossible to reach or denied access. In addition, measurement errors or device problems can occur during the survey, resulting in degradation of the radio map. We analyzed incidents that occurred in actual RSSI surveys that could disrupt the construction of the radio map and proposed methods to construct a more accurate radio map.


title: Music Source Separation with Band-split RNN

abstract: The performance of music source separation (MSS) models has been greatly improved in recent years thanks to the development of novel neural network architectures and training pipelines. However, recent model designs for MSS were mainly motivated by other audio processing tasks or other research fields, while the intrinsic characteristics and patterns of the music signals were not fully discovered. In this paper, we propose band-split RNN (BSRNN), a frequency-domain model that explictly splits the spectrogram of the mixture into subbands and perform interleaved band-level and sequence-level modeling. The choices of the bandwidths of the subbands can be determined by a priori knowledge or expert knowledge on the characteristics of the target source in order to optimize the performance on a certain type of target musical instrument. To better make use of unlabeled data, we also describe a semi-supervised model finetuning pipeline that can further improve the performance of the model. Experiment results show that BSRNN trained only on MUSDB18-HQ dataset significantly outperforms several top-ranking models in Music Demixing (MDX) Challenge 2021, and the semi-supervised finetuning stage further improves the performance on all four instrument tracks.


title: The First Photometric and Orbital Period Investigation of an Extremely Low Mass Ratio Contact Binary with a Sudden Period Change, TYC 4002-2628-1

abstract: Photometric observations for the totally eclipsing binary system TYC 4002-2628-1, were obtained between November 2020 and November 2021. To determine the stellar atmospheric parameters, a spectral image was taken with the 2.16 m telescope at National Astronomical Observatory of China (NAOC). TYC 4002-2628-1 is a low-amplitude (about 0.15 mag for $V$ band), short-period (0.3670495 d), contact eclipsing binary with a total secondary eclipse. Intrinsic light curve variations and the reversal of the O'Connell effect are detected in the light curves, which may be due to spot activity. Based on the photometric solutions derived from the multi-band time series light curves, TYC 4002-2628-1 is an extremely low mass ratio contact binary with a mass ratio of $q\sim$ 0.0482 and a fill-out factor of $f\sim5\%$. By analyzing the $O-C$ variations, we find that its orbital period remains unchanged when BJD < 2458321 . Then the orbital period changed suddenly around BJD 2458743 and has an increasing rate of $dP/dt=1.62\times{10^{-5}}day\cdot yr^{-1}=140$ $second\cdot century^{-1}$ . If confirmed, TYC 4002-2628-1 would be the contact binary with the highest orbital period increasing rate so far. By investigating the ratio of orbital angular momentum to the spin angular momentum ( $J_{orb}$/$J_{spin}$ $<3$) , the instability mass ratio ($q_{inst}/q=1.84$) and the instability separation ($A_{inst}/A=1.35$), TYC 4002-2628-1 can be regarded as a merger candidate.


title: Adaptive Sparse and Monotonic Attention for Transformer-based Automatic Speech Recognition

abstract: The Transformer architecture model, based on self-attention and multi-head attention, has achieved remarkable success in offline end-to-end Automatic Speech Recognition (ASR). However, self-attention and multi-head attention cannot be easily applied for streaming or online ASR. For self-attention in Transformer ASR, the softmax normalization function-based attention mechanism makes it impossible to highlight important speech information. For multi-head attention in Transformer ASR, it is not easy to model monotonic alignments in different heads. To overcome these two limits, we integrate sparse attention and monotonic attention into Transformer-based ASR. The sparse mechanism introduces a learned sparsity scheme to enable each self-attention structure to fit the corresponding head better. The monotonic attention deploys regularization to prune redundant heads for the multi-head attention structure. The experiments show that our method can effectively improve the attention mechanism on widely used benchmarks of speech recognition.


title: Domain Generalization -- A Causal Perspective

abstract: Machine learning models rely on various assumptions to attain high accuracy. One of the preliminary assumptions of these models is the independent and identical distribution, which suggests that the train and test data are sampled from the same distribution. However, this assumption seldom holds in the real world due to distribution shifts. As a result models that rely on this assumption exhibit poor generalization capabilities. Over the recent years, dedicated efforts have been made to improve the generalization capabilities of these models collectively known as -- \textit{domain generalization methods}. The primary idea behind these methods is to identify stable features or mechanisms that remain invariant across the different distributions. Many generalization approaches employ causal theories to describe invariance since causality and invariance are inextricably intertwined. However, current surveys deal with the causality-aware domain generalization methods on a very high-level. Furthermore, we argue that it is possible to categorize the methods based on how causality is leveraged in that method and in which part of the model pipeline is it used. To this end, we categorize the causal domain generalization methods into three categories, namely, (i) Invariance via Causal Data Augmentation methods which are applied during the data pre-processing stage, (ii) Invariance via Causal representation learning methods that are utilized during the representation learning stage, and (iii) Invariance via Transferring Causal mechanisms methods that are applied during the classification stage of the pipeline. Furthermore, this survey includes in-depth insights into benchmark datasets and code repositories for domain generalization methods. We conclude the survey with insights and discussions on future directions.


title: On the Quotient-Lift Matroid Relation

abstract: It is well known that a matroid L is a lift of a matroid M if and only if every circuit of L is the union of some circuits of M. In this paper we give a simpler proof of this important theorem. We also described a discrete homotopy theorem on two matroids of different ranks on the same ground set.


title: Physical Adversarial Attack meets Computer Vision: A Decade Survey

abstract: Despite the impressive achievements of Deep Neural Networks (DNNs) in computer vision, their vulnerability to adversarial attacks remains a critical concern. Extensive research has demonstrated that incorporating sophisticated perturbations into input images can lead to a catastrophic degradation in DNNs' performance. This perplexing phenomenon not only exists in the digital space but also in the physical world. Consequently, it becomes imperative to evaluate the security of DNNs-based systems to ensure their safe deployment in real-world scenarios, particularly in security-sensitive applications. To facilitate a profound understanding of this topic, this paper presents a comprehensive overview of physical adversarial attacks. Firstly, we distill four general steps for launching physical adversarial attacks. Building upon this foundation, we uncover the pervasive role of artifacts carrying adversarial perturbations in the physical world. These artifacts influence each step. To denote them, we introduce a new term: adversarial medium. Then, we take the first step to systematically evaluate the performance of physical adversarial attacks, taking the adversarial medium as a first attempt. Our proposed evaluation metric, hiPAA, comprises six perspectives: Effectiveness, Stealthiness, Robustness, Practicability, Aesthetics, and Economics. We also provide comparative results across task categories, together with insightful observations and suggestions for future research directions.


title: SCI: A Spectrum Concentrated Implicit Neural Compression for Biomedical Data

abstract: Massive collection and explosive growth of biomedical data, demands effective compression for efficient storage, transmission and sharing. Readily available visual data compression techniques have been studied extensively but tailored for natural images/videos, and thus show limited performance on biomedical data which are of different features and larger diversity. Emerging implicit neural representation (INR) is gaining momentum and demonstrates high promise for fitting diverse visual data in target-data-specific manner, but a general compression scheme covering diverse biomedical data is so far absent. To address this issue, we firstly derive a mathematical explanation for INR's spectrum concentration property and an analytical insight on the design of INR based compressor. Further, we propose a Spectrum Concentrated Implicit neural compression (SCI) which adaptively partitions the complex biomedical data into blocks matching INR's concentrated spectrum envelop, and design a funnel shaped neural network capable of representing each block with a small number of parameters. Based on this design, we conduct compression via optimization under given budget and allocate the available parameters with high representation accuracy. The experiments show SCI's superior performance to state-of-the-art methods including commercial compressors, data-driven ones, and INR based counterparts on diverse biomedical data. The source code can be found at https://github.com/RichealYoung/ImplicitNeuralCompression.git.


title: RL-MD: A Novel Reinforcement Learning Approach for DNA Motif Discovery

abstract: The extraction of sequence patterns from a collection of functionally linked unlabeled DNA sequences is known as DNA motif discovery, and it is a key task in computational biology. Several deep learning-based techniques have recently been introduced to address this issue. However, these algorithms can not be used in real-world situations because of the need for labeled data. Here, we presented RL-MD, a novel reinforcement learning based approach for DNA motif discovery task. RL-MD takes unlabelled data as input, employs a relative information-based method to evaluate each proposed motif, and utilizes these continuous evaluation results as the reward. The experiments show that RL-MD can identify high-quality motifs in real-world data.


title: Husformer: A Multi-Modal Transformer for Multi-Modal Human State Recognition

abstract: Human state recognition is a critical topic with pervasive and important applications in human-machine systems. Multi-modal fusion, the combination of metrics from multiple data sources, has been shown as a sound method for improving the recognition performance. However, while promising results have been reported by recent multi-modal-based models, they generally fail to leverage the sophisticated fusion strategies that would model sufficient cross-modal interactions when producing the fusion representation; instead, current methods rely on lengthy and inconsistent data preprocessing and feature crafting. To address this limitation, we propose an end-to-end multi-modal transformer framework for multi-modal human state recognition called Husformer. Specifically, we propose to use cross-modal transformers, which inspire one modality to reinforce itself through directly attending to latent relevance revealed in other modalities, to fuse different modalities while ensuring sufficient awareness of the cross-modal interactions introduced. Subsequently, we utilize a self-attention transformer to further prioritize contextual information in the fusion representation. Using two such attention mechanisms enables effective and adaptive adjustments to noise and interruptions in multi-modal signals during the fusion process and in relation to high-level features. Extensive experiments on two human emotion corpora (DEAP and WESAD) and two cognitive workload datasets (MOCAS and CogLoad) demonstrate that in the recognition of human state, our Husformer outperforms both state-of-the-art multi-modal baselines and the use of a single modality by a large margin, especially when dealing with raw multi-modal signals. We also conducted an ablation study to show the benefits of each component in Husformer.


title: Graphs with the same truncated cycle matroid

abstract: The classical Whitney's 2-Isomorphism Theorem describes the families of graphs having the same cycle matroid. In this paper we describe the families of graphs having the same truncated cycle matroid and prove, in particular, that every 3-connected graph, except for K4, is uniquely defined by its truncated cycle matroid.


title: Size-dependent transient nature of localized turbulence in transitional channel flow

abstract: It has been reported that a fully localized turbulent band in channel flow becomes sustained when the Reynolds number is above a threshold. Here we show evidences that turbulent bands are of a transient nature instead. When the band length is controlled to be fixed, lifetimes of turbulent bands appear to be stochastic and exponentially distributed, a sign of a memoryless transient nature. Besides increasing with the Reynolds number, the mean lifetime also strongly increases with the band length. Given that the band length always changes over time in real channel flow, this size dependence may translate into a time dependence, which needs to be taken into account when clarifying the relationship between channel flow transition and the directed percolation universality class.


title: Dephasing effect promotes the appearance of quantized Hall plateaus

abstract: The quantum Hall effect (QHE) is a topologically protected phenomenon which has been observed in various systems. In experiments, the size of Hall bar device to realize the QHE is generally much larger than the phase coherence length, in which the quantum coherence of electrons is destroyed. Here, we theoretically study the influence of dephasing effect on the quantized Hall (QH) plateaus. We find that the QH plateau disappears in perfectly quantum coherent systems if the coupling between leads and central region is imperfect. The Hall resistance is very large and strongly oscillates instead of presenting the QH plateau in this case. However, by introducing the dephasing, the Hall resistance decreases and the QH plateau appears gradually. Similar results can also be observed for the quantum anomalous Hall effect. Our results propose that dephasing effect promotes the appearance of QH plateaus, which opens a new topic of the dephasing effect on topological systems.


title: Leveraging Probabilistic Switching in Superparamagnets for Temporal Information Encoding in Neuromorphic Systems

abstract: Brain-inspired computing - leveraging neuroscientific principles underpinning the unparalleled efficiency of the brain in solving cognitive tasks - is emerging to be a promising pathway to solve several algorithmic and computational challenges faced by deep learning today. Nonetheless, current research in neuromorphic computing is driven by our well-developed notions of running deep learning algorithms on computing platforms that perform deterministic operations. In this article, we argue that taking a different route of performing temporal information encoding in probabilistic neuromorphic systems may help solve some of the current challenges in the field. The article considers superparamagnetic tunnel junctions as a potential pathway to enable a new generation of brain-inspired computing that combines the facets and associated advantages of two complementary insights from computational neuroscience -- how information is encoded and how computing occurs in the brain. Hardware-algorithm co-design analysis demonstrates $97.41\%$ accuracy of a state-compressed 3-layer spintronics enabled stochastic spiking network on the MNIST dataset with high spiking sparsity due to temporal information encoding.


title: Perspectives on Antiferromagnetic Spintronics

abstract: Although the development of spintronic devices has advanced significantly over the past decade with the use of ferromagnetic materials, the extensive implementation of such devices has been limited by the notable drawbacks of these materials. Antiferromagnets claim to resolve many of these shortcomings leading to faster, smaller, more energy-efficient, and more robust electronics. Antiferromagnets exhibit many desirable properties including zero net magnetization, imperviousness to external magnetic fields, intrinsic high-frequency dynamics with a characteristic precession frequency on the order of terahertz (THz), and the ability to serve as passive exchange-bias materials in multiple magnetoresistance (MR)- based devices. In this Perspective article, we will discuss the fundamental physics of magnetic structures in antiferromagnets and their interactions with external stimuli such as spin current, voltage, and magnons. A discussion on the challenges lying ahead is also provided along with an outlook of future research directions of these systems.


title: Quantum advantage through the magic pentagram problem

abstract: Through the two specific problems, the 2D hidden linear function problem and the 1D magic square problem, Bravyi et al. have recently shown that there exists a separation between $\mathbf{QNC^0}$ and $\mathbf{NC^0}$, where $\mathbf{QNC^0}$ and $\mathbf{NC^0}$ are the classes of polynomial-size and constant-depth quantum and classical circuits with bounded fan-in gates, respectively. In this paper, we present another problem with the same property, the magic pentagram problem based on the magic pentagram game, which is a nonlocal game. In other words, we show that the problem can be solved with certainty by a $\mathbf{QNC^0}$ circuit but not by any $\mathbf{NC^0}$ circuits.


title: Learning by Distilling Context

abstract: Language models significantly benefit from context tokens, such as prompts or scratchpads. They perform better when prompted with informative instructions, and they acquire new reasoning capabilities by generating a scratch-pad before predicting the final answers. However, they do not \textit{internalize} these performance gains, which disappear when the context tokens are gone. Our work proposes to apply context distillation so that a language model can improve itself by internalizing these gains. Concretely, given a synthetic unlabeled input for the target task, we condition the model on ``[instructions] + [task-input]'' to predict ``[scratch-pad] + [final answer]''; then we fine-tune the same model to predict its own ``[final answer]'' conditioned on the ``[task-input]'', without seeing the ``[instructions]'' or using the ``[scratch-pad]''. We show that context distillation is a general method to train language models, and it can effectively internalize 3 types of training signals. First, it can internalize abstract task instructions and explanations, so we can iteratively update the model parameters with new instructions and overwrite old ones. Second, it can internalize step-by-step reasoning for complex tasks (e.g., 8-digit addition), and such a newly acquired capability proves to be useful for other downstream tasks. Finally, it can internalize concrete training examples, and it outperforms directly learning with gradient descent by 9\% on the SPIDER Text-to-SQL dataset; furthermore, combining context distillation operations can internalize more training examples than the context window size allows.


title: Neural Integral Equations

abstract: Integral equations (IEs) are equations that model spatiotemporal systems with non-local interactions. They have found important applications throughout theoretical and applied sciences, including in physics, chemistry, biology, and engineering. While efficient algorithms exist for solving given IEs, no method exists that can learn an IE and its associated dynamics from data alone. In this paper, we introduce Neural Integral Equations (NIE), a method that learns an unknown integral operator from data through an IE solver. We also introduce Attentional Neural Integral Equations (ANIE), where the integral is replaced by self-attention, which improves scalability, capacity, and results in an interpretable model. We demonstrate that (A)NIE outperforms other methods in both speed and accuracy on several benchmark tasks in ODE, PDE, and IE systems of synthetic and real-world data.

categories: cs.LG cs.NA math.DS math.NA physics.comp-ph


title: Low PAPR Pilot for Delay-Doppler Domain Modulation

abstract: This paper studies the low PAPR pilot design in delay-Doppler domain modulation. We adopt a sequence based pilot design instead of the conventional pulse pilot, to mitigate the PAPR issue. We develop simple channel estimation algorithm composes of two-stages which are path identification and channel coefficient estimation. The quantitative analysis on the channel estimation error model is provided. Based on which the principle of pilot sequence design in delay-Doppler domain is revealed. Experiment results shows that the proposed scheme maintains a relatively low PAPR in time domain samples, while the channel estimation performance approaches the ideal channel estimation in limited-Doppler-Shift channel model.


title: Sound non-reciprocity based on synthetic magnetism

abstract: Synthetic magnetism has been recently realized using spatiotemporal modulation patterns, producing non-reciprocal steering of charge-neutral particles such as photons and phonons. Here, we design and experimentally demonstrate a non-reciprocal acoustic system composed of three compact cavities interlinked with both dynamic and static couplings, in which phase-correlated modulations induce a synthetic magnetic flux that breaks time-reversal symmetry. Within the rotating wave approximation, the transport properties of the system are controlled to efficiently realize large non-reciprocal acoustic transport. By optimizing the coupling strengths and modulation phases, we achieve frequency-preserved unidirectional transport with 45-dB isolation ratio and 0.85 forward transmission. Our results open to the realization of acoustic nonreciprocal technologies with high efficiency and large isolation, and offer a route towards Floquet topological insulators for sound.


title: Quantum Fourier Addition, Simplified to Toffoli Addition

abstract: Quantum addition circuits are considered being of two types: 1) Toffolli-adder circuits which use only classical reversible gates (CNOT and Toffoli), and 2) QFT-adder circuits based on the quantum Fourier transformation. We present the first systematic translation of the QFT-addition circuit into a Toffoli-based adder. This result shows that QFT-addition has fundamentally the same fault-tolerance cost (e.g. T-count) as the most cost-efficient Toffoli-adder: instead of using approximate decompositions of the gates from the QFT circuit, it is more efficient to merge gates. In order to achieve this, we formulated novel circuit identities for multi-controlled gates and apply the identities algorithmically. The employed techniques can be used to automate quantum circuit optimisation heuristics.


title: GKZ-system of the 2-loop self energy with 4 propagators

abstract: Applying the system of linear partial differential equations derived from the Mellin-Barnes representation and the Miller transformation, we present the GKZ-system of the Feynman integral of the 2-loop self energy diagram with 4 propagators. The codimension of the derived GKZ-system equals the number of independent dimensionless ratios among the external momentum squared and virtual mass squared. In total 536 hypergeometric functions are obtained in the neighborhoods of the origin and infinity, in which 30 linearly independent hypergeometric functions whose convergent regions have nonempty intersection constitute a fundamental solution system in a proper subset of the whole parameter space.


title: RF-Transformer: A Unified Backscatter Radio Hardware Abstraction

abstract: This paper presents RF-Transformer, a unified backscatter radio hardware abstraction that allows a low-power IoT device to directly communicate with heterogeneous wireless receivers at the minimum power consumption. Unlike existing backscatter systems that are tailored to a specific wireless communication protocol, RF-Transformer provides a programmable interface to the micro-controller, allowing IoT devices to synthesize different types of protocol-compliant backscatter signals sharing radically different PHY-layer designs. To show the efficacy of our design, we implement a PCB prototype of RF-Transformer on 2.4 GHz ISM band and showcase its capability on generating standard ZigBee, Bluetooth, LoRa, and Wi-Fi 802.11b/g/n/ac packets. Our extensive field studies show that RF-Transformer achieves 23.8 Mbps, 247.1 Kbps, 986.5 Kbps, and 27.3 Kbps throughput when generating standard Wi-Fi, ZigBee, Bluetooth, and LoRa signals while consuming 7.6-74.2 less power than their active counterparts. Our ASIC simulation based on the 65-nm CMOS process shows that the power gain of RF-Transformer can further grow to 92-678. We further integrate RF-Transformer with pressure sensors and present a case study on detecting foot traffic density in hallways. Our 7-day case studies demonstrate RFTransformer can reliably transmit sensor data to a commodity gateway by synthesizing LoRa packets on top of Wi-Fi signals. Our experimental results also verify the compatibility of RF-Transformer with commodity receivers. Code and hardware schematics can be found at: https://github.com/LeFsCC/RF-Transformer.


title: Continuous Gaze Tracking With Implicit Saliency-Aware Calibration on Mobile Devices

abstract: Gaze tracking is a useful human-to-computer interface, which plays an increasingly important role in a range of mobile applications. Gaze calibration is an indispensable component of gaze tracking, which transforms the eye coordinates to the screen coordinates. The existing approaches of gaze tracking either have limited accuracy or require the user's cooperation in calibration and in turn hurt the quality of experience. We in this paper propose vGaze, continuous gaze tracking with implicit saliency-aware calibration on mobile devices. The design of vGaze stems from our insight on the temporal and spatial dependent relation between the visual saliency and the user's gaze. vGaze is implemented as a light-weight software that identifies video frames with "useful" saliency information, sensing the user's head movement, performs opportunistic calibration using only those "useful" frames, and leverages historical information for accelerating saliency detection. We implement vGaze on a commercial mobile device and evaluate its performance in various scenarios. The results show that vGaze can work at real time with video playback applications. The average error of gaze tracking is 1.51 cm (2.884 degree) which decreases to 0.99 cm (1.891 degree) with historical information and 0.57 cm (1.089 degree) with an indicator.


title: Evaluation of taxonomic and neural embedding methods for calculating semantic similarity

abstract: Modelling semantic similarity plays a fundamental role in lexical semantic applications. A natural way of calculating semantic similarity is to access handcrafted semantic networks, but similarity prediction can also be anticipated in a distributional vector space. Similarity calculation continues to be a challenging task, even with the latest breakthroughs in deep neural language models. We first examined popular methodologies in measuring taxonomic similarity, including edge-counting that solely employs semantic relations in a taxonomy, as well as the complex methods that estimate concept specificity. We further extrapolated three weighting factors in modelling taxonomic similarity. To study the distinct mechanisms between taxonomic and distributional similarity measures, we ran head-to-head comparisons of each measure with human similarity judgements from the perspectives of word frequency, polysemy degree and similarity intensity. Our findings suggest that without fine-tuning the uniform distance, taxonomic similarity measures can depend on the shortest path length as a prime factor to predict semantic similarity; in contrast to distributional semantics, edge-counting is free from sense distribution bias in use and can measure word similarity both literally and metaphorically; the synergy of retrofitting neural embeddings with concept relations in similarity prediction may indicate a new trend to leverage knowledge bases on transfer learning. It appears that a large gap still exists on computing semantic similarity among different ranges of word frequency, polysemous degree and similarity intensity.


title: FoVR: Attention-based VR Streaming through Bandwidth-limited Wireless Networks

abstract: Consumer Virtual Reality (VR) has been widely used in various application areas, such as entertainment and medicine. In spite of the superb immersion experience, to enable high-quality VR on untethered mobile devices remains an extremely challenging task. The high bandwidth demands of VR streaming generally overburden a conventional wireless connection, which affects the user experience and in turn limits the usability of VR in practice. In this paper, we propose FoVR, attention-based hierarchical VR streaming through bandwidth-limited wireless networks. The design of FoVR stems from the insight that human's vision is hierarchical, so that different areas in the field of view (FoV) can be served with VR content of different qualities. By exploiting the gaze tracking capacity of the VR devices, FoVR is able to accurately predict the user's attention so that the streaming of hierarchical VR can be appropriately scheduled. In this way, FoVR significantly reduces the bandwidth cost and computing cost while keeping high quality of user experience. We implement FoVR on a commercial VR device and evaluate its performance in various scenarios. The experiment results show that FoVR reduces the bandwidth cost by 88.9% and 76.2%, respectively compared to the original VR streaming and the state-of-the-art approach.


title: Solving a puzzle in the rank 2 $\mathcal{N}=2$ classification by Argyres and Martone

abstract: Argyres and Martone have produced a beautiful and deep classification of the scale invariant Special Geometries in rank 2. They get a puzzle: the scale-invariant geometries with Coulomb dimensions $\{2,2\}$ appear to depend on four free complex parameters, while on physical grounds we expect only two marginal deformations. We show that the isoclasses of $\{2,2\}$ Special Geometries are indeed parametrized by a complex space of dimension 2, in facts by a non-singular del Pezzo surface of degree 5, a result which exactly matches the physical expectation by Gaiotto. This solves the puzzle.


title: An efficient encoder-decoder architecture with top-down attention for speech separation

abstract: Deep neural networks have shown excellent prospects in speech separation tasks. However, obtaining good results while keeping a low model complexity remains challenging in real-world applications. In this paper, we provide a bio-inspired efficient encoder-decoder architecture by mimicking the brain's top-down attention, called TDANet, with decreased model complexity without sacrificing performance. The top-down attention in TDANet is extracted by the global attention (GA) module and the cascaded local attention (LA) layers. The GA module takes multi-scale acoustic features as input to extract global attention signal, which then modulates features of different scales by direct top-down connections. The LA layers use features of adjacent layers as input to extract the local attention signal, which is used to modulate the lateral input in a top-down manner. On three benchmark datasets, TDANet consistently achieved competitive separation performance to previous state-of-the-art (SOTA) methods with higher efficiency. Specifically, TDANet's multiply-accumulate operations (MACs) are only 5\% of Sepformer, one of the previous SOTA models, and CPU inference time is only 10\% of Sepformer. In addition, a large-size version of TDANet obtained SOTA results on three datasets, with MACs still only 10\% of Sepformer and the CPU inference time only 24\% of Sepformer.


title: Hilbert expansion for kinetic equations with non-relativistic Coulomb collision

abstract: In this paper, we study the hydrodynamic limits of both the Landau equation and the Vlasov-Maxwell-Landau system in the whole space. Our main purpose is two-fold: the first one is to give a rigorous derivation of the compressible Euler equations from the Landau equation via the Hilbert expansion; while the second one is to prove, still in the setting of Hilbert expansion, that the unique classical solution of the Vlasov-Maxwell-Landau system converges, which is shown to be globally in time, to the resulting global smooth solution of the Euler-Maxwell system, as the Knudsen number goes to zero. The main ingredient of our analysis is to derive some novel interplay energy estimates on the solutions of the Landau equation and the Vlasov-Maxwell-Landau system which are small perturbations of both a local Maxwellian and a global Maxwellian, respectively. Our result solves an open problem in the hydrodynamic limit for the Landau-type equations with Coulomb potential and the approach developed in this paper can seamlessly be used to deal with the problem on the validity of the Hilbert expansion for other types of kinetic equations.


title: Synonym Detection Using Syntactic Dependency And Neural Embeddings

abstract: Recent advances on the Vector Space Model have significantly improved some NLP applications such as neural machine translation and natural language generation. Although word co-occurrences in context have been widely used in counting-/predicting-based distributional models, the role of syntactic dependencies in deriving distributional semantics has not yet been thoroughly investigated. By comparing various Vector Space Models in detecting synonyms in TOEFL, we systematically study the salience of syntactic dependencies in accounting for distributional similarity. We separate syntactic dependencies into different groups according to their various grammatical roles and then use context-counting to construct their corresponding raw and SVD-compressed matrices. Moreover, using the same training hyperparameters and corpora, we study typical neural embeddings in the evaluation. We further study the effectiveness of injecting human-compiled semantic knowledge into neural embeddings on computing distributional similarity. Our results show that the syntactically conditioned contexts can interpret lexical semantics better than the unconditioned ones, whereas retrofitting neural embeddings with semantic knowledge can significantly improve synonym detection.


title: Downlink Compression Improves TopK Sparsification

abstract: Training large neural networks is time consuming. To speed up the process, distributed training is often used. One of the largest bottlenecks in distributed training is communicating gradients across different nodes. Different gradient compression techniques have been proposed to alleviate the communication bottleneck, including topK gradient sparsification, which truncates the gradient to the largest K components before sending it to other nodes. While some authors have investigated topK gradient sparsification in the parameter-server framework by applying topK compression in both the worker-to-server (uplink) and server-to-worker (downlink) direction, the currently accepted belief says that adding extra compression degrades the convergence of the model. We demonstrate, on the contrary, that adding downlink compression can potentially improve the performance of topK sparsification: not only does it reduce the amount of communication per step, but also, counter-intuitively, can improve the upper bound in the convergence analysis. To show this, we revisit non-convex convergence analysis of topK stochastic gradient descent (SGD) and extend it from the unidirectional to the bidirectional setting. We also remove a restriction of the previous analysis that requires unrealistically large values of K. We experimentally evaluate bidirectional topK SGD against unidirectional topK SGD and show that models trained with bidirectional topK SGD will perform as well as models trained with unidirectional topK SGD while yielding significant communication benefits for large numbers of workers.


title: Quantum quenches in a pseudo-Hermitian Chern insulator

abstract: We propose to uncover the topology of a pseudo-Hermitian Chern insulator by quantum quench dynamics. The Bloch Hamiltonian of the pseudo-Hermitian Chern insulator is defined in the basis of the q-deformed Pauli matrices, which are related to the representation of the deformed algebras. We show the bulk-surface duality of the pseudo-Hermitian phases, then further build a concrete relation between the static band topology and quench dynamics, in terms of the time-averaged spin textures. The results are also generalized into a fully nonequilibrium case where the postquench evolution is governed by a Floquet pseudo-Hermitian Hamiltonian. Furthermore, we propose a possible scheme to realize the seemingly challenging model in a bilayer lattice and detect the dynamics with a double-quench protocol.


title: ASPiRe:Adaptive Skill Priors for Reinforcement Learning

abstract: We introduce ASPiRe (Adaptive Skill Prior for RL), a new approach that leverages prior experience to accelerate reinforcement learning. Unlike existing methods that learn a single skill prior from a large and diverse dataset, our framework learns a library of different distinction skill priors (i.e., behavior priors) from a collection of specialized datasets, and learns how to combine them to solve a new task. This formulation allows the algorithm to acquire a set of specialized skill priors that are more reusable for downstream tasks; however, it also brings up additional challenges of how to effectively combine these unstructured sets of skill priors to form a new prior for new tasks. Specifically, it requires the agent not only to identify which skill prior(s) to use but also how to combine them (either sequentially or concurrently) to form a new prior. To achieve this goal, ASPiRe includes Adaptive Weight Module (AWM) that learns to infer an adaptive weight assignment between different skill priors and uses them to guide policy learning for downstream tasks via weighted Kullback-Leibler divergences. Our experiments demonstrate that ASPiRe can significantly accelerate the learning of new downstream tasks in the presence of multiple priors and show improvement on competitive baselines.


title: What Makes Pre-trained Language Models Better Zero-shot Learners?

abstract: Current methods for prompt learning in zeroshot scenarios widely rely on a development set with sufficient human-annotated data to select the best-performing prompt template a posteriori. This is not ideal because in a realworld zero-shot scenario of practical relevance, no labelled data is available. Thus, we propose a simple yet effective method for screening reasonable prompt templates in zero-shot text classification: Perplexity Selection (Perplection). We hypothesize that language discrepancy can be used to measure the efficacy of prompt templates, and thereby develop a substantiated perplexity-based scheme allowing for forecasting the performance of prompt templates in advance. Experiments show that our method leads to improved prediction performance in a realistic zero-shot setting, eliminating the need for any labelled examples.


title: Mixture of experts models for multilevel data: modelling framework and approximation theory

abstract: Multilevel data are prevalent in many real-world applications. However, it remains an open research problem to identify and justify a class of models that flexibly capture a wide range of multilevel data. Motivated by the versatility of the mixture of experts (MoE) models in fitting regression data, in this article we extend upon the MoE and study a class of mixed MoE (MMoE) models for multilevel data. Under some regularity conditions, we prove that the MMoE is dense in the space of any continuous mixed effects models in the sense of weak convergence. As a result, the MMoE has a potential to accurately resemble almost all characteristics inherited in multilevel data, including the marginal distributions, dependence structures, regression links, random intercepts and random slopes. In a particular case where the multilevel data is hierarchical, we further show that a nested version of the MMoE universally approximates a broad range of dependence structures of the random effects among different factor levels.


title: Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel

abstract: Explaining generalizations and preventing over-confident predictions are central goals of studies on the loss landscape of neural networks. Flatness, defined as loss invariability on perturbations of a pre-trained solution, is widely accepted as a predictor of generalization in this context. However, the problem that flatness and generalization bounds can be changed arbitrarily according to the scale of a parameter was pointed out, and previous studies partially solved the problem with restrictions: Counter-intuitively, their generalization bounds were still variant for the function-preserving parameter scaling transformation or limited only to an impractical network structure. As a more fundamental solution, we propose new prior and posterior distributions invariant to scaling transformations by \textit{decomposing} the scale and connectivity of parameters, thereby allowing the resulting generalization bound to describe the generalizability of a broad class of networks with the more practical class of transformations such as weight decay with batch normalization. We also show that the above issue adversely affects the uncertainty calibration of Laplace approximation and propose a solution using our invariant posterior. We empirically demonstrate our posterior provides effective flatness and calibration measures with low complexity in such a practical parameter transformation case, supporting its practical effectiveness in line with our rationale.


title: Experimental demonstration of the criterion for the prepare-and-measure nonclassicality

abstract: The prepare-and-measure theory is a new type of quantum paradox that reveals the incompatibility between classical theory and quantum mechanics in terms of the dimensionality of physical systems.Just as the Horodecki criterion can determine whether given quantum states are capable of exhibiting Bell nonclassicality, a similar criterion is needed for the prepare-and-measure theory to determine whether given uantum states can exhibit the prepare-and-measure nonclassicality. Recently, Poderini et al. [Phys. Rev. Research 2, 043106 (2020)] presented such a criterion for the prepare-and-measure nonclassicality. In this work, we experimentally validate this criterion -- 52 different sets of quantum states are prepared and tested one by one using this criterion to determine whether they can exhibit the prepare-and-measure nonclassicality, and the experimental results are in good agreement with the theoretical expectations. The criterion experimentally verified here has the potential to be widely used in future research on the prepare-and-measure nonclassicality.


title: Multi-Prompt Alignment for Multi-Source Unsupervised Domain Adaptation

abstract: Most existing methods for unsupervised domain adaptation (UDA) rely on a shared network to extract domain-invariant features. However, when facing multiple source domains, optimizing such a network involves updating the parameters of the entire network, making it both computationally expensive and challenging, particularly when coupled with min-max objectives. Inspired by recent advances in prompt learning that adapts high-capacity models for downstream tasks in a computationally economic way, we introduce Multi-Prompt Alignment (MPA), a simple yet efficient framework for multi-source UDA. Given a source and target domain pair, MPA first trains an individual prompt to minimize the domain gap through a contrastive loss. Then, MPA denoises the learned prompts through an auto-encoding process and aligns them by maximizing the agreement of all the reconstructed prompts. Moreover, we show that the resulting subspace acquired from the auto-encoding process can easily generalize to a streamlined set of target domains, making our method more efficient for practical usage. Extensive experiments show that MPA achieves state-of-the-art results on three popular datasets with an impressive average accuracy of 54.1% on DomainNet.


title: Dual Progressive Transformations for Weakly Supervised Semantic Segmentation

abstract: Weakly supervised semantic segmentation (WSSS), which aims to mine the object regions by merely using class-level labels, is a challenging task in computer vision. The current state-of-the-art CNN-based methods usually adopt Class-Activation-Maps (CAMs) to highlight the potential areas of the object, however, they may suffer from the part-activated issues. To this end, we try an early attempt to explore the global feature attention mechanism of vision transformer in WSSS task. However, since the transformer lacks the inductive bias as in CNN models, it can not boost the performance directly and may yield the over-activated problems. To tackle these drawbacks, we propose a Convolutional Neural Networks Refined Transformer (CRT) to mine a globally complete and locally accurate class activation maps in this paper. To validate the effectiveness of our proposed method, extensive experiments are conducted on PASCAL VOC 2012 and CUB-200-2011 datasets. Experimental evaluations show that our proposed CRT achieves the new state-of-the-art performance on both the weakly supervised semantic segmentation task the weakly supervised object localization task, which outperform others by a large margin.


title: A Posteriori Risk Classification and Ratemaking with Random Effects in the Mixture-of-Experts Model

abstract: A well-designed framework for risk classification and ratemaking in automobile insurance is key to insurers' profitability and risk management, while also ensuring that policyholders are charged a fair premium according to their risk profile. In this paper, we propose to adapt a flexible regression model, called the Mixed LRMoE, to the problem of a posteriori risk classification and ratemaking, where policyholder-level random effects are incorporated to better infer their risk profile reflected by the claim history. We also develop a stochastic variational Expectation-Conditional-Maximization algorithm for estimating model parameters and inferring the posterior distribution of random effects, which is numerically efficient and scalable to large insurance portfolios. We then apply the Mixed LRMoE model to a real, multiyear automobile insurance dataset, where the proposed framework is shown to offer better fit to data and produce posterior premium which accurately reflects policyholders' claim history.


title: Catalysis in Action via Elementary Thermal Operations

abstract: We investigate catalysis in the framework of elementary thermal operations, leveraging the distinct features of such operations to illuminate catalytic dynamics. As groundwork, we establish new technical tools that enhance the computability of state transition rules for elementary thermal operations. Specifically, we provide a complete characterisation of state transitions for a qutrit system and special classes of initial states of arbitrary dimension. By employing these tools in conjunction with numerical methods, we find that by adopting a small catalyst, including just a qubit catalyst, one can significantly enlarge the set of state transitions for a qutrit system. This advancement notably narrows the gap of reachable states between elementary thermal operations and generic thermal operations. Furthermore, we decompose catalytic transitions into time-resolved evolution, which critically enables the tracking of nonequilibrium free energy exchanges between the system and bath. Our results provide evidence for the existence of simple and practicable catalytic advantage in thermodynamics while offering insight into analysing the mechanism of catalytic processes.


title: Construction and Applications of Billion-Scale Pre-Trained Multimodal Business Knowledge Graph

abstract: Business Knowledge Graphs (KGs) are important to many enterprises today, providing factual knowledge and structured data that steer many products and make them more intelligent. Despite their promising benefits, building business KG necessitates solving prohibitive issues of deficient structure and multiple modalities. In this paper, we advance the understanding of the practical challenges related to building KG in non-trivial real-world systems. We introduce the process of building an open business knowledge graph (OpenBG) derived from a well-known enterprise, Alibaba Group. Specifically, we define a core ontology to cover various abstract products and consumption demands, with fine-grained taxonomy and multimodal facts in deployed applications. OpenBG is an open business KG of unprecedented scale: 2.6 billion triples with more than 88 million entities covering over 1 million core classes/concepts and 2,681 types of relations. We release all the open resources (OpenBG benchmarks) derived from it for the community and report experimental results of KG-centric tasks. We also run up an online competition based on OpenBG benchmarks, and has attracted thousands of teams. We further pre-train OpenBG and apply it to many KG- enhanced downstream tasks in business scenarios, demonstrating the effectiveness of billion-scale multimodal knowledge for e-commerce. All the resources with codes have been released at \url{https://github.com/OpenBGBenchmark/OpenBG}.


title: INT: Towards Infinite-frames 3D Detection with An Efficient Framework

abstract: It is natural to construct a multi-frame instead of a single-frame 3D detector for a continuous-time stream. Although increasing the number of frames might improve performance, previous multi-frame studies only used very limited frames to build their systems due to the dramatically increased computational and memory cost. To address these issues, we propose a novel on-stream training and prediction framework that, in theory, can employ an infinite number of frames while keeping the same amount of computation as a single-frame detector. This infinite framework (INT), which can be used with most existing detectors, is utilized, for example, on the popular CenterPoint, with significant latency reductions and performance improvements. We've also conducted extensive experiments on two large-scale datasets, nuScenes and Waymo Open Dataset, to demonstrate the scheme's effectiveness and efficiency. By employing INT on CenterPoint, we can get around 7% (Waymo) and 15% (nuScenes) performance boost with only 2~4ms latency overhead, and currently SOTA on the Waymo 3D Detection leaderboard.


title: The Role of Time Delay in Sim2real Transfer of Reinforcement Learning for Cyber-Physical Systems

abstract: This paper analyzes the simulation to reality gap in reinforcement learning (RL) cyber-physical systems with fractional delays (i.e. delays that are non-integer multiple of the sampling period). The consideration of fractional delay has important implications on the nature of the cyber-physical system considered. Systems with delays are non-Markovian, and the system state vector needs to be extended to make the system Markovian. We show that this is not possible when the delay is in the output, and the problem would always be non-Markovian. Based on this analysis, a sampling scheme is proposed that results in efficient RL training and agents that perform well in realistic multirotor unmanned aerial vehicle simulations. We demonstrate that the resultant agents do not produce excessive oscillations, which is not the case with RL agents that do not consider time delay in the model.


title: Hyperbolic VAE via Latent Gaussian Distributions

abstract: We propose a Gaussian manifold variational auto-encoder (GM-VAE) whose latent space consists of a set of Gaussian distributions. It is known that the set of the univariate Gaussian distributions with the Fisher information metric form a hyperbolic space, which we call a Gaussian manifold. To learn the VAE endowed with the Gaussian manifolds, we propose a pseudo-Gaussian manifold normal distribution based on the Kullback-Leibler divergence, a local approximation of the squared Fisher-Rao distance, to define a density over the latent space. In experiments, we demonstrate the efficacy of GM-VAE on two different tasks: density estimation of image datasets and environment modeling in model-based reinforcement learning. GM-VAE outperforms the other variants of hyperbolic- and Euclidean-VAEs on density estimation tasks and shows competitive performance in model-based reinforcement learning. We observe that our model provides strong numerical stability, addressing a common limitation reported in previous hyperbolic-VAEs.


title: EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional Compression

abstract: In this work we focus our attention on distributed optimization problems in the context where the communication time between the server and the workers is non-negligible. We obtain novel methods supporting bidirectional compression (both from the server to the workers and vice versa) that enjoy new state-of-the-art theoretical communication complexity for convex and nonconvex problems. Our bounds are the first that manage to decouple the variance/error coming from the workers-to-server and server-to-workers compression, transforming a multiplicative dependence to an additive one. Moreover, in the convex regime, we obtain the first bounds that match the theoretical communication complexity of gradient descent. Even in this convex regime, our algorithms work with biased gradient estimators, which is non-standard and requires new proof techniques that may be of independent interest. Finally, our theoretical results are corroborated through suitable experiments.


title: Optimal Query Complexities for Dynamic Trace Estimation

abstract: We consider the problem of minimizing the number of matrix-vector queries needed for accurate trace estimation in the dynamic setting where our underlying matrix is changing slowly, such as during an optimization process. Specifically, for any $m$ matrices $A_1,...,A_m$ with consecutive differences bounded in Schatten-$1$ norm by $\alpha$, we provide a novel binary tree summation procedure that simultaneously estimates all $m$ traces up to $\epsilon$ error with $\delta$ failure probability with an optimal query complexity of $\widetilde{O}\left(m \alpha\sqrt{\log(1/\delta)}/\epsilon + m\log(1/\delta)\right)$, improving the dependence on both $\alpha$ and $\delta$ from Dharangutte and Musco (NeurIPS, 2021). Our procedure works without additional norm bounds on $A_i$ and can be generalized to a bound for the $p$-th Schatten norm for $p \in [1,2]$, giving a complexity of $\widetilde{O}\left(m \alpha\left(\sqrt{\log(1/\delta)}/\epsilon\right)^p +m \log(1/\delta)\right)$. By using novel reductions to communication complexity and information-theoretic analyses of Gaussian matrices, we provide matching lower bounds for static and dynamic trace estimation in all relevant parameters, including the failure probability. Our lower bounds (1) give the first tight bounds for Hutchinson's estimator in the matrix-vector product model with Frobenius norm error even in the static setting, and (2) are the first unconditional lower bounds for dynamic trace estimation, resolving open questions of prior work.


title: Assortment Optimization Under the Multivariate MNL Model

abstract: We study an assortment optimization problem under a multi-purchase choice model in which customers choose a bundle of up to one product from each of two product categories. Different bundles have different utilities and the bundle price is the summation of the prices of products in it. For the uncapacitated setting where any set of products can be offered, we prove that this problem is strongly NP-hard. We show that an adjusted-revenue-ordered assortment provides a 1/2-approximation. Furthermore, we develop an approximation framework based on a linear programming relaxation of the problem and obtain a 0.74-approximation algorithm. This approximation ratio almost matches the integrality gap of the linear program, which is proven to be at most 0.75. For the capacitated setting, we prove that there does not exist a constant-factor approximation algorithm assuming the Exponential Time Hypothesis. The same hardness result holds for settings with general bundle prices or more than two categories. Finally, we conduct numerical experiments on randomly generated problem instances. The average approximation ratios of our algorithms are over 99%.


title: Exact Spectral Function of One-Dimensional Bose Gases

abstract: Strong correlation in one-dimensional (1D) quantum systems drastically changes their dynamic and transport properties in the presence of the interaction. In this letter, combining quantum integrable theory with numerics, we exactly compute the spectral function of 1D Lieb-Liniger gas at a many-body level of large scales. It turns out that a full capture of the power-law singularities in the vicinities of thresholds requires system size as large as thousands of particles. Our research essentially confirms the validity of the nonlinear Tomonaga-Luttinger liquid and provides a reliable technique for studying critical behaviour emerged only in thermodynamic limit.


title: Searching for Signal of Primordial Black Hole from CMB Lensing and $\gamma$-ray Emissions

abstract: In this $\textit{Letter}$, we search for the signal of the primordial black holes (PBHs) by correlating the $\gamma$-ray emissions in the MeV energy band produced by the Hawking evaporation and the lensing effect of the cosmic microwave background (CMB). We use the conservative case of the astrophysical model as much as possible in the calculations, since the potential astrophysical origins dominate the observed emission in the MeV energy band. By carefully discussing the appropriate energy bands corresponding to different PBHs masses, it is worth expecting a tight constraint on the fraction of the Schwarzschild PBHs in the mass range of $10^{16} - 5\times10^{17}\,{\rm g}$, by simulations of the sensitivity of the future CMB-S4 project and the $\gamma$-ray telescope e-ASTROGAM. Furthermore, we also consider the PBHs model with spins, and find that the constraining ability of the PBHs fraction from the correlation between CMB lensing and $\gamma$-ray emissions can be improved by another order of magnitude, which could importantly fill the gaps with PBHs fraction limits in the mass range of $5\times 10^{17} - 2\times 10^{18}\,{\rm g}$.


title: ASTF: Visual Abstractions of Time-Varying Patterns in Radio Signals

abstract: A time-frequency diagram is a commonly used visualization for observing the time-frequency distribution of radio signals and analyzing their time-varying patterns of communication states in radio monitoring and management. While it excels when performing short-term signal analyses, it becomes inadaptable for long-term signal analyses because it cannot adequately depict signal time-varying patterns in a large time span on a space-limited screen. This research thus presents an abstract signal time-frequency (ASTF) diagram to address this problem. In the diagram design, a visual abstraction method is proposed to visually encode signal communication state changes in time slices. A time segmentation algorithm is proposed to divide a large time span into time slices.Three new quantified metrics and a loss function are defined to ensure the preservation of important time-varying information in the time segmentation. An algorithm performance experiment and a user study are conducted to evaluate the effectiveness of the diagram for long-term signal analyses.


title: Robust Unsupervised Multi-task and Transfer Learning on Gaussian Mixture Models

abstract: Unsupervised learning has been widely used in many real-world applications. One of the simplest and most important unsupervised learning models is the Gaussian mixture model (GMM). In this work, we study the multi-task learning problem on GMMs, which aims to leverage potentially similar GMM parameter structures among tasks to obtain improved learning performance compared to single-task learning. We propose a multi-task GMM learning procedure based on the EM algorithm that not only can effectively utilize unknown similarity between related tasks but is also robust against a fraction of outlier tasks from arbitrary distributions. The proposed procedure is shown to achieve minimax optimal rate of convergence for both parameter estimation error and the excess mis-clustering error, in a wide range of regimes. Moreover, we generalize our approach to tackle the problem of transfer learning for GMMs, where similar theoretical results are derived. Finally, we demonstrate the effectiveness of our methods through simulations and real data examples. To the best of our knowledge, this is the first work studying multi-task and transfer learning on GMMs with theoretical guarantees.


title: Induced cosmological anisotropy by a gauge-gravity interaction

abstract: We present a simple model which generates cosmological anisotropies on top of standard FLRW geometry. This is in some sense reminiscent of the mean field approximation, where the mean field cosmological model under consideration would be the standard FLRW, and the anisotropy is a small perturbative correction on top of it. Using a supergravity-inspired model, we confirm that the stable fixed point of our model corresponds to standard FLRW cosmology. We use a Bianchi VII$_0$-type model supplemented with a scalar and $U(1)$ gauge fields, and we show that the anisotropies of the geometry are generated by the non-trivial interaction between the gravity sector and the $U(1)$ gauge sector. Studying the attractor flow, we show that the anisotropies are present at early times (high redshift) and decay asymptotically to an FLRW attractor fixed point. With such a mechanism, observations of non-isotropy are not contradictory to FLRW geometry or indeed the $\Lambda$CDM model. Such models could in principle shed some insights on the present cosmological tensions.


title: Central concentration of warm and dense molecular gas in a strongly lensed submillimeter galaxy at z=6

abstract: We report the detection of the CO(12-11) line emission toward G09-83808 (or H-ATLAS J090045.4+004125), a strongly-lensed submillimeter galaxy at $z = 6.02$, with Atacama Large Millimeter/submillimeter Array observations. Combining previously detected [O III]$\,88\:\mathrm{\mu m}$, [N II]$\,205\:\mathrm{\mu m}$, and dust continuum at 0.6$\:$mm and 1.5$\:$mm, we investigate the physical properties of the multi-phase interstellar medium in G09-83808. A source-plane reconstruction reveals that the region of the CO(12-11) emission is compact ($R_\mathrm{{e, CO}}=0.49^{+0.29}_{-0.19}\,\mathrm{kpc}$) and roughly coincides with that of the dust continuum. Non-local thermodynamic equilibrium radiative transfer modeling of CO spectral-line energy distribution reveals that most of the CO(12-11) emission comes from a warm (kinetic temperature of $T_{\mathrm{kin}}=320\pm170\:$K) and dense ($\log(n_{\mathrm{H2}}/\mathrm{cm^{-3}})=5.4\pm0.6$) gas, indicating that the warm and dense molecular gas is concentrated in the central 0.5-kpc region. The luminosity ratio in G09-83808 is estimated to be $L_\mathrm{{CO(12-11)}} / L_\mathrm{{CO(6-5)}}=1.1\pm0.2$. The high ratio is consistent with those in local active galactic nuclei (AGNs) and $6<z<7$ quasars, the fact of which implies that G09-83808 would be a good target to explore dust-obscured AGNs in the epoch of reionization. In the reconstructed [O III]$\,88\:\mathrm{\mu m}$ and [N II]$\,205\:\mathrm{\mu m}$ cubes, we also find that a monotonic velocity gradient is extending over the central starburst region by a factor of two and that star-forming sub-components exist. High-resolution observations of bright [C II]$\,158\:\mathrm{\mu m}$ line emissions will enable us to characterize the kinematics of a possible rotating disk and the nature of the sub-components.


title: Low Thermal Conductivity Phase Change Memory Superlattices

abstract: Phase change memory devices are typically reset by melt-quenching a material to radically lower its electrical conductance. The high power and concomitantly high current density required to reset phase change materials is the major issue that limits the access times of 3D phase change memory architectures. Phase change superlattices were developed to lower the reset energy by confining the phase transition to the interface between two different phase change materials. However, the high thermal conductivity of the superlattices means that heat is poorly confined within the phase change material, and most of the thermal energy is wasted to the surrounding materials. Here, we identified Ti as a useful dopant for substantially lowering the thermal conductivity of Sb2Te3-GeTe superlattices whilst also stabilising the layered structure from unwanted disordering. We demonstrate via laser heating that lowering the thermal conductivity by doping the Sb2Te3 layers with Ti halves the switching energy compared to superlattices that only use interfacial phase change transitions and strain engineering. The thermally optimized superlattice has (0 0 l) crystallographic orientation yet a thermal conductivity of just 0.25 W/m.K in the "on" (set) state. Prototype phase change memory devices that incorporate this Ti-doped superlattice switch faster and and at a substantially lower voltage than the undoped superlattice. During switching the Ti-doped Sb2Te3 layers remain stable within the superlattice and only the Ge atoms are active and undergo interfacial phase transitions. In conclusion, we show the potential of thermally optimised Sb2Te3-GeTe superlattices for a new generation of energy-efficient electrical and optical phase change memory.


title: Laser Spectroscopy for the Study of Exotic Nuclei

abstract: Investigation into the properties and structure of unstable nuclei far from stability remains a key avenue of research in modern nuclear physics. These efforts are motivated by the continual observation of unexpected structure phenomena in nuclei with unusual proton-to-neutron ratios. In recent decades, laser spectroscopy techniques have made significant contributions in our understanding of exotic nuclei in different mass regions encompassing almost the entire nuclear chart. This is achieved through determining multiple fundamental properties of nuclear ground and isomeric states, such as nuclear spins, magnetic dipole and electric quadrupole moments and charge radii, via the measurement of hyperfine structures and isotope shifts in the atomic or ionic spectra of the nuclei of interest. These properties, when measured with sufficient precision for a long range of isotopes, offer prominent tests of recently developed state-of-the-art theory and help to stimulate new developments to improve the many-body methods and nucleon-nucleon interactions at the core of these models. With the aim of exploring more exotic short-lived nuclei located ever closer to the proton and neutron driplines, laser spectroscopy techniques, with their continuous technological developments towards higher resolution and higher sensitivity, are extensively employed at current- and next-generation radioactive ion beam facilities worldwide. Ongoing efforts in parallel promise to make even more exotic species available for study at next-generation facilities. Very recently, an innovative application of laser spectroscopy on molecules containing short-lived nuclei has been demonstrated offering additional opportunities for several fields of research, e.g. fundamental symmetry studies and astrophysics.


title: Auroras on Mars: from Discovery to New Developments

abstract: Auroras are emissions in a planetary atmosphere caused by its interactions with the surrounding plasma environment. They have been observed in most planets and some moons of the solar system. Since their first discovery in 2005, Mars auroras have been studied extensively and is now a rapidly growing area of research. Since Mars lacks an intrinsic global magnetic field, its crustal field is distributed throughout the planet and its interactions with the surrounding plasma environment lead to a number of complex processes resulting in several types of auroras uncommon on Earth. Martian auroras have been classified as diffuse, discrete and proton aurora. With new capability of synoptic observations made possible with the Hope probe, two new types of auroras have been observed. One of them, which occurs on a much larger spatial scale, covering much of the disk, is known as discrete sinuous aurora. The other subcategory is one of proton auroras observed in patches. Further study of these phenomena will provide insights into the interactions between the atmosphere, magnetosphere and the surrounding plasma environment of Mars. We provide a brief review of the work done on the subject in the past 17 years since their discovery, and report new developments based on observations with Hope probe.


title: The Replicator Dynamic, Chain Components and the Response Graph

abstract: In this paper we examine the relationship between the flow of the replicator dynamic, the continuum limit of Multiplicative Weights Update, and a game's response graph. We settle an open problem establishing that under the replicator, sink chain components -- a topological notion of long-run outcome of a dynamical system -- always exist and are approximated by the sink connected components of the game's response graph. More specifically, each sink chain component contains a sink connected component of the response graph, as well as all mixed strategy profiles whose support consists of pure profiles in the same connected component, a set we call the content of the connected component. As a corollary, all profiles are chain recurrent in games with strongly connected response graphs. In any two-player game sharing a response graph with a zero-sum game, the sink chain component is unique. In two-player zero-sum and potential games the sink chain components and sink connected components are in a one-to-one correspondence, and we conjecture that this holds in all games.


title: Superconductivity Induced by Site-Selective Arsenic Doping in Mo$_5$Si$_3$

abstract: Arsenic doping in silicides has been much less studied compared with phosphorus. In this study, superconductivity is successfully induced by As doping in Mo$_5$Si$_3$. The superconducting transition temperature ($T_c$) reaches 7.7 K, which is higher than those in previously known W$_5$Si$_3$-type superconductors. Mo$_5$Si$_2$As is a type-II BCS superconductor with upper and lower critical fields of 6.65 T and 22.4 mT, respectively. In addition, As atoms are found to selectively take the 8$h$ sites in Mo$_5$Si$_2$As. The emergence of superconductivity is possibly due to the shift of Fermi level as a consequence of As doping, as revealed by the specific heat measurements and first-principles calculations. Our work provides not only another example of As doping, but also a practical strategy to achieve superconductivity in silicides through Fermi level engineering.


title: Global regularity results for a class of singular/degenerate fully nonlinear elliptic equations

abstract: We provide the Alexandroff-Bakelman-Pucci estimate and global $C^{1, \alpha}$-regularity for a class of singular/degenerate fully nonlinear elliptic equations. We also derive the existence of a viscosity solution to the Dirichlet problem with the associated operator.


title: Advanced interfacial phase change material: structurally confined and interfacially extended superlattice

abstract: Interfacial Phase Change Memory (iPCM) retrench unnecessary power consumption due to wasted heat generated during phase change by reducing unnecessary entropic loss. In this study, an advanced iPCM (GeTe/Ti-Sb2Te3 Superlattice) is synthesized by doping Ti into Sb2Te3. Structural analysis and density functional theory (DFT) calculations confirm that bonding distortion and structurally well-confined layers contribute to improve phase change properties in iPCM. Ti-Sb2Te3 acts as an effective thermal barrier to localize the generated heat inside active region, which leads to reduction of switching energy. Since Ge-Te bonds adjacent to short and strong Ti-Te bonds are more elongated than the bonds near Sb-Te, it is easier for Ge atoms to break the bond with Te due to strengthened Peierls distortions (Rlong/Rshort) during phase change process. Properties of advanced iPCM (cycling endurance, write speed/energy) exceed previous records. Moreover, well-confined multi-level states are obtained with advanced iPCM, showing potential as a neuromorphic memory. Our work paves the way for designing superlattice based PCM by controlling confinement layers.


title: A polynomial construction of perfect sequence covering arrays

abstract: A PSCA$(v, t, \lambda)$ is a multiset of permutations of the $v$-element alphabet $\{0, \dots, v-1\}$ such that every sequence of $t$ distinct elements of the alphabet appears in the specified order in exactly $\lambda$ permutations. For $v \geq t$, let $g(v, t)$ be the smallest positive integer $\lambda$ such that a PSCA$(v, t, \lambda)$ exists. We present an explicit construction that proves $g(v,t) = O(v^{t(t-2)})$ for fixed $t \geq 4$. The method of construction involves taking a permutation representation of the group of projectivities of a suitable projective space of dimension $t - 2$ and deleting all but a certain number of symbols from each permutation. In the case that this space is a Desarguesian projective plane, we also show that there exists a permutation representation of the group of projectivities of the plane that covers the vast majority of 4-sequences of its points a fixed number of times.


title: Ram Pressure Stripping and ISM disc Truncation : Prediction vs. Observation

abstract: Ram pressure stripping (RPS) is known to be a key environmental effect that can remove interstellar gas from galaxies in a cluster. The RPS process is commonly described as a competition between the ram pressure by the intracluster medium (ICM) and the anchoring pressure on the interstellar medium (ISM) by the gravitational potential of a galaxy. However, the actual gas stripping process can be more complicated due to the complexity of gas physics such as compression and geometrical self-shielding as well as cooling and heating. In order to verify how well the observed signatures of the RPS process can be understood as simple momentum transfer, we compare the stripping radii of Virgo cluster galaxies in different stages of RPS measured from the HI observation with the predicted gas truncation radii for the given conditions. For the sample undergoing active RPS, we generally find good agreements between predictions and observations within a measurement uncertainty. On the other hand, galaxies likely in the early or later RPS stage and/or the ones with signs of environmental impacts other than RPS such as tidal interaction or starvation, show some discrepancies. Our results imply that the conventional RPS relation works reasonably well in a broad sense when RPS is the most dominant process and the galaxy is located where the surrounding environment can be well defined. Otherwise, more careful inspections on the second mechanism and local environment are required to assess the impact of RPS on the target.


title: Language-Family Adapters for Low-Resource Multilingual Neural Machine Translation

abstract: Large multilingual models trained with self-supervision achieve state-of-the-art results in a wide range of natural language processing tasks. Self-supervised pretrained models are often fine-tuned on parallel data from one or multiple language pairs for machine translation. Multilingual fine-tuning improves performance on low-resource languages but requires modifying the entire model and can be prohibitively expensive. Training a new adapter on each language pair or training a single adapter on all language pairs without updating the pretrained model has been proposed as a parameter-efficient alternative. However, the former does not permit any sharing between languages, while the latter shares parameters for all languages and is susceptible to negative interference. In this paper, we propose training language-family adapters on top of mBART-50 to facilitate cross-lingual transfer. Our approach outperforms related baselines, yielding higher translation scores on average when translating from English to 17 different low-resource languages. We also show that language-family adapters provide an effective method to translate to languages unseen during pretraining.


title: On the Spectral properties of power graphs over certain groups

abstract: The power graph $P(\Omega)$ of a group $\Omega$ is a graph with the vertex set $\Omega$ such that two distinct vertices form an edge if and only if one of them is an integral power of the other. In this article, we determine the power graph of the group $\mathcal{G} = \langle s,r \, : r^{2^kp} = s^2 = e,~ srs^{-1} = r^{2^{k-1}p-1}\rangle$. Further, we compute its characteristic polynomial for the adjacency, Laplacian, and signless Laplacian matrices associated with this power graph. In addition, we determine its spectrum, Laplacian spectrum, and Laplacian energy.


title: Efficient Graph based Recommender System with Weighted Averaging of Messages

abstract: We showcase a novel solution to a recommendation system problem where we face a perpetual soft item cold start issue. Our system aims to recommend demanded products to prospective sellers for listing in Amazon stores. These products always have only few interactions thereby giving rise to a perpetual soft item cold start situation. Modern collaborative filtering methods solve cold start using content attributes and exploit the existing implicit signals from warm start items. This approach fails in our use-case since our entire item set faces cold start issue always. Our Product Graph has over 500 Million nodes and over 5 Billion edges which makes training and inference using modern graph algorithms very compute intensive. To overcome these challenges we propose a system which reduces the dataset size and employs an improved modelling technique to reduce storage and compute without loss in performance. Particularly, we reduce our graph size using a filtering technique and then exploit this reduced product graph using Weighted Averaging of Messages over Layers (WAML) algorithm. WAML simplifies training on large graphs and improves over previous methods by reducing compute time to 1/7 of LightGCN and 1/26 of Graph Attention Network (GAT) and increasing recall$@100$ by 66% over LightGCN and 2.3x over GAT.


title: Smart Meters Integration in Distribution System State Estimation with Collaborative Filtering and Deep Gaussian Process

abstract: The problem of state estimations for electric distribution system is considered. A collaborative filtering approach is proposed in this paper to integrate the slow time-scale smart meter measurements in the distribution system state estimation, in which the deep Gaussian process is incorporated to infer the fast time-scale pseudo measurements and avoid anomalies. Numerical tests have demonstrated the higher estimation accuracy of the proposed method.


title: Universal Prompt Tuning for Graph Neural Networks

abstract: In recent years, prompt tuning has sparked a research surge in adapting pre-trained models. Unlike the unified pre-training strategy employed in the language field, the graph field exhibits diverse pre-training strategies, posing challenges in designing appropriate prompt-based tuning methods for graph neural networks. While some pioneering work has devised specialized prompting functions for models that employ edge prediction as their pre-training tasks, these methods are limited to specific pre-trained GNN models and lack broader applicability. In this paper, we introduce a universal prompt-based tuning method called Graph Prompt Feature (GPF) for pre-trained GNN models under any pre-training strategy. GPF operates on the input graph's feature space and can theoretically achieve an equivalent effect to any form of prompting function. Consequently, we no longer need to illustrate the prompting function corresponding to each pre-training strategy explicitly. Instead, we employ GPF to obtain the prompted graph for the downstream task in an adaptive manner. We provide rigorous derivations to demonstrate the universality of GPF and make guarantee of its effectiveness. The experimental results under various pre-training strategies indicate that our method performs better than fine-tuning, with an average improvement of about 1.4% in full-shot scenarios and about 3.2% in few-shot scenarios. Moreover, our method significantly outperforms existing specialized prompt-based tuning methods when applied to models utilizing the pre-training strategy they specialize in. These numerous advantages position our method as a compelling alternative to fine-tuning for downstream adaptations.


title: Helium-bearing superconductor at high pressure

abstract: Helium (He) is the most inert noble gas at ambient conditions. It adopts a hexagonal close packed structure (P63/mmc) and remains in the insulating phase up to 32 TPa. In contrast, lithium (Li) is one of the most reactive metals at zero pressure, while its cubic high-pressure phase (Fd-3m) is a weak metallic electride above 475 GPa. Strikingly, a stable compound of Li5He2 (R-3m) was formed by mixing Fd-3m Li with P63/mmc He above 700 GPa. The presence of helium promotes the lattice transformation from Fd-3m Li to Pm-3m Li, and tuns the three-dimensional distributed interstitial electrons into the mixture of zero- and two-dimensional anionic electrons. This significantly increases the degree of metallization at the Fermi level, consequently, the coupling of conductive anionic electrons with the Li-dominated vibrations is the key factor to the formation of superconducting electride Li5He2 with a transition temperature up to 26 K, dynamically stable to pressures down to 210 GPa.


title: Review of Electric Vehicle Charging Technologies, Configurations, and Architectures

abstract: Electric Vehicles (EVs) are projected to be one of the major contributors to energy transition in the global transportation due to their rapid expansion. The EVs will play a vital role in achieving a sustainable transportation system by reducing fossil fuel dependency and greenhouse gas (GHG) emissions. However, high level of EVs integration into the distribution grid has introduced many challenges for the power grid operation, safety, and network planning due to the increase in load demand, power quality impacts and power losses. An increasing fleet of electric mobility requires the advanced charging systems to enhance charging efficiency and utility grid support. Innovative EV charging technologies are obtaining much attention in recent research studies aimed at strengthening EV adoption while providing ancillary services. Therefore, analysis of the status of EV charging technologies is significant to accelerate EV adoption with advanced control strategies to discover a remedial solution for negative grid impacts, enhance desired charging efficiency and grid support. This paper presents a comprehensive review of the current deployment of EV charging systems, international standards, charging configurations, EV battery technologies, architecture of EV charging stations, and emerging technical challenges. The charging systems require a dedicated converter topology, a control strategy and international standards for charging and grid interconnection to ensure optimum operation and enhance grid support. An overview of different charging systems in terms of onboard and off-board chargers, AC-DC and DC-DC converter topologies, and AC and DC-based charging station architectures are evaluated.


title: Coexisting structural disorder and robust spin-polarization in half-metallic FeMnVAl

abstract: Half-metallic ferromagnets (HMF) are on one of the most promising materials in the field of spintronics due to their unique band structure consisting of one spin sub-band having metallic characteristics along with another sub-band with semiconductor-like behavior. In this work, we report the synthesis of a novel quaternary Heusler alloy FeMnVAl and have studied the structural, magnetic, transport, and electronic properties complemented with first-principles calculations. Among different possible structurally ordered arrangements, the optimal structure is identified by theoretical energy minimization. The corresponding spin-polarized band structure calculations indicates the presence of a half-metallic ferromagnetic ground state. A detailed and careful investigation of the x-ray diffraction data, M\"{o}ssbauer and nuclear magnetic resonance spectra suggest the presence of site-disorder between the Fe and Mn atoms in the stable ordered structure of the system. The magnetic susceptibility measurement clearly establishes a ferromagnetic-like transition below $\sim$213 K. The ${^{57}}$Fe M\"{o}ssbauer spectrometry measurements suggest only the Mn-spins could be responsible for the magnetic order, which is consistent with our theoretical calculation. Surprisingly, the density-functional-theory calculations reveal that the spin-polarization value is almost immunized (92.4\% ${\rightarrow}$ 90.4\%) from the Mn-Fe structural disorder, even when nonmagnetic Fe and moment carrying Mn sites are entangled inseparably. Robustness of spin polarization and half metallicity in the studied FeMnVAl compound comprising structural disorder is thus quite interesting and could provide a new direction to investigate and understand the exact role of disorders on spin polarization in these class of materials, over the available knowledge.


title: Flavor Hierarchy from Smooth Confinement

abstract: We present a model to explain the Standard Model flavor hierarchy. Our model is based on explicit smooth confinement (namely confinement without chiral symmetry breaking in a supersymmetric gauge theory) at an intermediate energy scale, before the electroweak symmetry breaking by the Higgs condensation at lower energy. In our context, the smooth confinement preserves the SU(3) and the chiral SU(2)$_L\times$ U(1)$_Y$ symmetry in a supersymmetric Standard Model, while this internal symmetry becomes dynamically gauged in the end. In contrast to Razamat-Tong's symmetric mass generation model also preserving the $G_{\rm SM} \equiv$ SU(3)$_C\times$ SU(2)$_L\times$ U(1)$_Y$ internal symmetry, our model introduces different matter contents with a different kind of superpotential deformation irrelevant at UV, which further induces Yukawa-Higgs terms marginal at IR, breaking the $G_{\rm SM}$ down to SU(3)$_C$ and the electromagnetic U(1)$_{\rm EM}$ only when Higgs condenses. In our model, the IR fermions in the first and second families are composite of UV fields, while the third family elementary fermions match between UV and IR. The smallness of the first and second family fermion masses is explained by the exponential hierarchy between the cutoff scale and the smooth confinement scale via dimensional transmutation. As a result, our UV Lagrangian only contains the natural parameters close to the order one.


title: Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction

abstract: Due to limited communication capacities of edge devices, most existing federated learning (FL) methods randomly select only a subset of devices to participate in training for each communication round. Compared with engaging all the available clients, the random-selection mechanism can lead to significant performance degradation on non-IID (independent and identically distributed) data. In this paper, we show our key observation that the essential reason resulting in such performance degradation is the class-imbalance of the grouped data from randomly selected clients. Based on our key observation, we design an efficient heterogeneity-aware client sampling mechanism, i.e., Federated Class-balanced Sampling (Fed-CBS), which can effectively reduce class-imbalance of the group dataset from the intentionally selected clients. In particular, we propose a measure of class-imbalance and then employ homomorphic encryption to derive this measure in a privacy-preserving way. Based on this measure, we also design a computation-efficient client sampling strategy, such that the actively selected clients will generate a more class-balanced grouped dataset with theoretical guarantees. Extensive experimental results demonstrate Fed-CBS outperforms the status quo approaches. Furthermore, it achieves comparable or even better performance than the ideal setting where all the available clients participate in the FL training.


title: Your Out-of-Distribution Detection Method is Not Robust!

abstract: Out-of-distribution (OOD) detection has recently gained substantial attention due to the importance of identifying out-of-domain samples in reliability and safety. Although OOD detection methods have advanced by a great deal, they are still susceptible to adversarial examples, which is a violation of their purpose. To mitigate this issue, several defenses have recently been proposed. Nevertheless, these efforts remained ineffective, as their evaluations are based on either small perturbation sizes, or weak attacks. In this work, we re-examine these defenses against an end-to-end PGD attack on in/out data with larger perturbation sizes, e.g. up to commonly used $\epsilon=8/255$ for the CIFAR-10 dataset. Surprisingly, almost all of these defenses perform worse than a random detection under the adversarial setting. Next, we aim to provide a robust OOD detection method. In an ideal defense, the training should expose the model to almost all possible adversarial perturbations, which can be achieved through adversarial training. That is, such training perturbations should based on both in- and out-of-distribution samples. Therefore, unlike OOD detection in the standard setting, access to OOD, as well as in-distribution, samples sounds necessary in the adversarial training setup. These tips lead us to adopt generative OOD detection methods, such as OpenGAN, as a baseline. We subsequently propose the Adversarially Trained Discriminator (ATD), which utilizes a pre-trained robust model to extract robust features, and a generator model to create OOD samples. Using ATD with CIFAR-10 and CIFAR-100 as the in-distribution data, we could significantly outperform all previous methods in the robust AUROC while maintaining high standard AUROC and classification accuracy. The code repository is available at https://github.com/rohban-lab/ATD .


title: About integer-valued variants of the theta and 6j symbols

abstract: These notes contain essentially a rewriting of several properties of two well-known quantities, the so-called theta symbol (or triangular symbol), which is rational, and the 6j symbol, which is usually irrational, in terms of two related integer-valued functions called gon and tet. Existence of these related integer-valued avatars, sharing most essential properties with their more popular partners, although a known fact, is often overlooked. The properties of gon and tet are easier to obtain, or to formulate, than those of the corresponding theta and 6j symbols, both in the classical and quantum situations. Their evaluation is also simpler (the paper displays a number of explicit formulae and evaluation procedures that may speed up some computer programs). These two integer-valued functions are unusual, in that their properties do not appear to be often discussed in the literature, but their features reflect those of related real-valued functions discussed in many places. Some of the properties that we shall discuss seem however to be new, in particular several relations between the function gon and the inverse Hilbert matrices.