title: Turning Noises to Fingerprint-Free "Credentials": Secure and Usable Drone Authentication

abstract: Drones have been widely used in various services, such as delivery and surveillance. Authentication forms the foundation of the security of these services. However, drones are expensive and may carry important payloads. To avoid being captured by attackers, drones should keep a safe distance from the verifier before authentication succeeds. This makes authentication methods that only work in very close proximity not applicable. Our work leverages drone noises for authentication. While using sounds for authentication is highly usable, how to handle various attacks that manipulate sounds is an \emph{unresolved challenge}. It is also unclear how to ensure robustness under various environmental sounds. Being the first in the literature, we address the two major challenges by exploiting unique characteristics of drone noises. We thereby build an authentication system that does \emph{not} rely on any drone sound fingerprints, keeps resilient to attacks, and is robust under environmental sounds. An extensive evaluation demonstrates its security and usability.


title: Exposing AI-Synthesized Human Voices Using Neural Vocoder Artifacts

abstract: The advancements of AI-synthesized human voices have introduced a growing threat of impersonation and disinformation. It is therefore of practical importance to developdetection methods for synthetic human voices. This work proposes a new approach to detect synthetic human voices based on identifying artifacts of neural vocoders in audio signals. A neural vocoder is a specially designed neural network that synthesizes waveforms from temporal-frequency representations, e.g., mel-spectrograms. The neural vocoder is a core component in most DeepFake audio synthesis models. Hence the identification of neural vocoder processing implies that an audio sample may have been synthesized. To take advantage of the vocoder artifacts for synthetic human voice detection, we introduce a multi-task learning framework for a binary-class RawNet2 model that shares the front-end feature extractor with a vocoder identification module. We treat the vocoder identification as a pretext task to constrain the front-end feature extractor to focus on vocoder artifacts and provide discriminative features for the final binary classifier. Our experiments show that the improved RawNet2 model based on vocoder identification achieves an overall high classification performance on the binary task.


title: Abundance matching analysis of the emission line galaxy sample in the extended Baryon Oscillation Spectroscopic Survey

abstract: We present the measurements of the small-scale clustering for the emission line galaxy (ELG) sample from the extended Baryon Oscillation Spectroscopic Survey (eBOSS) in the Sloan Digital Sky Survey IV (SDSS-IV). We use conditional abundance matching method to interpret the clustering measurements from $0.34h^{-1}\textrm{Mpc}$ to $70h^{-1}\textrm{Mpc}$. In order to account for the correlation between properties of emission line galaxies and their environment, we add a secondary connection between star formation rate of ELGs and halo accretion rate. Three parameters are introduced to model the ELG [OII] luminosity and to mimic the target selection of eBOSS ELGs. The parameters in our models are optimized using Markov Chain Monte Carlo (MCMC) method. We find that by conditionally matching star formation rate of galaxies and the halo accretion rate, we are able to reproduce the eBOSS ELG small scale clustering within 1$\sigma$ error level. Our best fit model shows that the eBOSS ELG sample only consists of $\sim 12\%$ of all star-forming galaxies, and the satellite fraction of eBOSS ELG sample is 19.3\%. We show that the effect of assembly bias is $\sim20\%$ on the two-point correlation function and $\sim5\%$ on the void probability function at scale of $r\sim 20 h^{-1}\rm Mpc$.


title: Brainomaly: Unsupervised Neurologic Disease Detection Utilizing Unannotated T1-weighted Brain MR Images

abstract: Harnessing the power of deep neural networks in the medical imaging domain is challenging due to the difficulties in acquiring large annotated datasets, especially for rare diseases, which involve high costs, time, and effort for annotation. Unsupervised disease detection methods, such as anomaly detection, can significantly reduce human effort in these scenarios. While anomaly detection typically focuses on learning from images of healthy subjects only, real-world situations often present unannotated datasets with a mixture of healthy and diseased subjects. Recent studies have demonstrated that utilizing such unannotated images can improve unsupervised disease and anomaly detection. However, these methods do not utilize knowledge specific to registered neuroimages, resulting in a subpar performance in neurologic disease detection. To address this limitation, we propose Brainomaly, a GAN-based image-to-image translation method specifically designed for neurologic disease detection. Brainomaly not only offers tailored image-to-image translation suitable for neuroimages but also leverages unannotated mixed images to achieve superior neurologic disease detection. Additionally, we address the issue of model selection for inference without annotated samples by proposing a pseudo-AUC metric, further enhancing Brainomaly's detection performance. Extensive experiments and ablation studies demonstrate that Brainomaly outperforms existing state-of-the-art unsupervised disease and anomaly detection methods by significant margins in Alzheimer's disease detection using a publicly available dataset and headache detection using an institutional dataset. The code is available from https://github.com/mahfuzmohammad/Brainomaly.


title: An Efficient Method for Joint Delay-Doppler Estimation of Moving Targets in Passive Radar

abstract: Passive radar systems can detect and track the moving targets of interest by exploiting non-cooperative illuminators-of-opportunity to transmit orthogonal frequency division multiplexing (OFDM) signals. These targets are searched using a bank of correlators tuned to the waveform corresponding to the given Doppler frequency shift and delay. In this paper, we study the problem of joint delay-Doppler estimation of moving targets in OFDM passive radar. This task of estimation is described as an atomic-norm regularized convex optimization problem, or equivalently, a semi-definite programming problem. The alternating direction method of multipliers (ADMM) can be employed which computes each variable in a Gauss-Seidel manner, but its convergence is lack of certificate. In this paper, we use a symmetric Gauss-Seidel (sGS) to the framework of ADMM, which only needs to compute some of the subproblems twice but has the ability to ensure convergence. We do some simulated experiments which illustrate that the sGS-ADMM is superior to ADMM in terms of accuracy and computing time.


title: Applications of integrated optical microcombs

abstract: Optical microcombs represent a new paradigm for generating laser frequency combs based on compact chip-scale devices, which have underpinned many modern technological advances for both fundamental science and industrial applications. Along with the surge in activity related to optical micro-combs in the past decade, their applications have also experienced rapid progress, not only in traditional fields such as frequency synthesis, signal processing, and optical communications, but also in new interdisciplinary fields spanning the frontiers of light detection and ranging (LiDAR), astronomical detection, neuromorphic computing, and quantum optics. This paper reviews the applications of optical microcombs. First, an overview of the devices and methods for generating optical microcombs is provided, which are categorized into material platforms, device architectures, soliton classes, and driving mechanisms. Second, the broad applications of optical microcombs are systematically reviewed, which are categorized into microwave photonics, optical communications, precision measurements, neuromorphic computing, and quantum optics. Finally, the current challenges and future perspectives are discussed.


title: Pattern formation of a pathway-based diffusion model: linear stability analysis and an asymptotic preserving method

abstract: We investigate the linear stability analysis of a pathway-based diffusion model (PBDM), which characterizes the dynamics of the engineered Escherichia coli populations [X. Xue and C. Xue and M. Tang, P LoS Computational Biology, 14 (2018), pp. e1006178]. This stability analysis considers small perturbations of the density and chemical concentration around two non-trivial steady states, and the linearized equations are transformed into a generalized eigenvalue problem. By formal analysis, when the internal variable responds to the outside signal fast enough, the PBDM converges to an anisotropic diffusion model, for which the probability density distribution in the internal variable becomes a delta function. We introduce an asymptotic preserving (AP) scheme for the PBDM that converges to a stable limit scheme consistent with the anisotropic diffusion model. Further numerical simulations demonstrate the theoretical results of linear stability analysis, i.e., the pattern formation, and the convergence of the AP scheme.


title: Geometry, quantum correlations, and phase transitions in the $\Lambda$-atomic configuration

abstract: The quantum phase diagram for a finite $3$-level system in the $\Lambda$ configuration, interacting with a two-mode electromagnetic field in a cavity, is determined by means of information measures such as fidelity, fidelity susceptibility and entanglement, applied to the reduced density matrix of the matter sector of the system. The quantum phases are explained by emphasizing the spontaneous symmetry breaking along the separatrix. Additionally, a description of the reduced density matrix of one atom in terms of a simplex allows a geometric representation of the entanglement and purity properties of the system. These concepts are calculated for both, the symmetry-adapted variational coherent states and the numerical diagonalisation of the Hamiltonian, and compared. The differences in purity and entanglement obtained in both calculations can be explained and visualised by means of this simplex representation.


title: Approximate Thompson Sampling via Epistemic Neural Networks

abstract: Thompson sampling (TS) is a popular heuristic for action selection, but it requires sampling from a posterior distribution. Unfortunately, this can become computationally intractable in complex environments, such as those modeled using neural networks. Approximate posterior samples can produce effective actions, but only if they reasonably approximate joint predictive distributions of outputs across inputs. Notably, accuracy of marginal predictive distributions does not suffice. Epistemic neural networks (ENNs) are designed to produce accurate joint predictive distributions. We compare a range of ENNs through computational experiments that assess their performance in approximating TS across bandit and reinforcement learning environments. The results indicate that ENNs serve this purpose well and illustrate how the quality of joint predictive distributions drives performance. Further, we demonstrate that the \textit{epinet} -- a small additive network that estimates uncertainty -- matches the performance of large ensembles at orders of magnitude lower computational cost. This enables effective application of TS with computation that scales gracefully to complex environments.


title: Computational study of defect complexes in $\beta$-LiGaO$_2$ and their relation to the donor-acceptor-pair recombination

abstract: Hybrid functional calculations are presented for defects in LiGaO$_2$ with the fraction of non-local exchange adjusted to reproduce the recently reported exciton gap of 6.0 eV. We study how the defect transition levels of the main native defects change with respect to the band edges compared to earlier calculations which assumed a smaller band gap near 5.1 eV. In addition, we consider defect complexes formed by combining the main native donor Ga$_\mathrm{Li}$ with the main acceptors, $V_\mathrm{Li}$ and Li$_\mathrm{Ga}$ antisites as function of their relative position. These results are used to tentatively identify the photoluminescence bands previous assigned to donor-acceptor-pair recombination.


title: RETVec: Resilient and Efficient Text Vectorizer

abstract: This paper describes RETVec, an efficient, resilient, and multilingual text vectorizer designed for neural-based text processing. RETVec combines a novel character encoding with an optional small embedding model to embed words into a 256-dimensional vector space. The RETVec embedding model is pre-trained using pair-wise metric learning to be robust against typos and character-level adversarial attacks. In this paper, we evaluate and compare RETVec to state-of-the-art vectorizers and word embeddings on popular model architectures and datasets. These comparisons demonstrate that RETVec leads to competitive, multilingual models that are significantly more resilient to typos and adversarial text attacks. RETVec is available under the Apache 2 license at https://github.com/google-research/retvec.


title: Bridge Damage Cause Estimation Using Multiple Images Based on Visual Question Answering

abstract: In this paper, a bridge member damage cause estimation framework is proposed by calculating the image position using Structure from Motion (SfM) and acquiring its information via Visual Question Answering (VQA). For this, a VQA model was developed that uses bridge images for dataset creation and outputs the damage or member name and its existence based on the images and questions. In the developed model, the correct answer rate for questions requiring the member's name and the damage's name were 67.4% and 68.9%, respectively. The correct answer rate for questions requiring a yes/no answer was 99.1%. Based on the developed model, a damage cause estimation method was proposed. In the proposed method, the damage causes are narrowed down by inputting new questions to the VQA model, which are determined based on the surrounding images obtained via SfM and the results of the VQA model. Subsequently, the proposed method was then applied to an actual bridge and shown to be capable of determining damage and estimating its cause. The proposed method could be used to prevent damage causes from being overlooked, and practitioners could determine inspection focus areas, which could contribute to the improvement of maintenance techniques. In the future, it is expected to contribute to infrastructure diagnosis automation.


title: Wigner Function Analysis of Finite Matter-Radiation Systems

abstract: We show that the behaviour in phase space of the Wigner function associated to the electromagnetic modes carries the information of both, the entanglement properties between matter and field, and the regions in parameter space where quantum phase transitions take place. A finer classification for the continuous phase transitions is obtained through the computation of the surface of minimum fidelity.


title: How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation

abstract: Generative Pre-trained Transformer (GPT) models have shown remarkable capabilities for natural language generation, but their performance for machine translation has not been thoroughly investigated. In this paper, we present a comprehensive evaluation of GPT models for machine translation, covering various aspects such as quality of different GPT models in comparison with state-of-the-art research and commercial systems, effect of prompting strategies, robustness towards domain shifts and document-level translation. We experiment with eighteen different translation directions involving high and low resource languages, as well as non English-centric translations, and evaluate the performance of three GPT models: ChatGPT, GPT3.5 (text-davinci-003), and text-davinci-002. Our results show that GPT models achieve very competitive translation quality for high resource languages, while having limited capabilities for low resource languages. We also show that hybrid approaches, which combine GPT models with other translation systems, can further enhance the translation quality. We perform comprehensive analysis and human evaluation to further understand the characteristics of GPT translations. We hope that our paper provides valuable insights for researchers and practitioners in the field and helps to better understand the potential and limitations of GPT models for translation.


title: Bayesian Covariance Estimation for Multi-group Matrix-variate Data

abstract: Multi-group covariance estimation for matrix-variate data with small within group sample sizes is a key part of many data analysis tasks in modern applications. To obtain accurate group-specific covariance estimates, shrinkage estimation methods which shrink an unstructured, group-specific covariance either across groups towards a pooled covariance or within each group towards a Kronecker structure have been developed. However, in many applications, it is unclear which approach will result in more accurate covariance estimates. In this article, we present a hierarchical prior distribution which flexibly allows for both types of shrinkage. The prior linearly combines shrinkage across groups towards a shared pooled covariance and shrinkage within groups towards a group-specific Kronecker covariance. We illustrate the utility of the proposed prior in speech recognition and an analysis of chemical exposure data.


title: HOPE: Human-Centric Off-Policy Evaluation for E-Learning and Healthcare

abstract: Reinforcement learning (RL) has been extensively researched for enhancing human-environment interactions in various human-centric tasks, including e-learning and healthcare. Since deploying and evaluating policies online are high-stakes in such tasks, off-policy evaluation (OPE) is crucial for inducing effective policies. In human-centric environments, however, OPE is challenging because the underlying state is often unobservable, while only aggregate rewards can be observed (students' test scores or whether a patient is released from the hospital eventually). In this work, we propose a human-centric OPE (HOPE) to handle partial observability and aggregated rewards in such environments. Specifically, we reconstruct immediate rewards from the aggregated rewards considering partial observability to estimate expected total returns. We provide a theoretical bound for the proposed method, and we have conducted extensive experiments in real-world human-centric tasks, including sepsis treatments and an intelligent tutoring system. Our approach reliably predicts the returns of different policies and outperforms state-of-the-art benchmarks using both standard validation methods and human-centric significance tests.


title: Giant twist-angle dependence of thermal conductivity in bilayer graphene originating from strong interlayer coupling

abstract: Recently, the twist-angle effect on 2D van der Walls (vdW) materials, such as bilayer graphene, has attracted great attention. Many novel electronic, magnetic and even optical properties induced by such effect have been discovered. However, the twist-angle effect on phononic property is not so remarkable. By investigating the thermal conductivity of twist bilayer graphene (TBG), here we reveal that the trivial twist-angle effect on phononic property observed in previous studies is owing to the non-localization nature of phonons. This characteristic makes phonons hardly trapped by the weak interlayer potentials induced by the twist-angle dependent Moir\'e pattern. We propose that the twist-angle effect can be effectively enhanced by increasing the interface coupling. In use of a sandwich structure composed of h-BN and TBG, we demonstrate that the thermal conductivity of TBG can be either significantly increased or dramatically decreased, under the synergistic modulation of interlayer coupling strength and twist angle. Particularly, the twist-angle effect can lead to a nontrivial reduction of thermal conductivity up to 78% when a strong interlayer coupling is applied. The reduction is several times larger than that observed in the freestanding TBG where the reduction is attributed to the twist-angle dependent phonon scatterings induced by the edge phonons. The underlying mechanism for the giant twist-angle dependence of thermal conductivity is further revealed on the basis of phonon transport theory. Our findings provide a platform for achieving efficient twist-angle modulation on phonon transport property of vdW materials.


title: Cost-effective Models for Detecting Depression from Speech

abstract: Depression is the most common psychological disorder and is considered as a leading cause of disability and suicide worldwide. An automated system capable of detecting signs of depression in human speech can contribute to ensuring timely and effective mental health care for individuals suffering from the disorder. Developing such automated system requires accurate machine learning models, capable of capturing signs of depression. However, state-of-the-art models based on deep acoustic representations require abundant data, meticulous selection of features, and rigorous training; the procedure involves enormous computational resources. In this work, we explore the effectiveness of two different acoustic feature groups - conventional hand-curated and deep representation features, for predicting the severity of depression from speech. We explore the relevance of possible contributing factors to the models' performance, including gender of the individual, severity of the disorder, content and length of speech. Our findings suggest that models trained on conventional acoustic features perform equally well or better than the ones trained on deep representation features at significantly lower computational cost, irrespective of other factors, e.g. content and length of speech, gender of the speaker and severity of the disorder. This makes such models a better fit for deployment where availability of computational resources is restricted, such as real time depression monitoring applications in smart devices.


title: Domain Agnostic Pipeline for Retina Vessel Segmentation

abstract: Automatic segmentation of retina vessels plays a pivotal role in clinical diagnosis of prevalent eye diseases, such as, Diabetic Retinopathy or Age-related Macular Degeneration. Due to the complex construction of blood vessels, with drastically varying thicknesses, accurate vessel segmentation can be quite a challenging task. In this work we show that it is possible to achieve near state-of-the-art performance, by crafting a careful thought pre-processing pipeline, without having to resort to complex networks and/or training routines. We also show that our model is able to maintain the same high segmentation performance across different datasets, very poor quality fundus images, as well as images of severe pathological cases. Code and models featured in this paper can be downloaded from http://github.com/farrell236/retina_segmentation. We also demonstrate the potential of our model at http://lazarus.ddns.net:8502.


title: Enhancing the accuracy of the Taylor polynomial by determining the remainder term

abstract: We determine the Lagrange function in Taylor polynomial approximation by solving an appropriate initial-value problem. Hence, we determine the remainder term which we then approximate by means of a natural cubic spline. This results in a significant improvement in the quality of the Taylor approximation. We observe improvements in the accuracy of the approximation of many orders of magnitude, including a case when the independent variable x lies beyond the relevant radius of convergence.


title: Identify local limiting factors of species distribution using min-linear logistic regression

abstract: Logistic regression is a commonly used building block in ecological modeling, but its additive structure among environmental predictors often assumes compensatory relationships between predictors, which can lead to problematic results. In reality, the distribution of species is often determined by the least-favored factor, according to von Liebig's Law of the Minimum, which is not addressed in modeling. To address this issue, we introduced the min-linear logistic regression model, which has a built-in minimum structure of competing factors. In our empirical analysis of the distribution of Asiatic black bears ($\textit{Ursus thibetanus}$), we found that the min-linear model performs well compared to other methods and has several advantages. By using the model, we were able to identify ecologically meaningful limiting factors on bear distribution across the survey area. The model's inherent simplicity and interpretability make it a promising tool for extending into other widely used ecological models.


title: Optimal Mix Among PAYGO, EET and Individual Savings

abstract: In order to deal with the aging problem, pension system is actively transformed into the funded scheme. However, the funded scheme does not completely replace PAYGO (Pay as You Go) scheme and there exist heterogeneous mixes among PAYGO, EET (Exempt, Exempt, Taxed) and individual savings in different countries. In this paper, we establish the optimal mix by solving a Nash equilibrium between the pension participants and the government. Given the obligatory PAYGO and EET contribution rates, the participants choose the optimal asset allocation of the individual savings and the consumption policies to achieve the objective. The results extend the ``Samuelson-Aaron" criterion to age-dependent preference orderings. And we identify three critical ages to distinguish the multiple outcomes of preference orderings based on heterogeneous characteristic parameters. The government is fully aware of the optimal feedback of the participants. It chooses the optimal PAYGO and EET contribution rates to maximize the overall utility of the participants weighted by each cohort's population. As such, the negative population growth rate leads to the decline of the PAYGO attractiveness as well as the increase of the older cohorts' weight in the government decision-making. The optimal mix is the comprehensive result of the two effects.


title: Complex saddles of three-dimensional de Sitter gravity via holography

abstract: We determine complex saddles of three-dimensional gravity with a positive cosmological constant by applying the recently proposed holography. It is sometimes useful to consider a complexified metric to study quantum gravity as in the case of the no-boundary proposal by Hartle and Hawking. However, there would be too many saddles for complexified gravity, and we should determine which saddles to take. We describe the gravity theory by three-dimensional SL$(2,\mathbb{C})$ Chern-Simons theory. At the leading order in the Newton constant, its holographic dual is given by Liouville theory with a large imaginary central charge. We examine geometry with a conical defect, called a de Sitter black hole, from a Liouville two-point function. We also consider geometry with two conical defects, whose saddles are determined by the monodromy matrix of Liouville four-point function. Utilizing Chern-Simons description, we extend the similar analysis to the case with higher-spin gravity.


title: A Tight Lower Bound for Compact Set Packing

abstract: This note is devoted to show a simple proof of a tight lower bound of the parameterized compact set packing problem, based on ETH.


title: Moby: Empowering 2D Models for Efficient Point Cloud Analytics on the Edge

abstract: 3D object detection plays a pivotal role in many applications, most notably autonomous driving and robotics. These applications are commonly deployed on edge devices to promptly interact with the environment, and often require near real-time response. With limited computation power, it is challenging to execute 3D detection on the edge using highly complex neural networks. Common approaches such as offloading to the cloud induce significant latency overheads due to the large amount of point cloud data during transmission. To resolve the tension between wimpy edge devices and compute-intensive inference workloads, we explore the possibility of empowering fast 2D detection to extrapolate 3D bounding boxes. To this end, we present Moby, a novel system that demonstrates the feasibility and potential of our approach. We design a transformation pipeline for Moby that generates 3D bounding boxes efficiently and accurately based on 2D detection results without running 3D detectors. Further, we devise a frame offloading scheduler that decides when to launch the 3D detector judiciously in the cloud to avoid the errors from accumulating. Extensive evaluations on NVIDIA Jetson TX2 with real-world autonomous driving datasets demonstrate that Moby offers up to 91.9% latency improvement with modest accuracy loss over state of the art.


title: A review of codebooks for CSI feedback in 5G new radio and beyond

abstract: Codebooks have been indispensable for wireless communication standard since the first release of the Long-Term Evolution in 2009. They offer an efficient way to acquire the channel state information (CSI) for multiple antenna systems. Nowadays, a codebook is not limited to a set of pre-defined precoders, it refers to a CSI feedback framework, which is more and more sophisticated. In this paper, we review the codebooks in 5G New Radio (NR) standards. The codebook timeline and the evolution trend are shown. Each codebook is elaborated with its motivation, the corresponding feedback mechanism, and the format of the precoding matrix indicator. Some insights are given to help grasp the underlying reasons and intuitions of these codebooks. Finally, we point out some unresolved challenges of the codebooks for future evolution of the standards. In general, this paper provides a comprehensive review of the codebooks in 5G NR and aims to help researchers understand the CSI feedback schemes from a standard and industrial perspective.


title: Existence of weak solutions to $p$-Navier-Stokes equations

abstract: We study the existence of weak solutions to the $p$-Navier-Stokes equations with a symmetric $p$-Laplacian on bounded domains. We construct a particular Schauder basis in $W_0^{1,p}(\Omega)$ with divergence free constraint and prove existence of weak solutions using the Galerkin approximation via this basis. Meanwhile, in the proof, we establish a chain rule for the $L^p$ norm of the weak solutions, which fixes a gap in our previous work. The equality of energy dissipation is also established for the weak solutions considered.


title: Maximizing the Bang Per Bit

abstract: Reducing memory traffic is critical to accelerate Lattice QCD computations on modern processors, given that such computations are memory-bandwidth bound. A commonly used strategy is mixed-precision solvers, however, these require careful treatment to ensure stable convergence. We give an overview of the strategies employed in QUDA to stabilize mixed-precision variants of Conjugate Gradient (CG), and its multi-shift brethren. Through the use of customized numerical storage formats we can significantly improve upon the precision achievable compared to IEEE numerical formats, increasing both the solver precision and stability achievable at fixed word size. We give examples using BiCGStab(l) and multi-shift CG solvers using the HISQ operator.


title: OMINACS: Online ML-Based IoT Network Attack Detection and Classification System

abstract: Several Machine Learning (ML) methodologies have been proposed to improve security in Internet Of Things (IoT) networks and reduce the damage caused by the action of malicious agents. However, detecting and classifying attacks with high accuracy and precision is still a major challenge. This paper proposes an online attack detection and network traffic classification system, which combines stream Machine Learning, Deep Learning, and Ensemble Learning technique. Using multiple stages of data analysis, the system can detect the presence of malicious traffic flows and classify them according to the type of attack they represent. Furthermore, we show how to implement this system both in an IoT network and from an ML point of view. The system was evaluated in three IoT network security datasets, in which it obtained accuracy and precision above 90% with a reduced false alarm rate.


title: Collapsars as Sites of r-process Nucleosynthesis: Systematic Near-Infrared Follow-up of Type Ic-BL Supernovae

abstract: One of the open questions following the discovery of GW170817 is whether neutron star mergers are the only astrophysical sites capable of producing $r$-process elements. Simulations have shown that 0.01-0.1M$_\odot$ of $r$-process material could be generated in the outflows originating from the accretion disk surrounding the rapidly rotating black hole that forms as a remnant to both neutron star mergers and collapsing massive stars associated with long-duration gamma-ray bursts (collapsars). The hallmark signature of $r$-process nucleosynthesis in the binary neutron star merger GW170817 was its long-lasting near-infrared emission, thus motivating a systematic photometric study of the light curves of broadlined stripped-envelope (Ic-BL) supernovae (SNe) associated with collapsars. We present the first systematic study of 25 SNe Ic-BL -- including 18 observed with the Zwicky Transient Facility and 7 from the literature -- in the optical/near-infrared bands to determine what quantity of $r$-process material, if any, is synthesized in these explosions. Using semi-analytic models designed to account for $r$-process production in SNe Ic-BL, we perform light curve fitting to derive constraints on the $r$-process mass for these SNe. We also perform independent light curve fits to models without $r$-process. We find that the $r$-process-free models are a better fit to the light curves of the objects in our sample. Thus we find no compelling evidence of $r$-process enrichment in any of our objects. Further high-cadence infrared photometric studies and nebular spectroscopic analysis would be sensitive to smaller quantities of $r$-process ejecta mass or indicate whether all collapsars are completely devoid of $r$-process nucleosynthesis.

categories: astro-ph.HE astro-ph.GA astro-ph.SR physics.data-an


title: Invertible Neural Skinning

abstract: Building animatable and editable models of clothed humans from raw 3D scans and poses is a challenging problem. Existing reposing methods suffer from the limited expressiveness of Linear Blend Skinning (LBS), require costly mesh extraction to generate each new pose, and typically do not preserve surface correspondences across different poses. In this work, we introduce Invertible Neural Skinning (INS) to address these shortcomings. To maintain correspondences, we propose a Pose-conditioned Invertible Network (PIN) architecture, which extends the LBS process by learning additional pose-varying deformations. Next, we combine PIN with a differentiable LBS module to build an expressive and end-to-end Invertible Neural Skinning (INS) pipeline. We demonstrate the strong performance of our method by outperforming the state-of-the-art reposing techniques on clothed humans and preserving surface correspondences, while being an order of magnitude faster. We also perform an ablation study, which shows the usefulness of our pose-conditioning formulation, and our qualitative results display that INS can rectify artefacts introduced by LBS well. See our webpage for more details: https://yashkant.github.io/invertible-neural-skinning/


title: Web Photo Source Identification based on Neural Enhanced Camera Fingerprint

abstract: With the growing popularity of smartphone photography in recent years, web photos play an increasingly important role in all walks of life. Source camera identification of web photos aims to establish a reliable linkage from the captured images to their source cameras, and has a broad range of applications, such as image copyright protection, user authentication, investigated evidence verification, etc. This paper presents an innovative and practical source identification framework that employs neural-network enhanced sensor pattern noise to trace back web photos efficiently while ensuring security. Our proposed framework consists of three main stages: initial device fingerprint registration, fingerprint extraction and cryptographic connection establishment while taking photos, and connection verification between photos and source devices. By incorporating metric learning and frequency consistency into the deep network design, our proposed fingerprint extraction algorithm achieves state-of-the-art performance on modern smartphone photos for reliable source identification. Meanwhile, we also propose several optimization sub-modules to prevent fingerprint leakage and improve accuracy and efficiency. Finally for practical system design, two cryptographic schemes are introduced to reliably identify the correlation between registered fingerprint and verified photo fingerprint, i.e. fuzzy extractor and zero-knowledge proof (ZKP). The codes for fingerprint extraction network and benchmark dataset with modern smartphone cameras photos are all publicly available at https://github.com/PhotoNecf/PhotoNecf.


title: Magnetoresistance signature of two-dimensional electronic states in Co$_3$Sn$_2$S$_2$

abstract: Two-dimensional (2D) Dirac bands and flat bands are characteristics of a kagome lattice. However, experimental studies on their electrical transport are few, because three-dimensional (3D) bulk bands of kagome materials, consisting of stacked 2D kagome layers, dominate the transport. We report a magnetoresistance (MR) study of a kagome material, Co$_3$Sn$_2$S$_2$. Based on analysis of the temperature, magnetic field, and field angle dependence of the resistivity, we obtain a complete anatomy of MR. Besides a magnon MR, a chirality-dependent MR, and a chiral-anomaly-induced MR, the most intriguing feature is an orbital MR that scales only with the out-of-plane field, which strongly indicates its 2D nature. We attribute it to the Dirac band of the kagome lattice.


title: VLN-Trans: Translator for the Vision and Language Navigation Agent

abstract: Language understanding is essential for the navigation agent to follow instructions. We observe two kinds of issues in the instructions that can make the navigation task challenging: 1. The mentioned landmarks are not recognizable by the navigation agent due to the different vision abilities of the instructor and the modeled agent. 2. The mentioned landmarks are applicable to multiple targets, thus not distinctive for selecting the target among the candidate viewpoints. To deal with these issues, we design a translator module for the navigation agent to convert the original instructions into easy-to-follow sub-instruction representations at each step. The translator needs to focus on the recognizable and distinctive landmarks based on the agent's visual abilities and the observed visual environment. To achieve this goal, we create a new synthetic sub-instruction dataset and design specific tasks to train the translator and the navigation agent. We evaluate our approach on Room2Room~(R2R), Room4room~(R4R), and Room2Room Last (R2R-Last) datasets and achieve state-of-the-art results on multiple benchmarks.


title: An explicit infinite homotopy in nonabelian Hodge theory in positive characteristic

abstract: This short note is extracted from Section 3 and Appendix of the paper entitled with Intersection de Rham complexes in positive characteristic by the same named authors, where an explicit infinite homotopy from a Higgs complex to the Frobenius pushforward of the corresponding de Rham complex in positive characteristic has been provided. The verification details, which are omitted therein, are provided here.


title: Transition between the stick and slip states in a simplified model of magnetic friction

abstract: We introduce a simplified model of magnetic friction, and investigate its behavior using both numerical and analytical methods. When resistance coefficient $\gamma$ is large, the movement of the system obeys the thermally activated process. In contrast, when $\gamma$ is sufficiently small, the slip and stick states behave as separate metastable states, and the lattice velocity depends on the probability that the slip state appears. We evaluate the velocities in both cases using several approximations and compare the results with those of numerical simulations.


title: Solving Boltzmann equation with neural sparse representation

abstract: We consider the neural sparse representation to solve Boltzmann equation with BGK and quadratic collision model, where a network-based ansatz that can approximate the distribution function with extremely high efficiency is proposed. Precisely, fully connected neural networks are employed in the time and spatial space so as to avoid the discretization in space and time. The different low-rank representations are utilized in the microscopic velocity for the BGK and quadratic collision model, resulting in a significant reduction in the degree of freedom. We approximate the discrete velocity distribution in the BGK model using the canonical polyadic decomposition. For the quadratic collision model, a data-driven, SVD-based linear basis is built based on the BGK solution. All these will significantly improve the efficiency of the network when solving Boltzmann equation. Moreover, the specially designed adaptive-weight loss function is proposed with the strategies as multi-scale input and Maxwellian splitting applied to further enhance the approximation efficiency and speed up the learning process. Several numerical experiments, including 1D wave and Sod problems and 2D wave problem, demonstrate the effectiveness of these neural sparse representation methods.


title: Electronic structure of InSb (001), (110), and (111)B surfaces

abstract: The electronic structure of various (001), (110), and (111)B surfaces of n-type InSb were studied with scanning tunneling microscopy and spectroscopy. The InSb(111)B (3x1) surface reconstruction is determined to be a disordered (111)B (3x3) surface reconstruction. The surface Fermi-level of the In rich and the equal In:Sb (001), (110), and (111)B surface reconstructions was observed to be pinned near the valence band edge. This observed pinning is consistent with a charge neutrality level lying near the valence band maximum. Sb termination was observed to shift the surface Fermi-level position by up to $254 \pm 35$ meV towards the conduction band on the InSb (001) surface and $60 \pm 35$ meV towards the conduction band on the InSb(111)B surface. The surface Sb on the (001) can shift the surface from electron depletion to electron accumulation. We propose the shift in the Fermi-level pinning is due to charge transfer from Sb clusters on the Sb terminated surfaces. Additionally, many sub-gap states were observed for the (111)B (3x1) surface, which are attributed to the disordered nature of this surface. This work demonstrates the tuning of the Fermi-level pinning position of InSb surfaces with Sb termination.


title: Generalization and Stability of Interpolating Neural Networks with Minimal Width

abstract: We investigate the generalization and optimization properties of shallow neural-network classifiers trained by gradient descent in the interpolating regime. Specifically, in a realizable scenario where model weights can achieve arbitrarily small training error $\epsilon$ and their distance from initialization is $g(\epsilon)$, we demonstrate that gradient descent with $n$ training data achieves training error $O(g(1/T)^2 /T)$ and generalization error $O(g(1/T)^2 /n)$ at iteration $T$, provided there are at least $m=\Omega(g(1/T)^4)$ hidden neurons. We then show that our realizable setting encompasses a special case where data are separable by the model's neural tangent kernel. For this and logistic-loss minimization, we prove the training loss decays at a rate of $\tilde O(1/ T)$ given polylogarithmic number of neurons $m=\Omega(\log^4 (T))$. Moreover, with $m=\Omega(\log^{4} (n))$ neurons and $T\approx n$ iterations, we bound the test loss by $\tilde{O}(1/n)$. Our results differ from existing generalization outcomes using the algorithmic-stability framework, which necessitate polynomial width and yield suboptimal generalization rates. Central to our analysis is the use of a new self-bounded weak-convexity property, which leads to a generalized local quasi-convexity property for sufficiently parameterized neural-network classifiers. Eventually, despite the objective's non-convexity, this leads to convergence and generalization-gap bounds that resemble those found in the convex setting of linear logistic regression.


title: Scalable Prompt Generation for Semi-supervised Learning with Language Models

abstract: Prompt-based learning methods in semi-supervised learning (SSL) settings have been shown to be effective on multiple natural language understanding (NLU) datasets and tasks in the literature. However, manually designing multiple prompts and verbalizers requires domain knowledge and human effort, making it difficult and expensive to scale across different datasets. In this paper, we propose two methods to automatically design multiple prompts and integrate automatic verbalizer in SSL settings without sacrificing performance. The first method uses various demonstration examples with learnable continuous prompt tokens to create diverse prompt models. The second method uses a varying number of soft prompt tokens to encourage language models to learn different prompts. For the verbalizer, we use the prototypical verbalizer to replace the manual one. In summary, we obtained the best average accuracy of 73.2% (a relative improvement of 2.52% over even the previous state-of-the-art SSL method with manual prompts and verbalizers) in different few-shot learning settings.


title: Characterizations of Network Auctions and Generalizations of VCG

abstract: With the growth of networks, promoting products through social networks has become an important problem. For auctions in social networks, items are needed to be sold to agents in a network, where each agent can bid and also diffuse the sale information to her neighbors. Thus, the agents' social relations are intervened with their bids in the auctions. In network auctions, the classical VCG mechanism fails to retain key properties. In order to better understand network auctions, in this paper, we characterize network auctions for the single-unit setting with respect to weak budget balance, individual rationality, incentive compatibility, efficiency, and other properties. For example, we present sufficient conditions for mechanisms to be efficient and (weakly) incentive compatible. With the help of these properties and new concepts such as rewards, participation rewards, and so on, we show how to design efficient mechanisms to satisfy incentive compatibility as much as possible, and incentive compatibility mechanisms to maximize the revenue. Our results provide insights into understanding auctions in social networks.


title: KLIF: An optimized spiking neuron unit for tuning surrogate gradient slope and membrane potential

abstract: Spiking neural networks (SNNs) have attracted much attention due to their ability to process temporal information, low power consumption, and higher biological plausibility. However, it is still challenging to develop efficient and high-performing learning algorithms for SNNs. Methods like artificial neural network (ANN)-to-SNN conversion can transform ANNs to SNNs with slight performance loss, but it needs a long simulation to approximate the rate coding. Directly training SNN by spike-based backpropagation (BP) such as surrogate gradient approximation is more flexible. Yet now, the performance of SNNs is not competitive compared with ANNs. In this paper, we propose a novel k-based leaky Integrate-and-Fire (KLIF) neuron model to improve the learning ability of SNNs. Compared with the popular leaky integrate-and-fire (LIF) model, KLIF adds a learnable scaling factor to dynamically update the slope and width of the surrogate gradient curve during training and incorporates a ReLU activation function that selectively delivers membrane potential to spike firing and resetting. The proposed spiking unit is evaluated on both static MNIST, Fashion-MNIST, CIFAR-10 datasets, as well as neuromorphic N-MNIST, CIFAR10-DVS, and DVS128-Gesture datasets. Experiments indicate that KLIF performs much better than LIF without introducing additional computational cost and achieves state-of-the-art performance on these datasets with few time steps. Also, KLIF is believed to be more biological plausible than LIF. The good performance of KLIF can make it completely replace the role of LIF in SNN for various tasks.


title: Faster Wavelet Tree Queries

abstract: Given a text, rank and select queries return the number of occurrences of a character up to a position (rank) or the position of a character with a given rank (select). These queries have applications in, e.g., compression, computational geometry, and most notably pattern matching in the form of the backward search -- the backbone of many compressed full-text indices. Currently, in practice, for text over non-binary alphabets, the wavelet tree is probably the most used data structure for rank and select queries. In this paper, we present techniques to speed up queries by a factor of two (access and select) up to three (rank), compared to the wavelet tree implementation contained in the widely used Succinct Data Structure Library (SDSL). To this end, we change the underlying tree structure from a binary tree to a 4-ary tree and reduce cache misses by approximating rank queries using a predictive model to prefetch all data required for the actual rank query.


title: Beamforming and Phase Shift Design for HR-IRS-aided Directional Modulation Network with a Malicious Attacker

abstract: In this paper, we propose to use hybrid relay-intelligent reflecting surface (HR-IRS) to improve the security performance of directional modulation (DM) system. In particular, the eavesdropper in this system works in full-duplex (FD) mode and he will eavesdrop on the confidential message (CM) as well as send malicious jamming. We aim to maximize the secrecy rate (SR) by jointly optimizing the receive beamforming, transmit beamforming and phase shift matrix (PSM) of HR-IRS. Since the optimization problem is un-convex and the variables are coupled to each other, we solve this problem by iteratively optimizing these variables. The receive beamforming and transmit beamforming are obtained based on generalized Rayleigh-Ritz theorem and Dinkelbach's Transform respectively. And for PSM, two methods, called separate optimization of PSM (SO-PSM) and joint optimization of PSM (JO-PSM) are proposed. Thus, two iterative algorithms are proposed accordingly, namely maximizing SR based on SO-PSM (Max-SR-SOP) and maximizing SR based on JO-PSM (Max-SR-JOP). The former has better performance and the latter has lower complexity. The simulation results show that when HR-IRS has sufficient power budget, the proposed Max-SR-SOP and Max-SR-JOP can enable HR-IRS-aided DM network to obtain higher SR than passive IRS-aided DM network.


title: Distributed Optimization for Reactive Power Sharing and Stability of Inverter-Based Resources Under Voltage Limits

abstract: Reactive power sharing and containment of voltages within limits for inverter-based resources (IBRs) are two important, yet coupled objectives in ac networks. In this article, we propose a distributed control technique to simultaneously achieve these objectives. Our controller consists of two components: a purely local nonlinear integral controller which adjusts the IBR voltage setpoint, and a distributed primal-dual optimizer that coordinates reactive power sharing between the IBRs. The controller prioritizes the voltage containment objective over reactive power sharing at all points in time; excluding the IBRs with saturated voltages, it provides reactive power sharing among all the IBRs. Considering the voltage saturation and the coupling between voltage and angle dynamics, a formal closed-loop stability analysis based on singular perturbation theory is provided, yielding practical tuning guidance for the overall control system. To validate the effectiveness of the proposed controller for different case studies, we apply it to a low-voltage microgrid and a microgrid adapted from the CIGRE medium-voltage network benchmark, both simulated in the MATLAB/Simulink environment.


title: Deciphering alloy composition in superconducting single-layer FeSe1-xSx on SrTiO3(001) substrates by machine learning of STM/S data

abstract: Scanning tunneling microscopy (STM) is a powerful technique for imaging atomic structure and inferring information on local elemental composition, chemical bonding, and electronic excitations. However, traditional methods of visual inspection can be challenging for such determination in multi-component alloys, particularly beyond the dilute limit due to chemical disorder and electronic inhomogeneity. One viable solution is to use machine learning to analyze STM data and identify patterns and correlations that may not be immediately apparent through visual inspection alone. Here, we apply this approach to determine the Se/S concentration in superconducting single-layer FeSe1-xSx alloy epitaxially grown on SrTiO3(100) substrate by molecular beam epitaxy. First, defect-related dI/dV tunneling spectra are identified by the K-means clustering method, followed by singular value decomposition to distinguish between those from S and Se. Such analysis provides an efficient and reliable determination of local elemental composition, and further reveals correlations of nanoscale chemical inhomogeneity to superconductivity in single-layer iron chalcogenide films.


title: A Federated Approach for Hate Speech Detection

abstract: Hate speech detection has been the subject of high research attention, due to the scale of content created on social media. In spite of the attention and the sensitive nature of the task, privacy preservation in hate speech detection has remained under-studied. The majority of research has focused on centralised machine learning infrastructures which risk leaking data. In this paper, we show that using federated machine learning can help address privacy the concerns that are inherent to hate speech detection while obtaining up to 6.81% improvement in terms of F1-score.


title: Dual-Domain Self-Supervised Learning for Accelerated Non-Cartesian MRI Reconstruction

abstract: While enabling accelerated acquisition and improved reconstruction accuracy, current deep MRI reconstruction networks are typically supervised, require fully sampled data, and are limited to Cartesian sampling patterns. These factors limit their practical adoption as fully-sampled MRI is prohibitively time-consuming to acquire clinically. Further, non-Cartesian sampling patterns are particularly desirable as they are more amenable to acceleration and show improved motion robustness. To this end, we present a fully self-supervised approach for accelerated non-Cartesian MRI reconstruction which leverages self-supervision in both k-space and image domains. In training, the undersampled data are split into disjoint k-space domain partitions. For the k-space self-supervision, we train a network to reconstruct the input undersampled data from both the disjoint partitions and from itself. For the image-level self-supervision, we enforce appearance consistency obtained from the original undersampled data and the two partitions. Experimental results on our simulated multi-coil non-Cartesian MRI dataset demonstrate that DDSS can generate high-quality reconstruction that approaches the accuracy of the fully supervised reconstruction, outperforming previous baseline methods. Finally, DDSS is shown to scale to highly challenging real-world clinical MRI reconstruction acquired on a portable low-field (0.064 T) MRI scanner with no data available for supervised training while demonstrating improved image quality as compared to traditional reconstruction, as determined by a radiologist study.


title: A stacky approach to identifying the semistable locus of bundles

abstract: We show that the semistable locus is the unique maximal open substack of the moduli stack of principal bundles over a curve that admits a schematic moduli space. For rank $2$ vector bundles it coincides with the unique maximal open substack that admits a separated moduli space, but for higher rank there exist other open substacks that admit separated moduli spaces.


title: Positronium Physics and Biomedical Applications

abstract: Positronium is the simplest bound state, built of an electron and a positron. Studies of positronium in vacuum and its decays in medium tell us about Quantum Electrodynamics, QED, and about the structure of matter and biological processes of living organisms at the nanoscale, respectively. Spectroscopic measurements constrain our understanding of QED bound state theory. Searches for rare decays and measurements of the effect of gravitation on positronium are used to look for new physics phenomena. In biological materials positronium decays are sensitive to the inter- and intra-molecular structure and to the metabolism of living organisms ranging from single cells to human beings. This leads to new ideas of positronium imaging in medicine using the fact that during positron emission tomography (PET) as much as 40% of positron annihilation occurs through the production of positronium atoms inside the patient's body. A new generation of the high sensitivity and multi-photon total-body PET systems opens perspectives for clinical applications of positronium as a biomarker of tissue pathology and the degree of tissue oxidation.


title: FastCod: Fast Brain Connectivity in Diffusion Imaging

abstract: Connectivity information derived from diffusion-weighted magnetic resonance images~(DW-MRIs) plays an important role in studying human subcortical gray matter structures. However, due to the $O(N^2)$ complexity of computing the connectivity of each voxel to every other voxel (or multiple ROIs), the current practice of extracting connectivity information is highly inefficient. This makes the processing of high-resolution images and population-level analyses very computationally demanding. To address this issue, we propose a more efficient way to extract connectivity information; briefly, we consider two regions/voxels to be connected if a white matter fiber streamline passes through them -- no matter where the streamline originates. We consider the thalamus parcellation task for demonstration purposes; our experiments show that our approach brings a 30 to 120 times speedup over traditional approaches with comparable qualitative parcellation results. We also demonstrate high-resolution connectivity features can be super-resolved from low-resolution DW-MRI in our framework. Together, these two innovations enable higher resolution connectivity analysis from DW-MRI. Our source code is availible at jasonbian97.github.io/fastcod.


title: Machine Love

abstract: While ML generates much economic value, many of us have problematic relationships with social media and other ML-powered applications. One reason is that ML often optimizes for what we want in the moment, which is easy to quantify but at odds with what is known scientifically about human flourishing. Thus, through its impoverished models of us, ML currently falls far short of its exciting potential, which is for it to help us to reach ours. While there is no consensus on defining human flourishing, from diverse perspectives across psychology, philosophy, and spiritual traditions, love is understood to be one of its primary catalysts. Motivated by this view, this paper explores whether there is a useful conception of love fitting for machines to embody, as historically it has been generative to explore whether a nebulous concept, such as life or intelligence, can be thoughtfully abstracted and reimagined, as in the fields of machine intelligence or artificial life. This paper forwards a candidate conception of machine love, inspired in particular by work in positive psychology and psychotherapy: to provide unconditional support enabling humans to autonomously pursue their own growth and development. Through proof of concept experiments, this paper aims to highlight the need for richer models of human flourishing in ML, provide an example framework through which positive psychology can be combined with ML to realize a rough conception of machine love, and demonstrate that current language models begin to enable embodying qualitative humanistic principles. The conclusion is that though at present ML may often serve to addict, distract, or divide us, an alternative path may be opening up: We may align ML to support our growth, through it helping us to align ourselves towards our highest aspirations.


title: Starspot mapping with adaptive parallel tempering. II. Application to TESS data for M-dwarf flare stars, AU Microscopii, YZ Canis Minoris, and EV Lacertae

abstract: Starspots and stellar flares are indicators of stellar magnetic activity.The magnetic energy stored around spots is thought to be the origin of flares, but the connection is not completely understood. To investigate the relation between spot locations deduced from the light curves and occurrence of flares therein, we perform starspot modeling for TESS light curves of three M-dwarf flare stars, AU Mic, YZ CMi, and EV Lac, using the code implemented in Paper I. The code enables to deduce multiple stellar/spot parameters by the adaptive parallel tempering algorithm efficiently. We found that flare occurrence frequency is not necessarily correlated with the rotation phases of the light curve for each star. The result of starspot modeling shows that either spot is always visible to the line of sight in all phases, and we suggest that this can be one of the reasons that there is no or less correlation between rotation phases and flare frequency. In addition, the amplitude and shape of the light curve for AU Mic and YZ CMi have varied in two years between different TESS Cycles. The result of starspot modeling suggests that this can be explained by the variations of spot size and latitude.


title: Distributed Planning with Asynchronous Execution with Local Navigation for Multi-agent Pickup and Delivery Problem

abstract: We propose a distributed planning method with asynchronous execution for multi-agent pickup and delivery (MAPD) problems for environments with occasional delays in agents' activities and flexible endpoints. MAPD is a crucial problem framework with many applications; however, most existing studies assume ideal agent behaviors and environments, such as a fixed speed of agents, synchronized movements, and a well-designed environment with many short detours for multiple agents to perform tasks easily. However, such an environment is often infeasible; for example, the moving speed of agents may be affected by weather and floor conditions and is often prone to delays. The proposed method can relax some infeasible conditions to apply MAPD in more realistic environments by allowing fluctuated speed in agents' actions and flexible working locations (endpoints). Our experiments showed that our method enables agents to perform MAPD in such an environment efficiently, compared to the baseline methods. We also analyzed the behaviors of agents using our method and discuss the limitations.


title: StyLIP: Multi-Scale Style-Conditioned Prompt Learning for CLIP-based Domain Generalization

abstract: Large-scale foundation models, such as CLIP, have demonstrated impressive zero-shot generalization performance on downstream tasks, leveraging well-designed language prompts. However, these prompt learning techniques often struggle with domain shift, limiting their generalization capabilities. In our study, we tackle this issue by proposing StyLIP, a novel approach for Domain Generalization (DG) that enhances CLIP's classification performance across domains. Our method focuses on a domain-agnostic prompt learning strategy, aiming to disentangle the visual style and content information embedded in CLIP's pre-trained vision encoder, enabling effortless adaptation to novel domains during inference. To achieve this, we introduce a set of style projectors that directly learn the domain-specific prompt tokens from the extracted multi-scale style features. These generated prompt embeddings are subsequently combined with the multi-scale visual content features learned by a content projector. The projectors are trained in a contrastive manner, utilizing CLIP's fixed vision and text backbones. Through extensive experiments conducted in five different DG settings on multiple benchmark datasets, we consistently demonstrate that StyLIP outperforms the current state-of-the-art (SOTA) methods.


title: Deep AHS: A Deep Learning Approach to Acoustic Howling Suppression

abstract: In this paper, we formulate acoustic howling suppression (AHS) as a supervised learning problem and propose a deep learning approach, called Deep AHS, to address it. Deep AHS is trained in a teacher forcing way which converts the recurrent howling suppression process into an instantaneous speech separation process to simplify the problem and accelerate the model training. The proposed method utilizes properly designed features and trains an attention based recurrent neural network (RNN) to extract the target signal from the microphone recording, thus attenuating the playback signal that may lead to howling. Different training strategies are investigated and a streaming inference method implemented in a recurrent mode used to evaluate the performance of the proposed method for real-time howling suppression. Deep AHS avoids howling detection and intrinsically prohibits howling from happening, allowing for more flexibility in the design of audio systems. Experimental results show the effectiveness of the proposed method for howling suppression under different scenarios.


title: The geometrically m-step solvable Grothendieck conjecture for affine hyperbolic curves over finitely generated fields

abstract: In this paper, we present some new results on the geometrically m-step solvable Grothendieck conjecture in anabelian geometry. Specifically, we show the (weak bi-anabelian and strong bi-anabelian) geometrically m-step solvable Grothendieck conjecture(s) for affine hyperbolic curves over fields finitely generated over the prime field. First of all, we show the conjecture over finite fields. Next, we show the geometrically m-step solvable version of the Oda-Tamagawa good reduction criterion for hyperbolic curves. Finally, by using these two results, we show the conjecture over fields finitely generated over the prime field.


title: Fabrication of Specimens for Atom Probe Tomography Using a Combined Gallium and Neon Focused Ion Beam Milling Approach

abstract: We demonstrate a new focused ion beam sample preparation method for atom probe tomography. The key aspect of the new method is that we use a neon ion beam for the final tip-shaping after conventional annulus milling using gallium ions. This dual-ion approach combines the benefits of the faster milling capability of the higher current gallium ion beam with the chemically inert and higher precision milling capability of the noble gas neon ion beam. Using a titanium-aluminum alloy and a layered aluminum/aluminum oxide material as test cases, we show that atom probe tips prepared using the combined gallium and neon ion approach are free from the gallium contamination that typically frustrates composition analysis of these materials due to implantation, diffusion, and embrittlement effects. We propose that by using a focused ion beam from a noble gas species, such as the neon ions demonstrated here, atom probe tomography can be more reliably performed on a larger range of materials than is currently possible using conventional techniques.


title: Clustered Covariate Regression

abstract: High covariate dimensionality is increasingly occurrent in model estimation, and existing techniques to address this issue typically require sparsity or discrete heterogeneity of the unobservable parameter vector. However, neither restriction may be supported by economic theory in some empirical contexts, leading to severe bias and misleading inference. The clustering-based grouped parameter estimator (GPE) introduced in this paper drops both restrictions in favour of the natural one that the parameter support be compact. GPE exhibits robust large sample properties under standard conditions and accommodates both sparse and non-sparse parameters whose support can be bounded away from zero. Extensive Monte Carlo simulations demonstrate the excellent performance of GPE in terms of bias reduction and size control compared to competing estimators. An empirical application of GPE to estimating price and income elasticities of demand for gasoline highlights its practical utility.


title: Multi-dimensional frequency dynamic convolution with confident mean teacher for sound event detection

abstract: Recently, convolutional neural networks (CNNs) have been widely used in sound event detection (SED). However, traditional convolution is deficient in learning time-frequency domain representation of different sound events. To address this issue, we propose multi-dimensional frequency dynamic convolution (MFDConv), a new design that endows convolutional kernels with frequency-adaptive dynamic properties along multiple dimensions. MFDConv utilizes a novel multi-dimensional attention mechanism with a parallel strategy to learn complementary frequency-adaptive attentions, which substantially strengthen the feature extraction ability of convolutional kernels. Moreover, in order to promote the performance of mean teacher, we propose the confident mean teacher to increase the accuracy of pseudo-labels from the teacher and train the student with high confidence labels. Experimental results show that the proposed methods achieve 0.470 and 0.692 of PSDS1 and PSDS2 on the DESED real validation dataset.


title: mmWave Coverage Extension Using Reconfigurable Intelligent Surfaces in Indoor Dense Spaces

abstract: In this work, we consider the deployment of reconfigurable intelligent surfaces (RISs) to extend the coverage of a millimeter-wave (mmWave) network in indoor dense spaces. We first integrate RIS into ray-tracing simulations to realistically capture the propagation characteristics, then formulate a non-convex optimization problem that minimizes the number of RISs under rate constraints. We propose a feasible point pursuit and successive convex approximation-based algorithm, which solves the problem by jointly selecting the RIS locations, optimizing the RIS phase-shifts, and allocating time resources to user equipments (UEs). The numerical results demonstrate substantial coverage extension by using at least four RISs, and a data rate of 130 Mbit/s is guaranteed for UEs in the considered area of an airplane cabin.


title: Digital Privacy Under Attack: Challenges and Enablers

abstract: Users have renewed interest in protecting their private data in the digital space. When they don't believe that their privacy is sufficiently covered by one platform, they will readily switch to another. Such an increasing level of privacy awareness has made privacy preservation an essential research topic. Nevertheless, new privacy attacks are emerging day by day. Therefore, a holistic survey to compare the discovered techniques on attacks over privacy preservation and their mitigation schemes is essential in the literature. We develop a study to fill this gap by assessing the resilience of privacy-preserving methods to various attacks and conducting a comprehensive review of countermeasures from a broader perspective. First, we introduce the fundamental concepts and critical components of privacy attacks. Second, we comprehensively cover major privacy attacks targeted at anonymous data, statistical aggregate data, and privacy-preserving models. We also summarize popular countermeasures to mitigate these attacks. Finally, some promising future research directions and related issues in the privacy community are envisaged. We believe this survey will successfully shed some light on privacy research and encourage researchers to entirely understand the resilience of different existing privacy-preserving approaches.


title: Energetics of the solar atmosphere

abstract: The excess temperature of the solar corona over the photosphere poses a challenge. Multiple energetic events contribute to maintaining the corona at such high temperatures. The energy released in different events can vary across several orders of magnitude. Large energy events of geomagnetic importance like flares and coronal mass ejections (CMEs) contribute little to the global energetics of the solar corona. Therefore, events with several (9-10) orders of magnitudes of lower energy, with much higher frequency of occurrence, need to be studied in great detail. Observations suggest that these impulsive events with different energies follow a power-law distribution, indicating a common underlying mechanism. We perform observation-motivated modeling of coronal loops (magnetic flux tubes) to understand the energetics of these small transient events and their similarity with impulsive events like flares. This thesis uses the EBTEL code based on the 0D hydrodynamical description of coronal loops. This approach is appropriate for getting quick estimates of the energetics of the system over a wide range of parameters. We then discuss the improvement of EBTEL to make it suitable over a broader range of parameters. This is followed by using improved EBTEL to explore the possibility of simulating impulsive events of different energy generated using a single power-law distribution. Comparison between observed emissions from various components of multi-thermal plasma and hydrodynamical models suggest the heating to be impulsive. Since field-aligned flows induced due to impulsive events are a crucial part of our modeling of coronal loops, we discuss the implications of such flows in the context of transition region heating.


title: Attribute-Specific Manipulation Based on Layer-Wise Channels

abstract: Image manipulation on the latent space of the pre-trained StyleGAN can control the semantic attributes of the generated images. Recently, some studies have focused on detecting channels with specific properties to directly manipulate the latent code, which is limited by the entanglement of the latent space. To detect the attribute-specific channels, we propose a novel detection method in the context of pre-trained classifiers. We analyse the gradients layer by layer on the style space. The intensities of the gradients indicate the channel's responses to specific attributes. The latent style codes of channels control separate attributes in the layers. We choose channels with top-$k$ gradients to control specific attributes in the maximum response layer. We implement single-channel and multi-channel manipulations with a certain attribute. Our methods can accurately detect relevant channels for a large number of face attributes. Extensive qualitative and quantitative results demonstrate that the proposed methods outperform state-of-the-art methods in generalization and scalability.


title: Quenching of low-energy optical absorption in bilayer C$_3$N polytypes

abstract: In this work we provide a first principles description of the electronic and optical properties of bilayers C$_3$N, with different stacking motifs AB, AB$'$ and AA$'$. Starting from quasi-particle electronic band-structures, we solve the Bethe Salpeter Equation (BSE) to access the excitonic properties of these bilayers. For all stacking sequences, we see strong optical absorption at energies lower than but close to that of the monolayer. Most relevant, we predict a strong quenching of the low-energy optical absorption, with negligible oscillator strength of low-lying bound excitons. This is a unique phenomenology that does not arise in the monolayer case, nor in other common homo-bilayers. We explain these findings in terms of the small interband dipole matrix elements associated to the valence-conduction transitions involved in these excitons, and discuss them in view of the different stacking motifs.


title: Optimal error bounds on the exponential wave integrator for the nonlinear Schr\"{o}dinger equation with low regularity potential and nonlinearity

abstract: We establish optimal error bounds for the exponential wave integrator (EWI) applied to the nonlinear Schr\"odinger equation (NLSE) with $ L^\infty $-potential and/or locally Lipschitz nonlinearity under the assumption of $ H^2 $-solution of the NLSE. For the semi-discretization in time by the first-order Gautschi-type EWI, we prove an optimal $ L^2 $-error bound at $ O(\tau) $ with $ \tau>0 $ being the time step size, together with a uniform $ H^2 $-bound of the numerical solution. For the full-discretization scheme obtained by using the Fourier spectral method in space, we prove an optimal $ L^2 $-error bound at $ O(\tau + h^2) $ without any coupling condition between $ \tau $ and $ h $, where $ h>0 $ is the mesh size. In addition, for $ W^{1, 4} $-potential and a little stronger regularity of the nonlinearity, under the assumption of $ H^3 $-solution, we obtain an optimal $ H^1 $-error bound. Furthermore, when the potential is of low regularity but the nonlinearity is sufficiently smooth, we propose an extended Fourier pseudospectral method which has the same error bound as the Fourier spectral method while its computational cost is similar to the standard Fourier pseudospectral method. Our new error bounds greatly improve the existing results for the NLSE with low regularity potential and/or nonlinearity. Extensive numerical results are reported to confirm our error estimates and to demonstrate that they are sharp.


title: Multistage Spatial Context Models for Learned Image Compression

abstract: Recent state-of-the-art Learned Image Compression methods feature spatial context models, achieving great rate-distortion improvements over hyperprior methods. However, the autoregressive context model requires serial decoding, limiting runtime performance. The Checkerboard context model allows parallel decoding at a cost of reduced RD performance. We present a series of multistage spatial context models allowing both fast decoding and better RD performance. We split the latent space into square patches and decode serially within each patch while different patches are decoded in parallel. The proposed method features a comparable decoding speed to Checkerboard while reaching the RD performance of Autoregressive and even also outperforming Autoregressive. Inside each patch, the decoding order must be carefully decided as a bad order negatively impacts performance; therefore, we also propose a decoding order optimization algorithm.


title: Catalyzed Single-Walled Carbon Nanotube Growth by Machine Learning Molecular Dynamics Simulation

abstract: Classical molecular dynamics (MD) simulations have been performed to elucidate the mechanism of single-walled carbon nanotube (SWCNT) growth. However, discussing the chirality has been challenging due to topological defects in simulated SWCNTs. Recently, the application of neural network to interatomic potentials has been actively studied and such interatomic potentials are called neural network potentials (NNPs). NNPs have a better ability for approximate functions and can predict complex systems' energy more accurately than conventional interatomic potentials. In this study, we developed an NNP to describe SWCNT growth more accurately. We successfully demonstrated defect-free and chirality-definable growth of SWCNTs on Fe nanoparticles by MD simulation. Furthermore, it was revealed that edge vacancies of SWCNTs caused defect formation and adatom diffusion contributed to vacancy healing, preventing defect formation.


title: Diffusive Molecular Communication with a Spheroidal Receiver for Organ-on-Chip Systems

abstract: Realistic models of the components and processes are required for molecular communication (MC) systems. In this paper, a spheroidal receiver structure is proposed for MC that is inspired by the 3D cell cultures known as spheroids being widely used in organ-on-chip systems. A simple diffusive MC system is considered where the spheroidal receiver and a point source transmitter are in an unbounded fluid environment. The spheroidal receiver is modeled as a porous medium for diffusive signaling molecules, then its boundary conditions and effective diffusion coefficient are characterized. It is revealed that the spheroid amplifies the diffusion signal, but also disperses the signal which reduces the information communication rate. Furthermore, we analytically formulate and derive the concentration Green's function inside and outside the spheroid in terms of infinite series-forms that are confirmed by a particle-based simulator (PBS).


title: Ionized carbon as a tracer of the assembly of interstellar clouds

abstract: Molecular hydrogen clouds are a key component of the interstellar medium because they are the birthplaces for stars. They are embedded in atomic gas that pervades the interstellar space. However, the details of how molecular clouds assemble from and interact with the atomic gas are still largely unknown. As a result of new observations of the 158~$\mu$m line of ionized carbon CII in the Cygnus region within the FEEDBACK program on SOFIA (Stratospheric Observatory for Infrared Astronomy), we present compelling evidence that CII unveils dynamic interactions between cloud ensembles. This process is neither a head-on collision of fully molecular clouds nor a gentle merging ofonly atomic clouds. Moreover, we demonstrate that the dense molecular clouds associated with the DR21 and W75N star-forming regions and a cloud at higher velocity are embedded in atomic gas and all components interact over a large range of velocities (20 km/s). The atomic gas has a density of 100 cm$^{-3}$ and a temperature of 100 K. We conclude that the CII 158 $\mu$m line is an excellent tracer to witness the processes involved in cloud interactions and anticipate further detections of this phenomenon in other regions


title: Stochastic Approximation Approaches to Group Distributionally Robust Optimization

abstract: This paper investigates group distributionally robust optimization (GDRO), with the purpose to learn a model that performs well over $m$ different distributions. First, we formulate GDRO as a stochastic convex-concave saddle-point problem, and demonstrate that stochastic mirror descent (SMD), using $m$ samples in each iteration, achieves an $O(m (\log m)/\epsilon^2)$ sample complexity for finding an $\epsilon$-optimal solution, which matches the $\Omega(m/\epsilon^2)$ lower bound up to a logarithmic factor. Then, we make use of techniques from online learning to reduce the number of samples required in each round from $m$ to $1$, keeping the same sample complexity. Specifically, we cast GDRO as a two-players game where one player simply performs SMD and the other executes an online algorithm for non-oblivious multi-armed bandits. Next, we consider a more practical scenario where the number of samples that can be drawn from each distribution is different, and propose a novel formulation of weighted GDRO, which allows us to derive distribution-dependent convergence rates. Denote by $n_i$ the sample budget for the $i$-th distribution, and assume $n_1 \geq n_2 \geq \cdots \geq n_m$. In the first approach, we incorporate non-uniform sampling into SMD such that the sample budget is satisfied in expectation, and prove that the excess risk of the $i$-th distribution decreases at an $O(\sqrt{n_1 \log m}/n_i)$ rate. In the second approach, we use mini-batches to meet the budget exactly and also reduce the variance in stochastic gradients, and then leverage stochastic mirror-prox algorithm, which can exploit small variances, to optimize a carefully designed weighted GDRO problem. Under appropriate conditions, it attains an $O((\log m)/\sqrt{n_i})$ convergence rate, which almost matches the optimal $O(\sqrt{1/n_i})$ rate of only learning from the $i$-th distribution with $n_i$ samples.


title: Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE

abstract: This technical report briefly describes our JDExplore d-team's submission Vega v1 on the General Language Understanding Evaluation (GLUE) leaderboard, where GLUE is a collection of nine natural language understanding tasks, including question answering, linguistic acceptability, sentiment analysis, text similarity, paraphrase detection, and natural language inference. [Method] We investigate several effective strategies and choose their best combination setting as the training recipes. As for model structure, we employ the vanilla Transformer with disentangled attention as the basic block encoder. For self-supervised training, we employ the representative denoising objective (i.e., replaced token detection) in phase 1 and combine the contrastive objective (i.e., sentence embedding contrastive learning) with it in phase 2. During fine-tuning, several advanced techniques such as transductive fine-tuning, self-calibrated fine-tuning, and adversarial fine-tuning are adopted. [Results] According to our submission record (Jan. 2022), with our optimized pretraining and fine-tuning strategies, our 1.3 billion model sets new state-of-the-art on 4/9 tasks, achieving the best average score of 91.3. Encouragingly, our Vega v1 is the first to exceed powerful human performance on the two challenging tasks, i.e., SST-2 and WNLI. We believe our empirically successful recipe with a bag of tricks could shed new light on developing efficient discriminative large language models.


title: Follow-up survey for the binary black hole merger GW200224_222234 using Subaru/HSC and GTC/OSIRIS

abstract: The LIGO/Virgo detected a gravitational wave (GW) event, named GW200224_222234 (a.k.a. S200224ca) and classified as a binary-black-hole coalescence, on February 24, 2020. Given its relatively small localization skymap (71 deg$^2$ for a 90% credible region; revised to 50 deg$^2$ in GWTC-3), we performed target-of-opportunity observations using the Subaru/Hyper Suprime-Cam (HSC) in the $r2$- and $z$-bands. Observations were conducted on February 25 and 28 and March 23, 2020, with the first epoch beginning 12.3 h after the GW detection. The survey covered the highest probability sky area of 56.6 deg$^2$, corresponding to a 91% probability. This was the first deep follow-up ($m_{r}\gtrsim24, m_{z}\gtrsim23$) for a binary-black-hole merger covering $>$90% of the localization. By performing image subtraction and candidate screening including light curve fitting with transient templates and examples, we found 22 off-nucleus transients that were not ruled out as the counterparts of GW200224_222234 with only our Subaru/HSC data. We also performed GTC/OSIRIS spectroscopy of the probable host galaxies for five candidates; two are likely to be located within the 3D skymap, whereas the others are not. In conclusion, 19 transients remain as possible optical counterparts of GW200224_222234; however, we could not identify a unique promising counterpart. If there are no counterparts in the remaining candidates, the upper limits of optical luminosity are $\nu L_{\nu} < 5.2^{+2.4}_{-1.9}\times 10^{41}$ erg s$^{-1}$ and $\nu L_{\nu} < 1.8^{+0.8}_{-0.6}\times 10^{42}$ erg s$^{-1}$ in the $r2$- and $z$-bands, respectively, at $\sim$12 h after GW detection. We also discuss improvements in the strategies of optical follow-ups for future GW events.


title: Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements

abstract: As generative large model capabilities advance, safety concerns become more pronounced in their outputs. To ensure the sustainable growth of the AI ecosystem, it's imperative to undertake a holistic evaluation and refinement of associated safety risks. This survey presents a framework for safety research pertaining to large models, delineating the landscape of safety risks as well as safety evaluation and improvement methods. We begin by introducing safety issues of wide concern, then delve into safety evaluation methods for large models, encompassing preference-based testing, adversarial attack approaches, issues detection, and other advanced evaluation methods. Additionally, we explore the strategies for enhancing large model safety from training to deployment, highlighting cutting-edge safety approaches for each stage in building large models. Finally, we discuss the core challenges in advancing towards more responsible AI, including the interpretability of safety mechanisms, ongoing safety issues, and robustness against malicious attacks. Through this survey, we aim to provide clear technical guidance for safety researchers and encourage further study on the safety of large models.


title: Entangling dynamics from effective rotor/spin-wave separation in U(1)-symmetric quantum spin models

abstract: The non-equilibrium dynamics of quantum spin models is a most challenging topic, due to the exponentiality of Hilbert space; and it is central to the understanding of the many-body entangled states that can be generated by state-of-the-art quantum simulators. A particularly important class of evolutions is the one governed by U(1) symmetric Hamiltonians, initialized in a state which breaks the U(1) symmetry -- the paradigmatic example being the evolution of the so-called one-axis-twisting (OAT) model, featuring infinite-range interactions between spins. In this work we show that the dynamics of the OAT model can be closely reproduced by systems with power-law-decaying interactions, thanks to an effective separation between the zero-momentum degrees of freedom, associated with the so-called Anderson tower of states, and reconstructing a OAT model; and finite-momentum ones, associated with spin-wave excitations. This mechanism explains quantitatively the recent numerical observation of spin squeezing and Schr\"odinger-cat generation in the dynamics of dipolar Hamiltonians; and it paves the way for the extension of this observation to a much larger class of models of immediate relevance for quantum simulations.


title: Opening the path to hard X-/soft gamma-ray focussing: the ASTENA-pathfinder mission

abstract: Hard X-/soft gamma-ray astronomy is a crucial field for transient, nuclear and multimessenger astrophysics. However, the spatial localization, imaging capabilities and sensitivity of the measurements are strongly limited for the energy range > 70 keV. To overcome these limitations, we have proposed a mission concept, ASTENA, submitted to ESA for its program Voyage 2050. We will report on a pathfinder of ASTENA, that we intend to propose to ASI as an Italian mission with international participation. It will be based on one of the two instruments aboard ASTENA: a Laue lens with 20 m focal length, able to focus hard X-rays in the 50-700 keV passband into a 3-d position sensitive focal plane spectrometer. The combination of the focussing properties of the lens and of the localization properties of the detector will provide unparalleled imaging and spectroscopic capabilities, thus enabling studies of phenomena such as gamma-ray bursts afterglows, supernova explosions, positron annihilation lines and many more.


title: Reinforcement Learning in the Wild with Maximum Likelihood-based Model Transfer

abstract: In this paper, we study the problem of transferring the available Markov Decision Process (MDP) models to learn and plan efficiently in an unknown but similar MDP. We refer to it as \textit{Model Transfer Reinforcement Learning (MTRL)} problem. First, we formulate MTRL for discrete MDPs and Linear Quadratic Regulators (LQRs) with continuous state actions. Then, we propose a generic two-stage algorithm, MLEMTRL, to address the MTRL problem in discrete and continuous settings. In the first stage, MLEMTRL uses a \textit{constrained Maximum Likelihood Estimation (MLE)}-based approach to estimate the target MDP model using a set of known MDP models. In the second stage, using the estimated target MDP model, MLEMTRL deploys a model-based planning algorithm appropriate for the MDP class. Theoretically, we prove worst-case regret bounds for MLEMTRL both in realisable and non-realisable settings. We empirically demonstrate that MLEMTRL allows faster learning in new MDPs than learning from scratch and achieves near-optimal performance depending on the similarity of the available MDPs and the target MDP.


title: Low-Complexity Pareto-Optimal 3D Beamforming for the Full-Dimensional Multi-User Massive MIMO Downlink

abstract: Full-dimensional (FD) multi-user massive multiple input multiple output (m-MIMO) systems employ large two-dimensional (2D) rectangular antenna arrays to control both the azimuth and elevation angles of signal transmission. We introduce the sum of two outer products of the azimuth and elevation beamforming vectors having moderate dimensions as a new class of FD beamforming. We show that this low-complexity class is capable of outperforming 2D beamforming relying on the single outer product of the azimuth and elevation beamforming vectors. It is also capable of performing close to its FD counterpart of massive dimensions in terms of either the users minimum rate or their geometric mean rate (GM-rate), or sum rate (SR). Furthermore, we also show that even FD beamforming may be outperformed by our outer product-based improper Gaussian signaling solution. Explicitly, our design is based on low-complexity algorithms relying on convex problems of moderate dimensions for max-min rate optimization or on closed-form expressions for GM-rate and SR maximization.


title: Structural Neural Additive Models: Enhanced Interpretable Machine Learning

abstract: Deep neural networks (DNNs) have shown exceptional performances in a wide range of tasks and have become the go-to method for problems requiring high-level predictive power. There has been extensive research on how DNNs arrive at their decisions, however, the inherently uninterpretable networks remain up to this day mostly unobservable "black boxes". In recent years, the field has seen a push towards interpretable neural networks, such as the visually interpretable Neural Additive Models (NAMs). We propose a further step into the direction of intelligibility beyond the mere visualization of feature effects and propose Structural Neural Additive Models (SNAMs). A modeling framework that combines classical and clearly interpretable statistical methods with the predictive power of neural applications. Our experiments validate the predictive performances of SNAMs. The proposed framework performs comparable to state-of-the-art fully connected DNNs and we show that SNAMs can even outperform NAMs while remaining inherently more interpretable.


title: Transformer-Based Neural Marked Spatio Temporal Point Process Model for Football Match Events Analysis

abstract: With recently available football match event data that record the details of football matches, analysts and researchers have a great opportunity to develop new performance metrics, gain insight, and evaluate key performance. However, most sports sequential events modeling methods and performance metrics approaches could be incomprehensive in dealing with such large-scale spatiotemporal data (in particular, temporal process), thereby necessitating a more comprehensive spatiotemporal model and a holistic performance metric. To this end, we proposed the Transformer-Based Neural Marked Spatio Temporal Point Process (NMSTPP) model for football event data based on the neural temporal point processes (NTPP) framework. In the experiments, our model outperformed the prediction performance of the baseline models. Furthermore, we proposed the holistic possession utilization score (HPUS) metric for a more comprehensive football possession analysis. For verification, we examined the relationship with football teams' final ranking, average goal score, and average xG over a season. It was observed that the average HPUS showed significant correlations regardless of not using goal and details of shot information. Furthermore, we show HPUS examples in analyzing possessions, matches, and between matches.


title: Promoting Cooperation in Multi-Agent Reinforcement Learning via Mutual Help

abstract: Multi-agent reinforcement learning (MARL) has achieved great progress in cooperative tasks in recent years. However, in the local reward scheme, where only local rewards for each agent are given without global rewards shared by all the agents, traditional MARL algorithms lack sufficient consideration of agents' mutual influence. In cooperative tasks, agents' mutual influence is especially important since agents are supposed to coordinate to achieve better performance. In this paper, we propose a novel algorithm Mutual-Help-based MARL (MH-MARL) to instruct agents to help each other in order to promote cooperation. MH-MARL utilizes an expected action module to generate expected other agents' actions for each particular agent. Then, the expected actions are delivered to other agents for selective imitation during training. Experimental results show that MH-MARL improves the performance of MARL both in success rate and cumulative reward.