title: Fast Hyperparameter Tuning for Ising Machines

abstract: In this paper, we propose a novel technique to accelerate Ising machines hyperparameter tuning. Firstly, we define Ising machine performance and explain the goal of hyperparameter tuning in regard to this performance definition. Secondly, we compare well-known hyperparameter tuning techniques, namely random sampling and Tree-structured Parzen Estimator (TPE) on different combinatorial optimization problems. Thirdly, we propose a new convergence acceleration method for TPE which we call "FastConvergence".It aims at limiting the number of required TPE trials to reach best performing hyperparameter values combination. We compare FastConvergence to previously mentioned well-known hyperparameter tuning techniques to show its effectiveness. For experiments, well-known Travel Salesman Problem (TSP) and Quadratic Assignment Problem (QAP) instances are used as input. The Ising machine used is Fujitsu's third generation Digital Annealer (DA). Results show, in most cases, FastConvergence can reach similar results to TPE alone within less than half the number of trials.


title: Traveling wave solutions of the generalized scale-invariant analogue of the KdV equation by tanh-coth method

abstract: In this work, the generalized scale-invariant analogue of the Korteweg-de Vries (gsiaKdV) equation is studied. For the first time, the tanh-coth methodology is used to find traveling wave solutions for this nonlinear equation. The considered generalized equation is a connection between the well-known KdV equation and the recently investigated SIdV equation. The obtained results show many families of solutions for the model, indicating that this equation also shares bell-shaped solutions with KdV and SIdV, as previously documented by other researchers. Finally, by executing the symbolic computation, we demonstrate that the employed technique is a valuable and effective mathematical tool that can be used to solve problems that arise in the cross-disciplinary nonlinear sciences and applied mathematics.


title: Simultaneous Estimation of Hand Configurations and Finger Joint Angles using Forearm Ultrasound

abstract: With the advancement in computing and robotics, it is necessary to develop fluent and intuitive methods for interacting with digital systems, augmented/virtual reality (AR/VR) interfaces, and physical robotic systems. Hand motion recognition is widely used to enable these interactions. Hand configuration classification and MCP joint angle detection is important for a comprehensive reconstruction of hand motion. sEMG and other technologies have been used for the detection of hand motions. Forearm ultrasound images provide a musculoskeletal visualization that can be used to understand hand motion. Recent work has shown that these ultrasound images can be classified using machine learning to estimate discrete hand configurations. Estimating both hand configuration and MCP joint angles based on forearm ultrasound has not been addressed in the literature. In this paper, we propose a CNN based deep learning pipeline for predicting the MCP joint angles. The results for the hand configuration classification were compared by using different machine learning algorithms. SVC with different kernels, MLP, and the proposed CNN have been used to classify the ultrasound images into 11 hand configurations based on activities of daily living. Forearm ultrasound images were acquired from 6 subjects instructed to move their hands according to predefined hand configurations. Motion capture data was acquired to get the finger angles corresponding to the hand movements at different speeds. Average classification accuracy of 82.7% for the proposed CNN and over 80% for SVC for different kernels was observed on a subset of the dataset. An average RMSE of 7.35 degrees was obtained between the predicted and the true MCP joint angles. A low latency (6.25 - 9.1 Hz) pipeline has been proposed for estimating both MCP joint angles and hand configuration aimed at real-time control of human-machine interfaces.


title: Scrambling and quantum chaos indicators from long-time properties of operator distributions

abstract: Scrambling is a key concept in the analysis of nonequilibrium properties of quantum many-body systems. Most studies focus on its characterization via out-of-time-ordered correlation functions (OTOCs), particularly through the early-time decay of the OTOC. However, scrambling is a complex process which involves operator spreading and operator entanglement, and a full characterization requires one to access more refined information on the operator dynamics at several timescales. In this work we analyze operator scrambling by expanding the target operator in a complete basis and studying the structure of the expansion coefficients treated as a coarse-grained probability distribution in the space of operators. We study different features of this distribution, such as its mean, variance, and participation ratio, for the Ising model with longitudinal and transverse fields, kicked collective spin models, and random circuit models. We show that the long-time properties of the operator distribution display common features across these cases, and discuss how these properties can be used as a proxy for the onset of quantum chaos. Finally, we discuss the connection with OTOCs and analyze the cost of probing the operator distribution experimentally using these correlation functions.


title: Spin-2 dark matter from anisotropic Universe in bigravity

abstract: Bigravity is one of the natural extensions of general relativity and contains an additional massive spin-2 field which can be a good candidate for dark matter. To discuss the production of spin-2 dark matter, we study fixed point solutions of the background equations for axisymmetric Bianchi type-I Universes in two bigravity theories without Boulware-Deser ghost, i.e., Hassan-Rosen bigravity and Minimal Theory of Bigravity. We investigate the local and global stability of the fixed points and classify them. Based on the general analysis, we propose a new scenario where spin-2 dark matter is produced by the transition from an anisotropic fixed point solution to isotropic one. The produced spin-2 dark matter can account for all or a part of dark matter and can be directly detected by laser interferometers in the same way as gravitational waves.


title: Some congruences involving generalized Bernoulli numbers and Bernoulli polynomials

abstract: Let $[x]$ be the integral part of $x$, $n>1$ be a positive integer and $\chi_n$ denote the trivial Dirichlet character modulo $n$. In this paper, we use an identity established by Z. H. Sun to get congruences of $T_{m,k}(n)=\sum_{x=1}^{[n/m]}\frac{\chi_n(x)}{x^k}\left(\bmod n^{r+1}\right)$ for $r\in \{1,2\}$, any positive integer $m $ with $n \equiv \pm 1 \left(\bmod m \right)$ in terms of Bernoulli polynomials. As its an application, we also obtain some new congruences involving binomial coefficients modulo $n^4$ in terms of generalized Bernoulli numbers.


title: Data Poisoning Attack Aiming the Vulnerability of Continual Learning

abstract: Generally, regularization-based continual learning models limit access to the previous task data to imitate the real-world constraints related to memory and privacy. However, this introduces a problem in these models by not being able to track the performance on each task. In essence, current continual learning methods are susceptible to attacks on previous tasks. We demonstrate the vulnerability of regularization-based continual learning methods by presenting a simple task-specific data poisoning attack that can be used in the learning process of a new task. Training data generated by the proposed attack causes performance degradation on a specific task targeted by the attacker. We experiment with the attack on the two representative regularization-based continual learning methods, Elastic Weight Consolidation (EWC) and Synaptic Intelligence (SI), trained with variants of MNIST dataset. The experiment results justify the vulnerability proposed in this paper and demonstrate the importance of developing continual learning models that are robust to adversarial attacks.


title: Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instances

abstract: We consider the problem of embodied visual navigation given an image-goal (ImageNav) where an agent is initialized in an unfamiliar environment and tasked with navigating to a location 'described' by an image. Unlike related navigation tasks, ImageNav does not have a standardized task definition which makes comparison across methods difficult. Further, existing formulations have two problematic properties; (1) image-goals are sampled from random locations which can lead to ambiguity (e.g., looking at walls), and (2) image-goals match the camera specification and embodiment of the agent; this rigidity is limiting when considering user-driven downstream applications. We present the Instance-specific ImageNav task (InstanceImageNav) to address these limitations. Specifically, the goal image is 'focused' on some particular object instance in the scene and is taken with camera parameters independent of the agent. We instantiate InstanceImageNav in the Habitat Simulator using scenes from the Habitat-Matterport3D dataset (HM3D) and release a standardized benchmark to measure community progress.


title: Effective Utilisation of Multiple Open-Source Datasets to Improve Generalisation Performance of Point Cloud Segmentation Models

abstract: Semantic segmentation of aerial point cloud data can be utilised to differentiate which points belong to classes such as ground, buildings, or vegetation. Point clouds generated from aerial sensors mounted to drones or planes can utilise LIDAR sensors or cameras along with photogrammetry. Each method of data collection contains unique characteristics which can be learnt independently with state-of-the-art point cloud segmentation models. Utilising a single point cloud segmentation model can be desirable in situations where point cloud sensors, quality, and structures can change. In these situations it is desirable that the segmentation model can handle these variations with predictable and consistent results. Although deep learning can segment point clouds accurately it often suffers in generalisation, adapting poorly to data which is different than the training data. To address this issue, we propose to utilise multiple available open source fully annotated datasets to train and test models that are better able to generalise. In this paper we discuss the combination of these datasets into a simple training set and challenging test set. Combining datasets allows us to evaluate generalisation performance on known variations in the point cloud data. We show that a naive combination of datasets produces a model with improved generalisation performance as expected. We go on to show that an improved sampling strategy which decreases sampling variations increases the generalisation performance substantially on top of this. Experiments to find which sample variations give this performance boost found that consistent densities are the most important.


title: Holomorphic anomaly equations for C^5/Z_5

abstract: We prove holomorphic anomaly equations for C^5/Z_5 based on the work arXiv:1707.02910 of Lho.


title: A decomposition theorem of surface vector fields and spectral structure of the Neumann-Poincar\'e operator in elasticity

abstract: We prove that the space of vector fields on the boundary of a bounded domain with the Lipschitz boundary in three dimensions is decomposed into three subspaces: elements of the first one extend to the inside the domain as divergence-free and rotation-free vector fields, the second one to the outside as divergence-free and rotation-free vector fields, and the third one to both the inside and the outside as divergence-free harmonic vector fields. We then show that each subspace in the decomposition is infinite-dimensional. We also prove under a mild regularity assumption on the boundary that the decomposition is almost direct in the sense that any intersection of two subspaces is finite-dimensional. We actually prove that the dimension of intersection is bounded by the first Betti number of the boundary. In particular, if the boundary is simply connected, then the decomposition is direct. We apply this decomposition theorem to investigate spectral properties of the Neumann-Poincar\'e operator in elasticity, whose cubic polynomial is known to be compact. We prove that each linear factor of the cubic polynomial is compact on each subspace of decomposition separately and those subspaces characterize eigenspaces of the Neumann-Poincar\'e operator. We then prove all the results for three dimensions, decomposition of surface vector fields and spectral structure, are extended to higher dimensions. We also prove analogous but different results in two dimensions.


title: Mirror descent of Hopfield model

abstract: Mirror descent is an elegant optimization technique that leverages a dual space of parametric models to perform gradient descent. While originally developed for convex optimization, it has increasingly been applied in the field of machine learning. In this study, we propose a novel approach for utilizing mirror descent to initialize the parameters of neural networks. Specifically, we demonstrate that by using the Hopfield model as a prototype for neural networks, mirror descent can effectively train the model with significantly improved performance compared to traditional gradient descent methods that rely on random parameter initialization. Our findings highlight the potential of mirror descent as a promising initialization technique for enhancing the optimization of machine learning models.


title: Latest data constraint of some parameterized dark energy models

abstract: Using various latest cosmological datasets including Type-Ia supernovae, cosmic microwave background radiation, baryon acoustic oscillations, and estimations of the Hubble parameter, we test some dark energy models with parameterized equations of state and try to distinguish or select observation-preferred models. We obtain the best fitting results of the six models and calculate their values of the Akaike Information Criteria and Bayes Information Criterion. And we can distinguish these dark energy models from each other by using these two information criterions. However, the $\Lambda $CDM model remains the best fit model. Furthermore, we perform geometric diagnostics including statefinder and Om diagnostics to understand the geometric behaviour of the dark energy models. We find that the six DE models can be distinguished from each other and from $\Lambda$CDM, Chaplygin gas, quintessence models after the statefinder and Om diagnostics were performed. Finally, we consider the growth factor of the dark energy models with comparison to $\Lambda $CDM model. Still, we find the models can be distinguished from each other and from $\Lambda $CDM model through the growth factor approximation.


title: Spectral analysis for a class of bounded linear operators

abstract: We study the Spectral Analysis for a class of bounded linear operators T = D + F in a non Archimedean Hilbert space E, where D is a diagonal linear operator and where F is a finite rank linear operator. In this study of the Spectral Analysis, we use extensively the Theory of Fredholm Operators to deduce some of our main results.


title: Yukawa Friedel-Tail pair potentials for warm dense matter applications

abstract: Accurate equations of state (EOS) and plasma transport properties are essential for numerical simulations of warm dense matter encountered in many high-energy-density situations. Molecular dynamics (MD) is a simulation method that generates EOS and transport data using an externally provided potential to dynamically evolve the particles without further reference to the electrons. To minimize computational cost, pair potentials needed in MD may be obtained from the neutral-pseudoatom (NPA) approach, a form of single-ion density functional theory (DFT), where many-ion effects are included via ion-ion correlation functionals. Standard $N$-ion DFT-MD provides pair potentials via the force matching technique but at much greater computational cost. Here we propose a simple analytic model for pair potentials with physically meaningful parameters based on a Yukawa form with a thermally damped Friedel tail (YFT) applicable to systems containing free electrons. The YFT model accurately fits NPA pair potentials or the non-parametric force-matched potentials from $N$-ion DFT-MD, showing excellent agreement for a wide range of conditions. The YFT form provides accurate extrapolations of the NPA or force-matched potentials for small and large particle separations within a physical model. Our method can be adopted to treat plasma mixtures, allowing for large-scale simulations of multi-species warm dense matter.


title: Semiconducting Electronic Structure of the Ferromagnetic Spinel $\mathbf{Hg}\mathbf{Cr}_2\mathbf{Se}_4$ Revealed by Soft-X-Ray Angle-Resolved Photoemission Spectroscopy

abstract: We study the electronic structure of the ferromagnetic spinel $\mathrm{Hg}\mathrm{Cr}_2\mathrm{Se}_4$ by soft-x-ray angle-resolved photoemission spectroscopy (SX-ARPES) and first-principles calculations. While a theoretical study has predicted that this material is a magnetic Weyl semimetal, SX-ARPES measurements give direct evidence for a semiconducting state in the ferromagnetic phase. Band calculations based on the density functional theory with hybrid functionals reproduce the experimentally determined band gap value, and the calculated band dispersion matches well with ARPES experiments. We conclude that the theoretical prediction of a Weyl semimetal state in $\mathrm{Hg}\mathrm{Cr}_2\mathrm{Se}_4$ underestimates the band gap, and this material is a ferromagnetic semiconductor.


title: A Primer on Twists in the Noncommutative Realm Focusing on Algebra, Representation Theory, and Geometry

abstract: We review several techniques that twist an algebra's multiplicative structure. We first consider twists by an automorphism, also known as Zhang twists, and we relate them to 2-cocycle twists of certain bialgebras. We then outline the classification and properties of twisted tensor products, and we examine twisted Segre products. Our exposition emphasizes clarity over generality, providing a wealth of interconnecting examples.


title: Approximating Martingale Process for Variance Reduction in Deep Reinforcement Learning with Large State Space

abstract: Approximating Martingale Process (AMP) is proven to be effective for variance reduction in reinforcement learning (RL) in specific cases such as Multiclass Queueing Networks. However, in the already proven cases, the state space is relatively small and all possible state transitions can be iterated through. In this paper, we consider systems in which state space is large and have uncertainties when considering state transitions, thus making AMP a generalized variance-reduction method in RL. Specifically, we will investigate the application of AMP in ride-hailing systems like Uber, where Proximal Policy Optimization (PPO) is incorporated to optimize the policy of matching drivers and customers.


title: Global Carleman Estimate and State Observation Problem for Ginzburg-Landau Equation

abstract: In this paper, we prove a global Carleman estimate for the complex Ginzburg- Landau operator with a cubic nonlinear term in a bounded domain of Rn, n = 2, 3. As applications, we study state observation problems for the Ginzburg-Landau equation.


title: UQ-ARMED: Uncertainty quantification of adversarially-regularized mixed effects deep learning for clustered non-iid data

abstract: This work demonstrates the ability to produce readily interpretable statistical metrics for model fit, fixed effects covariance coefficients, and prediction confidence. Importantly, this work compares 4 suitable and commonly applied epistemic UQ approaches, BNN, SWAG, MC dropout, and ensemble approaches in their ability to calculate these statistical metrics for the ARMED MEDL models. In our experiment for AD prognosis, not only do the UQ methods provide these benefits, but several UQ methods maintain the high performance of the original ARMED method, some even provide a modest (but not statistically significant) performance improvement. The ensemble models, especially the ensemble method with a 90% subsampling, performed well across all metrics we tested with (1) high performance that was comparable to the non-UQ ARMED model, (2) properly deweights the confounds probes and assigns them statistically insignificant p-values, (3) attains relatively high calibration of the output prediction confidence. Based on the results, the ensemble approaches, especially with a subsampling of 90%, provided the best all-round performance for prediction and uncertainty estimation, and achieved our goals to provide statistical significance for model fit, statistical significance covariate coefficients, and confidence in prediction, while maintaining the baseline performance of MEDL using ARMED


title: Simultaneous Best Subset Selection and Dimension Reduction via Primal-Dual Iterations

abstract: Sparse reduced rank regression is an essential statistical learning method. In the contemporary literature, estimation is typically formulated as a nonconvex optimization that often yields to a local optimum in numerical computation. Yet, their theoretical analysis is always centered on the global optimum, resulting in a discrepancy between the statistical guarantee and the numerical computation. In this research, we offer a new algorithm to address the problem and establish an almost optimal rate for the algorithmic solution. We also demonstrate that the algorithm achieves the estimation with a polynomial number of iterations. In addition, we present a generalized information criterion to simultaneously ensure the consistency of support set recovery and rank estimation. Under the proposed criterion, we show that our algorithm can achieve the oracle reduced rank estimation with a significant probability. The numerical studies and an application in the ovarian cancer genetic data demonstrate the effectiveness and scalability of our approach.


title: On Robust Learning from Noisy Labels: A Permutation Layer Approach

abstract: The existence of label noise imposes significant challenges (e.g., poor generalization) on the training process of deep neural networks (DNN). As a remedy, this paper introduces a permutation layer learning approach termed PermLL to dynamically calibrate the training process of the DNN subject to instance-dependent and instance-independent label noise. The proposed method augments the architecture of a conventional DNN by an instance-dependent permutation layer. This layer is essentially a convex combination of permutation matrices that is dynamically calibrated for each sample. The primary objective of the permutation layer is to correct the loss of noisy samples mitigating the effect of label noise. We provide two variants of PermLL in this paper: one applies the permutation layer to the model's prediction, while the other applies it directly to the given noisy label. In addition, we provide a theoretical comparison between the two variants and show that previous methods can be seen as one of the variants. Finally, we validate PermLL experimentally and show that it achieves state-of-the-art performance on both real and synthetic datasets.


title: An Extreme-Adaptive Time Series Prediction Model Based on Probability-Enhanced LSTM Neural Networks

abstract: Forecasting time series with extreme events has been a challenging and prevalent research topic, especially when the time series data are affected by complicated uncertain factors, such as is the case in hydrologic prediction. Diverse traditional and deep learning models have been applied to discover the nonlinear relationships and recognize the complex patterns in these types of data. However, existing methods usually ignore the negative influence of imbalanced data, or severe events, on model training. Moreover, methods are usually evaluated on a small number of generally well-behaved time series, which does not show their ability to generalize. To tackle these issues, we propose a novel probability-enhanced neural network model, called NEC+, which concurrently learns extreme and normal prediction functions and a way to choose among them via selective back propagation. We evaluate the proposed model on the difficult 3-day ahead hourly water level prediction task applied to 9 reservoirs in California. Experimental results demonstrate that the proposed model significantly outperforms state-of-the-art baselines and exhibits superior generalization ability on data with diverse distributions.


title: Shape resonances in photoionization cross sections and time delay

abstract: Shape resonances in photoionization of atoms and molecules arise from a particular geometry of the ionic potential which traps the receding photoelectron in a quasi-bound state in a particular partial wave. This mechanism allows us to connect the photoionization cross section in the resonant region with the photoelectron scattering phase in this partial wave by a simple formula $\sigma \propto \sin^2\delta_\ell$. Due to this relation, the phase $\delta_\ell$ can be extracted from an experimentally known cross section and then converted to the photoelectron group delay (Wigner time delay) $\tau_{\rm W} = \partial \delta_\ell/\partial E$ which is measurable by recently developed laser interferometric techniques. Such a direct connection of the photoionization cross section and the time delay is a fundamental property of shape resonances which provides a comprehensive test of novel measurements against a large body of older synchrotron data.


title: Adap DP-FL: Differentially Private Federated Learning with Adaptive Noise

abstract: Federated learning seeks to address the issue of isolated data islands by making clients disclose only their local training models. However, it was demonstrated that private information could still be inferred by analyzing local model parameters, such as deep neural network model weights. Recently, differential privacy has been applied to federated learning to protect data privacy, but the noise added may degrade the learning performance much. Typically, in previous work, training parameters were clipped equally and noises were added uniformly. The heterogeneity and convergence of training parameters were simply not considered. In this paper, we propose a differentially private scheme for federated learning with adaptive noise (Adap DP-FL). Specifically, due to the gradient heterogeneity, we conduct adaptive gradient clipping for different clients and different rounds; due to the gradient convergence, we add decreasing noises accordingly. Extensive experiments on real-world datasets demonstrate that our Adap DP-FL outperforms previous methods significantly.


title: HashEncoding: Autoencoding with Multiscale Coordinate Hashing

abstract: We present HashEncoding, a novel autoencoding architecture that leverages a non-parametric multiscale coordinate hash function to facilitate a per-pixel decoder without convolutions. By leveraging the space-folding behaviour of hashing functions, HashEncoding allows for an inherently multiscale embedding space that remains much smaller than the original image. As a result, the decoder requires very few parameters compared with decoders in traditional autoencoders, approaching a non-parametric reconstruction of the original image and allowing for greater generalizability. Finally, by allowing backpropagation directly to the coordinate space, we show that HashEncoding can be exploited for geometric tasks such as optical flow.


title: Composition based oxidation state prediction of materials using deep learning

abstract: Oxidation states are the charges of atoms after their ionic approximation of their bonds, which have been widely used in charge-neutrality verification, crystal structure determination, and reaction estimation. Currently only heuristic rules exist for guessing the oxidation states of a given compound with many exceptions. Recent work has developed machine learning models based on heuristic structural features for predicting the oxidation states of metal ions. However, composition based oxidation state prediction still remains elusive so far, which is more important in new material discovery for which the structures are not even available. This work proposes a novel deep learning based BERT transformer language model BERTOS for predicting the oxidation states of all elements of inorganic compounds given only their chemical composition. Our model achieves 96.82\% accuracy for all-element oxidation states prediction benchmarked on the cleaned ICSD dataset and achieves 97.61\% accuracy for oxide materials. We also demonstrate how it can be used to conduct large-scale screening of hypothetical material compositions for materials discovery.


title: Tripling energy storage density through order-disorder transition induced polar nanoregions in PbZrO3 thin films by ion implantation

abstract: Dielectric capacitors are widely used in pulsed power electronic devices due to their ultrahigh power densities and extremely fast charge/discharge speed. To achieve enhanced energy storage density, both maximum polarization (Pmax) and breakdown strength (Eb) need to be improved simultaneously. However, these two key parameters are inversely correlated. In this study, order-disorder transition induced polar nanoregions (PNRs) have been achieved in PbZrO3 thin films by making use of the low-energy ion implantation, enabling us overcome the trade-off between high polarizability and breakdown strength, which leads to the tripling of the energy storage density from 20.5 J/cm3 to 62.3 J/cm3 as well as the great enhancement of breakdown strength. This approach could be extended to other dielectric oxides to improve the energy storage performance, providing a new pathway for tailoring the oxide functionalities.


title: Learning Antidote Data to Individual Unfairness

abstract: Fairness is essential for machine learning systems deployed in high-stake applications. Among all fairness notions, individual fairness, deriving from a consensus that `similar individuals should be treated similarly,' is a vital notion to describe fair treatment for individual cases. Previous studies typically characterize individual fairness as a prediction-invariant problem when perturbing sensitive attributes on samples, and solve it by Distributionally Robust Optimization (DRO) paradigm. However, such adversarial perturbations along a direction covering sensitive information used in DRO do not consider the inherent feature correlations or innate data constraints, therefore could mislead the model to optimize at off-manifold and unrealistic samples. In light of this drawback, in this paper, we propose to learn and generate antidote data that approximately follows the data distribution to remedy individual unfairness. These generated on-manifold antidote data can be used through a generic optimization procedure along with original training data, resulting in a pure pre-processing approach to individual unfairness, or can also fit well with the in-processing DRO paradigm. Through extensive experiments on multiple tabular datasets, we demonstrate our method resists individual unfairness at a minimal or zero cost to predictive utility compared to baselines.


title: Resonant Mode Coupling in $\delta$ Scuti Stars

abstract: Delta Scuti ($\delta$ Sct) variables are intermediate mass stars that lie at the intersection of the main sequence and the instability strip on the Hertzsprung-Russel diagram. Various lines of evidence indicate that nonlinear mode interactions shape their oscillation spectra, including the particularly compelling detection of resonantly interacting mode triplets in the $\delta$ Sct star KIC 8054146. Motivated by these observations, we use the theory of three-mode coupling to study the strength and prevalence of nonlinear mode interactions in fourteen $\delta$ Sct models that span the instability strip. For each model, we calculate the frequency detunings and nonlinear coupling strengths of $\sim 10^4$ unique combinations of mode triplets. We find that all the models contain at least $\sim 100$ well-coupled triplets whose detunings and coupling strengths are consistent with the triplets identified in KIC 8054146. Our results suggest that resonant mode interactions can be significant in $\delta$ Sct stars and may explain why many exhibit rapid changes in amplitude and oscillation period.


title: FakeEdge: Alleviate Dataset Shift in Link Prediction

abstract: Link prediction is a crucial problem in graph-structured data. Due to the recent success of graph neural networks (GNNs), a variety of GNN-based models were proposed to tackle the link prediction task. Specifically, GNNs leverage the message passing paradigm to obtain node representation, which relies on link connectivity. However, in a link prediction task, links in the training set are always present while ones in the testing set are not yet formed, resulting in a discrepancy of the connectivity pattern and bias of the learned representation. It leads to a problem of dataset shift which degrades the model performance. In this paper, we first identify the dataset shift problem in the link prediction task and provide theoretical analyses on how existing link prediction methods are vulnerable to it. We then propose FakeEdge, a model-agnostic technique, to address the problem by mitigating the graph topological gap between training and testing sets. Extensive experiments demonstrate the applicability and superiority of FakeEdge on multiple datasets across various domains.


title: Towards More Robust Interpretation via Local Gradient Alignment

abstract: Neural network interpretation methods, particularly feature attribution methods, are known to be fragile with respect to adversarial input perturbations. To address this, several methods for enhancing the local smoothness of the gradient while training have been proposed for attaining \textit{robust} feature attributions. However, the lack of considering the normalization of the attributions, which is essential in their visualizations, has been an obstacle to understanding and improving the robustness of feature attribution methods. In this paper, we provide new insights by taking such normalization into account. First, we show that for every non-negative homogeneous neural network, a naive $\ell_2$-robust criterion for gradients is \textit{not} normalization invariant, which means that two functions with the same normalized gradient can have different values. Second, we formulate a normalization invariant cosine distance-based criterion and derive its upper bound, which gives insight for why simply minimizing the Hessian norm at the input, as has been done in previous work, is not sufficient for attaining robust feature attribution. Finally, we propose to combine both $\ell_2$ and cosine distance-based criteria as regularization terms to leverage the advantages of both in aligning the local gradient. As a result, we experimentally show that models trained with our method produce much more robust interpretations on CIFAR-10 and ImageNet-100 without significantly hurting the accuracy, compared to the recent baselines. To the best of our knowledge, this is the first work to verify the robustness of interpretation on a larger-scale dataset beyond CIFAR-10, thanks to the computational efficiency of our method.


title: Multi-robot Social-aware Cooperative Planning in Pedestrian Environments Using Multi-agent Reinforcement Learning

abstract: Safe and efficient co-planning of multiple robots in pedestrian participation environments is promising for applications. In this work, a novel multi-robot social-aware efficient cooperative planner that on the basis of off-policy multi-agent reinforcement learning (MARL) under partial dimension-varying observation and imperfect perception conditions is proposed. We adopt temporal-spatial graph (TSG)-based social encoder to better extract the importance of social relation between each robot and the pedestrians in its field of view (FOV). Also, we introduce K-step lookahead reward setting in multi-robot RL framework to avoid aggressive, intrusive, short-sighted, and unnatural motion decisions generated by robots. Moreover, we improve the traditional centralized critic network with multi-head global attention module to better aggregates local observation information among different robots to guide the process of individual policy update. Finally, multi-group experimental results verify the effectiveness of the proposed cooperative motion planner.


title: Simultaneous Spatial and Temporal Assignment for Fast UAV Trajectory Optimization using Bilevel Optimization

abstract: In this paper, we propose a framework for fast trajectory planning for unmanned aerial vehicles (UAVs). Our framework is reformulated from an existing bilevel optimization, in which the lower-level problem solves for the optimal trajectory with a fixed time allocation, whereas the upper-level problem updates the time allocation using analytical gradients. The lower-level problem incorporates the safety-set constraints (in the form of inequality constraints) and is cast as a convex quadratic program (QP). Our formulation modifies the lower-level QP by excluding the inequality constraints for the safety sets, which significantly reduces the computation time. The safety-set constraints are moved to the upper-level problem, where the feasible waypoints are updated together with the time allocation using analytical gradients enabled by the OptNet. We validate our approach in simulations, where our method's computation time scales linearly with respect to the number of safety sets, in contrast to the state-of-the-art that scales exponentially.


title: Equivalence Between SE(3) Equivariant Networks via Steerable Kernels and Group Convolution

abstract: A wide range of techniques have been proposed in recent years for designing neural networks for 3D data that are equivariant under rotation and translation of the input. Most approaches for equivariance under the Euclidean group $\mathrm{SE}(3)$ of rotations and translations fall within one of the two major categories. The first category consists of methods that use $\mathrm{SE}(3)$-convolution which generalizes classical $\mathbb{R}^3$-convolution on signals over $\mathrm{SE}(3)$. Alternatively, it is possible to use \textit{steerable convolution} which achieves $\mathrm{SE}(3)$-equivariance by imposing constraints on $\mathbb{R}^3$-convolution of tensor fields. It is known by specialists in the field that the two approaches are equivalent, with steerable convolution being the Fourier transform of $\mathrm{SE}(3)$ convolution. Unfortunately, these results are not widely known and moreover the exact relations between deep learning architectures built upon these two approaches have not been precisely described in the literature on equivariant deep learning. In this work we provide an in-depth analysis of both methods and their equivalence and relate the two constructions to multiview convolutional networks. Furthermore, we provide theoretical justifications of separability of $\mathrm{SE}(3)$ group convolution, which explain the applicability and success of some recent approaches. Finally, we express different methods using a single coherent formalism and provide explicit formulas that relate the kernels learned by different methods. In this way, our work helps to unify different previously-proposed techniques for achieving roto-translational equivariance, and helps to shed light on both the utility and precise differences between various alternatives. We also derive new TFN non-linearities from our equivalence principle and test them on practical benchmark datasets.


title: Graceful Coloring of Ladder Graphs

abstract: A graceful k-coloring of a non-empty graph $G=(V,E)$ is a proper vertex coloring $f:V(G)\rightarrow\lbrace 1,2,...,k \rbrace$, $k\geq 2$, which induces a proper edge coloring $f^{*}:E(G)\rightarrow\lbrace 1, 2, . . . , k-1 \rbrace $ defined by $f^{*}(uv) = |f(u)-f(v)|$, where $u,v\in V(G)$. The minimum $k$ for which $G$ has a graceful $k$-coloring is called graceful chromatic number, $\chi_{g}(G)$. The graceful chromatic number for a few variants of ladder graphs are investigated in this article.


title: Precision studies for string derived $Z'$ dynamics at the LHC

abstract: We consider $Z'$s in heterotic string derived models and study $Z'$ resonant production at the TeV scale at the Large Hadron Collider (LHC). We use various kinematic differential distributions for the Drell-Yan process at NNLO in QCD to explore the parameter space of such models and investigate $Z'$ couplings. In particular, we study the impact of $Z$-$Z'$ kinetic-mixing interactions on forward-backward asymmetry ($A_{FB}$) and other distributions at the LHC.


title: Modeling Reconstructed Images of Jets Launched by SANE Super-Eddington Accretion Flows Around SMBHs with the ngEHT

abstract: Tidal disruption events (TDEs) around super massive black holes (SMBHs) are a potential laboratory to study super-Eddington accretion disks and sometimes result in powerful jets or outflows which may shine in the radio and sub millimeter bands. In this work, we model the thermal synchrotron emission of jets from general relativistic radiation magneto-hydrodynamics (GRRMHD) simulations of a BH accretion disk/jet system which assumes the TDE resulted in a magnetized accretion disk around a BH accreting at $\sim 12-25$ times the Eddington accretion rate. Through synthetic observations with the Next Generation Event Horizon Telescope (ngEHT) and an image reconstruction analysis, we demonstrate that TDE jets may provide compelling targets, within the context of the models explored in this work. In particular, we find that jets launched by a SANE super-Eddington disk around a spin $a_*=0.9$ reach the ngEHT detection threshold at large distances (up to 100 Mpc in this work). A two-temperature plasma in the jet or weaker jets, such as a spin $a_*=0$ model, requires a much closer distance as we demonstrate detection at 10 Mpc for limiting cases of $a_*=0,\,\mathcal{R}=1$ or $a_*=0.9,\, \mathcal{R}=20$. We also demonstrate that TDE jets may appear as superluminal sources if the BH is rapidly rotating and the jet is viewed nearly face on.


title: Performance Evaluation, Optimization and Dynamic Decision in Blockchain Systems: A Recent Overview

abstract: With rapid development of blockchain technology as well as integration of various application areas, performance evaluation, performance optimization, and dynamic decision in blockchain systems are playing an increasingly important role in developing new blockchain technology. This paper provides a recent systematic overview of this class of research, and especially, developing mathematical modeling and basic theory of blockchain systems. Important examples include (a) performance evaluation: Markov processes, queuing theory, Markov reward processes, random walks, fluid and diffusion approximations, and martingale theory; (b) performance optimization: Linear programming, nonlinear programming, integer programming, and multi-objective programming; (c) optimal control and dynamic decision: Markov decision processes, and stochastic optimal control; and (d) artificial intelligence: Machine learning, deep reinforcement learning, and federated learning. So far, a little research has focused on these research lines. We believe that the basic theory with mathematical methods, algorithms and simulations of blockchain systems discussed in this paper will strongly support future development and continuous innovation of blockchain technology.


title: Satellite Constellation Avoidance with the Rubin Observatory Legacy Survey of Space and Time

abstract: We investigate a novel satellite avoidance strategy to mitigate the impact of large commercial satellite constellations in low-Earth orbit on the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST). We simulate the orbits of currently planned Starlink and OneWeb constellations ($\sim$40,000 satellites) to test how effectively an upgraded Rubin scheduler algorithm can avoid them, and assess how the overall survey is affected. Given a reasonably accurate satellite orbit forecast, we find it is possible to adjust the scheduler algorithm to effectively avoid some satellites. Overall, sacrificing 10% of LSST observing time to avoid satellites reduces the fraction of LSST visits with streaks by a factor of two. Whether such a mitigation will be required depends on the overall impact of streaks on science, which is not yet well quantified. This is due to a lack of adequate information about satellite brightness distributions as well as the impact of glints and low surface brightness residuals on alert purity and systematic errors in cosmological parameter estimation. A significant increase in the number of satellites or their brightness during Rubin Operations may make implementing this satellite avoidance strategy worthwhile.


title: Investigating pre-eruptive magnetic properties at the footprints of erupting magnetic flux ropes

abstract: It is well established that solar eruptions are powered by free magnetic energy stored in current-carrying magnetic field in the corona. It has also been generally accepted that magnetic flux ropes (MFRs) are a critical component of many coronal mass ejections (CMEs). What remains controversial is whether MFRs are present well before the eruption. Our aim is to identify progenitors of MFRs, and investigate pre-eruptive magnetic properties associated with these progenitors. Here we analyze 28 MFRs erupting within 45 deg from the disk center from 2010 to 2015. All MFRs'feet are well identified by conjugate coronal dimmings. We then calculate magnetic properties at the feet of the MFRs, prior to their eruptions, using Helioseismic and Magnetic Imager (HMI) vector magnetograms. Our results show that only 8 erupting MFRs are associated with significant non-neutralized electric currents, 4 of which also exhibit pre-eruptive dimmings at the foot-prints. Twist and current distributions are asymmetric at the two feet of these MFRs. The presence of pre-eruption dimmings associated with non-neutralized currents suggests the pre-existing MFRs. Furthermore, evolution of conjugate dimmings and electric currents within the foot-prints can provide clues about the internal structure of MFRs and their formation mechanism.


title: Low-overhead Beam Training Scheme for Extremely Large-Scale RIS in Near-field

abstract: Extremely large-scale reconfigurable intelligent surface (XL-RIS) has recently been proposed and is recognized as a promising technology that can further enhance the capacity of communication systems and compensate for severe path loss . However, the pilot overhead of beam training in XL-RIS-assisted wireless communication systems is enormous because the near-field channel model needs to be taken into account, and the number of candidate codewords in the codebook increases dramatically accordingly. To tackle this problem, we propose two deep learning-based near-field beam training schemes in XL-RIS-assisted communication systems, where deep residual networks are employed to determine the optimal near-field RIS codeword. Specifically, we first propose a far-field beam-based beam training (FBT) scheme in which the received signals of all far-field RIS codewords are fed into the neural network to estimate the optimal near-field RIS codeword. In order to further reduce the pilot overhead, a partial near-field beam-based beam training (PNBT) scheme is proposed, where only the received signals corresponding to the partial near-field XL-RIS codewords are served as input to the neural network. Moreover, we further propose an improved PNBT scheme to enhance the performance of beam training by fully exploring the neural network's output. Finally, simulation results show that the proposed schemes outperform the existing beam training schemes and can reduce the beam sweeping overhead by approximately 95%.


title: Space of one cycles and coniveau filtrations

abstract: We prove a structural result about the space of one cycles of a separably rationally connected variety or a separably rationally connected fibration over a curve, either as a topological group or as an h-sheaf. This has the following consequences: a proof that the strong coniveau filtration agrees with the coniveau filtration on degree 3 homology, and a result on the integral Tate conjecture for homologically trivial one cycles.


title: Optimizing Stock Option Forecasting with the Assembly of Machine Learning Models and Improved Trading Strategies

abstract: This paper introduced key aspects of applying Machine Learning (ML) models, improved trading strategies, and the Quasi-Reversibility Method (QRM) to optimize stock option forecasting and trading results. It presented the findings of the follow-up project of the research "Application of Convolutional Neural Networks with Quasi-Reversibility Method Results for Option Forecasting". First, the project included an application of Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) networks to provide a novel way of predicting stock option trends. Additionally, it examined the dependence of the ML models by evaluating the experimental method of combining multiple ML models to improve prediction results and decision-making. Lastly, two improved trading strategies and simulated investing results were presented. The Binomial Asset Pricing Model with discrete time stochastic process analysis and portfolio hedging was applied and suggested an optimized investment expectation. These results can be utilized in real-life trading strategies to optimize stock option investment results based on historical data.


title: Branch-Well-Structured Transition Systems and Extensions

abstract: We propose a relaxation to the definition of well-structured transition systems (\WSTS) while retaining the decidability of boundedness and non-termination. In this class, the well-quasi-ordered (wqo) condition is relaxed such that it is applicable only between states that are reachable one from another. Furthermore, the monotony condition is relaxed in the same way. While this retains the decidability of non-termination and boundedness, it appears that the coverability problem is undecidable. To this end, we define a new notion of monotony, called cover-monotony, which is strictly more general than the usual monotony and still allows us to decide a restricted form of the coverability problem.


title: Prompted Opinion Summarization with GPT-3.5

abstract: Large language models have shown impressive performance across a wide variety of tasks, including text summarization. In this paper, we show that this strong performance extends to opinion summarization. We explore several pipeline methods for applying GPT-3.5 to summarize a large collection of user reviews in a prompted fashion. To handle arbitrarily large numbers of user reviews, we explore recursive summarization as well as methods for selecting salient content to summarize through supervised clustering or extraction. On two datasets, an aspect-oriented summarization dataset of hotel reviews (SPACE) and a generic summarization dataset of Amazon and Yelp reviews (FewSum), we show that GPT-3.5 models achieve very strong performance in human evaluation. We argue that standard evaluation metrics do not reflect this, and introduce three new metrics targeting faithfulness, factuality, and genericity to contrast these different methods.


title: Local-global principle and integral Tate conjecture for certain varieties

abstract: We give a geometric criterion to check the validity of integral Tate conjecture for one cycles on separably rationally connected fibrations over a curve, and to check that the Brauer-Manin obstruction is the only obstruction to local-global principle for zero cycles on a separably rationally connected variety defined over a global function field. We prove that the Brauer-Manin obstruction is the only obstruction to local-global principle for zero cycles on all geometrically rational surfaces defined over a global function field, and to Hasse principle for rational points on del Pezzo surfaces of degree four defined over a global function field of odd characteristic. Along the way, we also prove some results about the space of one cycles on a separably rationally connected fibration over a curve, which leads to the equality of the coniveau filtration and the strong coniveau filtration (introduced by Benoist-Ottem and Voisin) on degree $3$ homology of such varieties.


title: BotSIM: An End-to-End Bot Simulation Toolkit for Commercial Task-Oriented Dialog Systems

abstract: We introduce BotSIM, a modular, open-source Bot SIMulation environment with dialog generation, user simulation and conversation analytics capabilities. BotSIM aims to serve as a one-stop solution for large-scale data-efficient end-to-end evaluation, diagnosis and remediation of commercial task-oriented dialog (TOD) systems to significantly accelerate commercial bot development and evaluation, reduce cost and time-to-market. BotSIM adopts a layered design comprising the infrastructure layer, the adaptor layer and the application layer. The infrastructure layer hosts key models and components to support BotSIM's major functionalities via a streamlined "generation-simulation-remediation" pipeline. The adaptor layer is used to extend BotSIM to accommodate new bot platforms. The application layer provides a suite of command line tools and a Web App to significantly lower the entry barrier for BotSIM users such as bot admins or practitioners. In this report, we focus on the technical designs of various system components. A detailed case study using Einstein BotBuilder is also presented to show how to apply BotSIM pipeline for bot evaluation and remediation. The detailed system descriptions can be found in our system demo paper. The toolkit is available at: https://github.com/salesforce/BotSIM .


title: On the geometry of an order unit space

abstract: We introduce the notion of $\mathit{skeleton}$ with a head in a non-zero real vector space. We prove that skeletons with heads describe order unit spaces geometrically. Next, we consider the notion of $\mathit{periphery}$ corresponding to an order unit space which is a part of the skeleton. We note that periphery consists of boundary elements of the positive cone with unit norms. We discuss some elementary properties of the periphery. We also find a condition under which $V$ would contain a copy of $\ell_{\infty}^n$ for some $n \in \mathbb{N}$ as an order unit subspace.


title: Similarity Distribution based Membership Inference Attack on Person Re-identification

abstract: While person Re-identification (Re-ID) has progressed rapidly due to its wide real-world applications, it also causes severe risks of leaking personal information from training data. Thus, this paper focuses on quantifying this risk by membership inference (MI) attack. Most of the existing MI attack algorithms focus on classification models, while Re-ID follows a totally different training and inference paradigm. Re-ID is a fine-grained recognition task with complex feature embedding, and model outputs commonly used by existing MI like logits and losses are not accessible during inference. Since Re-ID focuses on modelling the relative relationship between image pairs instead of individual semantics, we conduct a formal and empirical analysis which validates that the distribution shift of the inter-sample similarity between training and test set is a critical criterion for Re-ID membership inference. As a result, we propose a novel membership inference attack method based on the inter-sample similarity distribution. Specifically, a set of anchor images are sampled to represent the similarity distribution conditioned on a target image, and a neural network with a novel anchor selection module is proposed to predict the membership of the target image. Our experiments validate the effectiveness of the proposed approach on both the Re-ID task and conventional classification task.


title: Visser frames for sublogics of $\mathbf{IL}$

abstract: We study the modal completeness and the finite frame property of several sublogics of the logic $\mathbf{IL}$ of interpretability with respect to Visser frames, which are also called simplified Veltman frames. Among other things, we prove that the logic $\mathbf{CL}$ of conservativity has the finite frame property with respect to that frames. This is an affirmative solution to Ignatiev's problem.


title: Discrete Control in Real-World Driving Environments using Deep Reinforcement Learning

abstract: Training self-driving cars is often challenging since they require a vast amount of labeled data in multiple real-world contexts, which is computationally and memory intensive. Researchers often resort to driving simulators to train the agent and transfer the knowledge to a real-world setting. Since simulators lack realistic behavior, these methods are quite inefficient. To address this issue, we introduce a framework (perception, planning, and control) in a real-world driving environment that transfers the real-world environments into gaming environments by setting up a reliable Markov Decision Process (MDP). We propose variations of existing Reinforcement Learning (RL) algorithms in a multi-agent setting to learn and execute the discrete control in real-world environments. Experiments show that the multi-agent setting outperforms the single-agent setting in all the scenarios. We also propose reliable initialization, data augmentation, and training techniques that enable the agents to learn and generalize to navigate in a real-world environment with minimal input video data, and with minimal training. Additionally, to show the efficacy of our proposed algorithm, we deploy our method in the virtual driving environment TORCS.


title: Realization and Topological Properties of Third-Order Exceptional Lines Embedded in Exceptional Surfaces

abstract: As the counterpart of Hermitian nodal structures, the geometry formed by exceptional points (EPs), such as exceptional lines (ELs), entails intriguing spectral topology. We report the experimental realization of order-3 exceptional lines (EL3) that are entirely embedded in order-2 exceptional surfaces (ES2) in a three-dimensional periodic synthetic momentum space. The EL3 and the concomitant ES2, together with the topology of the underlying space, prohibit the evaluation of their topology in the eigenvalue manifold by prevailing topological characterization methods. We resolve this issue by defining a winding number that associates with the resultants of the Hamiltonian. This resultant winding number detects EL3 but ignores the ES2, allowing the diagnosis of the topological currents carried by the EL3, which enables the prediction of their evolution under perturbations. Our results exemplify unprecedented topology of higher-order exceptional geometries and may inspire new non-Hermitian topological applications.


title: The sheaf representation of residuated lattices

abstract: The residuated lattices form one of the most important algebras of fuzzy logics and have been heavily studied by people from various different points of view. Sheaf presentations provide a topological approach to many algebraic structures. In this paper, we study the topological properties of prime spectrum of residuated lattices, and then construct a sheaf space to obtain a sheaf representation for each residuated lattice.


title: Frequency Domain Gaussian Process Models for $H^\infty$ Uncertainties

abstract: Complex-valued Gaussian processes are used in Bayesian frequency-domain system identification as prior models for regression. If each realization of such a process were an $H_\infty$ function with probability one, then the same model could be used for probabilistic robust control, allowing for robustly safe learning. We investigate sufficient conditions for a general complex-domain Gaussian process to have this property. For the special case of processes whose Hermitian covariance is stationary, we provide an explicit parameterization of the covariance structure in terms of a summable sequence of nonnegative numbers.


title: Weakly Supervised Learning Significantly Reduces the Number of Labels Required for Intracranial Hemorrhage Detection on Head CT

abstract: Modern machine learning pipelines, in particular those based on deep learning (DL) models, require large amounts of labeled data. For classification problems, the most common learning paradigm consists of presenting labeled examples during training, thus providing strong supervision on what constitutes positive and negative samples. This constitutes a major obstacle for the development of DL models in radiology--in particular for cross-sectional imaging (e.g., computed tomography [CT] scans)--where labels must come from manual annotations by expert radiologists at the image or slice-level. These differ from examination-level annotations, which are coarser but cheaper, and could be extracted from radiology reports using natural language processing techniques. This work studies the question of what kind of labels should be collected for the problem of intracranial hemorrhage detection in brain CT. We investigate whether image-level annotations should be preferred to examination-level ones. By framing this task as a multiple instance learning problem, and employing modern attention-based DL architectures, we analyze the degree to which different levels of supervision improve detection performance. We find that strong supervision (i.e., learning with local image-level annotations) and weak supervision (i.e., learning with only global examination-level labels) achieve comparable performance in examination-level hemorrhage detection (the task of selecting the images in an examination that show signs of hemorrhage) as well as in image-level hemorrhage detection (highlighting those signs within the selected images). Furthermore, we study this behavior as a function of the number of labels available during training. Our results suggest that local labels may not be necessary at all for these tasks, drastically reducing the time and cost involved in collecting and curating datasets.


title: Device Modeling Bias in ReRAM-based Neural Network Simulations

abstract: Data-driven modeling approaches such as jump tables are promising techniques to model populations of resistive random-access memory (ReRAM) or other emerging memory devices for hardware neural network simulations. As these tables rely on data interpolation, this work explores the open questions about their fidelity in relation to the stochastic device behavior they model. We study how various jump table device models impact the attained network performance estimates, a concept we define as modeling bias. Two methods of jump table device modeling, binning and Optuna-optimized binning, are explored using synthetic data with known distributions for benchmarking purposes, as well as experimental data obtained from TiOx ReRAM devices. Results on a multi-layer perceptron trained on MNIST show that device models based on binning can behave unpredictably particularly at low number of points in the device dataset, sometimes over-promising, sometimes under-promising target network accuracy. This paper also proposes device level metrics that indicate similar trends with the modeling bias metric at the network level. The proposed approach opens the possibility for future investigations into statistical device models with better performance, as well as experimentally verified modeling bias in different in-memory computing and neural network architectures.


title: Interpretations Cannot Be Trusted: Stealthy and Effective Adversarial Perturbations against Interpretable Deep Learning

abstract: Deep learning methods have gained increased attention in various applications due to their outstanding performance. For exploring how this high performance relates to the proper use of data artifacts and the accurate problem formulation of a given task, interpretation models have become a crucial component in developing deep learning-based systems. Interpretation models enable the understanding of the inner workings of deep learning models and offer a sense of security in detecting the misuse of artifacts in the input data. Similar to prediction models, interpretation models are also susceptible to adversarial inputs. This work introduces two attacks, AdvEdge and AdvEdge$^{+}$, that deceive both the target deep learning model and the coupled interpretation model. We assess the effectiveness of proposed attacks against two deep learning model architectures coupled with four interpretation models that represent different categories of interpretation models. Our experiments include the attack implementation using various attack frameworks. We also explore the potential countermeasures against such attacks. Our analysis shows the effectiveness of our attacks in terms of deceiving the deep learning models and their interpreters, and highlights insights to improve and circumvent the attacks.


title: Compressing Cross-Lingual Multi-Task Models at Qualtrics

abstract: Experience management is an emerging business area where organizations focus on understanding the feedback of customers and employees in order to improve their end-to-end experiences. This results in a unique set of machine learning problems to help understand how people feel, discover issues they care about, and find which actions need to be taken on data that are different in content and distribution from traditional NLP domains. In this paper, we present a case study of building text analysis applications that perform multiple classification tasks efficiently in 12 languages in the nascent business area of experience management. In order to scale up modern ML methods on experience data, we leverage cross lingual and multi-task modeling techniques to consolidate our models into a single deployment to avoid overhead. We also make use of model compression and model distillation to reduce overall inference latency and hardware cost to the level acceptable for business needs while maintaining model prediction quality. Our findings show that multi-task modeling improves task performance for a subset of experience management tasks in both XLM-R and mBert architectures. Among the compressed architectures we explored, we found that MiniLM achieved the best compression/performance tradeoff. Our case study demonstrates a speedup of up to 15.61x with 2.60% average task degradation (or 3.29x speedup with 1.71% degradation) and estimated savings of 44% over using the original full-size model. These results demonstrate a successful scaling up of text classification for the challenging new area of ML for experience management.


title: Evidence for a Massive Andromeda Galaxy Using Satellite Galaxy Proper Motions

abstract: We present new mass estimates for Andromeda (M31) using the orbital angular momenta of four satellite galaxies (M33, NGC 185, NGC 147, IC 10) derived from existing proper motions, distances, and line-of-sight velocities. We infer two masses for M31: $M_{\rm vir}= 2.85^{+1.47}_{-0.77}\times10^{12}\, M_{\odot}$ using satellite galaxy phase space information derived with HST-based M31 proper motions and $M_{\rm vir}=3.02^{+1.30}_{-0.69}\times10^{12}\, M_{\odot}$ using phase space information derived with the weighted average of HST+Gaia-based M31 proper motions. The precision of our new M31 mass estimates (23-50%) improves by a factor of two compared to previous mass estimates using a similar methodology with just one satellite galaxy and places our results amongst the highest precision M31 estimates in recent literature. Furthermore, our results are consistent with recently revised estimates for the total mass of the Local Group (LG), with the stellar mass--halo mass relation, and with observed kinematic data for both M31 and its entire population of satellites. An M31 mass $> 2.5 \times 10^{12}\, M_{\odot}$ could have major implications for our understanding of LG dynamics, M31's merger and accretion history, and our understanding of LG galaxies in a cosmological context.


title: Backdoor Vulnerabilities in Normally Trained Deep Learning Models

abstract: We conduct a systematic study of backdoor vulnerabilities in normally trained Deep Learning models. They are as dangerous as backdoors injected by data poisoning because both can be equally exploited. We leverage 20 different types of injected backdoor attacks in the literature as the guidance and study their correspondences in normally trained models, which we call natural backdoor vulnerabilities. We find that natural backdoors are widely existing, with most injected backdoor attacks having natural correspondences. We categorize these natural backdoors and propose a general detection framework. It finds 315 natural backdoors in the 56 normally trained models downloaded from the Internet, covering all the different categories, while existing scanners designed for injected backdoors can at most detect 65 backdoors. We also study the root causes and defense of natural backdoors.


title: Offline Supervised Learning V.S. Online Direct Policy Optimization: A Comparative Study and A Unified Training Paradigm for Neural Network-Based Optimal Feedback Control

abstract: This work is concerned with solving neural network-based feedback controllers efficiently for optimal control problems. We first conduct a comparative study of two prevalent approaches: offline supervised learning and online direct policy optimization. Albeit the training part of the supervised learning approach is relatively easy, the success of the method heavily depends on the optimal control dataset generated by open-loop optimal control solvers. In contrast, direct policy optimization turns the optimal control problem into an optimization problem directly without any requirement of pre-computing, but the dynamics-related objective can be hard to optimize when the problem is complicated. Our results underscore the superiority of offline supervised learning in terms of both optimality and training time. To overcome the main challenges, dataset and optimization, in the two approaches respectively, we complement them and propose the Pre-train and Fine-tune strategy as a unified training paradigm for optimal feedback control, which further improves the performance and robustness significantly. Our code is accessible at https://github.com/yzhao98/DeepOptimalControl.


title: Posterior Sampling for Continuing Environments

abstract: We develop an extension of posterior sampling for reinforcement learning (PSRL) that is suited for a continuing agent-environment interface and integrates naturally into agent designs that scale to complex environments. The approach, continuing PSRL, maintains a statistically plausible model of the environment and follows a policy that maximizes expected $\gamma$-discounted return in that model. At each time, with probability $1-\gamma$, the model is replaced by a sample from the posterior distribution over environments. For a choice of discount factor that suitably depends on the horizon $T$, we establish an $\tilde{O}(\tau S \sqrt{A T})$ bound on the Bayesian regret, where $S$ is the number of environment states, $A$ is the number of actions, and $\tau$ denotes the reward averaging time, which is a bound on the duration required to accurately estimate the average reward of any policy. Our work is the first to formalize and rigorously analyze the resampling approach with randomized exploration.


title: Formal Bott-Thurston cocycle and part of a formal Riemann-Roch theorem

abstract: The Bott-Thurston cocycle is a $2$-cocycle on the group of orientation-preserving diffeomorphisms of the circle. We introduce and study a formal analog of Bott-Thurston cocycle. The formal Bott-Thurston cocycle is a $2$-cocycle on the group of continuous $A$-automorphisms of the algebra $A((t))$ of Laurent series over a commutative ring $A$ with values in the group $A^*$ of invertible elements of $A$. We prove that the central extension given by the formal Bott-Thurston cocycle is equivalent to the $12$-fold Baer sum of the determinantal central extension when $A$ is a $\mathbb Q$-algebra. As a consequence of this result we prove a part of new formal Riemann-Roch theorem. This Riemann-Roch theorem is applied to a ringed space on a separated scheme $S$ over $\mathbb Q$, where the structure sheaf of the ringed space is locally on $S$ isomorphic to the sheaf ${\mathcal O}_S((t))$ and the transition automorphisms are continuous. Locally on $S$ this ringed space corresponds to the punctured formal neighbourhood of a section of a smooth morphism to $U$ of relative dimension $1$, where an open subset $U \subset S$.


title: Maximal Atomic irRedundant Sets: a Usage-based Dataflow Partitioning Algorithm

abstract: Programs admitting a polyhedral representation can be transformed in many ways for locality and parallelism, notably loop tiling. Data flow analysis can then compute dependence relations between iterations and between tiles. When tiling is applied, certain iteration-wise dependences cross tile boundaries, creating the need for inter-tile data communication. Previous work computes it as the flow-in and flow-out sets of iteration tiles. In this paper, we propose a partitioning of the flow-out of a tile into the maximal sets of iterations that are entirely consumed and incur no redundant storage or transfer. The computation is described as an algorithm and performed on a selection of polyhedral programs. We then suggest possible applications of this decomposition in compression and memory allocation.


title: Causal identification for continuous-time stochastic processes

abstract: Many real-world processes are trajectories that may be regarded as continuous-time "functional data". Examples include patients' biomarker concentrations, environmental pollutant levels, and prices of stocks. Corresponding advances in data collection have yielded near continuous-time measurements, from e.g. physiological monitors, wearable digital devices, and environmental sensors. Statistical methodology for estimating the causal effect of a time-varying treatment, measured discretely in time, is well developed. But discrete-time methods like the g-formula, structural nested models, and marginal structural models do not generalize easily to continuous time, due to the entanglement of uncountably infinite variables. Moreover, researchers have shown that the choice of discretization time scale can seriously affect the quality of causal inferences about the effects of an intervention. In this paper, we establish causal identification results for continuous-time treatment-outcome relationships for general cadlag stochastic processes under continuous-time confounding, through orthogonalization and weighting. We use three concrete running examples to demonstrate the plausibility of our identification assumptions, as well as their connections to the discrete-time g methods literature.


title: QRPA calculations for M1 transitions with the noniterative finite amplitude method and the application to neutron radiative capture cross sections

abstract: We derive the equations of quasiparticle random-phase approximation (QRPA) based on the finite amplitude method (FAM) with the Hartree-Fock+BCS (HF+BCS) single-particle states, and calculate the magnetic dipole (M1) transition for deformed gadolinium isotopes. Our QRPA calculation shows both large spin-flip transitions in the 5 to 10 MeV excitation energy and the low energy orbital transition that would correspond to the M1 scissors mode observed in nuclear experiments. Then, we calculate neutron capture reactions based on the statistical Hauser-Feshbach theory with the photoabsorption cross sections given by QRPA. We find that the capture cross section is enhanced due to the contribution from the low energy M1 transition although the calculated capture cross section still underestimates the experimental data. This issue in the calculated capture cross section could be improved by uncertainties of low energy E1 transition neglected in our QRPA calculation.


title: Finding mixed-strategy equilibria of continuous-action games without gradients using randomized policy networks

abstract: We study the problem of computing an approximate Nash equilibrium of continuous-action game without access to gradients. Such game access is common in reinforcement learning settings, where the environment is typically treated as a black box. To tackle this problem, we apply zeroth-order optimization techniques that combine smoothed gradient estimators with equilibrium-finding dynamics. We model players' strategies using artificial neural networks. In particular, we use randomized policy networks to model mixed strategies. These take noise in addition to an observation as input and can flexibly represent arbitrary observation-dependent, continuous-action distributions. Being able to model such mixed strategies is crucial for tackling continuous-action games that lack pure-strategy equilibria. We evaluate the performance of our method using an approximation of the Nash convergence metric from game theory, which measures how much players can benefit from unilaterally changing their strategy. We apply our method to continuous Colonel Blotto games, single-item and multi-item auctions, and a visibility game. The experiments show that our method can quickly find high-quality approximate equilibria. Furthermore, they show that the dimensionality of the input noise is crucial for performance. To our knowledge, this paper is the first to solve general continuous-action games with unrestricted mixed strategies and without any gradient information.


title: Robustness Disparities in Face Detection

abstract: Facial analysis systems have been deployed by large companies and critiqued by scholars and activists for the past decade. Many existing algorithmic audits examine the performance of these systems on later stage elements of facial analysis systems like facial recognition and age, emotion, or perceived gender prediction; however, a core component to these systems has been vastly understudied from a fairness perspective: face detection, sometimes called face localization. Since face detection is a pre-requisite step in facial analysis systems, the bias we observe in face detection will flow downstream to the other components like facial recognition and emotion prediction. Additionally, no prior work has focused on the robustness of these systems under various perturbations and corruptions, which leaves open the question of how various people are impacted by these phenomena. We present the first of its kind detailed benchmark of face detection systems, specifically examining the robustness to noise of commercial and academic models. We use both standard and recently released academic facial datasets to quantitatively analyze trends in face detection robustness. Across all the datasets and systems, we generally find that photos of individuals who are $\textit{masculine presenting}$, $\textit{older}$, of $\textit{darker skin type}$, or have $\textit{dim lighting}$ are more susceptible to errors than their counterparts in other identities.


title: Segment-based fusion of multi-sensor multi-scale satellite soil moisture retrievals

abstract: Synergetic use of sensors for soil moisture retrieval is attracting considerable interest due to the different advantages of different sensors. Active, passive, and optic data integration could be a comprehensive solution for exploiting the advantages of different sensors aimed at preparing soil moisture maps. Typically, pixel-based methods are used for multi-sensor fusion. Since, different applications need different scales of soil moisture maps, pixel-based approaches are limited for this purpose. Object-based image analysis employing an image object instead of a pixel could help us to meet this need. This paper proposes a segment-based image fusion framework to evaluate the possibility of preparing a multi-scale soil moisture map through integrated Sentinel-1, Sentinel-2, and Soil Moisture Active Passive (SMAP) data. The results confirmed that the proposed methodology was able to improve soil moisture estimation in different scales up to 20% better compared to pixel-based fusion approach.


title: An Adaptive and Robust Deep Learning Framework for THz Ultra-Massive MIMO Channel Estimation

abstract: Terahertz ultra-massive MIMO (THz UM-MIMO) is envisioned as one of the key enablers of 6G wireless networks, for which channel estimation is highly challenging. Traditional analytical estimation methods are no longer effective, as the enlarged array aperture and the small wavelength result in a mixture of far-field and near-field paths, constituting a hybrid-field channel. Deep learning (DL)-based methods, despite the competitive performance, generally lack theoretical guarantees and scale poorly with the size of the array. In this paper, we propose a general DL framework for THz UM-MIMO channel estimation, which leverages existing iterative channel estimators and is with provable guarantees. Each iteration is implemented by a fixed point network (FPN), consisting of a closed-form linear estimator and a DL-based non-linear estimator. The proposed method perfectly matches the THz UM-MIMO channel estimation due to several unique advantages. First, the complexity is low and adaptive. It enjoys provable linear convergence with a low per-iteration cost and monotonically increasing accuracy, which enables an adaptive accuracy-complexity tradeoff. Second, it is robust to practical distribution shifts and can directly generalize to a variety of heavily out-of-distribution scenarios with almost no performance loss, which is suitable for the complicated THz channel conditions. For practical usage, the proposed framework is further extended to wideband THz UM-MIMO systems with beam squint effect. Theoretical analysis and extensive simulation results are provided to illustrate the advantages over the state-of-the-art methods in estimation accuracy, convergence rate, complexity, and robustness.


title: PiggyBack: Pretrained Visual Question Answering Environment for Backing up Non-deep Learning Professionals

abstract: We propose a PiggyBack, a Visual Question Answering platform that allows users to apply the state-of-the-art visual-language pretrained models easily. The PiggyBack supports the full stack of visual question answering tasks, specifically data processing, model fine-tuning, and result visualisation. We integrate visual-language models, pretrained by HuggingFace, an open-source API platform of deep learning technologies; however, it cannot be runnable without programming skills or deep learning understanding. Hence, our PiggyBack supports an easy-to-use browser-based user interface with several deep learning visual language pretrained models for general users and domain experts. The PiggyBack includes the following benefits: Free availability under the MIT License, Portability due to web-based and thus runs on almost any platform, A comprehensive data creation and processing technique, and ease of use on deep learning-based visual language pretrained models. The demo video is available on YouTube and can be found at https://youtu.be/iz44RZ1lF4s.


title: When Quantum Information Technologies Meet Blockchain in Web 3.0

abstract: With the drive to create a decentralized digital economy, Web 3.0 has become a cornerstone of digital transformation, developed on the basis of computing-force networking, distributed data storage, and blockchain. With the rapid realization of quantum devices, Web 3.0 is being developed in parallel with the deployment of quantum cloud computing and quantum Internet. In this regard, quantum computing first disrupts the original cryptographic systems that protect data security while reshaping modern cryptography with the advantages of quantum computing and communication. Therefore, in this paper, we introduce a quantum blockchain-driven Web 3.0 framework that provides information-theoretic security for decentralized data transferring and payment transactions. First, we present the framework of quantum blockchain-driven Web 3.0 with future-proof security during the transmission of data and transaction information. Next, we discuss the potential applications and challenges of implementing quantum blockchain in Web 3.0. Finally, we describe a use case for quantum non-fungible tokens (NFTs) and propose a quantum deep learning-based optimal auction for NFT trading to maximize the achievable revenue for sufficient liquidity in Web 3.0. In this way, the proposed framework can achieve proven security and sustainability for the next-generation decentralized digital society.


title: Universal quantum computer based on Carbon Nanotube Rotators

abstract: We propose a universal quantum computer based on a chain of carbon nanotube rotators where one metallic plate is attached to each rotator. The dynamical variable is the rotational angle $\phi$. The attached plate connected to ground electrostatically interacts with two fixed plates. Two angle positions $\phi =0,\pi$ are made stable by applying a voltage difference between the attached plate and the two fixed plates. We assign $\phi =0$ and $\pi $ to the qubit states $|0\rangle$ and $|1\rangle $. Then, considering a chain of rotators, we construct the arbitrary phase-shift gate, the NOT gate and the Ising gate, which constitute a set of universal quantum gates. They are executed by controlling the voltage between various plates.

categories: quant-ph cond-mat.mes-hall physics.app-ph physics.atm-clus


title: Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems

abstract: We propose a trust-region stochastic sequential quadratic programming algorithm (TR-StoSQP) to solve nonlinear optimization problems with stochastic objectives and deterministic equality constraints. We consider a fully stochastic setting, where at each step a single sample is generated to estimate the objective gradient. The algorithm adaptively selects the trust-region radius and, compared to the existing line-search StoSQP schemes, allows us to utilize indefinite Hessian matrices (i.e., Hessians without modification) in SQP subproblems. As a trust-region method for constrained optimization, our algorithm must address an infeasibility issue -- the linearized equality constraints and trust-region constraints may lead to infeasible SQP subproblems. In this regard, we propose an adaptive relaxation technique to compute the trial step, consisting of a normal step and a tangential step. To control the lengths of these two steps while ensuring a scale-invariant property, we adaptively decompose the trust-region radius into two segments, based on the proportions of the rescaled feasibility and optimality residuals to the rescaled full KKT residual. The normal step has a closed form, while the tangential step is obtained by solving a trust-region subproblem, to which a solution ensuring the Cauchy reduction is sufficient for our study. We establish a global almost sure convergence guarantee for TR-StoSQP, and illustrate its empirical performance on both a subset of problems in the CUTEst test set and constrained logistic regression problems using data from the LIBSVM collection.


title: The Effectiveness of World Models for Continual Reinforcement Learning

abstract: World models power some of the most efficient reinforcement learning algorithms. In this work, we showcase that they can be harnessed for continual learning - a situation when the agent faces changing environments. World models typically employ a replay buffer for training, which can be naturally extended to continual learning. We systematically study how different selective experience replay methods affect performance, forgetting, and transfer. We also provide recommendations regarding various modeling options for using world models. The best set of choices is called Continual-Dreamer, it is task-agnostic and utilizes the world model for continual exploration. Continual-Dreamer is sample efficient and outperforms state-of-the-art task-agnostic continual reinforcement learning methods on Minigrid and Minihack benchmarks.


title: Quantum Speed-ups for String Synchronizing Sets, Longest Common Substring, and k-mismatch Matching

abstract: Longest Common Substring (LCS) is an important text processing problem, which has recently been investigated in the quantum query model. The decisional version of this problem, LCS with threshold $d$, asks whether two length-$n$ input strings have a common substring of length $d$. The two extreme cases, $d=1$ and $d=n$, correspond respectively to Element Distinctness and Unstructured Search, two fundamental problems in quantum query complexity. However, the intermediate case $1\ll d\ll n$ was not fully understood. We show that the complexity of LCS with threshold $d$ smoothly interpolates between the two extreme cases up to $n^{o(1)}$ factors: LCS with threshold $d$ has a quantum algorithm in $n^{2/3+o(1)}/d^{1/6}$ query complexity and time complexity, and requires at least $\Omega(n^{2/3}/d^{1/6})$ quantum query complexity. Our result improves upon previous upper bounds $\tilde O(\min \{n/d^{1/2}, n^{2/3}\})$ (Le Gall and Seddighin ITCS 2022, Akmal and Jin SODA 2022), and answers an open question of Akmal and Jin. Our main technical contribution is a quantum speed-up of the powerful String Synchronizing Set technique introduced by Kempa and Kociumaka (STOC 2019). It consistently samples $n/\tau^{1-o(1)}$ synchronizing positions in the string depending on their length-$\Theta(\tau)$ contexts, and each synchronizing position can be reported by a quantum algorithm in $\tilde O(\tau^{1/2+o(1)})$ time. As another application of our quantum string synchronizing set, we study the $k$-mismatch Matching problem, which asks if the pattern has an occurrence in the text with at most $k$ Hamming mismatches. Using a structural result of Charalampopoulos, Kociumaka, and Wellnitz (FOCS 2020), we obtain a quantum algorithm for $k$-mismatch matching with $k^{3/4} n^{1/2+o(1)}$ query complexity and $\tilde O(kn^{1/2})$ time complexity.


title: Seven Concepts Attributed to Sim\'eon-Denis Poisson

abstract: Sim\'eon-Denis Poisson was 25 years old when he was appointed Professor of Mathematics at the \'Ecole Polytechnique in 1806. Elected to the Paris Acad\'emie des Sciences six years later, he soon became one of its most influential members. The origin and later developments of the many concepts in mathematics and physics that bear his name make interesting stories, a few of which we shall attempt to sketch in this paper.


title: Radiative corrections to inverse muon decay for accelerator neutrinos

abstract: Inverse muon decay ($\nu_\mu e^- \to \nu_e \mu^-$) is a promising tool to constrain neutrino fluxes with energies $E_{\nu} \ge 10.9~\mathrm{GeV}$. Radiative corrections introduce percent-level distortions to energy spectra of outgoing muons and depend on experimental details. In this paper, we calculate radiative corrections to the scattering processes $\nu_\mu e^- \to \nu_e \mu^-$ and $\bar{\nu}_e e^- \to \bar{\nu}_\mu \mu^-$. We present the muon energy spectrum for both channels, double-differential distributions in muon energy and muon scattering angle and in photon energy and photon scattering angle, and the photon energy spectrum for the dominant $\nu_\mu e^- \to \nu_e \mu^-$ process. Our results clarify and extend the region of applicability of previous results in the literature for the double differential distribution in muon energy and photon energy, and in the muon energy spectrum with a radiated photon above a threshold energy. We provide analytic expressions for single, double and triple differential cross sections, and discuss how radiative corrections modify experimentally interesting observable distributions.


title: Neural Vocoder Feature Estimation for Dry Singing Voice Separation

abstract: Singing voice separation (SVS) is a task that separates singing voice audio from its mixture with instrumental audio. Previous SVS studies have mainly employed the spectrogram masking method which requires a large dimensionality in predicting the binary masks. In addition, they focused on extracting a vocal stem that retains the wet sound with the reverberation effect. This result may hinder the reusability of the isolated singing voice. This paper addresses the issues by predicting mel-spectrogram of dry singing voices from the mixed audio as neural vocoder features and synthesizing the singing voice waveforms from the neural vocoder. We experimented with two separation methods. One is predicting binary masks in the mel-spectrogram domain and the other is directly predicting the mel-spectrogram. Furthermore, we add a singing voice detector to identify the singing voice segments over time more explicitly. We measured the model performance in terms of audio, dereverberation, separation, and overall quality. The results show that our proposed model outperforms state-of-the-art singing voice separation models in both objective and subjective evaluation except the audio quality.


title: A Mean-Field Treatment of Vacuum Fluctuations in Strong Light-Matter Coupling

abstract: Mean-field mixed quantum--classical dynamics could provide a much-needed means to inexpensively model quantum electrodynamical phenomena, by describing the optical field and its vacuum fluctuations classically. However, this approach is known to suffer from an unphysical transfer of energy out of the vacuum fluctuations when the light--matter coupling becomes strong. We highlight this issue for the case of an atom in an optical cavity, and resolve it by introducing an additional set of classical coordinates to specifically represent vacuum fluctuations whose light--matter interaction is scaled by the instantaneous ground-state population of the atom. This not only rigorously prevents the aforementioned unphysical energy transfer, but is also shown to yield a radically improved accuracy in terms of the atomic population and the optical field dynamics, generating results in excellent agreement with full quantum calculations. As such, the resulting method emerges as an attractive solution for the affordable modeling of strong light--matter coupling phenomena involving macroscopic numbers of optical modes.


title: Enhanced artificial intelligence-based diagnosis using CBCT with internal denoising: Clinical validation for discrimination of fungal ball, sinusitis, and normal cases in the maxillary sinus

abstract: The cone-beam computed tomography (CBCT) provides 3D volumetric imaging of a target with low radiation dose and cost compared with conventional computed tomography, and it is widely used in the detection of paranasal sinus disease. However, it lacks the sensitivity to detect soft tissue lesions owing to reconstruction constraints. Consequently, only physicians with expertise in CBCT reading can distinguish between inherent artifacts or noise and diseases, restricting the use of this imaging modality. The development of artificial intelligence (AI)-based computer-aided diagnosis methods for CBCT to overcome the shortage of experienced physicians has attracted substantial attention. However, advanced AI-based diagnosis addressing intrinsic noise in CBCT has not been devised, discouraging the practical use of AI solutions for CBCT. To address this issue, we propose an AI-based computer-aided diagnosis method using CBCT with a denoising module. This module is implemented before diagnosis to reconstruct the internal ground-truth full-dose scan corresponding to an input CBCT image and thereby improve the diagnostic performance. The external validation results for the unified diagnosis of sinus fungal ball, chronic rhinosinusitis, and normal cases show that the proposed method improves the micro-, macro-average AUC, and accuracy by 7.4, 5.6, and 9.6% (from 86.2, 87.0, and 73.4 to 93.6, 92.6, and 83.0%), respectively, compared with a baseline while improving human diagnosis accuracy by 11% (from 71.7 to 83.0%), demonstrating technical differentiation and clinical effectiveness. This pioneering study on AI-based diagnosis using CBCT indicates denoising can improve diagnostic performance and reader interpretability in images from the sinonasal area, thereby providing a new approach and direction to radiographic image reconstruction regarding the development of AI-based diagnostic solutions.


title: Feature-domain Adaptive Contrastive Distillation for Efficient Single Image Super-Resolution

abstract: Recently, CNN-based SISR has numerous parameters and high computational cost to achieve better performance, limiting its applicability to resource-constrained devices such as mobile. As one of the methods to make the network efficient, Knowledge Distillation (KD), which transfers teacher's useful knowledge to student, is currently being studied. More recently, KD for SISR utilizes Feature Distillation (FD) to minimize the Euclidean distance loss of feature maps between teacher and student networks, but it does not sufficiently consider how to effectively and meaningfully deliver knowledge from teacher to improve the student performance at given network capacity constraints. In this paper, we propose a feature-domain adaptive contrastive distillation (FACD) method for efficiently training lightweight student SISR networks. We show the limitations of the existing FD methods using Euclidean distance loss, and propose a feature-domain contrastive loss that makes a student network learn richer information from the teacher's representation in the feature domain. In addition, we propose an adaptive distillation that selectively applies distillation depending on the conditions of the training patches. The experimental results show that the student EDSR and RCAN networks with the proposed FACD scheme improves not only the PSNR performance of the entire benchmark datasets and scales, but also the subjective image quality compared to the conventional FD approaches.