title: Internal or external magma oceans in the earliest protoplanets -- perspectives from nitrogen and carbon fractionation

abstract: When the extent of protoplanetary melting approached magma ocean (MO)-like conditions, alloy melts efficiently segregated from the silicates to form metallic cores. The nature of the MO of a differentiating protoplanet, i.e., internal or external MO (IMO or EMO), not only determines the abundances of life-essential volatiles like nitrogen (N) and carbon (C) in its core and mantle reservoirs but also the timing and mechanism of volatile loss. Whether the earliest formed protoplanets had IMOs or EMOs is, however, poorly understood. Here we model equilibrium N and C partitioning between alloy and silicate melts in the absence (IMO) or presence (EMO) of vapor degassed atmospheres. Bulk N and C inventories of the protoplanets during core formation are constrained for IMOs and EMOs by comparing the predicted N and C abundances in the alloy melts from both scenarios with N and C concentrations in the parent cores of magmatic iron meteorites. Our results show that in comparison to EMOs, protoplanets having IMOs satisfy N and C contents of the parent cores with substantially lower amounts of bulk N and C present in the parent body during core formation. As the required bulk N and C contents for IMOs and EMOs are in the sub-chondritic and chondritic range, respectively, N and C fractionation models alone cannot be used to distinguish the prevalence of these two end-member differentiation regimes. A comparison of N and C abundances in chondrites with their peak metamorphic temperatures suggests that protoplanetary interiors could lose a substantial portion of their N and C inventories with increasing degrees of thermal metamorphism.


title: Unsupervised hyperspectral data mining and bioimaging by information entropy and self-modeling curve resolution

abstract: Unsupervised estimation of the dimensionality of hyperspectral microspectroscopy datasets containing pure and mixed spectral features, and extraction of their representative endmember spectra, remains a challenge in biochemical data mining. We report a new versatile algorithm building on semi-nonnegativity constrained self-modeling curve resolution and information entropy, to estimate the quantity of separable biochemical species from hyperspectral microspectroscopy, and extraction of their representative spectra. The algorithm is benchmarked with established methods from satellite remote sensing, spectral unmixing, and clustering. To demonstrate the widespread applicability of the developed algorithm, we collected hyperspectral datasets using spontaneous Raman, Coherent Anti-stokes Raman Scattering and Fourier Transform IR, of seven reference compounds, an oil-in-water emulsion, and tissue-engineered extracellular matrices on poly-L-lactic acid and porcine jejunum-derived small intestine submucosa scaffolds seeded with bovine chondrocytes. We show the potential of the developed algorithm by consolidating hyperspectral molecular information with sample microstructure, pertinent to fields ranging from gastrophysics to regenerative medicine.


title: Bad Citrus: Reducing Adversarial Costs with Model Distances

abstract: Recent work by Jia et al., showed the possibility of effectively computing pairwise model distances in weight space, using a model explanation technique known as LIME. This method requires query-only access to the two models under examination. We argue this insight can be leveraged by an adversary to reduce the net cost (number of queries) of launching an evasion campaign against a deployed model. We show that there is a strong negative correlation between the success rate of adversarial transfer and the distance between the victim model and the surrogate used to generate the evasive samples. Thus, we propose and evaluate a method to reduce adversarial costs by finding the closest surrogate model for adversarial transfer.


title: Physics-informed neural networks of the Saint-Venant equations for downscaling a large-scale river model

abstract: Large-scale river models are being refined over coastal regions to improve the scientific understanding of coastal processes, hazards and responses to climate change. However, coarse mesh resolutions and approximations in physical representations of tidal rivers limit the performance of such models at resolving the complex flow dynamics especially near the river-ocean interface, resulting in inaccurate simulations of flood inundation. In this research, we propose a machine learning (ML) framework based on the state-of-the-art physics-informed neural network (PINN) to simulate the downscaled flow at the subgrid scale. First, we demonstrate that PINN is able to assimilate observations of various types and solve the one-dimensional (1-D) Saint-Venant equations (SVE) directly. We perform the flow simulations over a floodplain and along an open channel in several synthetic case studies. The PINN performance is evaluated against analytical solutions and numerical models. Our results indicate that the PINN solutions of water depth have satisfactory accuracy with limited observations assimilated. In the case of flood wave propagation induced by storm surge and tide, a new neural network architecture is proposed based on Fourier feature embeddings that seamlessly encodes the periodic tidal boundary condition in the PINN's formulation. Furthermore, we show that the PINN-based downscaling can produce more reasonable subgrid solutions of the along-channel water depth by assimilating observational data. The PINN solution outperforms the simple linear interpolation in resolving the topography and dynamic flow regimes at the subgrid scale. This study provides a promising path towards improving emulation capabilities in large-scale models to characterize fine-scale coastal processes.


title: A Step Towards Uncovering The Structure of Multistable Neural Networks

abstract: We study how the connectivity within a recurrent neural network determines and is determined by the multistable solutions of network activity. To gain analytic tractability we let neural activation be a non-smooth Heaviside step function. This nonlinearity partitions the phase space into regions with different, yet linear dynamics. In each region either a stable equilibrium state exists, or network activity flows to outside of the region. The stable states are identified by their semipositivity constraints on the synaptic weight matrix. The restrictions can be separated by their effects on the signs or the strengths of the connections. Exact results on network topology, sign stability, weight matrix factorization, pattern completion and pattern coupling are derived and proven. Our work may lay the foundation for multistability in more complex recurrent neural networks.


title: Disentangling Mixtures of Unknown Causal Interventions

abstract: In many real-world scenarios, such as gene knockout experiments, targeted interventions are often accompanied by unknown interventions at off-target sites. Moreover, different units can get randomly exposed to different unknown interventions, thereby creating a mixture of interventions. Identifying different components of this mixture can be very valuable in some applications. Motivated by such situations, in this work, we study the problem of identifying all components present in a mixture of interventions on a given causal Bayesian Network. We construct an example to show that, in general, the components are not identifiable from the mixture distribution. Next, assuming that the given network satisfies a positivity condition, we show that, if the set of mixture components satisfy a mild exclusion assumption, then they can be uniquely identified. Our proof gives an efficient algorithm to recover these targets from the exponentially large search space of possible targets. In the more realistic scenario, where distributions are given via finitely many samples, we conduct a simulation study to analyze the performance of an algorithm derived from our identifiability proof.


title: Approximate Methods for Bayesian Computation

abstract: Rich data generating mechanisms are ubiquitous in this age of information and require complex statistical models to draw meaningful inference. While Bayesian analysis has seen enormous development in the last 30 years, benefitting from the impetus given by the successful application of Markov chain Monte Carlo (MCMC) sampling, the combination of big data and complex models conspire to produce significant challenges for the traditional MCMC algorithms. We review modern algorithmic developments addressing the latter and compare their performance using numerical experiments.


title: Reachability Analysis and Safety Verification of Neural Feedback Systems via Hybrid Zonotopes

abstract: Hybrid zonotopes generalize constrained zonotopes by introducing additional binary variables and possess some unique properties that make them convenient to represent nonconvex sets. This paper presents novel hybrid zonotope-based methods for the reachability analysis and safety verification of neural feedback systems. Algorithms are proposed to compute the input-output relationship of each layer of a feedforward neural network, as well as the exact reachable sets of neural feedback systems. In addition, a sufficient and necessary condition is formulated as a mixed-integer linear program to certify whether the trajectories of a neural feedback system can avoid unsafe regions. The proposed approach is shown to yield a formulation that provides the tightest convex relaxation for the reachable sets of the neural feedback system. Complexity reduction techniques for the reachable sets are developed to balance the computation efficiency and approximation accuracy. Two numerical examples demonstrate the superior performance of the proposed approach compared to other existing methods.


title: 21st Century Global and Regional Surface Temperature Projections

abstract: Many regions across the globe broke their surface temperature records in recent years, further sparking concerns about the impending arrival of "tipping points" later in the 21st century. This study analyzes observed global surface temperature trends in three target latitudinal regions: the Arctic Circle, the Tropics, and the Antarctic Circle. We show that global warming is accelerating unevenly across the planet, with the Arctic warming at approximately three times the average rate of our world. We further analyzed the reliability of latitude-dependent surface temperature simulations from a suite of Coupled Model Intercomparison Project Phase 6 models and their multi-model mean. We found that GISS-E2-1-G and FGOALS-g3 were the best-performing models based on their statistical abilities to reproduce observational, latitude-dependent data. Surface temperatures were projected from ensemble simulations of the Shared Socioeconomic Pathway 2-4.5 (SSP2-4.5). We estimate when the climate will warm by 1.5, 2.0, and 2.5 degrees C relative to the preindustrial period, globally and regionally. GISS-E2-1-G projects that global surface temperature anomalies would reach 1.5, 2.0, and 2.5 degrees C in 2024 (+/-1.34), 2039 (+/-2.83), and 2057 (+/-5.03) respectively, while FGOALS-g3 predicts these "tipping points" would arrive in 2024 (+/-2.50), 2054 (+/-7.90), and 2087 (+/-10.55) respectively. Our results reaffirm a dramatic, upward trend in projected climate warming acceleration, with upward concavity in 21st century projections of the Arctic, which could lead to catastrophic consequences across the Earth. Further studies are necessary to determine the most efficient solutions to reduce global warming acceleration and maintain a low SSP, both globally and regionally.


title: HealthE: Classifying Entities in Online Textual Health Advice

abstract: The processing of entities in natural language is essential to many medical NLP systems. Unfortunately, existing datasets vastly under-represent the entities required to model public health relevant texts such as health advice often found on sites like WebMD. People rely on such information for personal health management and clinically relevant decision making. In this work, we release a new annotated dataset, HealthE, consisting of 6,756 health advice. HealthE has a more granular label space compared to existing medical NER corpora and contains annotation for diverse health phrases. Additionally, we introduce a new health entity classification model, EP S-BERT, which leverages textual context patterns in the classification of entity classes. EP S-BERT provides a 4-point increase in F1 score over the nearest baseline and a 34-point increase in F1 when compared to off-the-shelf medical NER tools trained to extract disease and medication mentions from clinical texts. All code and data are publicly available on Github.


title: Modified hybrid inflation, reheating and stabilization of the electroweak vacuum

abstract: We propose a modification to the standard hybrid inflation model \cite{Linde:1993cn}, that connects a successful hybrid inflation scenario to the standard model Higgs sector, via the electroweak vacuum stability. The proposed model results in an effective inflation potential of a hilltop-type, with both the trans-Planckian and sub-Planckian inflation regimes consistent with the recent Planck/BICEP combined results. Reheating via the inflation sector decays to right-handed neutrinos is considered. An upper bound on the reheating temperature $T_{\rm R} \lsim 2\times 10^{11}~(1\times 10^{13})$ GeV, for large~(small) field inflation, will suppress contributions from one-loop quantum corrections to the inflation potential. This may push the neutrino Yukawa couplings to be ${\cal O}(1)$ and affect the vacuum stability. We show that the couplings of the SM Higgs to the inflation sector can guarantee the electroweak vacuum stability up to the Planck scale. The so-called hybrid Higgs-inflaton model leads to a positive correction for the Higgs quartic coupling at a threshold scale, which is shown to have a very significant effect in stabilizing the electroweak vacuum. We find that even with ${\cal O}(1)$ neutrino Yukawa couplings, threshold corrections leave the SM vacuum stability intact.


title: Direct stellarator coil optimization for nested magnetic surfaces with precise quasi-symmetry

abstract: We present a robust optimization algorithm for the design of electromagnetic coils that generate vacuum magnetic fields with nested flux surfaces and precise quasi-symmetry. The method is based on a bilevel optimization problem, where the outer coil optimization is constrained by a set of inner least-squares optimization problems whose solutions describe magnetic surfaces. The outer optimization objective targets coils that generate a field with nested magnetic surfaces and good quasi-symmetry. The inner optimization problems identify magnetic surfaces when they exist, and approximate surfaces in the presence of magnetic islands or chaos. We show that this formulation can be used to heal islands and chaos, thus producing coils that result in magnetic fields with precise quasi-symmetry. We show that the method can be initialized with coils from the traditional two stage coil design process, as well as coils from a near axis expansion optimization. We present a numerical example where island chains are healed and quasi-symmetry is optimized up to surfaces with aspect ratio 6. Another numerical example illustrates that the aspect ratio of nested flux surfaces with optimized quasi-symmetry can be decreased from 6 to approximately 4. The last example shows that our approach is robust and a cold-start using coils from a near-axis expansion optimization.


title: Joint Protection Scheme for Deep Neural Network Hardware Accelerators and Models

abstract: Deep neural networks (DNNs) are utilized in numerous image processing, object detection, and video analysis tasks and need to be implemented using hardware accelerators to achieve practical speed. Logic locking is one of the most popular methods for preventing chip counterfeiting. Nevertheless, existing logic-locking schemes need to sacrifice the number of input patterns leading to wrong output under incorrect keys to resist the powerful satisfiability (SAT)-attack. Furthermore, DNN model inference is fault-tolerant. Hence, using a wrong key for those SAT-resistant logic-locking schemes may not affect the accuracy of DNNs. This makes the previous SAT-resistant logic-locking scheme ineffective on protecting DNN accelerators. Besides, to prevent DNN models from being illegally used, the models need to be obfuscated by the designers before they are provided to end-users. Previous obfuscation methods either require long time to retrain the model or leak information about the model. This paper proposes a joint protection scheme for DNN hardware accelerators and models. The DNN accelerator is modified using a hardware key (Hkey) and a model key (Mkey). Different from previous logic locking, the Hkey, which is used to protect the accelerator, does not affect the output when it is wrong. As a result, the SAT attack can be effectively resisted. On the other hand, a wrong Hkey leads to substantial increase in memory accesses, inference time, and energy consumption and makes the accelerator unusable. A correct Mkey can recover the DNN model that is obfuscated by the proposed method. Compared to previous model obfuscation schemes, our proposed method avoids model retraining and does not leak model information.


title: Unsupervised Domain Adaptation for COVID-19 Information Service with Contrastive Adversarial Domain Mixup

abstract: In the real-world application of COVID-19 misinformation detection, a fundamental challenge is the lack of the labeled COVID data to enable supervised end-to-end training of the models, especially at the early stage of the pandemic. To address this challenge, we propose an unsupervised domain adaptation framework using contrastive learning and adversarial domain mixup to transfer the knowledge from an existing source data domain to the target COVID-19 data domain. In particular, to bridge the gap between the source domain and the target domain, our method reduces a radial basis function (RBF) based discrepancy between these two domains. Moreover, we leverage the power of domain adversarial examples to establish an intermediate domain mixup, where the latent representations of the input text from both domains could be mixed during the training process. Extensive experiments on multiple real-world datasets suggest that our method can effectively adapt misinformation detection systems to the unseen COVID-19 target domain with significant improvements compared to the state-of-the-art baselines.


title: Small Character Models Match Large Word Models for Autocomplete Under Memory Constraints

abstract: Autocomplete is a task where the user inputs a piece of text, termed prompt, which is conditioned by the model to generate semantically coherent continuation. Existing works for this task have primarily focused on datasets (e.g., email, chat) with high frequency user prompt patterns (or focused prompts) where word-based language models have been quite effective. In this work, we study the more challenging open-domain setting consisting of low frequency user prompt patterns (or broad prompts, e.g., prompt about 93rd academy awards) and demonstrate the effectiveness of character-based language models. We study this problem under memory-constrained settings (e.g., edge devices and smartphones), where character-based representation is effective in reducing the overall model size (in terms of parameters). We use WikiText-103 benchmark to simulate broad prompts and demonstrate that character models rival word models in exact match accuracy for the autocomplete task, when controlled for the model size. For instance, we show that a 20M parameter character model performs similar to an 80M parameter word model in the vanilla setting. We further propose novel methods to improve character models by incorporating inductive bias in the form of compositional information and representation transfer from large word models. Datasets and code used in this work are available at https://github.com/UBC-NLP/char_autocomplete.


title: Excitonic Condensate in Flat Valence and Conduction Bands of Opposite Chirality

abstract: Excitonic Bose-Einstein condensation (EBEC) has drawn increasing attention recently with the emergence of 2D materials. A general criterion for EBEC, as expected in an excitonic insulator (EI) state, is to have negative exciton formation energies in a semiconductor. Here, using exact diagonalization of multi-exciton Hamiltonian modelled in a diatomic Kagome lattice, we demonstrate that the negative exciton formation energies are only a prerequisite but insufficient condition for realizing an EI. By a comparative study between the cases of both a conduction and valence flat bands (FBs) versus that of a parabolic conduction band, we further show that the presence and increased FB contribution to exciton formation provide an attractive avenue to stabilize the EBEC, as confirmed by calculations and analyses of multi-exciton energies, wave functions and reduced density matrices. Our results warrant a similar many-exciton analysis for other known/new candidates of EIs, and demonstrate the FBs of opposite parity as a unique platform for studying exciton physics, paving the way to material realization of spinor BEC and spin-superfluidity.


title: Fast Automatic Bayesian Cubature Using Matching Kernels and Designs

abstract: Automatic cubatures approximate integrals to user-specified error tolerances. For high dimensional problems, it is difficult to adaptively change the sampling pattern to focus on peaks because peaks can hide more easily in high dimensional space. But, one can automatically determine the sample size, $n$, given a reasonable, fixed sampling pattern. This approach is pursued in Jagadeeswaran and Hickernell, Stat.\ Comput., 29:1214-1229, 2019, where a Bayesian perspective is used to construct a credible interval for the integral, and the computation is terminated when the half-width of the interval is no greater than the required error tolerance. Our earlier work employs integration lattice sampling, and the computations are expedited by the fast Fourier transform because the covariance kernels for the Gaussian process prior on the integrand are chosen to be shift-invariant. In this chapter, we extend our fast automatic Bayesian cubature to digital net sampling via \emph{digitally} shift-invariant covariance kernels and fast Walsh transforms. Our algorithm is implemented in the MATLAB Guaranteed Automatic Integration Library (GAIL) and the QMCPy Python library.


title: Network Intrusion Detection System in a Light Bulb

abstract: Internet of Things (IoT) devices are progressively being utilised in a variety of edge applications to monitor and control home and industry infrastructure. Due to the limited compute and energy resources, active security protections are usually minimal in many IoT devices. This has created a critical security challenge that has attracted researchers' attention in the field of network security. Despite a large number of proposed Network Intrusion Detection Systems (NIDSs), there is limited research into practical IoT implementations, and to the best of our knowledge, no edge-based NIDS has been demonstrated to operate on common low-power chipsets found in the majority of IoT devices, such as the ESP8266. This research aims to address this gap by pushing the boundaries on low-power Machine Learning (ML) based NIDSs. We propose and develop an efficient and low-power ML-based NIDS, and demonstrate its applicability for IoT edge applications by running it on a typical smart light bulb. We also evaluate our system against other proposed edge-based NIDSs and show that our model has a higher detection performance, and is significantly faster and smaller, and therefore more applicable to a wider range of IoT edge devices.


title: Damage Control During Domain Adaptation for Transducer Based Automatic Speech Recognition

abstract: Automatic speech recognition models are often adapted to improve their accuracy in a new domain. A potential drawback of model adaptation to new domains is catastrophic forgetting, where the Word Error Rate on the original domain is significantly degraded. This paper addresses the situation when we want to simultaneously adapt automatic speech recognition models to a new domain and limit the degradation of accuracy on the original domain without access to the original training dataset. We propose several techniques such as a limited training strategy and regularized adapter modules for the Transducer encoder, prediction, and joiner network. We apply these methods to the Google Speech Commands and to the UK and Ireland English Dialect speech data set and obtain strong results on the new target domain while limiting the degradation on the original domain.


title: Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negation

abstract: Negation is poorly captured by current language models, although the extent of this problem is not widely understood. We introduce a natural language inference (NLI) test suite to enable probing the capabilities of NLP methods, with the aim of understanding sub-clausal negation. The test suite contains premise--hypothesis pairs where the premise contains sub-clausal negation and the hypothesis is constructed by making minimal modifications to the premise in order to reflect different possible interpretations. Aside from adopting standard NLI labels, our test suite is systematically constructed under a rigorous linguistic framework. It includes annotation of negation types and constructions grounded in linguistic theory, as well as the operations used to construct hypotheses. This facilitates fine-grained analysis of model performance. We conduct experiments using pre-trained language models to demonstrate that our test suite is more challenging than existing benchmarks focused on negation, and show how our annotation supports a deeper understanding of the current NLI capabilities in terms of negation and quantification.


title: Confronting anomalous kaon correlations measured in Pb-Pb collisions at $\sqrt{s_{NN}} = 2.76$ TeV

abstract: Measurements of the dynamical correlations between neutral and charged kaons in central Pb-Pb collisions at $\sqrt{s_{NN}} = 2.76$ TeV by the ALICE Collaboration display anomalous behavior relative to conventional heavy-ion collision simulators such as AMPT, EPOS, and HIJING. We consider other conventional statistical models, none of which can reproduce the magnitude and centrality dependence of the correlations. The data can be reproduced by coherent emission from domains which grow in number and volume with increasing centrality. We show that the energy released by condensation of strange quarks may be sufficient to explain the anomaly.


title: Interpreting County Level COVID-19 Infection and Feature Sensitivity using Deep Learning Time Series Models

abstract: Interpretable machine learning plays a key role in healthcare because it is challenging in understanding feature importance in deep learning model predictions. We propose a novel framework that uses deep learning to study feature sensitivity for model predictions. This work combines sensitivity analysis with heterogeneous time-series deep learning model prediction, which corresponds to the interpretations of spatio-temporal features. We forecast county-level COVID-19 infection using the Temporal Fusion Transformer. We then use the sensitivity analysis extending Morris Method to see how sensitive the outputs are with respect to perturbation to our static and dynamic input features. The significance of the work is grounded in a real-world COVID-19 infection prediction with highly non-stationary, finely granular, and heterogeneous data. 1) Our model can capture the detailed daily changes of temporal and spatial model behaviors and achieves high prediction performance compared to a PyTorch baseline. 2) By analyzing the Morris sensitivity indices and attention patterns, we decipher the meaning of feature importance with observational population and dynamic model changes. 3) We have collected 2.5 years of socioeconomic and health features over 3142 US counties, such as observed cases and deaths, and a number of static (age distribution, health disparity, and industry) and dynamic features (vaccination, disease spread, transmissible cases, and social distancing). Using the proposed framework, we conduct extensive experiments and show our model can learn complex interactions and perform predictions for daily infection at the county level. Being able to model the disease infection with a hybrid prediction and description accuracy measurement with Morris index at the county level is a central idea that sheds light on individual feature interpretation via sensitivity analysis.


title: Considerations for Task Allocation in Human-Robot Teams

abstract: In human-robot teams where agents collaborate together, there needs to be a clear allocation of tasks to agents. Task allocation can aid in achieving the presumed benefits of human-robot teams, such as improved team performance. Many task allocation methods have been proposed that include factors such as agent capability, availability, workload, fatigue, and task and domain-specific parameters. In this paper, selected work on task allocation is reviewed. In addition, some areas for continued and further consideration in task allocation are discussed. These areas include level of collaboration, novel tasks, unknown and dynamic agent capabilities, negotiation and fairness, and ethics. Where applicable, we also mention some of our work on task allocation. Through continued efforts and considerations in task allocation, human-robot teaming can be improved.


title: The inverse problem beyond two-body interaction: the cubic mean-field Ising model

abstract: In this paper we solve the inverse problem for the cubic mean-field Ising model. Starting from configuration data generated according to the distribution of the model we reconstruct the free parameters of the system. We test the robustness of this inversion procedure both in the region of uniqueness of the solutions and in the region where multiple thermodynamics phases are present.


title: Adversarial Lagrangian Integrated Contrastive Embedding for Limited Size Datasets

abstract: Certain datasets contain a limited number of samples with highly various styles and complex structures. This study presents a novel adversarial Lagrangian integrated contrastive embedding (ALICE) method for small-sized datasets. First, the accuracy improvement and training convergence of the proposed pre-trained adversarial transfer are shown on various subsets of datasets with few samples. Second, a novel adversarial integrated contrastive model using various augmentation techniques is investigated. The proposed structure considers the input samples with different appearances and generates a superior representation with adversarial transfer contrastive training. Finally, multi-objective augmented Lagrangian multipliers encourage the low-rank and sparsity of the presented adversarial contrastive embedding to adaptively estimate the coefficients of the regularizers automatically to the optimum weights. The sparsity constraint suppresses less representative elements in the feature space. The low-rank constraint eliminates trivial and redundant components and enables superior generalization. The performance of the proposed model is verified by conducting ablation studies by using benchmark datasets for scenarios with small data samples.


title: Rado Numbers and SAT Computations

abstract: Given a linear equation $\mathcal{E}$, the $k$-color Rado number $R_k(\mathcal{E})$ is the smallest integer $n$ such that every $k$-coloring of $\{1,2,3,\dots,n\}$ contains a monochromatic solution to $\mathcal E$. The degree of regularity of $\mathcal E$, denoted $dor(\mathcal E)$, is the largest value $k$ such that $R_k(\mathcal E)$ is finite. In this article we present new theoretical and computational results about the Rado numbers $R_3(\mathcal{E})$ and the degree of regularity of three-variable equations $\mathcal{E}$. % We use SAT solvers to compute many new values of the three-color Rado numbers $R_3(ax+by+cz = 0)$ for fixed integers $a,b,$ and $c$. We also give a SAT-based method to compute infinite families of these numbers. In particular, we show that the value of $R_3(x-y = (m-2) z)$ is equal to $m^3-m^2-m-1$ for $m\ge 3$. This resolves a conjecture of Myers and implies the conjecture that the generalized Schur numbers $S(m,3) = R_3(x_1+x_2 + \dots x_{m-1} = x_m)$ equal $m^3-m^2-m-1$ for $m\ge 3$. Our SAT solver computations, combined with our new combinatorial results, give improved bounds on $dor(ax+by = cz)$ and exact values for $1\le a,b,c\le 5 $. We also give counterexamples to a conjecture of Golowich.


title: Moving monotonicity formulae for minimal submanifolds in constant curvature

abstract: We discover new monotonicity formulae for minimal submanifolds in space forms, which imply the sharp area bound for minimal submanifolds through a prescribed point in a geodesic ball. These monotonicity formulae involve an energy-like integral over sets which are, in general, not geodesic balls. In the Euclidean case, these sets reduce to the moving-centre balls introduced by the second author in [Zhu18].


title: Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters

abstract: Research has shown that personality is a key driver to improve engagement and user experience in conversational systems. Conversational agents should also maintain a consistent persona to have an engaging conversation with a user. However, text generation datasets are often crowd sourced and thereby have an averaging effect where the style of the generation model is an average style of all the crowd workers that have contributed to the dataset. While one can collect persona-specific datasets for each task, it would be an expensive and time consuming annotation effort. In this work, we propose a novel transfer learning framework which updates only $0.3\%$ of model parameters to learn style specific attributes for response generation. For the purpose of this study, we tackle the problem of stylistic story ending generation using the ROC stories Corpus. We learn style specific attributes from the PERSONALITY-CAPTIONS dataset. Through extensive experiments and evaluation metrics we show that our novel training procedure can improve the style generation by 200 over Encoder-Decoder baselines while maintaining on-par content relevance metrics with


title: Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Tasks

abstract: Adapting large-scale pretrained models to various downstream tasks via fine-tuning is a standard method in machine learning. Recently, parameter-efficient fine-tuning methods show promise in adapting a pretrained model to different tasks while training only a few parameters. Despite their success, most existing methods are proposed in Natural Language Processing tasks with language Transformers, and adaptation to Computer Vision tasks with Vision Transformers remains under-explored, especially for dense vision tasks. Further, in multi-task settings, individually fine-tuning and storing separate models for different tasks is inefficient. In this work, we provide an extensive multi-task parameter-efficient benchmark and examine existing parameter-efficient fine-tuning NLP methods for vision tasks. Our results on four different dense vision tasks showed that existing methods cannot be efficiently integrated due to the hierarchical nature of the Hierarchical Vision Transformers. To overcome this issue, we propose Polyhistor and Polyhistor-Lite, consisting of Decomposed HyperNetworks and Layer-wise Scaling Kernels, to share information across different tasks with a few trainable parameters. This leads to favorable performance improvements against existing parameter-efficient methods while using fewer trainable parameters. Specifically, Polyhistor achieves competitive accuracy compared to the state-of-the-art while only using ~10% of their trainable parameters. Furthermore, our methods show larger performance gains when large networks and more pretraining data are used.


title: Maximum Likelihood-based Gridless DoA Estimation Using Structured Covariance Matrix Recovery and SBL with Grid Refinement

abstract: We consider the parametric data model employed in applications such as line spectral estimation and direction-of-arrival estimation. We focus on the stochastic maximum likelihood estimation (MLE) framework and offer approaches to estimate the parameter of interest in a gridless manner, overcoming the model complexities of the past. This progress is enabled by the modern trend of reparameterization of the objective and exploiting the sparse Bayesian learning (SBL) approach. The latter is shown to be a correlation-aware method, and for the underlying problem it is identified as a grid-based technique for recovering a structured covariance matrix of the measurements. For the case when the structured matrix is expressible as a sampled Toeplitz matrix, such as when measurements are sampled in time or space at regular intervals, additional constraints and reparameterization of the SBL objective leads to the proposed structured matrix recovery technique based on MLE. The proposed optimization problem is non-convex, and we propose a majorization-minimization based iterative procedure to estimate the structured matrix; each iteration solves a semidefinite program. We recover the parameter of interest in a gridless manner by appealing to the Caratheodory-Fejer result on decomposition of PSD Toeplitz matrices. For the general case of irregularly spaced time or spatial samples, we propose an iterative SBL procedure that refines grid points to increase resolution near potential source locations, while maintaining a low per iteration complexity. We provide numerical results to evaluate and compare the performance of the proposed techniques with other gridless techniques, and the CRB. The proposed correlation-aware approach is more robust to environmental/system effects such as low number of snapshots, correlated sources, small separation between source locations and improves sources identifiability.


title: Meet the parents: the progenitor binary for the supermassive black hole candidate in E1821+643

abstract: The remnants of binary black hole mergers can be given recoil kick velocities up to $5,000\text{ km s$^{-1}$}$ due to anisotropic emission of gravitational waves. E1821+643 is a recoiling supermassive black hole moving at $\sim 2,100\text{ km s$^{-1}$}$ along the line-of-sight relative to its host galaxy. This suggests a recoil kick of $\sim 2,240\text{ km s$^{-1}$}$. Such a kick is powerful enough to eject E1821+643 from its $M_\text{gal}\sim 2 \times 10^{12}\text{ M}_{\odot}$ host galaxy. In this work, we address the question: what are the likely properties of the progenitor binary that formed E1821+643? Using astrophysically motivated priors, we infer that E1821+643 was likely formed from a binary black hole system with masses of $m_1\sim 1.9^{+5.0}_{-3.8}\times 10^{9}\text{ M}_{\odot}$, $m_2\sim 8.1^{+3.9}_{-3.2} \times 10^{8}\text{ M}_{\odot}$ (90\% credible intervals). The black holes in this binary were likely to be spinning rapidly with dimensionless spin magnitudes of ${\chi}_1 = 0.87^{+0.11}_{-0.26}$, ${\chi}_2 = 0.77^{+0.19}_{-0.37}$. Such a high recoil velocity is impossible for spins aligned to the orbital angular momentum axis. This suggest that E1821+643 merged in hot gas, which is thought to provide an environment where spin alignment from accretion proceeds slowly relative to the merger timescale. We infer that E1821+643 is likely to be rapidly rotating with dimensionless spin ${\chi} = 0.92\pm0.04$. A $2.6 \times 10^9 \text{ M}_{\odot}$ black hole, recoiling from a gas-rich environment at $v\sim 2,240\text{ km s$^{-1}$}$ is likely to persist as an active galactic nuclei for $\sim 860$ Myr, in which time it traverses $\sim 2$ Mpc.


title: Investigating the Global Properties of a Resource Theory of Contextuality

abstract: Resource theories constitute a powerful theoretical framework and a tool that captures, in an abstract structure, pragmatic aspects of the most varied theories and processes. For physical theories, while this framework deals directly with questions about the concrete possibilities of carrying out tasks and processes, resource theories also make it possible to recast these already established theories on a new language, providing not only new perspectives on the potential of physical phenomena as valuable resources for technological development, for example, but they also provide insights into the very foundations of these theories. In this work, we will investigate some properties of a resource theory for quantum contextuality, an essential characteristic of quantum phenomena that ensures the impossibility of interpreting the results of quantum measurements as revealing properties that are independent of the set of measurements being made. We will present the resource theory to be studied and investigate certain global properties of this theory using tools and methods that, although already developed and studied by the community in other resource theories, had not yet been used to characterize resource theories of contextuality. In particular, we will use the so called cost and yield monotones, extending the results of reference Quantum 4, 280 (2020) to general contextuality scenarios.


title: Multi-agent Deep Covering Skill Discovery

abstract: The use of skills (a.k.a., options) can greatly accelerate exploration in reinforcement learning, especially when only sparse reward signals are available. While option discovery methods have been proposed for individual agents, in multi-agent reinforcement learning settings, discovering collaborative options that can coordinate the behavior of multiple agents and encourage them to visit the under-explored regions of their joint state space has not been considered. In this case, we propose Multi-agent Deep Covering Option Discovery, which constructs the multi-agent options through minimizing the expected cover time of the multiple agents' joint state space. Also, we propose a novel framework to adopt the multi-agent options in the MARL process. In practice, a multi-agent task can usually be divided into some sub-tasks, each of which can be completed by a sub-group of the agents. Therefore, our algorithm framework first leverages an attention mechanism to find collaborative agent sub-groups that would benefit most from coordinated actions. Then, a hierarchical algorithm, namely HA-MSAC, is developed to learn the multi-agent options for each sub-group to complete their sub-tasks first, and then to integrate them through a high-level policy as the solution of the whole task. This hierarchical option construction allows our framework to strike a balance between scalability and effective collaboration among the agents. The evaluation based on multi-agent collaborative tasks shows that the proposed algorithm can effectively capture the agent interactions with the attention mechanism, successfully identify multi-agent options, and significantly outperforms prior works using single-agent options or no options, in terms of both faster exploration and higher task rewards.


title: TRADE: Object Tracking with 3D Trajectory and Ground Depth Estimates for UAVs

abstract: We propose TRADE for robust tracking and 3D localization of a moving target in cluttered environments, from UAVs equipped with a single camera. Ultimately TRADE enables 3d-aware target following. Tracking-by-detection approaches are vulnerable to target switching, especially between similar objects. Thus, TRADE predicts and incorporates the target 3D trajectory to select the right target from the tracker's response map. Unlike static environments, depth estimation of a moving target from a single camera is a ill-posed problem. Therefore we propose a novel 3D localization method for ground targets on complex terrain. It reasons about scene geometry by combining ground plane segmentation, depth-from-motion and single-image depth estimation. The benefits of using TRADE are demonstrated as tracking robustness and depth accuracy on several dynamic scenes simulated in this work. Additionally, we demonstrate autonomous target following using a thermal camera by running TRADE on a quadcopter's board computer.


title: On the bifurcation theory of the Ginzburg-Landau equations

abstract: We construct nonminimal and irreducible solutions to the Ginzburg-Landau equations on closed manifolds of arbitrary dimension with trivial first real cohomology. Our method uses bifurcation theory where the "bifurcation points" are characterized by the eigenvalues of a Laplace-type operator. To our knowledge these are the first such examples on nontrivial line bundles.


title: A Fast Transient Backend to Detect FRBs with the Tianlai Dish Pathfinder Array

abstract: The Tianlai Dish Pathfinder array is a radio interferometer array consisting of 16 six meter dish antennas. The original digital backend integration time is at the seconds level, designed for HI intensity mapping experiment. A new digital backend with millisecond response is added to enable it to search for fast radio burst (FRB) during its observations. The design and calibration of this backend, and the real time search pipeline for it are described in this paper. It is capable of forming 16 digital beams for each linear polarisation, covering an area of 19.6 square degrees. The search pipeline is capable of searching for, recording and classifying FRBs automatically in real time. In commissioning, we succeeded in capturing the signal pulses from the pulsars PSR B0329+54 and B2021+51.


title: A Unified Encoder-Decoder Framework with Entity Memory

abstract: Entities, as important carriers of real-world knowledge, play a key role in many NLP tasks. We focus on incorporating entity knowledge into an encoder-decoder framework for informative text generation. Existing approaches tried to index, retrieve, and read external documents as evidence, but they suffered from a large computational overhead. In this work, we propose an encoder-decoder framework with an entity memory, namely EDMem. The entity knowledge is stored in the memory as latent representations, and the memory is pre-trained on Wikipedia along with encoder-decoder parameters. To precisely generate entity names, we design three decoding methods to constrain entity generation by linking entities in the memory. EDMem is a unified framework that can be used on various entity-intensive question answering and generation tasks. Extensive experimental results show that EDMem outperforms both memory-based auto-encoder models and non-memory encoder-decoder models.


title: TCNL: Transparent and Controllable Network Learning Via Embedding Human-Guided Concepts

abstract: Explaining deep learning models is of vital importance for understanding artificial intelligence systems, improving safety, and evaluating fairness. To better understand and control the CNN model, many methods for transparency-interpretability have been proposed. However, most of these works are less intuitive for human understanding and have insufficient human control over the CNN model. We propose a novel method, Transparent and Controllable Network Learning (TCNL), to overcome such challenges. Towards the goal of improving transparency-interpretability, in TCNL, we define some concepts for specific classification tasks through scientific human-intuition study and incorporate concept information into the CNN model. In TCNL, the shallow feature extractor gets preliminary features first. Then several concept feature extractors are built right after the shallow feature extractor to learn high-dimensional concept representations. The concept feature extractor is encouraged to encode information related to the predefined concepts. We also build the concept mapper to visualize features extracted by the concept extractor in a human-intuitive way. TCNL provides a generalizable approach to transparency-interpretability. Researchers can define concepts corresponding to certain classification tasks and encourage the model to encode specific concept information, which to a certain extent improves transparency-interpretability and the controllability of the CNN model. The datasets (with concept sets) for our experiments will also be released (https://github.com/bupt-ai-cz/TCNL).


title: Achieving and Understanding Out-of-Distribution Generalization in Systematic Reasoning in Small-Scale Transformers

abstract: Out-of-distribution generalization (OODG) is a longstanding challenge for neural networks. This challenge is quite apparent in tasks with well-defined variables and rules, where explicit use of the rules could solve problems independently of the particular values of the variables, but networks tend to be tied to the range of values sampled in their training data. Large transformer-based language models have pushed the boundaries on how well neural networks can solve previously unseen problems, but their complexity and lack of clarity about the relevant content in their training data obfuscates how they achieve such robustness. As a step toward understanding how transformer-based systems generalize, we explore the question of OODG in small scale transformers trained with examples from a known distribution. Using a reasoning task based on the puzzle Sudoku, we show that OODG can occur on a complex problem if the training set includes examples sampled from the whole distribution of simpler component tasks. Successful generalization depends on carefully managing positional alignment when absolute position encoding is used, but we find that suppressing sensitivity to absolute positions overcomes this limitation. Taken together our results represent a small step toward understanding and promoting systematic generalization in transformers.


title: Excited States beyond Mott Gap in Half-Filled-Band Hubbard Model

abstract: In connection with recent experiments on excitation in which Mott insulators change to conductors, we study the properties of excited states beyond the Mott gap as quasi-stationary states for a two-dimensional Hubbard (t-t'-U) model at half filling. A variational Monte Carlo method is used with trial wave functions for paramagnetic or normal (PM), superconducting with dx2-y2-wave (d-SC), isotropic s-wave, and extended s-wave symmetries, and antiferromagnetic (AF) states. The excited states are generated by imposing a minimum number of doubly occupied sites (doublons) D_L on the lowest-energy states. For U>W (W: band width), d_L=D_L/Ns (Ns: number of sites) corresponds to the excitation intensity. It is found that the AF state is the most stable among the states we treated for d_L<0.14 and insulating. The PM and d-SC states become conductive over a threshold d_Lc, and the conduction is caused by unbound doublons and holons (empty sites). The PM state arises for d_L>0.14, but the d-SC state is always hidden by the AF state. The s-wave-type superconducting states are not stabilized for any parameter set.


title: Rethinking Normalization Methods in Federated Learning

abstract: Federated learning (FL) is a popular distributed learning framework that can reduce privacy risks by not explicitly sharing private data. In this work, we explicitly uncover external covariate shift problem in FL, which is caused by the independent local training processes on different devices. We demonstrate that external covariate shifts will lead to the obliteration of some devices' contributions to the global model. Further, we show that normalization layers are indispensable in FL since their inherited properties can alleviate the problem of obliterating some devices' contributions. However, recent works have shown that batch normalization, which is one of the standard components in many deep neural networks, will incur accuracy drop of the global model in FL. The essential reason for the failure of batch normalization in FL is poorly studied. We unveil that external covariate shift is the key reason why batch normalization is ineffective in FL. We also show that layer normalization is a better choice in FL which can mitigate the external covariate shift and improve the performance of the global model. We conduct experiments on CIFAR10 under non-IID settings. The results demonstrate that models with layer normalization converge fastest and achieve the best or comparable accuracy for three different model architectures.


title: Air-sea interactions on Titan: effect of radiative transfer on the lake evaporation and atmospheric circulation

abstract: Titan's northern high latitudes host many large hydrocarbon lakes. Like water lakes on Earth, Titan's lakes are constantly subject to evaporation. This process strongly affects the atmospheric methane abundance, the atmospheric temperature, the lake mixed layer temperature, and the local wind circulation. In this work we use a 2D atmospheric mesoscale model coupled to a slab lake model to investigate the effect of solar and infrared radiation on the exchange of energy and methane between Titan's lakes and atmosphere. The magnitude of solar radiation reaching the surface of Titan through its thick atmosphere is only a few $Wm^{-2}$. However, we find that this small energy input is important and is comparable in absolute magnitude to the latent and sensible heat fluxes, as suggested in the prior study by Rafkin and Soto (2020). The implementation of a gray radiative scheme in the model confirms the importance of radiation when studying lakes at the surface of Titan. Solar and infrared radiation change the energy balance of the system leading to an enhancement of the methane evaporation rate, an increase of the equilibrium lake temperature almost completely determined by its environment (humidity, insolation, and background wind), and a strengthening of the local sea breeze, which undergoes diurnal variations. The sea breeze efficiently transports methane vapor horizontally, from the lake to the land, and vertically due to rising motion along the sea breeze front and due to radiation-induced turbulence over the land.


title: Extended water wave systems of Boussinesq equations on a finite interval: Theory and numerical analysis

abstract: Considered here is a class of Boussinesq systems of Nwogu type. Such systems describe propagation of nonlinear and dispersive water waves of significant interest such as solitary and tsunami waves. The initial-boundary value problem on a finite interval for this family of systems is studied both theoretically and numerically. First, the linearization of a certain generalized Nwogu system is solved analytically via the unified transform of Fokas. The corresponding analysis reveals two types of admissible boundary conditions, thereby suggesting appropriate boundary conditions for the nonlinear Nwogu system on a finite interval. Then, well-posedness is established, both in the weak and in the classical sense, for a regularized Nwogu system in the context of an initial-boundary value problem that describes the dynamics of water waves in a basin with wall-boundary conditions. In addition, a new modified Galerkin method is suggested for the numerical discretization of this regularized system in time, and its convergence is proved along with optimal error estimates. Finally, numerical experiments illustrating the effect of the boundary conditions on the reflection of solitary waves by a vertical wall are also provided.


title: Real-Time Navigation for Bipedal Robots in Dynamic Environments

abstract: The popularity of mobile robots has been steadily growing, with these robots being increasingly utilized to execute tasks previously completed by human workers. For bipedal robots to see this same success, robust autonomous navigation systems need to be developed that can execute in real-time and respond to dynamic environments. These systems can be divided into three stages: perception, planning, and control. A holistic navigation framework for bipedal robots must successfully integrate all three components of the autonomous navigation problem to enable robust real-world navigation. In this paper, we present a real-time navigation framework for bipedal robots in dynamic environments. The proposed system addresses all components of the navigation problem: We introduce a depth-based perception system for obstacle detection, mapping, and localization. A two-stage planner is developed to generate collision-free trajectories robust to unknown and dynamic environments. And execute trajectories on the Digit bipedal robot's walking gait controller. The navigation framework is validated through a series of simulation and hardware experiments that contain unknown environments and dynamic obstacles.


title: Automatic Prediction of Rejected Edits in Stack Overflow

abstract: The content quality of shared knowledge in Stack Overflow (SO) is crucial in supporting software developers with their programming problems. Thus, SO allows its users to suggest edits to improve the quality of a post (i.e., question and answer). However, existing research shows that many suggested edits in SO are rejected due to undesired contents/formats or violating edit guidelines. Such a scenario frustrates or demotivates users who would like to conduct good-quality edits. Therefore, our research focuses on assisting SO users by offering them suggestions on how to improve their editing of posts. First, we manually investigate 764 (382 questions + 382 answers) rejected edits by rollbacks and produce a catalog of 19 rejection reasons. Second, we extract 15 texts and user-based features to capture those rejection reasons. Third, we develop four machine learning models using those features. Our best-performing model can predict rejected edits with 69.1% precision, 71.2% recall, 70.1% F1-score, and 69.8% overall accuracy. Fourth, we introduce an online tool named EditEx that works with the SO edit system. EditEx can assist users while editing posts by suggesting the potential causes of rejections. We recruit 20 participants to assess the effectiveness of EditEx. Half of the participants (i.e., treatment group) use EditEx and another half (i.e., control group) use the SO standard edit system to edit posts. According to our experiment, EditEx can support SO standard edit system to prevent 49% of rejected edits, including the commonly rejected ones. However, it can prevent 12% rejections even in free-form regular edits. The treatment group finds the potential rejection reasons identified by EditEx influential. Furthermore, the median workload suggesting edits using EditEx is half compared to the SO edit system.


title: Set2Box: Similarity Preserving Representation Learning of Sets

abstract: Sets have been used for modeling various types of objects (e.g., a document as the set of keywords in it and a customer as the set of the items that she has purchased). Measuring similarity (e.g., Jaccard Index) between sets has been a key building block of a wide range of applications, including, plagiarism detection, recommendation, and graph compression. However, as sets have grown in numbers and sizes, the computational cost and storage required for set similarity computation have become substantial, and this has led to the development of hashing and sketching based solutions. In this work, we propose Set2Box, a learning-based approach for compressed representations of sets from which various similarity measures can be estimated accurately in constant time. The key idea is to represent sets as boxes to precisely capture overlaps of sets. Additionally, based on the proposed box quantization scheme, we design Set2Box+, which yields more concise but more accurate box representations of sets. Through extensive experiments on 8 real-world datasets, we show that, compared to baseline approaches, Set2Box+ is (a) Accurate: achieving up to 40.8X smaller estimation error while requiring 60% fewer bits to encode sets, (b) Concise: yielding up to 96.8X more concise representations with similar estimation error, and (c) Versatile: enabling the estimation of four set-similarity measures from a single representation of each set.


title: Design Amortization for Bayesian Optimal Experimental Design

abstract: Bayesian optimal experimental design is a sub-field of statistics focused on developing methods to make efficient use of experimental resources. Any potential design is evaluated in terms of a utility function, such as the (theoretically well-justified) expected information gain (EIG); unfortunately however, under most circumstances the EIG is intractable to evaluate. In this work we build off of successful variational approaches, which optimize a parameterized variational model with respect to bounds on the EIG. Past work focused on learning a new variational model from scratch for each new design considered. Here we present a novel neural architecture that allows experimenters to optimize a single variational model that can estimate the EIG for potentially infinitely many designs. To further improve computational efficiency, we also propose to train the variational model on a significantly cheaper-to-evaluate lower bound, and show empirically that the resulting model provides an excellent guide for more accurate, but expensive to evaluate bounds on the EIG. We demonstrate the effectiveness of our technique on generalized linear models, a class of statistical models that is widely used in the analysis of controlled experiments. Experiments show that our method is able to greatly improve accuracy over existing approximation strategies, and achieve these results with far better sample efficiency.


title: Milnor operations and classifying spaces

abstract: We give an example of a nonzero odd degree element of the classifying space of a connected Lie group such that all higher Milnor operations vanish on it. It is a counterexample for a conjecture of Kono and Yagita.


title: Improved Caffarelli-Kohn-Nirenberg Inequalities and Uncertainty Principle

abstract: In this paper we prove some improved Caffarelli-Kohn-Nirenberg inequalities and uncertainty principle for complex- and vector-valued functions on $\mathbb R^n$, which is a further study of the results in \cite{Dang-Deng-Qian}. In particular, we introduce an analogue of "phase derivative" for vector-valued functions. Moreover, using the introduced "phase derivative", we extend the extra-strong uncertainty principle to cases for complex- and vector-valued functions defined on $\mathbb S^n,n\geq 2.$


title: Perspectives on a 6G Architecture

abstract: Mobile communications have been undergoing a generational change every ten years. Whilst we are just beginning to roll out 5G networks, significant efforts are planned to standardize 6G that is expected to be commercially introduced by 2030. This paper looks at the use cases for 6G and their impact on the network architecture to meet the anticipated performance requirements. The new architecture is based on integrating various network functions in virtual cloud environments, leveraging the advancement of artificial intelligence in all domains, integrating different sub-networks constituting the 6G system, and on enhanced means of exposing data and services to third parties.


title: Dirichlet Problem for Degenerate Fractional Parabolic Hyperbolic Equations

abstract: We are concerned in this paper with the degenerate fractional diffusion advection equations posed in bounded domains. Due to a suitable formulation, we show the existence of weak entropy solutions for measurable and bounded initial and Dirichlet boundary data. Moreover, we prove a $L^1-$type contraction property for weak entropy solutions obtained via parabolic perturbation. This is a weak selection principle which means that the weak entropy solutions are stable in this class.


title: Scientific Paper Classification Based on Graph Neural Network with Hypergraph Self-attention Mechanism

abstract: The number of scientific papers has increased rapidly in recent years. How to make good use of scientific papers for research is very important. Through the high-quality classification of scientific papers, researchers can quickly find the resource content they need from the massive scientific resources. The classification of scientific papers will effectively help researchers filter redundant information, obtain search results quickly and accurately, and improve the search quality, which is necessary for scientific resource management. This paper proposed a science-technique paper classification method based on hypergraph neural network(SPHNN). In the heterogeneous information network of scientific papers, the repeated high-order subgraphs are modeled as hyperedges composed of multiple related nodes. Then the whole heterogeneous information network is transformed into a hypergraph composed of different hyperedges. The graph convolution operation is carried out on the hypergraph structure, and the hyperedges self-attention mechanism is introduced to aggregate different types of nodes in the hypergraph, so that the final node representation can effectively maintain high-order nearest neighbor relationships and complex semantic information. Finally, by comparing with other methods, we proved that the model proposed in this paper has improved its performance.


title: Scalable Self-Supervised Representation Learning from Spatiotemporal Motion Trajectories for Multimodal Computer Vision

abstract: Self-supervised representation learning techniques utilize large datasets without semantic annotations to learn meaningful, universal features that can be conveniently transferred to solve a wide variety of downstream supervised tasks. In this work, we propose a self-supervised method for learning representations of geographic locations from unlabeled GPS trajectories to solve downstream geospatial computer vision tasks. Tiles resulting from a raster representation of the earth's surface are modeled as nodes on a graph or pixels of an image. GPS trajectories are modeled as allowed Markovian paths on these nodes. A scalable and distributed algorithm is presented to compute image-like representations, called reachability summaries, of the spatial connectivity patterns between tiles and their neighbors implied by the observed Markovian paths. A convolutional, contractive autoencoder is trained to learn compressed representations, called reachability embeddings, of reachability summaries for every tile. Reachability embeddings serve as task-agnostic, feature representations of geographic locations. Using reachability embeddings as pixel representations for five different downstream geospatial tasks, cast as supervised semantic segmentation problems, we quantitatively demonstrate that reachability embeddings are semantically meaningful representations and result in 4-23% gain in performance, as measured using area under the precision-recall curve (AUPRC) metric, when compared to baseline models that use pixel representations that do not account for the spatial connectivity between tiles. Reachability embeddings transform sequential, spatiotemporal mobility data into semantically meaningful tensor representations that can be combined with other sources of imagery and are designed to facilitate multimodal learning in geospatial computer vision.


title: Embedding Representation of Academic Heterogeneous Information Networks Based on Federated Learning

abstract: Academic networks in the real world can usually be portrayed as heterogeneous information networks (HINs) with multi-type, universally connected nodes and multi-relationships. Some existing studies for the representation learning of homogeneous information networks cannot be applicable to heterogeneous information networks because of the lack of ability to issue heterogeneity. At the same time, data has become a factor of production, playing an increasingly important role. Due to the closeness and blocking of businesses among different enterprises, there is a serious phenomenon of data islands. To solve the above challenges, aiming at the data information of scientific research teams closely related to science and technology, we proposed an academic heterogeneous information network embedding representation learning method based on federated learning (FedAHE), which utilizes node attention and meta path attention mechanism to learn low-dimensional, dense and real-valued vector representations while preserving the rich topological information and meta-path-based semantic information of nodes in network. Moreover, we combined federated learning with the representation learning of HINs composed of scientific research teams and put forward a federal training mechanism based on dynamic weighted aggregation of parameters (FedDWA) to optimize the node embeddings of HINs. Through sufficient experiments, the efficiency, accuracy and feasibility of our proposed framework are demonstrated.


title: A Relational Triple Extraction Method Based on Feature Reasoning for Technological Patents

abstract: The relation triples extraction method based on table filling can address the issues of relation overlap and bias propagation. However, most of them only establish separate table features for each relationship, which ignores the implicit relationship between different entity pairs and different relationship features. Therefore, a feature reasoning relational triple extraction method based on table filling for technological patents is proposed to explore the integration of entity recognition and entity relationship, and to extract entity relationship triples from multi-source scientific and technological patents data. Compared with the previous methods, the method we proposed for relational triple extraction has the following advantages: 1) The table filling method that saves more running space enhances the speed and efficiency of the model. 2) Based on the features of existing token pairs and table relations, reasoning the implicit relationship features, and improve the accuracy of triple extraction. On five benchmark datasets, we evaluated the model we suggested. The result suggest that our model is advanced and effective, and it performed well on most of these datasets.


title: Unsupervised Semantic Representation Learning of Scientific Literature Based on Graph Attention Mechanism and Maximum Mutual Information

abstract: Since most scientific literature data are unlabeled, this makes unsupervised graph-based semantic representation learning crucial. Therefore, an unsupervised semantic representation learning method of scientific literature based on graph attention mechanism and maximum mutual information (GAMMI) is proposed. By introducing a graph attention mechanism, the weighted summation of nearby node features make the weights of adjacent node features entirely depend on the node features. Depending on the features of the nearby nodes, different weights can be applied to each node in the graph. Therefore, the correlations between vertex features can be better integrated into the model. In addition, an unsupervised graph contrastive learning strategy is proposed to solve the problem of being unlabeled and scalable on large-scale graphs. By comparing the mutual information between the positive and negative local node representations on the latent space and the global graph representation, the graph neural network can capture both local and global information. Experimental results demonstrate competitive performance on various node classification benchmarks, achieving good results and sometimes even surpassing the performance of supervised learning.


title: PeF: Poisson's Equation Based Large-Scale Fixed-Outline Floorplanning

abstract: Floorplanning is the first stage of VLSI physical design. An effective floorplanning engine definitely has positive impact on chip design speed, quality and performance. In this paper, we present a novel mathematical model to characterize non-overlapping of modules, and propose a flat fixed-outline floorplanning algorithm based on the VLSI global placement approach using Poisson's equation. The algorithm consists of global floorplanning and legalization phases. In global floorplanning, we redefine the potential energy of each module based on the novel mathematical model for characterizing non-overlapping of modules and an analytical solution of Poisson's equation. In this scheme, the widths of soft modules appear as variables in the energy function and can be optimized. Moreover, we design a fast approximate computation scheme for partial derivatives of the potential energy. In legalization, based on the defined horizontal and vertical constraint graphs, we eliminate overlaps between modules remained after global floorplanning, by modifying relative positions of modules. Experiments on the MCNC, GSRC, HB+ and ami49\_x benchmarks show that, our algorithm improves the average wirelength by at least 2\% and 5\% on small and large scale benchmarks with certain whitespace, respectively, compared to state-of-the-art floorplanners.


title: Understanding Edge-of-Stability Training Dynamics with a Minimalist Example

abstract: Recently, researchers observed that gradient descent for deep neural networks operates in an ``edge-of-stability'' (EoS) regime: the sharpness (maximum eigenvalue of the Hessian) is often larger than stability threshold $2/\eta$ (where $\eta$ is the step size). Despite this, the loss oscillates and converges in the long run, and the sharpness at the end is just slightly below $2/\eta$. While many other well-understood nonconvex objectives such as matrix factorization or two-layer networks can also converge despite large sharpness, there is often a larger gap between sharpness of the endpoint and $2/\eta$. In this paper, we study EoS phenomenon by constructing a simple function that has the same behavior. We give rigorous analysis for its training dynamics in a large local region and explain why the final converging point has sharpness close to $2/\eta$. Globally we observe that the training dynamics for our example has an interesting bifurcating behavior, which was also observed in the training of neural nets.


title: Scientific and Technological News Recommendation Based on Knowledge Graph with User Perception

abstract: Existing research usually utilizes side information such as social network or item attributes to improve the performance of collaborative filtering-based recommender systems. In this paper, the knowledge graph with user perception is used to acquire the source of side information. We proposed KGUPN to address the limitations of existing embedding-based and path-based knowledge graph-aware recommendation methods, an end-to-end framework that integrates knowledge graph and user awareness into scientific and technological news recommendation systems. KGUPN contains three main layers, which are the propagation representation layer, the contextual information layer and collaborative relation layer. The propagation representation layer improves the representation of an entity by recursively propagating embeddings from its neighbors (which can be users, news, or relationships) in the knowledge graph. The contextual information layer improves the representation of entities by encoding the behavioral information of entities appearing in the news. The collaborative relation layer complements the relationship between entities in the news knowledge graph. Experimental results on real-world datasets show that KGUPN significantly outperforms state-of-the-art baselines in scientific and technological news recommendation.


title: GMA3D: Local-Global Attention Learning to Estimate Occluded Motions of Scene Flow

abstract: Scene flow represents the motion information of each point in the 3D point clouds. It is a vital downstream method applied to many tasks, such as motion segmentation and object tracking. However, there are always occlusion points between two consecutive point clouds, whether from the sparsity data sampling or real-world occlusion. In this paper, we focus on addressing occlusion issues in scene flow by the semantic self-similarity and motion consistency of the moving objects. We propose a GMA3D module based on the transformer framework, which utilizes local and global semantic similarity to infer the motion information of occluded points from the motion information of local and global non-occluded points respectively, and then uses an offset aggregator to aggregate them. Our module is the first to apply the transformer-based architecture to gauge the scene flow occlusion problem on point clouds. Experiments show that our GMA3D can solve the occlusion problem in the scene flow, especially in the real scene. We evaluated the proposed method on the occluded version of point cloud datasets and get state-of-the-art results on the real scene KITTI dataset. To testify that GMA3D is still beneficial to non-occluded scene flow, we also conducted experiments on non-occluded version datasets and achieved promising performance on FlyThings3D and KITTI. The code is available at https://anonymous.4open.science/r/GMA3D-E100.


title: Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systems

abstract: Decision-based attacks construct adversarial examples against a machine learning (ML) model by making only hard-label queries. These attacks have mainly been applied directly to standalone neural networks. However, in practice, ML models are just one component of a larger learning system. We find that by adding a single preprocessor in front of a classifier, state-of-the-art query-based attacks are up to 7$\times$ less effective at attacking a prediction pipeline than at attacking the model alone. We explain this discrepancy by the fact that most preprocessors introduce some notion of invariance to the input space. Hence, attacks that are unaware of this invariance inevitably waste a large number of queries to re-discover or overcome it. We, therefore, develop techniques to (i) reverse-engineer the preprocessor and then (ii) use this extracted information to attack the end-to-end system. Our preprocessors extraction method requires only a few hundred queries, and our preprocessor-aware attacks recover the same efficacy as when attacking the model alone. The code can be found at https://github.com/google-research/preprocessor-aware-black-box-attack.


title: Simulation of Transients in Natural Gas Networks via A Semi-analytical Solution Approach

abstract: Simulation and control of the transient flow in natural gas networks involve solving partial differential equations (PDEs). This paper proposes a semi-analytical solutions (SAS) approach for fast and accurate simulation of the natural gas transients. The region of interest is divided into a grid, and an SAS is derived for each grid cell in the form of the multivariate polynomials, of which the coefficients are identified according to the initial value and boundary value conditions. The solutions are solved in a ``time-stepping'' manner; that is, within one time step, the coefficients of the SAS are identified and the initial value of the next time step is evaluated. This approach achieves a much larger grid cell than the widely used finite difference method, and thus enhances the computational efficiency significantly. To further reduce the computation burden, the nonlinear terms in the model are simplified, which induces another SAS scheme that can greatly reduce the time consumption and have minor impact on accuracy. The simulation results on a single pipeline case and a 6-node network case validate the advantages of the proposed SAS approach in accuracy and computational efficiency.


title: A Learning-based Framework for Topology-Preserving Segmentation using Quasiconformal Mappings

abstract: We propose the Topology-Preserving Segmentation Network, a deformation-based model that can extract objects in an image while maintaining their topological properties. This network generates segmentation masks that have the same topology as the template mask, even when trained with limited data. The network consists of two components: the Deformation Estimation Network, which produces a deformation map that warps the template mask to enclose the region of interest, and the Beltrami Adjustment Module, which ensures the bijectivity of the deformation map by truncating the associated Beltrami coefficient based on Quasiconformal theories. The proposed network can also be trained in an unsupervised manner, eliminating the need for labeled training data. This is achieved by incorporating an unsupervised segmentation loss. Our experimental results on various image datasets show that TPSN achieves better segmentation accuracy than state-of-the-art models with correct topology. Furthermore, we demonstrate TPSN's ability to handle multiple object segmentation.


title: Multi-Robot Localization and Target Tracking with Connectivity Maintenance and Collision Avoidance

abstract: We study the problem that requires a team of robots to perform joint localization and target tracking task while ensuring team connectivity and collision avoidance. The problem can be formalized as a nonlinear, non-convex optimization program, which is typically hard to solve. To this end, we design a two-staged approach that utilizes a greedy algorithm to optimize the joint localization and target tracking performance and applies control barrier functions to ensure safety constraints, i.e., maintaining connectivity of the robot team and preventing inter-robot collisions. Simulated Gazebo experiments verify the effectiveness of the proposed approach. We further compare our greedy algorithm to a non-linear optimization solver and a random algorithm, in terms of the joint localization and tracking quality as well as the computation time. The results demonstrate that our greedy algorithm achieves high task quality and runs efficiently.


title: GOLLIC: Learning Global Context beyond Patches for Lossless High-Resolution Image Compression

abstract: Neural-network-based approaches recently emerged in the field of data compression and have already led to significant progress in image compression, especially in achieving a higher compression ratio. In the lossless image compression scenario, however, existing methods often struggle to learn a probability model of full-size high-resolution images due to the limitation of the computation source. The current strategy is to crop high-resolution images into multiple non-overlapping patches and process them independently. This strategy ignores long-term dependencies beyond patches, thus limiting modeling performance. To address this problem, we propose a hierarchical latent variable model with a global context to capture the long-term dependencies of high-resolution images. Besides the latent variable unique to each patch, we introduce shared latent variables between patches to construct the global context. The shared latent variables are extracted by a self-supervised clustering module inside the model's encoder. This clustering module assigns each patch the confidence that it belongs to any cluster. Later, shared latent variables are learned according to latent variables of patches and their confidence, which reflects the similarity of patches in the same cluster and benefits the global context modeling. Experimental results show that our global context model improves compression ratio compared to the engineered codecs and deep learning models on three benchmark high-resolution image datasets, DIV2K, CLIC.pro, and CLIC.mobile.


title: A method for constructing Weierstrass elliptic function solutions and their degenerated solutions of the mKdV equation

abstract: A Weierstrass type projective Riccati equation expansion method is proposed by using the Weierstrass elliptic function solutions of the projective Riccati equations and the conversion formulas which transform the Weierstrass elliptic functions into the hyperbolic and the trigonometric functions. The Weierstrass elliptic function solutions and their degenerated solutions including the solitary wave and the periodic wave solutions of the mKdV equation are obtained by using the proposed method.


title: Data-driven Approach to Differentiating between Depression and Dementia from Noisy Speech and Language Data

abstract: A significant number of studies apply acoustic and linguistic characteristics of human speech as prominent markers of dementia and depression. However, studies on discriminating depression from dementia are rare. Co-morbid depression is frequent in dementia and these clinical conditions share many overlapping symptoms, but the ability to distinguish between depression and dementia is essential as depression is often curable. In this work, we investigate the ability of clustering approaches in distinguishing between depression and dementia from human speech. We introduce a novel aggregated dataset, which combines narrative speech data from multiple conditions, i.e., Alzheimer's disease, mild cognitive impairment, healthy control, and depression. We compare linear and non-linear clustering approaches and show that non-linear clustering techniques distinguish better between distinct disease clusters. Our interpretability analysis shows that the main differentiating symptoms between dementia and depression are acoustic abnormality, repetitiveness (or circularity) of speech, word finding difficulty, coherence impairment, and differences in lexical complexity and richness.


title: Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding

abstract: Automatic International Classification of Diseases (ICD) coding aims to assign multiple ICD codes to a medical note with average length of 3,000+ tokens. This task is challenging due to a high-dimensional space of multi-label assignment (tens of thousands of ICD codes) and the long-tail challenge: only a few codes (common diseases) are frequently assigned while most codes (rare diseases) are infrequently assigned. This study addresses the long-tail challenge by adapting a prompt-based fine-tuning technique with label semantics, which has been shown to be effective under few-shot setting. To further enhance the performance in medical domain, we propose a knowledge-enhanced longformer by injecting three domain-specific knowledge: hierarchy, synonym, and abbreviation with additional pretraining using contrastive learning. Experiments on MIMIC-III-full, a benchmark dataset of code assignment, show that our proposed method outperforms previous state-of-the-art method in 14.5% in marco F1 (from 10.3 to 11.8, P<0.001). To further test our model on few-shot setting, we created a new rare diseases coding dataset, MIMIC-III-rare50, on which our model improves marco F1 from 17.1 to 30.4 and micro F1 from 17.2 to 32.6 compared to previous method.


title: How Do Data Science Workers Communicate Intermediate Results?

abstract: Data science workers increasingly collaborate on large-scale projects before communicating insights to a broader audience in the form of visualization. While prior work has modeled how data science teams, oftentimes with distinct roles and work processes, communicate knowledge to outside stakeholders, we have little knowledge of how data science workers communicate intermediately before delivering the final products. In this work, we contribute a nuanced description of the intermediate communication process within data science teams. By analyzing interview data with 8 self-identified data science workers, we characterized the data science intermediate communication process with four factors, including the types of audience, communication goals, shared artifacts, and mode of communication. We also identified overarching challenges in the current communication process. We also discussed design implications that might inform better tools that facilitate intermediate communication within data science teams.


title: Excited doubly heavy baryons production via top-quark decays

abstract: Within the framework of NRQCD, we calculate the production of excited doubly heavy baryons $\Xi_{bQ}$ through the semi-inclusive production process $t\rightarrow \langle bQ\rangle[n]\rightarrow \Xi_{bQ}+ \bar {Q} + W^+ $, where $Q= b$ or $c$ quark. The intermediate diquark state $\langle bQ\rangle[n]$ is in the excited $P$-wave state, including $[^1P_1]$ and $[^3P_J]$ ($J=$0, 1 or 2) in both color antitriplet state $\mathbf{\mathbf{\overline 3}}$ and color sixtuplet state $\mathbf{6}$, that is, $\langle bc\rangle[^{1}P_{1}]_{\mathbf{\overline 3}/ \mathbf{6}}$, $\langle bc\rangle[^{3}P_{J}]_{\mathbf{\overline 3}/ \mathbf{6}}$, $\langle bb\rangle[^{1}P_{1}]_{\mathbf{\overline 3}}$, and $\langle bb\rangle[^{3}P_{J}]_{\mathbf{6}}$. We find that the contributions from the P-wave states are about one order lower than the S-wave contributions, and this conclusion is consistent with others. We also analyze the invariant mass and angle differential distributions, and the theoretical uncertainty from the mass parameters and the renormalization scale. Finally, we can expect that about $1.14 \times10^{3-5}$ events of excited $\Xi_{bc}$ and $2.47 \times10^{1-3}$ events of excited $\Xi_{bb}$ can be produced per year at the LHC or HL-LHC with $\mathcal{L}$ =$10^{34-36}~\rm{cm}^{-2}~\rm{s}^{-1}$.


title: From the $B$-Toda to the BKP hierarchy

abstract: It is shown that all $\tau$-functions of BKP hierarchy can be written as Pfaffians of skew-symmetric matrices. $\tau$-functions of BKP hierarchy are parameterized by points in the universal orthogonal Grassmannian manifold (UOGM). The UOGM is a disjoint union of Schubert cells, we classify and give explicit parameterization for points in each Schubert cell by constructing a frame for UOGM in the sense of Sato. $\tau$-functions are then expressed in terms of these frames and Schur-Q functions. For concreteness we give a comprehensive study for the $\tau$-functions of $B$-Toda which can be viewed as a finite version of the BKP hierarchy. Along the way we also give a constructive description for complex pure spinors du E. Cartan. As an application of our construction, we reprove a theorem due to A. Alexandrov which states that KdV solves BKP up to rescaling of the time parameters by $2$. We prove this by showing that the KdV hierarchy can be viewed as $4$-reduction of the BKP hierarchy. This interpretation gives complete characterization for the KdV orbits inside the BKP hierarchy. Other than a few facts from representation theory, the main tools we use to show the above results, however, are surprisingly simple linear algebra.

categories: nlin.SI math-ph math.CO math.DS math.MP math.RT


title: Generative Augmented Flow Networks

abstract: The Generative Flow Network is a probabilistic framework where an agent learns a stochastic policy for object generation, such that the probability of generating an object is proportional to a given reward function. Its effectiveness has been shown in discovering high-quality and diverse solutions, compared to reward-maximizing reinforcement learning-based methods. Nonetheless, GFlowNets only learn from rewards of the terminal states, which can limit its applicability. Indeed, intermediate rewards play a critical role in learning, for example from intrinsic motivation to provide intermediate feedback even in particularly challenging sparse reward tasks. Inspired by this, we propose Generative Augmented Flow Networks (GAFlowNets), a novel learning framework to incorporate intermediate rewards into GFlowNets. We specify intermediate rewards by intrinsic motivation to tackle the exploration problem in sparse reward environments. GAFlowNets can leverage edge-based and state-based intrinsic rewards in a joint way to improve exploration. Based on extensive experiments on the GridWorld task, we demonstrate the effectiveness and efficiency of GAFlowNet in terms of convergence, performance, and diversity of solutions. We further show that GAFlowNet is scalable to a more complex and large-scale molecule generation domain, where it achieves consistent and significant performance improvement.


title: Localization for general Helmholtz

abstract: In \cite{gmw2022}, Guan, Murugan and Wei established the equivalence of the classical Helmholtz equation with a ``fractional Helmholtz" equation in which the Laplacian operator is replaced by the nonlocal fractional Laplacian operator. More general equivalence results are obtained for symbols which are complete Bernstein and satisfy additional regularity conditions. In this work we introduce a novel and general set-up for this Helmholtz equivalence problem. We show that under very mild and easy-to-check conditions on the Fourier multiplier, the general Helmholtz equation can be effectively reduced to a localization statement on the support of the symbol.


title: Scaling Forward Gradient With Local Losses

abstract: Forward gradient learning computes a noisy directional gradient and is a biologically plausible alternative to backprop for learning deep neural networks. However, the standard forward gradient algorithm, when applied naively, suffers from high variance when the number of parameters to be learned is large. In this paper, we propose a series of architectural and algorithmic modifications that together make forward gradient learning practical for standard deep learning benchmark tasks. We show that it is possible to substantially reduce the variance of the forward gradient estimator by applying perturbations to activations rather than weights. We further improve the scalability of forward gradient by introducing a large number of local greedy loss functions, each of which involves only a small number of learnable parameters, and a new MLPMixer-inspired architecture, LocalMixer, that is more suitable for local learning. Our approach matches backprop on MNIST and CIFAR-10 and significantly outperforms previously proposed backprop-free algorithms on ImageNet.


title: The trace of uniform hypergraphs with application to Estrada index

abstract: In this paper we investigate the traces of the adjacency tensor of hypergraphs (simply called the traces of hypergraphs). We give new expressions for the traces of hypertrees and linear unicyclic hypergraphs by the weight function assigned to their connected sub-hypergraphs, and provide some perturbation results for the traces of a hypergraph with cut vertices. As applications we determine the unique hypertree with maximum Estrada index among all hypertrees with fixed number of edges and perfect matchings, and the unique unicyclic hypergraph with maximum Estrada index among all unicyclic hypergraph with fixed number of edges and girth $3$.


title: Distillation-Resistant Watermarking for Model Protection in NLP

abstract: How can we protect the intellectual property of trained NLP models? Modern NLP models are prone to stealing by querying and distilling from their publicly exposed APIs. However, existing protection methods such as watermarking only work for images but are not applicable to text. We propose Distillation-Resistant Watermarking (DRW), a novel technique to protect NLP models from being stolen via distillation. DRW protects a model by injecting watermarks into the victim's prediction probability corresponding to a secret key and is able to detect such a key by probing a suspect model. We prove that a protected model still retains the original accuracy within a certain bound. We evaluate DRW on a diverse set of NLP tasks including text classification, part-of-speech tagging, and named entity recognition. Experiments show that DRW protects the original model and detects stealing suspects at 100% mean average precision for all four tasks while the prior method fails on two.


title: Slope parameters determined from CREX and PREX2

abstract: [Background] Very lately, the CREX group presents a skin value $\Delta R_{\rm skin}^{48}({\rm CREX}) =0.121 \pm 0.026\ {\rm (exp)} \pm 0.024\ {\rm (model)}=0.071\sim 0.171$~fm. Meanwhile, the PREX group reported a skin value $\Delta R_{\rm skin}^{208}({\rm PREX2}) = 0.283\pm 0.071=0.212 \sim 0.354$~fm. In our previous paper, we determined both the $L$--$\Delta R_{\rm skin}^{48}$ relation and the $L$--$\Delta R_{\rm skin}^{208}$ one, using 206 EoSs, where $L$ is a slope parameter. [Purpose] We determine $L$ from $\Delta R_{\rm skin}^{48}({\rm CREX})$ and $\Delta R_{\rm skin}^{208}({\rm PREX2}) $, using 207 EoSs. [Results] The $\Delta R_{\rm skin}^{48}({\rm CREX})$ yields $L({\rm CREX})=0 \sim 51$~MeV and the $\Delta R_{\rm skin}^{208}({\rm PREX2}) $ does $L({\rm PREX2})=76 \sim 165$~MeV. [Conclusion] There is no overlap between $L({\rm CREX})$ and $L({\rm PREX2})$. This is a big problem to be solved.


title: Uniformly convex neural networks and non-stationary iterated network Tikhonov (iNETT) method

abstract: We propose a non-stationary iterated network Tikhonov (iNETT) method for the solution of ill-posed inverse problems. The iNETT employs deep neural networks to build a data-driven regularizer, and it avoids the difficult task of estimating the optimal regularization parameter. To achieve the theoretical convergence of iNETT, we introduce uniformly convex neural networks to build the data-driven regularizer. Rigorous theories and detailed algorithms are proposed for the construction of convex and uniformly convex neural networks. In particular, given a general neural network architecture, we prescribe sufficient conditions to achieve a trained neural network which is component-wise convex or uniformly convex; moreover, we provide concrete examples of realizing convexity and uniform convexity in the modern U-net architecture. With the tools of convex and uniformly convex neural networks, the iNETT algorithm is developed and a rigorous convergence analysis is provided. Lastly, we show applications of the iNETT algorithm in 2D computerized tomography, where numerical examples illustrate the efficacy of the proposed algorithm.