title: Stochastic Image Transmission with CoAP for Extreme Environments

abstract: Communication in extreme environments is an important research topic for various use cases including environmental monitoring. A typical example is underwater acoustic communication for 6G mobile networks. The major challenges in such environments are extremely high-latency and high-error rate. They make real-time image transmission difficult using existing communication protocols. This is partly because frequent retransmission in noisy networks increases latency and leads to serious deterioration of real-timeness. To address this problem, this paper proposes a stochastic image transmission with Constrained Application Protocol (CoAP) for extreme environments. The goal of the proposed idea is to achieve approximate real-time image transmission without retransmission using CoAP over UDP. To this end, an image is divided into blocks, and value is assigned for each block based on the requirement. By the stochastic transmission of blocks, the reception probability is guaranteed without retransmission even when packets are lost in networks. We implemented the proposed scheme using Raspberry Pi 4 to demonstrate the feasibility. The performance of the proposed image transmission was confirmed from the experimental results.


title: SMLT: A Serverless Framework for Scalable and Adaptive Machine Learning Design and Training

abstract: In today's production machine learning (ML) systems, models are continuously trained, improved, and deployed. ML design and training are becoming a continuous workflow of various tasks that have dynamic resource demands. Serverless computing is an emerging cloud paradigm that provides transparent resource management and scaling for users and has the potential to revolutionize the routine of ML design and training. However, hosting modern ML workflows on existing serverless platforms has non-trivial challenges due to their intrinsic design limitations such as stateless nature, limited communication support across function instances, and limited function execution duration. These limitations result in a lack of an overarching view and adaptation mechanism for training dynamics and an amplification of existing problems in ML workflows. To address the above challenges, we propose SMLT, an automated, scalable, and adaptive serverless framework to enable efficient and user-centric ML design and training. SMLT employs an automated and adaptive scheduling mechanism to dynamically optimize the deployment and resource scaling for ML tasks during training. SMLT further enables user-centric ML workflow execution by supporting user-specified training deadlines and budget limits. In addition, by providing an end-to-end design, SMLT solves the intrinsic problems in serverless platforms such as the communication overhead, limited function execution duration, need for repeated initialization, and also provides explicit fault tolerance for ML training. SMLT is open-sourced and compatible with all major ML frameworks. Our experimental evaluation with large, sophisticated modern ML models demonstrate that SMLT outperforms the state-of-the-art VM based systems and existing serverless ML training frameworks in both training speed (up to 8X) and monetary cost (up to 3X)


title: Robustly Complete Finite-State Abstractions for Verification of Stochastic Systems

abstract: In this paper, we focus on discrete-time stochastic systems modelled by nonlinear stochastic difference equations and propose robust abstractions for verifying probabilistic linear temporal specifications. The current literature focuses on developing sound abstraction techniques for stochastic dynamics without perturbations. However, soundness thus far has only been shown for preserving the satisfaction probability of certain types of temporal-logic specification. We present constructive finite-state abstractions for verifying probabilistic satisfaction of general {\omega}-regular linear-time properties of more general nonlinear stochastic systems. Instead of imposing stability assumptions, we analyze the probabilistic properties from the topological view of metrizable space of probability measures. Such abstractions are both sound and approximately complete. That is, given a concrete discrete-time stochastic system and an arbitrarily small L1-perturbation of this system, there exists a family of finite-state Markov chains whose set of satisfaction probabilities contains that of the original system and meanwhile is contained by that of the slightly perturbed system. A direct consequence is that, given a probabilistic linear-time specification, initializing within the winning/losing region of the abstraction system can guarantee a satisfaction/dissatisfaction for the original system. We make an interesting observation that, unlike the deterministic case, point-mass (Dirac) perturbations cannot fulfill the purpose of robust completeness.


title: The Effect of Multiple Imputation of Routine Pathology Variables on Laboratory Diagnosis of Hepatitis C Infection

abstract: Pathology tests are central to modern healthcare in terms of diagnosis and patient management. Aggregated pathology results provide opportunities for research into fundamental and applied questions in health and medicine, but data analytic challenges appear since test profiles vary between medical practitioners, resulting in missing data. In this study we provide an analytical investigation of the laboratory diagnosis of Hepatitis C (HCV) infection and focus on how to maximize the predictive value of routine pathology data. We recommend using the Influx - Outflux measures to help construct the imputation model when using multiple imputation. Data from 14,320 community-patients aged 15 - 100 years were accessed via ACT Pathology (The Canberra Hospital, Australia). Influx and Outflux were calculated to identify which variables were potentially powerful predictors of missing values. Available Case analysis and Multiple Imputation were used to accommodate missing values in the dataset. Logistic regression model and stepwise selection method were used for analysing the imputed datasets. The predictive power of all methods was compared. The predictive power of the models on multiply imputed data was similar to the power of the models based on complete data. The advantage of multiply imputed data was that it allowed for the inclusion of all the completed variables in the logistic models, thus identifying a broader selection of test results that could lead to the enhanced laboratory prediction of HCV. Multiple imputation is an important statistical resource allowing all individuals in a study to contribute whatever data they have supplied to the analysis. MI in combination with the values of Influx and Outflux identifies potential predictors of HepC infection. Variables age, gender and alanine aminotransferase have been shown to be strong laboratory predictors of HCV infection.


title: A generalization of Hardy's operator and an asymptotic Muntz-Szasz Theorem

abstract: The Hardy operator has all the monomial functions as eigenvectors. We study bounded operators on L^2 that take monomial functions to multiples of other monomials, with a shifted exponent. We prove that they all leave the space of functions vanishing on [0,s] invariant. We prove an asymptotic Muntz-Szasz theorem, characterizing the set of functions that are limits of linear combinations of monomials with exponents between n and 2n.


title: Monomial Operators

abstract: We study monomial operators on $ L^2[0,1]$, that is bounded linear operators that map each monomial $x^n$ to a multiple of $x^{p_n}$ for some $p_n$. We show that they are all unitarily equivalent to weighted composition operators on a Hardy space. We characterize what sequences $p_n$ can arise. In the case that $p_n$ is a fixed translation of $n$, we give a criterion for boundedness of the operator.


title: DeeptDCS: Deep Learning-Based Estimation of Currents Induced During Transcranial Direct Current Stimulation

abstract: Objective: Transcranial direct current stimulation (tDCS) is a non-invasive brain stimulation technique used to generate conduction currents in the head and disrupt brain functions. To rapidly evaluate the tDCS-induced current density in near real-time, this paper proposes a deep learning-based emulator, named DeeptDCS. Methods: The emulator leverages Attention U-net taking the volume conductor models (VCMs) of head tissues as inputs and outputting the three-dimensional current density distribution across the entire head. The electrode configurations are also incorporated into VCMs without increasing the number of input channels; this enables the straightforward incorporation of the non-parametric features of electrodes (e.g., thickness, shape, size, and position) in the training and testing of the proposed emulator. Results: Attention U-net outperforms standard U-net and its other three variants (Residual U-net, Attention Residual U-net, and Multi-scale Residual U-net) in terms of accuracy. The generalization ability of DeeptDCS to non-trained electrode configurations can be greatly enhanced through fine-tuning the model. The computational time required by one emulation via DeeptDCS is a fraction of a second. Conclusion: DeeptDCS is at least two orders of magnitudes faster than a physics-based open-source simulator, while providing satisfactorily accurate results. Significance: The high computational efficiency permits the use of DeeptDCS in applications requiring its repetitive execution, such as uncertainty quantification and optimization studies of tDCS.


title: DEAR: A Novel Deep Learning-based Approach for Automated Program Repair

abstract: The existing deep learning (DL)-based automated program repair (APR) models are limited in fixing general software defects. % We present {\tool}, a DL-based approach that supports fixing for the general bugs that require dependent changes at once to one or multiple consecutive statements in one or multiple hunks of code. % We first design a novel fault localization (FL) technique for multi-hunk, multi-statement fixes that combines traditional spectrum-based (SB) FL with deep learning and data-flow analysis. It takes the buggy statements returned by the SBFL model, detects the buggy hunks to be fixed at once, and expands a buggy statement $s$ in a hunk to include other suspicious statements around $s$. We design a two-tier, tree-based LSTM model that incorporates cycle training and uses a divide-and-conquer strategy to learn proper code transformations for fixing multiple statements in the suitable fixing context consisting of surrounding subtrees. We conducted several experiments to evaluate {\tool} on three datasets: Defects4J (395 bugs), BigFix (+26k bugs), and CPatMiner (+44k bugs). On Defects4J dataset, {\tool} outperforms the baselines from 42\%--683\% in terms of the number of auto-fixed bugs with only the top-1 patches. On BigFix dataset, it fixes 31--145 more bugs than existing DL-based APR models with the top-1 patches. On CPatMiner dataset, among 667 fixed bugs, there are 169 (25.3\%) multi-hunk/multi-statement bugs. {\tool} fixes 71 and 164 more bugs, including 52 and 61 more multi-hunk/multi-statement bugs, than the state-of-the-art, DL-based APR models.


title: TOI-2046b, TOI-1181b and TOI-1516b, three new hot Jupiters from \textit{TESS}: planets orbiting a young star, a subgiant and a normal star

abstract: We present the confirmation and characterization of three hot Jupiters, TOI-1181b, TOI-1516b, and TOI-2046b, discovered by the TESS space mission. The reported hot Jupiters have orbital periods between 1.4 and 2.05 days. The masses of the three planets are $1.18\pm0.14$ M$_{\mathrm{J}}$, $3.16\pm0.12$\, M$_{\mathrm{J}}$, and 2.30 $\pm 0.28$ M$_{\mathrm{J}}$, for TOI-1181b, TOI-1516b, and TOI-2046b, respectively. The stellar host of TOI-1181b is a F9IV star, whereas TOI-1516b and TOI-2046b orbit F main sequence host stars. The ages of the first two systems are in the range of 2-5 Gyrs. However, TOI-2046 is among the few youngest known planetary systems hosting a hot Jupiter, with an age estimate of 100-400 Myrs. The main instruments used for the radial velocity follow-up of these three planets are located at Ond\v{r}ejov, Tautenburg and McDonald Observatory, and all three are mounted on 2-3 meter aperture telescopes, demonstrating that mid-aperture telescope networks can play a substantial role in the follow-up of gas giants discovered by \textit{TESS} and in the future by \textit{PLATO}.


title: Convergence analysis of the Newton-Schur method for the symmetric elliptic eigenvalue problem

abstract: In this paper, we consider the Newton-Schur method in Hilbert space and obtain quadratic convergence. For the symmetric elliptic eigenvalue problem discretized by the standard finite element method and non-overlapping domain decomposition method, we use the Steklov-Poincar\'e operator to reduce the eigenvalue problem on the domain $\Omega$ into the nonlinear eigenvalue subproblem on $\Gamma$, which is the union of subdomain boundaries. We prove that the convergence rate for the Newton-Schur method is $\epsilon_{N}\leq CH^{2}(1+\ln(H/h))^{2}\epsilon^{2}$, where the constant $C$ is independent of the fine mesh size $h$ and coarse mesh size $H$, and $\epsilon_{N}$ and $\epsilon$ are errors after and before one iteration step respectively. Numerical experiments confirm our theoretical analysis.


title: The Hardy-Weyl algebra

abstract: We study the algebra $\mathcal{A}$ generated by the Hardy operator $H$ and the operator $M_x$ of multiplication by $x$ on $L^2[0,1]$. We call $\mathcal{A}$ the Hardy-Weyl algebra. We show that its quotient by the compact operators is isomorphic to the algebra of functions that are continuous on $\Lambda$ and analytic on the interior of $\Lambda$ for a planar set $\Lambda$ = $[-1,0] \cup \bar{ \mathbb{D}(1,1)}$, which we call the lollipop. We find a Toeplitz-like short exact sequence for the $C^*$-algebra generated by $\mathcal{A}$. We study the operator $Z = H - M_x$, show that its point spectrum is $(-1,0] \cup \mathbb{D}(1,1)$, and that the eigenvalues grow in multiplicity as the points move to $0$ from the left.


title: Provably Confidential Language Modelling

abstract: Large language models are shown to memorize privacy information such as social security numbers in training data. Given the sheer scale of the training corpus, it is challenging to screen and filter these privacy data, either manually or automatically. In this paper, we propose Confidentially Redacted Training (CRT), a method to train language generation models while protecting the confidential segments. We borrow ideas from differential privacy (which solves a related but distinct problem) and show that our method is able to provably prevent unintended memorization by randomizing parts of the training process. Moreover, we show that redaction with an approximately correct screening policy amplifies the confidentiality guarantee. We implement the method for both LSTM and GPT language models. Our experimental results show that the models trained by CRT obtain almost the same perplexity while preserving strong confidentiality.


title: Depletion-Driven Morphological Control of Bundled Actin Networks

abstract: The actin cytoskeleton is a semiflexible biopolymer network whose morphology is controlled by a wide range of biochemical and physical factors. Actin is known to undergo a phase transition from a single-filament state to a bundled state by the addition of polyethylene glycol (PEG) molecules in sufficient concentration. While the depletion interaction experienced by these biopolymers is well-known, the effect of changing the molecular weight of the depletant is less well understood. Here, we experimentally identify a phase transition in solutions of actin from networks of filaments to networks of bundles by varying the molecular weight of PEG polymers, while holding the concentration of these PEG polymers constant. We examine the states straddling the phase transition in terms of micro and macroscale properties. We find that the mesh size, bundle diameter, persistence length, and intra-bundle spacing between filaments across the line of criticality do not show significant differences, while the relaxation time, storage modulus, and degree of bundling change between the two states do show significant differences. Our results demonstrate the ability to tune actin network morphology and mechanics by controlling depletant size, a property which could be exploited to develop actin-based materials with switchable rigidity.


title: Improved Limit on Tensor Currents in the Weak Interaction from $^8$Li $\beta$ Decay

abstract: The electroweak interaction in the Standard Model (SM) is described by a pure vector-axial-vector structure, though any Lorentz-invariant component could contribute. In this work, we present the most precise measurement of tensor currents in the low-energy regime by examining the $\beta$-$\bar{\nu}$ correlation of trapped $^{8}$Li ions with the Beta-decay Paul Trap. We find $a_{\beta\nu} = -0.3325 \pm 0.0013_{stat} \pm 0.0019_{syst}$ at $1\sigma$ for the case of coupling to right-handed neutrinos $(C_T=-C_T')$, which is consistent with the SM prediction.


title: Correlated Read Noise Reduction in Infrared Arrays Using Deep Learning

abstract: We present a new procedure rooted in deep learning to construct science images from data cubes collected by astronomical instruments using HxRG detectors in low-flux regimes. It improves on the drawbacks of the conventional algorithms to construct 2D images from multiple readouts by using the readout scheme of the detectors to reduce the impact of correlated readout noise. We train a convolutional recurrent neural network on simulated astrophysical scenes added to laboratory darks to estimate the flux on each pixel of science images. This method achieves a reduction of the noise on constructed science images when compared to standard flux-measurement schemes (correlated double sampling, up-the-ramp sampling), which results in a reduction of the error on the spectrum extracted from these science images. Over simulated data cubes created in a low signal-to-noise ratio regime where this method could have the largest impact, we find that the error on our constructed science images falls faster than a $1/\sqrt{N}$ decay, and that the spectrum extracted from the images has, averaged over a test set of three images, a standard error reduced by a factor of 1.85 in comparison to the standard up-the-ramp pixel sampling scheme. The code used in this project is publicly available on GitHub


title: Soft Metamaterials: Adaptation and Intelligence

abstract: We provide a perspective on soft metamaterials and biomatter with a special emphasis on future research directions at the interface between material intelligence and adaptation.


title: Hazard Exposure Heterophily: A Latent Characteristic in Socio-Spatial Networks Influencing Community Resilience

abstract: We present a latent characteristic in socio-spatial networks, hazard-exposure heterophily, to capture the extent to which populations with similar hazard exposure could assist each other through social ties. Heterophily is the tendency of unlike individuals to form social ties. Conversely, populations in spatial areas with significant hazard exposure similarity, homophily, would lack sufficient resourcefulness to aid each other to lessen the impact of hazards. In the context of the Houston metropolitan area, we use Meta's Social Connectedness data to construct a socio-spatial network in juxtaposition with flood exposure data from National Flood Hazard Layer to analyze flood hazard exposure of spatial areas. The results reveal the extent and spatial variation of hazard-exposure heterophily in the study area. Notably, the results show that lower-income areas have lower hazard-exposure heterophily possibly caused by income segregation and the tendency of affordable housing development to be located in flood zones. Less resourceful social ties due to high hazard-exposure homophily may inhibit low-income areas from better coping with hazard impacts and could contribute to their slower recovery. Overall, the results underscore the significance of characterizing hazard-exposure heterophily in socio-spatial networks to reveal community vulnerability and resilience to hazards.


title: The College Application Problem

abstract: This paper considers the maximization of the expected maximum value of a portfolio of random variables subject to a budget constraint. We refer to this as the optimal college application problem. When each variable's cost, or each college's application fee, is identical, we show that the optimal portfolios are nested in the budget constraint, yielding an exact polynomial-time algorithm. When colleges differ in their application fees, we show that the problem is NP-complete. We provide four algorithms for this more general setup: a branch-and-bound routine, a dynamic program that produces an exact solution in pseudopolynomial time, a different dynamic program that yields a fully polynomial-time approximation scheme, and a simulated-annealing heuristic. Numerical experiments demonstrate the algorithms' accuracy and efficiency.


title: Mathematical models and time-frequency heat maps for surface gravity waves generated by thin ships

abstract: Recent research suggests that studying the time-frequency response of ship wave signals has potential to shed light on a range of applications, such as inferring the dynamical and geometric properties of a moving vessel based on the surface elevation data detected at a single point in space. We continue this line of research here with a study of mathematical models for thin ships using standard Wigley hulls and Wigley transom-stern hulls as examples. Mathematical models of varying sophistication are considered. These include basic minimal models which use applied pressure distributions as proxies for the ship hull. The more complicated models are Michell's thin ship theory and the Hogner model, both of which explicitly take into account the shape of the hull. We outline a methodology for carefully choosing the form and parameter values in the minimal models such that they reproduce the key features of the more complicated models in the time-frequency domain. For example, we find that a two-pressure model is capable of producing wave elevation signals that have a similar time-frequency profile as that for Michell's thin ship theory applied to the Wigley hull, including the crucially important features caused by interference between waves created at the bow and stern of the ship. One of the key tools in our analysis is the spectrogram, which is a heat-map visualisation in the time-frequency domain. Our work here extends the existing knowledge on the topic of spectrograms of ship waves. The theoretical results in this study are supported by experimental data collected in a towing tank at the Australian Maritime College using model versions of the standard Wigley hulls and Wigley transom-stern hulls.


title: UCL-Dehaze: Towards Real-world Image Dehazing via Unsupervised Contrastive Learning

abstract: While the wisdom of training an image dehazing model on synthetic hazy data can alleviate the difficulty of collecting real-world hazy/clean image pairs, it brings the well-known domain shift problem. From a different yet new perspective, this paper explores contrastive learning with an adversarial training effort to leverage unpaired real-world hazy and clean images, thus bridging the gap between synthetic and real-world haze is avoided. We propose an effective unsupervised contrastive learning paradigm for image dehazing, dubbed UCL-Dehaze. Unpaired real-world clean and hazy images are easily captured, and will serve as the important positive and negative samples respectively when training our UCL-Dehaze network. To train the network more effectively, we formulate a new self-contrastive perceptual loss function, which encourages the restored images to approach the positive samples and keep away from the negative samples in the embedding space. Besides the overall network architecture of UCL-Dehaze, adversarial training is utilized to align the distributions between the positive samples and the dehazed images. Compared with recent image dehazing works, UCL-Dehaze does not require paired data during training and utilizes unpaired positive/negative data to better enhance the dehazing performance. We conduct comprehensive experiments to evaluate our UCL-Dehaze and demonstrate its superiority over the state-of-the-arts, even only 1,800 unpaired real-world images are used to train our network. Source code has been available at https://github.com/yz-wang/UCL-Dehaze.


title: A smectic liquid crystal model in the periodic setting

abstract: We consider the asymptotic behavior as $\varepsilon $ goes to zero of the 2D smectics model in the periodic setting given by \begin{equation*} \mathcal{E}_{\varepsilon }( w) =\frac{1}{2}\int_{\mathbb{T}^{2}}\frac{1}{ \varepsilon }\left( \left\vert \partial_{1}\right\vert ^{-1}\left( \partial_{2}w-\partial_{1}\frac{1}{2}w^{2}\right) \right) ^{2}+\varepsilon \left( \partial_{1}w\right) ^{2}dx . \end{equation*} We show that the energy $\mathcal{E}_\varepsilon(w)$ controls suitable $L^p$ and Besov norms of $w$ and use this to demonstrate the existence of minimizers for $\mathcal{E}_\varepsilon(w)$, which has not been proved for this smectics model before, and compactness in $L^p$ for an energy-bounded sequence. We also prove an asymptotic lower bound for $\mathcal{E}_\varepsilon(w)$ as $\varepsilon \to 0$ by means of an entropy argument.


title: Wasserstein Adversarial Learning based Temporal Knowledge Graph Embedding

abstract: Research on knowledge graph embedding (KGE) has emerged as an active field in which most existing KGE approaches mainly focus on static structural data and ignore the influence of temporal variation involved in time-aware triples. In order to deal with this issue, several temporal knowledge graph embedding (TKGE) approaches have been proposed to integrate temporal and structural information in recent years. However, these methods only employ a uniformly random sampling to construct negative facts. As a consequence, the corrupted samples are often too simplistic for training an effective model. In this paper, we propose a new temporal knowledge graph embedding framework by introducing adversarial learning to further refine the performance of traditional TKGE models. In our framework, a generator is utilized to construct high-quality plausible quadruples and a discriminator learns to obtain the embeddings of entities and relations based on both positive and negative samples. Meanwhile, we also apply a Gumbel-Softmax relaxation and the Wasserstein distance to prevent vanishing gradient problems on discrete data; an inherent flaw in traditional generative adversarial networks. Through comprehensive experimentation on temporal datasets, the results indicate that our proposed framework can attain significant improvements based on benchmark models and also demonstrate the effectiveness and applicability of our framework.


title: Joint Image Compression and Denoising via Latent-Space Scalability

abstract: When it comes to image compression in digital cameras, denoising is traditionally performed prior to compression. However, there are applications where image noise may be necessary to demonstrate the trustworthiness of the image, such as court evidence and image forensics. This means that noise itself needs to be coded, in addition to the clean image itself. In this paper, we present a learning-based image compression framework where image denoising and compression are performed jointly. The latent space of the image codec is organized in a scalable manner such that the clean image can be decoded from a subset of the latent space (the base layer), while the noisy image is decoded from the full latent space at a higher rate. Using a subset of the latent space for the denoised image allows denoising to be carried out at a lower rate. Besides providing a scalable representation of the noisy input image, performing denoising jointly with compression makes intuitive sense because noise is hard to compress; hence, compressibility is one of the criteria that may help distinguish noise from the signal. The proposed codec is compared against established compression and denoising benchmarks, and the experiments reveal considerable bitrate savings compared to a cascade combination of a state-of-the-art codec and a state-of-the-art denoiser.


title: Machine Learning based Framework for Robust Price-Sensitivity Estimation with Application to Airline Pricing

abstract: We consider the problem of dynamic pricing of a product in the presence of feature-dependent price sensitivity. Developing practical algorithms that can estimate price elasticities robustly, especially when information about no purchases (losses) is not available, to drive such automated pricing systems is a challenge faced by many industries. Based on the Poisson semi-parametric approach, we construct a flexible yet interpretable demand model where the price related part is parametric while the remaining (nuisance) part of the model is non-parametric and can be modeled via sophisticated machine learning (ML) techniques. The estimation of price-sensitivity parameters of this model via direct one-stage regression techniques may lead to biased estimates due to regularization. To address this concern, we propose a two-stage estimation methodology which makes the estimation of the price-sensitivity parameters robust to biases in the estimators of the nuisance parameters of the model. In the first-stage we construct estimators of observed purchases and prices given the feature vector using sophisticated ML estimators such as deep neural networks. Utilizing the estimators from the first-stage, in the second-stage we leverage a Bayesian dynamic generalized linear model to estimate the price-sensitivity parameters. We test the performance of the proposed estimation schemes on simulated and real sales transaction data from the Airline industry. Our numerical studies demonstrate that our proposed two-stage approach reduces the estimation error in price-sensitivity parameters from 25\% to 4\% in realistic simulation settings. The two-stage estimation techniques proposed in this work allows practitioners to leverage modern ML techniques to robustly estimate price-sensitivities while still maintaining interpretability and allowing ease of validation of its various constituent parts.


title: fairlib: A Unified Framework for Assessing and Improving Classification Fairness

abstract: This paper presents fairlib, an open-source framework for assessing and improving classification fairness. It provides a systematic framework for quickly reproducing existing baseline models, developing new methods, evaluating models with different metrics, and visualizing their results. Its modularity and extensibility enable the framework to be used for diverse types of inputs, including natural language, images, and audio. In detail, we implement 14 debiasing methods, including pre-processing, at-training-time, and post-processing approaches. The built-in metrics cover the most commonly used fairness criterion and can be further generalized and customized for fairness evaluation.


title: Quantum dialogue without information leakage based on the entanglement swapping between any two Bell states and the shared secret Bell state

abstract: In order to avoid the risk of information leakage during the information mutual transmission between two authorized participants, i.e., Alice and Bob, a quantum dialogue protocol based on the entanglement swapping between any two Bell states and the shared secret Bell state is proposed. The proposed protocol integrates the ideas of block transmission, two-step transmission and unitary operation encoding together using the Bell states as the information carriers. Besides the entanglement swapping between any two Bell states, a shared secret Bell state is also used to overcome the information leakage problem, which not only makes Bob aware of the prepared initial state but also is used for Bob's encoding and entanglement swapping. Security analysis shows that the proposed protocol can resist the general active attacks from an outside eavesdropper Eve. Moreover, the relation between the maximal amount of information Eve can gain and the detection probability is derived.


title: Exploring Entity Interactions for Few-Shot Relation Learning (Student Abstract)

abstract: Few-shot relation learning refers to infer facts for relations with a limited number of observed triples. Existing metric-learning methods for this problem mostly neglect entity interactions within and between triples. In this paper, we explore this kind of fine-grained semantic meanings and propose our model TransAM. Specifically, we serialize reference entities and query entities into sequence and apply transformer structure with local-global attention to capture both intra- and inter-triple entity interactions. Experiments on two public benchmark datasets NELL-One and Wiki-One with 1-shot setting prove the effectiveness of TransAM.


title: A Nonlinear Car-following Controller Design Inspired By Human-driving Behaviors to Increase Comfort and Enhance Safety

abstract: This paper investigates the car-following problem and proposes a nonlinear controller that considers driving comfort, safety concerns, steady-state response and transient response. This controller is designed based on the demands of lower cost, faster response, increased comfort, enhanced safety and elevated extendability from the automotive industry. Design insights and intuitions are provided in detail. Also, theoretical analysis are performed on plant stability, string stability and tracking performance of the closed-loop system. Conditions and guidelines are provided on the selection of control parameters. Comprehensive simulations are conducted to demonstrate the efficacy of the proposed controller in different driving scenarios.


title: Surface states induced weak anti-localization effect in Bi0.85Sb0.15 topological single crystal

abstract: We report, an experimental evidence of surface states (SS) driven magneto-transport in a Bi0.85Sb0.15 single crystal. Detailed high field (up to 12T) and low temperature (down to 2K) magneto-transport measurements are been carried out on the studied Bi0.85Sb0.15 single crystal. The phase, composition and Raman modes are studied through X-ray diffraction, Energy dispersive X-ray, and Raman spectroscopy. The obtained crystal shows non-saturating magnetoresistance (4250%) at 2K and 12T, along with the existence of weak-anti localization (WAL) effect at around zero magnetic field. Further, the Hikami-Larkin-Nagaoka (HLN) analysis is performed to analyse the WAL effect. The prefactor and phase coherence length are deduced at various temperatures, which signified the presence of more than one conduction channel in the studied Bi0.85Sb0.15 single crystal. The effect of quantum scattering, bulk contribution from underneath the surface states and defects are been studied by adding various field dependent quadratic, linear and constant terms to the SS driven HLN equation. Various possible scattering mechanism are studied by analysing the temperature dependence of the phase coherence length. Angle dependent magneto-conductivity of the studied Bi0.85Sb0.15 single crystal clearly confirmed the surface states dominated transport in present crystal.


title: Quantum secure dialogue with quantum encryption

abstract: How to solve the information leakage problem has become the research focus of quantum dialogue. In this paper, in order to overcome the information leakage problem in quantum dialogue, a novel approach for sharing the initial quantum state privately between communicators, i.e., quantum encryption sharing, is proposed by utilizing the idea of quantum encryption. The proposed protocol uses EPR pairs as the private quantum key to encrypt and decrypt the traveling photons, which can be repeatedly used after rotation. Due to quantum encryption sharing, the public announcement on the state of the initial quantum state is omitted, thus the information leakage problem is overcome. The information-theoretical efficiency of the proposed protocol is nearly 100%, much higher than previous information leakage resistant quantum dialogue protocols. Moreover, the proposed protocol only needs single-photon measurements and nearly uses single photons as quantum resource so that it is convenient to implement in practice.


title: Approximating Choice Data by Discrete Choice Models

abstract: We obtain a necessary and sufficient condition under which random-coefficient discrete choice models, such as mixed-logit models, are rich enough to approximate any nonparametric random utility models arbitrarily well across choice sets. The condition turns out to be the affine-independence of the set of characteristic vectors. When the condition fails, resulting in some random utility models that cannot be closely approximated, we identify preferences and substitution patterns that are challenging to approximate accurately. We also propose algorithms to quantify the magnitude of approximation errors.


title: All You May Need for VQA are Image Captions

abstract: Visual Question Answering (VQA) has benefited from increasingly sophisticated models, but has not enjoyed the same level of engagement in terms of data creation. In this paper, we propose a method that automatically derives VQA examples at volume, by leveraging the abundance of existing image-caption annotations combined with neural models for textual question generation. We show that the resulting data is of high-quality. VQA models trained on our data improve state-of-the-art zero-shot accuracy by double digits and achieve a level of robustness that lacks in the same model trained on human-annotated VQA data.


title: A deep domain decomposition method based on Fourier features

abstract: In this paper we present a Fourier feature based deep domain decomposition method (F-D3M) for partial differential equations (PDEs). Currently, deep neural network based methods are actively developed for solving PDEs, but their efficiency can degenerate for problems with high frequency modes. In this new F-D3M strategy, overlapping domain decomposition is conducted for the spatial domain, such that high frequency modes can be reduced to relatively low frequency ones. In each local subdomain, multi Fourier feature networks (MFFNets) are constructed, where efficient boundary and interface treatments are applied for the corresponding loss functions. We present a general mathematical framework of F-D3M, validate its accuracy and demonstrate its efficiency with numerical experiments.


title: Holographic Schwinger effect and electric instability with anisotropy

abstract: According to the gauge-gravity duality, we systematically study the Schwinger effect and electric instability with anisotropy in a top-down holographic approach. The anisotropic black brane and bubble (soliton) background in IIB supergravity are employed and the dual theories in these backgrounds are expected to be anisotropic theory at finite temperature and anisotropic theory with confinement respectively. Then performing the potential analysis, we find due to the anisotropy, the potential barrier behaves oppositely with parallel and perpendicular electric fields, and this behavior agrees with the previous study about the quark potential with anisotropy in this system. Afterwards, we evaluate the pair production rate by solving the equation of motion for a fundamental string numerically which reveals the consistent behavior with the potential analysis. Furthermore, the probe D7-brane as flavor is introduced into the bulk in order to investigate the electric instability. The vacuum decay rate can be obtained by evaluating the imaginary part of the D7-brane action which again agrees with our potential analysis. Solving the associated constraint of gauge field strength on the flavor brane, we finally obtain the V-A curve displaying the distinct behavior of the conductivity in parallel and perpendicular direction which is in agreement with some bottom-up and phenomenologically holographic approaches in anisotropic fluid. Accordingly, we believe this work may be remarkable to study the electric features in strongly coupled anisotropic system.


title: P^3 Ranker: Mitigating the Gaps between Pre-training and Ranking Fine-tuning with Prompt-based Learning and Pre-finetuning

abstract: Compared to other language tasks, applying pre-trained language models (PLMs) for search ranking often requires more nuances and training signals. In this paper, we identify and study the two mismatches between pre-training and ranking fine-tuning: the training schema gap regarding the differences in training objectives and model architectures, and the task knowledge gap considering the discrepancy between the knowledge needed in ranking and that learned during pre-training. To mitigate these gaps, we propose Pre-trained, Prompt-learned and Pre-finetuned Neural Ranker (P^3 Ranker). P^3 Ranker leverages prompt-based learning to convert the ranking task into a pre-training like schema and uses pre-finetuning to initialize the model on intermediate supervised tasks. Experiments on MS MARCO and Robust04 show the superior performances of P^3 Ranker in few-shot ranking. Analyses reveal that P^3 Ranker is able to better accustom to the ranking task through prompt-based learning and retrieve necessary ranking-oriented knowledge gleaned in pre-finetuning, resulting in data-efficient PLM adaptation. Our code is available at https://github.com/NEUIR/P3Ranker.


title: Uncertainty estimation of pedestrian future trajectory using Bayesian approximation

abstract: Past research on pedestrian trajectory forecasting mainly focused on deterministic predictions which provide only point estimates of future states. These future estimates can help an autonomous vehicle plan its trajectory and avoid collision. However, under dynamic traffic scenarios, planning based on deterministic predictions is not trustworthy. Rather, estimating the uncertainty associated with the predicted states with a certain level of confidence can lead to robust path planning. Hence, the authors propose to quantify this uncertainty during forecasting using stochastic approximation which deterministic approaches fail to capture. The current method is simple and applies Bayesian approximation during inference to standard neural network architectures for estimating uncertainty. The authors compared the predictions between the probabilistic neural network (NN) models with the standard deterministic models. The results indicate that the mean predicted path of probabilistic models was closer to the ground truth when compared with the deterministic prediction. Further, the effect of stochastic dropout of weights and long-term prediction on future state uncertainty has been studied. It was found that the probabilistic models produced better performance metrics like average displacement error (ADE) and final displacement error (FDE). Finally, the study has been extended to multiple datasets providing a comprehensive comparison for each model.


title: The Mazur-Ulam property for a Banach space which satisfies a separation condition

abstract: We study $C$-rich spaces, lush spaces, and $C$-extremely regular spaces concerning with the Mazur-Ulam property. We show that a uniform algebra and the real part of a uniform algebra with the supremum norm are $C$-rich spaces, hence lush spaces. We prove that a uniformly closed subalgebra of the algebra of complex-valued continuous functions on a locally compact Hausdorff space which vanish at infinity is $C$-extremely regular provided that it separates the points of the underlying space and has no common zeros. In section 3 we exhibit descriptions on the Choquet bounday, the \vSilov bounday, the strong boundary points. We also recall the definition that a function space strongly separates the points in the underlying space. We need to avoid the confusion which appears because of the variety of names of these concepts; they sometimes differs from authors to authors. After some preparation, we study the Mazur-Ulam property in sections 4 through 6. We exhibit a sufficient condition on a Banach space which has the Mazur-Ulam property and the complex Mazur-Ulam property. In section 5 we consider a Banach space with a separation condition $(*)$ (Definition 5.1). We prove that a real Banach space satisfying $(*)$ has the Mazur-Ulam propety (Theorem 6.1), and a complex Banach space satisfying $(*)$ has the complex Mazur-Ulam property (Theorem 6.3). Applying the results in the previous sections we prove that an extremely $C$-regular complex linear subspace has the complex Mazur-Ulam property (Corollary 6.4) in section 6. As a consequence we prove that any closed subalgebra of the algebra of all complex-valued continuous functions defined on a locally compact Hausdorff space has the complex Mazur-Ulam property (Corollary 6.5).


title: Improving Multi-Document Summarization through Referenced Flexible Extraction with Credit-Awareness

abstract: A notable challenge in Multi-Document Summarization (MDS) is the extremely-long length of the input. In this paper, we present an extract-then-abstract Transformer framework to overcome the problem. Specifically, we leverage pre-trained language models to construct a hierarchical extractor for salient sentence selection across documents and an abstractor for rewriting the selected contents as summaries. However, learning such a framework is challenging since the optimal contents for the abstractor are generally unknown. Previous works typically create pseudo extraction oracle to enable the supervised learning for both the extractor and the abstractor. Nevertheless, we argue that the performance of such methods could be restricted due to the insufficient information for prediction and inconsistent objectives between training and testing. To this end, we propose a loss weighting mechanism that makes the model aware of the unequal importance for the sentences not in the pseudo extraction oracle, and leverage the fine-tuned abstractor to generate summary references as auxiliary signals for learning the extractor. Moreover, we propose a reinforcement learning method that can efficiently apply to the extractor for harmonizing the optimization between training and testing. Experiment results show that our framework substantially outperforms strong baselines with comparable model sizes and achieves the best results on the Multi-News, Multi-XScience, and WikiCatSum corpora.


title: Correlation-induced magnetism in substrate-supported 2D metal-organic frameworks

abstract: Two-dimensional (2D) metal-organic frameworks (MOFs) in a kagome lattice can exhibit strong electron-electron interactions, which can lead to tunable quantum phases including many exotic magnetic phases. While technological developments of 2D MOFs typically take advantage of substrates for growth, support, and electrical contacts, investigations often ignore substrates and their dramatic influence on electronic properties of MOFs. Here, we show how substrates alter the correlated magnetic phases in kagome MOFs using systematic density functional theory and mean-field Hubbard calculations. We demonstrate that MOF-substrate coupling, MOF-substrate charge transfer, strain, and external electric fields are key variables, activating and deactivating magnetic phases in these materials. While we consider the example of kagome-arranged 9,10-dicyanoanthracene molecules coordinated with copper atoms, our findings should generalise to any kagome lattice. This work offers useful predictions for tunable interaction-induced magnetism in surface-supported 2D organic materials, opening the door to solid-state electronic and spintronic technologies based on such systems.


title: Magnon Nernst Effect in Magnon Spin Hall Systems

abstract: Magnon spin Hall systems could hardly show experimentally observable particle and thermal transport phenomena intrinsically due to the spin cancellation. Here we demonstrated that the magnon spin Hall systems can exhibit magnon Nernst effect and thermal Hall effect under external magnetic field by considering two typical systems, i.e. the antiferromagnetically (AFM) coupled bilayer honeycomb ferromagnets and monolayer collinear honeycomb antiferromagnet. The both systems experience magnetic phase transitions from AFM phase to a field-polarized phase via a spin-flop (SF) phase or directly. In both systems, there exist magnon Nerst effect and also thermal Hall effect under a longitudinal temperature gradient, which can be regarded as the indicator of the magnetic phase transitions, with the Hall conductivity dependence on magnetic field consistent with the order of magnetic phase transition.


title: Unsupervised Domain Adaptation Learning for Hierarchical Infant Pose Recognition with Synthetic Data

abstract: The Alberta Infant Motor Scale (AIMS) is a well-known assessment scheme that evaluates the gross motor development of infants by recording the number of specific poses achieved. With the aid of the image-based pose recognition model, the AIMS evaluation procedure can be shortened and automated, providing early diagnosis or indicator of potential developmental disorder. Due to limited public infant-related datasets, many works use the SMIL-based method to generate synthetic infant images for training. However, this domain mismatch between real and synthetic training samples often leads to performance degradation during inference. In this paper, we present a CNN-based model which takes any infant image as input and predicts the coarse and fine-level pose labels. The model consists of an image branch and a pose branch, which respectively generates the coarse-level logits facilitated by the unsupervised domain adaptation and the 3D keypoints using the HRNet with SMPLify optimization. Then the outputs of these branches will be sent into the hierarchical pose recognition module to estimate the fine-level pose labels. We also collect and label a new AIMS dataset, which contains 750 real and 4000 synthetic infants images with AIMS pose labels. Our experimental results show that the proposed method can significantly align the distribution of synthetic and real-world datasets, thus achieving accurate performance on fine-grained infant pose recognition.


title: Crystal Twins: Self-supervised Learning for Crystalline Material Property Prediction

abstract: Machine learning (ML) models have been widely successful in the prediction of material properties. However, large labeled datasets required for training accurate ML models are elusive and computationally expensive to generate. Recent advances in Self-Supervised Learning (SSL) frameworks capable of training ML models on unlabeled data have mitigated this problem and demonstrated superior performance in computer vision and natural language processing tasks. Drawing inspiration from the developments in SSL, we introduce Crystal Twins (CT): an SSL method for crystalline materials property prediction. Using a large unlabeled dataset, we pre-train a Graph Neural Network (GNN) by applying the redundancy reduction principle to the graph latent embeddings of augmented instances obtained from the same crystalline system. By sharing the pre-trained weights when fine-tuning the GNN for regression tasks, we significantly improve the performance for 7 challenging material property prediction benchmarks


title: Results on bar-core partitions, core shifted Young diagrams, and doubled distinct cores

abstract: Simultaneous bar-cores, core shifted Young diagrams (or CSYDs), and doubled distinct cores have been studied since Morris and Yaseen introduced the concept of bar-cores. In this paper, our goal is to give a formula for the number of these core partitions on $(s,t)$-cores and $(s,s+d,s+2d)$-cores for the remaining cases that are not covered yet. In order to achieve this goal, we observe a characterization of $\bar{s}$-core partitions to obtain characterizations of doubled distinct $s$-core partitions and $s$-CSYDs. By using them, we construct $NE$ lattice path interpretations of these core partitions on $(s,t)$-cores. Also, we give free Motzkin path interpretations of these core partitions on $(s,s+d,s+2d)$-cores.


title: Study of the $tH$ production at 14TeV and 100TeV $pp$ colliders

abstract: The flavor-changing neutral current interactions in the standard model are suppressed seriously and such interactions can be used to search the new physics beyond SM. The top quark and Higgs bosons are heavier than the other particles in SM, we can expect the new physics plays a more important role in their interactions. In this work, we study the flavor-changing neutral current interactions between the top quark $\bar{t}q H$ through the production of the single top associated with a Higgs boson on 14TeV and 100TeV $pp$ colliders. We consider the leptonic decay channels of Higgs and study the signal. We find a sensitive region $0.4\leq\Delta R\leq1.4$ for the leptons from Higgs bosons and $\Delta R\geq1.8$ between the jets from top quark and leptons. We investigate the detective abilities of the hadron colliders for the processes. Although this process seems not comparable with $pp\to t\bar{t}$ with $t\to H q$ decay, it is still attractive since this process can be used to distinguish the $y_{tu}$ and $y_{tc}$.


title: An Online Generalized Multiscale finite element method for heat and mass transfer problem with artificial ground freezing

abstract: In this paper, we present an Online Generalized Multiscale Finite Element Method(Online GMsFEM) for heat and mass transfer problem in heterogeneous media with artificial ground freezing pipes. The mathematical model of the process is based on the classical Stefan model, which describes heat transfer with a phase transition and takes into account filtration in a porous medium. The model is described by a system of equations for temperature and pressure. For fine grid solution, we use a finite element method using the fictitious domain method. To derive a solution on the coarse grid, we use a model reduction procedure based on Online GMsFEM. Online version of GMsFEM allows to us to take less number of offline multiscale basis functions. In our approach, we use decoupled offline basis functions constructed with snapshot space and based on spectral problems. This is the standard approach of basis construction. To take into account artificial ground freezing pipes, we compute an additional basis functions on the offline stage. For the accurate approximation of phase change we add online multiscale basis functions. We construct online basis that minimizes error by values of local residuals. Online procedure is significantly improves the accuracy of standard GMsFEM. We present numerical results in two-dimensional domain with layered heterogeneity. To investigate accuracy of the method, we present results with different number of offline and online basis functions. The presented results show that Online GMsFEM can produce solution with high accuracy and requires small computational resources.


title: Virtual Analog Modeling of Distortion Circuits Using Neural Ordinary Differential Equations

abstract: Recent research in deep learning has shown that neural networks can learn differential equations governing dynamical systems. In this paper, we adapt this concept to Virtual Analog (VA) modeling to learn the ordinary differential equations (ODEs) governing the first-order and the second-order diode clipper. The proposed models achieve performance comparable to state-of-the-art recurrent neural networks (RNNs) albeit using fewer parameters. We show that this approach does not require oversampling and allows to increase the sampling rate after the training has completed, which results in increased accuracy. Using a sophisticated numerical solver allows to increase the accuracy at the cost of slower processing. ODEs learned this way do not require closed forms but are still physically interpretable.


title: Go Back in Time: Generating Flashbacks in Stories with Event Temporal Prompts

abstract: Stories or narratives are comprised of a sequence of events. To compose interesting stories, professional writers often leverage a creative writing technique called flashback that inserts past events into current storylines as we commonly observe in novels and plays. However, it is challenging for machines to generate flashback as it requires a solid understanding of event temporal order (e.g. "feeling hungry" before "eat," not vice versa), and the creativity to arrange storylines so that earlier events do not always appear first in narrative order. Two major issues in existing systems that exacerbate the challenges: 1) temporal bias in pertaining and story datasets that leads to monotonic event temporal orders; 2) lack of explicit guidance that helps machines decide where to insert flashbacks. We propose to address these issues using structured storylines to encode events and their pair-wise temporal relations (before, after and vague) as temporal prompts that guide how stories should unfold temporally. We leverage a Plan-and-Write framework enhanced by reinforcement learning to generate storylines and stories end-to-end. Evaluation results show that the proposed method can generate more interesting stories with flashbacks while maintaining textual diversity, fluency, and temporal coherence.


title: A Tool For Debugging Quantum Circuits

abstract: As the scale of quantum programs grows to match that of classical software, the nascent field of quantum software engineering must mature and tools such as debuggers will become increasingly important. However, developing a quantum debugger is challenging due to the nature of a quantum computer; sneaking a peek at the value of a quantum state will cause either partial or complete collapse of the superposition and may destroy the necessary entanglement. As a first step to developing a full quantum circuit debugger, we have designed and implemented a quantum circuit debugging tool. The tool allows the user to divide the circuit vertically or horizontally into smaller chunks known as slices, and manage their simulation or execution for either interactive debugging or automated testing. The tool also enables developers to track gates within the overall circuit and each chunk to understand their behavior better. Feedback on usefulness and usability from early users shows that using the tool to slice and test their circuits has helped make the debugging process more time-efficient for them.


title: Resonant enhancement of particle emission from a parametrically driven condensate in a one-dimensional lattice

abstract: Motivated by recent experiments, we investigate particle emission from a Bose-Einstein condensate in a one-dimensional lattice, where the interaction strength is periodically modulated. The modulated interactions parametrically excite a collective mode, leading to density oscillations. These collective oscillations in turn drive particle emission. This multistep process amplifies the drive, producing larger particle jets. We find that the amplitude dependence of the emission rate has a characteristic threshold behavior, as seen in experiments.


title: Spatial-Temporal Meta-path Guided Explainable Crime Prediction

abstract: Exposure to crime and violence can harm individuals' quality of life and the economic growth of communities. In light of the rapid development in machine learning, there is a rise in the need to explore automated solutions to prevent crimes. With the increasing availability of both fine-grained urban and public service data, there is a recent surge in fusing such cross-domain information to facilitate crime prediction. By capturing the information about social structure, environment, and crime trends, existing machine learning predictive models have explored the dynamic crime patterns from different views. However, these approaches mostly convert such multi-source knowledge into implicit and latent representations (e.g., learned embeddings of districts), making it still a challenge to investigate the impacts of explicit factors for the occurrences of crimes behind the scenes. In this paper, we present a Spatial-Temporal Metapath guided Explainable Crime prediction (STMEC) framework to capture dynamic patterns of crime behaviours and explicitly characterize how the environmental and social factors mutually interact to produce the forecasts. Extensive experiments show the superiority of STMEC compared with other advanced spatiotemporal models, especially in predicting felonies (e.g., robberies and assaults with dangerous weapons).


title: Pik-Fix: Restoring and Colorizing Old Photos

abstract: Restoring and inpainting the visual memories that are present, but often impaired, in old photos remains an intriguing but unsolved research topic. Decades-old photos often suffer from severe and commingled degradation such as cracks, defocus, and color-fading, which are difficult to treat individually and harder to repair when they interact. Deep learning presents a plausible avenue, but the lack of large-scale datasets of old photos makes addressing this restoration task very challenging. Here we present a novel reference-based end-to-end learning framework that is able to both repair and colorize old, degraded pictures. Our proposed framework consists of three modules: a restoration sub-network that conducts restoration from degradations, a similarity network that performs color histogram matching and color transfer, and a colorization subnet that learns to predict the chroma elements of images conditioned on chromatic reference signals. The overall system makes uses of color histogram priors from reference images, which greatly reduces the need for large-scale training data. We have also created a first-of-a-kind public dataset of real old photos that are paired with ground truth ''pristine'' photos that have been manually restored by PhotoShop experts. We conducted extensive experiments on this dataset and synthetic datasets, and found that our method significantly outperforms previous state-of-the-art models using both qualitative comparisons and quantitative measurements. The code is available at https://github.com/DerrickXuNu/Pik-Fix.


title: Self-Taught Metric Learning without Labels

abstract: We present a novel self-taught framework for unsupervised metric learning, which alternates between predicting class-equivalence relations between data through a moving average of an embedding model and learning the model with the predicted relations as pseudo labels. At the heart of our framework lies an algorithm that investigates contexts of data on the embedding space to predict their class-equivalence relations as pseudo labels. The algorithm enables efficient end-to-end training since it demands no off-the-shelf module for pseudo labeling. Also, the class-equivalence relations provide rich supervisory signals for learning an embedding space. On standard benchmarks for metric learning, it clearly outperforms existing unsupervised learning methods and sometimes even beats supervised learning models using the same backbone network. It is also applied to semi-supervised metric learning as a way of exploiting additional unlabeled data, and achieves the state of the art by boosting performance of supervised learning substantially.


title: ImAiR: Airwriting Recognition framework using Image Representation of IMU Signals

abstract: The problem of Airwriting Recognition is focused on identifying letters written by movement of finger in free space. It is a type of gesture recognition where the dictionary corresponds to letters in a specific language. In particular, airwriting recognition using sensor data from wrist-worn devices can be used as a medium of user input for applications in Human-Computer Interaction (HCI). Recognition of in-air trajectories using such wrist-worn devices is limited in literature and forms the basis of the current work. In this paper, we propose an airwriting recognition framework by first encoding the time-series data obtained from a wearable Inertial Measurement Unit (IMU) on the wrist as images and then utilizing deep learning-based models for identifying the written alphabets. The signals recorded from 3-axis accelerometer and gyroscope in IMU are encoded as images using different techniques such as Self Similarity Matrix (SSM), Gramian Angular Field (GAF) and Markov Transition Field (MTF) to form two sets of 3-channel images. These are then fed to two separate classification models and letter prediction is made based on an average of the class conditional probabilities obtained from the two models. Several standard model architectures for image classification such as variants of ResNet, DenseNet, VGGNet, AlexNet and GoogleNet have been utilized. Experiments performed on two publicly available datasets demonstrate the efficacy of the proposed strategy. The code for our implementation will be made available at https://github.com/ayushayt/ImAiR.


title: Three-dimensional Geospatial Interlinking with JedAI-spatial

abstract: Geospatial data constitutes a considerable part of (Semantic) Web data, but so far, its sources are inadequately interlinked in the Linked Open Data cloud. Geospatial Interlinking aims to cover this gap by associating geometries with topological relations like those of the Dimensionally Extended 9-Intersection Model. Due to its quadratic time complexity, various algorithms aim to carry out Geospatial Interlinking efficiently. We present JedAI-spatial, a novel, open-source system that organizes these algorithms according to three dimensions: (i) Space Tiling, which determines the approach that reduces the search space, (ii) Budget-awareness, which distinguishes interlinking algorithms into batch and progressive ones, and (iii) Execution mode, which discerns between serial algorithms, running on a single CPU-core, and parallel ones, running on top of Apache Spark. We analytically describe JedAI-spatial's architecture and capabilities and perform thorough experiments to provide interesting insights about the relative performance of its algorithms.


title: ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters

abstract: The incredible feats of athleticism demonstrated by humans are made possible in part by a vast repertoire of general-purpose motor skills, acquired through years of practice and experience. These skills not only enable humans to perform complex tasks, but also provide powerful priors for guiding their behaviors when learning new tasks. This is in stark contrast to what is common practice in physics-based character animation, where control policies are most typically trained from scratch for each task. In this work, we present a large-scale data-driven framework for learning versatile and reusable skill embeddings for physically simulated characters. Our approach combines techniques from adversarial imitation learning and unsupervised reinforcement learning to develop skill embeddings that produce life-like behaviors, while also providing an easy to control representation for use on new downstream tasks. Our models can be trained using large datasets of unstructured motion clips, without requiring any task-specific annotation or segmentation of the motion data. By leveraging a massively parallel GPU-based simulator, we are able to train skill embeddings using over a decade of simulated experiences, enabling our model to learn a rich and versatile repertoire of skills. We show that a single pre-trained model can be effectively applied to perform a diverse set of new tasks. Our system also allows users to specify tasks through simple reward functions, and the skill embedding then enables the character to automatically synthesize complex and naturalistic strategies in order to achieve the task objectives.


title: Cross-lingual Word Embeddings in Hyperbolic Space

abstract: Cross-lingual word embeddings can be applied to several natural language processing applications across multiple languages. Unlike prior works that use word embeddings based on the Euclidean space, this short paper presents a simple and effective cross-lingual Word2Vec model that adapts to the Poincar\'e ball model of hyperbolic space to learn unsupervised cross-lingual word representations from a German-English parallel corpus. It has been shown that hyperbolic embeddings can capture and preserve hierarchical relationships. We evaluate the model on both hypernymy and analogy tasks. The proposed model achieves comparable performance with the vanilla Word2Vec model on the cross-lingual analogy task, the hypernymy task shows that the cross-lingual Poincar\'e Word2Vec model can capture latent hierarchical structure from free text across languages, which are absent from the Euclidean-based Word2Vec representations. Our results show that by preserving the latent hierarchical information, hyperbolic spaces can offer better representations for cross-lingual embeddings.


title: Rings of invariants for three dimensional modular representations

abstract: Let $p>3$ be a prime number. We compute the rings of invariants of the elementary abelian $p$-group $(\mathbb Z/p\mathbb Z)^r$ for $3$-dimensional generic representations. Furthermore we show that these rings of invariants are complete intersections rings with embedding dimension $\lceil r/2\rceil +3$. This proves a conjecture of Campbell, Shank and Wehlau in [CSW], which they proved for $r=3$, and later Pierron and Shank proved it for $r=4$.


title: Modeling Task Interactions in Document-Level Joint Entity and Relation Extraction

abstract: We target on the document-level relation extraction in an end-to-end setting, where the model needs to jointly perform mention extraction, coreference resolution (COREF) and relation extraction (RE) at once, and gets evaluated in an entity-centric way. Especially, we address the two-way interaction between COREF and RE that has not been the focus by previous work, and propose to introduce explicit interaction namely Graph Compatibility (GC) that is specifically designed to leverage task characteristics, bridging decisions of two tasks for direct task interference. Our experiments are conducted on DocRED and DWIE; in addition to GC, we implement and compare different multi-task settings commonly adopted in previous work, including pipeline, shared encoders, graph propagation, to examine the effectiveness of different interactions. The result shows that GC achieves the best performance by up to 2.3/5.1 F1 improvement over the baseline.


title: The Fredholm Navier-Stokes type equations for the de Rham complex over weighted H\"older spaces

abstract: We consider a family of initial problems for the Navier-Stokes type equations generated by the de Rham complex in ${\mathbb R}^n \times [0,T]$, $n\geq 2$, with a positive time $T$ over a scale weighted anisotropic H\"older spaces. As the weights control the order of zero at the infinity with respect to the space variables for vectors fields under the consideration, this actually leads to initial problems over a compact manifold with the singular conic point at the infinity. We prove that each problem from the family induces Fredholm open injective mappings on elements of the scales. At the step $1$ of the complex we may apply the results to the classical Navier-Stokes equations for incompressible viscous fluid.


title: Constraints on the dark photon from parity violation and the $W$ mass

abstract: We present an analysis of the experimental data for parity-violating electron scattering (PVES) and atomic parity-violation, including the effects of a dark photon. We derive the favored region of dark photon parameter space, which provides a good description of the experimental data from the Qweak Collaboration and the Jefferson Lab PVDIS Collaboration and simultaneously relieves the tension between the neutron skin thickness determined in the PREX-II experiment and nuclear-model predictions. In addition, we extract the parameter region required to explain the latest W-boson mass anomaly. Our results indicate that a heavy dark photon with mass above the Z boson mass is favored, while other sources of new physics beyond the Standard Model in addition to the dark photon would also be expected.


title: A Scalable Algorithm for Shape Optimization with Geometric Constraints in Banach Spaces

abstract: This work develops an algorithm for PDE-constrained shape optimization based on Lipschitz transformations. Building on previous work in this field, the $p$-Laplace operator is utilized to approximate a descent method for Lipschitz shapes. In particular, it is shown how geometric constraints are algorithmically incorporated avoiding penalty terms by assigning them to the subproblem of finding a suitable descent direction. A special focus is placed on the scalability of the proposed methods for large scale parallel computers via the application of multigrid solvers. The preservation of mesh quality under large deformations, where shape singularities have to be smoothed or generated within the optimization process, is also discussed. It is shown that the interaction of hierarchically refined grids and shape optimization can be realized by the choice of appropriate descent directions. The performance of the proposed methods is demonstrated for energy dissipation minimization in fluid dynamics applications.


title: Ionic Bonds Control Ferroelectric Behavior in Wurtzite Nitrides

abstract: Ferroelectricity enables key integrated technologies from non-volatile memory to precision ultrasound. Wurtzite ferroelectric Al1-xScxN has recently attracted attention because of its robust ferroelectricity and Si process compatibility in addition to being the first known ferroelectric wurtzite. However, the origin and control of ferroelectricity in wurtzite materials is not yet fully understood. Here we show that the local bond ionicity, rather than simply the change in tetrahedral distortion, is key to controlling the macroscopic ferroelectric response, according to our coupled experimental and computational results. Across the composition gradient in Sc < 0.35 range and 140-260 nm thickness in combinatorial thin films of Al1-xScxN, the pure wurtzite phase exhibits a similar c/a ratio regardless of the Sc content, due to elastic interaction with neighboring crystals. The coercive field and spontaneous polarization significantly decrease with increasing Sc content despite this invariant c/a ratio, due to the more ionic bonding nature of Sc-N relative to the more covalent Al-N bonds, supported by DFT calculations. Based on these insights, ionicity engineering is introduced as an approach to reduce coercive field of Al1-xScxN for memory and other applications and to control ferroelectric properties in other wurtzites.


title: Total positivity of copulas from a Markov kernel perspective

abstract: The underlying dependence structure between two random variables can be described in manifold ways. This includes the examination of certain dependence properties such as lower tail decreasingness (LTD), stochastic increasingness (SI) or total positivity of order 2, the latter usually considered for a copula (TP2) or (if existent) its density (d-TP2). In the present paper we investigate total positivity of order 2 for a copula's Markov kernel (MK-TP2 for short), a positive dependence property that is stronger than TP2 and SI, weaker than d-TP2 but, unlike d-TP2, is not restricted to absolutely continuous copulas, making it presumably the strongest dependence property defined for any copula (including those with a singular part such as Marshall-Olkin copulas). We examine the MK-TP2 property for different copula families, among them the class of Archimedean copulas and the class of extreme value copulas. In particular we show that, within the class of Archimedean copulas, the dependence properties SI and MK-TP2 are equivalent.


title: Generalized Knowledge Distillation via Relationship Matching

abstract: The knowledge of a well-trained deep neural network (a.k.a. the "teacher") is valuable for learning similar tasks. Knowledge distillation extracts knowledge from the teacher and integrates it with the target model (a.k.a. the "student"), which expands the student's knowledge and improves its learning efficacy. Instead of enforcing the teacher to work on the same task as the student, we borrow the knowledge from a teacher trained from a general label space -- in this "Generalized Knowledge Distillation (GKD)", the classes of the teacher and the student may be the same, completely different, or partially overlapped. We claim that the comparison ability between instances acts as an essential factor threading knowledge across tasks, and propose the RElationship FacIlitated Local cLassifiEr Distillation (REFILLED) approach, which decouples the GKD flow of the embedding and the top-layer classifier. In particular, different from reconciling the instance-label confidence between models, REFILLED requires the teacher to reweight the hard tuples pushed forward by the student and then matches the similarity comparison levels between instances. An embedding-induced classifier based on the teacher model supervises the student's classification confidence and adaptively emphasizes the most related supervision from the teacher. REFILLED demonstrates strong discriminative ability when the classes of the teacher vary from the same to a fully non-overlapped set w.r.t. the student. It also achieves state-of-the-art performance on standard knowledge distillation, one-step incremental learning, and few-shot learning tasks.


title: Identifying diverging-effective mass in MOSFET and $^3$He systems

abstract: Emerging devices such as a neuromorphic device and a qubit can use the Mott transition phenomenon, but in particular, the diverging mechanism of the phenomenon remains to be clarified. The diverging-effective mass near Mott insulators was measured in strongly correlated Mott systems such as a fermion $^3$He and a Si metal-oxide-semiconductor-field-effect transistor, and is closely fitted by the effective mass obtained by the extension of the Brinkman-Rice(BR) picture, $m^*/m=1/[1-(U/U_c)^2]=1/(1-{\kappa}^2_{BR}{\rho}^4)$ when ${\kappa}^2_{BR}{\approx}1({\neq}$1), where $0<U/U_c={\kappa}_{BR}{\rho}^2<1$, correlation strength is ${\kappa}_{BR}$, band-filling is ${\rho}$. Its identification is a percolation of a constant mass in the Brinkman-Rice picture. Over ${\kappa}_{BR}{\approx}0.96$ is evaluated.


title: CoCa: Contrastive Captioners are Image-Text Foundation Models

abstract: Exploring large-scale pretrained foundation models is of significant interest in computer vision because these models can be quickly transferred to many downstream tasks. This paper presents Contrastive Captioner (CoCa), a minimalist design to pretrain an image-text encoder-decoder foundation model jointly with contrastive loss and captioning loss, thereby subsuming model capabilities from contrastive approaches like CLIP and generative methods like SimVLM. In contrast to standard encoder-decoder transformers where all decoder layers attend to encoder outputs, CoCa omits cross-attention in the first half of decoder layers to encode unimodal text representations, and cascades the remaining decoder layers which cross-attend to the image encoder for multimodal image-text representations. We apply a contrastive loss between unimodal image and text embeddings, in addition to a captioning loss on the multimodal decoder outputs which predicts text tokens autoregressively. By sharing the same computational graph, the two training objectives are computed efficiently with minimal overhead. CoCa is pretrained end-to-end and from scratch on both web-scale alt-text data and annotated images by treating all labels simply as text, seamlessly unifying natural language supervision for representation learning. Empirically, CoCa achieves state-of-the-art performance with zero-shot transfer or minimal task-specific adaptation on a broad range of downstream tasks, spanning visual recognition (ImageNet, Kinetics-400/600/700, Moments-in-Time), crossmodal retrieval (MSCOCO, Flickr30K, MSR-VTT), multimodal understanding (VQA, SNLI-VE, NLVR2), and image captioning (MSCOCO, NoCaps). Notably on ImageNet classification, CoCa obtains 86.3% zero-shot top-1 accuracy, 90.6% with a frozen encoder and learned classification head, and new state-of-the-art 91.0% top-1 accuracy on ImageNet with a finetuned encoder.


title: Quantization of Interacting Galilean Field theories

abstract: We present the quantum field description of Galilean electrodynamics minimally coupled to massless Galilean fermion in (3 + 1) dimensions. At the classical level, the Lagrangian is obtained as a null reduction of a relativistic theory in one higher dimension. We use functional techniques to develop the quantum field description of the theory. Quantum corrections to the propagators and vertex are obtained upto first order and the theory is found to be renormalizable to this order. The beta function of the theory is found to grow linearly; the theory is not asymptotically free.


title: Photon surfaces, shadows and accretion disks in gravity with minimally coupled scalar field

abstract: In this article, we conduct a sequential study of possible observable images of black hole simulators described by two recently obtained rotating geometries in Einstein gravity, minimally coupled to a scalar field. One of them, "Kerr-like" (KL), can be seen as a legitimate alternative to the rotating Fisher-Janis-Newman-Winicour (FJNW) solution, and the other (TSL) is a scalar generalization of the Tomimatsu-Sato solution. Unlike the previous version of the rotating FJNW, these solutions do indeed satisfy the system's equations of motion. Our study includes both analytical and numerical calculations of equatorial circular orbits, photon regions, gravitational shadows, and radiation from thin accretion disks for various values of the object's angular momentum and scalar charge. The TSL solution was found to simulate Kerr for all valid parameter values with high accuracy. The maximum difference between the deviations of shadows from a circle for the Kerr and TSL cases does not exceed 1% and fits into the experimental observational data M87*. However, near-extreme objects show two times smaller peak values of the observed outflow luminosity of the accretion disk than for the Kerr black hole. The KL solution cannot be ruled out by the experimental data for small values of the scalar charge either. As the scalar charge increases, the optical properties change dramatically. The shadow can become multiply connected, strongly oblate, and the photon region does not hide the singularity, so it should be classified as a strong singularity.


title: Scene Clustering Based Pseudo-labeling Strategy for Multi-modal Aerial View Object Classification

abstract: Multi-modal aerial view object classification (MAVOC) in Automatic target recognition (ATR), although an important and challenging problem, has been under studied. This paper firstly finds that fine-grained data, class imbalance and various shooting conditions preclude the representational ability of general image classification. Moreover, the MAVOC dataset has scene aggregation characteristics. By exploiting these properties, we propose Scene Clustering Based Pseudo-labeling Strategy (SCP-Label), a simple yet effective method to employ in post-processing. The SCP-Label brings greater accuracy by assigning the same label to objects within the same scene while also mitigating bias and confusion with model ensembles. Its performance surpasses the official baseline by a large margin of +20.57% Accuracy on Track 1 (SAR), and +31.86% Accuracy on Track 2 (SAR+EO), demonstrating the potential of SCP-Label as post-processing. Finally, we win the championship both on Track1 and Track2 in the CVPR 2022 Perception Beyond the Visible Spectrum (PBVS) Workshop MAVOC Challenge. Our code is available at https://github.com/HowieChangchn/SCP-Label.


title: Second Order Path Variationals in Non-Stationary Online Learning

abstract: We consider the problem of universal dynamic regret minimization under exp-concave and smooth losses. We show that appropriately designed Strongly Adaptive algorithms achieve a dynamic regret of $\tilde O(d^2 n^{1/5} C_n^{2/5} \vee d^2)$, where $n$ is the time horizon and $C_n$ a path variational based on second order differences of the comparator sequence. Such a path variational naturally encodes comparator sequences that are piecewise linear -- a powerful family that tracks a variety of non-stationarity patterns in practice (Kim et al, 2009). The aforementioned dynamic regret rate is shown to be optimal modulo dimension dependencies and poly-logarithmic factors of $n$. Our proof techniques rely on analysing the KKT conditions of the offline oracle and requires several non-trivial generalizations of the ideas in Baby and Wang, 2021, where the latter work only leads to a slower dynamic regret rate of $\tilde O(d^{2.5}n^{1/3}C_n^{2/3} \vee d^{2.5})$ for the current problem.


title: Performance evaluations on the parallel CHAracteristic-Spectral-Mixed (CHASM) scheme

abstract: Performance evaluations on the deterministic algorithms for 6-D problems are rarely found in literatures except some recent advances in the Vlasov and Boltzmann community [Dimarco et al. (2018), Kormann et al. (2019)], due to the extremely high complexity. Thus a detailed comparison among various techniques shall be useful to the researchers in the related fields. We try to make a thorough evaluation on a parallel CHAracteristic-Spectral-Mixed (CHASM) scheme to support its usage. CHASM utilizes the cubic B-spline expansion in the spatial space and spectral expansion in the momentum space, which many potentially overcome the computational burden in solving classical and quantum kinetic equations in 6-D phase space. Our purpose is three-pronged. First, we would like show that by imposing some effective Hermite boundary conditions, the local cubic spline can approximate to the global one as accurately as possible. Second, we will illustrate the necessity of adopting the truncated kernel method in calculating the pseudodifferential operator with a singular symbol, since the widely used pseudo-spectral method [Ringhofer (1990)] might fail to properly tackle the singularity. Finally, we make a comparison among non-splitting Lawson schemes and Strang operator splitting. Our numerical results demonstrate the advantage of the one-stage Lawson predictor-corrector scheme over multi-stage ones as well as the splitting scheme in both accuracy and stability.


title: Exploring the hadronic origin of LHAASO J1908+0621

abstract: Recent observations by the Large High Altitude Air Shower Observatory (LHAASO) have paved the way for the observational detection of PeVatrons in the Milky Way Galaxy, thus revolutionizing the field of $\gamma$-ray astrophysics. In this paper, we study one such detected source, LHAASO J1908+0621, and explore the origin of multi-TeV $\gamma$-ray emission from this source. A middle-aged radio supernova remnant SNR G40.5-0.5 and a GeV pulsar PSR J1907+0602 are co-spatial with LHAASO J1908+0621. Dense molecular clouds are also found to be associated with SNR G40.5-0.5. We explain the multi-TeV $\gamma$-ray emission observed from the direction of LHAASO J1908+0621, by the hadronic interaction between accelerated protons that escaped from the SNR shock front and cold protons present inside the dense molecular clouds, and the leptonic emission from the pulsar wind nebula (PWN) associated with the pulsar J1907+0602. Moreover, we explain lower energy $\gamma$-ray emission by considering the radiative cooling of the electrons that escaped from SNR G40.5-0.5. Finally, the combined lepto-hadronic scenario was used to explain the multi-wavelength spectral energy distribution (SED) of LHAASO J1908+0621. Although not yet significant, an ICECUBE hotspot of neutrino emission is spatially associated with LHAASO J1908+0621, indicating a possible hadronic contribution. In this paper, we show that if a hadronic component is present in LHAASO J1908+0621, then the second generation ICECUBE observatory will detect neutrino from this source.


title: Zero-Episode Few-Shot Contrastive Predictive Coding: Solving intelligence tests without prior training

abstract: Video prediction models often combine three components: an encoder from pixel space to a small latent space, a latent space prediction model, and a generative model back to pixel space. However, the large and unpredictable pixel space makes training such models difficult, requiring many training examples. We argue that finding a predictive latent variable and using it to evaluate the consistency of a future image enables data-efficient predictions because it precludes the necessity of a generative model training. To demonstrate it, we created sequence completion intelligence tests in which the task is to identify a predictably changing feature in a sequence of images and use this prediction to select the subsequent image. We show that a one-dimensional Markov Contrastive Predictive Coding (M-CPC_1D) model solves these tests efficiently, with only five examples. Finally, we demonstrate the usefulness of M-CPC_1D in solving two tasks without prior training: anomaly detection and stochastic movement video prediction.


title: Continued Fractions and Probability Estimations in the Shor Algorithm -- A Detailed and Self-Contained Treatise

abstract: The algorithm of Shor for prime factorization is a hybrid algorithm consisting of a quantum part and a classical part. The main focus of the classical part is a continued fraction analysis. The presentation of this is often short, pointing to text books on number theory. In this contribution, we present the relevant results and proofs from the theory of continued fractions in detail (even in more detail than in text books) filling the gap to allow a complete comprehension of the algorithm of Shor. Similarly, we provide a detailed computation of the estimation of the probability that convergents will provide the period required for determining a prime factor.


title: Freeness of type $B$ and conditional freeness for random matrices

abstract: The asymptotic freeness of independent unitarily invariant $N\times N$ random matrices holds in expectation up to $O(N^{-2})$. An already known consequence is the infinitesimal freeness in expectation. We put in evidence another consequence for unitarily invariant random matrices: the almost sure asymptotic freeness of type $B$. As byproducts, we recover the asymptotic cyclic monotonicity, and we get the asymptotic conditional freeness. In particular, the eigenvector empirical spectral distribution of the sum of two randomly rotated random matrices converges towards the conditionally free convolution. We also show new connections between infinitesimal freeness, freeness of type $B$, conditional freeness, cyclic monotonicity and monotone independence. Finally, we show rigorously that the BBP phase transition for an additive rank-one perturbation of a GUE matrix is a consequence of the asymptotic conditional freeness, and the arguments extend to the study of the outlier eigenvalues of other unitarily invariant ensembles.


title: Probabilistic Symmetry for Multi-Agent Dynamics

abstract: Learning multi-agent dynamics is a core AI problem with broad applications in robotics and autonomous driving. While most existing works focus on deterministic prediction, producing probabilistic forecasts to quantify uncertainty and assess risks is critical for downstream decision-making tasks such as motion planning and collision avoidance. Multi-agent dynamics often contains internal symmetry. By leveraging symmetry, specifically rotation equivariance, we can improve not only the prediction accuracy but also uncertainty calibration. We introduce Energy Score, a proper scoring rule, to evaluate probabilistic predictions. We propose a novel deep dynamics model, Probabilistic Equivariant Continuous COnvolution (PECCO) for probabilistic prediction of multi-agent trajectories. PECCO extends equivariant continuous convolution to model the joint velocity distribution of multiple agents. It uses dynamics integration to propagate the uncertainty from velocity to position. On both synthetic and real-world datasets, PECCO shows significant improvements in accuracy and calibration compared to non-equivariant baselines.


title: Some identities involving degenerate Stirling numbers associated with several degenerate polynomials and numbers

abstract: The aim of this paper is to investigate some properties, recurrence relations and identities involving degenerate Stirling numbers of both kinds associated with degenerate hyperharmonic numbers and also with degenerate Bernolli, degenerate Euler, degenerate Bell and degenerate Fubini polynomials.


title: Explain to Not Forget: Defending Against Catastrophic Forgetting with XAI

abstract: The ability to continuously process and retain new information like we do naturally as humans is a feat that is highly sought after when training neural networks. Unfortunately, the traditional optimization algorithms often require large amounts of data available during training time and updates wrt. new data are difficult after the training process has been completed. In fact, when new data or tasks arise, previous progress may be lost as neural networks are prone to catastrophic forgetting. Catastrophic forgetting describes the phenomenon when a neural network completely forgets previous knowledge when given new information. We propose a novel training algorithm called training by explaining in which we leverage Layer-wise Relevance Propagation in order to retain the information a neural network has already learned in previous tasks when training on new data. The method is evaluated on a range of benchmark datasets as well as more complex data. Our method not only successfully retains the knowledge of old tasks within the neural networks but does so more resource-efficiently than other state-of-the-art solutions.


title: Explainable Anomaly Detection for Industrial Control System Cybersecurity

abstract: Industrial Control Systems (ICSs) are becoming more and more important in managing the operation of many important systems in smart manufacturing, such as power stations, water supply systems, and manufacturing sites. While massive digital data can be a driving force for system performance, data security has raised serious concerns. Anomaly detection, therefore, is essential for preventing network security intrusions and system attacks. Many AI-based anomaly detection methods have been proposed and achieved high detection performance, however, are still a "black box" that is hard to be interpreted. In this study, we suggest using Explainable Artificial Intelligence to enhance the perspective and reliable results of an LSTM-based Autoencoder-OCSVM learning model for anomaly detection in ICS. We demonstrate the performance of our proposed method based on a well-known SCADA dataset.


title: Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unlabeled, unannotated pathology slides

abstract: Definitive cancer diagnosis and management depend upon the extraction of information from microscopy images by pathologists. These images contain complex information requiring time-consuming expert human interpretation that is prone to human bias. Supervised deep learning approaches have proven powerful for classification tasks, but they are inherently limited by the cost and quality of annotations used for training these models. To address this limitation of supervised methods, we developed Histomorphological Phenotype Learning (HPL), a fully blue{self-}supervised methodology that requires no expert labels or annotations and operates via the automatic discovery of discriminatory image features in small image tiles. Tiles are grouped into morphologically similar clusters which constitute a library of histomorphological phenotypes, revealing trajectories from benign to malignant tissue via inflammatory and reactive phenotypes. These clusters have distinct features which can be identified using orthogonal methods, linking histologic, molecular and clinical phenotypes. Applied to lung cancer tissues, we show that they align closely with patient survival, with histopathologically recognised tumor types and growth patterns, and with transcriptomic measures of immunophenotype. We then demonstrate that these properties are maintained in a multi-cancer study. These results show the clusters represent recurrent host responses and modes of tumor growth emerging under natural selection. Code, pre-trained models, learned embeddings, and documentation are available to the community at https://github.com/AdalbertoCq/Histomorphological-Phenotype-Learning


title: Early Detection of Spam Domains with Passive DNS and SPF

abstract: Spam domains are sources of unsolicited mails and one of the primary vehicles for fraud and malicious activities such as phishing campaigns or malware distribution. Spam domain detection is a race: as soon as the spam mails are sent, taking down the domain or blacklisting it is of relative use, as spammers have to register a new domain for their next campaign. To prevent malicious actors from sending mails, we need to detect them as fast as possible and, ideally, even before the campaign is launched. In this paper, using near-real-time passive DNS data from Farsight Security, we monitor the DNS traffic of newly registered domains and the contents of their TXT records, in particular, the configuration of the Sender Policy Framework, an anti-spoofing protocol for domain names and the first line of defense against devastating Business Email Compromise scams. Because spammers and benign domains have different SPF rules and different traffic profiles, we build a new method to detect spam domains using features collected from passive DNS traffic. Using the SPF configuration and the traffic to the TXT records of a domain, we accurately detect a significant proportion of spam domains with a low false positives rate demonstrating its potential in real-world deployments. Our classification scheme can detect spam domains before they send any mail, using only a single DNS query and later on, it can refine its classification by monitoring more traffic to the domain name.