title: A semi-analytical model to simulate the spin-diode effect and accelerate its use in neuromorphic computing

abstract: The spin-diode effect is studied both experimentally and with our original semi-analytical method. The latter is based on an improved version of the Thiele equation approach (TEA) that we combine to micromagnetic simulation data to accurately model the non-linear dynamics of spin-torque vortex oscillator (STVO). This original method, called data-driven Thiele equation approach (DD-TEA), absorbs the difference between the analytical model and micromagnetic simulations to provide a both ultra-fast and quantitative model. The DD-TEA model predictions also agree very well with the experimental data. The reversal of the spin-diode effect with the chirality of the vortex, the impact of the input current and the origin of a variation at half of the STVO frequency are presented as well as the ability of the model to reproduce the experimental behavior. Finally, the spin-diode effect and its simulation using the DD-TEA model are discussed as a promising perspective in the framework of neuromorphic computing.


title: Unearthing InSights into Mars: Unsupervised Source Separation with Limited Data

abstract: Source separation involves the ill-posed problem of retrieving a set of source signals that have been observed through a mixing operator. Solving this problem requires prior knowledge, which is commonly incorporated by imposing regularity conditions on the source signals, or implicitly learned through supervised or unsupervised methods from existing data. While data-driven methods have shown great promise in source separation, they often require large amounts of data, which rarely exists in planetary space missions. To address this challenge, we propose an unsupervised source separation scheme for domains with limited data access that involves solving an optimization problem in the wavelet scattering covariance representation space$\unicode{x2014}$an interpretable, low-dimensional representation of stationary processes. We present a real-data example in which we remove transient, thermally-induced microtilts$\unicode{x2014}$known as glitches$\unicode{x2014}$from data recorded by a seismometer during NASA's InSight mission on Mars. Thanks to the wavelet scattering covariances' ability to capture non-Gaussian properties of stochastic processes, we are able to separate glitches using only a few glitch-free data snippets.


title: Strategy evolution on dynamic networks

abstract: Models of strategy evolution on static networks help us understand how population structure can promote the spread of traits like cooperation. One key mechanism is the formation of altruistic spatial clusters, where neighbors of a cooperative individual are likely to reciprocate, which protects prosocial traits from exploitation. But most real-world interactions are ephemeral and subject to exogenous restructuring, so that social networks change over time. Strategic behavior on dynamic networks is difficult to study, and much less is known about the resulting evolutionary dynamics. Here, we provide an analytical treatment of cooperation on dynamic networks, allowing for arbitrary spatial and temporal heterogeneity. We show that transitions among a large class of network structures can favor the spread of cooperation, even if each individual social network would inhibit cooperation when static. Furthermore, we show that spatial heterogeneity tends to inhibit cooperation, whereas temporal heterogeneity tends to promote it. Dynamic networks can have profound effects on the evolution of prosocial traits, even when individuals have no agency over network structures.


title: Complexity and chaotic behavior of the U.S. rivers and estimation of their prediction horizon

abstract: A streamflow time series encompasses a large amount of hidden information and reliable prediction of its behavior in the future remains a challenge. It seems that the use of information measures can significantly contribute to determining the time horizon of rivers and improving predictability. Using the Kolmogorov complexity (KC) and its derivatives (KC spectrum and its highest value), and Lyapunov exponent (LE), it has previously been shown that the degree of streamflow predictability depends on human activities, environmental factors, and natural characteristics. This paper applied the KC and LE measures to investigate the randomness and chaotic behavior of monthly streamflow of 1879 rivers from the United States for a period from 1950 to 2015 and evaluated their time horizons via the Lyapunov and Kolmogorov time (LT and KT, respectively).


title: Dual Control of Exploration and Exploitation for Auto-Optimisation Control with Active Learning

abstract: The quest for optimal operation in environments with unknowns and uncertainties is highly desirable but critically challenging across numerous fields. This paper develops a dual control framework for exploration and exploitation (DCEE) to solve an auto-optimisation problem in such complex settings. In general, there is a fundamental conflict between tracking an unknown optimal operational condition and parameter identification. The DCEE framework stands out by eliminating the need for additional perturbation signals, a common requirement in existing adaptive control methods. Instead, it inherently incorporates an exploration mechanism, actively probing the uncertain environment to diminish belief uncertainty. An ensemble based multi-estimator approach is developed to learn the environmental parameters and in the meanwhile quantify the estimation uncertainty in real time. The control action is devised with dual effects, which not only minimises the tracking error between the current state and the believed unknown optimal operational condition but also reduces belief uncertainty by proactively exploring the environment. Formal properties of the proposed DCEE framework like convergence are established. A numerical example is used to validate the effectiveness of the proposed DCEE. Simulation results for maximum power point tracking are provided to further demonstrate the potential of this new framework in real world applications.


title: The triple-peaked afterglow of GRB 210731A from X-ray to radio frequencies

abstract: GRB 210731A was a long-duration gamma-ray burst discovered by the Burst Alert Telescope (BAT) aboard the Neil Gehrels Swift observatory. Swift triggered the wide-field, robotic MeerLICHT optical telescope in Sutherland; it began observing the BAT error circle 286 seconds after the Swift trigger and discovered the optical afterglow of GRB 210731A in its first 60-second q-band exposure. Multi-colour observations of the afterglow with MeerLICHT revealed a light curve that showed three peaks of similar brightness within the first four hours. We present the results of our follow-up campaign and interpret our observations in the framework of the synchrotron forward shock model. We performed temporal and spectral fits to determine the spectral regime and external medium density profile, and performed detailed multi-wavelength theoretical modelling of the afterglow following the last optical peak at 0.2 days to determine the intrinsic blast wave parameters. We find a preference for a stellar wind density profile consistent with a massive star origin, while our theoretical modelling results in fairly typical shock microphysics parameters. Based on the energy released in gamma-rays and the kinetic energy in the blast wave, we determine a low radiative efficiency of ~0.02. The first peak in the optical light curve is likely the onset of the afterglow. We find that energy injection into the forward shock offers the simplest explanation for the subsequent light curve evolution, and that the blast wave kinetic energy increasing by a factor of ~1000 from the first peak to the last peak is indicative of substantial energy injection. Our highest-likelihood theoretical model overpredicts the 1.4 GHz flux by a factor of approximately three with respect to our upper limits, possibly implying a population of thermal electrons within the shocked region.


title: Enhancing Face Recognition with Latent Space Data Augmentation and Facial Posture Reconstruction

abstract: The small amount of training data for many state-of-the-art deep learning-based Face Recognition (FR) systems causes a marked deterioration in their performance. Although a considerable amount of research has addressed this issue by inventing new data augmentation techniques, using either input space transformations or Generative Adversarial Networks (GAN) for feature space augmentations, these techniques have yet to satisfy expectations. In this paper, we propose an approach named the Face Representation Augmentation (FRA) for augmenting face datasets. To the best of our knowledge, FRA is the first method that shifts its focus towards manipulating the face embeddings generated by any face representation learning algorithm to create new embeddings representing the same identity and facial emotion but with an altered posture. Extensive experiments conducted in this study convince of the efficacy of our methodology and its power to provide noiseless, completely new facial representations to improve the training procedure of any FR algorithm. Therefore, FRA can help the recent state-of-the-art FR methods by providing more data for training FR systems. The proposed method, using experiments conducted on the Karolinska Directed Emotional Faces (KDEF) dataset, improves the identity classification accuracies by 9.52 %, 10.04 %, and 16.60 %, in comparison with the base models of MagFace, ArcFace, and CosFace, respectively.


title: $J/\psi$ polarization in large-$P_T$ semi-inclusive deep-inelastic scattering at the EIC

abstract: We present a detailed phenomenological study of $J/\psi$ polarization in semi-inclusive deep inelastic scattering processes, focusing on the kinematics accessible at the future Electron-Ion Collider. We show theoretical estimates for the standard polarization parameters for different frames usually adopted in the literature, in the large $P_T$ region, namely $P_T\gg \Lambda_\text{QCD}$, where collinear factorization is expected to hold. We adopt both the Color Singlet Model and the Nonrelativistic QCD approach, paying special attention to the role of different sets of Long Distance Matrix Elements. Finally we present a preliminary analysis of some frame independent polarization invariants.


title: Energy-Efficient Distributed Algorithms for Synchronous Networks

abstract: We study the design of energy-efficient algorithms for the LOCAL and CONGEST models. Specifically, as a measure of complexity, we consider the maximum, taken over all the edges, or over all the nodes, of the number of rounds at which an edge, or a node, is active in the algorithm. We first show that every Turing-computable problem has a CONGEST algorithm with constant node-activation complexity, and therefore constant edge-activation complexity as well. That is, every node (resp., edge) is active in sending (resp., transmitting) messages for only $O(1)$ rounds during the whole execution of the algorithm. In other words, every Turing-computable problem can be solved by an algorithm consuming the least possible energy. In the LOCAL model, the same holds obviously, but with the additional feature that the algorithm runs in $O(\mbox{poly}(n))$ rounds in $n$-node networks. However, we show that insisting on algorithms running in $O(\mbox{poly}(n))$ rounds in the CONGEST model comes with a severe cost in terms of energy. Namely, there are problems requiring $\Omega(\mbox{poly}(n))$ edge-activations (and thus $\Omega(\mbox{poly}(n))$ node-activations as well) in the CONGEST model whenever solved by algorithms bounded to run in $O(\mbox{poly}(n))$ rounds. Finally, we demonstrate the existence of a sharp separation between the edge-activation complexity and the node-activation complexity in the CONGEST model, for algorithms bounded to run in $O(\mbox{poly}(n))$ rounds. Specifically, under this constraint, there is a problem with $O(1)$ edge-activation complexity but $\tilde{\Omega}(n^{1/4})$ node-activation complexity.


title: Practical Differentially Private Hyperparameter Tuning with Subsampling

abstract: Tuning the hyperparameters of differentially private (DP) machine learning (ML) algorithms often requires use of sensitive data and this may leak private information via hyperparameter values. Recently, Papernot and Steinke (2022) proposed a certain class of DP hyperparameter tuning algorithms, where the number of random search samples is randomized itself. Commonly, these algorithms still considerably increase the DP privacy parameter $\varepsilon$ over non-tuned DP ML model training and can be computationally heavy as evaluating each hyperparameter candidate requires a new training run. We focus on lowering both the DP bounds and the computational cost of these methods by using only a random subset of the sensitive data for the hyperparameter tuning and by extrapolating the optimal values to a larger dataset. We provide a R\'enyi differential privacy analysis for the proposed method and experimentally show that it consistently leads to better privacy-utility trade-off than the baseline method by Papernot and Steinke.


title: Alignment with human representations supports robust few-shot learning

abstract: Should we care whether AI systems have representations of the world that are similar to those of humans? We provide an information-theoretic analysis that suggests that there should be a U-shaped relationship between the degree of representational alignment with humans and performance on few-shot learning tasks. We confirm this prediction empirically, finding such a relationship in an analysis of the performance of 491 computer vision models. We also show that highly-aligned models are more robust to both natural adversarial attacks and domain shifts. Our results suggest that human-alignment is often a sufficient, but not necessary, condition for models to make effective use of limited data, be robust, and generalize well.


title: Real-time non-perturbative dynamics of jet production: quantum entanglement and vacuum modification

abstract: The production of jets should allow testing the real-time response of the QCD vacuum disturbed by the propagation of high-momentum color charges. Addressing this problem theoretically requires a real-time, non-perturbative method. It is well known that the Schwinger model [QED in $(1+1)$ dimensions] shares many common properties with QCD, including confinement, chiral symmetry breaking, and the existence of vacuum fermion condensate. As a step in developing such an approach, we report here on fully quantum simulations of a massive Schwinger model coupled to external sources representing quark and antiquark jets as produced in $e^+e^-$ annihilation. We study, for the first time, the modification of the vacuum chiral condensate by the propagating jets and the quantum entanglement between the fragmenting jets. Our results indicate strong entanglement between the fragmentation products of the two jets at rapidity separations $\Delta \eta \leq 2$, which can potentially exist also in QCD and can be studied in experiments.


title: Data sparse multilevel covariance estimation in optimal complexity

abstract: We consider the $\mathcal{H}^2$-formatted compression and computational estimation of covariance functions on a compact set in $\mathbb{R}^d$. The classical sample covariance or Monte Carlo estimator is prohibitively expensive for many practically relevant problems, where often approximation spaces with many degrees of freedom and many samples for the estimator are needed. In this article, we propose and analyze a data sparse multilevel sample covariance estimator, i.e., a multilevel Monte Carlo estimator. For this purpose, we generalize the notion of asymptotically smooth kernel functions to a Gevrey type class of kernels for which we derive new variable-order $\mathcal{H}^2$-approximation rates. These variable-order $\mathcal{H}^2$-approximations can be considered as a variant of $hp$-approximations. Our multilevel sample covariance estimator then uses an approximate multilevel hierarchy of variable-order $\mathcal{H}^2$-approximations to compress the sample covariances on each level. The non-nestedness of the different levels makes the reduction to the final estimator nontrivial and we present a suitable algorithm which can handle this task in linear complexity. This allows for a data sparse multilevel estimator of Gevrey covariance kernel functions in the best possible complexity for Monte Carlo type multilevel estimators, which is quadratic. Numerical examples which estimate covariance matrices with tens of billions of entries are presented.


title: Quantum Langevin theory for two coupled phase-conjugated electromagnetic waves

abstract: While loss-gain-induced Langevin noises have been intensively studied in quantum optics, the effect of a complex-valued nonlinear coupling coefficient on the noises of two coupled phase-conjugated optical fields has never been questioned before. Here, we provide a general macroscopic phenomenological formula of quantum Langevin equations for two coupled phase-conjugated fields with linear loss (gain) and complex nonlinear coupling coefficient. The macroscopic phenomenological formula is obtained from the coupling matrix to preserve the field commutation relations and correlations, which does not require knowing the microscopic details of light-matter interaction and internal atomic structures. To validate this phenomenological formula, we take spontaneous four-wave mixing in a double-$\Lambda$ four-level atomic system as an example to numerically confirm that our macroscopic phenomenological result is consistent with that obtained from the microscopic Heisenberg-Langevin theory. Finally, we apply the quantum Langevin equations to study the effects of linear gain and loss, complex phase mismatching, as well as complex nonlinear coupling coefficient in entangled photon pair (biphoton) generation, particularly to their temporal quantum correlations.


title: Gender and Prestige Bias in Coronavirus News Reporting

abstract: Journalists play a vital role in surfacing issues of societal importance, but their choices of what to highlight and who to interview are influenced by societal biases. In this work, we use natural language processing tools to measure these biases in a large corpus of news articles about the Covid-19 pandemic. Specifically, we identify when experts are quoted in news and extract their names and institutional affiliations. We enrich the data by classifying each expert's gender, the type of organization they belong to, and for academic institutions, their ranking. Our analysis reveals disparities in the representation of experts in news. We find a substantial gender gap, where men are quoted three times more than women. The gender gap varies by partisanship of the news source, with conservative media exhibiting greater gender bias. We also identify academic prestige bias, where journalists turn to experts from highly-ranked academic institutions more than experts from less prestigious institutions, even if the latter group has more public health expertise. Liberal news sources exhibit slightly more prestige bias than conservative sources. Equality of representation is essential to enable voices from all groups to be heard. By auditing bias, our methods help identify blind spots in news coverage.


title: A New Symmetric Homomorphic Functional Encryption over a Hidden Ring for Polynomial Public Key Encapsulations

abstract: This paper proposes a new homomorphic functional encryption using modular multiplications over a hidden ring. Unlike traditional homomorphic encryption where users can only passively perform ciphertext addition or multiplication, the homomorphic functional encryption retains homomorphic addition and scalar multiplication properties, but also allows for the user's inputs through polynomial variables. The proposed homomorphic encryption can be applied to any polynomials over a finite field, with their coefficients considered as their privacy. We denote the polynomials before homomorphic encryption as plain polynomials and after homomorphic encryption as cipher polynomials. A cipher polynomial can be evaluated with variables from the finite field, GF(p), by calculating the monomials of variables modulo a prime p. These properties allow functional homomorphic encryption to be used for public key encryption of certain asymmetric cryptosystems to hide the structure of its central map construction. We propose a new variant of MPKC with homomorphic encryption of its public key. We propose to use a single plaintext vector and a noise vector of multiple variables to be associated with the central map, in place of the secret plaintext vector to be encrypted in MPKC. We call this variant of encrypted MPKC, a Homomorphic Polynomial Public Key algorithm or HPPK algorithm. The HPPK algorithm holds the property of indistinguishability under the chosen-plaintext attacks or IND-CPA. The overall classical complexity to crack the HPPK algorithm is exponential in the size of the prime field GF(p). We briefly report on benchmarking performance results using the SUPERCOP toolkit. Benchmarking results demonstrate that HPPK offers rather fast performance, which is comparable and in some cases outperforms the NIST PQC finalists for key generation, encryption, and decryption.


title: $L^2$ Diffusive Expansion For Neutron Transport Equation

abstract: Grazing set singularity leads to a surprising counter-example and breakdown of the classical mathematical theory for $L^{\infty}$ diffusive expansion of neutron transport equation with in-flow boundary condition in term of the Knudsen number $\varepsilon$, one of the most classical problems in the kinetic theory. Even though a satisfactory new theory has been established by constructing new boundary layers with favorable $\varepsilon$-geometric correction for convex domains, the severe grazing singularity from non-convex domains has prevented any positive mathematical progress. We develop a novel and optimal $L^2$ expansion theory for general domain (including non-convex domain) by discovering a surprising $\varepsilon^{\frac{1}{2}}$ gain for the average of remainder.


title: Prompt-Based Editing for Text Style Transfer

abstract: Prompting approaches have been recently explored in text style transfer, where a textual prompt is used to query a pretrained language model to generate style-transferred texts word by word in an autoregressive manner. However, such a generation process is less controllable and early prediction errors may affect future word predictions. In this paper, we present a prompt-based editing approach for text style transfer. Specifically, we prompt a pretrained language model for style classification and use the classification probability to compute a style score. Then, we perform discrete search with word-level editing to maximize a comprehensive scoring function for the style-transfer task. In this way, we transform a prompt-based generation problem into a classification one, which is a training-free process and more controllable than the autoregressive generation of sentences. In our experiments, we performed both automatic and human evaluation on three style-transfer benchmark datasets, and show that our approach largely outperforms the state-of-the-art systems that have 20 times more parameters. Additional empirical analyses further demonstrate the effectiveness of our approach.


title: Augmented Reality's Potential for Identifying and Mitigating Home Privacy Leaks

abstract: Users face various privacy risks in smart homes, yet there are limited ways for them to learn about the details of such risks, such as the data practices of smart home devices and their data flow. In this paper, we present Privacy Plumber, a system that enables a user to inspect and explore the privacy "leaks" in their home using an augmented reality tool. Privacy Plumber allows the user to learn and understand the volume of data leaving the home and how that data may affect a user's privacy -- in the same physical context as the devices in question, because we visualize the privacy leaks with augmented reality. Privacy Plumber uses ARP spoofing to gather aggregate network traffic information and presents it through an overlay on top of the device in an smartphone app. The increased transparency aims to help the user make privacy decisions and mend potential privacy leaks, such as instruct Privacy Plumber on what devices to block, on what schedule (i.e., turn off Alexa when sleeping), etc. Our initial user study with six participants demonstrates participants' increased awareness of privacy leaks in smart devices, which further contributes to their privacy decisions (e.g., which devices to block).


title: Particle-Number Threshold for Non-Abelian Geometric Phases

abstract: When a quantum state traverses a path, while being under the influence of a gauge potential, it acquires a geometric phase that is often more than just a scalar quantity. The variety of unitary transformations that can be realised by this form of parallel transport depends crucially on the number of particles involved in the evolution. Here, we introduce a particle-number threshold (PNT) that assesses a system's capabilities to perform purely geometric manipulations of quantum states. This threshold gives the minimal number of particles necessary to fully exploit a system's potential to generate non-Abelian geometric phases. Therefore, the PNT might be useful for evaluating the resource demands of a holonomic quantum computer. We benchmark our findings on bosonic systems relevant to linear and nonlinear quantum optics.


title: Beyond Classroom: Making a Difference in Diversity in Tech

abstract: With all the opportunities and risks that technology holds in connection to our safe and sustainable future, it is becoming increasingly important to involve a larger portion of our society in becoming active co-creators of our digitalized future -- moving from the passenger seat to the driver seat. Yet, despite extensive efforts around the world, little progress has been made in growing the representation of certain communities and groups in software engineering. This chapter shares one successful project, called Czechitas, triggering a major social change in Czechia, involving 1 000+ volunteers to support 50 000+ women on their way towards software engineering education and career.


title: Vertex-based reachability analysis for verifying ReLU deep neural networks

abstract: Neural networks achieved high performance over different tasks, i.e. image identification, voice recognition and other applications. Despite their success, these models are still vulnerable regarding small perturbations, which can be used to craft the so-called adversarial examples. Different approaches have been proposed to circumvent their vulnerability, including formal verification systems, which employ a variety of techniques, including reachability, optimization and search procedures, to verify that the model satisfies some property. In this paper we propose three novel reachability algorithms for verifying deep neural networks with ReLU activations. The first and third algorithms compute an over-approximation for the reachable set, whereas the second one computes the exact reachable set. Differently from previously proposed approaches, our algorithms take as input a V-polytope. Our experiments on the ACAS Xu problem show that the Exact Polytope Network Mapping (EPNM) reachability algorithm proposed in this work surpass the state-of-the-art results from the literature, specially in relation to other reachability methods.


title: New isomeric transition in $^{36}$Mg: Bridging the N=20 and N=28 islands of inversion

abstract: We observed a new isomeric gamma transition at 168 keV in $^{36}$Mg, with a half-life of T$_{1/2}$=[130-500]$(\pm40)(^{+800}_{-20})_{sys}$ ns. We propose that the observed transition de-excites a new 0$^+$ isomeric state and populates the previously known first 2$^+$ state. The existence of this isomer is consistent with the predictions of the large-scale shell model calculations of $^{36}$Mg using the sdpf-u-mix interaction. The observed excitation energy of the second 0$^+$ state is caused by the small energy separation between two prolate-deformed configurations where the intruder configuration corresponds to two neutron excitations from the {\it sd} to the {\it pf} shell. Within this interpretation, $^{36}$Mg becomes the crossing point between nuclei in which ground state deformed/superdeformed configurations are caused by the dominance of N=20 intruders ($^{32,34}$Mg) and nuclei where deformed configurations are associated with N=28 intruders ($^{38}$Mg and beyond). We found the lack of three-body monopole corrections in other effective interactions results in a predominance of N=20 intruder configurations past $^{38}$Mg incompatible with our observation. We conclude that $^{36}$Mg bridges the N=20 and N=28 islands of inversion, forming the so-called Big Island of Deformation.


title: Minimizing Trajectory Curvature of ODE-based Generative Models

abstract: Recent ODE/SDE-based generative models, such as diffusion models, rectified flows, and flow matching, define a generative process as a time reversal of a fixed forward process. Even though these models show impressive performance on large-scale datasets, numerical simulation requires multiple evaluations of a neural network, leading to a slow sampling speed. We attribute the reason to the high curvature of the learned generative trajectories, as it is directly related to the truncation error of a numerical solver. Based on the relationship between the forward process and the curvature, here we present an efficient method of training the forward process to minimize the curvature of generative trajectories without any ODE/SDE simulation. Experiments show that our method achieves a lower curvature than previous models and, therefore, decreased sampling costs while maintaining competitive performance. Code is available at https://github.com/sangyun884/fast-ode.


title: Understanding the Effectiveness of Very Large Language Models on Dialog Evaluation

abstract: Language models have steadily increased in size over the past few years. They achieve a high level of performance on various natural language processing (NLP) tasks such as question answering and summarization. Large language models (LLMs) have been used for generation and can now output human-like text. Due to this, there are other downstream tasks in the realm of dialog that can now harness the LLMs' language understanding capabilities. Dialog evaluation is one task that this paper will explore. It concentrates on prompting with LLMs: BLOOM, OPT, GPT-3, Flan-T5, InstructDial and TNLGv2. The paper shows that the choice of datasets used for training a model contributes to how well it performs on a task as well as on how the prompt should be structured. Specifically, the more diverse and relevant the group of datasets that a model is trained on, the better dialog evaluation performs. This paper also investigates how the number of examples in the prompt and the type of example selection used affect the model's performance.


title: EmbedDistill: A Geometric Knowledge Distillation for Information Retrieval

abstract: Large neural models (such as Transformers) achieve state-of-the-art performance for information retrieval (IR). In this paper, we aim to improve distillation methods that pave the way for the resource-efficient deployment of such models in practice. Inspired by our theoretical analysis of the teacher-student generalization gap for IR models, we propose a novel distillation approach that leverages the relative geometry among queries and documents learned by the large teacher model. Unlike existing teacher score-based distillation methods, our proposed approach employs embedding matching tasks to provide a stronger signal to align the representations of the teacher and student models. In addition, it utilizes query generation to explore the data manifold to reduce the discrepancies between the student and the teacher where training data is sparse. Furthermore, our analysis also motivates novel asymmetric architectures for student models which realizes better embedding alignment without increasing online inference cost. On standard benchmarks like MSMARCO, we show that our approach successfully distills from both dual-encoder (DE) and cross-encoder (CE) teacher models to 1/10th size asymmetric students that can retain 95-97% of the teacher performance.


title: Improved knowledge distillation by utilizing backward pass knowledge in neural networks

abstract: Knowledge distillation (KD) is one of the prominent techniques for model compression. In this method, the knowledge of a large network (teacher) is distilled into a model (student) with usually significantly fewer parameters. KD tries to better-match the output of the student model to that of the teacher model based on the knowledge extracts from the forward pass of the teacher network. Although conventional KD is effective for matching the two networks over the given data points, there is no guarantee that these models would match in other areas for which we do not have enough training samples. In this work, we address that problem by generating new auxiliary training samples based on extracting knowledge from the backward pass of the teacher in the areas where the student diverges greatly from the teacher. We compute the difference between the teacher and the student and generate new data samples that maximize the divergence. This is done by perturbing data samples in the direction of the gradient of the difference between the student and the teacher. Augmenting the training set by adding this auxiliary improves the performance of KD significantly and leads to a closer match between the student and the teacher. Using this approach, when data samples come from a discrete domain, such as applications of natural language processing (NLP) and language understanding, is not trivial. However, we show how this technique can be used successfully in such applications. We evaluated the performance of our method on various tasks in computer vision and NLP domains and got promising results.


title: On Semidefinite Representations of Second-order Conic Optimization Problems

abstract: Second-order conic optimization (SOCO) can be considered as a special case of semidefinite optimization (SDO). In the literature it has been advised that a SOCO problem can be embedded in an SDO problem using the arrow-head matrix transformation. However, a primal-dual solution pair cannot be mapped simultaneously using the arrow-head transformation as we might lose complementarity and duality in some cases. To address this issue, we investigate the relationship between SOCO problems, and their SDO counterpart. Through derivation of standard semidefinite representations of SOCO problems, we introduce admissible mappings. We show that the proposed mappings preserve both feasibility and optimality. Further, we discuss how the optimal partition of a SOCO problem maps to the optimal partition of its SDO counterpart.


title: A Deep Potential model for liquid-vapor equilibrium and cavitation rates of water

abstract: Computational studies of liquid water and its phase transition into vapor have traditionally been performed using classical water models. Here we utilize the Deep Potential methodology -- a machine learning approach -- to study this ubiquitous phase transition, starting from the phase diagram in the liquid-vapor coexistence regime. The machine learning model is trained on ab initio energies and forces based on the SCAN density functional which has been previously shown to reproduce solid phases and other properties of water. Here, we compute the surface tension, saturation pressure and enthalpy of vaporization for a range of temperatures spanning from 300 to 600 K, and evaluate the Deep Potential model performance against experimental results and the semi-empirical TIP4P/2005 classical model. Moreover, by employing the seeding technique, we evaluate the free energy barrier and nucleation rate at negative pressures for the isotherm of 296.4 K. We find that the nucleation rates obtained from the Deep Potential model deviate from those computed for the TIP4P/2005 water model, due to an underestimation in the surface tension from the Deep Potential model. From analysis of the seeding simulations, we also evaluate the Tolman length for the Deep Potential water model, which is (0.091 $\pm$ 0.008) nm at 296.4 K. Lastly, we identify that water molecules display a preferential orientation in the liquid-vapor interface, in which H atoms tend to point towards the vapor phase to maximize the enthalpic gain of interfacial molecules. We find that this behaviour is more pronounced for planar interfaces than for the curved interfaces in bubbles. This work represents the first application of Deep Potential models to the study of liquid-vapor coexistence and water cavitation.


title: Inference for all variants of the multivariate coefficient of variation in factorial designs

abstract: The multivariate coefficient of variation (MCV) is an attractive and easy-to-interpret effect size for the dispersion in multivariate data. Recently, the first inference methods for the MCV were proposed by Ditzhaus and Smaga (2022) for general factorial designs covering k-sample settings but also complex higher-way layouts. However, two questions are still pending: (1) The theory on inference methods for MCV is primarily derived for one special MCV variant while there are several reasonable proposals. (2) When rejecting a global null hypothesis in factorial designs, a more in-depth analysis is typically of high interest to find the specific contrasts of MCV leading to the aforementioned rejection. In this paper, we tackle both by, first, extending the aforementioned nonparametric permutation procedure to the other MCV variants and, second, by proposing a max-type test for post hoc analysis. To improve the small sample performance of the latter, we suggest a novel studentized bootstrap strategy and prove its asymptotic validity. The actual performance of all proposed tests and post hoc procedures are compared in an extensive simulation study and illustrated by a real data analysis.


title: Leveraging Google's Tensor Processing Units for tsunami-risk mitigation planning in the Pacific Northwest and beyond

abstract: Tsunami-risk and flood-risk mitigation planning has particular importance for communities like those of the Pacific Northwest, where coastlines are extremely dynamic and a seismically-active subduction zone looms large. The challenge does not stop here for risk managers: mitigation options have multiplied since communities have realized the viability and benefits of nature-based solutions. To identify suitable mitigation options for their community, risk managers need the ability to rapidly evaluate several different options through fast and accessible tsunami models, but may lack high-performance computing infrastructure. The goal of this work is to leverage the newly developed Google's Tensor Processing Unit (TPU), a high-performance hardware accessible via the Google Cloud framework, to enable the rapid evaluation of different tsunami-risk mitigation strategies available to all communities. We establish a starting point through a numerical solver of the nonlinear shallow-water equations that uses a fifth-order Weighted Essentially Non-Oscillatory method with the Lax-Friedrichs flux splitting, and a Total Variation Diminishing third-order Runge-Kutta method for time discretization. We verify numerical solutions through several analytical solutions and benchmarks, reproduce several findings about one particular tsunami-risk mitigation strategy, and model tsunami runup at Crescent City, California whose topography comes from a high-resolution Digital Elevation Model. The direct measurements of the simulations performance, energy usage, and ease of execution show that our code could be a first step towards a community-based, user-friendly virtual laboratory that can be run by a minimally trained user on the cloud thanks to the ease of use of the Google Cloud Platform.


title: Sensing with shallow recurrent decoder networks

abstract: Sensing is a universal task in science and engineering. Downstream tasks from sensing include inferring full state estimates of a system (system identification), control decisions, and forecasting. These tasks are exceptionally challenging to achieve with limited sensors, noisy measurements, and corrupt or missing data. We propose a SHallow REcurrent Decoder (SHRED) neural network structure for sensing which incorporates (i) a recurrent neural network (LSTM) to learn a latent representation of the temporal dynamics of the sensors, and (ii) a shallow decoder that learns a mapping between this latent representation and the high-dimensional state space. By explicitly accounting for the time-history, or trajectory, of the sensor measurements, SHRED enables accurate reconstructions with far fewer sensors, outperforms existing techniques when more measurements are available, and is agnostic towards sensor placement. In addition, a compressed representation of the high-dimensional state is directly obtained from sensor measurements, which provides an on-the-fly compression for modeling physical and engineering systems. Forecasting is also achieved from the sensor time-series data alone, producing an efficient paradigm for predicting temporal evolution with an exceptionally limited number of sensors. In the example cases explored, including turbulent flows, complex spatio-temporal dynamics can be characterized with exceedingly limited sensors that can be randomly placed with minimal loss of performance.


title: In-Distribution Barrier Functions: Self-Supervised Policy Filters that Avoid Out-of-Distribution States

abstract: Learning-based control approaches have shown great promise in performing complex tasks directly from high-dimensional perception data for real robotic systems. Nonetheless, the learned controllers can behave unexpectedly if the trajectories of the system divert from the training data distribution, which can compromise safety. In this work, we propose a control filter that wraps any reference policy and effectively encourages the system to stay in-distribution with respect to offline-collected safe demonstrations. Our methodology is inspired by Control Barrier Functions (CBFs), which are model-based tools from the nonlinear control literature that can be used to construct minimally invasive safe policy filters. While existing methods based on CBFs require a known low-dimensional state representation, our proposed approach is directly applicable to systems that rely solely on high-dimensional visual observations by learning in a latent state-space. We demonstrate that our method is effective for two different visuomotor control tasks in simulation environments, including both top-down and egocentric view settings.


title: Cybersecurity Threat Hunting and Vulnerability Analysis Using a Neo4j Graph Database of Open Source Intelligence

abstract: Open source intelligence is a powerful tool for cybersecurity analysts to gather information both for analysis of discovered vulnerabilities and for detecting novel cybersecurity threats and exploits. However the scale of information that is relevant for information security on the internet is always increasing, and is intractable for analysts to parse comprehensively. Therefore methods of condensing the available open source intelligence, and automatically developing connections between disparate sources of information, is incredibly valuable. In this research, we present a system which constructs a Neo4j graph database formed by shared connections between open source intelligence text including blogs, cybersecurity bulletins, news sites, antivirus scans, social media posts (e.g., Reddit and Twitter), and threat reports. These connections are comprised of possible indicators of compromise (e.g., IP addresses, domains, hashes, email addresses, phone numbers), information on known exploits and techniques (e.g., CVEs and MITRE ATT&CK Technique ID's), and potential sources of information on cybersecurity exploits such as twitter usernames. The construction of the database of potential IoCs is detailed, including the addition of machine learning and metadata which can be used for filtering of the data for a specific domain (for example a specific natural language) when needed. Examples of utilizing the graph database for querying connections between known malicious IoCs and open source intelligence documents, including threat reports, are shown. We show that this type of relationship querying can allow for more effective use of open source intelligence for threat hunting, malware family clustering, and vulnerability analysis.


title: A hierarchy on non-archimedean Polish groups admitting a compatible complete left-invariant metric

abstract: In this article, we introduce a hierarchy on the class of non-archimedean Polish groups that admit a compatible complete left-invariant metric. We denote this hierarchy by $\alpha$-CLI and L-$\alpha$-CLI where $\alpha$ is a countable ordinal. We establish three results: \begin{enumerate} \item $G$ is $0$-CLI iff $G=\{1_G\}$; \item $G$ is $1$-CLI iff $G$ admits a compatible complete two-sided invariant metric; and \item $G$ is L-$\alpha$-CLI iff $G$ is locally $\alpha$-CLI, i.e., $G$ contains an open subgroup that is $\alpha$-CLI. \end{enumerate} Subsequently, we show this hierarchy is proper by constructing non-archimedean CLI Polish groups $G_\alpha$ and $H_\alpha$ for $\alpha<\omega_1$, such that \begin{enumerate} \item $H_\alpha$ is $\alpha$-CLI but not L-$\beta$-CLI for $\beta<\alpha$; and \item $G_\alpha$ is $(\alpha+1)$-CLI but not L-$\alpha$-CLI. \end{enumerate}


title: A Variant Prescribed Curvature Flow on Closed Surfaces with Negative Euler Characteristic

abstract: On a closed Riemannian surface $(M,\bar g)$ with negative Euler characteristic, we study the problem of finding conformal metrics with prescribed volume $A>0$ and the property that their Gauss curvatures $f_\lambda= f + \lambda$ are given as the sum of a prescribed function $f \in C^\infty(M)$ and an additive constant $\lambda$. Our main tool in this study is a new variant of the prescribed Gauss curvature flow, for which we establish local well-posedness and global compactness results. In contrast to previous work, our approach does not require any sign conditions on $f$. Moreover, we exhibit conditions under which the function $f_\lambda$ is sign changing and the standard prescribed Gauss curvature flow is not applicable.


title: Improved analytical modeling of the non-linear power spectrum in modified gravity cosmologies

abstract: Reliable analytical modeling of the non-linear power spectrum (PS) of matter perturbations is among the chief pre-requisites for cosmological analyses from the largest sky surveys. This is especially true for the models that extend the standard general-relativity paradigm by adding the fifth force, where numerical simulations can be prohibitively expensive. Here we present a method for building accurate PS models for two modified gravity (MG) variants: namely the Hu-Sawicki $f(R)$, and the normal branch of the Dvali-Gabadadze-Porrati (nDGP) braneworld. We start by modifying the standard halo model (HM) with respect to the baseline Lambda-Cold-Dark-Matter ($\Lambda$CDM) scenario, by using the HM components with specific MG extensions. We find that our $P(k)_{\text{HM}}$ retains 5% accuracy only up to mildly non-linear scales ($k \lesssim 0.3$ $h/\,\mbox{Mpc}$) when compared to PS from numerical simulations. At the same time, our HM prescription much more accurately captures the ratio $\Upsilon(k) = P(k)_{\text{MG}}/P(k)_{\Lambda \text{CDM}}$ up to non-linear scales. We show that using HM-derived $\Upsilon(k)$ together with a viable non-linear $\Lambda$CDM $P(k)$ prescription (such as HALOFIT), we render a much better and more accurate PS predictions in MG. The new approach yields considerably improved performance, with modeled $P(k)_{\text{MG}}$ being now accurate to within 5% all the way to non-linear scales of $k \lesssim 2.5-3$ $h/\,\mbox{Mpc}$. The magnitude of deviations from GR as fostered by these MG models is typically $\mathcal{O}(10\%)$ in these regimes. Therefore reaching 5% PS modeling is enough for forecasting constraints on modern-era cosmological observables.


title: Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Cases

abstract: Improving the deployment efficiency of transformer-based language models has been challenging given their high computation and memory cost. While INT8 quantization has recently been shown to be effective in reducing both the memory cost and latency while preserving model accuracy, it remains unclear whether we can leverage INT4 (which doubles peak hardware throughput) to achieve further latency improvement. In this study, we explore the feasibility of employing INT4 weight and activation (W4A4) quantization for language models. Our findings indicate that W4A4 quantization introduces no to negligible accuracy degradation for encoder-only and encoder-decoder models, but causes a significant accuracy drop for decoder-only models. To materialize the performance gain using W4A4, we develop a highly optimized end-to-end W4A4 encoder inference pipeline supporting different quantization strategies. Our INT4 pipeline is $8.5\times$ faster for latency-oriented scenarios and up to $3\times$ for throughput-oriented scenarios compared to the inference of FP16, and improves the SOTA BERT INT8 performance from FasterTransformer by up to $1.7\times$. We provide insights into the failure cases when applying W4A4 to decoder-only models, and further explore the compatibility of INT4 quantization with other compression methods, like pruning and layer reduction.


title: Stochastic and Mixed Density Functional Theory within the projector augmented wave formalism for the simulation of warm dense matter

abstract: Stochastic and mixed stochastic-deterministic density functional theory (DFT) are promising new approaches for the calculation of the equation-of-state and transport properties in materials under extreme conditions. In the intermediate warm dense matter regime, a state between correlated condensed matter and kinetic plasma, electrons can range from being highly localized around nuclei to delocalized over the whole simulation cell. The plane-wave basis pseudo-potential approach is thus the typical tool of choice for modeling such systems at the DFT level. Unfortunately, the stochastic DFT methods scale as the square of the maximum plane-wave energy in this basis. To reduce the effect of this scaling, and improve the overall description of the electrons within the pseudo-potential approximation, we present stochastic and mixed DFT developed and implemented within the projector augmented wave formalism. We compare results between the different DFT approaches for both single-point and molecular dynamics trajectories and present calculations of self-diffusion coefficients of solid density carbon from 1 to 50 eV.


title: A Greedy Sensor Selection Algorithm for Hyperparameterized Linear Bayesian Inverse Problems

abstract: We consider optimal sensor placement for a family of linear Bayesian inverse problems characterized by a deterministic hyper-parameter. The hyper-parameter describes distinct configurations in which measurements can be taken of the observed physical system. To optimally reduce the uncertainty in the system's model with a single set of sensors, the initial sensor placement needs to account for the non-linear state changes of all admissible configurations. We address this requirement through an observability coefficient which links the posteriors' uncertainties directly to the choice of sensors. We propose a greedy sensor selection algorithm to iteratively improve the observability coefficient for all configurations through orthogonal matching pursuit. The algorithm allows explicitly correlated noise models even for large sets of candidate sensors, and remains computationally efficient for high-dimensional forward models through model order reduction. We demonstrate our approach on a large-scale geophysical model of the Perth Basin, and provide numerical studies regarding optimality and scalability with regard to classic optimal experimental design utility functions.


title: General relativistic hydrodynamic simulations of perturbed transonic accretion

abstract: Comparison of horizon-scale observations of Sgr A* and M87* with numerical simulations has provided considerable insight in their interpretation. Most of these simulations are variations of the same physical scenario consisting of a rotation-supported torus seeded with poloidal magnetic fields. This setup has several well known limitations, most notably, it differs in important ways from what observed in simulations of accretion from large scales. We aim to study the flow patterns that arise at horizon scales in more general scenarios, that have a clearer connection with the large scale flow and are at the same time controlled by a reduced set of parameters. As a first step in this direction, we perform three dimensional general relativistic hydrodynamic simulations of rotating transonic flows with velocity perturbations injected from a spherical boundary located 1000 gravitational radii away from the central object. We study the general properties of these flows with varying angular momentum and perturbation amplitudes. We observe a rich phenomenology in accretion patterns, that includes smooth Bondi-like flows, turbulent torus-like structures, shocks, filaments, and complex sonic structures. For sufficiently large perturbations and angular momentum, radial profiles deviate from the constant entropy and angular momentum profiles used for initialization and resemble those of advection dominated accretion flows, showing evidence of entropy generation and angular momentum redistribution not mediated by magnetic fields. Fluctuations are amplified and extend further in frequency than the injected white noise spectrum, producing a red noise spectrum for synthetic Bremsstrahlung light curves. Future inclusion of magnetic fields and radiative cooling could make this type of simulations a viable alternative for numerical modeling of general low-luminosity active galactic nuclei.


title: The quotient set of the quadratic distance set over finite fields

abstract: Let $\mathbb F_q^d$ be the $d$-dimensional vector space over the finite field $\mathbb F_q$ with $q$ elements. For each non-zero $r$ in $\mathbb F_q$ and $E\subset \mathbb F_q^d$, we define $W(r)$ as the number of quadruples $(x,y,z,w)\in E^4$ such that $ Q(x-y)/Q(z-w)=r,$ where $Q$ is a non-degenerate quadratic form in $d$ variables over $\mathbb F_q.$ When $Q(\alpha)=\sum_{i=1}^d \alpha_i^2$ with $\alpha=(\alpha_1, \ldots, \alpha_d)\in \mathbb F_q^d,$ Pham (2022) recently used the machinery of group actions and proved that if $E\subset \mathbb F_q^2$ with $q\equiv 3 \pmod{4}$ and $|E|\ge C q$, then we have $W(r)\ge c |E|^4/q$ for any non-zero square number $r \in \mathbb F_q,$ where $C$ is a sufficiently large constant, $ c$ is some number between $0$ and $1,$ and $|E|$ denotes the cardinality of the set $E.$ In this article, we improve and extend Pham's result in two dimensions to arbitrary dimensions with general non-degenerate quadratic distances. As a corollary of our results, we also generalize the sharp results on the Falconer type problem for the quotient set of distance set due to the first two authors and Parshall (2019). Furthermore, we provide improved constants for the size conditions of the underlying sets. The key new ingredient is to relate the estimate of the $W(r)$ to a quadratic homogeneous variety in $2d$-dimensional vector space. This approach is fruitful because it allows us to take advantage of Gauss sums which are more handleable than the Kloosterman sums appearing in the standard distance type problems.


title: Epsilon-Identifiability of Causal Quantities

abstract: Identifying the effects of causes and causes of effects is vital in virtually every scientific field. Often, however, the needed probabilities may not be fully identifiable from the data sources available. This paper shows how partial identifiability is still possible for several probabilities of causation. We term this epsilon-identifiability and demonstrate its usefulness in cases where the behavior of certain subpopulations can be restricted to within some narrow bounds. In particular, we show how unidentifiable causal effects and counterfactual probabilities can be narrowly bounded when such allowances are made. Often those allowances are easily measured and reasonably assumed. Finally, epsilon-identifiability is applied to the unit selection problem.


title: Meta Temporal Point Processes

abstract: A temporal point process (TPP) is a stochastic process where its realization is a sequence of discrete events in time. Recent work in TPPs model the process using a neural network in a supervised learning framework, where a training set is a collection of all the sequences. In this work, we propose to train TPPs in a meta learning framework, where each sequence is treated as a different task, via a novel framing of TPPs as neural processes (NPs). We introduce context sets to model TPPs as an instantiation of NPs. Motivated by attentive NP, we also introduce local history matching to help learn more informative features. We demonstrate the potential of the proposed method on popular public benchmark datasets and tasks, and compare with state-of-the-art TPP methods.


title: Stability of Finite Receding Horizon Control: A Complementary Approach

abstract: This paper presents a complementary approach to establish stability of finite receding horizon control with a terminal cost. First a new augmented stage cost is defined by rotating the terminal cost. Then a one-step optimisation problem is defined based on this augmented stage cost. It is shown that a slightly modified Model Predictive Control (MPC) algorithm is stable if the value function of the augmented one-step cost (OSVF) is a control Lyapunov function. The proposed stability condition is completely complementary to the existing terminal cost based MPC stability conditions in the sense that they are mutually excluded with each other. By using this approach, we are able to establish stability for MPC algorithms with zero terminal cost or even negative terminal cost as special cases. Combining this new approach with the existing MPC stability theory, we are able to significantly relax the stability requirement on MPC and extend the design space where stability are guaranteed. The proposed approach will help to further reduce the gap between stability theory and practical applications of MPC and other optimisation based control methods.


title: Cross-Architectural Positive Pairs improve the effectiveness of Self-Supervised Learning

abstract: Existing self-supervised techniques have extreme computational requirements and suffer a substantial drop in performance with a reduction in batch size or pretraining epochs. This paper presents Cross Architectural - Self Supervision (CASS), a novel self-supervised learning approach that leverages Transformer and CNN simultaneously. Compared to the existing state-of-the-art self-supervised learning approaches, we empirically show that CASS-trained CNNs and Transformers across four diverse datasets gained an average of 3.8% with 1% labeled data, 5.9% with 10% labeled data, and 10.13% with 100% labeled data while taking 69% less time. We also show that CASS is much more robust to changes in batch size and training epochs than existing state-of-the-art self-supervised learning approaches. We have open-sourced our code at https://github.com/pranavsinghps1/CASS.


title: G-formula for causal inference via multiple imputation

abstract: G-formula is a popular approach for estimating treatment or exposure effects from longitudinal data that are subject to time-varying confounding. G-formula estimation is typically performed by Monte-Carlo simulation, with non-parametric bootstrapping used for inference. We show that G-formula can be implemented by exploiting existing methods for multiple imputation (MI) for synthetic data. This involves using an existing modified version of Rubin's variance estimator. In practice missing data is ubiquitous in longitudinal datasets. We show that such missing data can be readily accommodated as part of the MI procedure when using G-formula, and describe how MI software can be used to implement the approach. We explore its performance using a simulation study and an application from cystic fibrosis.


title: Lp Quasi-norm Minimization: Algorithm and Applications

abstract: Sparsity finds applications in areas as diverse as statistics, machine learning, and signal processing. Computations over sparse structures are less complex compared to their dense counterparts, and their storage consumes less space. This paper proposes a heuristic method for retrieving sparse approximate solutions of optimization problems via minimizing the $\ell_{p}$ quasi-norm, where $0<p<1$. An iterative two-block ADMM algorithm for minimizing the $\ell_{p}$ quasi-norm subject to convex constraints is proposed. For $p=s/q<1$, $s,q \in \mathbb{Z}_{+}$, the proposed algorithm requires solving for the roots of a scalar degree $2q$ polynomial as opposed to applying a soft thresholding operator in the case of $\ell_{1}$. The merit of that algorithm relies on its ability to solve the $\ell_{p}$ quasi-norm minimization subject to any convex set of constraints. However, it suffers from low speed, due to a convex projection step in each iteration, and the lack of mathematical convergence guarantee. We then aim to vanquish these shortcomings by relaxing the assumption on the constraints set to be the set formed due to convex and differentiable, with Lipschitz continuous gradient, functions, i.e. specifically, polytope sets. Using a proximal gradient step, we mitigate the convex projection step and hence enhance the algorithm speed while proving its convergence. We then present various applications where the proposed algorithm excels, namely, matrix rank minimization, sparse signal reconstruction from noisy measurements, sparse binary classification, and system identification. The results demonstrate the significant gains obtained by the proposed algorithm compared to those via $\ell_{1}$ minimization.


title: An Arithmetic Theory for the Poly-Time Random Functions

abstract: We introduce a new bounded theory RS^1_2 and show that the functions which are Sigma^b_1-representable in it are precisely random functions which can be computed in polynomial time. Concretely, we pass through a class of oracle functions over string, called POR, together with the theory of arithmetic RS^1_2. Then, we show that functions computed by poly-time PTMs are arithmetically characterized by a class of probabilistic bounded formulas.


title: Multi-task Highly Adaptive Lasso

abstract: We propose a novel, fully nonparametric approach for the multi-task learning, the Multi-task Highly Adaptive Lasso (MT-HAL). MT-HAL simultaneously learns features, samples and task associations important for the common model, while imposing a shared sparse structure among similar tasks. Given multiple tasks, our approach automatically finds a sparse sharing structure. The proposed MTL algorithm attains a powerful dimension-free convergence rate of $o_p(n^{-1/4})$ or better. We show that MT-HAL outperforms sparsity-based MTL competitors across a wide range of simulation studies, including settings with nonlinear and linear relationships, varying levels of sparsity and task correlations, and different numbers of covariates and sample size.


title: TiLT: A Time-Centric Approach for Stream Query Optimization and Parallelization

abstract: Stream processing engines (SPEs) are widely used for large scale streaming analytics over unbounded time-ordered data streams. Modern day streaming analytics applications exhibit diverse compute characteristics and demand strict latency and throughput requirements. Over the years, there has been significant attention in building hardware-efficient stream processing engines (SPEs) that support several query optimization, parallelization, and execution strategies to meet the performance requirements of large scale streaming analytics applications. However, in this work, we observe that these strategies often fail to generalize well on many real-world streaming analytics applications due to several inherent design limitations of current SPEs. We further argue that these limitations stem from the shortcomings of the fundamental design choices and the query representation model followed in modern SPEs. To address these challenges, we first propose TiLT, a novel intermediate representation (IR) that offers a highly expressive temporal query language amenable to effective query optimization and parallelization strategies. We subsequently build a compiler backend for TiLT that applies such optimizations on streaming queries and generates hardware-efficient code to achieve high performance on multi-core stream query executions. We demonstrate that TiLT achieves up to 326x (20.49x on average) higher throughput compared to state-of-the-art SPEs (e.g., Trill) across eight real-world streaming analytics applications. TiLT source code is available at https://github.com/ampersand-projects/tilt.git.


title: Context Matters: A Strategy to Pre-train Language Model for Science Education

abstract: This study aims at improving the performance of scoring student responses in science education automatically. BERT-based language models have shown significant superiority over traditional NLP models in various language-related tasks. However, science writing of students, including argumentation and explanation, is domain-specific. In addition, the language used by students is different from the language in journals and Wikipedia, which are training sources of BERT and its existing variants. All these suggest that a domain-specific model pre-trained using science education data may improve model performance. However, the ideal type of data to contextualize pre-trained language model and improve the performance in automatically scoring student written responses remains unclear. Therefore, we employ different data in this study to contextualize both BERT and SciBERT models and compare their performance on automatic scoring of assessment tasks for scientific argumentation. We use three datasets to pre-train the model: 1) journal articles in science education, 2) a large dataset of students' written responses (sample size over 50,000), and 3) a small dataset of students' written responses of scientific argumentation tasks. Our experimental results show that in-domain training corpora constructed from science questions and responses improve language model performance on a wide variety of downstream tasks. Our study confirms the effectiveness of continual pre-training on domain-specific data in the education domain and demonstrates a generalizable strategy for automating science education tasks with high accuracy. We plan to release our data and SciEdBERT models for public use and community engagement.


title: BinaryVQA: A Versatile Test Set to Evaluate the Out-of-Distribution Generalization of VQA Models

abstract: We introduce a new test set for visual question answering (VQA) called BinaryVQA to push the limits of VQA models. Our dataset includes 7,800 questions across 1,024 images and covers a wide variety of objects, topics, and concepts. For easy model evaluation, we only consider binary questions. Questions and answers are formulated and verified carefully and manually. Around 63% of the questions have positive answers. The median number of questions per image and question length are 7 and 5, respectively. The state of the art OFA model achieves 75% accuracy on BinaryVQA dataset, which is significantly lower than its performance on the VQA v2 test-dev dataset (94.7%). We also analyze the model behavior along several dimensions including: a) performance over different categories such as text, counting and gaze direction, b) model interpretability, c) the effect of question length on accuracy, d) bias of models towards positive answers and introduction of a new score called the ShuffleAcc, and e) sensitivity to spelling and grammar errors. Our investigation demonstrates the difficulty of our dataset and shows that it can challenge VQA models for next few years. Data and code are publicly available at: DATA and CODE.


title: Norm-based Generalization Bounds for Compositionally Sparse Neural Networks

abstract: In this paper, we investigate the Rademacher complexity of deep sparse neural networks, where each neuron receives a small number of inputs. We prove generalization bounds for multilayered sparse ReLU neural networks, including convolutional neural networks. These bounds differ from previous ones, as they consider the norms of the convolutional filters instead of the norms of the associated Toeplitz matrices, independently of weight sharing between neurons. As we show theoretically, these bounds may be orders of magnitude better than standard norm-based generalization bounds and empirically, they are almost non-vacuous in estimating generalization in various simple classification problems. Taken together, these results suggest that compositional sparsity of the underlying target function is critical to the success of deep neural networks.


title: Surface-termination-dependent electronic states in kagome superconductors AV3Sb5 (A = K, Rb, Cs) studied by micro-ARPES

abstract: Recently discovered kagome superconductors AV3Sb5 (A = K, Rb, Cs) exhibit exotic bulk and surface physical properties such as charge-density wave (CDW) and chirality, whereas their origins remain unresolved. By using micro-focused angle-resolved photoemission spectroscopy, we discovered that AV3Sb5 commonly exhibits two distinct polar surfaces depending on the termination; electron- and hole-doped ones for the A- and Sb-termination, respectively. We observed that the kagome-derived band shows a clear splitting in the A-terminated surface while it is absent in the Sb-terminated counterpart, indicative of the polarity-dependent CDW at the surface. Close comparison of the band-dependent splitting reveals that the three-dimensional CDW structure of the K-terminated surface is different from that of the Rb- or Cs-terminated surface, suggesting the diversity of the CDW ground state. These results provide important insight into the origin of CDW in kagome superconductors AV3Sb5.


title: State Machine-based Waveforms for Channels With 1-Bit Quantization and Oversampling With Time-Instance Zero-Crossing Modulation

abstract: Systems with 1-bit quantization and oversampling are promising for the Internet of Things (IoT) devices in order to reduce the power consumption of the analog-to-digital-converters. The novel time-instance zero-crossing (TI ZX) modulation is a promising approach for this kind of channels but existing studies rely on optimization problems with high computational complexity and delay. In this work, we propose a practical waveform design based on the established TI ZX modulation for a multiuser multi-input multi-output (MIMO) downlink scenario with 1-bit quantization and temporal oversampling at the receivers. In this sense, the proposed temporal transmit signals are constructed by concatenating segments of coefficients which convey the information into the time-instances of zero-crossings according to the TI ZX mapping rules. The proposed waveform design is compared with other methods from the literature. The methods are compared in terms of bit error rate and normalized power spectral density. Numerical results show that the proposed technique is suitable for multiuser MIMO system with 1-bit quantization while tolerating some small amount of out-of-band radiation.


title: Analyzing Robustness of the Deep Reinforcement Learning Algorithm in Ramp Metering Applications Considering False Data Injection Attack and Defense

abstract: Ramp metering is the act of controlling on-going vehicles to the highway mainlines. Decades of practices of ramp metering have proved that ramp metering can decrease total travel time, mitigate shockwaves, decrease rear-end collisions by smoothing the traffic interweaving process, etc. Besides traditional control algorithm like ALINEA, Deep Reinforcement Learning (DRL) algorithms have been introduced to build a finer control. However, two remaining challenges still hinder DRL from being implemented in the real world: (1) some assumptions of algorithms are hard to be matched in the real world; (2) the rich input states may make the model vulnerable to attacks and data noises. To investigate these issues, we propose a Deep Q-Learning algorithm using only loop detectors information as inputs in this study. Then, a set of False Data Injection attacks and random noise attack are designed to investigate the robustness of the model. The major benefit of the model is that it can be applied to almost any ramp metering sites regardless of the road geometries and layouts. Besides outcompeting the ALINEA method, the Deep Q-Learning method also shows a good robustness through training among very different demands and geometries. For example, during the testing case in I-24 near Murfreesboro, TN, the model shows its robustness as it still outperforms ALINEA algorithm under Fast Gradient Sign Method attacks. Unlike many previous studies, the model is trained and tested in completely different environments to show the capabilities of the model.


title: Possible depth-resolved reconstruction of shear moduli in the cornea following collagen crosslinking (CXL) with optical coherence tomography and elastography

abstract: Collagen crosslinking of the cornea (CXL) is commonly employed to prevent or treat keratoconus. Although the change of corneal stiffness induced by CXL surgery can be monitored with non-contact dynamic Optical Coherence Elastography (OCE) by tracking mechanical wave propagation, the depth dependence of this change is still unclear if the cornea is not crosslinked through the whole depth. Here we propose to combine phase-decorrelation measurement applied to OCT structural images and acoustic micro-tapping (A$\mu$T) OCE to explore possible depth reconstruction of stiffness within crosslinked corneas in an ex vivo human cornea sample. The analysis of experimental OCT images is used to define the penetration depth of CXL into the cornea, which varies from $\sim$100$\mu m$ in the periphery to $\sim$150$\mu m$ in the central area and exhibits a sharp transition between areas. This information was used in a two-layer analytical model to quantify the stiffness of the treated layer. We also discuss how the elastic moduli of partially CXL-treated cornea layers reconstructed from OCE measurements reflect the effective mechanical stiffness of the entire cornea to properly quantify surgical outcome.


title: STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning

abstract: Directed Exploration is a crucial challenge in reinforcement learning (RL), especially when rewards are sparse. Information-directed sampling (IDS), which optimizes the information ratio, seeks to do so by augmenting regret with information gain. However, estimating information gain is computationally intractable or relies on restrictive assumptions which prohibit its use in many practical instances. In this work, we posit an alternative exploration incentive in terms of the integral probability metric (IPM) between a current estimate of the transition model and the unknown optimal, which under suitable conditions, can be computed in closed form with the kernelized Stein discrepancy (KSD). Based on KSD, we develop a novel algorithm \algo: \textbf{STE}in information dir\textbf{E}cted exploration for model-based \textbf{R}einforcement Learn\textbf{ING}. To enable its derivation, we develop fundamentally new variants of KSD for discrete conditional distributions. {We further establish that {\algo} archives sublinear Bayesian regret, improving upon prior learning rates of information-augmented MBRL.} Experimentally, we show that the proposed algorithm is computationally affordable and outperforms several prior approaches.


title: Harnessing the Power of Decision Trees to Detect IoT Malware

abstract: Due to its simple installation and connectivity, the Internet of Things (IoT) is susceptible to malware attacks. Being able to operate autonomously. As IoT devices have become more prevalent, they have become the most tempting targets for malware. Weak, guessable, or hard-coded passwords, and a lack of security measures contribute to these vulnerabilities along with insecure network connections and outdated update procedures. To understand IoT malware, current methods and analysis ,using static methods, are ineffective. The field of deep learning has made great strides in recent years due to their tremendous data mining, learning, and expression capabilities, cybersecurity has enjoyed tremendous growth in recent years. As a result, malware analysts will not have to spend as much time analyzing malware. In this paper, we propose a novel detection and analysis method that harnesses the power and simplicity of decision trees. The experiments are conducted using a real word dataset, MaleVis which is a publicly available dataset. Based on the results, we show that our proposed approach outperforms existing state-of-the-art solutions in that it achieves 97.23% precision and 95.89% recall in terms of detection and classification. A specificity of 96.58%, F1-score of 96.40%, an accuracy of 96.43.


title: ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts

abstract: Current protein language models (PLMs) learn protein representations mainly based on their sequences, thereby well capturing co-evolutionary information, but they are unable to explicitly acquire protein functions, which is the end goal of protein representation learning. Fortunately, for many proteins, their textual property descriptions are available, where their various functions are also described. Motivated by this fact, we first build the ProtDescribe dataset to augment protein sequences with text descriptions of their functions and other important properties. Based on this dataset, we propose the ProtST framework to enhance Protein Sequence pre-training and understanding by biomedical Texts. During pre-training, we design three types of tasks, i.e., unimodal mask prediction, multimodal representation alignment and multimodal mask prediction, to enhance a PLM with protein property information with different granularities and, at the same time, preserve the PLM's original representation power. On downstream tasks, ProtST enables both supervised learning and zero-shot prediction. We verify the superiority of ProtST-induced PLMs over previous ones on diverse representation learning benchmarks. Under the zero-shot setting, we show the effectiveness of ProtST on zero-shot protein classification, and ProtST also enables functional protein retrieval from a large-scale database without any function annotation.


title: Vehicle-to-Grid Fleet Service Provision considering Nonlinear Battery Behaviors

abstract: The surging adoption of electric vehicles (EV) calls for accurate and efficient approaches to coordinate with the power grid operation. By being responsive to distribution grid limits and time-varying electricity prices, EV charging stations can minimize their charging costs while aiding grid operation simultaneously. In this study, we investigate the economic benefit of vehicle-to-grid (V2G) using real-time price data from New York State and a real-world charging network dataset. We incorporate nonlinear battery models and price uncertainty into the V2G management design to provide a realistic estimation of cost savings from different V2G options. The proposed control method is computationally tractable when scaling up to real-world applications. We show that our proposed algorithm leads to an average of 35% charging cost savings compared to uncontrolled charging when considering unidirectional charging, and bi-directional V2G enables additional 18% cost savings compared to unidirectional smart charging. Our result also shows the importance of using more accurate nonlinear battery models in V2G controllers and evaluating the cost of price uncertainties over V2G.


title: The gauge-invariant formulation of the local expansion rate driven by the local average density in an inhomogeneous universe

abstract: The Hubble tension casts a blight on the standard cosmology. As a possible solution to the problem, the local variation of the expansion rate has been proposed where the spatial averaging over a finite domain was introduced in order to restore the local Friedmannian behavior in an inhomogeneous cosmology. So far, however, the approaches are limited to the particular choices of the gauges, and it has been unclear whether the results are gauge-invariant. In this paper, we present the gauge-invariant formulation of the local expansion rate which is driven by the spatial average of the gauge-invariant inhomogeneous density. We show that the local cosmological parameters in the finite domain may change from the global parameters, and the relations between them are expressed by the gauge-invariant averaged density.


title: Parsimonious System Identification from Fragmented Quantized Measurements

abstract: Quantization is the process of mapping an input signal from an infinite continuous set to a countable set with a finite number of elements. It is a non-linear irreversible process, which makes the traditional methods of system identification no longer applicable. In this work, we propose a method for parsimonious linear time invariant system identification when only quantized observations, discerned from noisy data, are available. More formally, given a priori information on the system, represented by a compact set containing the poles of the system, and quantized realizations, our algorithm aims at identifying the least order system that is compatible with the available information. The proposed approach takes also into account that the available data can be subject to fragmentation. Our proposed algorithm relies on an ADMM approach to solve a $\ell_{p},(0<p<1),$ quasi-norm objective problem. Numerical results highlight the performance of the proposed approach when compared to the $\ell_{1}$ minimization in terms of the sparsity of the induced solution.


title: Supergeo Design: Generalized Matching for Geographic Experiments

abstract: We propose a generalization of the standard matched pairs design in which experimental units (often geographic regions or geos) may be combined into larger units/regions called "supergeos" in order to improve the average matching quality. Unlike optimal matched pairs design which can be found in polynomial time (Lu et al. 2011), this generalized matching problem is NP-hard. We formulate it as a mixed-integer program (MIP) and show that experimental design obtained by solving this MIP can often provide a significant improvement over the standard design regardless of whether the treatment effects are homogeneous or heterogeneous. Furthermore, we present the conditions under which trimming techniques that often improve performance in the case of homogeneous effects (Chen and Au, 2022), may lead to biased estimates and show that the proposed design does not introduce such bias. We use empirical studies based on real-world advertising data to illustrate these findings.


title: Forward selection and post-selection inference in factorial designs

abstract: Ever since the seminal work of R. A. Fisher and F. Yates, factorial designs have been an important experimental tool to simultaneously estimate the effects of multiple treatment factors. In factorial designs, the number of treatment combinations grows exponentially with the number of treatment factors, which motivates the forward selection strategy based on the sparsity, hierarchy, and heredity principles for factorial effects. Although this strategy is intuitive and has been widely used in practice, its rigorous statistical theory has not been formally established. To fill this gap, we establish design-based theory for forward factor selection in factorial designs based on the potential outcome framework. We not only prove a consistency property for the factor selection procedure but also discuss statistical inference after factor selection. In particular, with selection consistency, we quantify the advantages of forward selection based on asymptotic efficiency gain in estimating factorial effects. With inconsistent selection in higher-order interactions, we propose two strategies and investigate their impact on subsequent inference. Our formulation differs from the existing literature on variable selection and post-selection inference because our theory is based solely on the physical randomization of the factorial design and does not rely on a correctly specified outcome model.


title: Semantic Adversarial Attacks on Face Recognition through Significant Attributes

abstract: Face recognition is known to be vulnerable to adversarial face images. Existing works craft face adversarial images by indiscriminately changing a single attribute without being aware of the intrinsic attributes of the images. To this end, we propose a new Semantic Adversarial Attack called SAA-StarGAN that tampers with the significant facial attributes for each image. We predict the most significant attributes by applying the cosine similarity or probability score. The probability score method is based on training a Face Verification model for an attribute prediction task to obtain a class probability score for each attribute. The prediction process will help craft adversarial face images more easily and efficiently, as well as improve the adversarial transferability. Then, we change the most significant facial attributes, with either one or more of the facial attributes for impersonation and dodging attacks in white-box and black-box settings. Experimental results show that our method could generate diverse and realistic adversarial face images meanwhile avoid affecting human perception of the face recognition. SAA-StarGAN achieves an 80.5% attack success rate against black-box models, outperforming existing methods by 35.5% under the impersonation attack. Concerning the black-box setting, SAA-StarGAN achieves high attack success rates on various models. The experiments confirm that predicting the most important attributes significantly affects the success of adversarial attacks in both white-box and black-box settings and could enhance the transferability of the crafted adversarial examples.


title: Backpropagation of Unrolled Solvers with Folded Optimization

abstract: The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. A central challenge in this setting is backpropagation through the solution of an optimization problem, which typically lacks a closed form. One typical strategy is algorithm unrolling, which relies on automatic differentiation through the operations of an iterative solver. While flexible and general, unrolling can encounter accuracy and efficiency issues in practice. These issues can be avoided by analytical differentiation of the optimization, but current frameworks impose rigid requirements on the optimization problem's form. This paper provides theoretical insights into the backward pass of unrolled optimization, leading to a system for generating efficiently solvable analytical models of backpropagation. Additionally, it proposes a unifying view of unrolling and analytical differentiation through optimization mappings. Experiments over various model-based learning tasks demonstrate the advantages of the approach both computationally and in terms of enhanced expressiveness.


title: Making Reconstruction-based Method Great Again for Video Anomaly Detection

abstract: Anomaly detection in videos is a significant yet challenging problem. Previous approaches based on deep neural networks employ either reconstruction-based or prediction-based approaches. Nevertheless, existing reconstruction-based methods 1) rely on old-fashioned convolutional autoencoders and are poor at modeling temporal dependency; 2) are prone to overfit the training samples, leading to indistinguishable reconstruction errors of normal and abnormal frames during the inference phase. To address such issues, firstly, we get inspiration from transformer and propose ${\textbf S}$patio-${\textbf T}$emporal ${\textbf A}$uto-${\textbf T}$rans-${\textbf E}$ncoder, dubbed as $\textbf{STATE}$, as a new autoencoder model for enhanced consecutive frame reconstruction. Our STATE is equipped with a specifically designed learnable convolutional attention module for efficient temporal learning and reasoning. Secondly, we put forward a novel reconstruction-based input perturbation technique during testing to further differentiate anomalous frames. With the same perturbation magnitude, the testing reconstruction error of the normal frames lowers more than that of the abnormal frames, which contributes to mitigating the overfitting problem of reconstruction. Owing to the high relevance of the frame abnormality and the objects in the frame, we conduct object-level reconstruction using both the raw frame and the corresponding optical flow patches. Finally, the anomaly score is designed based on the combination of the raw and motion reconstruction errors using perturbed inputs. Extensive experiments on benchmark video anomaly detection datasets demonstrate that our approach outperforms previous reconstruction-based methods by a notable margin, and achieves state-of-the-art anomaly detection performance consistently. The code is available at https://github.com/wyzjack/MRMGA4VAD.


title: Photoinduced pseudospin-wave emission from charge-density-wave domain wall with superconductivity

abstract: We study photoinduced dynamics triggered by an inhomogeneity due to competition between charge density waves (CDWs) and superconductivity. As a simple example, we consider the superconducting (SC) interface between two CDW domains with opposite signs. The real-time dynamics are calculated within the time-dependent Hartree--Fock--Bogoliubov framework, where the order parameter dynamics and the nonequilibrium quasiparticle distribution functions are studied. We also calculate the various dynamical response functions within a generalized random phase approximation. Through comparisons between the real time dynamics and the analysis of the response functions, it is found that the photo-driven SC interface can emit collective modes of the SC order parameter. This is analogous to the spin wave emission from the magnetic domain wall in an antiferromagnet, particularly in the case of a low driving frequency, where the order parameters can be mapped onto the pseudospin picture. In the high-frequency case, we find a domain wall melting caused by changes in the quasiparticle distribution, which induces superconductivity in the whole system.


title: Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling

abstract: Reinforcement learning (RL) agents typically learn tabula rasa, without prior knowledge of the world. However, if initialized with knowledge of high-level subgoals and transitions between subgoals, RL agents could utilize this Abstract World Model (AWM) for planning and exploration. We propose using few-shot large language models (LLMs) to hypothesize an AWM, that will be verified through world experience, to improve sample efficiency of RL agents. Our DECKARD agent applies LLM-guided exploration to item crafting in Minecraft in two phases: (1) the Dream phase where the agent uses an LLM to decompose a task into a sequence of subgoals, the hypothesized AWM; and (2) the Wake phase where the agent learns a modular policy for each subgoal and verifies or corrects the hypothesized AWM. Our method of hypothesizing an AWM with LLMs and then verifying the AWM based on agent experience not only increases sample efficiency over contemporary methods by an order of magnitude but is also robust to and corrects errors in the LLM, successfully blending noisy internet-scale information from LLMs with knowledge grounded in environment dynamics.


title: Predicting Students' Exam Scores Using Physiological Signals

abstract: While acute stress has been shown to have both positive and negative effects on performance, not much is known about the impacts of stress on students grades during examinations. To answer this question, we examined whether a correlation could be found between physiological stress signals and exam performance. We conducted this study using multiple physiological signals of ten undergraduate students over three different exams. The study focused on three signals, i.e., skin temperature, heart rate, and electrodermal activity. We extracted statistics as features and fed them into a variety of binary classifiers to predict relatively higher or lower grades. Experimental results showed up to 0.81 ROC-AUC with k-nearest neighbor algorithm among various machine learning algorithms.


title: Leveraging Importance Weights in Subset Selection

abstract: We present a subset selection algorithm designed to work with arbitrary model families in a practical batch setting. In such a setting, an algorithm can sample examples one at a time but, in order to limit overhead costs, is only able to update its state (i.e. further train model weights) once a large enough batch of examples is selected. Our algorithm, IWeS, selects examples by importance sampling where the sampling probability assigned to each example is based on the entropy of models trained on previously selected batches. IWeS admits significant performance improvement compared to other subset selection algorithms for seven publicly available datasets. Additionally, it is competitive in an active learning setting, where the label information is not available at selection time. We also provide an initial theoretical analysis to support our importance weighting approach, proving generalization and sampling rate bounds.


title: Weakly Supervised Image Segmentation Beyond Tight Bounding Box Annotations

abstract: Weakly supervised image segmentation approaches in the literature usually achieve high segmentation performance using tight bounding box supervision and decrease the performance greatly when supervised by loose bounding boxes. However, compared with loose bounding box, it is much more difficult to acquire tight bounding box due to its strict requirements on the precise locations of the four sides of the box. To resolve this issue, this study investigates whether it is possible to maintain good segmentation performance when loose bounding boxes are used as supervision. For this purpose, this work extends our previous parallel transformation based multiple instance learning (MIL) for tight bounding box supervision by integrating an MIL strategy based on polar transformation to assist image segmentation. The proposed polar transformation based MIL formulation works for both tight and loose bounding boxes, in which a positive bag is defined as pixels in a polar line of a bounding box with one endpoint located inside the object enclosed by the box and the other endpoint located at one of the four sides of the box. Moreover, a weighted smooth maximum approximation is introduced to incorporate the observation that pixels closer to the origin of the polar transformation are more likely to belong to the object in the box. The proposed approach was evaluated on two public datasets using dice coefficient when bounding boxes at different precision levels were considered in the experiments. The results demonstrate that the proposed approach achieves state-of-the-art performance for bounding boxes at all precision levels and is robust to mild and moderate errors in the loose bounding box annotations. The codes are available at \url{https://github.com/wangjuan313/wsis-beyond-tightBB}.


title: Adversarial Learning Networks: Source-free Unsupervised Domain Incremental Learning

abstract: This work presents an approach for incrementally updating deep neural network (DNN) models in a non-stationary environment. DNN models are sensitive to changes in input data distribution, which limits their application to problem settings with stationary input datasets. In a non-stationary environment, updating a DNN model requires parameter re-training or model fine-tuning. We propose an unsupervised source-free method to update DNN classification models. The contributions of this work are two-fold. First, we use trainable Gaussian prototypes to generate representative samples for future iterations; second, using unsupervised domain adaptation, we incrementally adapt the existing model using unlabelled data. Unlike existing methods, our approach can update a DNN model incrementally for non-stationary source and target tasks without storing past training data. We evaluated our work on incremental sentiment prediction and incremental disease prediction applications and compared our approach to state-of-the-art continual learning, domain adaptation, and ensemble learning methods. Our results show that our approach achieved improved performance compared to existing incremental learning methods. We observe minimal forgetting of past knowledge over many iterations, which can help us develop unsupervised self-learning systems.