title: Towards Privacy Engineering for Real-Time Analytics in the Human-Centered Internet of Things

abstract: Big data applications offer smart solutions to many urgent societal challenges, such as health care, traffic coordination, energy management, etc. The basic premise for these applications is "the more data the better". The focus often lies on sensing infrastructures in the public realm that produce an ever-increasing amount of data. Yet, any smartphone and smartwatch owner could be a continuous source of valuable data and contribute to many useful big data applications. However, such data can reveal a lot of sensitive information, like the current location or the heart rate of the owner of such devices. Protection of personal data is important in our society and for example manifested in the EU General Data Protection Regulation (GDPR). However, privacy protection and useful big data applications are hard to bring together, particularly in the human-centered IoT. Implementing proper privacy protection requires skills that are typically not in the focus of data analysts and big data developers. Thus, many individuals tend to share none of their data if in doubt whether it will be properly protected. There exist excellent privacy solutions between the "all or nothing" approach. For example, instead of continuously publishing the current location of individuals one might aggregate this data and only publish information of how many individuals are in a certain area of the city. Thus, personal data is not revealed, while useful information for certain applications like traffic coordination is retained. The goal of the Parrot project is to provide tools for real-time data analysis applications that leverage this "middle ground". Data analysts should only be required to specify their data needs, and end-users can select the privacy requirements for their data as well as the applications and end-users they want to share their data with.


title: Proposal of FPGA logic change after service launch for environment adaptation

abstract: In order to make full use of heterogeneous hardware, it is necessary to have a technical skill of hardware such as OpenCL, and the current situation is that the barrier is high. Based on this background, I have proposed environment-adaptive software that enables high-performance operation by automatically converting application code written for normal CPUs by engineers according to the deployed environment and setting appropriate amount of resources. Until now, I only considered conversions and settings before the start of operation. In this paper, I verify that the logic is reconfigured according to the usage characteristics during operation. I confirm that the application running on the FPGA is reconfigured into another application according to the usage characteristics.


title: The Transition from the Kuiper Belt to the Jupiter-Family (Comets)

abstract: Kuiper Belt Objects, or more generally Trans-Neptunian Objects (TNOs), are planetesimals found beyond the orbit of Neptune. Some TNOs evolve onto Neptune-crossing orbits and become Centaurs. Many Centaurs, in turn, reach Jupiter-crossing orbits and become Jupiter-family comets (JFCs). TNOs are the main source of the JFCs. TNOs offer a different window than the JFCs, of more primordial bodies and over a different size and temperature range. It is in that context that this chapter is written. Here we discuss the dynamical pathways taken from the trans-Neptunian region to the JFCs, and the most important properties of TNOs that relate to the JFC population, including considerations of their origins, compositions, morphologies, and size distributions. We relate these properties to the JFCs whenever possible. We reflect on a few key outstanding issues regarding our incomplete knowledge of TNOs as they pertain to the Centaurs and JFC populations. We finish with a short discussion of notable new and upcoming facilities and the impacts they will have regarding these outstanding questions.


title: QuDPy: A Python-Based Tool For Computing Ultrafast Non-linear Optical Responses

abstract: Nonlinear Optical Spectroscopy is a well-developed field with theoretical and experimental advances that have aided multiple fields including chemistry, biology and physics. However, accurate quantum dynamical simulations based on model Hamiltonians are need to interpret the corresponding multi-dimensional spectral signals properly. In this article, we present the initial release of our code, QuDPy (quantum dynamics in python) which addresses the need for a robust numerical platform for performing quantum dynamics simulations based on model systems, including open quantum systems. An important feature of our approach is that one can specify various high-order optical response pathways in the form of double-sided Feynman diagrams via a straightforward input syntax that specifies the time-ordering of ket-sided or bra-sided optical interactions acting upon the time-evolving density matrix of the system. We use the quantum dynamics capabilities of QuTip for simulating the spectral response of complex systems to compute essentially any n-th-order optical response of the model system. We provide a series of example calculations to illustrate the utility of our approach.


title: Quantum Systems at The Brink

abstract: We present a method to calculate the asymptotic behavior of eigenfunctions of Schr\"odinger operators that also works at the threshold of the essential spectrum. It can be viewed as a higher order correction to the well-known WKB method which does need a safety distance to the essential spectrum. We illustrate its usefulness on examples of quantum particles in a potential well with a long-range repulsive term outside the well.


title: Minimum Kernel Discrepancy Estimators

abstract: For two decades, reproducing kernels and their associated discrepancies have facilitated elegant theoretical analyses in the setting of quasi Monte Carlo. These same tools are now receiving interest in statistics and related fields, as criteria that can be used to select an appropriate statistical model for a given dataset. The focus of this article is on minimum kernel discrepancy estimators, whose use in statistical applications is reviewed, and a general theoretical framework for establishing their asymptotic properties is presented.


title: Applicability of machine learning in uncertainty quantification of turbulence models

abstract: White paper: The aim of this work is to apply and analyze machine learning methods for uncertainty quantification of turbulence models. In this work we investigate the classical and data-driven variants of the eigenspace perturbation method. This methodology is designed to estimate the uncertainties related to the shape of the modeled Reynolds stress tensor in the Navier-Stokes equations for Computational Fluid Dynamics (CFD). The underlying methodology is extended by adding a data-driven, physics-constrained machine learning approach in order to predict local perturbations of the Reynolds stress tensor. Using separated two-dimensional flows, we investigate the generalization properties of the machine learning models and shed a light on impacts of applying a data-driven extension.


title: Forcing More DC Over the Chang Model Using the Thorn Sequence

abstract: In the context of $\mathsf{ZF}+\mathsf{DC}$, we force $\mathsf{DC}_\kappa$ for relations on $\mathcal{P}(\kappa)$ for $\kappa{}<\aleph_\omega$ over the Chang model $\mathrm{L}(\mathrm{Ord}^\omega)$ making some assumptions on the thorn sequence defined by ${\it \unicode{xFE}}_0=\omega$, ${\it \unicode{xFE}}_{\alpha{}+1}$ as the least ordinal not a surjective image of ${\it \unicode{xFE}}_\alpha^\omega$ (i.e. no $f:{\it \unicode{xFE}}_{\alpha}^\omega{}\rightarrow {\it \unicode{xFE}}_{\alpha{}+1}$ is surjective) and ${\it \unicode{xFE}}_\gamma{}=\sup_{\alpha{}<\gamma}{\it \unicode{xFE}}_\alpha$ for limit $\gamma$. These assumptions are motivated from results about $\Theta$ in the context of determinacy, and could be reasonable ways of thinking about the Chang model. Explicitly, we assume cardinals $\lambda$ on the thorn sequence are strongly regular (meaning regular and functions $f:\kappa{}^{<\kappa}\rightarrow \lambda$ are bounded whenever $\kappa{}<\lambda$ is on the thorn sequence) and justified (meaning $\mathcal{P}(\kappa^\omega)\cap \mathrm{L}(\mathrm{Ord}^\omega)\subseteq \mathrm{L}_{\lambda}(\lambda^\omega{},X)$ for some $X\subseteq \lambda$ for any $\kappa{}<\lambda$ on the thorn sequence). This allow us to use Cohen forcing and establish more dependent choice.


title: Bundles of strongly self-absorbing $C^*$-algebras with a Clifford grading

abstract: We extend our previous results on generalized Dixmier-Douady theory to graded $C^*$-algebras, as means for explicit computations of the invariants arising for bundles of ungraded $C^*$-algebras. For a strongly self-absorbing $C^*$-algebra $D$ and complex Clifford algebras $\mathbb{C}\ell_{n}$ we show that the classifying spaces of the groups of graded automorphisms $\mathrm{Aut}_{\text{gr}}(\mathbb{C}\ell_{n}\otimes \mathcal{K }\otimes D)$ admit compatible infinite loop space structures giving rise to a cohomology theory $\hat{E}^*_D(X)$. For $D$ stably finite and $X$ a finite CW-complex, we show that the tensor product operation defines a group structure on the isomorphism classes of locally trivial bundles of graded $C^*$-algebras with fibers $ \mathbb{C}\ell_{k}\otimes D \otimes \mathcal{K}$ and that this group is isomorphic to $H^0(X,\mathbb{Z}/2)\oplus \hat{E}^1_{D}(X)$. Moreover, we establish isomorphisms $\hat{E}^1_{D}(X)\cong H^1(X;\mathbb{Z}/2) \times_{_{tw}} E^1_{D}(X)$ and $\hat{E}^1_{D}(X)\cong E^1_{D\otimes \mathcal{O}_\infty}(X)$, where $E^1_{D}(X)$ is the group that classifies the locally trivial bundles with fibers $D\otimes \mathcal{K}$. In particular $E^1_{\mathcal{O}_\infty}(X)\cong H^1(X;\mathbb{Z}/2) \times_{_{tw}} E^1_{\mathcal{Z}}(X)$ where $\mathcal{Z}$ is the Jiang-Su algebra and the multiplication on the last two factors is twisted similarly to the Brauer theory for bundles with fibers the graded compact operators on a finite and respectively infinite dimensional Hilbert space.


title: Regularized numerical methods for the nonlinear Schr\"odinger equation with singular nonlinearity

abstract: We present different regularizations and numerical methods for the nonlinear Schr\"odinger equation with singular nonlinearity (sNLSE) including the regularized Lie-Trotter time-splitting (LTTS) methods and regularized Lawson-type exponential integrator (LTEI) methods. Due to the blowup of the singular nonlinearity, i.e., $f(\rho)=\rho^{\alpha}$ with a fixed exponent $\alpha<0$ goes to infinity when $\rho \to 0^+$ ($\rho = |\psi|^2$ represents the density with $\psi$ being the complex-valued wave function or order parameter), there are significant difficulties in designing accurate and efficient numerical schemes to solve the sNLSE. In order to suppress the round-off error and avoid blowup near $\rho = 0^+$, two types of regularizations for the sNLSE are proposed with a small regularization parameter $0 < \eps \ll 1$. One is based on the local energy regularization (LER) for the sNLSE via regularizing the energy density $F(\rho) = \frac{1}{\alpha+1}\rho^{\alpha+1}$ locally near $\rho = 0^+$ with a polynomial approximation and then obtaining a local energy regularized nonlinear Schr\"odinger equation via energy variation. The other one is the global nonlinearity regularization which directly regularizes the singular nonlinearity $f(\rho)=\rho^{\alpha}$ to avoid blowup near $\rho = 0^+$. For the regularized models, we apply the first-order Lie-Trotter time-splitting method and Lawson-type exponential integrator method for temporal discretization and combine with the Fourier pseudospectral method in space to numerically solve them. Numerical examples are provided to show the convergence of the regularized models to the sNLSE and they suggest that the local energy regularization performs better than directly regularizing the singular nonlinearity globally.


title: Monolithically integrated 940 nm half VCSELs on bulk Ge substrates

abstract: High quality n-type AlGaAs distributed Bragg reflectors (DBRs) and lnGaAs multiple quantum wells were successfully monolithically grown on 4-inch off-cut Ge (100) wafers. The grown structures have photoluminescence spectra and reflectance spectra comparable to those grown on conventional bulk GaAs wafers and have smooth morphology and reasonable uniformity. These results strongly support full VCSEL growth and fabrication on larger-area bulk Ge substrates for the mass production of AlGaAs-based VCSELs.


title: Predicting Brain Age using Transferable coVariance Neural Networks

abstract: The deviation between chronological age and biological age is a well-recognized biomarker associated with cognitive decline and neurodegeneration. Age-related and pathology-driven changes to brain structure are captured by various neuroimaging modalities. These datasets are characterized by high dimensionality as well as collinearity, hence applications of graph neural networks in neuroimaging research routinely use sample covariance matrices as graphs. We have recently studied covariance neural networks (VNNs) that operate on sample covariance matrices using the architecture derived from graph convolutional networks, and we showed VNNs enjoy significant advantages over traditional data analysis approaches. In this paper, we demonstrate the utility of VNNs in inferring brain age using cortical thickness data. Furthermore, our results show that VNNs exhibit multi-scale and multi-site transferability for inferring {brain age}. In the context of brain age in Alzheimer's disease (AD), our experiments show that i) VNN outputs are interpretable as brain age predicted using VNNs is significantly elevated for AD with respect to healthy subjects for different datasets; and ii) VNNs can be transferable, i.e., VNNs trained on one dataset can be transferred to another dataset with different dimensions without retraining for brain age prediction.


title: Uncertainty Driven Active Learning of Coarse Grained Free Energy Models

abstract: Coarse graining techniques play an essential role in accelerating molecular simulations of systems with large length and time scales. Theoretically grounded bottom-up models are appealing due to their thermodynamic consistency with the underlying all-atom models. In this direction, machine learning approaches hold great promise to fitting complex many-body data. However, training models may require collection of large amounts of expensive data. Moreover, quantifying trained model accuracy is challenging, especially in cases of non-trivial free energy configurations, where training data may be sparse. We demonstrate a path towards uncertainty-aware models of coarse grained free energy surfaces. Specifically, we show that principled Bayesian model uncertainty allows for efficient data collection through an on-the-fly active learning framework and open the possibility of adaptive transfer of models across different chemical systems. Uncertainties also characterize models' accuracy of free energy predictions, even when training is performed only on forces. This work helps pave the way towards efficient autonomous training of reliable and uncertainty aware many-body machine learned coarse grain models.


title: Elastic Weight Consolidation Improves the Robustness of Self-Supervised Learning Methods under Transfer

abstract: Self-supervised representation learning (SSL) methods provide an effective label-free initial condition for fine-tuning downstream tasks. However, in numerous realistic scenarios, the downstream task might be biased with respect to the target label distribution. This in turn moves the learned fine-tuned model posterior away from the initial (label) bias-free self-supervised model posterior. In this work, we re-interpret SSL fine-tuning under the lens of Bayesian continual learning and consider regularization through the Elastic Weight Consolidation (EWC) framework. We demonstrate that self-regularization against an initial SSL backbone improves worst sub-group performance in Waterbirds by 5% and Celeb-A by 2% when using the ViT-B/16 architecture. Furthermore, to help simplify the use of EWC with SSL, we pre-compute and publicly release the Fisher Information Matrix (FIM), evaluated with 10,000 ImageNet-1K variates evaluated on large modern SSL architectures including ViT-B/16 and ResNet50 trained with DINO.


title: Second release of the CoRe database of binary neutron star merger waveforms

abstract: We present the second data release of gravitational waveforms from binary neutron star merger simulations performed by the Computational Relativity (CoRe) collaboration. The current database consists of 254 different binary neutron star configurations and a total of 590 individual numerical-relativity simulations using various grid resolutions. The released waveform data contain the strain and the Weyl curvature multipoles up to $\ell=m=4$. They span a significant portion of the mass, mass-ratio,spin and eccentricity parameter space and include targeted configurations to the events GW170817 and GW190425. CoRe simulations are performed with 18 different equations of state, seven of which are finite temperature models, and three of which account for non-hadronic degrees of freedom. About half of the released data are computed with high-order hydrodynamics schemes for tens of orbits to merger; the other half is computed with advanced microphysics. We showcase a standard waveform error analysis and discuss the accuracy of the database in terms of faithfulness. We present ready-to-use fitting formulas for equation of state-insensitive relations at merger (e.g. merger frequency), luminosity peak, and post-merger spectrum.


title: LAKEE: A Lightweight Authenticated Key Exchange Protocol for Power Constrained Devices

abstract: The rapid development of IoT networks has led to a research trend in designing effective security features for them. Due to the power-constrained nature of IoT devices, the security features should remain as lightweight as possible. Currently, most of the IoT network traffic is unencrypted. The leakage of smart devices' unencrypted data can come with the significant cost of a privacy breach. To have a secure channel with encrypted traffic, two endpoints in a network have to authenticate each other and calculate a short-term key. They can then communicate through an authenticated and secure channel. This process is referred to as authenticated key exchange (AKE). Although Datagram Transport Layer Security (DTLS) offers an AKE protocol for IoT networks, research has proposed more efficient and case-specific alternatives. This paper presents LAKEE, a straightforward, lightweight AKE protocol for IoT networks. Our protocol employs elliptic curve cryptography for generating a short-term session key. It reduces the communication and computational overhead of its alternatives while maintaining or improving their security strength. The simplicity and low overhead of our protocol make it a fit for a network of constrained devices.


title: Dark energy based on exotic statistics

abstract: Dark energy is an elusive concept, which has been introduced two decades ago in order to make the acceleration of the universe a comprehensible phenomenon. However, the nature of this energy is far from being understood, both from a fundamental as well as an observational way. In this work we study cosmological consequences of the existence of particles (which we called ``ewkons'' in a previous work) which are quasi distinguishable, obey unorthodox statistics, and have an equation of state similar to many existent dark energy candidates (including negative relation between pressure and energy density). We find an effective scalar field description of this ewkon fluid, and obtain cosmological solutions for the dark energy dominated epoch. This can be considered as a one-parameter class of dark energy models.


title: Browder's Theorem: from One-Dimensional Parameter Space to General Parameter Space

abstract: A parametric version of Brouwer's Fixed Point Theorem, which is proven using the fixed-point index, states that for every continuous mapping $f : (X \times Y) \to Y$, where $X$ is nonempty, compact, and connected subset of a Hausdorff topological space and $Y$ is a nonempty, convex, and compact subset of a locally-convex topological vector space, the set of fixed points of $f$, defined by $C_f := \{ (x,y) \in X \times Y \colon f(x,y)=y\}$, has a connected component whose projection onto the first coordinate is $X$. In this note we provide an elementary proof for this result, using its reduction to the case $X = [0,1]$.


title: Intermodulation of optical frequency combs in a multimode optomechanical system

abstract: Phonons offer the possibility to connect the microwave and optical domains while being efficiently transduced with electronic and optical signals. Here, we present a multimodal optomechanical platform, consisting of a mechanical-optical-mechanical resonator configuration. The mechanical modes, with frequencies at 265 MHz and 6.8 GHz, can be simultaneously excited into a phonon lasing regime as supported by a stability analysis of the system. Both the MHz and the GHz modes enter a self-sustained oscillation regime, leading to the intermodulation of two frequency combs in the optical field. We characterize this platform experimentally, demonstrating previously unexplored dynamical regimes. These results suggest the possibility to control multiple mechanical degrees of freedom via a single optical mode, with implications in GHz phononic devices, signal processing, and optical comb sensing applications.


title: Distributed Black-box Attack against Image Classification Cloud Services

abstract: Black-box adversarial attacks can fool image classifiers into misclassifying images without requiring access to model structure and weights. Recent studies have reported attack success rates of over 95% with less than 1,000 queries. The question then arises of whether black-box attacks have become a real threat against IoT devices that rely on cloud APIs to achieve image classification. To shed some light on this, note that prior research has primarily focused on increasing the success rate and reducing the number of queries. However, another crucial factor for black-box attacks against cloud APIs is the time required to perform the attack. This paper applies black-box attacks directly to cloud APIs rather than to local models, thereby avoiding mistakes made in prior research that applied the perturbation before image encoding and pre-processing. Further, we exploit load balancing to enable distributed black-box attacks that can reduce the attack time by a factor of about five for both local search and gradient estimation methods.


title: Remarks on the Clauser-Horne-Shimony-Holt inequality in relativistic quantum field theory

abstract: We present an investigation of the $CHSH$ inequality within a relativistic quantum field theory model built up with a pair of free massive scalar fields $(\varphi_A, \varphi_B)$ where, as it is customary, the indices $(A,B)$ refer to Alice and Bob, respectively. A set of bounded Hermitian operators is introduced by making use of the Weyl operators. A $CHSH$ type correlator is constructed and evaluated in the Fock vacuum by means of the canonical quantization. Although the observed violation of the $CHSH$ inequality turns out to be rather small as compared to Tsirelson's bound of Quantum Mechanics, the model can be employed for the study of Bell's inequalities in the more physical case of gauge theories such as: the Higgs models, for which local $BRST$ invariant operators describing both the massive gauge boson as well as the Higgs particle have been devised. These operators can be naturally exponentiated, leading to $BRST$ invariant type of Weyl operators useful to analyze Bell's inequalities within an invariant $BRST$ environment.


title: Continuous Attribution of Episodical Outcomes for More Efficient and Targeted Online Measurement

abstract: Online experimentation platforms collect user feedback at low cost and large scale. Some systems even support real-time or near real-time data processing, and can update metrics and statistics continuously. Many commonly used metrics, such as clicks and page views, can be observed without much delay. However, many important signals can only be observed after several hours or days, with noise adding up over the duration of the episode. When episodical outcomes follow a complex sequence of user-product interactions, it is difficult to understand which interactions lead to the final outcome. There is no obvious attribution logic for us to associate a positive or negative outcome back to the actions and choices we made at different times. This attribution logic is critical to unlocking more targeted and efficient measurement at a finer granularity that could eventually lead to the full capability of reinforcement learning. In this paper, we borrow the idea of Causal Surrogacy to model a long-term outcome using leading indicators that are incrementally observed and apply it as the value function to track the progress towards the final outcome and attribute incrementally to various user-product interaction steps. Applying this approach to the guest booking metric at Airbnb resulted in significant variance reductions of 50% to 85%, while aligning well with the booking metric itself. Continuous attribution allows us to assign a utility score to each product page-view, and this score can be flexibly further aggregated to a variety of units of interest, such as searches and listings. We provide multiple real-world applications of attribution to illustrate its versatility.


title: Evaluation of lattice light shift at low 10$^{-19}$ uncertainty for a shallow lattice Sr optical clock

abstract: A Wannier-Stark optical lattice clock has demonstrated unprecedented measurement precision for optical atomic clocks. We present a systematic evaluation of the lattice light shift, a necessary next step for establishing this system as an accurate atomic clock. With precise control of the atomic motional states in the lattice, we report accurate measurements of the multipolar and the hyperpolar contributions and the operational lattice light shift with a fractional frequency uncertainty of 3.5$\times$10${}^{-19}$.


title: SoftBart: Soft Bayesian Additive Regression Trees

abstract: Bayesian additive regression tree (BART) models have seen increased attention in recent years as a general-purpose nonparametric modeling technique. BART combines the flexibility of modern machine learning techniques with the principled uncertainty quantification of Bayesian inference, and it has been shown to be uniquely appropriate for addressing the high-noise problems that occur commonly in many areas of science, including medicine and the social sciences. This paper introduces the SoftBart package for fitting the Soft BART algorithm of Linero and Yang (2018). In addition to improving upon the predictive performance of other BART packages, a major goal of this package has been to facilitate the inclusion of BART in larger models, making it ideal for researchers in Bayesian statistics. I show both how to use this package for standard prediction tasks and how to embed BART models in larger models; I illustrate by using SoftBart to implement a nonparametric probit regression model, a semiparametric varying coefficient model, and a partial linear model.


title: A Heintze--Karcher inequality with free boundaries and applications to capillarity theory

abstract: In this paper we analyze the shape of a droplet inside a smooth container. To characterize their shape in the capillarity regime, we obtain a new form of the Heintze--Karcher inequality for mean convex hypersufaces with boundary lying on curved substrates.


title: Least-squares finite elements for distributed optimal control problems

abstract: We provide a framework for the numerical approximation of distributed optimal control problems, based on least-squares finite element methods. Our proposed method simultaneously solves the state and adjoint equations and is $\inf$--$\sup$ stable for any choice of conforming discretization spaces. A reliable and efficient a posteriori error estimator is derived for problems where box constraints are imposed on the control. It can be localized and therefore used to steer an adaptive algorithm. For unconstrained optimal control problems, i.e., the set of controls being a Hilbert space, we obtain a coercive least-squares method and, in particular, quasi-optimality for any choice of discrete approximation space. For constrained problems we derive and analyze a variational inequality where the PDE part is tackled by least-squares finite element methods. We show that the abstract framework can be applied to a wide range of problems, including scalar second-order PDEs, the Stokes problem, and parabolic problems on space-time domains. Numerical examples for some selected problems are presented.


title: Modeling and Rapid Prototyping of Integrated Transmission-Distribution OPF Formulations with PowerModelsITD.jl

abstract: Conventional electric power systems are composed of different unidirectional power flow stages of generation, transmission, and distribution, managed independently by transmission system and distribution system operators. However, as distribution systems increase in complexity due to the integration of distributed energy resources, coordination between transmission and distribution networks will be imperative for the optimal operation of the power grid. However, coupling models and formulations between transmission and distribution is non-trivial, in particular due to the common practice of modeling transmission systems as single-phase, and distribution systems as multi-conductor phase-unbalanced. To enable the rapid prototyping of power flow formulations, in particular in the modeling of the boundary conditions between these two seemingly incompatible data models, we introduce PowerModelsITD.jl, a free, open-source toolkit written in Julia for integrated transmission-distribution (ITD) optimization that leverages mature optimization libraries from the InfrastructureModels.jl-ecosystem. The primary objective of the proposed framework is to provide baseline implementations of steady-state ITD optimization problems, while providing a common platform for the evaluation of emerging formulations and optimization problems. In this work, we introduce the nonlinear formulations currently supported in PowerModelsITD.jl, which include AC-polar, AC-rectangular, current-voltage, and a linear network transportation model. Results are validated using combinations of IEEE transmission and distribution networks.


title: Resonant Spin Hall Effect of Light in Random Photonic Arrays

abstract: It has been recently shown that the coherent component of light propagating in transversally disordered media, the so-called coherent mode, exhibits an optical spin Hall effect (SHE). In non-resonant materials, however, this phenomenon shows up at a spatial scale much larger than the mean free path, making its observation challenging due to the exponential attenuation of the coherent mode. Here, we show that in disordered photonic arrays exhibiting Mie resonances, the SHE on the contrary appears at a scale smaller than the mean free path if one operates in the close vicinity of the lowest transverse-magnetic resonance of the array. In combination with a weak measurement, this gives rise to a giant SHE that should be observable in optically-thin media. Furthermore, we show that by additionally exploiting the cooperative emission of a flash of light following the abrupt extinction of the incoming beam, one can achieve a time-dependent SHE, observable at large optical thickness as well.


title: Incorporating Crowdsourced Annotator Distributions into Ensemble Modeling to Improve Classification Trustworthiness for Ancient Greek Papyri

abstract: Performing classification on noisy, crowdsourced image datasets can prove challenging even for the best neural networks. Two issues which complicate the problem on such datasets are class imbalance and ground-truth uncertainty in labeling. The AL-ALL and AL-PUB datasets - consisting of tightly cropped, individual characters from images of ancient Greek papyri - are strongly affected by both issues. The application of ensemble modeling to such datasets can help identify images where the ground-truth is questionable and quantify the trustworthiness of those samples. As such, we apply stacked generalization consisting of nearly identical ResNets with different loss functions: one utilizing sparse cross-entropy (CXE) and the other Kullback-Liebler Divergence (KLD). Both networks use labels drawn from a crowd-sourced consensus. This consensus is derived from a Normalized Distribution of Annotations (NDA) based on all annotations for a given character in the dataset. For the second network, the KLD is calculated with respect to the NDA. For our ensemble model, we apply a k-nearest neighbors model to the outputs of the CXE and KLD networks. Individually, the ResNet models have approximately 93% accuracy, while the ensemble model achieves an accuracy of > 95%, increasing the classification trustworthiness. We also perform an analysis of the Shannon entropy of the various models' output distributions to measure classification uncertainty. Our results suggest that entropy is useful for predicting model misclassifications.


title: Not Another Day Zero: Design Hackathons for Community-Based Water Quality Monitoring

abstract: This study looks at water quality monitoring and management as a new form of community engagement. Through a series of a unique research method called `design hackathons', we engaged with a hyperlocal community of citizens who are actively involved in monitoring and management of their local watershed. These design hackathons sought to understand the motivation, practices, collaboration and experiences of these citizens. Qualitative analysis of data revealed the nature of the complex stakeholder network, workflow practices, initiatives to engage with a larger community, current state of technological infrastructure being used, and innovative design scenarios proposed by the hackathon participants. Based on this comprehensive analysis, we conceptualize water quality monitoring and management as community-based monitoring and management, and water data as community data. Such a conceptualization sheds light on how these practices can help in preempting water crisis by empowering citizens through increased awareness, active participation and informal learning of water data and resources.


title: The Hurricane Track Fit Consensus Model for Improving Hurricane Forecasting

abstract: We present a new method for creating a model consensus to improve real-time hurricane track prediction. The method is based on the statistical fitting of historic numerical model track forecasts to the observed storm positions and learning from their historical errors and biases. Our method is closest to the HFIP Corrected Consensus Approach (HCCA) methodology while using an alternative model formulation. Our method creates a separate consensus model for each forecast hour making it possible to independently correct the bias of each input model for that specific hour. This approach, which we call the Hurricane Track Fit (HFIT) model, is computationally efficient and scalable to additional numerical models as input, and it produces interpretable coefficients weighing model contributions. The new method is evaluated for the 2014-2021 hurricane seasons in the Atlantic basin using the input from the best-performing operational track forecast guidance at the National Hurricane Center (NHC): the U.S. National Weather Service Global Forecast System deterministic and ensemble mean models, European Centre for Medium-Range Weather Forecasts deterministic model, the NWS Hurricane Weather Research and Forecasting model and the NHC equally weighted numerical model track consensus (TVCA). The results of the cross-validation for the 2014-2021 hurricane track dataset show that the HFIT consensus model consistently reduces the track forecast errors compared to those from the input models and the official NHC forecasts (OFCL). For example, at 24h the HFIT track forecast errors are smaller by 18.5% and 15.6% than those in AVNI and EMXI respectively, and 23% and 15% smaller at 72h. The HFIT forecasts show a reduction of errors compared to OFCL by 8.1% at 24h and 7.5% at 72h. We also discuss the successful real-time operational performance of HFIT during the 2022 hurricane season.


title: Formalizing Anisotropic Inflation in Modified Gravity

abstract: Motivated by the fact that the pre-inflationary era may evolve in an exotic way, in this work we formalize anisotropic evolution in the context of modified gravity, focusing on pre-inflationary and near the vicinity of the inflationary epochs. We specialize on specific metrics like Bianchi and Taub and we formalize the inflationary theory in vacuum $F(R)$ gravity, in $F(R)$ gravity with an extra scalar field and in Gauss-Bonnet gravity. We discuss the qualitative effects of the anisotropies on the evolution of the Universe and also we consider several specific solutions, like the de Sitter solution, in both the isotropic and anisotropic contexts. Furthermore, several exotic modified gravity cosmological solutions, like the ones which contain finite time singularities, are also discussed in brief.


title: A remark on geodesics in the Banach Mazur distance

abstract: We show that there are uncountably many geodesics between any two non-isometric $n$-dimensional normed spaces. We construct two explicit geodesics that can be used to describe all the points of the other geodesics.


title: Optimization of Hydrogen Blending in Natural Gas Networks for Carbon Emissions Reduction

abstract: We present an economic optimization problem for allocating the flow of natural gas and hydrogen blends through a large-scale transportation pipeline network. Physical flow of the gas mixture is modeled using a steady-state relation between pressure decrease and flow rate, which depends on mass concentration of the constituents as it varies by location in the network. The objective reflects the economic value provided by the system, accounting for delivered energy in withdrawn flows, the cost of natural gas and hydrogen injections, and avoided carbon emissions. The problem is solved subject to physical flow equations, nodal balance and mixing laws, and engineering inequality constraints. The desired energy delivery rate and minimum hydrogen concentration can be specified as upper and lower bound values, respectively, of inequality constraints, and we examine the sensitivity of the physical pressure and flow solution to these parameters for two test networks. The results confirm that increasing hydrogen concentration requires greater energy expended for compression to deliver the same energy content, and the formulation could be used for valuation of the resulting mitigation of carbon emissions.


title: Non-Stationary Bandits with Auto-Regressive Temporal Dependency

abstract: Traditional multi-armed bandit (MAB) frameworks, predominantly examined under stochastic or adversarial settings, often overlook the temporal dynamics inherent in many real-world applications such as recommendation systems and online advertising. This paper introduces a novel non-stationary MAB framework that captures the temporal structure of these real-world dynamics through an auto-regressive (AR) reward structure. We propose an algorithm that integrates two key mechanisms: (i) an alternation mechanism adept at leveraging temporal dependencies to dynamically balance exploration and exploitation, and (ii) a restarting mechanism designed to discard out-of-date information. Our algorithm achieves a regret upper bound that nearly matches the lower bound, with regret measured against a robust dynamic benchmark. Finally, via a real-world case study on tourism demand prediction, we demonstrate both the efficacy of our algorithm and the broader applicability of our techniques to more complex, rapidly evolving time series.


title: An Integrated Photon-Pair Source with Monolithic Piezoelectric Frequency Tunability

abstract: This work demonstrates the capabilities of an entangled photon-pair source at telecom wavelengths, based on a photonic integrated Si$_3$N$_4$ microresonator with monolithically integrated piezoelectric frequency tuning. Previously, frequency tuning of photon-pairs generated by microresonators has only been demonstrated using thermal control, however these have limited actuation bandwidth, and are not compatible with cryogenic environments. Here, the frequency-tunable photon-pair generation capabilities of a Si$_3$N$_4$ microresonator with a monolithically integrated aluminium nitride layer are shown. Fast-frequency locking of the microresonator to an external laser is demonstrated, with a resulting locking bandwidth orders of magnitude larger than reported previously using thermal locking. These abilities will have direct application in future schemes which interface such sources with quantum memories based on e.g. trapped-ion or rare-earth ion schemes.


title: The electromagnetic vacuum field as an essential ingredient of the quantum-mechanical ontology

abstract: Abstract This paper provides elements in support of the random zero-point radiation field (zpf) as an essential ontological ingredient needed to explain distinctive properties of quantum-mechanical systems. We show that when an otherwise classical particle is connected to the zpf, a drastic, qualitative change in the dynamics takes place, leading eventually to the quantum dynamics. In particular, we demonstrate that in parallel with the evolution of the particle canonical variables into quantum operators satisfying the basic commutator \left[\hat{x},\hat{p}\right]=i\hbar, also the field canonical variables are transformed, giving rise to the corresponding creation and annihilation operators \hat{a}^{\dagger},\hat{a}, satisfying \left[\hat{a},\hat{a}^{\dagger}\right]=1. This allows for an explanation of quantum features such as quantum fluctuations, stationary states and transitions, and establishes a natural contact with (nonrelativistic) quantum electrodynamics.


title: A Complete Hierarchy of Linear Systems for Certifying Quantum Entanglement of Subspaces

abstract: We introduce a hierarchy of linear systems for showing that a given subspace of pure quantum states is entangled (i.e., contains no product states). This hierarchy outperforms known methods already at the first level, and it is complete in the sense that every entangled subspace is shown to be so at some finite level of the hierarchy. It generalizes straightforwardly to the case of higher Schmidt rank, as well as the multipartite cases of completely and genuinely entangled subspaces. These hierarchies work extremely well in practice even in very large quantum systems, as they can be implemented via elementary linear algebra techniques rather than the semidefinite programming techniques that are required by previously-known hierarchies.


title: Emergence of Layer Stacking Disorder in c-axis Confined MoTe$_2$

abstract: The layer stacking order in 2D materials strongly affects functional properties and holds promise for next generation electronic devices. In bulk, octahedral MoTe$_2$ possesses two stacking arrangements, the Weyl semimetal T$_d$ phase, and the higher-order topological insulator 1T' phase; however, it remains unclear if thin exfoliated flakes of MoTe$_2$ follow the T$_d$, 1T', or an alternative stacking sequence. Here, we resolve this debate using atomic-resolution imaging within the transmission electron microscope. We find that the layer stacking in thin flakes of MoTe$_2$ is highly disordered and pseudo-random, which we attribute to intrinsic confinement effects. Conversely, WTe$_2$, which is isostructural and isoelectronic to MoTe$_2$, displays ordered stacking even for thin exfoliated flakes. Our results are important for understanding the quantum properties of MoTe$_2$ devices, and suggest that thickness may be used to alter the layer stacking in other 2D materials.


title: Radically Lower Data-Labeling Costs for Visually Rich Document Extraction Models

abstract: A key bottleneck in building automatic extraction models for visually rich documents like invoices is the cost of acquiring the several thousand high-quality labeled documents that are needed to train a model with acceptable accuracy. We propose Selective Labeling to simplify the labeling task to provide "yes/no" labels for candidate extractions predicted by a model trained on partially labeled documents. We combine this with a custom active learning strategy to find the predictions that the model is most uncertain about. We show through experiments on document types drawn from 3 different domains that selective labeling can reduce the cost of acquiring labeled data by $10\times$ with a negligible loss in accuracy.


title: Physics-aware Graph Neural Network for Accurate RNA 3D Structure Prediction

abstract: Biological functions of RNAs are determined by their three-dimensional (3D) structures. Thus, given the limited number of experimentally determined RNA structures, the prediction of RNA structures will facilitate elucidating RNA functions and RNA-targeted drug discovery, but remains a challenging task. In this work, we propose a Graph Neural Network (GNN)-based scoring function trained only with the atomic types and coordinates on limited solved RNA 3D structures for distinguishing accurate structural models. The proposed Physics-aware Multiplex Graph Neural Network (PaxNet) separately models the local and non-local interactions inspired by molecular mechanics. Furthermore, PaxNet contains an attention-based fusion module that learns the individual contribution of each interaction type for the final prediction. We rigorously evaluate the performance of PaxNet on two benchmarks and compare it with several state-of-the-art baselines. The results show that PaxNet significantly outperforms all the baselines overall, and demonstrate the potential of PaxNet for improving the 3D structure modeling of RNA and other macromolecules. Our code is available at https://github.com/zetayue/Physics-aware-Multiplex-GNN.


title: Semimetallic and semiconducting graphene-hBN multilayers with parallel or reverse stacking

abstract: We theoretically investigate 3D layered crystals of alternating graphene and hBN layers with different symmetries. Depending on the hopping parameters between the graphene layers, we find that these synthetic 3D materials can feature semimetallic, gapped, or Weyl semimetal phases. Our results demonstrate that 3D crystals stacked from individual 2D materials represent a synthetic materials class with emergent properties different from their constituents.


title: HeartSiam: A Domain Invariant Model for Heart Sound Classification

abstract: Cardiovascular disease is one of the leading causes of death according to WHO. Phonocardiography (PCG) is a costeffective, non-invasive method suitable for heart monitoring. The main aim of this work is to classify heart sounds into normal/abnormal categories. Heart sounds are recorded using different stethoscopes, thus varying in the domain. Based on recent studies, this variability can affect heart sound classification. This work presents a Siamese network architecture for learning the similarity between normal vs. normal or abnormal vs. abnormal signals and the difference between normal vs. abnormal signals. By applying this similarity and difference learning across all domains, the task of domain invariant heart sound classification can be well achieved. We have used the multi-domain 2016 Physionet/CinC challenge dataset for the evaluation method. Results: On the evaluation set provided by the challenge, we have achieved a sensitivity of 82.8%, specificity of 75.3%, and mean accuracy of 79.1%. While overcoming the multi-domain problem, the proposed method has surpassed the first-place method of the Physionet challenge in terms of specificity up to 10.9% and mean accuracy up to 5.6%. Also, compared with similar state-of-the-art domain invariant methods, our model converges faster and performs better in specificity (4.1%) and mean accuracy (1.5%) with an equal number of epochs learned.


title: Ensure Differential Privacy and Convergence Accuracy in Consensus Tracking and Aggregative Games with Coupling Constraints

abstract: We address differential privacy for fully distributed aggregative games with shared coupling constraints. By co-designing the generalized Nash equilibrium (GNE) seeking mechanism and the differential-privacy noise injection mechanism, we propose the first GNE seeking algorithm that can ensure both provable convergence to the GNE and rigorous epsilon-differential privacy, even with the number of iterations tending to infinity. As a basis of the co-design, we also propose a new consensus-tracking algorithm that can achieve rigorous epsilon-differential privacy while maintaining accurate tracking performance, which, to our knowledge, has not been achieved before. To facilitate the convergence analysis, we also establish a general convergence result for stochastically-perturbed nonstationary fixed-point iteration processes, which lie at the core of numerous optimization and variational problems. Numerical simulation results confirm the effectiveness of the proposed approach.


title: Enhanced Energy-Saving Mechanisms in TSCH Networks for the IIoT: the PRIL Approach

abstract: Lifetime of motes in wireless sensor networks can be enlarged by decreasing the energy spent for communication. Approaches like time slotted channel hopping pursue this goal by performing frame exchanges according to a predefined schedule, which helps reducing the duty cycle. Unfortunately, whenever the receiving radio interface is active but nobody in the network is transmitting, idle listening occurs. If the traffic pattern is known in advance, as in the relevant case of periodic sensing, proactive reduction of idle listening (PRIL) noticeably lowers energy waste by disabling receivers when no frames are expected for them. Optimal PRIL operation demands that, at any time, the transmitter and receiver sides of a link have a coherent view of its state (either enabled or disabled). However, this is not ensured in the presence of acknowledgment frame losses. This paper presents and analyzes some strategies to cope with such events. An extensive experimental campaign has been carried out through discrete event simulation to determine what consequences above errors may have from both a functional and performance viewpoint. Results show that, although no strategy is optimal in all circumstances, different solutions can be profitably adopted depending on the specific operating conditions.


title: Infrared Effects and the Unruh State

abstract: Detailed behaviors of the modes of quantized scalar fields in the Unruh state for various eternal black holes in two dimensions are investigated. It is shown that the late-time behaviors of some of the modes of the quantum fields and of the symmetric two-point function are determined by infrared effects. The nature of these effects depends upon whether there is an effective potential in the mode equation and what form this potential takes. Here, three cases are considered, one with no potential and two with potentials that are nonnegative everywhere and are zero on the event horizon of the black hole and zero at either infinity or the cosmological horizon. Specifically, the potentials are a delta function potential and the potential that occurs for a massive scalar field in Schwarzschild-de Sitter spacetime. In both cases, scattering effects remove infrared divergences in the mode functions that would otherwise arise from the normalization process. When such infrared divergences are removed, it is found that the modes that are positive frequency with respect to the Kruskal time on the past black hole horizon approach zero in the limit that the radial coordinate is fixed and the time coordinate goes to infinity. In contrast, when there is no potential and thus infrared divergences occur, the same modes approach nonzero constant values in the late-time limit when the radial coordinate is held fixed. The behavior of the symmetric two-point function when the field is in the Unruh state is investigated for the case of a delta function potential in certain asymptotically flat black hole spacetimes in two dimensions. The removal of the infrared divergences in the mode functions results in the elimination of terms that grow linearly in time.


title: System Demo: Tool and Infrastructure for Offensive Language Error Analysis (OLEA) in English

abstract: The automatic detection of offensive language is a pressing societal need. Many systems perform well on explicit offensive language but struggle to detect more complex, nuanced, or implicit cases of offensive and hateful language. OLEA is an open-source Python library that provides easy-to-use tools for error analysis in the context of detecting offensive language in English. OLEA also provides an infrastructure for re-distribution of new datasets and analysis methods requiring very little coding.


title: U-Net-based Models for Skin Lesion Segmentation: More Attention and Augmentation

abstract: According to WHO[1], since the 1970s, diagnosis of melanoma skin cancer has been more frequent. However, if detected early, the 5-year survival rate for melanoma can increase to 99 percent. In this regard, skin lesion segmentation can be pivotal in monitoring and treatment planning. In this work, ten models and four augmentation configurations are trained on the ISIC 2016 dataset. The performance and overfitting are compared utilizing five metrics. Our results show that the U-Net-Resnet50 and the R2U-Net have the highest metrics value, along with two data augmentation scenarios. We also investigate CBAM and AG blocks in the U-Net architecture, which enhances segmentation performance at a meager computational cost. In addition, we propose using pyramid, AG, and CBAM blocks in a sequence, which significantly surpasses the results of using the two individually. Finally, our experiments show that models that have exploited attention modules successfully overcome common skin lesion segmentation problems. Lastly, in the spirit of reproducible research, we implement models and codes publicly available.


title: Flatter, faster: scaling momentum for optimal speedup of SGD

abstract: Commonly used optimization algorithms often show a trade-off between good generalization and fast training times. For instance, stochastic gradient descent (SGD) tends to have good generalization; however, adaptive gradient methods have superior training times. Momentum can help accelerate training with SGD, but so far there has been no principled way to select the momentum hyperparameter. Here we study training dynamics arising from the interplay between SGD with label noise and momentum in the training of overparametrized neural networks. We find that scaling the momentum hyperparameter $1-\beta$ with the learning rate to the power of $2/3$ maximally accelerates training, without sacrificing generalization. To analytically derive this result we develop an architecture-independent framework, where the main assumption is the existence of a degenerate manifold of global minimizers, as is natural in overparametrized models. Training dynamics display the emergence of two characteristic timescales that are well-separated for generic values of the hyperparameters. The maximum acceleration of training is reached when these two timescales meet, which in turn determines the scaling limit we propose. We confirm our scaling rule for synthetic regression problems (matrix sensing and teacher-student paradigm) and classification for realistic datasets (ResNet-18 on CIFAR10, 6-layer MLP on FashionMNIST), suggesting the robustness of our scaling rule to variations in architectures and datasets.


title: The Fisher-Rao Loss for Learning under Label Noise

abstract: Choosing a suitable loss function is essential when learning by empirical risk minimisation. In many practical cases, the datasets used for training a classifier may contain incorrect labels, which prompts the interest for using loss functions that are inherently robust to label noise. In this paper, we study the Fisher-Rao loss function, which emerges from the Fisher-Rao distance in the statistical manifold of discrete distributions. We derive an upper bound for the performance degradation in the presence of label noise, and analyse the learning speed of this loss. Comparing with other commonly used losses, we argue that the Fisher-Rao loss provides a natural trade-off between robustness and training dynamics. Numerical experiments with synthetic and MNIST datasets illustrate this performance.


title: GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexity

abstract: We study a class of distributed optimization algorithms that aim to alleviate high communication costs by allowing the clients to perform multiple local gradient-type training steps prior to communication. While methods of this type have been studied for about a decade, the empirically observed acceleration properties of local training eluded all attempts at theoretical understanding. In a recent breakthrough, Mishchenko et al. (ICML 2022) proved that local training, when properly executed, leads to provable communication acceleration, and this holds in the strongly convex regime without relying on any data similarity assumptions. However, their method ProxSkip requires all clients to take the same number of local training steps in each communication round. Inspired by a common sense intuition, we start our investigation by conjecturing that clients with ``less important'' data should be able to get away with fewer local training steps without this impacting the overall communication complexity of the method. It turns out that this intuition is correct: we managed to redesign the original ProxSkip method to achieve this. In particular, we prove that our modified method, for which coin the name GradSkip, converges linearly under the same assumptions, has the same accelerated communication complexity, while the number of local gradient steps can be reduced relative to a local condition number. We further generalize our method by extending the randomness of probabilistic alternations to arbitrary unbiased compression operators and considering a generic proximable regularizer. This generalization, which we call GradSkip+, recovers several related methods in the literature as special cases. Finally, we present an empirical study on carefully designed toy problems that confirm our theoretical claims.


title: The first gradual solar energetic particle event with enhanced 3He abundance on Solar Orbiter

abstract: The origin of 3He abundance enhancements in coronal mass ejection (CME)-driven shock gradual solar energetic particle (SEP) events remains largely unexplained. Two mechanisms have been suggested - the re-acceleration of remnant flare material in interplanetary space and concomitant activity in the corona. We explore the first gradual SEP event with enhanced 3He abundance observed by Solar Orbiter. The event started on 2020 November 24 and was associated with a relatively fast halo CME. During the event, the spacecraft was at 0.9 au from the Sun. The event averaged 3He/4He abundance ratio is 24 times higher than the coronal or solar wind value, and the 3He intensity had timing similar to other species. We inspected available imaging, radio observations, and spacecraft magnetic connection to the CME source. It appears the most probable cause of the enhanced 3He abundance are residual 3He ions remaining from a preceding long period of 3He-rich SEPs on 2020 November 17-23.


title: Experimental Evaluation of Seamless Redundancy Applied to Industrial Wi-Fi Networks

abstract: Seamless redundancy can be profitably exploited to improve predictability of wireless networks in general and, in particular, IEEE 802.11. According to this approach, packets are transmitted by senders on two (or more) channels at the same time and duplicate copies are discarded by receivers. As long as the behavior of physical channels is uncorrelated, communication quality improves noticeably, in terms of both transmission latencies and percentage of dropped frames. In this paper, communication over redundant links has been analyzed by means of a thorough experimental campaign, based on measurements carried out on real devices. Results confirm that, under typical operating conditions, the assumption of independence among channels in properly designed systems is verified reasonably well. Indeed, in our experiments, measured link quality indices did not differ more than 10% from what we expected from theory. This grants for redundant solutions tangible advantages over conventional Wi-Fi networks.


title: Evaluation of Categorical Generative Models -- Bridging the Gap Between Real and Synthetic Data

abstract: The machine learning community has mainly relied on real data to benchmark algorithms as it provides compelling evidence of model applicability. Evaluation on synthetic datasets can be a powerful tool to provide a better understanding of a model's strengths, weaknesses, and overall capabilities. Gaining these insights can be particularly important for generative modeling as the target quantity is completely unknown. Multiple issues related to the evaluation of generative models have been reported in the literature. We argue those problems can be avoided by an evaluation based on ground truth. General criticisms of synthetic experiments are that they are too simplified and not representative of practical scenarios. As such, our experimental setting is tailored to a realistic generative task. We focus on categorical data and introduce an appropriately scalable evaluation method. Our method involves tasking a generative model to learn a distribution in a high-dimensional setting. We then successively bin the large space to obtain smaller probability spaces where meaningful statistical tests can be applied. We consider increasingly large probability spaces, which correspond to increasingly difficult modeling tasks and compare the generative models based on the highest task difficulty they can reach before being detected as being too far from the ground truth. We validate our evaluation procedure with synthetic experiments on both synthetic generative models and current state-of-the-art categorical generative models.


title: Gallai's Conjecture for Complete and "Nearly Complete" Graphs

abstract: The famous Gallai's Conjecture states that any connected graph with n vertices has a path decomposition containing at most (n+1)/2 paths. In this note, we explore graphs generated from removing edges from complete graphs. We first provide an explicit construction for a path decomposition of complete graphs that satisfies Gallai's Conjecture. We then use that construction to prove that we can remove stars and certain tadpoles such that the resulting graph still satisfies Gallai's Conjecture. We also introduce a potential general approach through analyzing non-isomorphic path decompositions of complete graphs.


title: Just-DREAM-about-it: Figurative Language Understanding with DREAM-FLUTE

abstract: Figurative language (e.g., "he flew like the wind") is challenging to understand, as it is hard to tell what implicit information is being conveyed from the surface form alone. We hypothesize that to perform this task well, the reader needs to mentally elaborate the scene being described to identify a sensible meaning of the language. We present DREAM-FLUTE, a figurative language understanding system that does this, first forming a "mental model" of situations described in a premise and hypothesis before making an entailment/contradiction decision and generating an explanation. DREAM-FLUTE uses an existing scene elaboration model, DREAM, for constructing its "mental model." In the FigLang2022 Shared Task evaluation, DREAM-FLUTE achieved (joint) first place (Acc@60=63.3%), and can perform even better with ensemble techniques, demonstrating the effectiveness of this approach. More generally, this work suggests that adding a reflective component to pretrained language models can improve their performance beyond standard fine-tuning (3.3% improvement in Acc@60).


title: Heterogeneous Position Effects and the Power of Rankings

abstract: Many online retailers and search intermediaries present products on ranked product lists. Because changing the ordering of products influences their revenues and consumer welfare, these platforms can design rankings to maximize either of these two metrics. In this paper, I study how rankings serving these two objectives differ and what determines this difference. First, I highlight that when position effects are heterogeneous, rankings can increase both revenues and consumer welfare by increasing the overall conversion rate. Second, I provide empirical evidence for this heterogeneity, highlighting that cheaper alternatives have stronger position effects. Finally, I quantify revenue and consumer welfare effects across the different rankings. To this end, I develop an estimation procedure for the search and discovery model of Greminger (2022) that yields a smooth likelihood function by construction. The results from the model show that rankings designed to maximize revenues can also increase consumer welfare. Moreover, these revenue-based rankings reduce consumer welfare only to a limited extent relative to a consumer-welfare-maximizing ranking.


title: An innovative materials design protocol for the development of novel refractory high-entropy alloys for extreme environments

abstract: In the quest of new materials that can withstand severe irradiation and mechanical extremes for advanced applications (e.g. fission reactors, fusion devices, space applications, etc), design, prediction and control of advanced materials beyond current material designs become a paramount goal. Here, though a combined experimental and simulation methodology, the design of a new nanocrystalline refractory high entropy alloy (RHEA) system is established. Compositions of this alloy, assessed under extreme environments and in situ electron-microscopy, revealed both high mechanical strength and thermal stability, grain refinement under heavy ion irradiation and outstanding irradiation resistance to dual-beam irradiation and helium implantation, marked by remarkable resistance to defect generation, growth and coalescence. The experimental and modeling results, which demonstrated notable agreement, can be applied to design and rapidly assess other alloys subjected to extreme environmental conditions.


title: Tuning the conductance topology in solids

abstract: The inertia of trapping and detrapping of nonequilibrium charge carriers affects the electrochemical and transport properties of both bulk and nanoscopic structures in a very peculiar way. An emerging memory response with a hysteresis in the current-voltage response and its eventual multiple crossing, produced by this universally available ingredient, are signatures of this process. Here, we deliver a microscopic and analytical solution for these behaviors, understood as the modulation of the topology of the current-voltage loops. The memory emergence becomes thus a characterization tool for intrinsic features that affect the electronic transport of solids such as the nature and number of trapping sites, intrinsic symmetry constraints, and natural relaxation time scales. This method is also able to reduce the seeming complexity of frequency-dependent electrochemical impedance and cyclic voltammetry observable for a variety of systems to a combination of simple microscopic ingredients.


title: Recurrent Convolutional Deep Neural Networks for Modeling Time-Resolved Wildfire Spread Behavior

abstract: The increasing incidence and severity of wildfires underscores the necessity of accurately predicting their behavior. While high-fidelity models derived from first principles offer physical accuracy, they are too computationally expensive for use in real-time fire response. Low-fidelity models sacrifice some physical accuracy and generalizability via the integration of empirical measurements, but enable real-time simulations for operational use in fire response. Machine learning techniques offer the ability to bridge these objectives by learning first-principles physics while achieving computational speedup. While deep learning approaches have demonstrated the ability to predict wildfire propagation over large time periods, time-resolved fire-spread predictions are needed for active fire management. In this work, we evaluate the ability of deep learning approaches in accurately modeling the time-resolved dynamics of wildfires. We use an autoregressive process in which a convolutional recurrent deep learning model makes predictions that propagate a wildfire over 15 minute increments. We demonstrate the model in application to three simulated datasets of increasing complexity, containing both field fires with homogeneous fuel distribution as well as real-world topologies sampled from the California region of the United States. We show that even after 100 autoregressive predictions representing more than 24 hours of simulated fire spread, the resulting models generate stable and realistic propagation dynamics, achieving a Jaccard score between 0.89 and 0.94 when predicting the resulting fire scar.


title: A State-Augmented Approach for Learning Optimal Resource Management Decisions in Wireless Networks

abstract: We consider a radio resource management (RRM) problem in a multi-user wireless network, where the goal is to optimize a network-wide utility function subject to constraints on the ergodic average performance of users. We propose a state-augmented parameterization for the RRM policy, where alongside the instantaneous network states, the RRM policy takes as input the set of dual variables corresponding to the constraints. We provide theoretical justification for the feasibility and near-optimality of the RRM decisions generated by the proposed state-augmented algorithm. Focusing on the power allocation problem with RRM policies parameterized by a graph neural network (GNN) and dual variables sampled from the dual descent dynamics, we numerically demonstrate that the proposed approach achieves a superior trade-off between mean, minimum, and 5th percentile rates than baseline methods.


title: When does mixup promote local linearity in learned representations?

abstract: Mixup is a regularization technique that artificially produces new samples using convex combinations of original training points. This simple technique has shown strong empirical performance, and has been heavily used as part of semi-supervised learning techniques such as mixmatch~\citep{berthelot2019mixmatch} and interpolation consistent training (ICT)~\citep{verma2019interpolation}. In this paper, we look at Mixup through a \emph{representation learning} lens in a semi-supervised learning setup. In particular, we study the role of Mixup in promoting linearity in the learned network representations. Towards this, we study two questions: (1) how does the Mixup loss that enforces linearity in the \emph{last} network layer propagate the linearity to the \emph{earlier} layers?; and (2) how does the enforcement of stronger Mixup loss on more than two data points affect the convergence of training? We empirically investigate these properties of Mixup on vision datasets such as CIFAR-10, CIFAR-100 and SVHN. Our results show that supervised Mixup training does not make \emph{all} the network layers linear; in fact the \emph{intermediate layers} become more non-linear during Mixup training compared to a network that is trained \emph{without} Mixup. However, when Mixup is used as an unsupervised loss, we observe that all the network layers become more linear resulting in faster training convergence.


title: Meta-Learning Biologically Plausible Plasticity Rules with Random Feedback Pathways

abstract: Backpropagation is widely used to train artificial neural networks, but its relationship to synaptic plasticity in the brain is unknown. Some biological models of backpropagation rely on feedback projections that are symmetric with feedforward connections, but experiments do not corroborate the existence of such symmetric backward connectivity. Random feedback alignment offers an alternative model in which errors are propagated backward through fixed, random backward connections. This approach successfully trains shallow models, but learns slowly and does not perform well with deeper models or online learning. In this study, we develop a meta-learning approach to discover interpretable, biologically plausible plasticity rules that improve online learning performance with fixed random feedback connections. The resulting plasticity rules show improved online training of deep models in the low data regime. Our results highlight the potential of meta-learning to discover effective, interpretable learning rules satisfying biological constraints.


title: Cluster Randomized Designs for One-Sided Bipartite Experiments

abstract: The conclusions of randomized controlled trials may be biased when the outcome of one unit depends on the treatment status of other units, a problem known as interference. In this work, we study interference in the setting of one-sided bipartite experiments in which the experimental units - where treatments are randomized and outcomes are measured - do not interact directly. Instead, their interactions are mediated through their connections to interference units on the other side of the graph. Examples of this type of interference are common in marketplaces and two-sided platforms. The cluster-randomized design is a popular method to mitigate interference when the graph is known, but it has not been well-studied in the one-sided bipartite experiment setting. In this work, we formalize a natural model for interference in one-sided bipartite experiments using the exposure mapping framework. We first exhibit settings under which existing cluster-randomized designs fail to properly mitigate interference under this model. We then show that minimizing the bias of the difference-in-means estimator under our model results in a balanced partitioning clustering objective with a natural interpretation. We further prove that our design is minimax optimal over the class of linear potential outcomes models with bounded interference. We conclude by providing theoretical and experimental evidence of the robustness of our design to a variety of interference graphs and potential outcomes models.


title: Highly-Excited Rydberg Excitons in Synthetic Thin-Film Cuprous Oxide

abstract: Cuprous oxide (Cu${}_2$O) has recently been proposed as a promising solid-state host for excitonic Rydberg states with large principal quantum numbers ($n$), whose exaggerated wavefunction sizes ($\propto n^2)$ facilitate gigantic dipole-dipole ($\propto n^4$) and van der Waals ($\propto n^{11}$) interactions, making them an ideal basis for solid-state quantum technology. Synthetic, thin-film Cu${}_2$O samples are of particular interest because they can be made defect-free via carefully controlled fabrication and are, in principle, suitable for the observation of extreme single-photon nonlinearities caused by the Rydberg blockade. Here, we present spectroscopic absorption and photoluminescence studies of Rydberg excitons in synthetic Cu${}_2$O grown on a transparent substrate, reporting yellow exciton series up to $n = 7$. We perform these studies at powers up to 2 mW and temperatures up to 150 K, the highest temperature where Rydberg series can be observed. These results open a new portal to scalable and integrable on-chip Rydberg-based quantum devices.


title: Similarities and differences in the sensitivity of Soil Organic Matter (SOM) dynamics to biogeochemical parameters for different vegetation inputs and climates

abstract: The biogeochemical complexity of environmental models is increasing continuously and model reliability must be reanalysed when new implementations are brought about. This work aim to identify influential biogeochemical parameters that control the Soil Organic Matter (SOM) dynamics and greenhouse gas emissions in different ecosystems and climates predicted by a physically-based mechanistic model. This explicitly accounts for four pools of organic polymers, seven pools of organic monomers, five microbial functional groups, and inorganic N and C species. We first benchmarked our model against vertical SOM profiles measured in a temperate forest in North-Eastern Bavaria, Germany (Staudt and Foken, 2007). Next, we conducted a sensitivity analysis to biogeochemical parameters using modified Morris indices for target SOM pools and gas emissions from a tropical, a temperate, and a semi-arid grassland in Australia. We found that greenhouse gas emissions, the SOM stock, and the fungi-to-bacteria ratio in the top soil were more sensitive to the mortality of aerobic bacteria than other biogeochemical parameters. The larger CO2 emission rates in forests than in grasslands were explained by a greater dissolved SOM content. Finally, we found that the soil N availability was largely controlled by vegetation inputs in forests and by atmospheric fixation in grasslands


title: First results from a multiplexed and massive instrument with sub-electron noise Skipper-CCDs

abstract: We present a new instrument composed of a large number of sub-electron noise Skipper-CCDs operated with a two stage analog multiplexed readout scheme suitable for scaling to thousands of channels. New, thick, $1.35$ Mpix sensors, from a new foundry, are glued into a Multi-Chip Module (MCM) printed circuit board on a ceramic substrate which has 16 sensors each. The instrument, that can hold up-to 16 MCMs, a total of 256 Skipper-CCD sensors (called a Super-Module with $\approx 130$ grams of active mass and $346$ Mpix), is part of the R$\&$D effort of the OSCURA experiment which will have $\approx 94$ super-modules. Experimental results with $10$ MCMs and $160$ Skipper-CCDs sensors are presented in this paper. This is already the largest ever build instrument with single electron sensitivity CCDs using nondestructive readout, both, in terms of active mass and number of channels.


title: Thermal Multi-scale Entanglement Renormalization Ansatz for Variational Gibbs State Preparation

abstract: Many simulation tasks require that one first prepare a system's Gibbs state. We present a family of quantum circuits for variational preparation of thermal Gibbs states on a quantum computer; we call them the thermal multi-scale entanglement renormalization ansatz (TMERA). TMERA circuits transform input qubits to wavepacket modes localized to varying length scales and approximate a systems Gibbs state as a mixed state of these modes. The TMERA is a based on the deep multi-scale entanglement renormalization ansatz (DMERA); a TMERA modifies a ground-state DMERA circuit by preparing each input qubit as a mixed state. The excitation probabilities for input qubits serve as variational parameters used to target particular temperature Gibbs states. Since a TMERA is a special case of the product spectrum ansatz for thermal states, it is simple to prepare, analyze, and optimize. We benchmark the TMERA on the transverse field Ising model in one dimension and find that for $D=6$ it produces global fidelities $\mathcal F > 0.4$ for 512-site systems across all temperatures.


title: Supramolecular self-assembly as a tool to preserve electronic purity of perylene diimide chromophores

abstract: Small molecule organic semiconductors hold great promise for efficient, printable, and flexible optoelectronic applications like solar cells and displays. However, strong excited-state quenching due to uncontrolled aggregation currently limits their performance and employability in devices. Here, we report on the self-assembly of a supramolecular pseudo-cube formed from six modified tetradentate perylene diimides (PDIs). The rigid, shape-persistent cage sets the distance and orientation of the PDI chromophores and suppresses intramolecular rotations and vibrations, leading to non-aggregated, monomer-like electronic properties in solution as well as in the solid state, in contrast to the fast fluorescence quenching in the free ligand. The stabilized excited state and electronic purity of the cage enable the observation of delayed fluorescence due to a bright excited multimer state, which acts as an excited state reservoir, due to a rare case of benign inter-chromophore interactions in the cage. Our results suggest that not only the photophysical properties of the subcomponents but the geometric structure is crucial for the overall optoelectronic properties of supramolecular systems. We show that self-assembly provides a powerful tool for retaining and controlling the electronic properties of well-studied chromophores, providing a route to bring molecular electronics applications in reach.


title: The Impact of Beam Variations on Power Spectrum Estimation for 21-cm Cosmology I: Simulations of Foreground Contamination for HERA

abstract: Detecting cosmological signals from the Epoch of Reionization (EoR) requires high-precision calibration to isolate the cosmological signals from foreground emission. In radio interferometery, perturbed primary beams of antenna elements can disrupt the precise calibration, which results in contaminating the foreground-free region, or the EoR window, in the cylindrically averaged power spectrum. For Hydrogen Epoch of Reionization Array (HERA), we simulate and characterize the perturbed primary beams induced by feed motions such as axial, lateral, and tilting motions, above the 14-meter dish. To understand the effect of the perturbed beams, visibility measurements are modeled with two different foreground components, point sources and diffuse sources, and we find different feed motions present a different reaction to each type of sky source. HERA's redundant-baseline calibration in the presence of non-redundant antenna beams due to feed motions introduces chromatic errors in gain solutions, which produces foreground power leakage into the EoR window. The observed leakage from vertical feed motions comes predominately from point sources around zenith. Furthermore, the observed leakage from horizontal and tilting feed motion comes predominately from the diffuse components near the horizon. Mitigation of chromatic gain errors will be necessary for robust detection of the EoR signals with minimal foreground bias, and this will be discussed in the subsequent paper.


title: Toward Unifying Text Segmentation and Long Document Summarization

abstract: Text segmentation is important for signaling a document's structure. Without segmenting a long document into topically coherent sections, it is difficult for readers to comprehend the text, let alone find important information. The problem is only exacerbated by a lack of segmentation in transcripts of audio/video recordings. In this paper, we explore the role that section segmentation plays in extractive summarization of written and spoken documents. Our approach learns robust sentence representations by performing summarization and segmentation simultaneously, which is further enhanced by an optimization-based regularizer to promote selection of diverse summary sentences. We conduct experiments on multiple datasets ranging from scientific articles to spoken transcripts to evaluate the model's performance. Our findings suggest that the model can not only achieve state-of-the-art performance on publicly available benchmarks, but demonstrate better cross-genre transferability when equipped with text segmentation. We perform a series of analyses to quantify the impact of section segmentation on summarizing written and spoken documents of substantial length and complexity.


title: Transferability-based Chain Motion Mapping from Humans to Humanoids for Teleoperation

abstract: Although data-driven motion mapping methods are promising to allow intuitive robot control and teleoperation that generate human-like robot movement, they normally require tedious pair-wise training for each specific human and robot pair. This paper proposes a transferability-based mapping scheme to allow new robot and human input systems to leverage the mapping of existing trained pairs to form a mapping transfer chain, which will reduce the number of new pair-specific mappings that need to be generated. The first part of the mapping schematic is the development of a Synergy Mapping via Dual-Autoencoder (SyDa) method. This method uses the latent features from two autoencoders to extract the common synergy of the two agents. Secondly, a transferability metric is created that approximates how well the mapping between a pair of agents will perform compared to another pair before creating the motion mapping models. Thus, it can guide the formation of an optimal mapping chain for the new human-robot pair. Experiments with human subjects and a Pepper robot demonstrated 1) The SyDa method improves the accuracy and generalizability of the pair mappings, 2) the SyDa method allows for bidirectional mapping that does not prioritize the direction of mapping motion, and 3) the transferability metric measures how compatible two agents are for accurate teleoperation. The combination of the SyDa method and transferability metric creates generalizable and accurate mapping need to create the transfer mapping chain.


title: Machine Unlearning of Federated Clusters

abstract: Federated clustering (FC) is an unsupervised learning problem that arises in a number of practical applications, including personalized recommender and healthcare systems. With the adoption of recent laws ensuring the "right to be forgotten", the problem of machine unlearning for FC methods has become of significant importance. We introduce, for the first time, the problem of machine unlearning for FC, and propose an efficient unlearning mechanism for a customized secure FC framework. Our FC framework utilizes special initialization procedures that we show are well-suited for unlearning. To protect client data privacy, we develop the secure compressed multiset aggregation (SCMA) framework that addresses sparse secure federated learning (FL) problems encountered during clustering as well as more general problems. To simultaneously facilitate low communication complexity and secret sharing protocols, we integrate Reed-Solomon encoding with special evaluation points into our SCMA pipeline, and prove that the client communication cost is logarithmic in the vector dimension. Additionally, to demonstrate the benefits of our unlearning mechanism over complete retraining, we provide a theoretical analysis for the unlearning performance of our approach. Simulation results show that the new FC framework exhibits superior clustering performance compared to previously reported FC baselines when the cluster sizes are highly imbalanced. Compared to completely retraining K-means++ locally and globally for each removal request, our unlearning procedure offers an average speed-up of roughly 84x across seven datasets. Our implementation for the proposed method is available at https://github.com/thupchnsky/mufc.


title: Feasibility study for the hard x-ray free electron laser based on synergistic use of conventional and plasma accelerator technologies

abstract: We access the possibility of using the conventional RF accelerator as an injector for the plasma driven wakefield accelerator. Conventional accelerators deliver high quality beams with low emittance and low energy spread. Once injected into the plasma wake, the emittance may be preserved upon proper beam matching while the energy spread may not due to long beam duration delivered by the conventional accelerators. Parameters of the overall accelerator system and the free electron laser based on such an accelerator are estimated.


title: Distinguishing some genus one knots using finite quotients

abstract: We give a criterion for distinguishing a prime knot $K$ in $S^3$ from every other knot in $S^3$ using the finite quotients of $\pi_1(S^3\setminus K)$. Using recent work of Baldwin-Sivek, we apply this criterion to the hyperbolic knots $5_2$, $15n_{43522}$, and the three-strand pretzel knots $P(-3,3,2n+1)$ for every integer $n$.


title: Semi-Quantum Random Number Generation

abstract: Semi-quantum cryptography involves at least one user who is semi-quantum or "classical" in nature. Such a user can only interact with the quantum channel in a very restricted way. Many semi-quantum key distribution protocols have been developed, some with rigorous proofs of security. Here we show for the first time, to our knowledge, that quantum random number generation is possible in the semi-quantum setting. We also develop a rigorous proof of security, deriving a bound on the random bit generation rate of the protocol as a function of noise in the channel. Our protocol and proof may be broadly applicable to other quantum and semi-quantum cryptographic scenarios where users are limited in their capabilities.


title: Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention

abstract: Audio captioning aims to generate text descriptions of audio clips. In the real world, many objects produce similar sounds. How to accurately recognize ambiguous sounds is a major challenge for audio captioning. In this work, inspired by inherent human multimodal perception, we propose visually-aware audio captioning, which makes use of visual information to help the description of ambiguous sounding objects. Specifically, we introduce an off-the-shelf visual encoder to extract video features and incorporate the visual features into an audio captioning system. Furthermore, to better exploit complementary audio-visual contexts, we propose an audio-visual attention mechanism that adaptively integrates audio and visual context and removes the redundant information in the latent space. Experimental results on AudioCaps, the largest audio captioning dataset, show that our proposed method achieves state-of-the-art results on machine translation metrics.


title: Components and Exit Times of Brownian Motion in two or more $p$-Adic Dimensions

abstract: The fundamental solution of a pseudo-differential equation for functions defined on the $d$-fold product of the $p$-adic numbers, $\mathbb{Q}_p$, induces an analogue of the Wiener process in $\mathbb{Q}_p^d$. As in the real setting, the components are $1$-dimensional $p$-adic Brownian motions with the same diffusion constant and exponent as the original process. Asymptotic analysis of the conditional probabilities shows that the vector components are dependent for all time. Exit time probabilities for the higher dimensional processes reveal a concrete effect of the component dependency.


title: Second Order Finite Volume Scheme for Shallow Water Equations on Manifolds

abstract: In this work, we propose a second-order accurate scheme for shallow water equations in general covariant coordinates over manifolds. In particular, the covariant parametrization in general covariant coordinates is induced by the metric tensor associated to the manifold. The model is then re-written in a hyperbolic form with a tuple of conserved variables composed both of the evolving physical quantities and the metric coefficients. This formulation allows the numerical scheme to i) automatically compute the curvature of the manifold as long as the physical variables are evolved and ii) numerically study complex physical domains over simple computational domains.


title: DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attention

abstract: Vision-and-language (V-L) tasks require the system to understand both vision content and natural language, thus learning fine-grained joint representations of vision and language (a.k.a. V-L representations) is of paramount importance. Recently, various pre-trained V-L models are proposed to learn V-L representations and achieve improved results in many tasks. However, the mainstream models process both vision and language inputs with the same set of attention matrices. As a result, the generated V-L representations are entangled in one common latent space. To tackle this problem, we propose DiMBERT (short for Disentangled Multimodal-Attention BERT), which is a novel framework that applies separated attention spaces for vision and language, and the representations of multi-modalities can thus be disentangled explicitly. To enhance the correlation between vision and language in disentangled spaces, we introduce the visual concepts to DiMBERT which represent visual information in textual format. In this manner, visual concepts help to bridge the gap between the two modalities. We pre-train DiMBERT on a large amount of image-sentence pairs on two tasks: bidirectional language modeling and sequence-to-sequence language modeling. After pre-train, DiMBERT is further fine-tuned for the downstream tasks. Experiments show that DiMBERT sets new state-of-the-art performance on three tasks (over four datasets), including both generation tasks (image captioning and visual storytelling) and classification tasks (referring expressions). The proposed DiM (short for Disentangled Multimodal-Attention) module can be easily incorporated into existing pre-trained V-L models to boost their performance, up to a 5% increase on the representative task. Finally, we conduct a systematic analysis and demonstrate the effectiveness of our DiM and the introduced visual concepts.


title: Uncertainty Quantification of Nonlinear Lagrangian Data Assimilation Using Linear Stochastic Forecast Models

abstract: Lagrangian data assimilation exploits the trajectories of moving tracers as observations to recover the underlying flow field. One major challenge in Lagrangian data assimilation is the intrinsic nonlinearity that impedes using exact Bayesian formulae for the state estimation of high-dimensional systems. In this paper, an analytically tractable mathematical framework for continuous-in-time Lagrangian data assimilation is developed. It preserves the nonlinearity in the observational processes while approximating the forecast model of the underlying flow field using linear stochastic models (LSMs). A critical feature of the framework is that closed analytic formulae are available for solving the posterior distribution, which facilitates mathematical analysis and numerical simulations. First, an efficient iterative algorithm is developed in light of the analytically tractable statistics. It accurately estimates the parameters in the LSMs using only a small number of the observed tracer trajectories. Next, the framework facilitates the development of several computationally efficient approximate filters and the quantification of the associated uncertainties. A cheap approximate filter with a diagonal posterior covariance derived from the asymptotic analysis of the posterior estimate is shown to be skillful in recovering incompressible flows. It is also demonstrated that randomly selecting a small number of tracers at each time step as observations can reduce the computational cost while retaining the data assimilation accuracy. Finally, based on a prototype model in geophysics, the framework with LSMs is shown to be skillful in filtering nonlinear turbulent flow fields with strong non-Gaussian features.


title: Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models

abstract: Fully-parametric language models generally require a huge number of model parameters to store the necessary knowledge for solving multiple natural language tasks in zero/few-shot settings. In addition, it is hard to adapt to the evolving world knowledge without the costly model re-training. In this paper, we develop a novel semi-parametric language model architecture, Knowledge-in-Context (KiC), which empowers a parametric text-to-text language model with a knowledge-rich external memory. Specifically, the external memory contains six different types of knowledge: entity, dictionary, commonsense, event, script, and causality knowledge. For each input instance, the KiC model adaptively selects a knowledge type and retrieves the most helpful pieces of knowledge. The input instance along with its knowledge augmentation is fed into a text-to-text model (e.g., T5) to generate the output answer, where both the input and the output are in natural language forms after prompting. Interestingly, we find that KiC can be identified as a special mixture-of-experts (MoE) model, where the knowledge selector plays the role of a router that is used to determine the sequence-to-expert assignment in MoE. This key observation inspires us to develop a novel algorithm for training KiC with an instance-adaptive knowledge selector. As a knowledge-rich semi-parametric language model, KiC only needs a much smaller parametric part to achieve superior zero-shot performance on unseen tasks. By evaluating on 40+ different tasks, we show that KiC_Large with 770M parameters easily outperforms large language models (LMs) that are 4-39x larger by a large margin. We also demonstrate that KiC exhibits emergent abilities at a much smaller model scale compared to the fully-parametric models.