title: The $\ell^p$ norm of the Riesz--Titchmarsh transform for even integer $p$

abstract: The long-standing conjecture that for $p \in (1, \infty)$ the $\ell^p(\mathbb Z)$ norm of the Riesz--Titchmarsh discrete Hilbert transform is the same as the $L^p(\mathbb R)$ norm of the classical Hilbert transform, is verified when $p = 2 n$ or $\frac{p}{p - 1} = 2 n$, for $n \in \mathbb N$. The proof, which is algebraic in nature, depends in a crucial way on the sharp estimate for the $\ell^p(\mathbb Z)$ norm of a different variant of this operator for the full range of $p$. The latter result was recently proved by the authors in [Ba\~nuelos, Kwa\'snicki, On the $\ell^p$-norm of the discrete Hilbert transform, Duke Math. J. 168(3) (2019): 471-504].


title: High Resolution HST Imaging Survey of Local Star-Forming Galaxies I: Spatially-Resolved Obscured Star Formation with H$\alpha$ and Paschen-$\beta$ Recombination Lines

abstract: We present a sample of 24 local star-forming galaxies observed with broad- and narrow-band photometry from the Hubble Space Telescope, that are part of the GOALS survey of local luminous and ultra-luminous infrared galaxies. With narrow-band filters around the emission lines H$\alpha$ (and [NII]) and Pa$\beta$, we obtain robust estimates of the dust attenuation affecting the gas in each galaxy, probing higher attenuation than can be traced by the optical Balmer decrement H$\alpha$/H$\beta$ alone by a factor of $>1$ mag. We also infer the dust attenuation towards the stars via a spatially-resolved SED-fitting procedure that uses all available HST imaging filters. We use various indicators to obtain the star formation rate (SFR) per spatial bin, and find that Pa$\beta$ traces star-forming regions where the H$\alpha$ and the optical stellar continuum are heavily obscured. The dust-corrected Pa$\beta$ SFR recovers the 24$\mu$m-inferred SFR with a ratio $-0.14\pm0.32$ dex and the SFR inferred from the $8\mathrm{-}1000\,\mu\mathrm{m}$ infrared luminosity at $-0.04\pm0.23$ dex. Both in a spatially-resolved and integrated sense, rest-frame near infrared recombination lines can paint a more comprehensive picture of star formation across cosmic time, particularly with upcoming JWST observations of Paschen-series line emission in galaxies as early as the epoch of reionization.


title: Defining a credible interval is not always possible with "point-null'' priors: A lesser-known correlate of the Jeffreys-Lindley paradox

abstract: In many common situations, a Bayesian credible interval will be, given the same data, very similar to a frequentist confidence interval, and researchers will interpret these intervals in a similar fashion. However, no predictable similarity exists when credible intervals are based on model-averaged posteriors whenever one of the two nested models under consideration is a so called ''point-null''. Not only can this model-averaged credible interval be quite different than the frequentist confidence interval, in some cases it may be undefined. This is a lesser-known correlate of the Jeffreys-Lindley paradox and is of particular interest given the popularity of the Bayes factor for testing point-null hypotheses.


title: VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training

abstract: Reward and representation learning are two long-standing challenges for learning an expanding set of robot manipulation skills from sensory observations. Given the inherent cost and scarcity of in-domain, task-specific robot data, learning from large, diverse, offline human videos has emerged as a promising path towards acquiring a generally useful visual representation for control; however, how these human videos can be used for general-purpose reward learning remains an open question. We introduce $\textbf{V}$alue-$\textbf{I}$mplicit $\textbf{P}$re-training (VIP), a self-supervised pre-trained visual representation capable of generating dense and smooth reward functions for unseen robotic tasks. VIP casts representation learning from human videos as an offline goal-conditioned reinforcement learning problem and derives a self-supervised dual goal-conditioned value-function objective that does not depend on actions, enabling pre-training on unlabeled human videos. Theoretically, VIP can be understood as a novel implicit time contrastive objective that generates a temporally smooth embedding, enabling the value function to be implicitly defined via the embedding distance, which can then be used to construct the reward for any goal-image specified downstream task. Trained on large-scale Ego4D human videos and without any fine-tuning on in-domain, task-specific data, VIP's frozen representation can provide dense visual reward for an extensive set of simulated and $\textbf{real-robot}$ tasks, enabling diverse reward-based visual control methods and significantly outperforming all prior pre-trained representations. Notably, VIP can enable simple, $\textbf{few-shot}$ offline RL on a suite of real-world robot tasks with as few as 20 trajectories.


title: Dynamical consistency conditions for rapid turn inflation

abstract: We derive consistency conditions for sustained slow roll and rapid turn inflation in two-field cosmological models with oriented scalar field space, which imply that inflationary models with field-space trajectories of this type are non-generic. In particular, we show that third order adiabatic slow roll, together with large and slowly varying turn rate, requires the scalar potential of the model to satisfy a certain nonlinear second order PDE, whose coefficients depend on the scalar field metric. We also derive consistency conditions for slow roll inflationary solutions in the so called ``rapid turn attractor'' approximation, as well as study the consistency conditions for circular rapid turn trajectories with slow roll in two-field models with rotationally invariant field space metric. Finally, we argue that the rapid turn regime tends to have a natural exit after a limited number of e-folds.


title: Direct Embedding of Temporal Network Edges via Time-Decayed Line Graphs

abstract: Temporal networks model a variety of important phenomena involving timed interactions between entities. Existing methods for machine learning on temporal networks generally exhibit at least one of two limitations. First, time is assumed to be discretized, so if the time data is continuous, the user must determine the discretization and discard precise time information. Second, edge representations can only be calculated indirectly from the nodes, which may be suboptimal for tasks like edge classification. We present a simple method that avoids both shortcomings: construct the line graph of the network, which includes a node for each interaction, and weigh the edges of this graph based on the difference in time between interactions. From this derived graph, edge representations for the original network can be computed with efficient classical methods. The simplicity of this approach facilitates explicit theoretical analysis: we can constructively show the effectiveness of our method's representations for a natural synthetic model of temporal networks. Empirical results on real-world networks demonstrate our method's efficacy and efficiency on both edge classification and temporal link prediction.


title: Projectivity and effective global generation of determinantal line bundles on quiver moduli

abstract: We give a moduli-theoretic treatment of the existence and properties of moduli spaces of semistable quiver representations, avoiding methods from geometric invariant theory. Using the existence criteria of Alper--Halpern-Leistner--Heinloth, we show that for many stability functions, the stack of semistable representations admits an adequate moduli space, and prove that this moduli space is proper over the moduli space of semisimple representations. We construct a natural determinantal line bundle that descends to a semiample line bundle on the moduli space and provide new effective bounds for global generation. For an acyclic quiver, we show that this line bundle is ample, thus giving a modern proof of the fact that the moduli space is projective.


title: Offset-value coding in database query processing

abstract: Recent work shows how offset-value coding speeds up database query execution, not only sorting but also duplicate removal and grouping (aggregation) in sorted streams, order-preserving exchange (shuffle), merge join, and more. It already saves thousands of CPUs in Google's Napa and F1 Query systems, e.g., in grouping algorithms and in log-structured merge-forests. In order to realize the full benefit of interesting orderings, however, query execution algorithms must not only consume and exploit offset-value codes but also produce offset-value codes for the next operator in the pipeline. Our research has sought ways to produce offset-value codes without comparing successive output rows one-by-one, column-by-column. This short paper introduces a new theorem and, based on its proof and a simple corollary, describes in detail how order-preserving algorithms (from filter to merge join and even shuffle) can compute offset-value codes for their outputs. These computations are surprisingly simple and very efficient.


title: Neural Causal Models for Counterfactual Identification and Estimation

abstract: Evaluating hypothetical statements about how the world would be had a different course of action been taken is arguably one key capability expected from modern AI systems. Counterfactual reasoning underpins discussions in fairness, the determination of blame and responsibility, credit assignment, and regret. In this paper, we study the evaluation of counterfactual statements through neural models. Specifically, we tackle two causal problems required to make such evaluations, i.e., counterfactual identification and estimation from an arbitrary combination of observational and experimental data. First, we show that neural causal models (NCMs) are expressive enough and encode the structural constraints necessary for performing counterfactual reasoning. Second, we develop an algorithm for simultaneously identifying and estimating counterfactual distributions. We show that this algorithm is sound and complete for deciding counterfactual identification in general settings. Third, considering the practical implications of these results, we introduce a new strategy for modeling NCMs using generative adversarial networks. Simulations corroborate with the proposed methodology.


title: Differentially Private Bias-Term Fine-tuning of Foundation Models

abstract: We study the problem of differentially private (DP) fine-tuning of large pre-trained models -- a recent privacy-preserving approach suitable for solving downstream tasks with sensitive data. Existing work has demonstrated that high accuracy is possible under strong privacy constraint, yet requires significant computational overhead or modifications to the network architecture. We propose differentially private bias-term fine-tuning (DP-BiTFiT), which matches the state-of-the-art accuracy for DP algorithms and the efficiency of the standard BiTFiT. DP-BiTFiT is model agnostic (not modifying the network architecture), parameter efficient (only training about 0.1% of the parameters), and computation efficient (almost removing the overhead caused by DP, in both the time and space complexity). On a wide range of tasks, DP-BiTFiT is 2~30X faster and uses 2~8X less memory than DP full fine-tuning, even faster than the standard full fine-tuning. This amazing efficiency enables us to conduct DP fine-tuning on language and vision tasks with long-sequence texts and high-resolution images, which were computationally difficult using existing methods. We open-source our code at FastDP (https://github.com/awslabs/fast-differential-privacy).


title: A Complete Set of Connectivity-aware Local Topology Manipulation Operations for Robot Swarms

abstract: The topology of a robotic swarm affects the convergence speed of consensus and the mobility of the robots. In this paper, we prove the existence of a complete set of local topology manipulation operations that allow the transformation of a swarm topology. The set is complete in the sense that any other possible set of manipulation operations can be performed by a sequence of operations from our set. The operations are local as they depend only on the first and second hop neighbors' information to transform any initial spanning tree of the network's graph to any other connected tree with the same number of nodes. The flexibility provided by our method is similar to global methods that require full knowledge of the swarm network. We prove the existence of a sequence of transformations for any tree-to-tree transformation, and derive sequences of operations to form a line or star from any initial spanning tree. Our work provides a theoretical and practical framework for topological control of a swarm, establishing global properties using only local information.


title: Differentially Private Optimization on Large Model at Small Cost

abstract: Differentially private (DP) optimization is the standard paradigm to learn large neural networks that are accurate and privacy-preserving. The computational cost for DP deep learning, however, is notoriously heavy due to the per-sample gradient clipping. Existing DP implementations are 2-1000X more costly in time and space complexity than the standard (non-private) training. In this work, we develop a novel Book-Keeping (BK) technique that implements existing DP optimizers (thus achieving the same accuracy), with a substantial improvement on the computational cost. Specifically, BK enables DP training on large models and high dimensional data to be roughly as fast and memory-saving as the standard training, whereas previous DP algorithms can be inefficient or incapable of training due to memory error. The computational advantage of BK is supported by the complexity analysis as well as extensive experiments on vision and language tasks. Our implementation achieves state-of-the-art (SOTA) accuracy with very small extra cost: on GPT2 and at almost the same memory cost (<1% overhead), BK has 1.03X the time complexity of the standard training (0.83X training speed in practice), and 0.61X the time complexity of the most efficient DP implementation (1.36X training speed in practice). We open-source the codebase for the BK algorithm at the FastDP library (https://github.com/awslabs/fast-differential-privacy).


title: a characterization of the centers of chordal graphs

abstract: A graph is $k$-chordal if it does not have an induced cycle with length greater than $k$. We call a graph chordal if it is $3$-chordal. Let $G$ be a graph. The distance between the vertices $x$ and $y$, denoted by $d_{G}(x,y)$, is the length of a shortest path from $x$ to $y$ in $G$. The eccentricity of a vertex $x$ is defined as $\epsilon_{G}(x)= \max\{d_{G}(x,y)|y\in V(G)\}$. The radius of $G$ is defined as $Rad(G)=\min\{\epsilon_{G}(x)|x\in V(G)\}$. The diameter of $G$ is defined as $Diam(G)=\max\{\epsilon_{G}(x)|x\in V(G)\}$. The graph induced by the set of vertices of $G$ with eccentricity equal to the radius is called the center of $G$. In this paper we present new bounds for the diameter of $k$-chordal graphs, and we give a concise characterization of the centers of chordal graphs.


title: Utility of the Koopman operator in output regulation of disturbed nonlinear systems

abstract: This paper studies the problem of output regulation for a class of nonlinear systems experiencing matched input disturbances. It is assumed that the disturbance signal is generated by an external autonomous dynamical system. First, we show that for a class of nonlinear systems admitting a finite-dimensional Koopman representation, the problem is equivalent to a bilinear output regulation. We then prove that a linear dynamic output feedback controller, inspired by the linear output regulation framework, locally solves the original nonlinear problem. Numerical results validate our analysis.


title: Communication-Enabled Deep Reinforcement Learning to Optimise Energy-Efficiency in UAV-Assisted Networks

abstract: Unmanned aerial vehicles (UAVs) are increasingly deployed to provide wireless connectivity to static and mobile ground users in situations of increased network demand or points of failure in existing terrestrial cellular infrastructure. However, UAVs are energy-constrained and experience the challenge of interference from nearby UAV cells sharing the same frequency spectrum, thereby impacting the system's energy efficiency (EE). Recent approaches focus on optimising the system's EE by optimising the trajectory of UAVs serving only static ground users and neglecting mobile users. Several others neglect the impact of interference from nearby UAV cells, assuming an interference-free network environment. Despite growing research interest in decentralised control over centralised UAVs' control, direct collaboration among UAVs to improve coordination while optimising the systems' EE has not been adequately explored. To address this, we propose a direct collaborative communication-enabled multi-agent decentralised double deep Q-network (CMAD-DDQN) approach. The CMAD-DDQN is a collaborative algorithm that allows UAVs to explicitly share their telemetry via existing 3GPP guidelines by communicating with their nearest neighbours. This allows the agent-controlled UAVs to optimise their 3D flight trajectories by filling up knowledge gaps and converging to optimal policies. Simulation results show that the proposed approach outperforms existing baselines in terms of maximising the systems' EE without degrading coverage performance in the network. The CMAD-DDQN approach outperforms the MAD-DDQN that neglects direct collaboration among UAVs, the multi-agent deep deterministic policy gradient (MADDPG) and random policy approaches that consider a 2D UAV deployment design while neglecting interference from nearby UAV cells by about 15%, 65% and 85%, respectively.


title: Direct Estimation of Porosity from Seismic Data using Rock and Wave Physics Informed Neural Networks (RW-PINN)

abstract: Petrophysical inversion is an important aspect of reservoir modeling. However due to the lack of a unique and straightforward relationship between seismic traces and rock properties, predicting petrophysical properties directly from seismic data is a complex task. Many studies have attempted to identify the direct end-to-end link using supervised machine learning techniques, but face different challenges such as a lack of large petrophysical training dataset or estimates that may not conform with physics or depositional history of the rocks. We present a rock and wave physics informed neural network (RW-PINN) model that can estimate porosity directly from seismic image traces with no or limited number of wells, with predictions that are consistent with rock physics and geologic knowledge of deposition. As an example, we use the uncemented sand rock physics model and normal-incidence wave physics to guide the learning of RW-PINN to eventually get good estimates of porosities from normal-incidence seismic traces and limited well data. Training RW-PINN with few wells (weakly supervised) helps in tackling the problem of non-uniqueness as different porosity logs can give similar seismic traces. We use weighted normalized root mean square error loss function to train the weakly supervised network and demonstrate the impact of different weights on porosity predictions. The RW-PINN estimated porosities and seismic traces are compared to predictions from a completely supervised model, which gives slightly better porosity estimates but poorly matches the seismic traces, in addition to requiring a large amount of labeled training data. In this paper, we demonstrate the complete workflow for executing petrophysical inversion of seismic data using self-supervised or weakly supervised rock physics informed neural networks.


title: Search for new heavy resonances decaying to WW, WZ, ZZ, WH, or ZH boson pairs in the all-jets final state in proton-proton collisions at $\sqrt{s}$ = 13 TeV

abstract: A search for new heavy resonances decaying to WW, WZ, ZZ, WH, or ZH boson pairs in the all-jets final state is presented. The analysis is based on proton-proton collision data recorded by the CMS detector in 2016-2018 at a centre-of-mass energy of 13 TeV at the CERN LHC, corresponding to an integrated luminosity of 138 fb$^{-1}$. The search is sensitive to resonances with masses between 1.3 and 6 TeV, decaying to bosons that are highly Lorentz-boosted such that each of the bosons forms a single large-radius jet. Machine learning techniques are employed to identify such jets. No significant excess over the estimated standard model background is observed. A maximum local significance of 3.6 standard deviations, corresponding to a global significance of 2.3 standard deviations, is observed at masses of 2.1 and 2.9 TeV. In a heavy vector triplet model, spin-1 Z' and W' resonances with masses below 4.8 TeV are excluded at the 95% confidence level (CL). These limits are the most stringent to date. In a bulk graviton model, spin-2 gravitons and spin-0 radions with masses below 1.4 and 2.7 TeV, respectively, are excluded at 95% CL. Production of heavy resonances through vector boson fusion is constrained with upper cross section limits at 95% CL as low as 0.1 fb.


title: Task Formulation Matters When Learning Continually: A Case Study in Visual Question Answering

abstract: Continual learning aims to train a model incrementally on a sequence of tasks without forgetting previous knowledge. Although continual learning has been widely studied in computer vision, its application to Vision+Language tasks is not that straightforward, as settings can be parameterized in multiple ways according to their input modalities. In this paper, we present a detailed study of how different settings affect performance for Visual Question Answering. We first propose three plausible task formulations and demonstrate their impact on the performance of continual learning algorithms. We break down several factors of task similarity, showing that performance and sensitivity to task order highly depend on the shift of the output distribution. We also investigate the potential of pretrained models and compare the robustness of transformer models with different visual embeddings. Finally, we provide an analysis interpreting model representations and their impact on forgetting. Our results highlight the importance of stabilizing visual representations in deeper layers.


title: Calibrating Sequence likelihood Improves Conditional Language Generation

abstract: Conditional language models are predominantly trained with maximum likelihood estimation (MLE), giving probability mass to sparsely observed target sequences. While MLE trained models assign high probability to plausible sequences given the context, the model probabilities often do not accurately rank-order generated sequences by quality. This has been empirically observed in beam search decoding as output quality degrading with large beam sizes, and decoding strategies benefiting from heuristics such as length normalization and repetition-blocking. In this work, we introduce sequence likelihood calibration (SLiC) where the likelihood of model generated sequences are calibrated to better align with reference sequences in the model's latent space. With SLiC, decoding heuristics become unnecessary and decoding candidates' quality significantly improves regardless of the decoding method. Furthermore, SLiC shows no sign of diminishing returns with model scale, and presents alternative ways to improve quality with limited training and inference budgets. With SLiC, we exceed or match SOTA results on a wide range of generation tasks spanning abstractive summarization, question generation, abstractive question answering and data-to-text generation, even with modest-sized models.


title: Impact of the tangential traction for radial hydraulic fracture

abstract: The radial (penny-shaped) model of hydraulic fracture is considered. The tangential traction on the fracture walls is incorporated, including an updated evaluation of the energy release rate (fracture criterion), system asymptotics and the need to account for stagnant zone formation near the injection point. The impact of incorporating the shear stress on the construction of solvers, and the effectiveness of approximating system parameters using the first term of the crack tip asymptotics, is discussed. A full quantitative investigation of the impact of tangential traction on solution is undertaken, utilizing an extremely effective (in-house build) adaptive time-space solver.


title: The $D^6 R^4$ interaction as a Poincar\'e series, and a related shifted convolution sum

abstract: We complete the program, initiated in a 2015 paper of Green, Miller, and Vanhove, of directly constructing the automorphic solution to the string theory $D^6 R^4$ differential equation $(\Delta-12)f=-E_{3/2}^2$ for $SL(2,\mathbb{Z})$. The construction is via a type of Poincar\'e series, and requires explicitly evaluating a particular double integral. We also show how to derive the predicted vanishing of one type of term appearing in $f$'s Fourier expansion, confirming a conjecture made by Chester, Green, Pufu, Wang, and Wen motivated by Yang-Mills theory.


title: Axioms for Constant Function Market Makers

abstract: We study axiomatic foundations for different classes of constant-function automated market makers (CFMMs). We focus particularly on separability and on different invariance properties under scaling. Our main results are an axiomatic characterization of a natural generalization of constant product market makers (CPMMs), popular in decentralized finance, on the one hand, and a characterization of the Logarithmic Scoring Rule Market Makers (LMSR), popular in prediction markets, on the other hand. The first class is characterized by the combination of independence and scale invariance, whereas the second is characterized by the combination of independence and translation invariance. The two classes are therefore distinguished by a different invariance property that is motivated by different interpretations of the num\'eraire in the two applications. However, both are pinned down by the same separability property. Moreover, we characterize the CPMM as an extremal point within the class of scale invariant, independent, symmetric AMMs with non-concentrated liquidity provision. Our results add to a formal analysis of mechanisms that are currently used for decentralized exchanges and connect the most popular class of DeFi AMMs to the most popular class of prediction market AMMs.


title: Bayes factor functions for reporting outcomes of hypothesis tests

abstract: Bayes factors represent the ratio of probabilities assigned to data by competing scientific hypotheses. Drawbacks of Bayes factors are their dependence on prior specifications that define null and alternative hypotheses and difficulties encountered in their computation. To address these problems, we define Bayes factor functions (BFF) directly from common test statistics. BFFs depend on a single non-centrality parameter that can be expressed as a function of standardized effect sizes, and plots of BFFs versus effect size provide informative summaries of hypothesis tests that can be easily aggregated across studies. Such summaries eliminate the need for arbitrary P-value thresholds to define ``statistical significance.'' BFFs are available in closed form and can be computed easily from z, t, chi-squared, and F statistics.


title: Distributionally Robust Covariance Steering with Optimal Risk Allocation

abstract: This article extends the optimal covariance steering (CS) problem for discrete time linear stochastic systems modeled using moment-based ambiguity sets. To hedge against the uncertainty in the state distributions while performing covariance steering, distributionally robust risk constraints are employed during the optimal allocation of the risk. Specifically, a distributionally robust iterative risk allocation (DR-IRA) formalism is used to solve the optimal risk allocation problem for the CS problem using a two-stage approach. The upper-stage of DR-IRA is a convex problem that optimizes the risk, while the lower-stage optimizes the controller with the new distributionally robust risk constraints. The proposed framework results in solutions that are robust against arbitrary distributions in the considered ambiguity set. Finally, we demonstrate our proposed approach using numerical simulations. Addressing the covariance steering problem through the lens of distributional robustness marks the novel contribution of this article.


title: Force/Torque Sensing for Soft Grippers using an External Camera

abstract: Robotic manipulation can benefit from wrist-mounted force/torque (F/T) sensors, but conventional F/T sensors can be expensive, difficult to install, and damaged by high loads. We present Visual Force/Torque Sensing (VFTS), a method that visually estimates the 6-axis F/T measurement that would be reported by a conventional F/T sensor. In contrast to approaches that sense loads using internal cameras placed behind soft exterior surfaces, our approach uses an external camera with a fisheye lens that observes a soft gripper. VFTS includes a deep learning model that takes a single RGB image as input and outputs a 6-axis F/T estimate. We trained the model with sensor data collected while teleoperating a robot (Stretch RE1 from Hello Robot Inc.) to perform manipulation tasks. VFTS outperformed F/T estimates based on motor currents, generalized to a novel home environment, and supported three autonomous tasks relevant to healthcare: grasping a blanket, pulling a blanket over a manikin, and cleaning a manikin's limbs. VFTS also performed well with a manually operated pneumatic gripper. Overall, our results suggest that an external camera observing a soft gripper can perform useful visual force/torque sensing for a variety of manipulation tasks.


title: New examples of Anosov flows on higher dimensional manifolds which fibre over $3$-dimensional Anosov flows

abstract: We construct non-algebraic Anosov flows in dimension $3+2n$, $n\geq 2$, by suspending the action of the fundamental group of a finite cover of the Bonatti-Langevin flow.


title: Kernel Normalized Convolutional Networks for Privacy-Preserving Machine Learning

abstract: Normalization is an important but understudied challenge in privacy-related application domains such as federated learning (FL), differential privacy (DP), and differentially private federated learning (DP-FL). While the unsuitability of batch normalization for these domains has already been shown, the impact of other normalization methods on the performance of federated or differentially private models is not well-known. To address this, we draw a performance comparison among layer normalization (LayerNorm), group normalization (GroupNorm), and the recently proposed kernel normalization (KernelNorm) in FL, DP, and DP-FL settings. Our results indicate LayerNorm and GroupNorm provide no performance gain compared to the baseline (i.e. no normalization) for shallow models in FL and DP. They, on the other hand, considerably enhance the performance of shallow models in DP-FL and deeper models in FL and DP. KernelNorm, moreover, significantly outperforms its competitors in terms of accuracy and convergence rate (or communication efficiency) for both shallow and deeper models in all considered learning environments. Given these key observations, we propose a kernel normalized ResNet architecture called KNResNet-13 for differentially private learning. Using the proposed architecture, we provide new state-of-the-art accuracy values on the CIFAR-10 and Imagenette datasets, when trained from scratch.


title: Volatility density estimation by multiplicative deconvolution

abstract: We study the non-parametric estimation of an unknown stationary density fV of an unobserved strictly stationary volatility process $(\bm V_t)_{t\geq 0}$ on $\IRp^2 := (0,\infty)^2$ based on discrete-time observations in a stochastic volatility model. We identify the underlying multiplicative measurement error model and build an estimator based on the estimation of the Mellin transform of the scaled, integrated volatility process and a spectral cut-off regularisation of the inverse of the Mellin transform. We prove that the proposed estimator leads to a consistent estimation strategy. A fully data-driven choice of $\bm k \in \IRp^2$ is proposed and upper bounds for the mean integrated squared risk are provided. Throughout our study, regularity properties of the volatility process are necessary for the analsysis of the estimator. These assumptions are fulfilled by several examples of volatility processes which are listed and used in a simulation study to illustrate a reasonable behaviour of the proposed estimator.


title: MaskTune: Mitigating Spurious Correlations by Forcing to Explore

abstract: A fundamental challenge of over-parameterized deep learning models is learning meaningful data representations that yield good performance on a downstream task without over-fitting spurious input features. This work proposes MaskTune, a masking strategy that prevents over-reliance on spurious (or a limited number of) features. MaskTune forces the trained model to explore new features during a single epoch finetuning by masking previously discovered features. MaskTune, unlike earlier approaches for mitigating shortcut learning, does not require any supervision, such as annotating spurious features or labels for subgroup samples in a dataset. Our empirical results on biased MNIST, CelebA, Waterbirds, and ImagenNet-9L datasets show that MaskTune is effective on tasks that often suffer from the existence of spurious correlations. Finally, we show that MaskTune outperforms or achieves similar performance to the competing methods when applied to the selective classification (classification with rejection option) task. Code for MaskTune is available at https://github.com/aliasgharkhani/Masktune.


title: Solvability of time-varying infinite-dimensional linear port-Hamiltonian systems

abstract: Thirty years after the introduction of port-Hamiltonian systems, interest in this system class still remains high among systems and control researchers. Very recently, Jacob and Laasri obtained strong results on the solvability and well-posedness of time-varying linear port-Hamil\-to\-nian systems with boundary control and boundary observation. In this paper, we complement their results by discussing the solvability of linear, infinite-dimensional time-varying port-Hamiltonian systems not necessarily of boundary control type. The theory is illustrated on a system with a delay component in the state dynamics.


title: Paraconsistent and Paracomplete Zermelo-Fraenkel Set Theory

abstract: We present a novel treatment of set theory in a four-valued paraconsistent and paracomplete logic, i.e., a logic in which propositions can be both true and false, and neither true nor false. Our approach is a significant departure from previous research in paraconsistent set theory, which has almost exclusively been motivated by a desire to avoid Russell's paradox and fulfil naive comprehension. Instead, we prioritise setting up a system with a clear ontology of non-classical sets, which can be used to reason informally about incomplete and inconsistent phenomena, and is sufficiently similar to ZFC to enable the development of interesting mathematics. We propose an axiomatic system BZFC, obtained by analysing the ZFC-axioms and translating them to a four-valued setting in a careful manner, avoiding many of the obstacles encountered by other attempted formalizations. We introduce the anti-classicality axiom postulating the existence of non-classical sets, and prove a surprising results stating that the existence of a single non-classical set is sufficient to produce any other type of non-classical set. Our theory is naturally bi-interpretable with ZFC, and provides a philosophically satisfying view in which non-classical sets can be seen as a natural extension of classical ones, in a similar way to the non-well-founded sets of Peter Aczel. Finally, we provide an interesting application concerning Tarski semantics, showing that the classical definition of the satisfaction relation yields a logic precisely reflecting the non-classicality in the meta-theory.


title: Hardware Trojan Threats to Cache Coherence in Modern 2.5D Chiplet Systems

abstract: As industry moves toward chiplet-based designs, the insertion of hardware Trojans poses a significant threat to the security of these systems. These systems rely heavily on cache coherence for coherent data communication, making coherence an attractive target. Critically, unlike prior work, which focuses only on malicious packet modifications, a Trojan attack that exploits coherence can modify data in memory that was never touched and is not owned by the chiplet which contains the Trojan. Further, the Trojan need not even be physically between the victim and the memory controller to attack the victim's memory transactions. Here, we explore the fundamental attack vectors possible in chiplet-based systems and provide an example Trojan implementation capable of directly modifying victim data in memory. This work aims to highlight the need for developing mechanisms that can protect and secure the coherence scheme from these forms of attacks.


title: Nonlinear automorphism of the conformal algebra in 2D and continuous $\sqrt{T\bar{T}}$ deformations

abstract: The conformal algebra in 2D (Diff($S^{1}$)$\oplus$Diff($S^{1}$)) is shown to be preserved under a nonlinear map that mixes both chiral (holomorphic) generators $T$ and $\bar{T}$. It depends on a single real parameter and it can be regarded as a ``nonlinear $SO(1,1)$ automorphism.'' The map preserves the form of the momentum density and naturally induces a flow on the energy density by a marginal $\sqrt{T\bar{T}}$ deformation. In turn, the general solution of the corresponding flow equation of the deformed action can be analytically solved in closed form, recovering the nonlinear automorphism. The deformed CFT$_{2}$ can also be described through the original theory on a field-dependent curved metric whose lapse and shift functions are given by the variation of the deformed Hamiltonian with respect to the energy and momentum densities, respectively. The conformal symmetries of the deformed theories can then also be seen to arise from diffeomorphisms that fulfill suitably deformed conformal Killing equations. Besides, Cardy formula is shown to map to itseft under the nonlinear automorphism. As a simple example, the deformation of $N$ free bosons is briefly addressed, making contact with recent related results and the dimensional reduction of the ModMax theory. Furthermore, the nonlinear map between the conformal algebra in 2D and its ultra/non-relativistic versions (BMS$_{3}$$\approx$CCA$_{2}$$\approx$GCA$_{2}$), including the corresponding finite $\sqrt{T\bar{T}}$ deformation, are recovered from a limiting case of the nonlinear automorphism. The extension to a three-parameter nonlinear $ISO(1,1)$ automorphism of the conformal algebra, and a discrete nonlinear automorphism of BMS$_{3}$ are also briefly discussed.


title: FedTrees: A Novel Computation-Communication Efficient Federated Learning Framework Investigated in Smart Grids

abstract: Smart energy performance monitoring and optimisation at the supplier and consumer levels is essential to realising smart cities. In order to implement a more sustainable energy management plan, it is crucial to conduct a better energy forecast. The next-generation smart meters can also be used to measure, record, and report energy consumption data, which can be used to train machine learning (ML) models for predicting energy needs. However, sharing fine-grained energy data and performing centralised learning may compromise users' privacy and leave them vulnerable to several attacks. This study addresses this issue by utilising federated learning (FL), an emerging technique that performs ML model training at the user level, where data resides. We introduce FedTrees, a new, lightweight FL framework that benefits from the outstanding features of ensemble learning. Furthermore, we developed a delta-based early stopping algorithm to monitor FL training and stop it when it does not need to continue. The simulation results demonstrate that FedTrees outperforms the most popular federated averaging (FedAvg) framework and the baseline Persistence model for providing accurate energy forecasting patterns while taking only 2% of the computation time and 13% of the communication rounds compared to FedAvg, saving considerable amounts of computation and communication resources.


title: $K$-theory of Bernoulli shifts of finite groups on UHF-Algebras

abstract: We show that the Bernoulli shift and the trivial action of a finite group $G$ on a UHF-algebra of infinite type are $KK^G$-equivalent and that the Bernoulli shift absorbs the trivial action up to conjugacy. As an application, we compute the $K$-theory of crossed products by approximately inner flips on classifiable $C^*$-algebras.


title: Learning Robust Kernel Ensembles with Kernel Average Pooling

abstract: Model ensembles have long been used in machine learning to reduce the variance in individual model predictions, making them more robust to input perturbations. Pseudo-ensemble methods like dropout have also been commonly used in deep learning models to improve generalization. However, the application of these techniques to improve neural networks' robustness against input perturbations remains underexplored. We introduce Kernel Average Pooling (KAP), a neural network building block that applies the mean filter along the kernel dimension of the layer activation tensor. We show that ensembles of kernels with similar functionality naturally emerge in convolutional neural networks equipped with KAP and trained with backpropagation. Moreover, we show that when trained on inputs perturbed with additive Gaussian noise, KAP models are remarkably robust against various forms of adversarial attacks. Empirical evaluations on CIFAR10, CIFAR100, TinyImagenet, and Imagenet datasets show substantial improvements in robustness against strong adversarial attacks such as AutoAttack without training on any adversarial examples.


title: DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases

abstract: Question answering over knowledge bases (KBs) aims to answer natural language questions with factual information such as entities and relations in KBs. Previous methods either generate logical forms that can be executed over KBs to obtain final answers or predict answers directly. Empirical results show that the former often produces more accurate answers, but it suffers from non-execution issues due to potential syntactic and semantic errors in the generated logical forms. In this work, we propose a novel framework DecAF that jointly generates both logical forms and direct answers, and then combines the merits of them to get the final answers. Moreover, different from most of the previous methods, DecAF is based on simple free-text retrieval without relying on any entity linking tools -- this simplification eases its adaptation to different datasets. DecAF achieves new state-of-the-art accuracy on WebQSP, FreebaseQA, and GrailQA benchmarks, while getting competitive results on the ComplexWebQuestions benchmark.


title: CEREAL: Few-Sample Clustering Evaluation

abstract: Evaluating clustering quality with reliable evaluation metrics like normalized mutual information (NMI) requires labeled data that can be expensive to annotate. We focus on the underexplored problem of estimating clustering quality with limited labels. We adapt existing approaches from the few-sample model evaluation literature to actively sub-sample, with a learned surrogate model, the most informative data points for annotation to estimate the evaluation metric. However, we find that their estimation can be biased and only relies on the labeled data. To that end, we introduce CEREAL, a comprehensive framework for few-sample clustering evaluation that extends active sampling approaches in three key ways. First, we propose novel NMI-based acquisition functions that account for the distinctive properties of clustering and uncertainties from a learned surrogate model. Next, we use ideas from semi-supervised learning and train the surrogate model with both the labeled and unlabeled data. Finally, we pseudo-label the unlabeled data with the surrogate model. We run experiments to estimate NMI in an active sampling pipeline on three datasets across vision and language. Our results show that CEREAL reduces the area under the absolute error curve by up to 57% compared to the best sampling baseline. We perform an extensive ablation study to show that our framework is agnostic to the choice of clustering algorithm and evaluation metric. We also extend CEREAL from clusterwise annotations to pairwise annotations. Overall, CEREAL can efficiently evaluate clustering with limited human annotations.


title: Application of Deep Q Learning with Simulation Results for Elevator Optimization

abstract: This paper presents a methodology for combining programming and mathematics to optimize elevator wait times. Based on simulated user data generated according to the canonical three-peak model of elevator traffic, we first develop a naive model from an intuitive understanding of the logic behind elevators. We take into consideration a general array of features including capacity, acceleration, and maximum wait time thresholds to adequately model realistic circumstances. Using the same evaluation framework, we proceed to develop a Deep Q Learning model in an attempt to match the hard-coded naive approach for elevator control. Throughout the majority of the paper, we work under a Markov Decision Process (MDP) schema, but later explore how the assumption fails to characterize the highly stochastic overall Elevator Group Control System (EGCS).


title: Improving Policy Learning via Language Dynamics Distillation

abstract: Recent work has shown that augmenting environments with language descriptions improves policy learning. However, for environments with complex language abstractions, learning how to ground language to observations is difficult due to sparse, delayed rewards. We propose Language Dynamics Distillation (LDD), which pretrains a model to predict environment dynamics given demonstrations with language descriptions, and then fine-tunes these language-aware pretrained representations via reinforcement learning (RL). In this way, the model is trained to both maximize expected reward and retain knowledge about how language relates to environment dynamics. On SILG, a benchmark of five tasks with language descriptions that evaluate distinct generalization challenges on unseen environments (NetHack, ALFWorld, RTFM, Messenger, and Touchdown), LDD outperforms tabula-rasa RL, VAE pretraining, and methods that learn from unlabeled demonstrations in inverse RL and reward shaping with pretrained experts. In our analyses, we show that language descriptions in demonstrations improve sample-efficiency and generalization across environments, and that dynamics modelling with expert demonstrations is more effective than with non-experts.


title: The COVID-19 Pandemic and the Future of Telecommuting in the United States

abstract: This study focuses on an important transport-related long-term effect of the COVID-19 pandemic in the United States: an increase in telecommuting. Analyzing a nationally representative panel survey of adults, we find that 40-50% of workers expect to telecommute at least a few times per month post-pandemic, up from 24% pre-COVID. If given the option, 90-95% of those who first telecommuted during the pandemic plan to continue the practice regularly. We also find that new telecommuters are demographically similar to pre-COVID telecommuters. Both pre- and post-COVID, higher educational attainment and income, together with certain job categories, largely determine whether workers have the option to telecommute. Despite growth in telecommuting, approximately half of workers expect to remain unable to telecommute and between 2/3 and 3/4 of workers expect their post-pandemic telecommuting patterns to be unchanged from their pre-COVID patterns. This limits the contribution telecommuting can make to reducing peak hour transport demand.


title: Multi-Task Option Learning and Discovery for Stochastic Path Planning

abstract: This paper addresses the problem of reliably and efficiently solving broad classes of long-horizon stochastic path planning problems. Starting with a vanilla RL formulation with a stochastic dynamics simulator and an occupancy matrix of the environment, our approach computes useful options with policies as well as high-level paths that compose the discovered options. Our main contributions are (1) data-driven methods for creating abstract states that serve as endpoints for helpful options, (2) methods for computing option policies using auto-generated option guides in the form of dense pseudo-reward functions, and (3) an overarching algorithm for composing the computed options. We show that this approach yields strong guarantees of executability and solvability: under fairly general conditions, the computed option guides lead to composable option policies and consequently ensure downward refinability. Empirical evaluation on a range of robots, environments, and tasks shows that this approach effectively transfers knowledge across related tasks and that it outperforms existing approaches by a significant margin.


title: Topological Singularity Detection at Multiple Scales

abstract: The manifold hypothesis, which assumes that data lies on or close to an unknown manifold of low intrinsic dimension, is a staple of modern machine learning research. However, recent work has shown that real-world data exhibits distinct non-manifold structures, i.e. singularities, that can lead to erroneous findings. Detecting such singularities is therefore crucial as a precursor to interpolation and inference tasks. We address this issue by developing a topological framework that (i) quantifies the local intrinsic dimension, and (ii) yields a Euclidicity score for assessing the 'manifoldness' of a point along multiple scales. Our approach identifies singularities of complex spaces, while also capturing singular structures and local geometric complexity in image data.


title: Ultrafast generation of hidden phases via energy-tuned electronic photoexcitation in magnetite

abstract: Metal-insulator transitions (MIT) occurring in non-adiabatic conditions can evolve through high-energy intermediate states that are difficult to observe and control via static methods. By monitoring the out-of-equilibrium structural dynamics of a magnetite (Fe3O4) crystal via ultrafast electron diffraction, we show that MITs can evolve through different pathways by properly selecting the electronic excitation with light. Near-infrared (800 nm) photons inducing d-d electronic transitions is found to favor the destruction of the long-range zigzag network of the trimerons and to generate a phase separation between cubic-metallic and monoclinic-insulating regions. Instead, visible light (400 nm) further promotes the long-range order of the trimerons by stabilizing the charge density wave fluctuations through the excitation of the oxygen 2p to iron 3d charge transfer and, thus, fosters a reinforcement of the monoclinic insulating phase. Our experiments demonstrate that tailored light pulses can drive strongly correlated materials into different hidden phases, influencing the lifetime and emergent properties of the intermediate states.


title: A Time-Efficient, Data Driven Modelling Approach For Predicting The Geomagnetic Impact of Coronal Mass Ejections

abstract: To understand the global-scale physical processes behind coronal mass ejection (CME)-driven geomagnetic storms and predict their intensity as a space weather forecasting measure, we develop an interplanetary CME flux rope-magnetosphere interaction module using 3D magnetohydrodynamics. The simulations adequately describe ICME-forced dynamics of the magnetosphere including the imposed magnetotail torsion. These interactions also result in induced currents which is used to calculate the geomagnetic perturbation. Through a suitable calibration, we estimate a proxy of geoeffectiveness -- the Storm Intensity index (STORMI) -- that compares well with the Dst/SYM-H Index. Simulated impacts of two contrasting coronal mass ejections quantified by the STORMI index exhibit a high linear correlation with the corresponding Dst and SYM-H indices. Our approach is relatively simple, has fewer parameters to be fine-tuned, is time-efficient compared to complex fluid-kinetic methods. Furthermore, we demonstrate that flux rope erosion does not significantly affect our results. Thus our method has the potential to significantly extend the time window for predictability -- an outstanding challenge in geospace environment forecasting -- if early predictions of near-Earth CME flux rope structures based on near-Sun observations are available as inputs. This study paves the way for early warnings based on operational predictions of CME-driven geomagnetic storms.


title: Probing Ultra-light Primordial Black Holes as a Dark Matter Candidate

abstract: Dark Matter (DM) is omnipresent in our universe. Despite its abundance, the microscopic identity of DM still remains a mystery. Primordial black holes (PBHs), possibly formed via gravitational collapse of large density perturbations in the early universe, are one of the earliest proposed and viable DM candidates. Recent studies indicate that PBHs can make up a large or even entire fraction of the DM density for a wide range of masses. Here, we briefly review the observational constraints on PBHs as DM, concentrating on the ultra-light mass window. Ultra-light PBHs emit particles via Hawking radiation and can be probed by observing such Hawking evaporated particles in various space as well as ground based detectors. We also outline how next-generation gamma ray telescopes can set a stringent exclusion limit on ultra-light PBH DM by probing low energy photons.


title: Digital Twin and Artificial Intelligence Incorporated With Surrogate Modeling for Hybrid and Sustainable Energy Systems

abstract: Surrogate modeling has brought about a revolution in computation in the branches of science and engineering. Backed by Artificial Intelligence, a surrogate model can present highly accurate results with a significant reduction in computation time than computer simulation of actual models. Surrogate modeling techniques have found their use in numerous branches of science and engineering, energy system modeling being one of them. Since the idea of hybrid and sustainable energy systems is spreading rapidly in the modern world for the paradigm of the smart energy shift, researchers are exploring the future application of artificial intelligence-based surrogate modeling in analyzing and optimizing hybrid energy systems. One of the promising technologies for assessing applicability for the energy system is the digital twin, which can leverage surrogate modeling. This work presents a comprehensive framework/review on Artificial Intelligence-driven surrogate modeling and its applications with a focus on the digital twin framework and energy systems. The role of machine learning and artificial intelligence in constructing an effective surrogate model is explained. After that, different surrogate models developed for different sustainable energy sources are presented. Finally, digital twin surrogate models and associated uncertainties are described.


title: Leveraging Industry 4.0 -- Deep Learning, Surrogate Model and Transfer Learning with Uncertainty Quantification Incorporated into Digital Twin for Nuclear System

abstract: Industry 4.0 targets the conversion of the traditional industries into intelligent ones through technological revolution. This revolution is only possible through innovation, optimization, interconnection, and rapid decision-making capability. Numerical models are believed to be the key components of Industry 4.0, facilitating quick decision-making through simulations instead of costly experiments. However, numerical investigation of precise, high-fidelity models for optimization or decision-making is usually time-consuming and computationally expensive. In such instances, data-driven surrogate models are excellent substitutes for fast computational analysis and the probabilistic prediction of the output parameter for new input parameters. The emergence of Internet of Things (IoT) and Machine Learning (ML) has made the concept of surrogate modeling even more viable. However, these surrogate models contain intrinsic uncertainties, originate from modeling defects, or both. These uncertainties, if not quantified and minimized, can produce a skewed result. Therefore, proper implementation of uncertainty quantification techniques is crucial during optimization, cost reduction, or safety enhancement processes analysis. This chapter begins with a brief overview of the concept of surrogate modeling, transfer learning, IoT and digital twins. After that, a detailed overview of uncertainties, uncertainty quantification frameworks, and specifics of uncertainty quantification methodologies for a surrogate model linked to a digital twin is presented. Finally, the use of uncertainty quantification approaches in the nuclear industry has been addressed.


title: Mixed-Reality Robot Behavior Replay: A System Implementation

abstract: As robots become increasingly complex, they must explain their behaviors to gain trust and acceptance. However, it may be difficult through verbal explanation alone to fully convey information about past behavior, especially regarding objects no longer present due to robots' or humans' actions. Humans often try to physically mimic past movements to accompany verbal explanations. Inspired by this human-human interaction, we describe the technical implementation of a system for past behavior replay for robots in this tool paper. Specifically, we used Behavior Trees to encode and separate robot behaviors, and schemaless MongoDB to structurally store and query the underlying sensor data and joint control messages for future replay. Our approach generalizes to different types of replays, including both manipulation and navigation replay, and visual (i.e., augmented reality (AR)) and auditory replay. Additionally, we briefly summarize a user study to further provide empirical evidence of its effectiveness and efficiency. Sample code and instructions are available on GitHub at https://github.com/umhan35/robot-behavior-replay.


title: Correlated twin-photon generation in a silicon nitrite loaded thin film PPLN waveguide

abstract: Photon-pair sources based on thin film lithium niobate on insulator technology have a great potential for integrated optical quantum information processing. We report on such a source of correlated twin-photon pairs generated by spontaneous parametric down conversion in a silicon nitride (SiN) rib loaded thin film periodically poled lithium niobate (LN) waveguide. The generated photon pairs have a wavelength centred at 1560\,nm compatible with present telecom infrastructure, a large bandwidth (21\,THz) and a brightness of $\sim 2.5\times 10^5$\,pairs/s/mW/GHz. The photons are correlated and exhibit a cross correlation $g^{(2)}(0)$ of about 8000. Using the Hanbury Brown and Twiss effect, we have also shown heralded single photon emission, achieving an autocorrelation $g^{(2)}_H(0) \simeq 0.04$.


title: E-Branchformer: Branchformer with Enhanced merging for speech recognition

abstract: Conformer, combining convolution and self-attention sequentially to capture both local and global information, has shown remarkable performance and is currently regarded as the state-of-the-art for automatic speech recognition (ASR). Several other studies have explored integrating convolution and self-attention but they have not managed to match Conformer's performance. The recently introduced Branchformer achieves comparable performance to Conformer by using dedicated branches of convolution and self-attention and merging local and global context from each branch. In this paper, we propose E-Branchformer, which enhances Branchformer by applying an effective merging method and stacking additional point-wise modules. E-Branchformer sets new state-of-the-art word error rates (WERs) 1.81% and 3.65% on LibriSpeech test-clean and test-other sets without using any external training data.


title: A 2-categorical proof of Frobenius for fibrations defined from a generic point

abstract: Consider a locally cartesian closed category with an object I and a class of trivial fibrations, which admit sections and are stable under pushforward and retract as arrows. Define the fibrations to be those maps whose Leibniz exponential with the generic point of I defines a trivial fibration. Then the fibrations are also closed under pushforward.


title: Causal Estimation for Text Data with (Apparent) Overlap Violations

abstract: Consider the problem of estimating the causal effect of some attribute of a text document; for example: what effect does writing a polite vs. rude email have on response time? To estimate a causal effect from observational data, we need to adjust for confounding aspects of the text that affect both the treatment and outcome -- e.g., the topic or writing level of the text. These confounding aspects are unknown a priori, so it seems natural to adjust for the entirety of the text (e.g., using a transformer). However, causal identification and estimation procedures rely on the assumption of overlap: for all levels of the adjustment variables, there is randomness leftover so that every unit could have (not) received treatment. Since the treatment here is itself an attribute of the text, it is perfectly determined, and overlap is apparently violated. The purpose of this paper is to show how to handle causal identification and obtain robust causal estimation in the presence of apparent overlap violations. In brief, the idea is to use supervised representation learning to produce a data representation that preserves confounding information while eliminating information that is only predictive of the treatment. This representation then suffices for adjustment and can satisfy overlap. Adapting results on non-parametric estimation, we find that this procedure is robust to conditional outcome misestimation, yielding a low-bias estimator with valid uncertainty quantification under weak conditions. Empirical results show strong improvements in bias and uncertainty quantification relative to the natural baseline.


title: Longitudinal mode-coupling instabilities of proton bunches in the CERN Super Proton Synchrotron

abstract: In this paper, we study single-bunch instabilities observed in the CERN Super Proton Synchrotron (SPS). According to the linearized Vlasov theory, radial or azimuthal mode-coupling instabilities result from a coupling of bunch-oscillation modes, which belong to either the same or adjacent azimuthal modes, respectively. We show that both instability mechanisms exist in the SPS by applying the Oide-Yokoya approach to compute van Kampen modes for the realistic longitudinal impedance model of the SPS. The results agree with macroparticle simulations and are consistent with beam measurements. In particular, we see that the uncontrolled longitudinal emittance blow-up of single bunches observed before the recent impedance reduction campaign (2018-2021) is due to the radial mode-coupling instability. Unexpectedly, this instability is as strong as the azimuthal mode-coupling instability, which is possible in the SPS for other combinations of bunch length and intensity. We also demonstrate the significant role of rf nonlinearity and potential-well distortion in determining these instability thresholds. Finally, we discuss the effect of the recent impedance reduction campaign on beam stability in single- and double-rf configurations.


title: Distributed Implementation of Minimax Adaptive Controller For Finite Set of Linear Systems

abstract: This paper deals with a distributed implementation of minimax adaptive control algorithm for networked dynamical systems modeled by a finite set of linear models. To hedge against the uncertainty arising out of finite number of possible dynamics in each node in the network, it collects only the historical data of its neighboring nodes to decide its control action along its edges. This makes our proposed distributed approach scalable. Numerical simulations demonstrate that once each node has sufficiently estimated the uncertain parameters, the distributed minimax adaptive controller behaves like the optimal distributed H-infinity controller in hindsight.


title: Asymptotic analysis of a family of Sobolev orthogonal polynomials related to the generalized Charlier polynomials

abstract: In this paper we tackle the asymptotic behavior of a family of orthogonal polynomials with respect to a nonstandard inner product involving the forward operator {\Delta}. Concretely, we treat the generalized Charlier weights in the framework of {\Delta}--Sobolev orthogonality. We obtain an asymptotic expansion for this orthogonal polynomials where the falling factorial polynomials play an important role.


title: Spectral shaping of an ultrafast modelocked Ytterbium fiber laser output through a passive intracavity optical filter; a simple and reliable route to sub-45 fs pulses

abstract: Here we investigate the use of passive intracavity optical filters for controlling the laser output spectrum of a polarization-mode-locked, ultrafast Ytterbium fiber laser. With strategic placement of the filter cutoff frequency, the overall lasing bandwidth can be increased or extended. Overall laser performance, including pulse compression and intensity noise, is investigated for both shortpass and longpass filters with a range of cutoff frequencies, tuned by rotating the filters. We demonstrate the use of an intracavity filter not only shapes the output spectra, it provides a route for overall broader bandwidths and shorter pulses in Yb:fiber lasers. These results demonstrate that spectral shaping is a useful tool to routinely achieve sub-45 fs pulse durations in Yb:fiber lasers.


title: Contrastive Graph Few-Shot Learning

abstract: Prevailing deep graph learning models often suffer from label sparsity issue. Although many graph few-shot learning (GFL) methods have been developed to avoid performance degradation in face of limited annotated data, they excessively rely on labeled data, where the distribution shift in the test phase might result in impaired generalization ability. Additionally, they lack a general purpose as their designs are coupled with task or data-specific characteristics. To this end, we propose a general and effective Contrastive Graph Few-shot Learning framework (CGFL). CGFL leverages a self-distilled contrastive learning procedure to boost GFL. Specifically, our model firstly pre-trains a graph encoder with contrastive learning using unlabeled data. Later, the trained encoder is frozen as a teacher model to distill a student model with a contrastive loss. The distilled model is finally fed to GFL. CGFL learns data representation in a self-supervised manner, thus mitigating the distribution shift impact for better generalization and making model task and data-independent for a general graph mining purpose. Furthermore, we introduce an information-based method to quantitatively measure the capability of CGFL. Comprehensive experiments demonstrate that CGFL outperforms state-of-the-art baselines on several graph mining tasks in the few-shot scenario. We also provide quantitative measurement of CGFL's success.


title: Development of a Full Monte Carlo Therapeutic Dose Calculation Toolkit for Halcyon Using Geant4

abstract: Purpose: To develop a Monte Carlo (MC) therapeutic dose calculation toolkit of a recently released ring gantry linac in Geant4 (Version 10.7) for secondary dose validation of radiotherapy plan. Methods: For the Halcyon (Varian Medical Systems), the DSMLC was modeled and radiation transport in DSMLC and patient phantom was simulated using Geant4. Radiation source was sampled from a phase space file for linac head above the DSMLC. The phase space file was obtained using a cloud-based Monte Carlo (MC) simulator, VirtuaLinac (VL) provide by Varian. Dosimetric profiles for different square field widths (2x2, 4x4, 6x6, 8x8, 10x10, 20x20, and 28x28 cm2), i.e., percent depth dose (PDD) curves and lateral profiles are simulated and compared against the experimental profiles. IMRT (intensity modulated radiation therapy) plans in two anatomical sites (prostate and brain) were also calculated using the developed toolkit and compared against the TPS calculated dose (Acuros, Eclipse 15.6). 3D dose difference and 3D gamma analysis were used to evaluate the simulation accuracy compared against the TPS calculated dose. Results: The simulated lateral dose profiles and PDD curves in water phantom match well with the measured ones for all the simulated field sizes with relative difference +-2%. For the prostate and brain IMRT plans, the simulated dose showed a good agreement with the TPS calculated dose. The 3D gamma pass rate (3%/3mm) are 98.08% and 95.4% for the two prostate and brain plans, respectively. Conclusion: The developed full MC dose calculation toolkit for Halcyon performs well in dose calculations in water phantom and patient CT phantom. The developed toolkit shows promising possibility for future secondary dose calculation for IMRT and serve as clinical quality assurance (QA) tool for Halcyon.


title: The Diophantine problem for systems of algebraic equations with exponents

abstract: Consider the equation $q_1\alpha^{x_1}+\dots+q_k\alpha^{x_k} = q$, with constants $\alpha \in \overline{\mathbb{Q}} \setminus \{0,1\}$, $q_1,\ldots,q_k,q\in\overline{\mathbb{Q}}$ and unknowns $x_1,\ldots,x_k$, referred to in this paper as an \emph{algebraic equation with exponents}. We prove that the problem to decide if a given equation has an integer solution is $\textbf{NP}$-complete, and that the same holds for systems of equations (whether $\alpha$ is fixed or given as part of the input). Furthermore, we describe the set of all solutions for a given system of algebraic equations with exponents and prove that it is semilinear.


title: D-Align: Dual Query Co-attention Network for 3D Object Detection Based on Multi-frame Point Cloud Sequence

abstract: LiDAR sensors are widely used for 3D object detection in various mobile robotics applications. LiDAR sensors continuously generate point cloud data in real-time. Conventional 3D object detectors detect objects using a set of points acquired over a fixed duration. However, recent studies have shown that the performance of object detection can be further enhanced by utilizing spatio-temporal information obtained from point cloud sequences. In this paper, we propose a new 3D object detector, named D-Align, which can effectively produce strong bird's-eye-view (BEV) features by aligning and aggregating the features obtained from a sequence of point sets. The proposed method includes a novel dual-query co-attention network that uses two types of queries, including target query set (T-QS) and support query set (S-QS), to update the features of target and support frames, respectively. D-Align aligns S-QS to T-QS based on the temporal context features extracted from the adjacent feature maps and then aggregates S-QS with T-QS using a gated attention mechanism. The dual queries are updated through multiple attention layers to progressively enhance the target frame features used to produce the detection results. Our experiments on the nuScenes dataset show that the proposed D-Align method greatly improved the performance of a single frame-based baseline method and significantly outperformed the latest 3D object detectors.


title: Statistical learning for $\psi$-weakly dependent processes

abstract: We consider statistical learning question for $\psi$-weakly dependent processes, that unifies a large class of weak dependence conditions such as mixing, association,$\cdots$ The consistency of the empirical risk minimization algorithm is established. We derive the generalization bounds and provide the learning rate, which, on some H{\"o}lder class of hypothesis, is close to the usual $O(n^{-1/2})$ obtained in the {\it i.i.d.} case. Application to time series prediction is carried out with an example of causal models with exogenous covariates.


title: A Multi-label Time Series Classification Approach for Non-intrusive Water End-Use Monitoring

abstract: Numerous real-world problems from a diverse set of application areas exist that exhibit temporal dependencies. We focus on a specific type of time series classification which we refer to as aggregated time series classification. We consider an aggregated sequence of a multi-variate time series, and propose a methodology to make predictions based solely on the aggregated information. As a case study, we apply our methodology to the challenging problem of household water end-use dissagregation when using non-intrusive water monitoring. Our methodology does not require a-priori identification of events, and to our knowledge, it is considered for the first time. We conduct an extensive experimental study using a residential water-use simulator, involving different machine learning classifiers, multi-label classification methods, and successfully demonstrate the effectiveness of our methodology.


title: Data-driven discovery of non-Newtonian astronomy via learning non-Euclidean Hamiltonian

abstract: Incorporating the Hamiltonian structure of physical dynamics into deep learning models provides a powerful way to improve the interpretability and prediction accuracy. While previous works are mostly limited to the Euclidean spaces, their extension to the Lie group manifold is needed when rotations form a key component of the dynamics, such as the higher-order physics beyond simple point-mass dynamics for N-body celestial interactions. Moreover, the multiscale nature of these processes presents a challenge to existing methods as a long time horizon is required. By leveraging a symplectic Lie-group manifold preserving integrator, we present a method for data-driven discovery of non-Newtonian astronomy. Preliminary results show the importance of both these properties in training stability and prediction accuracy.


title: Factorized Fusion Shrinkage for Dynamic Relational Data

abstract: Modern data science applications often involve complex relational data with dynamic structures. An abrupt change in such dynamic relational data is typically observed in systems that undergo regime changes due to interventions. In such a case, we consider a factorized fusion shrinkage model in which all decomposed factors are dynamically shrunk towards group-wise fusion structures, where the shrinkage is obtained by applying global-local shrinkage priors to the successive differences of the row vectors of the factorized matrices. The proposed priors enjoy many favorable properties in comparison and clustering of the estimated dynamic latent factors. Comparing estimated latent factors involves both adjacent and long-term comparisons, with the time range of comparison considered as a variable. Under certain conditions, we demonstrate that the posterior distribution attains the minimax optimal rate up to logarithmic factors. In terms of computation, we present a structured mean-field variational inference framework that balances optimal posterior inference with computational scalability, exploiting both the dependence among components and across time. The framework can accommodate a wide variety of models, including dynamic matrix factorization, latent space models for networks and low-rank tensors. The effectiveness of our methodology is demonstrated through extensive simulations and real-world data analysis.


title: Federated Training of Dual Encoding Models on Small Non-IID Client Datasets

abstract: Dual encoding models that encode a pair of inputs are widely used for representation learning. Many approaches train dual encoding models by maximizing agreement between pairs of encodings on centralized training data. However, in many scenarios, datasets are inherently decentralized across many clients (user devices or organizations) due to privacy concerns, motivating federated learning. In this work, we focus on federated training of dual encoding models on decentralized data composed of many small, non-IID (independent and identically distributed) client datasets. We show that existing approaches that work well in centralized settings perform poorly when naively adapted to this setting using federated averaging. We observe that, we can simulate large-batch loss computation on individual clients for loss functions that are based on encoding statistics. Based on this insight, we propose a novel federated training approach, Distributed Cross Correlation Optimization (DCCO), which trains dual encoding models using encoding statistics aggregated across clients, without sharing individual data samples. Our experimental results on two datasets demonstrate that the proposed DCCO approach outperforms federated variants of existing approaches by a large margin.


title: Shockwave: Fair and Efficient Cluster Scheduling for Dynamic Adaptation in Machine Learning

abstract: Dynamic adaptation has become an essential technique in accelerating distributed machine learning (ML) training. Recent studies have shown that dynamically adjusting model structure (e.g., lottery ticket hypothesis) or hyperparameters (e.g., batch size) can significantly accelerate training without sacrificing accuracy. However, existing ML cluster schedulers are not designed to handle dynamic adaptation. We show that existing schemes fail to provide fairness and degrade system efficiency when the training throughput changes over time under dynamic adaptation. We design Shockwave, a scheduler with future planning that builds on two key ideas. First, Shockwave extends classic market theory from static settings to dynamic settings to co-optimize efficiency and fairness. Second, Shockwave utilizes stochastic dynamic programming to handle dynamic changes. We build a system for Shockwave and validate its performance with both trace-driven simulation and cluster experiments. Results show that for traces of ML jobs with dynamic adaptation, Shockwave improves makespan by 1.3X and fairness by 2X when compared with existing fair scheduling schemes.


title: Improving Robustness with Adaptive Weight Decay

abstract: We propose adaptive weight decay, which automatically tunes the hyper-parameter for weight decay during each training iteration. For classification problems, we propose changing the value of the weight decay hyper-parameter on the fly based on the strength of updates from the classification loss (i.e., gradient of cross-entropy), and the regularization loss (i.e., $\ell_2$-norm of the weights). We show that this simple modification can result in large improvements in adversarial robustness -- an area which suffers from robust overfitting -- without requiring extra data across various datasets and architecture choices. For example, our reformulation results in $20\%$ relative robustness improvement for CIFAR-100, and $10\%$ relative robustness improvement on CIFAR-10 comparing to the best tuned hyper-parameters of traditional weight decay resulting in models that have comparable performance to SOTA robustness methods. In addition, this method has other desirable properties, such as less sensitivity to learning rate, and smaller weight norms, which the latter contributes to robustness to overfitting to label noise, and pruning.


title: Safety-Critical Adaptation in Self-Adaptive Systems

abstract: Modern systems are designed to operate in increasingly variable and uncertain environments. Not only are these environments complex, in the sense that they contain a tremendous number of variables, but they also change over time. Systems must be able to adjust their behaviour at run-time to manage these uncertainties. These self-adaptive systems have been studied extensively. This paper proposes a definition of a safety-critical self-adaptive system and then describes a taxonomy for classifying adaptations into different types based on their impact on the system's safety and the system's safety case. The taxonomy expresses criteria for classification and then describes specific criteria that the safety case for a self-adaptive system must satisfy, depending on the type of adaptations performed. Each type in the taxonomy is illustrated using the example of a safety-critical self-adaptive water heating system.


title: Quantum Discord and Logarithmic Negativity in the Generalized n-qubit Werner State

abstract: Quantum Discord (QD) is a measure of the total quantum non-local correlations of a quantum system. The formalism of quantum discord has been applied to various two-qubit mixed states and it has been reported that there is a non-zero quantum discord even when the states are unentangled. To this end, we have calculated the Quantum Discord for higher than two qubit mixed state, that is, the generalized n-qubit Werner state with a bipartite split. We found that the QD saturates to a straight line with unit slope in the thermodynamic limit. Qualitative studies of entanglement between the two subsystems using logarithmic negativity revealed that the entanglement content between them increases non-uniformly with the number of qubits leading to its saturation. We have proved the above claims both analytically and numerically.


title: A Layered Spin 1/2 polymorph of titanium triiodide

abstract: A previously unreported layered spin 1/2 triangular lattice polymorph of TiI3 is described, synthesized under 6 GPa of applied pressure at 900 C, but stable at atmospheric pressure. This air-sensitive material has a CdI2-type layered structure (P-3m1 (#164), a = 4.012 A and c = 6.641 A at 120 K, Z = 1 of Ti0.667I2) with an in-plane triangular lattice, related to that of TiI4 (Ti0.5I2). Although the TiI3 formula is consistent with expectations for a layered honeycomb lattice of spin 1/2 Ti(III), there appears to be disorder in the crystal structure. Magnetic susceptibility and heat capacity measurements suggest that the material undergoes several low temperature phase transitions.


title: Towards complete representation of bacterial contents in metagenomic samples

abstract: Background: In the metagenome assembly of a microbiome community, we may think abundant species would be easier to assemble due to their deeper coverage. However, this conjucture is rarely tested. We often do not know how many abundant species we are missing and do not have an approach to recover these species. Results: Here we proposed k-mer based and 16S RNA based methods to measure the completeness of metagenome assembly. We showed that even with PacBio High-Fidelity (HiFi) reads, abundant species are often not assembled as high strain diversity may lead to fragmented contigs. We developed a novel algorithm to recover abundant metagenome-assembled genomes (MAGs) by identifying circular assembly subgraphs. Our algorithm is reference-free and complement to standard metagenome binning. Evaluated on 14 real datasets, it rescued many abundant species that would be missing with existing methods. Conclusions: Our work stresses the importance of metagenome completeness which is often overlooked before. Our algorithm generates more circular MAGs and moves a step closer to the complete representation of microbiome communities.


title: Interplay of gross and fine structures in strongly-curved sheets

abstract: Although thin films are typically manufactured in planar sheets or rolls, they are often forced into three-dimensional shapes, producing a plethora of structures across multiple length-scales. Existing theoretical approaches have made progress by separating the behaviors at different scales and limiting their scope to one. Under large confinement, a geometric model has been proposed to predict the gross shape of the sheet, which averages out the fine features. However, the actual meaning of the gross shape, and how it constrains the fine features, remains unclear. Here, we study a thin-membraned balloon as a prototypical system that involves a doubly curved gross shape with large amplitude undulations. By probing its profiles and cross sections, we discover that the geometric model captures the mean behavior of the film. We then propose a minimal model for the balloon cross sections, as independent elastic filaments subjected to an effective pinning potential around the mean shape. This approach allows us to combine the global and local features consistently. Despite the simplicity of our model, it reproduces a broad range of phenomena seen in the experiments, from how the morphology changes with pressure to the detailed shape of the wrinkles and folds. Our results establish a new route to understanding finite buckled structures over an enclosed surface, which could aid the design of inflatable structures, or provide insight into biological patterns.


title: Image-Based Detection of Modifications in Gas Pump PCBs with Deep Convolutional Autoencoders

abstract: In this paper, we introduce an approach for detecting modifications in assembled printed circuit boards based on photographs taken without tight control over perspective and illumination conditions. One instance of this problem is the visual inspection of gas pumps PCBs, which can be modified by fraudsters wishing to deceive costumers or evade taxes. Given the uncontrolled environment and the huge number of possible modifications, we address the problem as a case of anomaly detection, proposing an approach that is directed towards the characteristics of that scenario, while being well-suited for other similar applications. The proposed approach employs a deep convolutional autoencoder trained to reconstruct images of an unmodified board, but which remains unable to do the same for images showing modifications. By comparing the input image with its reconstruction, it is possible to segment anomalies and modifications in a pixel-wise manner. Experiments performed on a dataset built to represent real-world situations (and which we will make publicly available) show that our approach outperforms other state-of-the-art approaches for anomaly segmentation in the considered scenario, while producing comparable results on the popular MVTec-AD dataset for a more general object anomaly detection task.


title: Aeroacoustic airfoil shape optimization enhanced by autoencoders

abstract: We present a framework for airfoil shape optimization to reduce the trailing edge noise for the design of wind turbine blades. Far-field noise is evaluated using Amiet's theory coupled with the TNO-Blake model to calculate the wall pressure spectrum and fast turn-around XFOIL simulations to evaluate the boundary layer parameters. The computational framework is first validated using a NACA0012 airfoil at zero angle of attack. Particle swarm optimization is used to find the optimized airfoil configuration. The multi-objective optimization minimizes the A-weighted overall sound pressure level at various angles of attack, while ensuring enough lift and minimum drag. We compare classic parametrization methods to define the airfoil geometry (i.e., CST) to a machine learning method (i.e., a variational autoencoder). We observe that variational autoencoders can represent a wide variety of geometries, with only four encoded variables, leading to efficient optimizations, which result in improved optimal shapes. When compared to the baseline geometry, a NACA0012, the autoencoder-based optimized airfoil reduces by 3% (1.75 dBA) the overall sound pressure level (with decreased noise across the entire frequency range), while maintaining favorable aerodynamic properties in terms of lift and drag.


title: MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initialization

abstract: Training graph neural networks (GNNs) on large graphs is complex and extremely time consuming. This is attributed to overheads caused by sparse matrix multiplication, which are sidestepped when training multi-layer perceptrons (MLPs) with only node features. MLPs, by ignoring graph context, are simple and faster for graph data, however they usually sacrifice prediction accuracy, limiting their applications for graph data. We observe that for most message passing-based GNNs, we can trivially derive an analog MLP (we call this a PeerMLP) with an equivalent weight space, by setting the trainable parameters with the same shapes, making us curious about \textbf{\emph{how do GNNs using weights from a fully trained PeerMLP perform?}} Surprisingly, we find that GNNs initialized with such weights significantly outperform their PeerMLPs, motivating us to use PeerMLP training as a precursor, initialization step to GNN training. To this end, we propose an embarrassingly simple, yet hugely effective initialization method for GNN training acceleration, called MLPInit. Our extensive experiments on multiple large-scale graph datasets with diverse GNN architectures validate that MLPInit can accelerate the training of GNNs (up to 33X speedup on OGB-Products) and often improve prediction performance (e.g., up to $7.97\%$ improvement for GraphSAGE across $7$ datasets for node classification, and up to $17.81\%$ improvement across $4$ datasets for link prediction on metric Hits@10). The code is available at \href{https://github.com/snap-research/MLPInit-for-GNNs}.


title: Variability in Protoplanetary Nebulae: IX. Evidence for Evolution in a Decade

abstract: We have carried out a new photometric V,Rc study of 12 protoplanetary nebulae, objects in the short-lived transition between the AGB and PN phases of stellar evolution. These had been the subjects of an earlier study, using data from 1994-2007, that found that all 12 varied periodically, with pulsation periods in the range of ~38 to ~150 days. They are all carbon-rich, with F-G spectral types. We combined our new (2008-2018) data with publicly-available ASAS-SN data and determined new periods for their variability. The older and newer period values were compared to investigate evidence of period change, for which there is theoretical support that it might be detectable in a decade or two in some cases. Such a detection is challenging since the light curves are complicated, with multiple periods, changing amplitudes, and evidence of shocks. Nevertheless, we found one, and possibly two, such cases, which are associated with the higher temperature stars in the sample (7250 and 8000 K). These results are most consistent with the evolution of stars at the lower end of the mass range of carbon stars, ~1.5-2 M(sun). Several of the stars show longer-term trends of increasing (six cases) or decreasing (one case) brightness, which we think most likely due to changes in the circumstellar dust opacity. There is one case of a possible ~1.8 yr period in addition to the shorter pulsation. This is interpreted as possible evidence of an orbiting companion.


title: High-Energy Photon Opacity in the Twisted Magnetospheres of Magnetars

abstract: Magnetars are neutron stars characterized by strong surface magnetic fields generally exceeding the quantum critical value of 44.1 TeraGauss. High-energy photons propagating in their magnetospheres can be attenuated by QED processes like photon splitting and magnetic pair creation. In this paper, we compute the opacities due to photon splitting and pair creation by photons emitted anywhere in the magnetosphere of a magnetar. Axisymmetric, twisted dipole field configurations embedded in the Schwarzschild metric are treated. The paper computes the maximum energies for photon transparency that permit propagation to infinity in curved spacetime. Special emphasis is given to cases where photons are generated along magnetic field loops and/or in polar regions; these cases directly relate to resonant inverse Compton scattering models for the hard X-ray emission from magnetars and Comptonized soft gamma-ray emission from giant flares. We find that increases in magnetospheric twists raise or lower photon opacities, depending on both the emission locale, and the competition between field line straightening and field strength enhancement. Consequently, given the implicit spectral transparency of hard X-ray bursts and persistent "tail" emission of magnetars, photon splitting considerations constrain their emission region locales and the twist angle of the magnetosphere; these constraints can be probed by future soft gamma-ray telescopes such as COSI and AMEGO. The inclusion of twists generally increases the opaque volume of pair creation by photons above its threshold, except when photons are emitted in polar regions and approximately parallel to the field.


title: A Decade of Knowledge Graphs in Natural Language Processing: A Survey

abstract: In pace with developments in the research field of artificial intelligence, knowledge graphs (KGs) have attracted a surge of interest from both academia and industry. As a representation of semantic relations between entities, KGs have proven to be particularly relevant for natural language processing (NLP), experiencing a rapid spread and wide adoption within recent years. Given the increasing amount of research work in this area, several KG-related approaches have been surveyed in the NLP research community. However, a comprehensive study that categorizes established topics and reviews the maturity of individual research streams remains absent to this day. Contributing to closing this gap, we systematically analyzed 507 papers from the literature on KGs in NLP. Our survey encompasses a multifaceted review of tasks, research types, and contributions. As a result, we present a structured overview of the research landscape, provide a taxonomy of tasks, summarize our findings, and highlight directions for future work.


title: A broad survey of spectro-temporal properties from FRB 20121102A

abstract: We survey the spectro-temporal properties of fast radio bursts from FRB 20121102A observed by earlier studies across a wide range of frequencies. We investigate 167 bursts from FRB 20121102A spanning frequencies 1--7.5GHz, durations of less than 1 ms to approximately 10 ms, with low and high energies, and with different wait-times. We find from this sample of bursts a strong agreement with the inverse relationship between sub-burst slope and duration and with other predictions made by the triggered relativistic dynamical model (TRDM). Earlier results found agreement with those predictions across three different repeating FRB sources. For this sample of bursts, we find that the sub-burst slope as well as the 'sad trombone' drift rate are consistent with being in a quadratic relationship with frequency and that both these quantities are inversely proportional to the duration. We also find that the duration decreases with increasing frequency as well as a statistically significant correlation between the sub-burst duration and bandwidth (proportional to $t^{-1/2}$) that is unexpected. No distinct group of bursts in this sample deviated from these relationships, however significant scatter can be seen in measurements. This study demonstrates the consistent existence of relationships between the spectro-temporal properties of bursts from a repeating FRB source. A simple explanation for the inverse relation between the sub-burst slope and duration is an inherently narrowband emission process. We make all measurements available as well as a graphical user interface called Frbgui developed and used to perform measurements of burst waterfalls.


title: Contrastive Corpus Attribution for Explaining Representations

abstract: Despite the widespread use of unsupervised models, very few methods are designed to explain them. Most explanation methods explain a scalar model output. However, unsupervised models output representation vectors, the elements of which are not good candidates to explain because they lack semantic meaning. To bridge this gap, recent works defined a scalar explanation output: a dot product-based similarity in the representation space to the sample being explained (i.e., an explicand). Although this enabled explanations of unsupervised models, the interpretation of this approach can still be opaque because similarity to the explicand's representation may not be meaningful to humans. To address this, we propose contrastive corpus similarity, a novel and semantically meaningful scalar explanation output based on a reference corpus and a contrasting foil set of samples. We demonstrate that contrastive corpus similarity is compatible with many post-hoc feature attribution methods to generate COntrastive COrpus Attributions (COCOA) and quantitatively verify that features important to the corpus are identified. We showcase the utility of COCOA in two ways: (i) we draw insights by explaining augmentations of the same image in a contrastive learning setting (SimCLR); and (ii) we perform zero-shot object localization by explaining the similarity of image representations to jointly learned text representations (CLIP).


title: ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks

abstract: Early backdoor attacks against machine learning set off an arms race in attack and defence development. Defences have since appeared demonstrating some ability to detect backdoors in models or even remove them. These defences work by inspecting the training data, the model, or the integrity of the training procedure. In this work, we show that backdoors can be added during compilation, circumventing any safeguards in the data preparation and model training stages. The attacker can not only insert existing weight-based backdoors during compilation, but also a new class of weight-independent backdoors, such as ImpNet. These backdoors are impossible to detect during the training or data preparation processes, because they are not yet present. Next, we demonstrate that some backdoors, including ImpNet, can only be reliably detected at the stage where they are inserted and removing them anywhere else presents a significant challenge. We conclude that ML model security requires assurance of provenance along the entire technical pipeline, including the data, model architecture, compiler, and hardware specification.