title: The Synergy of Speculative Decoding and Batching in Serving Large Language Models

abstract: Large Language Models (LLMs) like GPT are state-of-the-art text generation models that provide significant assistance in daily routines. However, LLM execution is inherently sequential, since they only produce one token at a time, thus incurring low hardware utilization on modern GPUs. Batching and speculative decoding are two techniques to improve GPU hardware utilization in LLM inference. To study their synergy, we implement a prototype implementation and perform an extensive characterization analysis on various LLM models and GPU architectures. We observe that the optimal speculation length depends on the batch size used. We analyze the key observation and build a quantitative model to explain it. Based on our analysis, we propose a new adaptive speculative decoding strategy that chooses the optimal speculation length for different batch sizes. Our evaluations show that our proposed method can achieve equal or better performance than the state-of-the-art speculation decoding schemes with fixed speculation length.


title: Stability of Random Forests and Coverage of Random-Forest Prediction Intervals

abstract: We establish stability of random forests under the mild condition that the squared response ($Y^2$) does not have a heavy tail. In particular, our analysis holds for the practical version of random forests that is implemented in popular packages like \texttt{randomForest} in \texttt{R}. Empirical results show that stability may persist even beyond our assumption and hold for heavy-tailed $Y^2$. Using the stability property, we prove a non-asymptotic lower bound for the coverage probability of prediction intervals constructed from the out-of-bag error of random forests. With another mild condition that is typically satisfied when $Y$ is continuous, we also establish a complementary upper bound, which can be similarly established for the jackknife prediction interval constructed from an arbitrary stable algorithm. We also discuss the asymptotic coverage probability under assumptions weaker than those considered in previous literature. Our work implies that random forests, with its stability property, is an effective machine learning method that can provide not only satisfactory point prediction but also justified interval prediction at almost no extra computational cost.


title: Rethinking Semi-Supervised Federated Learning: How to co-train fully-labeled and fully-unlabeled client imaging data

abstract: The most challenging, yet practical, setting of semi-supervised federated learning (SSFL) is where a few clients have fully labeled data whereas the other clients have fully unlabeled data. This is particularly common in healthcare settings where collaborating partners (typically hospitals) may have images but not annotations. The bottleneck in this setting is the joint training of labeled and unlabeled clients as the objective function for each client varies based on the availability of labels. This paper investigates an alternative way for effective training with labeled and unlabeled clients in a federated setting. We propose a novel learning scheme specifically designed for SSFL which we call Isolated Federated Learning (IsoFed) that circumvents the problem by avoiding simple averaging of supervised and semi-supervised models together. In particular, our training approach consists of two parts - (a) isolated aggregation of labeled and unlabeled client models, and (b) local self-supervised pretraining of isolated global models in all clients. We evaluate our model performance on medical image datasets of four different modalities publicly available within the biomedical image classification benchmark MedMNIST. We further vary the proportion of labeled clients and the degree of heterogeneity to demonstrate the effectiveness of the proposed method under varied experimental settings.


title: Adaptive Test-Time Personalization for Federated Learning

abstract: Personalized federated learning algorithms have shown promising results in adapting models to various distribution shifts. However, most of these methods require labeled data on testing clients for personalization, which is usually unavailable in real-world scenarios. In this paper, we introduce a novel setting called test-time personalized federated learning (TTPFL), where clients locally adapt a global model in an unsupervised way without relying on any labeled data during test-time. While traditional test-time adaptation (TTA) can be used in this scenario, most of them inherently assume training data come from a single domain, while they come from multiple clients (source domains) with different distributions. Overlooking these domain interrelationships can result in suboptimal generalization. Moreover, most TTA algorithms are designed for a specific kind of distribution shift and lack the flexibility to handle multiple kinds of distribution shifts in FL. In this paper, we find that this lack of flexibility partially results from their pre-defining which modules to adapt in the model. To tackle this challenge, we propose a novel algorithm called ATP to adaptively learns the adaptation rates for each module in the model from distribution shifts among source domains. Theoretical analysis proves the strong generalization of ATP. Extensive experiments demonstrate its superiority in handling various distribution shifts including label shift, image corruptions, and domain shift, outperforming existing TTA methods across multiple datasets and model architectures. Our code is available at https://github.com/baowenxuan/ATP .


title: Low-redshift Lyman Continuum Survey: Radio continuum properties of low-$z$ Lyman continuum emitters

abstract: Sources that leak Lyman-continuum (LyC) photons and lead to the reionisation of the universe are intensely studied using multiple observing facilities. Recently, the Low-redshift LyC Survey (LzLCS) has found the first large sample of LyC emitting galaxies at low redshift ($z\sim 0.3$) with the Hubble Space Telescope/Cosmic Origins Spectrograph. The LzLCS sample contains a robust estimate of the LyC escape fraction ($f_\mathrm{esc}^\mathrm{LyC}$) for 66 galaxies spanning a wide range of $f_\mathrm{esc}^\mathrm{LyC}$. Here we, for the first time, aim to study the radio continuum (RC) properties of LzLCS sources and their dependence on $f_\mathrm{esc}^\mathrm{LyC}$. We present Karl G. Jansky Very Large Array RC observations at C (4-8 GHz), S (2-4 GHz) and L (1-2 GHz) bands for a sub-sample of the LzLCS sources. The radio spectral index ($\alpha^{\mathrm{3GHz}}_\mathrm{6GHz}$) spans a wide range from being flat ( $\geq -0.1$) to very steep ($\leq -1.0$). The strongest leakers in our sample show flat $\alpha^{\mathrm{3GHz}}_\mathrm{6GHz}$, weak leakers have $\alpha^{\mathrm{3GHz}}_\mathrm{6GHz}$ close to normal star-forming galaxies, and non-leakers are characterized by steep $\alpha^{\mathrm{3GHz}}_\mathrm{6GHz}$. We argue that a combination of young ages, free-free absorption, and a flat cosmic-ray energy spectrum can altogether lead to a flat $\alpha^{\mathrm{3GHz}}_\mathrm{6GHz}$ for strong leakers. Non-leakers are characterized by steep spectra which can arise due to break/cutoff at high frequencies. Such a cutoff in the spectrum can arise in a single injection model of CRs characteristic of galaxies which have recently stopped star formation. Such a relation between $\alpha^{\mathrm{3GHz}}_\mathrm{6GHz}$ and $f_\mathrm{esc}^\mathrm{LyC}$ hints at the interesting role of supernovae, CRs, and magnetic fields in facilitating the escape ( and/or the lack) of LyC photons. (Abridged)


title: A thousand fermions in a 3D harmonic trap via Monte Carlo simulations

abstract: By use of a special wave function derived from similarly transformed propagators, this work shows that the energy of a thousand spin-balanced fermions in a three-dimensional harmonic potential can be accurately computed using the Monte Carlo method.


title: On the mapping class groups of simply-connected smooth 4-manifolds

abstract: The mapping class group $M(X)$ of a smooth manifold $X$ is the group of smooth isotopy classes of orientation preserving diffeomorphisms of $X$. We prove a number of results about the mapping class groups of compact, simply-connected, smooth $4$-manifolds. We prove that $M(X)$ is non-finitely generated for $X = 2n \mathbb{CP}^2 # 10n \overline{\mathbb{CP}^2}$, where $n \ge 3$ is odd. Let $\Gamma(X)$ denote the group of automorphisms of the intersection lattice of $X$ that can be realised by diffeomorphisms. Then $M(X)$ is an extension of $\Gamma(X)$ by $T(X)$, the Torelli group of isotopy classes of diffeomorphisms that act trivially in cohomology. We prove that this extension is split for connected sums of $\mathbb{CP}^2$, but is not split for $2\mathbb{CP}^2 # n \overline{\mathbb{CP}^2}$, where $n \ge 11$. We prove that the Nielsen realisation problem fails for certain finite subgroups of $M( p \mathbb{CP}^2 # q \overline{\mathbb{CP}^2} )$ whenever $p+q \ge 4$. Lastly we study the extension $M_1(X) \to M(X)$, where $M_1(X)$ is the group of isotopy classes of diffeomorphisms of $X$ which fix a neighbourhood of a point. When $X = K3$ or $K3 # (S^2 \times S^2)$ we prove that $M_1(X) \to M(X)$ is a non-trivial extension of $M(X)$ by $\mathbb{Z}_2$. Moreover, we completely determine the extension class of $M_1(K3) \to M(K3)$.


title: Demand-Side Threats to Power Grid Operations from IoT-Enabled Edge

abstract: The growing adoption of Internet-of-Things (IoT)-enabled energy smart appliances (ESAs) at the consumer end, such as smart heat pumps, electric vehicle chargers, etc., is seen as key to enabling demand-side response (DSR) services. However, these smart appliances are often poorly engineered from a security point of view and present a new threat to power grid operations. They may become convenient entry points for malicious parties to gain access to the system and disrupt important grid operations by abruptly changing the demand. Unlike utility-side and SCADA assets, ESAs are not monitored continuously due to their large numbers and the lack of extensive monitoring infrastructure at consumer sites. This article presents an in-depth analysis of the demand side threats to power grid operations including (i) an overview of the vulnerabilities in ESAs and the wider risk from the DSR ecosystem and (ii) key factors influencing the attack impact on power grid operations. Finally, it presents measures to improve the cyber-physical resilience of power grids, putting them in the context of ongoing efforts from the industry and regulatory bodies worldwide.


title: Efficient OPA tomography of non-Gaussian states of light

abstract: Current advances in nonlinear optics have made it possible to perform a homodyne-like tomography of an unknown state without highly efficient detectors or a strong local oscillator. Thereby, a new experimental direction has been opened into multimode and large-bandwidth quantum optics. An optical parametric amplifier (OPA) allows us to reconstruct the quadrature distribution of an unknown state directly from the measured intensity distribution with high precision. We propose adding a controllable displacement to the standard scheme, obtaining an improved method applicable even to asymmetric and non-Gaussian states while significantly increasing estimation accuracy and lowering the OPA amplification requirement. To demonstrate the power of our method, we accurately detect the sub-Planck phase-space structure by a distillable squeezing from the OPA estimates of various non-Gaussian states. With the improvements, OPA tomography became a generally applicable loss-tolerant and efficient alternative to OPA-assisted homodyne detection.


title: Donor-acceptor recombination emission in hydrogen-terminated nanodiamond: Novel single-photon source for room-temperature quantum photonics

abstract: In fluorescence spectra of nanodiamonds (NDs) synthesized at high pressure from adamantane and other organic compounds, very narrow (~1 nm) lines of unknown origin are observed in a wide spectroscopic range from ~500 to 800 nm. Here, we propose and experimentally substantiate the hypothesis that these mysterious lines arise from radiative recombination of donor-acceptor pairs (DAPs). To confirm our hypothesis, we study the fluorescence spectra of undoped and nitrogen-doped NDs of different sizes, before and after thermal oxidation of their surface. The results obtained with a high degree of confidence allowed us to conclude that the DAPs are formed through the interaction of donor-like substitutional nitrogen present in the diamond lattice, and a 2D layer of acceptors resulting from the transfer doping effect on the surface of hydrogen-terminated NDs. A specific behavior of the DAP-induced lines was discovered in the temperature range of 100-10 K: their energy increases and most lines are split into 2 or more components with decreasing temperature. It is shown that the majority of the studied DAP emitters are sources of single photons, with an emission rate of up to >1 million counts/s at room temperature, which significantly surpasses that of nitrogen-vacancy and silicon-vacancy centers under the same detection conditions. Despite an observed temporal instability in the emission, the DAP emitters of H-terminated NDs represent a powerful room-temperature single-photon source for quantum optical technologies.


title: Successfully Applying Lottery Ticket Hypothesis to Diffusion Model

abstract: Despite the success of diffusion models, the training and inference of diffusion models are notoriously expensive due to the long chain of the reverse process. In parallel, the Lottery Ticket Hypothesis (LTH) claims that there exists winning tickets (i.e., aproperly pruned sub-network together with original weight initialization) that can achieve performance competitive to the original dense neural network when trained in isolation. In this work, we for the first time apply LTH to diffusion models. We empirically find subnetworks at sparsity 90%-99% without compromising performance for denoising diffusion probabilistic models on benchmarks (CIFAR-10, CIFAR-100, MNIST). Moreover, existing LTH works identify the subnetworks with a unified sparsity along different layers. We observe that the similarity between two winning tickets of a model varies from block to block. Specifically, the upstream layers from two winning tickets for a model tend to be more similar than the downstream layers. Therefore, we propose to find the winning ticket with varying sparsity along different layers in the model. Experimental results demonstrate that our method can find sparser sub-models that require less memory for storage and reduce the necessary number of FLOPs. Codes are available at https://github.com/osier0524/Lottery-Ticket-to-DDPM.


title: Intrinsic Gaussian Vector Fields on Manifolds

abstract: Various applications ranging from robotics to climate science require modeling signals on non-Euclidean domains, such as the sphere. Gaussian process models on manifolds have recently been proposed for such tasks, in particular when uncertainty quantification is needed. In the manifold setting, vector-valued signals can behave very differently from scalar-valued ones, with much of the progress so far focused on modeling the latter. The former, however, are crucial for many applications, such as modeling wind speeds or force fields of unknown dynamical systems. In this paper, we propose novel Gaussian process models for vector-valued signals on manifolds that are intrinsically defined and account for the geometry of the space in consideration. We provide computational primitives needed to deploy the resulting Hodge-Mat\'ern Gaussian vector fields on the two-dimensional sphere and the hypertori. Further, we highlight two generalization directions: discrete two-dimensional meshes and "ideal" manifolds like hyperspheres, Lie groups, and homogeneous spaces. Finally, we show that our Gaussian vector fields constitute considerably more refined inductive biases than the extrinsic fields proposed before.


title: A Fuzzy Time Series-Based Model Using Particle Swarm Optimization and Weighted Rules

abstract: During the last decades, a myriad of fuzzy time series models have been proposed in scientific literature. Among the most accurate models found in fuzzy time series, the high-order ones are the most accurate. The research described in this paper tackles three potential limitations associated with the application of high-order fuzzy time series models. To begin with, the adequacy of forecast rules lacks consistency. Secondly, as the model's order increases, data utilization diminishes. Thirdly, the uniformity of forecast rules proves to be highly contingent on the chosen interval partitions. To address these likely drawbacks, we introduce a novel model based on fuzzy time series that amalgamates the principles of particle swarm optimization (PSO) and weighted summation. Our results show that our approach models accurately the time series in comparison with previous methods.


title: Nonlinear concentric water waves of moderate amplitude

abstract: We consider the outward-propagating nonlinear concentric water waves within the scope of the 2D Boussinesq system. The problem is axisymmetric, and we derive the slow radius versions of the cylindrical Korteweg - de Vries (cKdV) and extended cKdV (ecKdV) models. Numerical runs are initially performed using the full axisymmetric Boussinesq system. At some distance away from the origin, we use the numerical solution of the Boussinesq system as the "initial condition" for the derived cKdV and ecKdV models. We then compare the evolution of the waves as described by both reduced models and the direct numerical simulations of the axisymmetric Boussinesq system. The main conclusion of the paper is that the extended cKdV model provides a much more accurate description of the waves and extends the range of validity of the weakly-nonlinear modelling to the waves of moderate amplitude.


title: All Things Considered: Detecting Partisan Events from News Media with Cross-Article Comparison

abstract: Public opinion is shaped by the information news media provide, and that information in turn may be shaped by the ideological preferences of media outlets. But while much attention has been devoted to media bias via overt ideological language or topic selection, a more unobtrusive way in which the media shape opinion is via the strategic inclusion or omission of partisan events that may support one side or the other. We develop a latent variable-based framework to predict the ideology of news articles by comparing multiple articles on the same story and identifying partisan events whose inclusion or omission reveals ideology. Our experiments first validate the existence of partisan event selection, and then show that article alignment and cross-document comparison detect partisan events and article ideology better than competitive baselines. Our results reveal the high-level form of media bias, which is present even among mainstream media with strong norms of objectivity and nonpartisanship. Our codebase and dataset are available at https://github.com/launchnlp/ATC.


title: Kerr-Enhanced Optical Spring

abstract: We propose and experimentally demonstrate the generation of enhanced optical springs using the optical Kerr effect. A nonlinear optical crystal is inserted into a Fabry-Perot cavity with a movable mirror, and a chain of second-order nonlinear optical effects in the phase-mismatched condition induces the Kerr effect. The optical spring constant is enhanced by a factor of $1.6\pm0.1$ over linear theory. To our knowledge, this is the first realization of optomechanical coupling enhancement using a nonlinear optical effect, which has been theoretically investigated to overcome the performance limitations of linear optomechanical systems. The tunable nonlinearity of demonstrated system has a wide range of potential applications, from observing gravitational waves emitted by binary neutron star post-merger remnants to cooling macroscopic oscillators to their quantum ground state.


title: Abundant radiation of soft photons: a puzzle lasting four decades

abstract: The observed enhancement of low-kT photons in comparison with incorrect calculations, should not be treated as a puzzle. The paper by Low considered a large rapidity gap process of diffractive excitation of a hadron, h -> h+\gamma, rather than multiple hadron production spanning all over the rapidity interval between colliding hadrons. The optical theorem connects these two processes, and what is inner bremsstrahlung, suppressed according to Low, corresponds to radiation from final state hadrons. Thus, the main result of the Low theorem, based on gauge invariance of the diffractive bremsstrahlung amplitude, supplemented with the optical theorem, contradicts the so-called bremsstrahlung model. The latter has been used for comparison with data, leading to the longstanding soft photon puzzle.


title: Translating away Translationese without Parallel Data

abstract: Translated texts exhibit systematic linguistic differences compared to original texts in the same language, and these differences are referred to as translationese. Translationese has effects on various cross-lingual natural language processing tasks, potentially leading to biased results. In this paper, we explore a novel approach to reduce translationese in translated texts: translation-based style transfer. As there are no parallel human-translated and original data in the same language, we use a self-supervised approach that can learn from comparable (rather than parallel) mono-lingual original and translated data. However, even this self-supervised approach requires some parallel data for validation. We show how we can eliminate the need for parallel validation data by combining the self-supervised loss with an unsupervised loss. This unsupervised loss leverages the original language model loss over the style-transferred output and a semantic similarity loss between the input and style-transferred output. We evaluate our approach in terms of original vs. translationese binary classification in addition to measuring content preservation and target-style fluency. The results show that our approach is able to reduce translationese classifier accuracy to a level of a random classifier after style transfer while adequately preserving the content and fluency in the target original style.


title: Paired 2-disjoint path covers of burnt pancake graphs with faulty elements

abstract: The burnt pancake graph $BP_n$ is the Cayley graph of the hyperoctahedral group using prefix reversals as generators. Let $\{u,v\}$ and $\{x,y\}$ be any two pairs of distinct vertices of $BP_n$ for $n\geq 4$. We show that there are $u-v$ and $x-y$ paths whose vertices partition the vertex set of $BP_n$ even if $BP_n$ has up to $n-4$ faulty elements. On the other hand, for every $n\ge3$ there is a set of $n-2$ faulty edges or faulty vertices for which such a fault-free disjoint path cover does not exist.


title: Responsible AI (RAI) Games and Ensembles

abstract: Several recent works have studied the societal effects of AI; these include issues such as fairness, robustness, and safety. In many of these objectives, a learner seeks to minimize its worst-case loss over a set of predefined distributions (known as uncertainty sets), with usual examples being perturbed versions of the empirical distribution. In other words, aforementioned problems can be written as min-max problems over these uncertainty sets. In this work, we provide a general framework for studying these problems, which we refer to as Responsible AI (RAI) games. We provide two classes of algorithms for solving these games: (a) game-play based algorithms, and (b) greedy stagewise estimation algorithms. The former class is motivated by online learning and game theory, whereas the latter class is motivated by the classical statistical literature on boosting, and regression. We empirically demonstrate the applicability and competitive performance of our techniques for solving several RAI problems, particularly around subpopulation shift.


title: Model-based Control of the Scanning Tunneling Microscope: Enabling New Modes of Imaging, Spectroscopy, and Lithography

abstract: The invention of scanning tunneling microscope (STM) dates back to the work of Binnig and Rohrer in the early 1980s, whose seminal contribution was rewarded by the 1986 Nobel Prize in Physics for the design of the scanning tunneling microscope. Forty years later, the STM remains the best existing tool for studying electronic, chemical, and physical properties of conducting and semiconducting surfaces with atomic precision. It has opened entirely new fields of research, enabling scientists to gain invaluable insight into properties and structure of matter at the atomic scale. Recent breakthroughs in STM-based automated hydrogen depassivation lithography (HDL) on silicon have resulted in the STM being considered a viable tool for fabrication of error-free silicon-based quantum-electronic devices. Despite the STM's unique ability to interrogate and manipulate matter with atomic precision, it remains a challenging tool to use. It turns out that many issues can be traced back to the STM's feedback control system, which has remained essentially unchanged since its invention about 40 years ago. This article explains the role of feedback control system of the STM and reviews some of the recent progress made possible in imaging, spectroscopy, and lithography by making appropriate changes to the STM's feedback control loop. We believe that the full potential of the STM is yet to be realized, and the key to new innovations will be the application of advanced model-based control and estimation techniques to this system.


title: Automating the Correctness Assessment of AI-generated Code for Security Contexts

abstract: Evaluating the correctness of code generated by AI is a challenging open problem. In this paper, we propose a fully automated method, named ACCA, to evaluate the correctness of AI-generated code for security purposes. The method uses symbolic execution to assess whether the AI-generated code behaves as a reference implementation. We use ACCA to assess four state-of-the-art models trained to generate security-oriented assembly code and compare the results of the evaluation with different baseline solutions, including output similarity metrics, widely used in the field, and the well-known ChatGPT, the AI-powered language model developed by OpenAI. Our experiments show that our method outperforms the baseline solutions and assesses the correctness of the AI-generated code similar to the human-based evaluation, which is considered the ground truth for the assessment in the field. Moreover, ACCA has a very strong correlation with the human evaluation (Pearson's correlation coefficient r=0.84 on average). Finally, since it is a fully automated solution that does not require any human intervention, the proposed method performs the assessment of every code snippet in ~0.17s on average, which is definitely lower than the average time required by human analysts to manually inspect the code, based on our experience.


title: Experience-weighted attraction learning in network coordination games

abstract: This paper studies the action dynamics of network coordination games with bounded-rational agents. I apply the experience-weighted attraction (EWA) model to the analysis as the EWA model has several free parameters that can capture different aspects of agents' behavioural features. I show that the set of possible long-term action patterns can be largely different when the behavioural parameters vary, ranging from a unique possibility in which all agents favour the risk-dominant option to some set of outcomes richer than the collection of Nash equilibria. Monotonicity and non-monotonicity in the relationship between the number of possible long-term action profiles and the behavioural parameters are explored. I also study the question of influential agents in terms of whose initial predispositions are important to the actions of the whole network. The importance of agents can be represented by a left eigenvector of a Jacobian matrix provided that agents' initial attractions are close to some neutral level. Numerical calculations examine the predictive power of the eigenvector for the long-run action profile and how agents' influences are impacted by their behavioural features and network positions.


title: Design of Cluster-Randomized Trials with Cross-Cluster Interference

abstract: Cluster-randomized trials often involve units that are irregularly distributed in space without well-separated communities. In these settings, cluster construction is a critical aspect of the design due to the potential for cross-cluster interference. The existing literature relies on partial interference models, which take clusters as given and assume no cross-cluster interference. We relax this assumption by allowing interference to decay with geographic distance between units. This induces a bias-variance trade-off: constructing fewer, larger clusters reduces bias due to interference but increases variance. We propose new estimators that exclude units most potentially impacted by cross-cluster interference and show that this substantially reduces asymptotic bias relative to conventional difference-in-means estimators. We provide formal justification for a new design that chooses the number of clusters to balance the asymptotic bias and variance of our estimators and uses unsupervised learning to automate cluster construction.


title: Pair Plasma Cascade in Rotating Black Hole Magnetospheres with Split Monopole Flux Model

abstract: An electron-positron cascade in the magnetospheres of Kerr Black Holes (BH) is a fundamental ingredient to fueling the relativistic $\gamma$-ray jets seen at the polar regions of galactic supermassive BHs (SMBH). This leptonic cascade occurs in the "spark gap" region of a BH magnetosphere where the unscreen electric field parallel to the magnetic field is present, hence it is affected by the magnetic field structure. A previous study explored the case of a thin accretion disk, representative of Active Galactic Nuclei (AGN). Here we explore the case of a quasi-spherical gas distribution, as is expected to be present around the SMBH Sgr A* in the center of our Milky Way galaxy, for example. The properties and efficiency of the leptonic cascade are studied. The findings of our study and the implications for SMBH systems in various spectral and accretion states are discussed. The relationships and scalings derived from varying the mass of the BH and background photon spectra are further used to analyze the leptonic cascade process to power jets seen in astronomical observations. In particular, one finds the efficiency of the cascade in a quasi-spherical gas distribution peaks at the jet axis. Observationally, this should lead to a more prominent jet core, in contrast to the thin disk accretion case, where it peaks around the jet-disk interface. One also finds the spectrum of the background photons to play a key role. The cascade efficiency is maximum for a spectral index of two, while harder and softer spectra lead to a less efficient cascade.


title: High Q-factor diamond optomechanical resonators with silicon vacancy centers at millikelvin temperatures

abstract: Phonons are envisioned as coherent intermediaries between different types of quantum systems. Engineered nanoscale devices such as optomechanical crystals (OMCs) provide a platform to utilize phonons as quantum information carriers. Here we demonstrate OMCs in diamond designed for strong interactions between phonons and a silicon vacancy (SiV) spin. Using optical measurements at millikelvin temperatures, we measure a linewidth of 13 kHz (Q-factor of ~440,000) for 6 GHz acoustic modes, a record for diamond in the GHz frequency range and within an order of magnitude of state-of-the-art linewidths for OMCs in silicon. We investigate SiV optical and spin properties in these devices and outline a path towards a coherent spin-phonon interface.


title: The Telehealth Chain: a protocol for secure and transparent telemedicine transactions on the blockchain

abstract: Blockchain technology provides a secure and decentralized platform for storing and transferring sensitive medical data, which can be utilized to enable remote medical consultations. This paper proposes a theoretical framework for creating a blockchain-based digital entity to facilitate telemedicine services. The proposed framework utilizes blockchain technology to provide a secure and reliable platform for medical practitioners to remotely interact with patient transactions. The blockchain will serve as a one-stop digital service to secure patient data, ensure privacy, and facilitate payments. The proposed framework leverages the existing Hyperledger Fabric platform to build a secure blockchain-assisted telemedicine platform.


title: Customizing 360-Degree Panoramas through Text-to-Image Diffusion Models

abstract: Personalized text-to-image (T2I) synthesis based on diffusion models has attracted significant attention in recent research. However, existing methods primarily concentrate on customizing subjects or styles, neglecting the exploration of global geometry. In this study, we propose an approach that focuses on the customization of 360-degree panoramas, which inherently possess global geometric properties, using a T2I diffusion model. To achieve this, we curate a paired image-text dataset specifically designed for the task and subsequently employ it to fine-tune a pre-trained T2I diffusion model with LoRA. Nevertheless, the fine-tuned model alone does not ensure the continuity between the leftmost and rightmost sides of the synthesized images, a crucial characteristic of 360-degree panoramas. To address this issue, we propose a method called StitchDiffusion. Specifically, we perform pre-denoising operations twice at each time step of the denoising process on the stitch block consisting of the leftmost and rightmost image regions. Furthermore, a global cropping is adopted to synthesize seamless 360-degree panoramas. Experimental results demonstrate the effectiveness of our customized model combined with the proposed StitchDiffusion in generating high-quality 360-degree panoramic images. Moreover, our customized model exhibits exceptional generalization ability in producing scenes unseen in the fine-tuning dataset. Code is available at https://github.com/littlewhitesea/StitchDiffusion.


title: A randomized algorithm for nonconvex minimization with inexact evaluations and complexity guarantees

abstract: We consider minimization of a smooth nonconvex function with inexact oracle access to gradient and Hessian (without assuming access to the function value) to achieve approximate second-order optimality. A novel feature of our method is that if an approximate direction of negative curvature is chosen as the step, we choose its sense to be positive or negative with equal probability. We allow gradients to be inexact in a relative sense and relax the coupling between inexactness thresholds for the first- and second-order optimality conditions. Our convergence analysis includes both an expectation bound based on martingale analysis and a high-probability bound based on concentration inequalities. We apply our algorithm to empirical risk minimization problems and obtain improved gradient sample complexity over existing works.


title: Optimization of Nonlinear Turbulence in Stellarators

abstract: We present new stellarator equilibria that have been optimized for reduced turbulent transport using nonlinear gyrokinetic simulations within the optimization loop. The optimization routine involves coupling the pseudo-spectral GPU-native gyrokinetic code GX with the stellarator equilibrium and optimization code DESC. Since using GX allows for fast nonlinear simulations, we directly optimize for reduced nonlinear heat fluxes. To handle the noisy heat flux traces returned by these simulations, we employ the simultaneous perturbation stochastic approximation (SPSA) method that only uses two objective function evaluations for a simple estimate of the gradient. We show several examples that optimize for both reduced heat fluxes and good quasisymmetry as a proxy for low neoclassical transport. Finally, we run full transport simulations using the T3D stellarator transport code to evaluate the changes in the macroscopic profiles.


title: Ultralong-term high-density data storage with atomic defects in SiC

abstract: There is an urgent need to increase the global data storage capacity, as current approaches lag behind the exponential growth of data generation driven by the Internet, social media and cloud technologies. In addition to increasing storage density, new solutions should provide long-term data archiving that goes far beyond traditional magnetic memory, optical disks and solid-state drives. Here, we propose a concept of energy-efficient, ultralong, high-density data archiving based on optically active atomic-size defects in a radiation resistance material, silicon carbide (SiC). The information is written in these defects by focused ion beams and read using photoluminescence or cathodoluminescence. The temperature-dependent deactivation of these defects suggests a retention time minimum over a few generations under ambient conditions. With near-infrared laser excitation, grayscale encoding and multi-layer data storage, the areal density corresponds to that of Blu-ray discs. Furthermore, we demonstrate that the areal density limitation of conventional optical data storage media due to the light diffraction can be overcome by focused electron-beam excitation.


title: BanditPAM++: Faster $k$-medoids Clustering

abstract: Clustering is a fundamental task in data science with wide-ranging applications. In $k$-medoids clustering, cluster centers must be actual datapoints and arbitrary distance metrics may be used; these features allow for greater interpretability of the cluster centers and the clustering of exotic objects in $k$-medoids clustering, respectively. $k$-medoids clustering has recently grown in popularity due to the discovery of more efficient $k$-medoids algorithms. In particular, recent research has proposed BanditPAM, a randomized $k$-medoids algorithm with state-of-the-art complexity and clustering accuracy. In this paper, we present BanditPAM++, which accelerates BanditPAM via two algorithmic improvements, and is $O(k)$ faster than BanditPAM in complexity and substantially faster than BanditPAM in wall-clock runtime. First, we demonstrate that BanditPAM has a special structure that allows the reuse of clustering information $\textit{within}$ each iteration. Second, we demonstrate that BanditPAM has additional structure that permits the reuse of information $\textit{across}$ different iterations. These observations inspire our proposed algorithm, BanditPAM++, which returns the same clustering solutions as BanditPAM but often several times faster. For example, on the CIFAR10 dataset, BanditPAM++ returns the same results as BanditPAM but runs over 10$\times$ faster. Finally, we provide a high-performance C++ implementation of BanditPAM++, callable from Python and R, that may be of interest to practitioners at https://github.com/motiwari/BanditPAM. Auxiliary code to reproduce all of our experiments via a one-line script is available at https://github.com/ThrunGroup/BanditPAM_plusplus_experiments.


title: Application of Collaborative Learning Paradigms within Software Engineering Education: A Systematic Mapping Study

abstract: Collaboration is used in Software Engineering (SE) to develop software. Industry seeks SE graduates with collaboration skills to contribute to productive software development. SE educators can use Collaborative Learning (CL) to help students develop collaboration skills. This paper uses a Systematic Mapping Study (SMS) to examine the application of the CL educational theory in SE Education. The SMS identified 14 papers published between 2011 and 2022. We used qualitative analysis to classify the papers into four CL paradigms: Conditions, Effect, Interactions, and Computer-Supported Collaborative Learning (CSCL). We found a high interest in CSCL, with a shift in student interaction research to computer-mediated technologies. We discussed the 14 papers in depth, describing their goals and further analysing the CSCL research. Almost half the papers did not achieve the appropriate level of supporting evidence; however, calibrating the instruments presented could strengthen findings and support multiple CL paradigms, especially opportunities to learn at the social and community levels, where research was lacking. Though our results demonstrate limited CL educational theory applied in SE Education, we discuss future work to layer the theory on existing study designs for more effective teaching strategies.


title: INCODE: Implicit Neural Conditioning with Prior Knowledge Embeddings

abstract: Implicit Neural Representations (INRs) have revolutionized signal representation by leveraging neural networks to provide continuous and smooth representations of complex data. However, existing INRs face limitations in capturing fine-grained details, handling noise, and adapting to diverse signal types. To address these challenges, we introduce INCODE, a novel approach that enhances the control of the sinusoidal-based activation function in INRs using deep prior knowledge. INCODE comprises a harmonizer network and a composer network, where the harmonizer network dynamically adjusts key parameters of the activation function. Through a task-specific pre-trained model, INCODE adapts the task-specific parameters to optimize the representation process. Our approach not only excels in representation, but also extends its prowess to tackle complex tasks such as audio, image, and 3D shape reconstructions, as well as intricate challenges such as neural radiance fields (NeRFs), and inverse problems, including denoising, super-resolution, inpainting, and CT reconstruction. Through comprehensive experiments, INCODE demonstrates its superiority in terms of robustness, accuracy, quality, and convergence rate, broadening the scope of signal representation. Please visit the project's website for details on the proposed method and access to the code.


title: Bird's Eye View Based Pretrained World model for Visual Navigation

abstract: Sim2Real transfer has gained popularity because it helps transfer from inexpensive simulators to real world. This paper presents a novel system that fuses components in a traditional World Model into a robust system, trained entirely within a simulator, that Zero-Shot transfers to the real world. To facilitate transfer, we use an intermediary representation that is based on \textit{Bird's Eye View (BEV)} images. Thus, our robot learns to navigate in a simulator by first learning to translate from complex \textit{First-Person View (FPV)} based RGB images to BEV representations, then learning to navigate using those representations. Later, when tested in the real world, the robot uses the perception model that translates FPV-based RGB images to embeddings that were learned by the FPV to BEV translator and that can be used by the downstream policy. The incorporation of state-checking modules using \textit{Anchor images} and Mixture Density LSTM not only interpolates uncertain and missing observations but also enhances the robustness of the model in the real-world. We trained the model using data from a Differential drive robot in the CARLA simulator. Our methodology's effectiveness is shown through the deployment of trained models onto a real-world Differential drive robot. Lastly we release a comprehensive codebase, dataset and models for training and deployment (\url{https://sites.google.com/view/value-explicit-pretraining}).


title: Optimal concentration level of anisotropic Trudinger-Moser functionals on any bounded domain

abstract: Let $F$ be convex and homogeneous of degree $1$, its polar $F^{o}$ represent a finsler metric on $\mathbb{R}^{n}$, and $\Omega$ be any bounded open set in $\mathbb{R}^{n}$. In this paper, we first construct the theoretical structure of anisotropic harmonic transplantation. Using the anisotropic harmonic transplantation, co-area formula, limiting Sobolev approximation method, delicate estimate of level set of Green function, we investigate the optimal concentration level of the Trudinger-Moser functional \[ \int_{\Omega}e^{\lambda_{n}|u|^{\frac{n}{n-1}}}dx \] under the anisotropic Dirichlet norm constraint $\int_{\Omega}F^{n}\left( \nabla{{u}}\right) dx\leq1$, where $\lambda_{n}=n^{\frac{n}{n-1}}\kappa _{n}^{\frac{1}{n-1}}\ $ denotes the sharp constant of anisotropic Trudinger-Moser inequality in bounded domain and $\kappa_{n}$ is the Lebesgue measure of the unit Wulff ball. As an application. we can immediately deduce the existence of extremals for anisotropic Trudinger-Moser inequality on bounded domain. Finally, we also consider the optimal concentration level of the anisotropic singular Trudinger-Moser functional. The method is based on the limiting Hardy-Sobolev approximation method and constructing a suitable normalized anisotropic concentrating sequence.


title: Deep Learning-based Compressed Domain Multimedia for Man and Machine: A Taxonomy and Application to Point Cloud Classification

abstract: In the current golden age of multimedia, human visualization is no longer the single main target, with the final consumer often being a machine which performs some processing or computer vision tasks. In both cases, deep learning plays a undamental role in extracting features from the multimedia representation data, usually producing a compressed representation referred to as latent representation. The increasing development and adoption of deep learning-based solutions in a wide area of multimedia applications have opened an exciting new vision where a common compressed multimedia representation is used for both man and machine. The main benefits of this vision are two-fold: i) improved performance for the computer vision tasks, since the effects of coding artifacts are mitigated; and ii) reduced computational complexity, since prior decoding is not required. This paper proposes the first taxonomy for designing compressed domain computer vision solutions driven by the architecture and weights compatibility with an available spatio-temporal computer vision processor. The potential of the proposed taxonomy is demonstrated for the specific case of point cloud classification by designing novel compressed domain processors using the JPEG Pleno Point Cloud Coding standard under development and adaptations of the PointGrid classifier. Experimental results show that the designed compressed domain point cloud classification solutions can significantly outperform the spatial-temporal domain classification benchmarks when applied to the decompressed data, containing coding artifacts, and even surpass their performance when applied to the original uncompressed data.


title: Exploring Data Augmentations on Self-/Semi-/Fully- Supervised Pre-trained Models

abstract: Data augmentation has become a standard component of vision pre-trained models to capture the invariance between augmented views. In practice, augmentation techniques that mask regions of a sample with zero/mean values or patches from other samples are commonly employed in pre-trained models with self-/semi-/fully-supervised contrastive losses. However, the underlying mechanism behind the effectiveness of these augmentation techniques remains poorly explored. To investigate the problems, we conduct an empirical study to quantify how data augmentation affects performance. Concretely, we apply 4 types of data augmentations termed with Random Erasing, CutOut, CutMix and MixUp to a series of self-/semi-/fully- supervised pre-trained models. We report their performance on vision tasks such as image classification, object detection, instance segmentation, and semantic segmentation. We then explicitly evaluate the invariance and diversity of the feature embedding. We observe that: 1) Masking regions of the images decreases the invariance of the learned feature embedding while providing a more considerable diversity. 2) Manual annotations do not change the invariance or diversity of the learned feature embedding. 3) The MixUp approach improves the diversity significantly, with only a marginal decrease in terms of the invariance.


title: Probing Parity Manifest Minimal Left-Right Symmetric Model through $CP$ Violation & Anomalous Magnetic Moment of Charged Leptons Constrained by Lepton Flavor Violating $l_i \rightarrow l_j\gamma$ & $l_i \rightarrow 3l_j$ Channels

abstract: While the Standard Model remains the prevailing description of natural phenomena, several observed phenomena continue to elude its explanation. To address these challenges, we investigate the Minimal Left-Right Symmetric Model, an immediate extension of the Standard Model. This model adeptly resolves issues related to parity violation and neutrino mass smallness by incorporating the seesaw mechanism. Our investigation focuses on assessing the Parity Manifest Minimal Left-Right Symmetric Model's contributions to specific phenomena, namely the anomalous magnetic moment (AMM) and CP violation of charged leptons. First, the model's coupling parameter space is derived from the experimental bounds of charged lepton flavor-violating $l_i \rightarrow l_j\gamma$ and $l_i \rightarrow 3l_j$ channels. These bounds emanate from the extended Higgs sector of the model. Subsequently, we evaluate the one-loop contributions of the model to magnetic dipole moment (MDM) and electric dipole moment (EDM), which provide insights into the model's estimations for AMM and CP violation of charged leptons based on the experimental bounds of the aforementioned processes. The outcomes of this analysis will shed light on whether the Parity Manifest Minimal Left-Right Symmetric Model stands as a highly promising candidate for physics beyond the Standard Model.


title: AI for Open Science: A Multi-Agent Perspective for Ethically Translating Data to Knowledge

abstract: AI for Science (AI4Science), particularly in the form of self-driving labs, has the potential to sideline human involvement and hinder scientific discovery within the broader community. While prior research has focused on ensuring the responsible deployment of AI applications, enhancing security, and ensuring interpretability, we also propose that promoting openness in AI4Science discoveries should be carefully considered. In this paper, we introduce the concept of AI for Open Science (AI4OS) as a multi-agent extension of AI4Science with the core principle of maximizing open knowledge translation throughout the scientific enterprise rather than a single organizational unit. We use the established principles of Knowledge Discovery and Data Mining (KDD) to formalize a language around AI4OS. We then discuss three principle stages of knowledge translation embedded in AI4Science systems and detail specific points where openness can be applied to yield an AI4OS alternative. Lastly, we formulate a theoretical metric to assess AI4OS with a supporting ethical argument highlighting its importance. Our goal is that by drawing attention to AI4OS we can ensure the natural consequence of AI4Science (e.g., self-driving labs) is a benefit not only for its developers but for society as a whole.


title: Liquid Hopfield model: retrieval and localization in heterogeneous liquid mixtures

abstract: Biological mixtures, such as the cellular cytoplasm, are composed of a large number of different components. From this heterogeneity, ordered mesoscopic structures emerge, such as liquid phases with controlled composition. These structures compete with each other for the same components. This raises several questions, such as what types of interactions allow the retrieval of multiple ordered mesoscopic structures, and what are the physical limitations for the retrieval of said structures. In this work, we develop an analytically tractable model for liquids capable of retrieving states with target compositions. We name this model the liquid Hopfield model in reference to corresponding work in the theory of associative neural networks. By solving this model, we show that non-linear repulsive interactions are necessary for retrieval of target structures. We demonstrate that this is because liquid mixtures at low temperatures tend to transition to phases with few components, a phenomenon that we term localization. Taken together, our results demonstrate a trade-off between retrieval and localization phenomena in liquid mixtures.

categories: physics.bio-ph cond-mat.dis-nn cond-mat.soft q-bio.BM


title: On the role of postulates in the understanding of Weyl gauge symmetry

abstract: Understanding of Weyl gauge symmetry is rarely associated with underlying postulates. Here we show that such an omission leads to discrepancies in regard to the reach and consequences of gauge symmetry within gravitational theories. Replacement of the postulates which underlie the conventional approach, by just the opposite assumptions, leads to a radically different interpretation of Weyl gauge symmetry, which shares the same mathematical foundations and carries observational consequences.


title: Ergodic theory on coded shift spaces

abstract: We study ergodic-theoretic properties of coded shift spaces. A coded shift space is defined as a closure of all bi-infinite concatenations of words from a fixed countable generating set. We derive sufficient conditions for the uniqueness of measures of maximal entropy and equilibrium states of H\"{o}lder continuous potentials based on the partition of the coded shift into its sequential set (sequences that are concatenations of generating words) and its residual set (sequences added under the closure). In this case we provide a simple explicit description of the measure of maximal entropy. We also obtain flexibility results for the entropy on the sequential and residual set. Finally, we prove a local structure theorem for intrinsically ergodic coded shift spaces which shows that our results apply to a larger class of coded shift spaces compared to previous works by Climenhaga, Climenhaga and Thompson, and Pavlov.


title: Stochastic modeling of superconducting qudits in the dispersive regime

abstract: The field of superconducting quantum computing, based on Josephson junctions, has recently seen remarkable strides in scaling the number of logical qubits. In particular, the fidelities of one- and two-qubit gates have reached the breakeven point with the novel error mitigation and correction methods. Parallel to these advances is the effort to expand the Hilbert space within a single junction or device by employing high-dimensional qubits, otherwise known as qudits. Research has demonstrated the possibility of driving higher-order transitions in a transmon or designing innovative multimode superconducting circuits, termed multimons. These advances can significantly expand the computational basis while simplifying the interconnects in a large-scale quantum processor. In this work we extend the measurement theory of a conventional superconducting qubit to that of a qudit, focusing on modeling the dispersive quadrature measurement in an open quantum system. Under the Markov assumption, the qudit Lindblad and stochastic master equations are formulated and analyzed; in addition, both the ensemble-averaged and the quantum-jump approach of decoherence analysis are detailed with analytical and numerical comparisons. We verify our stochastic model with a series of experimental results on a transmon-type qutrit, verifying the validity of our high-dimensional formalism.


title: Toward Local Madelung Mechanics in Spacetime

abstract: It has recently been shown that relativistic quantum theory leads to a local interpretation of quantum mechanics wherein the universal wavefunction in configuration space is entirely replaced with an ensemble of local fluid equations in spacetime. For want of a fully relativistic quantum fluid treatment, we develop a model using the nonrelativistic Madelung equations, and obtain conditions for them to be local in spacetime. Every particle in the Madelung fluid is equally real, and has a definite position, momentum, kinetic energy, and potential energy. These are obtained by defining quantum momentum and kinetic energy densities for the fluid and separating the momentum into average and symmetric parts, and kinetic energy into classical kinetic and quantum potential parts. The two types of momentum naturally give rise to a single classical kinetic energy density, which contains the expected kinetic energy, even for stationary states, and we define the reduced quantum potential as the remaining part of the quantum kinetic energy density. We treat the quantum potential as a novel mode of internal energy storage within the fluid particles, which explains most of the nonclassical behavior of the Madelung fluid. For example, we show that in tunneling phenomena the quantum potential negates the barrier so that nothing prevents the fluid from flowing through. We show how energy flows and transforms in this model, and that enabling local conservation of energy requires defining a quantum potential energy current that flows through the fluid rather than only flowing with it. The nonrelativistic treatment generally contains singularities in the velocity field, which undermines the goal of local dynamics, but we expect a proper relativistic treatment will bound the fluid particle velocities at $c$.


title: Estimating a function of the scale parameter in a gamma distribution with bounded variance

abstract: Given a gamma population with known shape parameter $\alpha$, we develop a general theory for estimating a function $g(\cdot)$ of the scale parameter $\beta$ with bounded variance. We begin by defining a sequential sampling procedure with $g(\cdot)$ satisfying some desired condition in proposing the stopping rule, and show the procedure enjoys appealing asymptotic properties. After these general conditions, we substitute $g(\cdot)$ with specific functions including the gamma mean, the gamma variance, the gamma rate parameter, and a gamma survival probability as four possible illustrations. For each illustration, Monte Carlo simulations are carried out to justify the remarkable performance of our proposed sequential procedure. This is further substantiated with a real data study on weights of newly born babies.


title: SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models

abstract: Mixture-of-Experts (MoE) has emerged as a favorable architecture in the era of large models due to its inherent advantage, i.e., enlarging model capacity without incurring notable computational overhead. Yet, the realization of such benefits often results in ineffective GPU memory utilization, as large portions of the model parameters remain dormant during inference. Moreover, the memory demands of large models consistently outpace the memory capacity of contemporary GPUs. Addressing this, we introduce SiDA-MoE ($\textbf{S}$parsity-$\textbf{i}$nspired $\textbf{D}$ata-$\textbf{A}$ware), an efficient inference approach tailored for large MoE models. SiDA-MoE judiciously exploits both the system's main memory, which is now abundant and readily scalable, and GPU memory by capitalizing on the inherent sparsity on expert activation in MoE models. By adopting a data-aware perspective, SiDA-MoE achieves enhanced model efficiency with a neglectable performance drop. Specifically, SiDA-MoE attains a remarkable speedup in MoE inference with up to $3.93\times$ throughput increasing, up to $72\%$ latency reduction, and up to $80\%$ GPU memory saving with down to $1\%$ performance drop. This work paves the way for scalable and efficient deployment of large MoE models, even with constrained resources. Code is available at: https://github.com/timlee0212/SiDA-MoE.


title: Bayes beats Cross Validation: Efficient and Accurate Ridge Regression via Expectation Maximization

abstract: We present a novel method for tuning the regularization hyper-parameter, $\lambda$, of a ridge regression that is faster to compute than leave-one-out cross-validation (LOOCV) while yielding estimates of the regression parameters of equal, or particularly in the setting of sparse covariates, superior quality to those obtained by minimising the LOOCV risk. The LOOCV risk can suffer from multiple and bad local minima for finite $n$ and thus requires the specification of a set of candidate $\lambda$, which can fail to provide good solutions. In contrast, we show that the proposed method is guaranteed to find a unique optimal solution for large enough $n$, under relatively mild conditions, without requiring the specification of any difficult to determine hyper-parameters. This is based on a Bayesian formulation of ridge regression that we prove to have a unimodal posterior for large enough $n$, allowing for both the optimal $\lambda$ and the regression coefficients to be jointly learned within an iterative expectation maximization (EM) procedure. Importantly, we show that by utilizing an appropriate preprocessing step, a single iteration of the main EM loop can be implemented in $O(\min(n, p))$ operations, for input data with $n$ rows and $p$ columns. In contrast, evaluating a single value of $\lambda$ using fast LOOCV costs $O(n \min(n, p))$ operations when using the same preprocessing. This advantage amounts to an asymptotic improvement of a factor of $l$ for $l$ candidate values for $\lambda$ (in the regime $q, p \in O(\sqrt{n})$ where $q$ is the number of regression targets).


title: Peer-to-Peer Deep Learning for Beyond-5G IoT

abstract: We present P2PL, a practical multi-device peer-to-peer deep learning algorithm that, unlike the federated learning paradigm, does not require coordination from edge servers or the cloud. This makes P2PL well-suited for the sheer scale of beyond-5G computing environments like smart cities that otherwise create range, latency, bandwidth, and single point of failure issues for federated approaches. P2PL introduces max norm synchronization to catalyze training, retains on-device deep model training to preserve privacy, and leverages local inter-device communication to implement distributed consensus. Each device iteratively alternates between two phases: 1) on-device learning and 2) peer-to-peer cooperation where they combine model parameters with nearby devices. We empirically show that all participating devices achieve the same test performance attained by federated and centralized training -- even with 100 devices and relaxed singly stochastic consensus weights. We extend these experimental results to settings with diverse network topologies, sparse and intermittent communication, and non-IID data distributions.


title: Counterfactually Probing Language Identity in Multilingual Models

abstract: Techniques in causal analysis of language models illuminate how linguistic information is organized in LLMs. We use one such technique, AlterRep, a method of counterfactual probing, to explore the internal structure of multilingual models (mBERT and XLM-R). We train a linear classifier on a binary language identity task, to classify tokens between Language X and Language Y. Applying a counterfactual probing procedure, we use the classifier weights to project the embeddings into the null space and push the resulting embeddings either in the direction of Language X or Language Y. Then we evaluate on a masked language modeling task. We find that, given a template in Language X, pushing towards Language Y systematically increases the probability of Language Y words, above and beyond a third-party control language. But it does not specifically push the model towards translation-equivalent words in Language Y. Pushing towards Language X (the same direction as the template) has a minimal effect, but somewhat degrades these models. Overall, we take these results as further evidence of the rich structure of massive multilingual language models, which include both a language-specific and language-general component. And we show that counterfactual probing can be fruitfully applied to multilingual models.


title: The diminishing state of shared reality on US television news

abstract: The potential for a large, diverse population to coexist peacefully is thought to depend on the existence of a ``shared reality:'' a public sphere in which participants are exposed to similar facts about similar topics. A generation ago, broadcast television news was widely considered to serve this function; however, since the rise of cable news in the 1990s, critics and scholars have worried that the corresponding fragmentation and segregation of audiences along partisan lines has caused this shared reality to be lost. Here we examine this concern using a unique combination of data sets tracking the production (since 2012) and consumption (since 2016) of television news content on the three largest cable and broadcast networks respectively. With regard to production, we find strong evidence for the ``loss of shared reality hypothesis:'' while broadcast continues to cover similar topics with similar language, cable news networks have become increasingly distinct, both from broadcast news and each other, diverging both in terms of content and language. With regard to consumption, we find more mixed evidence: while broadcast news has indeed declined in popularity, it remains the dominant source of news for roughly 50\% more Americans than does cable; moreover, its decline, while somewhat attributable to cable, appears driven more by a shift away from news consumption altogether than a growth in cable consumption. We conclude that shared reality on US television news is indeed diminishing, but is more robust than previously thought and is declining for somewhat different reasons.


title: $T$-matrix Analysis of Static Wilson Line Correlators from Lattice QCD at Finite Temperature

abstract: We utilize a previously constructed thermodynamic $T$-matrix approach to the quark-gluon plasma (QGP) to calculate Wilson line correlators (WLCs) of a static quark-antiquark pair and apply them to the results from 2+1-flavor lattice-QCD (lQCD) computations with realistic pion mass. The self-consistent $T$-matrix results, which include constraints from the lQCD equation of state in the light-parton sector, can describe the lQCD data for WLCs fairly well once refinements of the input parameters are implemented. In particular, the input potential requires less screening than used in previous $T$-matrix analyses. Pertinent predictions for the spectral and transport properties of the QGP are discussed, including the spatial diffusion coefficient for heavy quarks which turns out to have a rather weak temperature dependence, in approximate agreement with recent lQCD results.


title: MUST: A Multilingual Student-Teacher Learning approach for low-resource speech recognition

abstract: Student-teacher learning or knowledge distillation (KD) has been previously used to address data scarcity issue for training of speech recognition (ASR) systems. However, a limitation of KD training is that the student model classes must be a proper or improper subset of the teacher model classes. It prevents distillation from even acoustically similar languages if the character sets are not same. In this work, the aforementioned limitation is addressed by proposing a MUltilingual Student-Teacher (MUST) learning which exploits a posteriors mapping approach. A pre-trained mapping model is used to map posteriors from a teacher language to the student language ASR. These mapped posteriors are used as soft labels for KD learning. Various teacher ensemble schemes are experimented to train an ASR model for low-resource languages. A model trained with MUST learning reduces relative character error rate (CER) up to 9.5% in comparison with a baseline monolingual ASR.


title: Astrometry with PRAIA

abstract: PRAIA - Package for the Reduction of Astronomical Images Automatically - is a suite of astrometric and photometric tasks designed to cope with huge amounts of heterogeneous observations with fast processing, no human intervention, minimum parametrization and yet maximum possible accuracy and precision. It is the main tool used to analyse astronomical observations by an international collaboration involving Brazilian, French and Spanish researchers under the Lucky Star umbrella for Solar System studies. In this paper, we focus on the astrometric concepts underneath PRAIA, used in reference system works, natural satellite and NEA astrometry for dynamical and ephemeris studies, and lately for the precise prediction of stellar occultations by planetary satellites, dwarf-planets, TNOs, Centaurs and Trojan asteroids. We highlight novelties developed by us and never reported before in the literature, which significantly enhance astrometry precision and automation. Such as the robust object detection and aperture characterization (BOIA), which explains the long standing empirical photometry/astrometry axiom that recommends using apertures with 2 - 3 sigma (Gaussian width) radius. We give examples showing the astrometry performance, discuss the advantages of PRAIA over other astrometry packages and comment about future planed astrometry implementations. PRAIA codes and input files are publicly available for the first time at: https://ov.ufrj.br/en/PRAIA/. PRAIA astrometry is useful for Solar System as well as astrophysical observations.


title: Prompt-Engineering and Transformer-based Question Generation and Evaluation

abstract: Question generation has numerous applications in the educational context. Question generation can prove helpful for students when reviewing content and testing themselves. Furthermore, a question generation model can aid teachers by lessening the burden of creating assessments and other practice material. This paper aims to find the best method to generate questions from textual data through a transformer model and prompt engineering. In this research, we finetuned a pretrained distilBERT model on the SQuAD question answering dataset to generate questions. In addition to training a transformer model, prompt engineering was applied to generate questions effectively using the LLaMA model. The generated questions were compared against the baseline questions in the SQuAD dataset to evaluate the effectiveness of four different prompts. All four prompts demonstrated over 60% similarity on average. Of the prompt-generated questions, 30% achieved a high similarity score greater than 70%.


title: Correlation Aware Sparsified Mean Estimation Using Random Projection

abstract: We study the problem of communication-efficient distributed vector mean estimation, a commonly used subroutine in distributed optimization and Federated Learning (FL). Rand-$k$ sparsification is a commonly used technique to reduce communication cost, where each client sends $k < d$ of its coordinates to the server. However, Rand-$k$ is agnostic to any correlations, that might exist between clients in practical scenarios. The recently proposed Rand-$k$-Spatial estimator leverages the cross-client correlation information at the server to improve Rand-$k$'s performance. Yet, the performance of Rand-$k$-Spatial is suboptimal. We propose the Rand-Proj-Spatial estimator with a more flexible encoding-decoding procedure, which generalizes the encoding of Rand-$k$ by projecting the client vectors to a random $k$-dimensional subspace. We utilize Subsampled Randomized Hadamard Transform (SRHT) as the projection matrix and show that Rand-Proj-Spatial with SRHT outperforms Rand-$k$-Spatial, using the correlation information more efficiently. Furthermore, we propose an approach to incorporate varying degrees of correlation and suggest a practical variant of Rand-Proj-Spatial when the correlation information is not available to the server. Experiments on real-world distributed optimization tasks showcase the superior performance of Rand-Proj-Spatial compared to Rand-$k$-Spatial and other more sophisticated sparsification techniques.


title: Comparison of integral structures on the space of modular forms of full level N

abstract: Let $N\geq3$ and $r\geq1$ be integers and $p\geq2$ be a prime such that $p\nmid N$. One can consider two different integral structures on the space of modular forms over $\mathbb{Q}$, one coming from arithmetic via $q$-expansions, the other coming from geometry via integral models of modular curves. Both structures are stable under the Hecke operators; furthermore, their quotient is finite torsion. Our goal is to investigate the exponent of the annihilator of the quotient. We will apply methods due to Brian Conrad to the situation of modular forms of even weight and level $\Gamma(Np^{r})$ over $\mathbb{Q}_{p}(\zeta_{Np^{r}})$ to obtain an upper bound for the exponent. We also use Klein forms to construct explicit modular forms of level $p^{r}$ whenever $p^{r}>3$, allowing us to compute a lower bound which agrees with the upper bound. Hence we are able to compute the exponent precisely.


title: Hunter self-similar implosion profiles for the gravitational Euler-Poisson system

abstract: Our result is a construction of infinitely many radial self-similar implosion profiles for the gravitational Euler-Poisson system. The problem can be expressed as solving a system of non-autonomous non-linear ODEs. The first rigorous existence result for a non-trivial solution to these ODEs is due to Guo, Had\v{z}i\'c and Jang [Comm. Math. Phys. 386.3 (2021)], in which they construct a solution found numerically by Larson and Penston independently [Monthly Notices of the Royal Astronomical Society 145.3 (1969), Monthly Notices of the Royal Astronomical Society 144.4 (1969)]. The solutions we construct belong to a different regime and correspond to a strict subset of the family of profiles discovered numerically by Hunter. Our proof adapts a technique developed by Collot, Rapha\"el and Szeftel in [Mem. Amer. Math. Soc. 260.1255 (2019)], in which they study blowup for a family of energy-supercritical focusing semilinear heat equations. In our case, the quasilinearity presents complications, most severely near the sonic point where the system degenerates.


title: Compressed Gradient Tracking Algorithms for Distributed Nonconvex Optimization

abstract: In this paper, we study the distributed nonconvex optimization problem, which aims to minimize the average value of the local nonconvex cost functions using local information exchange. To reduce the communication overhead, we introduce three general classes of compressors, i.e., compressors with bounded relative compression error, compressors with globally bounded absolute compression error, and compressors with locally bounded absolute compression error. By integrating them with distributed gradient tracking algorithm, we then propose three compressed distributed nonconvex optimization algorithms. For each algorithm, we design a novel Lyapunov function to demonstrate its sublinear convergence to a stationary point if the local cost functions are smooth. Furthermore, when the global cost function satisfies the Polyak--{\L}ojasiewicz (P--{\L}) condition, we show that our proposed algorithms linearly converge to a global optimal point. It is worth noting that, for compressors with bounded relative compression error and globally bounded absolute compression error, our proposed algorithms' parameters do not require prior knowledge of the P--{\L} constant. The theoretical results are illustrated by numerical examples, which demonstrate the effectiveness of the proposed algorithms in significantly reducing the communication burden while maintaining the convergence performance. Moreover, simulation results show that the proposed algorithms outperform state-of-the-art compressed distributed nonconvex optimization algorithms.


title: Lambda bracket and Intertwiners

abstract: We describe the intertwiners between modules of a vertex algebra using the language of lambda bracket. We apply this formalism to obtain some classical results on conformal field theory.


title: Combined exponential patterns in multiplicative $IP^{\star}$ sets

abstract: $IP$ sets play fundamental role in arithmetic Ramsey theory. A set is called an additive $IP$ set if it is of the form $FS\left(\langle x_{n}\rangle_{n\in \mathbb{N}}\right)=\left\{ \sum_{t\in H}x_{t}:H\right.$ is a nonempty finite subset of $\left.\mathbb{N}\right\}$, whereas it is called a multiplicative $IP$ set if it is of the form $FP\left(\langle x_{n}\rangle_{n\in \mathbb{N}}\right)=\left\{ \prod_{t\in H}x_{t}:H\right.$ is a nonempty finite subset of $\left. \mathbb{N}\right\}$ for some injective sequence $\langle x_{n}\rangle_{n\in \mathbb{N}}.$ An additive $IP^{\star}$ (resp. multiplicative $IP^{\star}$) set is a set which intersects every additive $IP$ set (resp. multiplicative $IP$ set). In \cite{key-1}, V. Bergelson and N. Hindman studied how rich additive $IP^{\star}$ sets are. They proved additive $IP^{\star}$ sets ($AIP^{\star}$ in short) contain finite sums and finite products of a single sequence. An analogous study was made by A. Sisto in\cite{key-3}, where he proved that multiplicative $IP^{\star}$ sets ($MIP^{\star}$ in short) contain exponential tower\footnote{will be defined later} and finite product of a single sequence. However exponential patterns can be defined in two different ways. In this article we will prove that $MIP^{\star}$ sets contain two different exponential patterns and finite product of a single sequence. This immediately improves the result of A. Sisto. We also construct a $MIP^\star$ set, not arising from the recurrence of measurable dynamical systems. Throughout our work we will use the machinery of the algebra of the Stone-\v{C}ech Compactification of $\mathbb{N}$.


title: HDMNet: A Hierarchical Matching Network with Double Attention for Large-scale Outdoor LiDAR Point Cloud Registration

abstract: Outdoor LiDAR point clouds are typically large-scale and complexly distributed. To achieve efficient and accurate registration, emphasizing the similarity among local regions and prioritizing global local-to-local matching is of utmost importance, subsequent to which accuracy can be enhanced through cost-effective fine registration. In this paper, a novel hierarchical neural network with double attention named HDMNet is proposed for large-scale outdoor LiDAR point cloud registration. Specifically, A novel feature consistency enhanced double-soft matching network is introduced to achieve two-stage matching with high flexibility while enlarging the receptive field with high efficiency in a patch-to patch manner, which significantly improves the registration performance. Moreover, in order to further utilize the sparse matching information from deeper layer, we develop a novel trainable embedding mask to incorporate the confidence scores of correspondences obtained from pose estimation of deeper layer, eliminating additional computations. The high-confidence keypoints in the sparser point cloud of the deeper layer correspond to a high-confidence spatial neighborhood region in shallower layer, which will receive more attention, while the features of non-key regions will be masked. Extensive experiments are conducted on two large-scale outdoor LiDAR point cloud datasets to demonstrate the high accuracy and efficiency of the proposed HDMNet.


title: Feature calibration for computer models

abstract: Computer model calibration involves using partial and imperfect observations of the real world to learn which values of a model's input parameters lead to outputs that are consistent with real-world observations. When calibrating models with high-dimensional output (e.g. a spatial field), it is common to represent the output as a linear combination of a small set of basis vectors. Often, when trying to calibrate to such output, what is important to the credibility of the model is that key emergent physical phenomena are represented, even if not faithfully or in the right place. In these cases, comparison of model output and data in a linear subspace is inappropriate and will usually lead to poor model calibration. To overcome this, we present kernel-based history matching (KHM), generalising the meaning of the technique sufficiently to be able to project model outputs and observations into a higher-dimensional feature space, where patterns can be compared without their location necessarily being fixed. We develop the technical methodology, present an expert-driven kernel selection algorithm, and then apply the techniques to the calibration of boundary layer clouds for the French climate model IPSL-CM.


title: Thermal conductivity of MgO in giant planetary interior conditions predicted by deep potential

abstract: Thermal conductivity $\kappa$ of MgO plays a fundamental role in understanding the thermal evolution and mantle convection in the interior of terrestrial planets. However, previous theoretical calculations deviate from each other and the $\kappa$ of high-pressure B2 phase remains undetermined. Here, by combining molecular dynamics and deep potential trained with first-principles data, we systematically investigate the $\kappa$ of MgO from ambient state to the core-mantle boundary (CMB) of super-Earth with $5M_{\oplus}$. We point out the significance of 4-phonon scatterings and modify the conventional thermal conductivity model of MgO by considering the density-dependent proportion of 3-phonon and 4-phonon scatterings. The $\kappa$ profiles of MgO in Earth and super-Earth are further estimated. For super-Earth, we predict a significant reduction of $\kappa$ at the B1-B2 phase transition area near the CMB. This work provides new insights into thermal transport under extreme conditions and an improved thermal model for terrestrial planets.


title: Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emotion Recognition

abstract: Previous work has established that a person's demographics and speech style affect how well speech processing models perform for them. But where does this bias come from? In this work, we present the Speech Embedding Association Test (SpEAT), a method for detecting bias in one type of model used for many speech tasks: pre-trained models. The SpEAT is inspired by word embedding association tests in natural language processing, which quantify intrinsic bias in a model's representations of different concepts, such as race or valence (something's pleasantness or unpleasantness) and capture the extent to which a model trained on large-scale socio-cultural data has learned human-like biases. Using the SpEAT, we test for six types of bias in 16 English speech models (including 4 models also trained on multilingual data), which come from the wav2vec 2.0, HuBERT, WavLM, and Whisper model families. We find that 14 or more models reveal positive valence (pleasantness) associations with abled people over disabled people, with European-Americans over African-Americans, with females over males, with U.S. accented speakers over non-U.S. accented speakers, and with younger people over older people. Beyond establishing that pre-trained speech models contain these biases, we also show that they can have real world effects. We compare biases found in pre-trained models to biases in downstream models adapted to the task of Speech Emotion Recognition (SER) and find that in 66 of the 96 tests performed (69%), the group that is more associated with positive valence as indicated by the SpEAT also tends to be predicted as speaking with higher valence by the downstream model. Our work provides evidence that, like text and image-based models, pre-trained speech based-models frequently learn human-like biases. Our work also shows that bias found in pre-trained models can propagate to the downstream task of SER.


title: Asymptotic profiles for the Cauchy problem of damped beam equation with two variable coefficients and derivative nonlinearity

abstract: In this article we investigate the asymptotic profile of solutions for the Cauchy problem of the nonlinear damped beam equation with two variable coefficients: \[ \partial_t^2 u + b(t) \partial_t u - a(t) \partial_x^2 u + \partial_x^4 u = \partial_x \left( N(\partial_x u) \right). \] In the authors' previous article [17], the asymptotic profile of solutions for linearized problem ($N \equiv 0$) was classified depending on the assumptions for the coefficients $a(t)$ and $b(t)$ and proved the asymptotic behavior in effective damping cases. We here give the conditions of the coefficients and the nonlinear term in order that the solution behaves as the solution for the heat equation: $b(t) \partial_t u - a(t) \partial_x^2 u=0$ asymptotically as $t \to \infty$.


title: Coupling conduction-band valleys in SiGe heterostructures via shear strain and Ge concentration oscillations

abstract: Engineering conduction-band valley couplings is a key challenge for Si-based spin qubits. Recent work has shown that the most reliable method for enhancing valley couplings entails adding Ge concentration oscillations to the quantum well. However, ultrashort oscillation periods are difficult to grow, while long oscillation periods do not provide useful improvements. Here, we show that the main benefits of short-wavelength oscillations can be achieved in long-wavelength structures through a second-order coupling process involving Brillouin-zone folding induced by shear strain. We finally show that such strain can be achieved through common fabrication techniques, making this an exceptionally promising system for scalable quantum computing.


title: Nature abhors a vacuum: A simple rigorous example of thermalization in an isolated macroscopic quantum system

abstract: We show, without relying on any unproven assumptions, that a low-density free fermion chain exhibits thermalization in the following (restricted) sense. We choose the initial state as a pure state drawn randomly from the Hilbert space in which all particles are in half of the chain. This represents a nonequilibrium state such that the half chain containing all particles is in equilibrium at infinite temperature, and the other half chain is a vacuum. We let the system evolve according to the unitary time evolution determined by the Hamiltonian and, at a sufficiently large typical time, measure the particle number in an arbitrary macroscopic region in the chain. In this setup, it is proved that the measured number is close to the equilibrium value with probability very close to one. Our result establishes the presence of thermalization in a concrete model in a mathematically rigorous manner. The most important theoretical ingredient for the proof of thermalization is the demonstration that a nonequilibrium initial state generated as above typically has a sufficiently large effective dimension. Here, we first give general proof of thermalization based on two assumptions, namely, the absence of degeneracy in energy eigenvalues and a property about the particle distribution in energy eigenstates. We then justify these assumptions in a concrete free-fermion model, where the absence of degeneracy is established by using number-theoretic results. This means that our general result also applies to any lattice gas models in which the above two assumptions are justified. To confirm the potential wide applicability of our theory, we discuss some other models for which the essential assumption about the particle distribution is easily verified, and some non-random initial states whose effective dimensions are sufficiently large.

categories: cond-mat.stat-mech math-ph math.MP quant-ph


title: Efficient separate quantification of state preparation errors and measurement errors on quantum computers and their mitigation

abstract: Current noisy quantum computers have multiple types of errors, which can occur in the state preparation, measurement/readout, and gate operation, as well as intrinsic decoherence and relaxation. Partly motivated by the booming of intermediate-scale quantum processors, measurement and gate errors have been recently extensively studied, and several methods of mitigating them have been proposed and formulated in software packages (e.g., in IBM Qiskit). Despite this, the state preparation error and the procedure to quantify it have not yet been standardized, as state preparation and measurement errors are usually considered not directly separable. Inspired by a recent work of Laflamme, Lin, and Mor [Phys. Rev. A 106, 012439 (2022)], we propose a simple and resource-efficient approach to quantify separately the state preparation and readout error rates. With these two errors separately quantified, we also propose methods to mitigate them separately, especially mitigating state preparation errors with linear (with the number of qubits) complexity. As a result of the separate mitigation, we show that the fidelity of the outcome can be improved by an order of magnitude compared to the standard measurement error mitigation scheme. We also show that the quantification and mitigation scheme is resilient against gate noise and can be immediately applied to current noisy quantum computers. To demonstrate this, we present results from cloud experiments on IBM's superconducting quantum computers. The results indicate that the state preparation error rate is also an important metric for qubit metrology that can be efficiently obtained.


title: Differentiable Learning of Generalized Structured Matrices for Efficient Deep Neural Networks

abstract: This paper investigates efficient deep neural networks (DNNs) to replace dense unstructured weight matrices with structured ones that possess desired properties. The challenge arises because the optimal weight matrix structure in popular neural network models is obscure in most cases and may vary from layer to layer even in the same network. Prior structured matrices proposed for efficient DNNs were mostly hand-crafted without a generalized framework to systematically learn them. To address this issue, we propose a generalized and differentiable framework to learn efficient structures of weight matrices by gradient descent. We first define a new class of structured matrices that covers a wide range of structured matrices in the literature by adjusting the structural parameters. Then, the frequency-domain differentiable parameterization scheme based on the Gaussian-Dirichlet kernel is adopted to learn the structural parameters by proximal gradient descent. On the image and language tasks, our method learns efficient DNNs with structured matrices, achieving lower complexity and/or higher performance than prior approaches that employ low-rank, block-sparse, or block-low-rank matrices.


title: A Zeroth-Order Variance-Reduced Method for Decentralized Stochastic Non-convex Optimization

abstract: In this paper, we consider a distributed stochastic non-convex optimization problem, which is about minimizing a sum of $n$ local cost functions over a network with only zeroth-order information. A novel single-loop Decentralized Zeroth-Order Variance Reduction algorithm, called DZOVR, is proposed, which combines two-point gradient estimation, momentum-based variance reduction technique, and gradient tracking. Under mild assumptions, we show that the algorithm is able to achieve $\mathcal{O}(dn^{-1}\epsilon^{-3})$ sampling complexity at each node to reach an $\epsilon$-accurate stationary point and also exhibits network-independent and linear speedup properties. To the best of our knowledge, this is the first stochastic decentralized zeroth-order algorithm that achieves this sampling complexity. Numerical experiments demonstrate that DZOVR outperforms the other state-of-the-art algorithms and has network-independent and linear speedup properties.


title: Simple and Asymmetric Graph Contrastive Learning without Augmentations

abstract: Graph Contrastive Learning (GCL) has shown superior performance in representation learning in graph-structured data. Despite their success, most existing GCL methods rely on prefabricated graph augmentation and homophily assumptions. Thus, they fail to generalize well to heterophilic graphs where connected nodes may have different class labels and dissimilar features. In this paper, we study the problem of conducting contrastive learning on homophilic and heterophilic graphs. We find that we can achieve promising performance simply by considering an asymmetric view of the neighboring nodes. The resulting simple algorithm, Asymmetric Contrastive Learning for Graphs (GraphACL), is easy to implement and does not rely on graph augmentations and homophily assumptions. We provide theoretical and empirical evidence that GraphACL can capture one-hop local neighborhood information and two-hop monophily similarity, which are both important for modeling heterophilic graphs. Experimental results show that the simple GraphACL significantly outperforms state-of-the-art graph contrastive learning and self-supervised learning methods on homophilic and heterophilic graphs. The code of GraphACL is available at https://github.com/tengxiao1/GraphACL.


title: A foundational neural operator that continuously learns without forgetting

abstract: Machine learning has witnessed substantial growth, leading to the development of advanced artificial intelligence models crafted to address a wide range of real-world challenges spanning various domains, such as computer vision, natural language processing, and scientific computing. Nevertheless, the creation of custom models for each new task remains a resource-intensive undertaking, demanding considerable computational time and memory resources. In this study, we introduce the concept of the Neural Combinatorial Wavelet Neural Operator (NCWNO) as a foundational model for scientific computing. This model is specifically designed to excel in learning from a diverse spectrum of physics and continuously adapt to the solution operators associated with parametric partial differential equations (PDEs). The NCWNO leverages a gated structure that employs local wavelet experts to acquire shared features across multiple physical systems, complemented by a memory-based ensembling approach among these local wavelet experts. This combination enables rapid adaptation to new challenges. The proposed foundational model offers two key advantages: (i) it can simultaneously learn solution operators for multiple parametric PDEs, and (ii) it can swiftly generalize to new parametric PDEs with minimal fine-tuning. The proposed NCWNO is the first foundational operator learning algorithm distinguished by its (i) robustness against catastrophic forgetting, (ii) the maintenance of positive transfer for new parametric PDEs, and (iii) the facilitation of knowledge transfer across dissimilar tasks. Through an extensive set of benchmark examples, we demonstrate that the NCWNO can outperform task-specific baseline operator learning frameworks with minimal hyperparameter tuning at the prediction stage. We also show that with minimal fine-tuning, the NCWNO performs accurate combinatorial learning of new parametric PDEs.


title: Non-abelian cohomology and Seifert-Van Kampen theorem

abstract: The first goal of the present paper it to present a simple and elementary proof of the standard Seifert-van Kampen theorem based on ideas of P. Olum. The key tool is the singular cohomology theory with non-abelian coefficients in dimensions 0 and 1. After this we apply non-abelian cohomology to prove Crowell-Fox version of Seifert-van Kampent theorem and its improvement die to Brown-Salleh (this proof shows that the Lebesgue dimension of the disc is irrelevant for this improvement). Finally, we apply non-abelian cohomology to prove some theorem of van Kampen, which are more general than the standard version (and, in particular, include the computation of the fundamental group of the circle).


title: Dynamo-Depth: Fixing Unsupervised Depth Estimation for Dynamical Scenes

abstract: Unsupervised monocular depth estimation techniques have demonstrated encouraging results but typically assume that the scene is static. These techniques suffer when trained on dynamical scenes, where apparent object motion can equally be explained by hypothesizing the object's independent motion, or by altering its depth. This ambiguity causes depth estimators to predict erroneous depth for moving objects. To resolve this issue, we introduce Dynamo-Depth, an unifying approach that disambiguates dynamical motion by jointly learning monocular depth, 3D independent flow field, and motion segmentation from unlabeled monocular videos. Specifically, we offer our key insight that a good initial estimation of motion segmentation is sufficient for jointly learning depth and independent motion despite the fundamental underlying ambiguity. Our proposed method achieves state-of-the-art performance on monocular depth estimation on Waymo Open and nuScenes Dataset with significant improvement in the depth of moving objects. Code and additional results are available at https://dynamo-depth.github.io.


title: D2NO: Efficient Handling of Heterogeneous Input Function Spaces with Distributed Deep Neural Operators

abstract: Neural operators have been applied in various scientific fields, such as solving parametric partial differential equations, dynamical systems with control, and inverse problems. However, challenges arise when dealing with input functions that exhibit heterogeneous properties, requiring multiple sensors to handle functions with minimal regularity. To address this issue, discretization-invariant neural operators have been used, allowing the sampling of diverse input functions with different sensor locations. However, existing frameworks still require an equal number of sensors for all functions. In our study, we propose a novel distributed approach to further relax the discretization requirements and solve the heterogeneous dataset challenges. Our method involves partitioning the input function space and processing individual input functions using independent and separate neural networks. A centralized neural network is used to handle shared information across all output functions. This distributed methodology reduces the number of gradient descent back-propagation steps, improving efficiency while maintaining accuracy. We demonstrate that the corresponding neural network is a universal approximator of continuous nonlinear operators and present four numerical examples to validate its performance.