title: Parametrically controlled chiral interface for superconducting quantum devices

abstract: Nonreciprocal microwave routing plays a crucial role for measuring quantum circuits, and allows for realizing cascaded quantum systems for generating and stabilizing entanglement between non-interacting qubits. The most commonly used tools for implementing directionality are ferrite-based circulators. These devices are versatile, but suffer from excess loss, a large footprint, and fixed directionality. For utilizing nonreciprocity in scalable quantum circuits it is desirable to develop efficient integration of low-loss and in-situ controllable directional elements. Here, we report the design and experimental realization of a controllable directional interface that may be integrated directly with superconducting qubits. In the presented device, nonreciprocity is realized through a combination of interference and phase-controlled parametric pumping. We have achieved a maximum directionality of around 30\,dB, and the performance of the device is predicted quantitatively from independent calibration measurements. Using the excellent agreement of model and experiment, we predict that the circuit will be useable as a chiral qubit interface with inefficiencies at the one-percent level or below. Our work provides a route toward isolator-free qubit readout schemes and high-fidelity entanglement generation in all-to-all connected networks of superconducting quantum devices.


title: Comparing eccentric waveform models based on post-Newtonian and effective-one-body approaches, over an observationally relevant parameter space

abstract: We used two numerical models, namely the \texttt{CBwaves} and \texttt{SEOBNRE} algorithms, based on the post-Newtonian and effective-one-body approaches for binary black holes evolving on eccentric orbits. We performed 20.000 new simulations for non-spinning and 240.000 simulations for aligned-spin configurations on a common grid of parameter values over the parameter space spanned by the mass ratio $q\equiv m_1/m_2\in[0.1,\,1]$, the gravitational mass $m_i \in [10M_\odot,\, 100M_\odot]$ of each component labeled by $i$, the corresponding spin magnitude $S_i \in [0,\,0.6]$ and a constant initial orbital eccentricity $e_{0}$. A detailed investigation was conducted to ascertain whether there was a discrepancy in the waveforms generated by the two codes. This involved an in-depth analysis of the mismatch. Furthermore, an extensive comparison was carried out on the outlier points between the two codes.


title: Adaptive Dynamic Bitvectors

abstract: While operations \emph{rank} and \emph{select} on static bitvectors can be supported in constant time, lower bounds show that supporting updates raises the cost per operation to $\Theta(\log n/ \log\log n)$. This is a shame in scenarios where updates are possible but uncommon. We develop a representation of bitvectors that, if there are $q = \Omega(\log^2 n)$ queries per update, supports all the operations in $O(\log(n/q))$ amortized time. Our experimental results support the theoretical findings, displaying speedups of orders of magnitude compared to standard dynamic implementations.


title: Targeted Nakamoto: A Bitcoin Protocol to Balance Network Security and Energy Consumption

abstract: In a Proof-of-Work blockchain such as Bitcoin mining hashrate is increasing in the block reward. An increase in hashrate reduces network vulnerability to attack (a reduction in security cost) while increasing carbon emissions and electricity cost (an increase in externalities cost). This implies a tradeoff in total cost at different levels of hashrate and the existence of a hashrate interval where total cost is minimized. Targeted Nakamoto is a Proof-of-Work protocol augmentation that incentivizes miners to hone in on a target hashrate interval. When hashrate is above target a ceiling is placed on the block reward a miner can receive. When hashrate is below target a floor is placed underneath the miner's block reward. Monetary neutrality is maintained by a proportional increase in spending potential among addresses holding UTXO's to match a deduction from total block reward when the ceiling is operative and a proportional reduction in spending potential among addresses holding UTXO's to match an increase over the total block reward when the floor is binding.


title: Pure Exploration for Constrained Best Mixed Arm Identification with a Fixed Budget

abstract: In this paper, we introduce the constrained best mixed arm identification (CBMAI) problem with a fixed budget. This is a pure exploration problem in a stochastic finite armed bandit model. Each arm is associated with a reward and multiple types of costs from unknown distributions. Unlike the unconstrained best arm identification problem, the optimal solution for the CBMAI problem may be a randomized mixture of multiple arms. The goal thus is to find the best mixed arm that maximizes the expected reward subject to constraints on the expected costs with a given learning budget $N$. We propose a novel, parameter-free algorithm, called the Score Function-based Successive Reject (SFSR) algorithm, that combines the classical successive reject framework with a novel score-function-based rejection criteria based on linear programming theory to identify the optimal support. We provide a theoretical upper bound on the mis-identification (of the the support of the best mixed arm) probability and show that it decays exponentially in the budget $N$ and some constants that characterize the hardness of the problem instance. We also develop an information theoretic lower bound on the error probability that shows that these constants appropriately characterize the problem difficulty. We validate this empirically on a number of average and hard instances.


title: Nilpotent Symplectic Alternating Algebras

abstract: We develop a structure theory for nilpotent symplectic alternating algebras. We then give a classification of all nilpotent symplectic alternating algebras of dimension up to 10 over any field. The study reveals a new subclasses of powerful groups that we call powerfully nilpotent groups and powerfully soluble groups.


title: Dissociation of Faithful and Unfaithful Reasoning in LLMs

abstract: Large language models (LLMs) improve their performance in downstream tasks when they generate Chain of Thought reasoning text before producing an answer. Our research investigates how LLMs recover from errors in Chain of Thought, reaching the correct final answer despite mistakes in the reasoning text. Through analysis of these error recovery behaviors, we find evidence for unfaithfulness in Chain of Thought, but we also identify many clear examples of faithful error recovery behaviors. We identify factors that shift LLM recovery behavior: LLMs recover more frequently from obvious errors and in contexts that provide more evidence for the correct answer. However, unfaithful recoveries show the opposite behavior, occurring more frequently for more difficult error positions. Our results indicate that there are distinct mechanisms driving faithful and unfaithful error recoveries. Our results challenge the view that LLM reasoning is a uniform, coherent process.


title: Real-Time and Accurate: Zero-shot High-Fidelity Singing Voice Conversion with Multi-Condition Flow Synthesis

abstract: Singing voice conversion is to convert the source sing voice into the target sing voice except for the content. Currently, flow-based models can complete the task of voice conversion, but they struggle to effectively extract latent variables in the more rhythmically rich and emotionally expressive task of singing voice conversion, while also facing issues with low efficiency in speech processing. In this paper, we propose a high-fidelity flow-based model based on multi-decoupling feature constraints, which enhances the capture of vocal details by integrating multiple encoders. We also use iSTFT to enhance the speed of speech processing by replacing some layers of the Vocoder. We compare the synthesized singing voice with other models from multiple dimensions, and our proposed model is highly consistent with the current state-of-the-art, with the demo which is available at \url{https://lazycat1119.github.io/RASVC-demo/}


title: ULTRA-MC: A Unified Approach to Learning Mixtures of Markov Chains via Hitting Times

abstract: This study introduces a novel approach for learning mixtures of Markov chains, a critical process applicable to various fields, including healthcare and the analysis of web users. Existing research has identified a clear divide in methodologies for learning mixtures of discrete and continuous-time Markov chains, while the latter presents additional complexities for recovery accuracy and efficiency. We introduce a unifying strategy for learning mixtures of discrete and continuous-time Markov chains, focusing on hitting times, which are well defined for both types. Specifically, we design a reconstruction algorithm that outputs a mixture which accurately reflects the estimated hitting times and demonstrates resilience to noise. We introduce an efficient gradient-descent approach, specifically tailored to manage the computational complexity and non-symmetric characteristics inherent in the calculation of hitting time derivatives. Our approach is also of significant interest when applied to a single Markov chain, thus extending the methodologies previously established by Hoskins et al. and Wittmann et al. We complement our theoretical work with experiments conducted on synthetic and real-world datasets, providing a comprehensive evaluation of our methodology.


title: Compilation for Dynamically Field-Programmable Qubit Arrays with Efficient and Provably Near-Optimal Scheduling

abstract: Dynamically field-programmable qubit arrays based on neutral atoms have high fidelity and highly parallel gates for quantum computing. However, it is challenging for compilers to fully leverage the novel flexibility offered by such hardware while respecting its various constraints. In this study, we break down the compilation for this architecture into three tasks: scheduling, placement, and routing. We formulate these three problems and present efficient solutions to them. Notably, our scheduling based on graph edge coloring is provably near-optimal in terms of two-qubit gate stage count (at most one more than the optimum), the fidelity bottleneck of this platform. As a result, our compiler, Enola, produces higher fidelity results compared to existing works, e.g., 3.7X stage reduction and 5.9X fidelity improvement on the benchmark set used by OLSQ-DPQA, the current state of the art. Additionally, Enola is highly scalable, e.g., within 30 minutes, it can compile circuits with 10,000 qubits, a scale sufficient for the current era of quantum computing. Enola is open source at https://github.com/UCLA-VAST/Enola


title: Music Genre Classification: Training an AI model

abstract: Music genre classification is an area that utilizes machine learning models and techniques for the processing of audio signals, in which applications range from content recommendation systems to music recommendation systems. In this research I explore various machine learning algorithms for the purpose of music genre classification, using features extracted from audio signals.The systems are namely, a Multilayer Perceptron (built from scratch), a k-Nearest Neighbours (also built from scratch), a Convolutional Neural Network and lastly a Random Forest wide model. In order to process the audio signals, feature extraction methods such as Short-Time Fourier Transform, and the extraction of Mel Cepstral Coefficients (MFCCs), is performed. Through this extensive research, I aim to asses the robustness of machine learning models for genre classification, and to compare their results.


title: Contrastive and Consistency Learning for Neural Noisy-Channel Model in Spoken Language Understanding

abstract: Recently, deep end-to-end learning has been studied for intent classification in Spoken Language Understanding (SLU). However, end-to-end models require a large amount of speech data with intent labels, and highly optimized models are generally sensitive to the inconsistency between the training and evaluation conditions. Therefore, a natural language understanding approach based on Automatic Speech Recognition (ASR) remains attractive because it can utilize a pre-trained general language model and adapt to the mismatch of the speech input environment. Using this module-based approach, we improve a noisy-channel model to handle transcription inconsistencies caused by ASR errors. We propose a two-stage method, Contrastive and Consistency Learning (CCL), that correlates error patterns between clean and noisy ASR transcripts and emphasizes the consistency of the latent features of the two transcripts. Experiments on four benchmark datasets show that CCL outperforms existing methods and improves the ASR robustness in various noisy environments. Code is available at https://github.com/syoung7388/CCL.


title: Magnetic Resonance Image Processing Transformer for General Reconstruction

abstract: Purpose: To develop and evaluate a deep learning model for general accelerated MRI reconstruction. Materials and Methods: This retrospective study built a magnetic resonance image processing transformer (MR-IPT) which includes multi-head-tails and a single shared window transformer main body. Three mutations of MR-IPT with different transformer structures were implemented to guide the design of our MR-IPT model. Pre-trained on the MRI set of RadImageNet including 672675 images with multiple anatomy categories, the model was further migrated and evaluated on fastMRI knee dataset with 25012 images for downstream reconstruction tasks. We performed comparison studies with three CNN-based conventional networks in zero- and few-shot learning scenarios. Transfer learning process was conducted on both MR-IPT and CNN networks to further validate the generalizability of MR-IPT. To study the model performance stability, we evaluated our model with various downstream dataset sizes ranging from 10 to 2500 images. Result: The MR-IPT model provided superior performance in multiple downstream tasks compared to conventional CNN networks. MR-IPT achieved a PSNR/SSIM of 26.521/0.6102 (4-fold) and 24.861/0.4996 (8-fold) in 10-epoch learning, surpassing UNet128 at 25.056/0.5832 (4-fold) and 22.984/0.4637 (8-fold). With the same large-scale pre-training, MR-IPT provided a 5% performance boost compared to UNet128 in zero-shot learning in 8-fold and 3% in 4-fold. Conclusion: MR-IPT framework benefits from its transformer-based structure and large-scale pre-training and can serve as a solid backbone in other downstream tasks with zero- and few-shot learning.


title: Stability analysis of nonlinear stochastic flexibility function in smart energy systems

abstract: Demand-side management provides a great potential for improving the efficiency and reliability of energy systems. This requires a mechanism to connect the market level and the demand side. The flexibility function is a novel approach that bridges the gap between the markets and the dynamics of physical assets at the lower levels of the energy systems and activates demand-side flexibility with the purpose of decision-making as well as for offering a new framework for balancing and grid services. Employing this function as a key for many decision-making and control algorithms reveals that a mathematically rigorous stability analysis is required for it. In this paper, we investigate the stability properties of two nonlinear flexibility functions, as a dynamic mapping between electricity price and power consumption. Specifically, we analyze the stability of a deterministic flexibility function and an It\^{o} stochastic flexibility function. Simulation results are also provided to demonstrate the dynamics of the flexibility functions and to show that the analytical results hold.


title: Mobile Robot Sensory Coverage in 2-D Environments: An Optimization Approach with Efficiency Bounds

abstract: This paper considers three related mobile robot multi-target sensory coverage and inspection planning problems in 2-D environments. In the first problem, a mobile robot must find the shortest path to observe multiple targets with a limited range sensor in an obstacle free environment. In the second problem, the mobile robot must efficiently observe multiple targets while taking advantage of multi-target views in an obstacle free environment. The third problem considers multi-target sensory coverage in the presence of obstacles that obstruct sensor views of the targets. We show how all three problems can be formulated in a MINLP optimization framework. Because exact solutions to these problems are NP-hard, we introduce polynomial time approximation algorithms for each problem. These algorithms combine polynomial-time methods to approximate the optimal target sensing order, combined with efficient convex optimization methods that incorporate the constraints posed by the robot sensor footprint and obstacles in the environment. Importantly, we develop bounds that limit the gap between the exact and approximate solutions. Algorithms for all problems are fully implemented and illustrated with examples. Beyond the utility of our algorithms, the bounds derived in the paper contribute to the theory of optimal coverage planning algorithms.


title: Social Zone as a Barrier Function for Socially-Compliant Robot Navigation

abstract: This study addresses the challenge of integrating social norms into robot navigation, which is essential for ensuring that robots operate safely and efficiently in human-centric environments. Social norms, often unspoken and implicitly understood among people, are difficult to explicitly define and implement in robotic systems. To overcome this, we derive these norms from real human trajectory data, utilizing the comprehensive ATC dataset to identify the minimum social zones humans and robots must respect. These zones are integrated into the robot' navigation system by applying barrier functions, ensuring the robot consistently remains within the designated safety set. Simulation results demonstrate that our system effectively mimics human-like navigation strategies, such as passing on the right side and adjusting speed or pausing in constrained spaces. The proposed framework is versatile, easily comprehensible, and tunable, demonstrating the potential to advance the development of robots designed to navigate effectively in human-centric environments.


title: The Fraction of Dust Mass in the Form of PAHs on 10-50 pc Scales in Nearby Galaxies

abstract: Polycyclic aromatic hydrocarbons (PAHs) are a ubiquitous component of the interstellar medium (ISM) in z~0 massive, star-forming galaxies and play key roles in ISM energy balance, chemistry, and shielding. Wide field of view, high resolution mid-infrared (MIR) images from JWST provides the ability to map the fraction of dust in the form of PAHs and the properties of these key dust grains at 10-50 pc resolution in galaxies outside the Local Group. We use MIR JWST photometric observations of a sample of 19 nearby galaxies from the "Physics at High Angular Resolution in Nearby GalaxieS" (PHANGS) survey to investigate the variations of the PAH fraction. By comparison to lower resolution far-IR mapping, we show that a combination of the MIRI filters (R$_{\rm{PAH}}$ = [F770W+F1130W]/F2100W) traces the fraction of dust by mass in the form of PAHs (i.e., the PAH fraction, or q$_{\rm{PAH}}$). Mapping R$_{\rm{PAH}}$ across the 19 PHANGS galaxies, we find that the PAH fraction steeply decreases in HII regions, revealing the destruction of these small grains in regions of ionized gas. Outside HII regions, we find R$_{\rm{PAH}}$ is constant across the PHANGS sample with an average value of 3.43$\pm$0.98, which, for an illuminating radiation field of intensity 2-5 times that of the radiation field in the solar neighborhood, corresponds to q$_{\rm{PAH}}$ values of 3-6%.


title: The Rarity of Musical Audio Signals Within the Space of Possible Audio Generation

abstract: A white noise signal can access any possible configuration of values, though statistically over many samples tends to a uniform spectral distribution, and is highly unlikely to produce intelligible sound. But how unlikely? The probability that white noise generates a music-like signal over different durations is analyzed, based on some necessary features observed in real music audio signals such as mostly proximate movement and zero crossing rate. Given the mathematical results, the rarity of music as a signal is considered overall. The applicability of this study is not just to show that music has a precious rarity value, but that examination of the size of music relative to the overall size of audio signal space provides information to inform new generations of algorithmic music system (which are now often founded on audio signal generation directly, and may relate to white noise via such machine learning processes as diffusion). Estimated upper bounds on the rarity of music to the size of various physical and musical spaces are compared, to better understand the magnitude of the results (pun intended). Underlying the research are the questions `how much music is still out there?' and `how much music could a machine learning process actually reach?'.


title: A finiteness result for common zeros of iterates of rational maps

abstract: Answering a question asked by Hsia and Tucker in their paper on the finiteness of greatest common divisors of iterates of polynomials, we prove that if $f, g \in \mathbb{C}(X)$ are compositionally independent rational functions and $c \in \mathbb{C}(X)$, then there are at most finitely many $\lambda\in\mathbb{C}$ with the property that there is an $n$ such that $f^n(\lambda) = g^n(\lambda) = c(\lambda)$, except for a few families of $f, g \in Aut(\mathbb{P}^1_\mathbb{C})$ which gives counterexamples.


title: Certified Inventory Control of Critical Resources

abstract: Inventory control is subject to service-level requirements, in which sufficient stock levels must be maintained despite an unknown demand. We propose a data-driven order policy that certifies any prescribed service level under minimal assumptions on the unknown demand process. The policy achieves this using any online learning method along with integral action. We further propose an inference method that is valid in finite samples. The properties and theoretical guarantees of the method are illustrated using both synthetic and real-world data.


title: Conformal Classification with Equalized Coverage for Adaptively Selected Groups

abstract: This paper introduces a conformal inference method to evaluate uncertainty in classification by generating prediction sets with valid coverage conditional on adaptively chosen features. These features are carefully selected to reflect potential model limitations or biases. This can be useful to find a practical compromise between efficiency -- by providing informative predictions -- and algorithmic fairness -- by ensuring equalized coverage for the most sensitive groups. We demonstrate the validity and effectiveness of this method on simulated and real data sets.


title: Is Algorithmic Stability Testable? A Unified Framework under Computational Constraints

abstract: Algorithmic stability is a central notion in learning theory that quantifies the sensitivity of an algorithm to small changes in the training data. If a learning algorithm satisfies certain stability properties, this leads to many important downstream implications, such as generalization, robustness, and reliable predictive inference. Verifying that stability holds for a particular algorithm is therefore an important and practical question. However, recent results establish that testing the stability of a black-box algorithm is impossible, given limited data from an unknown distribution, in settings where the data lies in an uncountably infinite space (such as real-valued data). In this work, we extend this question to examine a far broader range of settings, where the data may lie in any space -- for example, categorical data. We develop a unified framework for quantifying the hardness of testing algorithmic stability, which establishes that across all settings, if the available data is limited then exhaustive search is essentially the only universally valid mechanism for certifying algorithmic stability. Since in practice, any test of stability would naturally be subject to computational constraints, exhaustive search is impossible and so this implies fundamental limits on our ability to test the stability property for a black-box algorithm.


title: Correlation functions of boundary and defect conformal field theories

abstract: Applying the holographic method, we investigate correlation functions of boundary and defect conformal field theories. To describe boundary conformal field theory, we consider an end of the world brane in an asymptotic AdS space which behaves as a boundary in the dual conformal field theory. In this holographic setup, we calculate correlation functions involving the reflection effect at the boundary. We show that, when the end of the world brane has no degrees of freedom, the holographic calculation reproduces the correlation functions known in the boundary conformal field theory. When the boundary has nontrivial boundary entropy, we calculate one- and two-point functions nontrivially relying on the boundary entropy. We further study correlation functions of defect conformal field theory after introducing a $p$-brane. We directly derive a bulk-to-defect two-point function without introducing an image operator and determine the coefficient of the two-point function exactly in the holographic setup.


title: An Update to the SBML Human-Readable Antimony Language

abstract: Antimony is a high-level, human-readable text-based language designed for defining and sharing models in the systems biology community. It enables scientists to describe biochemical networks and systems using a simple and intuitive syntax. It allows users to easily create, modify, and distribute reproducible computational models. By allowing the concise representation of complex biological processes, Antimony enhances collaborative efforts, improves reproducibility, and accelerates the iterative development of models in systems biology. This paper provides an update to the Antimony language since it was introduced in 2009. In particular, we highlight new annotation features, support for flux balance analysis, a new rateOf method, support for probability distributions and uncertainty, named stochiometries, and algebraic rules. Antimony is also now distributed as a C/C++ library, together with python and Julia bindings, as well as a JavaScript version for use within a web browser. Availability: https://github.com/sys-bio/antimony.


title: CHARP: Conversation History AwaReness Probing for Knowledge-grounded Dialogue Systems

abstract: In this work, we dive deep into one of the popular knowledge-grounded dialogue benchmarks that focus on faithfulness, FaithDial. We show that a significant portion of the FaithDial data contains annotation artifacts, which may bias models towards completely ignoring the conversation history. We therefore introduce CHARP, a diagnostic test set, designed for an improved evaluation of hallucinations in conversational model. CHARP not only measures hallucination but also the compliance of the models to the conversation task. Our extensive analysis reveals that models primarily exhibit poor performance on CHARP due to their inability to effectively attend to and reason over the conversation history. Furthermore, the evaluation methods of FaithDial fail to capture these shortcomings, neglecting the conversational history. Our findings indicate that there is substantial room for contribution in both dataset creation and hallucination evaluation for knowledge-grounded dialogue, and that CHARP can serve as a tool for monitoring the progress in this particular research area. CHARP is publicly available at https://huggingface.co/datasets/huawei-noah/CHARP


title: Invariant uniformization

abstract: Standard results in descriptive set theory provide sufficient conditions for a set $P \subseteq \mathbb{N}^\mathbb{N} \times \mathbb{N}^\mathbb{N}$ to admit a Borel uniformization, namely, when $P$ has ''small'' sections or ''large'' sections. We consider an invariant analogue of these results: Given a Borel equivalence relation $E$ and an $E$-invariant set $P$ with ''small'' or ''large'' sections, does $P$ admit an $E$-invariant Borel uniformization? Given $E$, we show that every such $P$ admits an $E$-invariant Borel uniformization if and only if $E$ is smooth. We also compute the definable complexity of counterexamples in the case where $E$ is not smooth, using category, measure, and Ramsey-theoretic methods. We provide two new proofs of a dichotomy of Miller classifying the pairs $(E, P)$ such that $P$ admits an $E$-invariant uniformization, for $P$ with countable sections. In the process, we prove an $\aleph_0$-dimensional $(G_0, H_0)$ dichotomy, generalizing dichotomies of Miller and Lecomte. We also show that the set of pairs $(E, P)$ such that $P$ has ''large'' sections and admits an $E$-invariant Borel uniformization is $\boldsymbol{\Sigma^1_2}$-complete; in particular, there is no analog of Miller's dichotomy for $P$ with ''large'' sections. Finally, we consider a less strict notion of invariant uniformization, where we select a countable nonempty subset of each section instead of a single point.


title: Splicing positroid varieties

abstract: We construct an explicit isomorphism between an open subset in the open positroid variety $\Pi_{k,n}^{\circ}$ in the Grassmannian $\mathrm{Gr}(k,n)$ and the product of two open positroid varieties $\Pi_{k,n-a+1}^{\circ}\times \Pi_{k,a+k-1}^{\circ}$. In the respective cluster structures, this isomorphism is given by freezing a certain subset of cluster variables and applying a cluster quasi-equivalence.


title: A Wearable Resistance Devices Motor Learning Effects in Exercise

abstract: The integration of technology into exercise regimens has emerged as a strategy to enhance normal human capabilities and return human motor function after injury or illness by enhancing motor learning and retention. Much research has focused on how active devices, whether confined to a lab or made into a wearable format, can apply forces at set times and conditions to optimize the process of learning. However, the focus on active force production often forces devices to either be confined to simple movements or interventions. As such, in this paper, we investigate how passive device behaviors can contribute to the process of motor learning by themselves. Our approach involves using a wearable resistance (WR) device, which is outfitted with elastic bands, to apply a force field that changes in response to a person's movements while performing exercises. We develop a method to measure the produced forces from the device without impeding the function and we characterize the device's force generation abilities. We then present a study assessing the impact of the WR device on motor learning of proper squat form compared to visual or no feedback. Biometrics such as knee and hip angles were used to monitor and assess subject performance. Our findings indicate that the force fields produced while training with the WR device can improve performance in full-body exercises similarly to a more direct visual feedback mechanism, though the improvement is not consistent across all performance metrics. Through our research, we contribute important insights into the application of passive wearable resistance technology in practical exercise settings.


title: Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learning

abstract: Conventional recommender systems (RSs) face challenges in precisely capturing users' fine-grained preferences. Large language models (LLMs) have shown capabilities in commonsense reasoning and leveraging external tools that may help address these challenges. However, existing LLM-based RSs suffer from hallucinations, misalignment between the semantic space of items and the behavior space of users, or overly simplistic control strategies (e.g., whether to rank or directly present existing results). To bridge these gap, we introduce ToolRec, a framework for LLM-empowered recommendations via tool learning that uses LLMs as surrogate users, thereby guiding the recommendation process and invoking external tools to generate a recommendation list that aligns closely with users' nuanced preferences. We formulate the recommendation process as a process aimed at exploring user interests in attribute granularity. The process factors in the nuances of the context and user preferences. The LLM then invokes external tools based on a user's attribute instructions and probes different segments of the item pool. We consider two types of attribute-oriented tools: rank tools and retrieval tools. Through the integration of LLMs, ToolRec enables conventional recommender systems to become external tools with a natural language interface. Extensive experiments verify the effectiveness of ToolRec, particularly in scenarios that are rich in semantic content.


title: Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification

abstract: Predicting simple function classes has been widely used as a testbed for developing theory and understanding of the trained Transformer's in-context learning (ICL) ability. In this paper, we revisit the training of Transformers on linear regression tasks, and different from all the existing literature, we consider a bi-objective prediction task of predicting both the conditional expectation $\mathbb{E}[Y|X]$ and the conditional variance Var$(Y|X)$. This additional uncertainty quantification objective provides a handle to (i) better design out-of-distribution experiments to distinguish ICL from in-weight learning (IWL) and (ii) make a better separation between the algorithms with and without using the prior information of the training distribution. Theoretically, we show that the trained Transformer reaches near Bayes-optimum, suggesting the usage of the information of the training distribution. Our method can be extended to other cases. Specifically, with the Transformer's context window $S$, we prove a generalization bound of $\tilde{\mathcal{O}}(\sqrt{\min\{S, T\}/(n T)})$ on $n$ tasks with sequences of length $T$, providing sharper analysis compared to previous results of $\tilde{\mathcal{O}}(\sqrt{1/n})$. Empirically, we illustrate that while the trained Transformer behaves as the Bayes-optimal solution as a natural consequence of supervised training in distribution, it does not necessarily perform a Bayesian inference when facing task shifts, in contrast to the \textit{equivalence} between these two proposed in many existing literature. We also demonstrate the trained Transformer's ICL ability over covariates shift and prompt-length shift and interpret them as a generalization over a meta distribution.


title: Quantifying the Gain in Weak-to-Strong Generalization

abstract: Recent advances in large language models have shown capabilities that are extraordinary and near-superhuman. These models operate with such complexity that reliably evaluating and aligning them proves challenging for humans. This leads to the natural question: can guidance from weak models (like humans) adequately direct the capabilities of strong models? In a recent and somewhat surprising work, Burns et al. (2023) empirically demonstrated that when strong models (like GPT-4) are finetuned using labels generated by weak supervisors (like GPT-2), the strong models outperform their weaker counterparts -- a phenomenon they term weak-to-strong generalization. In this work, we present a theoretical framework for understanding weak-to-strong generalization. Specifically, we show that the improvement in performance achieved by strong models over their weaker counterparts is quantified by the misfit error incurred by the strong model on labels generated by the weaker model. Our theory reveals several curious algorithmic insights. For instance, we can predict the amount by which the strong model will improve over the weak model, and also choose among different weak models to train the strong model, based on its misfit error. We validate our theoretical findings through various empirical assessments.


title: A Pilot Study from the First Course-Based Undergraduate Research Experience for Online Degree-Seeking Astronomy Students

abstract: Research-based active learning approaches are critical for the teaching and learning of undergraduate STEM majors. Course-based undergraduate research experiences (CUREs) are becoming more commonplace in traditional, in-person academic environments, but have only just started to be utilized in online education. Online education has been shown to create accessible pathways to knowledge for individuals from nontraditional student backgrounds, and increasing the diversity of STEM fields has been identified as a priority for future generations of scientists and engineers. We developed and instructed a rigorous, six-week curriculum on the topic of observational astronomy, dedicated to educating second year online astronomy students in practices and techniques for astronomical research. Throughout the course, the students learned about telescopes, the atmosphere, filter systems, adaptive optics systems, astronomical catalogs, and image viewing and processing tools. We developed a survey informed by previous research validated assessments aimed to evaluate course feedback, course impact, student self-efficacy, student science identity and community values, and student sense of belonging. The survey was administered at the conclusion of the course to all eleven students yielding eight total responses. Although preliminary, the results of our analysis indicate that student confidence in utilizing the tools and skills taught in the course was significant. Students also felt a great sense of belonging to the astronomy community and increased confidence in conducting astronomical research in the future.


title: GS-Hider: Hiding Messages into 3D Gaussian Splatting

abstract: 3D Gaussian Splatting (3DGS) has already become the emerging research focus in the fields of 3D scene reconstruction and novel view synthesis. Given that training a 3DGS requires a significant amount of time and computational cost, it is crucial to protect the copyright, integrity, and privacy of such 3D assets. Steganography, as a crucial technique for encrypted transmission and copyright protection, has been extensively studied. However, it still lacks profound exploration targeted at 3DGS. Unlike its predecessor NeRF, 3DGS possesses two distinct features: 1) explicit 3D representation; and 2) real-time rendering speeds. These characteristics result in the 3DGS point cloud files being public and transparent, with each Gaussian point having a clear physical significance. Therefore, ensuring the security and fidelity of the original 3D scene while embedding information into the 3DGS point cloud files is an extremely challenging task. To solve the above-mentioned issue, we first propose a steganography framework for 3DGS, dubbed GS-Hider, which can embed 3D scenes and images into original GS point clouds in an invisible manner and accurately extract the hidden messages. Specifically, we design a coupled secured feature attribute to replace the original 3DGS's spherical harmonics coefficients and then use a scene decoder and a message decoder to disentangle the original RGB scene and the hidden message. Extensive experiments demonstrated that the proposed GS-Hider can effectively conceal multimodal messages without compromising rendering quality and possesses exceptional security, robustness, capacity, and flexibility. Our project is available at: https://xuanyuzhang21.github.io/project/gshider.


title: Bayesian Optimization of Functions over Node Subsets in Graphs

abstract: We address the problem of optimizing over functions defined on node subsets in a graph. The optimization of such functions is often a non-trivial task given their combinatorial, black-box and expensive-to-evaluate nature. Although various algorithms have been introduced in the literature, most are either task-specific or computationally inefficient and only utilize information about the graph structure without considering the characteristics of the function. To address these limitations, we utilize Bayesian Optimization (BO), a sample-efficient black-box solver, and propose a novel framework for combinatorial optimization on graphs. More specifically, we map each $k$-node subset in the original graph to a node in a new combinatorial graph and adopt a local modeling approach to efficiently traverse the latter graph by progressively sampling its subgraphs using a recursive algorithm. Extensive experiments under both synthetic and real-world setups demonstrate the effectiveness of the proposed BO framework on various types of graphs and optimization tasks, where its behavior is analyzed in detail with ablation studies.


title: A Counterfactual Analysis of the Dishonest Casino

abstract: The dishonest casino is a well-known hidden Markov model (HMM) used in educational settings to introduce HMMs and graphical models. Here, a sequence of die rolls is observed, with the casino switching between a fair and a loaded die. Typically, the goal is to use the observed rolls to infer the pattern of fair and loaded dice, leading to filtering, smoothing, and Viterbi algorithms. This paper, however, explores how much of the winnings is attributable to the casino's cheating, a counterfactual question beyond the scope of HMM primitives. To address this, we introduce a structural causal model (SCM) consistent with the HMM and show that the expected winnings attributable to cheating (EWAC) can be bounded using linear programs (LPs). Through numerical experiments, we compute these bounds and develop intuition using benchmark SCMs based on independence, comonotonic, and counter-monotonic copulas. We show that tighter bounds are obtained with a time-homogeneity condition on the SCM, while looser bounds allow for an almost explicit LP solution. Domain-specific knowledge like pathwise monotonicity or counterfactual stability can be incorporated via linear constraints. Our work contributes to bounding counterfactuals in causal inference and is the first to develop LP bounds in a dynamic HMM setting, benefiting educational contexts where counterfactual inference is taught.


title: Two-octave frequency combs from all-silica-fiber implementation

abstract: Mid-infrared frequency comb spectroscopy enables measurement of molecular at megahertz spectral resolution, sub-hertz frequency accuracy and microsecond acquisition speed. However, the widespread adoption of this technique has been hindered by the complexity and alignment sensitivity of mid-infrared frequency comb sources. Leveraging the underexplored mid-infrared window of silica fibers presents a promising approach to address these challenges. In this study, we present the first experimental demonstration and quantitative numerical description of mid-infrared frequency comb generation in silica fibers. Our all-silica-fiber frequency comb spans over two octaves (0.8 $\mu$m to 3.5 $\mu$m) with a power output of 100 mW in the mid-infrared region. The amplified quantum noise is suppressed using four-cycle (25 fs) driving pulses, with the carrier-envelope offset frequency exhibiting a signal-to-noise ratio of 40 dB and a free-running bandwidth of 90 kHz. Our developed model provides quantitative guidelines for mid-infrared frequency comb generation in silica fibers, enabling all-fiber frequency comb spectroscopy in diverse fields such as organic synthesis, pharmacokinetics processes, and environmental monitoring.


title: Generalizable and Scalable Multistage Biomedical Concept Normalization Leveraging Large Language Models

abstract: Background: Biomedical entity normalization is critical to biomedical research because the richness of free-text clinical data, such as progress notes, can often be fully leveraged only after translating words and phrases into structured and coded representations suitable for analysis. Large Language Models (LLMs), in turn, have shown great potential and high performance in a variety of natural language processing (NLP) tasks, but their application for normalization remains understudied. Methods: We applied both proprietary and open-source LLMs in combination with several rule-based normalization systems commonly used in biomedical research. We used a two-step LLM integration approach, (1) using an LLM to generate alternative phrasings of a source utterance, and (2) to prune candidate UMLS concepts, using a variety of prompting methods. We measure results by $F_{\beta}$, where we favor recall over precision, and F1. Results: We evaluated a total of 5,523 concept terms and text contexts from a publicly available dataset of human-annotated biomedical abstracts. Incorporating GPT-3.5-turbo increased overall $F_{\beta}$ and F1 in normalization systems +9.5 and +7.3 (MetaMapLite), +13.9 and +10.9 (QuickUMLS), and +10.5 and +10.3 (BM25), while the open-source Vicuna model achieved +10.8 and +12.2 (MetaMapLite), +14.7 and +15 (QuickUMLS), and +15.6 and +18.7 (BM25). Conclusions: Existing general-purpose LLMs, both propriety and open-source, can be leveraged at scale to greatly improve normalization performance using existing tools, with no fine-tuning.


title: Probeable Problems for Beginner-level Programming-with-AI Contests

abstract: To broaden participation, competitive programming contests may include beginner-level problems that do not require knowledge of advanced Computer Science concepts (e.g., algorithms and data structures). However, since most participants have easy access to AI code-generation tools, these problems often become trivial to solve. For beginner-friendly programming contests that do not prohibit the use of AI tools, we propose Probeable Problems: code writing tasks that provide (1) a problem specification that deliberately omits certain details, and (2) a mechanism to probe for these details by asking clarifying questions and receiving immediate feedback. To evaluate our proposal, we conducted a 2-hour programming contest for undergraduate Computer Science students from multiple institutions, where each student was an active member of their institution's computing club. The contest comprised of six Probeable Problems for which a popular code-generation tool (GitHub Copilot) was unable to generate accurate solutions due to the absence of details. Students were permitted to work individually or in groups, and were free to use AI tools. We obtained consent from 26 groups (67 students) to use their submissions for research. We analyze the extent to which the code submitted by these groups identifies missing details and identify ways in which Probeable Problems can support learning in formal and informal CS educational contexts.


title: Scaling Law for Time Series Forecasting

abstract: Scaling law that rewards large datasets, complex models and enhanced data granularity has been observed in various fields of deep learning. Yet, studies on time series forecasting have cast doubt on scaling behaviors of deep learning methods for time series forecasting: while more training data improves performance, more capable models do not always outperform less capable models, and longer input horizons may hurt performance for some models. We propose a theory for scaling law for time series forecasting that can explain these seemingly abnormal behaviors. We take into account the impact of dataset size and model complexity, as well as time series data granularity, particularly focusing on the look-back horizon, an aspect that has been unexplored in previous theories. Furthermore, we empirically evaluate various models using a diverse set of time series forecasting datasets, which (1) verifies the validity of scaling law on dataset size and model complexity within the realm of time series forecasting, and (2) validates our theoretical framework, particularly regarding the influence of look back horizon. We hope our findings may inspire new models targeting time series forecasting datasets of limited size, as well as large foundational datasets and models for time series forecasting in future works.\footnote{Codes for our experiments will be made public at: \url{https://github.com/JingzheShi/ScalingLawForTimeSeriesForecasting}.


title: HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting

abstract: High dynamic range (HDR) novel view synthesis (NVS) aims to create photorealistic images from novel viewpoints using HDR imaging techniques. The rendered HDR images capture a wider range of brightness levels containing more details of the scene than normal low dynamic range (LDR) images. Existing HDR NVS methods are mainly based on NeRF. They suffer from long training time and slow inference speed. In this paper, we propose a new framework, High Dynamic Range Gaussian Splatting (HDR-GS), which can efficiently render novel HDR views and reconstruct LDR images with a user input exposure time. Specifically, we design a Dual Dynamic Range (DDR) Gaussian point cloud model that uses spherical harmonics to fit HDR color and employs an MLP-based tone-mapper to render LDR color. The HDR and LDR colors are then fed into two Parallel Differentiable Rasterization (PDR) processes to reconstruct HDR and LDR views. To establish the data foundation for the research of 3D Gaussian splatting-based methods in HDR NVS, we recalibrate the camera parameters and compute the initial positions for Gaussian point clouds. Experiments demonstrate that our HDR-GS surpasses the state-of-the-art NeRF-based method by 3.84 and 1.91 dB on LDR and HDR NVS while enjoying 1000x inference speed and only requiring 6.3% training time. Code, models, and recalibrated data will be publicly available at https://github.com/caiyuanhao1998/HDR-GS


title: Imaging topological polar structures in marginally twisted 2D semiconductors

abstract: Moire superlattices formed in van der Waals heterostructures due to twisting, lattice mismatch and strain present an opportunity for creating novel metamaterials with unique properties not present in the individual layers themselves. Ferroelectricity for example, arises due to broken inversion symmetry in twisted and strained bilayers of 2D semiconductors with stacking domains of alternating out-of-plane polarization. However, understanding the individual contributions of twist and strain to the formation of topological polar nanostructures remains to be established and has proven to be experimentally challenging. Inversion symmetry breaking has been predicted to give rise to an in-plane component of polarization along the domain walls, leading to the formation of a network of topologically non-trivial merons (half-skyrmions) that are Bloch-type for twisted and Neel-type for strained systems. Here we utilise angle-resolved, high-resolution vector piezoresponse force microscopy (PFM) to spatially resolve polarization components and topological polar nanostructures in marginally twisted bilayer WSe2, and provide experimental proof for the existence of topologically non-trivial meron/antimeron structures. We observe both Bloch-type and Neel-type merons, allowing us to differentiate between moire superlattices formed due to twist or heterogeneous strain. This first demonstration of non-trivial real-space topology in a twisted van der Waals heterostructure opens pathways for exploring the connection between twist and topology in engineered nano-devices.


title: Benchmarking Hierarchical Image Pyramid Transformer for the classification of colon biopsies and polyps in histopathology images

abstract: Training neural networks with high-quality pixel-level annotation in histopathology whole-slide images (WSI) is an expensive process due to gigapixel resolution of WSIs. However, recent advances in self-supervised learning have shown that highly descriptive image representations can be learned without the need for annotations. We investigate the application of the recent Hierarchical Image Pyramid Transformer (HIPT) model for the specific task of classification of colorectal biopsies and polyps. After evaluating the effectiveness of TCGA-learned features in the original HIPT model, we incorporate colon biopsy image information into HIPT's pretraining using two distinct strategies: (1) fine-tuning HIPT from the existing TCGA weights and (2) pretraining HIPT from random weight initialization. We compare the performance of these pretraining regimes on two colorectal biopsy classification tasks: binary and multiclass classification.


title: Fluctuations around the mean-field limit for attractive Riesz potentials in the moderate regime

abstract: A central limit theorem is shown for moderately interacting particles in the whole space. The interaction potential approximates singular attractive or repulsive potentials of sub-Coulomb type. It is proved that the fluctuations become asymptotically Gaussians in the limit of infinitely many particles. The methodology is inspired by the classical work of Oelschl\"ager on fluctuations for the porous-medium equation. The novelty in this work is that we can allow for attractive potentials in the moderate regime and still obtain asymptotic Gaussian fluctuations. The key element of the proof is the mean-square convergence in expectation for smoothed empirical measures associated to moderately interacting $N$-particle systems with rate $N^{-1/2-\varepsilon}$ for some $\varepsilon>0$. To allow for attractive potentials, the proof uses a quantitative mean-field convergence in probability with any algebraic rate and a law-of-large-numbers estimate as well as a systematic separation of the terms to be estimated in a mean-field part and a law-of-large-numbers part.


title: ADMM for Nonsmooth Composite Optimization under Orthogonality Constraints

abstract: We consider a class of structured, nonconvex, nonsmooth optimization problems under orthogonality constraints, where the objectives combine a smooth function, a nonsmooth concave function, and a nonsmooth weakly convex function. This class of problems finds diverse applications in statistical learning and data science. Existing ADMMs for addressing these problems often fail to exploit the specific structure of orthogonality constraints, struggle with nonsmooth functions and nonconvex constraint sets, or result in suboptimal oracle complexity. We propose {\sf OADMM}, an Alternating Direction Method of Multipliers (ADMM) designed to solve this class of problems using efficient proximal linearized strategies. Two specific variants of {\sf OADMM} are explored: one based on Euclidean Projection ({\sf OADMM-EP}) and the other on Riemannian retraction ({\sf OADMM-RR}). We integrate a Nesterov extrapolation strategy into {\sf OADMM-EP} and a monotone Barzilai-Borwein strategy into {\sf OADMM-RR} to potentially accelerate primal convergence. Additionally, we adopt an over-relaxation strategy in both {\sf OADMM-EP} and {\sf OADMM-RR} for rapid dual convergence. Under mild assumptions, we prove that {\sf OADMM} converges to the critical point of the problem with a provable convergence rate of $\mathcal{O}(1/\epsilon^{3})$. We also establish the convergence rate of {\sf OADMM} under the Kurdyka-Lojasiewicz (KL) inequality. Numerical experiments are conducted to demonstrate the advantages of the proposed method.


title: OptLLM: Optimal Assignment of Queries to Large Language Models

abstract: Large Language Models (LLMs) have garnered considerable attention owing to their remarkable capabilities, leading to an increasing number of companies offering LLMs as services. Different LLMs achieve different performance at different costs. A challenge for users lies in choosing the LLMs that best fit their needs, balancing cost and performance. In this paper, we propose a framework for addressing the cost-effective query allocation problem for LLMs. Given a set of input queries and candidate LLMs, our framework, named OptLLM, provides users with a range of optimal solutions to choose from, aligning with their budget constraints and performance preferences, including options for maximizing accuracy and minimizing cost. OptLLM predicts the performance of candidate LLMs on each query using a multi-label classification model with uncertainty estimation and then iteratively generates a set of non-dominated solutions by destructing and reconstructing the current solution. To evaluate the effectiveness of OptLLM, we conduct extensive experiments on various types of tasks, including text classification, question answering, sentiment analysis, reasoning, and log parsing. Our experimental results demonstrate that OptLLM substantially reduces costs by 2.40% to 49.18% while achieving the same accuracy as the best LLM. Compared to other multi-objective optimization algorithms, OptLLM improves accuracy by 2.94% to 69.05% at the same cost or saves costs by 8.79% and 95.87% while maintaining the highest attainable accuracy.


title: Computational toolkit for predicting thickness of 2D materials using machine learning and autogenerated dataset by large language model

abstract: The thickness of 2D materials not only plays a crucial role in determining the performance of nanoelectronic and optoelectronic devices but also introduces complexities in predicting volume-dependent properties such as energy storage capacity, due to the intrinsic vacuum within these materials. Although a plethora of experimental techniques, including but not limited to optical contrast, Raman spectroscopy, nonlinear optical spectroscopy, near-field optical imaging, and hyperspectral imaging, facilitate the measurement of 2D material thickness, comprehensive data for many materials remains elusive. Over the last decade, the exponential proliferation of 2D materials and their heterostructures has outstripped the capabilities of conventional experimental and computational approaches. In this evolving landscape, machine learning (ML) has emerged as an indispensable tool, offering novel avenues to augment these traditional methodologies. Addressing the critical gap, we introduce THICK2D - Thickness Hierarchy Inference and Calculation Kit for 2D Materials. This Python-based computational framework harnesses an autogenerated thickness database, developed using large language models (LLMs), and advanced ML algorithms to facilitate the rapid and scalable estimation of material thickness, relying solely on crystallographic data. To demonstrate the utility and robustness of THICK2D, we successfully employed the toolkit to predict the thickness of more than 8000 2D-based materials, sourced from two extensive 2D material databases. THICK2D is disseminated as an open-source utility, accessible on GitHub https://github.com/gmp007/THICK2D, and archived on Zenodo at https://doi.org/10.5281/zenodo.11216648}{10.5281/zenodo.11216648.


title: Beyond the noise: intrinsic dimension estimation with optimal neighbourhood identification

abstract: The Intrinsic Dimension (ID) is a key concept in unsupervised learning and feature selection, as it is a lower bound to the number of variables which are necessary to describe a system. However, in almost any real-world dataset the ID depends on the scale at which the data are analysed. Quite typically at a small scale, the ID is very large, as the data are affected by measurement errors. At large scale, the ID can also be erroneously large, due to the curvature and the topology of the manifold containing the data. In this work, we introduce an automatic protocol to select the sweet spot, namely the correct range of scales in which the ID is meaningful and useful. This protocol is based on imposing that for distances smaller than the correct scale the density of the data is constant. Since to estimate the density it is necessary to know the ID, this condition is imposed self-consistently. We illustrate the usefulness and robustness of this procedure by benchmarks on artificial and real-world datasets.

categories: stat.ML cs.LG math.ST stat.CO stat.ME stat.TH


title: Modeling of Hydroacoustic Noise from Marine Propellers with Tip Vortex Cavitation

abstract: The present work aims to study the cavitating turbulent flow of a full-scale marine propeller and explore the physical mechanism underpinning the underwater radiated noise. We employ the standard dynamic large eddy simulation for the turbulent wake flow and the Schnerr-Sauer cavitation model, while the Ffowcs-Williams-Hawkings acoustic analogy is considered for the hydroacoustic modeling. For the current investigation, we consider a well-known Potsdam Propeller Test Case to analyze the turbulent cavitating flow and the associated hydroacoustic emissions. To begin, the modeling framework is validated using the available experimental data, and distinctive double-helical tip vortex cavitation and its qualitative patterns along the vortex trajectory are captured. In comparison to the non-cavitating condition, the pressure distribution on the propeller surface is more disordered for the cavitating condition, which is further reflected by a relatively stronger power of both low-frequency tonal peaks and high-frequency broadband components in the spectrum of thrust generation. Specifically, the generation of cavitation leads to the enhancement of the monopole noise source and the breakdown of cavitation bubbles as well as vortex structures in the turbulent wake. Furthermore, the tonal noise with the frequency corresponding to the harmonics of blade passing frequency is also enhanced. Generally speaking, the generation of cavitation structures enhances the hydroacoustics energy of URN at all orientations, especially in the downstream direction with sound pressure level increasing up to 20 dB.


title: Efficient Biomedical Entity Linking: Clinical Text Standardization with Low-Resource Techniques

abstract: Clinical text is rich in information, with mentions of treatment, medication and anatomy among many other clinical terms. Multiple terms can refer to the same core concepts which can be referred as a clinical entity. Ontologies like the Unified Medical Language System (UMLS) are developed and maintained to store millions of clinical entities including the definitions, relations and other corresponding information. These ontologies are used for standardization of clinical text by normalizing varying surface forms of a clinical term through Biomedical entity linking. With the introduction of transformer-based language models, there has been significant progress in Biomedical entity linking. In this work, we focus on learning through synonym pairs associated with the entities. As compared to the existing approaches, our approach significantly reduces the training data and resource consumption. Moreover, we propose a suite of context-based and context-less reranking techniques for performing the entity disambiguation. Overall, we achieve similar performance to the state-of-the-art zero-shot and distant supervised entity linking techniques on the Medmentions dataset, the largest annotated dataset on UMLS, without any domain-based training. Finally, we show that retrieval performance alone might not be sufficient as an evaluation metric and introduce an article level quantitative and qualitative analysis to reveal further insights on the performance of entity linking methods.


title: Exploring the Evolution of Hidden Activations with Live-Update Visualization

abstract: Monitoring the training of neural networks is essential for identifying potential data anomalies, enabling timely interventions and conserving significant computational resources. Apart from the commonly used metrics such as losses and validation accuracies, the hidden representation could give more insight into the model progression. To this end, we introduce SentryCam, an automated, real-time visualization tool that reveals the progression of hidden representations during training. Our results show that this visualization offers a more comprehensive view of the learning dynamics compared to basic metrics such as loss and accuracy over various datasets. Furthermore, we show that SentryCam could facilitate detailed analysis such as task transfer and catastrophic forgetting to a continual learning setting. The code is available at https://github.com/xianglinyang/SentryCam.


title: Convergence analysis of a weak Galerkin finite element method on a Bakhvalov-type mesh for a singularly perturbed convection-diffusion equation in 2D

abstract: In this paper, we propose a weak Galerkin finite element method (WG) for solving singularly perturbed convection-diffusion problems on a Bakhvalov-type mesh in 2D. Our method is flexible and allows the use of discontinuous approximation functions on the meshe. An error estimate is devised in a suitable norm and the optimal convergence order is obtained. Finally, numerical experiments are given to support the theory and to show the efficiency of the proposed method.


title: An Approximate Dynamic Programming Framework for Occlusion-Robust Multi-Object Tracking

abstract: In this work, we consider data association problems involving multi-object tracking (MOT). In particular, we address the challenges arising from object occlusions. We propose a framework called approximate dynamic programming track (ADPTrack), which applies dynamic programming principles to improve an existing method called the base heuristic. Given a set of tracks and the next target frame, the base heuristic extends the tracks by matching them to the objects of this target frame directly. In contrast, ADPTrack first processes a few subsequent frames and applies the base heuristic starting from the next target frame to obtain tentative tracks. It then leverages the tentative tracks to match the objects of the target frame. This tends to reduce the occlusion-based errors and leads to an improvement over the base heuristic. When tested on the MOT17 video dataset, the proposed method demonstrates a 0.7% improvement in the association accuracy (IDF1 metric) over a state-of-the-art method that is used as the base heuristic. It also obtains improvements with respect to all the other standard metrics. Empirically, we found that the improvements are particularly pronounced in scenarios where the video data is obtained by fixed-position cameras.


title: Particle acceleration and magnetic field amplification by relativistic shocks in inhomogeneous media

abstract: Particle acceleration and magnetic field amplification in relativistic shocks propagating in inhomogeneous media are investigated by three-dimensional magnetohydrodynamical (MHD) simulations and test-particle simulations. The MHD simulations show that the interaction between the relativistic shock and dense clumps amplifies the downstream magnetic field to the value expected from observations of the gamma-ray burst. The test-particle simulations in the electromagnetic field given by the MHD simulation show that particles are accelerated by the downstream turbulence and the relativistic shock. We provide the injection energy to the shock acceleration in this system. If the amplitude of upstream density fluctuations is sufficiently large, low-energy particles are initially accelerated to the injection energy by the downstream turbulence and then rapidly accelerated to higher energies by the relativistic shock. Therefore, the density fluctuation significantly affects particle acceleration in the relativistic shock.


title: Self-Gravitational Dynamics Within the Inner Oort Cloud

abstract: The formation of the Inner Oort Cloud (IOC) - a vast halo of icy bodies residing far beyond Neptune's orbit - is an expected outcome of the solar system's primordial evolution within a stellar cluster. Recent models have shown that the process of early planetesimal capture within the trans-Neptunian region may have been sufficiently high for the cumulative mass of the Cloud to approach several Earth masses. In light of this, here we examine the dynamical evolution of the IOC, driven by its own self-gravity. We show that the collective gravitational potential of the IOC is adequately approximated by the Miyamoto-Nagai model, and use a semi-analytic framework to demonstrate that the resulting secular oscillations are akin to the von Zeipel-Lidov-Kozai resonance. We verify our results with direct $N$-body calculations, and examine the effects of IOC self-gravity on the long-term behavior of the solar system's minor bodies using a detailed simulation. Cumulatively, we find that while the modulation of perihelion distances and inclinations can occur within an observationally relevant range, the associated timescales vastly surpass the age of the sun, indicating that the influence of IOC self-gravity on the architecture of the solar system is negligible.


title: Better Membership Inference Privacy Measurement through Discrepancy

abstract: Membership Inference Attacks have emerged as a dominant method for empirically measuring privacy leakage from machine learning models. Here, privacy is measured by the {\em{advantage}} or gap between a score or a function computed on the training and the test data. A major barrier to the practical deployment of these attacks is that they do not scale to large well-generalized models -- either the advantage is relatively low, or the attack involves training multiple models which is highly compute-intensive. In this work, inspired by discrepancy theory, we propose a new empirical privacy metric that is an upper bound on the advantage of a family of membership inference attacks. We show that this metric does not involve training multiple models, can be applied to large Imagenet classification models in-the-wild, and has higher advantage than existing metrics on models trained with more recent and sophisticated training recipes. Motivated by our empirical results, we also propose new membership inference attacks tailored to these training losses.


title: Likelihood distortion and Bayesian local robustness

abstract: Robust Bayesian analysis has been mainly devoted to detecting and measuring robustness to the prior distribution. Indeed, many contributions in the literature aim to define suitable classes of priors which allow the computation of variations of quantities of interest while the prior changes within those classes. The literature has devoted much less attention to the robustness of Bayesian methods to the likelihood function due to mathematical and computational complexity, and because it is often arguably considered a more objective choice compared to the prior. In this contribution, a new approach to Bayesian local robustness to the likelihood function is proposed and extended to robustness to the prior and to both. This approach is based on the notion of distortion function introduced in the literature on risk theory, and then successfully adopted to build suitable classes of priors for Bayesian global robustness to the prior. The novel robustness measure is a local sensitivity measure that turns out to be very tractable and easy to compute for certain classes of distortion functions. Asymptotic properties are derived and numerical experiments illustrate the theory and its applicability for modelling purposes.


title: Characterization of Gradient Condition for Asymmetric Partial Exclusion Processes and Their Scaling Limits

abstract: We consider partial exclusion processes~(PEPs) on the one-dimensional square lattice, that is, a system of interacting particles where each particle random walks according to a jump rate satisfying an exclusion rule that allows up to a certain number of particles can exist on each site. Particularly, we assume that the jump rate is given as a product of two functions depending on occupation variables on the original and target sites. Our interest is to study the limiting behavior, especially to derive some macroscopic PDEs by means of (fluctuating) hydrodynamics, of fluctuation fields associated with PEPs, starting from an invariant measure. The so-called gradient condition, meaning that the symmetric part of the instantaneous current is written in a gradient form, and that the invariant measures are given as a product measure is technically crucial. Our first main result is to clarify the relationship between these two conditions, and we show that the gradient condition and the existence of product invariant measures are mutually equivalent, provided the jump rate is given in the above simple form, as it is imposed in most of the literature, and the dynamics is asymmetric. Moreover, when the width of the lattice tends to zero and the process is accelerated in diffusive time-scaling, we show that the family of fluctuation fields converges to the stationary energy solution of the stochastic Burgers equation (SBE), under the setting that the jump rate to the right neighboring site is a bit larger than the one to the left side, of which discrepancy is given as square root of the width of the underlying lattice. This fills the gap at the level of universality of SBE since it has been proved for the exclusion process (a special case of PEP) and for the zero-range process.


title: Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models

abstract: Go-Explore is a powerful family of algorithms designed to solve hard-exploration problems, built on the principle of archiving discovered states, and iteratively returning to and exploring from the most promising states. This approach has led to superhuman performance across a wide variety of challenging problems including Atari games and robotic control, but requires manually designing heuristics to guide exploration, which is time-consuming and infeasible in general. To resolve this, we propose Intelligent Go-Explore (IGE) which greatly extends the scope of the original Go-Explore by replacing these heuristics with the intelligence and internalized human notions of interestingness captured by giant foundation models (FMs). This provides IGE with a human-like ability to instinctively identify how interesting or promising any new state is (e.g. discovering new objects, locations, or behaviors), even in complex environments where heuristics are hard to define. Moreover, IGE offers the exciting and previously impossible opportunity to recognize and capitalize on serendipitous discoveries that cannot be predicted ahead of time. We evaluate IGE on a range of language-based tasks that require search and exploration. In Game of 24, a multistep mathematical reasoning problem, IGE reaches 100% success rate 70.8% faster than the best classic graph search baseline. Next, in BabyAI-Text, a challenging partially observable gridworld, IGE exceeds the previous SOTA with orders of magnitude fewer online samples. Finally, in TextWorld, we show the unique ability of IGE to succeed in settings requiring long-horizon exploration where prior SOTA FM agents like Reflexion completely fail. Overall, IGE combines the tremendous strengths of FMs and the powerful Go-Explore algorithm, opening up a new frontier of research into creating more generally capable agents with impressive exploration capabilities.


title: Ultra-sensitive solid-state organic molecular microwave quantum receiver

abstract: High-accuracy microwave sensing is widely demanded in various fields, ranging from cosmology to microwave quantum technology. Quantum receivers based on inorganic solid-state spin systems are promising candidates for such purpose because of the stability and compatibility, but their best sensitivity is currently limited to a few pT/$\sqrt{\rm{Hz}}$. Here, by utilising an enhanced readout scheme with the state-of-the-art solid-state maser technology, we develop a robust microwave quantum receiver functioned by organic molecular spins at ambient conditions. Owing to the maser amplification, the sensitivity of the receiver achieves 6.14 $\pm$ 0.17 fT/$\sqrt{\rm{Hz}}$ which exceeds three orders of magnitude than that of the inorganic solid-state quantum receivers. The heterodyne detection without additional local oscillators improves bandwidth of the receiver and allows frequency detection. The scheme can be extended to other solid-state spin systems without complicated control pulses and thus enables practical applications such as electron spin resonance spectroscopy, dark matter searches, and astronomical observations.


title: CulturePark: Boosting Cross-cultural Understanding in Large Language Models

abstract: Cultural bias is pervasive in many large language models (LLMs), largely due to the deficiency of data representative of different cultures. Typically, cultural datasets and benchmarks are constructed either by extracting subsets of existing datasets or by aggregating from platforms such as Wikipedia and social media. However, these approaches are highly dependent on real-world data and human annotations, making them costly and difficult to scale. Inspired by cognitive theories on social communication, this paper introduces CulturePark, an LLM-powered multi-agent communication framework for cultural data collection. CulturePark simulates cross-cultural human communication with LLM-based agents playing roles in different cultures. It generates high-quality cross-cultural dialogues encapsulating human beliefs, norms, and customs. Using CulturePark, we generated 41,000 cultural samples to fine-tune eight culture-specific LLMs. We evaluated these models across three downstream tasks: content moderation, cultural alignment, and cultural education. Results show that for content moderation, our GPT-3.5-based models either match or outperform GPT-4 on datasets. Regarding cultural alignment, our models surpass GPT-4 on Hofstede's VSM 13 framework. Furthermore, for cultural education of human participants, our models demonstrate superior outcomes in both learning efficacy and user experience compared to GPT-4. CulturePark proves an important step in addressing cultural bias and advancing the democratization of AI, highlighting the critical role of culturally inclusive data in model training.


title: On the acceptance, commissioning, and quality assurance of electron FLASH units

abstract: Background & Purpose: FLASH or ultra-high dose rate (UHDR) radiation therapy (RT) has gained attention in recent years for its ability to spare normal tissues relative to conventional dose rate (CDR) RT in various preclinical trials. However, clinical implementation of this promising treatment option has been limited because of the lack of availability of accelerators capable of delivering UHDR RT. We established a framework for the acceptance, commissioning, and periodic quality assurance (QA) of electron FLASH units and present an example of commissioning. Methods: A protocol for acceptance, commissioning, and QA of UHDR linear accelerators was established by combining and adapting standards and professional recommendations for standard linear accelerators based on the experience with UHDR at four clinical centers that use different UHDR devices. Non-standard dosimetric beam parameters considered included pulse width, pulse repetition frequency, dose per pulse, and instantaneous dose rate, together with recommendations on how to acquire these measurements. Results: The 6 and 9 MeV beams of an UHDR electron device were commissioned by using this developed protocol. Measurements were acquired with a combination of ion chambers, beam current transformers (BCTs), and dose rate independent passive dosimeters. The unit was calibrated according to the concept of redundant dosimetry using a reference setup. Conclusions: This study provides detailed recommendations for the acceptance testing, commissioning, and routine QA of low-energy electron UHDR linear accelerators. The proposed framework is not limited to any specific unit, making it applicable to all existing eFLASH units in the market. Through practical insights and theoretical discourse, this document establishes a benchmark for the commissioning of UHDR devices for clinical use.


title: The generalized 4-connectivity of godan graphs

abstract: The generalized $k$-connectivity of a graph $G$, denoted by $\kappa_k(G)$, is the minimum number of internally edge disjoint $S$-trees for any $S\subseteq V(G)$ and $|S|=k$. The generalized $k$-connectivity is a natural extension of the classical connectivity and plays a key role in applications related to the modern interconnection networks. The godan graph $EA_n$ is a kind of Cayley graphs which posses many desirable properties. In this paper, we shall study the generalized 4-connectivity of $EA_n$ and show that $\kappa_4(EA_n)=n-1$ for $n\ge 3$.


title: Dynamically corrected gates in silicon singlet-triplet spin qubits

abstract: Fault-tolerant quantum computation requires low physical-qubit gate errors. Many approaches exist to reduce gate errors, including both hardware- and control-optimization strategies. Dynamically corrected gates are designed to cancel specific errors and offer the potential for high-fidelity gates, but they have yet to be implemented in singlet-triplet spin qubits in semiconductor quantum dots, due in part to the stringent control constraints in these systems. In this work, we experimentally implement dynamically corrected gates designed to mitigate hyperfine noise in a singlet-triplet qubit realized in a Si/SiGe double quantum dot. The corrected gates reduce infidelities by about a factor of three, resulting in gate fidelities above 0.99 for both identity and Hadamard gates. The gate performances depend sensitively on pulse distortions, and their specific performance reveals an unexpected distortion in our experimental setup.


title: Uniform Calder\'{o}n-Zygmund estimates in multiscale elliptic homogenization

abstract: This paper is concerned with the elliptic equation $-\text{div} (A_\varepsilon \nabla u_\varepsilon) = \text{div} f$ in a bounded $C^1$ domain, where $A_\varepsilon$ takes a form of $A_\varepsilon(x) = A(x/\varepsilon_1, x/\varepsilon_2,\cdots, x/\varepsilon_n)$, with $A(y_1,y_2,\cdots,y_n)$ being 1-periodic in each $y_i$. We prove the uniform Calder\'{o}n-Zygmund estimate, namely, the uniform $L^p$ boundedness of the linear map $f\mapsto \nabla u_\varepsilon$ for any $p\in (1,\infty)$ with a constant independent of small parameters $(\varepsilon_1,\varepsilon_2,\cdots, \varepsilon_n) \in (0,1]^n$. Our result includes the uniform Calder\'{o}n-Zygmund estimate in quasiperiodic elliptic homogenization (even without the Diophantine condition), which was previously unknown. The proof novelly combines the Dirichlet's theorem on the simultaneous Diophantine approximation from number theory, a technique of reperiodization, reiterated periodic homogenization and a large-scale real-variable argument. Using the idea of reperiodization, we also obtain some large-scale or mesoscopic-scale Lipschitz estimates.


title: Enhancing Learning with Label Differential Privacy by Vector Approximation

abstract: Label differential privacy (DP) is a framework that protects the privacy of labels in training datasets, while the feature vectors are public. Existing approaches protect the privacy of labels by flipping them randomly, and then train a model to make the output approximate the privatized label. However, as the number of classes $K$ increases, stronger randomization is needed, thus the performances of these methods become significantly worse. In this paper, we propose a vector approximation approach, which is easy to implement and introduces little additional computational overhead. Instead of flipping each label into a single scalar, our method converts each label into a random vector with $K$ components, whose expectations reflect class conditional probabilities. Intuitively, vector approximation retains more information than scalar labels. A brief theoretical analysis shows that the performance of our method only decays slightly with $K$. Finally, we conduct experiments on both synthesized and real datasets, which validate our theoretical analysis as well as the practical performance of our method.


title: NeB-SLAM: Neural Blocks-based Salable RGB-D SLAM for Unknown Scenes

abstract: Neural implicit representations have recently demonstrated considerable potential in the field of visual simultaneous localization and mapping (SLAM). This is due to their inherent advantages, including low storage overhead and representation continuity. However, these methods necessitate the size of the scene as input, which is impractical for unknown scenes. Consequently, we propose NeB-SLAM, a neural block-based scalable RGB-D SLAM for unknown scenes. Specifically, we first propose a divide-and-conquer mapping strategy that represents the entire unknown scene as a set of sub-maps. These sub-maps are a set of neural blocks of fixed size. Then, we introduce an adaptive map growth strategy to achieve adaptive allocation of neural blocks during camera tracking and gradually cover the whole unknown scene. Finally, extensive evaluations on various datasets demonstrate that our method is competitive in both mapping and tracking when targeting unknown environments.


title: Machine Unlearning in Large Language Models

abstract: Machine unlearning, a novel area within artificial intelligence, focuses on addressing the challenge of selectively forgetting or reducing undesirable knowledge or behaviors in machine learning models, particularly in the context of large language models (LLMs). This paper introduces a methodology to align LLMs, such as Open Pre-trained Transformer Language Models, with ethical, privacy, and safety standards by leveraging the gradient ascent algorithm for knowledge unlearning. Our approach aims to selectively erase or modify learned information in LLMs, targeting harmful responses and copyrighted content. This paper presents a dual-pronged approach to enhance the ethical and safe behavior of large language models (LLMs) by addressing the issues of harmful responses and copyrighted content. To mitigate harmful responses, we applied gradient ascent on the PKU dataset, achieving a 75\% reduction in harmful responses for Open Pre-trained Transformer Language Models (OPT1.3b and OPT2.7b) \citet{zhang2022opt} while retaining previous knowledge using the TruthfulQA dataset \citet{DBLP:journals/corr/abs-2109-07958}. For handling copyrighted content, we constructed a custom dataset based on the Lord of the Rings corpus and aligned LLMs (OPT1.3b and OPT2.7b) \citet{zhang2022opt} through LoRA: Low-Rank Adaptation of Large Language Models \citet{DBLP:journals/corr/abs-2106-09685} finetuning. Subsequently, we employed gradient ascent to unlearn the Lord of the Rings content, resulting in a remarkable reduction in the presence of copyrighted material. To maintain a diverse knowledge base, we utilized the Book Corpus dataset. Additionally, we propose a new evaluation technique for assessing the effectiveness of harmful unlearning.


title: Optimal Reference Nodes Deployment for Positioning Seafloor Anchor Nodes

abstract: Seafloor anchor nodes, which form a geodetic network, are designed to provide surface and underwater users with positioning, navigation and timing (PNT) services. Due to the non-uniform distribution of underwater sound speed, accurate positioning of underwater anchor nodes is a challenge work. Traditional anchor node positioning typically uses cross or circular shapes, however, how to optimize the deployment of reference nodes for positioning underwater anchor nodes considering the variability of sound speed has not yet been studied. This paper focuses on the optimal reference nodes deployment strategies for time--of--arrival (TOA) localization in the three-dimensional (3D) underwater space. We adopt the criterion that minimizing the trace of the inverse Fisher information matrix (FIM) to determine optimal reference nodes deployment with Gaussian measurement noise, which is positive related to the signal propagation path. A comprehensive analysis of optimal reference-target geometries is provided in the general circumstance with no restriction on the number of reference nodes, elevation angle and reference-target range. A new semi-closed form solution is found to detemine the optimal geometries. To demonstrate the findings in this paper, we conducted both simulations and sea trials on underwater anchor node positioning. Both the simulation and experiment results are consistent with theoretical analysis.


title: Online Prompt Pricing based on Combinatorial Multi-Armed Bandit and Hierarchical Stackelberg Game

abstract: Generation models have shown promising performance in various tasks, making trading around machine learning models possible. In this paper, we aim at a novel prompt trading scenario, prompt bundle trading (PBT) system, and propose an online pricing mechanism. Based on the combinatorial multi-armed bandit (CMAB) and three-stage hierarchical Stackelburg (HS) game, our pricing mechanism considers the profits of the consumer, platform, and seller, simultaneously achieving the profit satisfaction of these three participants. We break down the pricing issue into two steps, namely unknown category selection and incentive strategy optimization. The former step is to select a set of categories with the highest qualities, and the latter is to derive the optimal strategy for each participant based on the chosen categories. Unlike the existing fixed pricing mode, the PBT pricing mechanism we propose is more flexible and diverse, which is more in accord with the transaction needs of real-world scenarios. We test our method on a simulated text-to-image dataset. The experimental results demonstrate the effectiveness of our algorithm, which provides a feasible price-setting standard for the prompt marketplaces.


title: CLIP model is an Efficient Online Lifelong Learner

abstract: Online Lifelong Learning (OLL) addresses the challenge of learning from continuous and non-stationary data streams. Existing online lifelong learning methods based on image classification models often require preset conditions such as the total number of classes or maximum memory capacity, which hinders the realization of real never-ending learning and renders them impractical for real-world scenarios. In this work, we propose that vision-language models, such as Contrastive Language-Image Pretraining (CLIP), are more suitable candidates for online lifelong learning. We discover that maintaining symmetry between image and text is crucial during Parameter-Efficient Tuning (PET) for CLIP model in online lifelong learning. To this end, we introduce the Symmetric Image-Text (SIT) tuning strategy. We conduct extensive experiments on multiple lifelong learning benchmark datasets and elucidate the effectiveness of SIT through gradient analysis. Additionally, we assess the impact of lifelong learning on generalizability of CLIP and found that tuning the image encoder is beneficial for lifelong learning, while tuning the text encoder aids in zero-shot learning.


title: Spectral fittings of warm coronal radiation with high seed photon temperature: apparent low-temperature and flat soft excess in AGNs

abstract: A warm corona has been widely proposed to explain the soft X-ray excess (SE) above the 2--10 keV power law extrapolation in AGNs. In actual spectral fittings, the warm coronal seed photon temperature ($T_{\rm s}$) is usually assumed to be far away from the soft X-ray, but $kT_{\rm s}$ can reach close to 0.1 keV in standard accretion disc model. In this study, we used Monte Carlo simulations to obtain radiation spectra from a slab-like warm corona and fitted the spectra using the spherical-geometry-based routine \textsc{thcomp} or a thermal component. Our findings reveal that high $T_{\rm s}$ can influence the fitting results. A moderately high $kT_{\rm s}$ (around 0.03 keV) can result in an apparent low-temperature and flat SE, while an extremely high $kT_{\rm s}$ (around 0.07 keV) can even produce an unobserved blackbody-like SE. Our conclusions indicate that, for spectral fittings of the warm coronal radiation (SE in AGNs), $kT_{\rm s}$ should be treated as a free parameter with an upper limit, and an accurate coronal geometry is necessary when $kT_{\rm s}>0.01$ keV.


title: Rethinking Class-Incremental Learning from a Dynamic Imbalanced Learning Perspective

abstract: Deep neural networks suffer from catastrophic forgetting when continually learning new concepts. In this paper, we analyze this problem from a data imbalance point of view. We argue that the imbalance between old task and new task data contributes to forgetting of the old tasks. Moreover, the increasing imbalance ratio during incremental learning further aggravates the problem. To address the dynamic imbalance issue, we propose Uniform Prototype Contrastive Learning (UPCL), where uniform and compact features are learned. Specifically, we generate a set of non-learnable uniform prototypes before each task starts. Then we assign these uniform prototypes to each class and guide the feature learning through prototype contrastive learning. We also dynamically adjust the relative margin between old and new classes so that the feature distribution will be maintained balanced and compact. Finally, we demonstrate through extensive experiments that the proposed method achieves state-of-the-art performance on several benchmark datasets including CIFAR100, ImageNet100 and TinyImageNet.


title: ProtFAD: Introducing function-aware domains as implicit modality towards protein function perception

abstract: Protein function prediction is currently achieved by encoding its sequence or structure, where the sequence-to-function transcendence and high-quality structural data scarcity lead to obvious performance bottlenecks. Protein domains are "building blocks" of proteins that are functionally independent, and their combinations determine the diverse biological functions. However, most existing studies have yet to thoroughly explore the intricate functional information contained in the protein domains. To fill this gap, we propose a synergistic integration approach for a function-aware domain representation, and a domain-joint contrastive learning strategy to distinguish different protein functions while aligning the modalities. Specifically, we associate domains with the GO terms as function priors to pre-train domain embeddings. Furthermore, we partition proteins into multiple sub-views based on continuous joint domains for contrastive training under the supervision of a novel triplet InfoNCE loss. Our approach significantly and comprehensively outperforms the state-of-the-art methods on various benchmarks, and clearly differentiates proteins carrying distinct functions compared to the competitor.


title: Leveraging Gated Recurrent Units for Iterative Online Precise Attitude Control for Geodetic Missions

abstract: In this paper, we consider the problem of precise attitude control for geodetic missions, such as the GRACE Follow-on (GRACE-FO) mission. Traditional and well-established control methods, such as Proportional-Integral-Derivative (PID) controllers, have been the standard in attitude control for most space missions, including the GRACE-FO mission. Instead of significantly modifying (or replacing) the original PID controllers that are being used for these missions, we introduce an iterative modification to the PID controller that ensures improved attitude control precision (i.e., reduction in attitude error). The proposed modification leverages Gated Recurrent Units (GRU) to learn and predict external disturbance trends derived from incoming attitude measurements from the GRACE satellites. Our analysis has revealed a distinct trend in the external disturbance time-series data, suggesting the potential utility of GRU's to predict future disturbances acting on the system. The learned GRU model compensates for these disturbances within the standard PID control loop in real time via an additive correction term which is updated at regular time intervals. The simulation results verify the significant reduction in attitude error, verifying the efficacy of our proposed approach.


title: ARVideo: Autoregressive Pretraining for Self-Supervised Video Representation Learning

abstract: This paper presents a new self-supervised video representation learning framework, ARVideo, which autoregressively predicts the next video token in a tailored sequence order. Two key designs are included. First, we organize autoregressive video tokens into clusters that span both spatially and temporally, thereby enabling a richer aggregation of contextual information compared to the standard spatial-only or temporal-only clusters. Second, we adopt a randomized spatiotemporal prediction order to facilitate learning from multi-dimensional data, addressing the limitations of a handcrafted spatial-first or temporal-first sequence order. Extensive experiments establish ARVideo as an effective paradigm for self-supervised video representation learning. For example, when trained with the ViT-B backbone, ARVideo competitively attains 81.2% on Kinetics-400 and 70.9% on Something-Something V2, which are on par with the strong benchmark set by VideoMAE. Importantly, ARVideo also demonstrates higher training efficiency, i.e., it trains 14% faster and requires 58% less GPU memory compared to VideoMAE.