title: Probabilistic Inference in the Era of Tensor Networks and Differential Programming

abstract: Probabilistic inference is a fundamental task in modern machine learning. Recent advances in tensor network (TN) contraction algorithms have enabled the development of better exact inference methods. However, many common inference tasks in probabilistic graphical models (PGMs) still lack corresponding TN-based adaptations. In this work, we advance the connection between PGMs and TNs by formulating and implementing tensor-based solutions for the following inference tasks: (i) computing the partition function, (ii) computing the marginal probability of sets of variables in the model, (iii) determining the most likely assignment to a set of variables, and (iv) the same as (iii) but after having marginalized a different set of variables. We also present a generalized method for generating samples from a learned probability distribution. Our work is motivated by recent technical advances in the fields of quantum circuit simulation, quantum many-body physics, and statistical physics. Through an experimental evaluation, we demonstrate that the integration of these quantum technologies with a series of algorithms introduced in this study significantly improves the effectiveness of existing methods for solving probabilistic inference tasks.


title: Meanings and Feelings of Large Language Models: Observability of Latent States in Generative AI

abstract: We tackle the question of whether Large Language Models (LLMs), viewed as dynamical systems with state evolving in the embedding space of symbolic tokens, are observable. That is, whether there exist multiple 'mental' state trajectories that yield the same sequence of generated tokens, or sequences that belong to the same Nerode equivalence class ('meaning'). If not observable, mental state trajectories ('experiences') evoked by an input ('perception') or by feedback from the model's own state ('thoughts') could remain self-contained and evolve unbeknown to the user while being potentially accessible to the model provider. Such "self-contained experiences evoked by perception or thought" are akin to what the American Psychological Association (APA) defines as 'feelings'. Beyond the lexical curiosity, we show that current LLMs implemented by autoregressive Transformers cannot have 'feelings' according to this definition: The set of state trajectories indistinguishable from the tokenized output is a singleton. But if there are 'system prompts' not visible to the user, then the set of indistinguishable trajectories becomes non-trivial, and there can be multiple state trajectories that yield the same verbalized output. We prove these claims analytically, and show examples of modifications to standard LLMs that engender such 'feelings.' Our analysis sheds light on possible designs that would enable a model to perform non-trivial computation that is not visible to the user, as well as on controls that the provider of services using the model could take to prevent unintended behavior.


title: ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles

abstract: We present ChatScene, a Large Language Model (LLM)-based agent that leverages the capabilities of LLMs to generate safety-critical scenarios for autonomous vehicles. Given unstructured language instructions, the agent first generates textually described traffic scenarios using LLMs. These scenario descriptions are subsequently broken down into several sub-descriptions for specified details such as behaviors and locations of vehicles. The agent then distinctively transforms the textually described sub-scenarios into domain-specific languages, which then generate actual code for prediction and control in simulators, facilitating the creation of diverse and complex scenarios within the CARLA simulation environment. A key part of our agent is a comprehensive knowledge retrieval component, which efficiently translates specific textual descriptions into corresponding domain-specific code snippets by training a knowledge database containing the scenario description and code pairs. Extensive experimental results underscore the efficacy of ChatScene in improving the safety of autonomous vehicles. For instance, the scenarios generated by ChatScene show a 15% increase in collision rates compared to state-of-the-art baselines when tested against different reinforcement learning-based ego vehicles. Furthermore, we show that by using our generated safety-critical scenarios to fine-tune different RL-based autonomous driving models, they can achieve a 9% reduction in collision rates, surpassing current SOTA methods. ChatScene effectively bridges the gap between textual descriptions of traffic scenarios and practical CARLA simulations, providing a unified way to conveniently generate safety-critical scenarios for safety testing and improvement for AVs.


title: On sets of orthogonal exponentials on the disk

abstract: We show that if $A$ is a set of mutually orthogonal exponentials with respect to the unit disk then $|A \cap [-R, R]^2| \lesssim_\varepsilon R^{3/5+\varepsilon}$ holds. This improves the previous bound of $R^{2/3}$ by Iosevich--Kolountzakis. The main new ingredient in the proof is a variant of Marstrand's projection theorem.


title: Building a stable classifier with the inflated argmax

abstract: We propose a new framework for algorithmic stability in the context of multiclass classification. In practice, classification algorithms often operate by first assigning a continuous score (for instance, an estimated probability) to each possible label, then taking the maximizer -- i.e., selecting the class that has the highest score. A drawback of this type of approach is that it is inherently unstable, meaning that it is very sensitive to slight perturbations of the training data, since taking the maximizer is discontinuous. Motivated by this challenge, we propose a pipeline for constructing stable classifiers from data, using bagging (i.e., resampling and averaging) to produce stable continuous scores, and then using a stable relaxation of argmax, which we call the "inflated argmax," to convert these scores to a set of candidate labels. The resulting stability guarantee places no distributional assumptions on the data, does not depend on the number of classes or dimensionality of the covariates, and holds for any base classifier. Using a common benchmark data set, we demonstrate that the inflated argmax provides necessary protection against unstable classifiers, without loss of accuracy.


title: Full spectroscopic model and trihybrid experimental-perturbative-variational line list for NH

abstract: Imidogen (NH) is a reactive molecule whose presence in astrochemical environments is of interest due to its role in the formation of nitrogen-containing molecules and as a potential probe of nitrogen abundance. Spectroscopic NH monitoring is useful for Earth-based combustion and photolysis processes of ammonia and other nitrogen-containing species. NH is also relevant to ultracold molecular physics and plasma studies. To enable these diverse applications, high-quality molecular spectroscopic data is required. Here, a new line list with significant advantages over existing data is presented. Most notably, this line list models isotopologue spectroscopy and forbidden transitions (important for NH visible absorption), alongside some overall improvements to accuracy and completeness. This approach takes advantage of existing experimental data (from a previous MARVEL compilation) and perturbative line lists together with new MRCI ab initio electronic data. These are used to produce a novel variational spectroscopic model and trihybrid line list for the main 14N1H isotopologue, as well as isotopologue-extrapolated hybrid line lists for the 14N2H, 15N1H and 15N2H isotopologues. The new 14N1H ExoMol-style trihybrid line list, kNigHt, comprises 4,076 energy levels (1,078 experimental) and 327,014 transitions up to 47,500 cm-1 (211 nm) between five low-lying electronic states (X 3{\Sigma}-, a 1{\Delta}, b 1{\Sigma}+, A 3{\Pi} and c 1{\Pi}). For most anticipated applications aside from far-IR studies, this line list will be of sufficient quality; any improvements should focus on the b 1{\Sigma}+ energies, and the a 1{\Delta} - A 3{\Pi} and b 1{\Sigma}+ - A 3{\Pi} spin-orbit couplings.


title: Online Classification with Predictions

abstract: We study online classification when the learner has access to predictions about future examples. We design an online learner whose expected regret is never worse than the worst-case regret, gracefully improves with the quality of the predictions, and can be significantly better than the worst-case regret when the predictions of future examples are accurate. As a corollary, we show that if the learner is always guaranteed to observe data where future examples are easily predictable, then online learning can be as easy as transductive online learning. Our results complement recent work in online algorithms with predictions and smoothed online classification, which go beyond a worse-case analysis by using machine-learned predictions and distributional assumptions respectively.


title: ABI Approach: Automatic Bias Identification in Decision-Making Under Risk based in an Ontology of Behavioral Economics

abstract: Organizational decision-making is crucial for success, yet cognitive biases can significantly affect risk preferences, leading to suboptimal outcomes. Risk seeking preferences for losses, driven by biases such as loss aversion, pose challenges and can result in severe negative consequences, including financial losses. This research introduces the ABI approach, a novel solution designed to support organizational decision-makers by automatically identifying and explaining risk seeking preferences during decision-making. This research makes a novel contribution by automating the identification and explanation of risk seeking preferences using Cumulative Prospect theory (CPT) from Behavioral Economics. The ABI approach transforms theoretical insights into actionable, real-time guidance, making them accessible to a broader range of organizations and decision-makers without requiring specialized personnel. By contextualizing CPT concepts into business language, the approach facilitates widespread adoption and enhances decision-making processes with deep behavioral insights. Our systematic literature review identified significant gaps in existing methods, especially the lack of automated solutions with a concrete mechanism for automatically identifying risk seeking preferences, and the absence of formal knowledge representation, such as ontologies, for identifying and explaining the risk preferences. The ABI Approach addresses these gaps, offering a significant contribution to decision-making research and practice. Furthermore, it enables automatic collection of historical decision data with risk preferences, providing valuable insights for enhancing strategic management and long-term organizational performance. An experiment provided preliminary evidence on its effectiveness in helping decision-makers recognize their risk seeking preferences during decision-making in the loss domain.


title: Verifying Cake-Cutting, Faster

abstract: Envy-free cake-cutting protocols procedurally divide an infinitely divisible good among a set of agents so that no agent prefers another's allocation to their own. These protocols are highly complex and difficult to prove correct. Recently, Bertram, Levinson, and Hsu introduced a language called Slice for describing and verifying cake-cutting protocols. Slice programs can be translated to formulas encoding envy-freeness, which are solved by SMT. While Slice works well on smaller protocols, it has difficulty scaling to more complex cake-cutting protocols. We improve Slice in two ways. First, we show any protocol execution in Slice can be replicated using piecewise uniform valuations. We then reduce Slice's constraint formulas to formulas within the theory of linear real arithmetic, showing that verifying envy-freeness is efficiently decidable. Second, we design and implement a linear type system which enforces that no two agents receive the same part of the good. We implement our methods and verify a range of challenging examples, including the first nontrivial four-agent protocol.


title: Control landscapes for high-fidelity generation of C-NOT and C-PHASE gates with coherent and environmental driving

abstract: High fidelity generation of two-qubit gates is important for quantum computation, since such gates are components of popular universal sets of gates. Here we consider the problem of high fidelity generation of two-qubit C-NOT and C-PHASE (with a detailed study of C-Z) gates in presence of the environment. We consider the general situation when qubits are manipulated by coherent and incoherent controls; the latter is used to induce generally time-dependent decoherence rates. For estimating efficiency of optimization methods for high fidelity generation of these gates, we study quantum control landscapes which describe the behaviour of the fidelity as a function of the controls. For this, we generate and analyze the statistical distributions of best objective values obtained by incoherent GRadient Ascent Pulse Engineering (inGRAPE) approach. We also apply inGRAPE and stochastic zero-order method to numerically estimate minimal infidelity values. The results are different from the case of single-qubit gates and indicate a smooth trap-free behaviour of the fidelity.


title: The tilting property for $F_*^e\mathcal O_X$ on Fano surfaces and threefolds

abstract: Let $X$ be a smooth variety over a field of characteristic $p$. It is a natural question whether the Frobenius pushforwards $F_*^e\mathcal O_X$ of the structure sheaf are tilting bundles. We show if $X$ is a smooth del Pezzo surface of degree $\leq 3$ or a Fano threefold with $\mathrm{vol}(K_X)<24$ over a field of characteristic $p$, then $\mathrm{Ext}^i(F_*^e\mathcal O_X,F^e_*\mathcal O_X)\neq 0$ and thus $F_*^e\mathcal O_X$ is not tilting.


title: Revisiting linear stability of black hole odd-parity perturbations in Einstein-Aether gravity

abstract: In Einstein-Aether gravity, we revisit the issue of linear stabilities of black holes against odd-parity perturbations on a static and spherically symmetric background. In this theory, superluminal propagation is allowed and there is a preferred timelike direction along the unit Aether vector field. If we choose the usual spherically symmetric background coordinates with respect to the Killing time $t$ and the areal radius $r$, it may not be appropriate for unambiguously determining the black hole stability because the constant $t$ hypersurfaces are not necessarily always spacelike. Unlike past related works of black hole perturbations, we choose an Aether-orthogonal frame in which the timelike Aether field is orthogonal to spacelike hypersurfaces over the whole background spacetime. In the short wavelength limit, we show that no-ghost conditions as well as radial and angular propagation speeds coincide with those of vector and tensor perturbations on the Minkowski background. Thus, the odd-parity linear stability of black holes for large radial and angular momentum modes is solely determined by constant coefficients of the Aether derivative couplings.


title: Problem-informed Graphical Quantum Generative Learning

abstract: Leveraging the intrinsic probabilistic nature of quantum systems, generative quantum machine learning (QML) offers the potential to outperform classical learning models. Current generative QML algorithms mostly rely on general-purpose models that, while being very expressive, face several training challenges. A potential way to address these setbacks involves constructing problem-informed models capable of more efficient training on structured problems. In particular, probabilistic graphical models provide a flexible framework for representing structure in generative learning problems and can thus be exploited to incorporate inductive bias in QML algorithms. In this work, we propose a problem-informed quantum circuit Born machine Ansatz for learning the joint probability distribution of random variables, with independence relations efficiently represented by a Markov network (MN). We further demonstrate the applicability of the MN framework in constructing generative learning benchmarks and compare our model's performance to previous designs, showing it outperforms problem-agnostic circuits. Based on a preliminary analysis of trainability, we narrow down the class of MNs to those exhibiting favorable trainability properties. Finally, we discuss the potential of our model to offer quantum advantage in the context of generative learning.


title: PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learning

abstract: Designing generalizable agents capable of adapting to diverse embodiments has achieved significant attention in Reinforcement Learning (RL), which is critical for deploying RL agents in various real-world applications. Previous Cross-Embodiment RL approaches have focused on transferring knowledge across embodiments within specific tasks. These methods often result in knowledge tightly coupled with those tasks and fail to adequately capture the distinct characteristics of different embodiments. To address this limitation, we introduce the notion of Cross-Embodiment Unsupervised RL (CEURL), which leverages unsupervised learning to enable agents to acquire embodiment-aware and task-agnostic knowledge through online interactions within reward-free environments. We formulate CEURL as a novel Controlled Embodiment Markov Decision Process (CE-MDP) and systematically analyze CEURL's pre-training objectives under CE-MDP. Based on these analyses, we develop a novel algorithm Pre-trained Embodiment-Aware Control (PEAC) for handling CEURL, incorporating an intrinsic reward function specifically designed for cross-embodiment pre-training. PEAC not only provides an intuitive optimization strategy for cross-embodiment pre-training but also can integrate flexibly with existing unsupervised RL methods, facilitating cross-embodiment exploration and skill discovery. Extensive experiments in both simulated (e.g., DMC and Robosuite) and real-world environments (e.g., legged locomotion) demonstrate that PEAC significantly improves adaptation performance and cross-embodiment generalization, demonstrating its effectiveness in overcoming the unique challenges of CEURL.


title: Enhancing Critical Infrastructure Cybersecurity: Collaborative DNN Synthesis in the Cloud Continuum

abstract: Researchers are exploring the integration of IoT and the cloud continuum, together with AI to enhance the cost-effectiveness and efficiency of critical infrastructure (CI) systems. This integration, however, increases susceptibility of CI systems to cyberattacks, potentially leading to disruptions like power outages, oil spills, or even a nuclear mishap. CI systems are inherently complex and generate vast amounts of heterogeneous and high-dimensional data, which crosses many trust boundaries in their journey across the IoT, edge, and cloud domains over the communication network interconnecting them. As a result, they face expanded attack surfaces. To ensure the security of these dataflows, researchers have used deep neural network models with encouraging results. Nevertheless, two important challenges that remain are tackling the computational complexity of these models to reduce convergence times and preserving the accuracy of detection of integrity-violating intrusions. In this paper, we propose an innovative approach that utilizes trained edge cloud models to synthesize central cloud models, effectively overcoming these challenges. We empirically validate the effectiveness of the proposed method by comparing it with traditional centralized and distributed techniques, including a contemporary collaborative technique.


title: $T^2$ of Thoughts: Temperature Tree Elicits Reasoning in Large Language Models

abstract: Large Language Models (LLMs) have emerged as powerful tools in artificial intelligence, especially in complex decision-making scenarios, but their static problem-solving strategies often limit their adaptability to dynamic environments. We explore the enhancement of reasoning capabilities in LLMs through Temperature Tree ($T^2$) prompting via Particle Swarm Optimization, termed as $T^2$ of Thoughts ($T^2oT$). The primary focus is on enhancing decision-making processes by dynamically adjusting search parameters, especially temperature, to improve accuracy without increasing computational demands. We empirically validate that our hybrid $T^2oT$ approach yields enhancements in, single-solution accuracy, multi-solution generation and text generation quality. Our findings suggest that while dynamic search depth adjustments based on temperature can yield mixed results, a fixed search depth, when coupled with adaptive capabilities of $T^2oT$, provides a more reliable and versatile problem-solving strategy. This work highlights the potential for future explorations in optimizing algorithmic interactions with foundational language models, particularly illustrated by our development for the Game of 24 and Creative Writing tasks.


title: Seifert forms and slice Euler characteristic of links

abstract: We define the Witt coindex of a link with non-trivial Alexander polynomial, as a concordance invariant from the Seifert form. We show that it provides an upper bound for the (locally flat) slice Euler characteristic of the link, extending the work of Levine on algebraically slice knots and Taylor on the genera of knots. Then we extend the techniques by Levine on isometric structures and characterize completely the forms of coindex $1$ under the condition that the symmetrized Seifert form is non-degenerate. We illustrate our results with examples where the coindex is used to show that a two-component link does not bound a locally flat cylinder in the four-ball, whereas any other known restriction does not show it.


title: Learning to Transform Dynamically for Better Adversarial Transferability

abstract: Adversarial examples, crafted by adding perturbations imperceptible to humans, can deceive neural networks. Recent studies identify the adversarial transferability across various models, \textit{i.e.}, the cross-model attack ability of adversarial samples. To enhance such adversarial transferability, existing input transformation-based methods diversify input data with transformation augmentation. However, their effectiveness is limited by the finite number of available transformations. In our study, we introduce a novel approach named Learning to Transform (L2T). L2T increases the diversity of transformed images by selecting the optimal combination of operations from a pool of candidates, consequently improving adversarial transferability. We conceptualize the selection of optimal transformation combinations as a trajectory optimization problem and employ a reinforcement learning strategy to effectively solve the problem. Comprehensive experiments on the ImageNet dataset, as well as practical tests with Google Vision and GPT-4V, reveal that L2T surpasses current methodologies in enhancing adversarial transferability, thereby confirming its effectiveness and practical significance. The code is available at https://github.com/RongyiZhu/L2T.


title: A finite time analysis of distributed Q-learning

abstract: Multi-agent reinforcement learning (MARL) has witnessed a remarkable surge in interest, fueled by the empirical success achieved in applications of single-agent reinforcement learning (RL). In this study, we consider a distributed Q-learning scenario, wherein a number of agents cooperatively solve a sequential decision making problem without access to the central reward function which is an average of the local rewards. In particular, we study finite-time analysis of a distributed Q-learning algorithm, and provide a new sample complexity result of $\tilde{\mathcal{O}}\left( \min\left\{\frac{1}{\epsilon^2}\frac{t_{\text{mix}}}{(1-\gamma)^6 d_{\min}^4 } ,\frac{1}{\epsilon}\frac{\sqrt{|\gS||\gA|}}{(1-\sigma_2(\boldsymbol{W}))(1-\gamma)^4 d_{\min}^3} \right\}\right)$ under tabular lookup


title: Advancing Transportation Mode Share Analysis with Built Environment: Deep Hybrid Models with Urban Road Network

abstract: Transportation mode share analysis is important to various real-world transportation tasks as it helps researchers understand the travel behaviors and choices of passengers. A typical example is the prediction of communities' travel mode share by accounting for their sociodemographics like age, income, etc., and travel modes' attributes (e.g. travel cost and time). However, there exist only limited efforts in integrating the structure of the urban built environment, e.g., road networks, into the mode share models to capture the impacts of the built environment. This task usually requires manual feature engineering or prior knowledge of the urban design features. In this study, we propose deep hybrid models (DHM), which directly combine road networks and sociodemographic features as inputs for travel mode share analysis. Using graph embedding (GE) techniques, we enhance travel demand models with a more powerful representation of urban structures. In experiments of mode share prediction in Chicago, results demonstrate that DHM can provide valuable spatial insights into the sociodemographic structure, improving the performance of travel demand models in estimating different mode shares at the city level. Specifically, DHM improves the results by more than 20\% while retaining the interpretation power of the choice models, demonstrating its superiority in interpretability, prediction accuracy, and geographical insights.


title: Fano line shape metamorphosis in resonant two-photon ionization

abstract: Two-photon atomic ionization driven by time-locked XUV and IR pulses allows to study dynamics of Fano resonances in time and energy domains. Different time evolution of the two interfering pathways leading to a Fano resonance can be exploited to turn the Fano profile of the two-photon XUV/IR ionization into a symmetric Gaussian once the directly ejected photoelectron leaves the parent ion and cannot any longer absorb an IR photon. This line shape transformation allows for the direct determination of the resonant lifetime from the spectroscopic measurements without need for an extremely fine energy resolution. Ubiquitous nature of Fano resonances makes this determination a universal tool in diverse quantum systems ranging from nuclei to nano-fabricated solids.


title: Laboratory-scale Perpendicular Collisionless Shock Generation and Ion Acceleration in Magnetized Head-on Colliding Plasmas

abstract: Magnetized collisionless shocks drive particle acceleration broadly in space and astrophysics. We perform the first large-scale particle-in-cell simulations with realistic laboratory parameters (density, temperature, and velocity) to investigate the magnetized shock in head-on colliding plasmas with an applied magnetic field of tens of Tesla. It is shown that a perpendicular collisionless shock is formed with about fourfold density jump when two pre-magnetized flows collide. This shock is also characterized by rapid increase of neutron yield, triggered by the beam-beam nuclear reactions between injected deuterons and ones reflected by the shock. Distinct from the shocks arising from the interaction of injected flows with a magnetized background, the self-generated magnetic field in this colliding plasmas experiences a significant amplification due to the increasing diamagnetic current, approximately 30 times of upstream magnetic field. Moreover, we find that ions, regardless of whether they pass through or are reflected by the shock, can gain energy by the shock surfing acceleration, generating a power-law energy spectrum. In addition, we also demonstrate that the shock mediated only by filamentation instability cannot be generated under the prevailing unmagnetized experimental parameters. These results provide a direct connection of astrophysical field amplification to the magnetized shock formation and nonthermal ion generation.


title: Exclusively Penalized Q-learning for Offline Reinforcement Learning

abstract: Constraint-based offline reinforcement learning (RL) involves policy constraints or imposing penalties on the value function to mitigate overestimation errors caused by distributional shift. This paper focuses on a limitation in existing offline RL methods with penalized value function, indicating the potential for underestimation bias due to unnecessary bias introduced in the value function. To address this concern, we propose Exclusively Penalized Q-learning (EPQ), which reduces estimation bias in the value function by selectively penalizing states that are prone to inducing estimation errors. Numerical results show that our method significantly reduces underestimation bias and improves performance in various offline control tasks compared to other offline RL methods


title: Log motivic nearby cycles

abstract: We define the log motivic nearby cycles functor. We show that this sends the motive of a proper smooth scheme over the fraction field of a DVR to the motive of the boundary of a log smooth model assuming absolute purity, which is unconditional in the equal characteristic case. In characteristic $0$, we show that the $\infty$-categories of motives over the standard log point and rigid analytic motives are equivalent, and we relate log motivic nearby cycles functor with Ayoub's motivic nearby cycles functor.


title: Algebraic Independence of Special Points on Shimura Varieties

abstract: Given a correspondence $V$ between a connected Shimura variety $S$, a commutative connected algebraic group $G$, and $n \in \mathbb{N}$, we prove that the $V$-images of any $n$ special points on $S$ outside a proper Zariski closed subset are algebraically independent. Our result unifies previous unlikely intersection results on multiplicative independence and linear independence. We prove multiplicative independence of differences of singular moduli, generalizing previous results by Pila-Tsimerman, and Aslanlyan-Eterovi\'c-Fowler. We also give an application to abelian varieties by proving that the special points of $S$ whose $V$-images lie in a finite-rank subgroup of $T$ are contained in a finite union of proper special subvarieties of $S$, only dependent on the rank of the subgroup. In this way, our result is a generalization of the works of Pila-Tsimerman and Buium-Poonen.


title: Quantumness and Randomness of Quantum Random Number Generators via Photon Statistics

abstract: Several quantum random number generator (QRNG) models have been proposed to produce random numbers, which, due to the quantum theory, are more secure than their classical counterparts. Many QRNG devices are commercially available as off-the-shelf black-box devices. Computationally, it is not possible to distinguish between a PRNG and a QRNG just by observing their outputs. The common practice for testing quantumness is a direct comparison between the mean and the variance of the experimental photon count. However, statistically, this is not a feasible solution either, because of finite sample size. A QRNG that uses single photons as the quantum source produces true quantum random numbers. Since single photons follow sub-Poissonian statistics, by determining the underlying distribution one can conclude whether a QRNG is truly quantum or not. In this work, we point out the limitations of existing methods of such a decision-making processes, and propose a more efficient two-fold statistical method, which can ensure whether an optical source is quantum or not up to a desired confidence level. Also, QRNGs can not produce true random numbers without deterministic classical post-processing. In this work, we also show that the two models of QRNGs, one producing random numbers from exponential distribution and the other from uniform distribution, become essentially similar under device noise. Detector outputs of both the above models can be tested to quantify the randomness coming from the quantum source of a QRNG, which in turn, dictates how much post-processing is required to produce good random numbers retaining quantumness. In this context, we also derive a relation when the underlying sampling distributions of the QRNGs will be $\epsilon$-random. Depending on this relation, a suitable post-processing algorithm can be chosen.


title: Classification of Lagrangian translators and Lagrangian self-expanders in $\mathbb{C}^{2}$

abstract: In this paper, we obtain several classification results of $2$-dimensional complete Lagrangian translators and lagrangian self-expanders with constant squared norm $|\vec{H}|^{2}$ of the mean curvature vector in $\mathbb{C}^{2}$ by using a new Omori-Yau type maximum principle which was proved by Chen and Qiu \cite{CQ}. The same idea is also used to give a similar result of Lagrangian $\xi$-translators in $\mathbb{C}^{2}$.


title: Finitely generated congruences on tropical rational function semifields

abstract: We prove that the congruence on the tropical rational function semifield in $n$-variables associated with a subset $V$ of $\boldsymbol{R}^n$ is finitely generated if and only if the closure of $V$ is a finite union of $\boldsymbol{R}$-rational polyhedral sets. With this fact, we characterize rational function semifields of tropical curves.


title: High-dimensional Learning with Noisy Labels

abstract: This paper provides theoretical insights into high-dimensional binary classification with class-conditional noisy labels. Specifically, we study the behavior of a linear classifier with a label noisiness aware loss function, when both the dimension of data $p$ and the sample size $n$ are large and comparable. Relying on random matrix theory by supposing a Gaussian mixture data model, the performance of the linear classifier when $p,n\to \infty$ is shown to converge towards a limit, involving scalar statistics of the data. Importantly, our findings show that the low-dimensional intuitions to handle label noise do not hold in high-dimension, in the sense that the optimal classifier in low-dimension dramatically fails in high-dimension. Based on our derivations, we design an optimized method that is shown to be provably more efficient in handling noisy labels in high dimensions. Our theoretical conclusions are further confirmed by experiments on real datasets, where we show that our optimized approach outperforms the considered baselines.


title: Improved Canonicalization for Model Agnostic Equivariance

abstract: This work introduces a novel approach to achieving architecture-agnostic equivariance in deep learning, particularly addressing the limitations of traditional equivariant architectures and the inefficiencies of the existing architecture-agnostic methods. Building equivariant models using traditional methods requires designing equivariant versions of existing models and training them from scratch, a process that is both impractical and resource-intensive. Canonicalization has emerged as a promising alternative for inducing equivariance without altering model architecture, but it suffers from the need for highly expressive and expensive equivariant networks to learn canonical orientations accurately. We propose a new method that employs any non-equivariant network for canonicalization. Our method uses contrastive learning to efficiently learn a unique canonical orientation and offers more flexibility for the choice of canonicalization network. We empirically demonstrate that this approach outperforms existing methods in achieving equivariance for large pretrained models and significantly speeds up the canonicalization process, making it up to 2 times faster.


title: Actively Learning Combinatorial Optimization Using a Membership Oracle

abstract: We consider solving a combinatorial optimization problem with an unknown linear constraint using a membership oracle that, given a solution, determines whether it is feasible or infeasible with absolute certainty. The goal of the decision maker is to find the best possible solution subject to a budget on the number of oracle calls. Inspired by active learning based on Support Vector Machines (SVMs), we adapt a classical framework in order to solve the problem by learning and exploiting a surrogate linear constraint. The resulting new framework includes training a linear separator on the labeled points and selecting new points to be labeled, which is achieved by applying a sampling strategy and solving a 0-1 integer linear program. Following the active learning literature, one can consider using SVM as a linear classifier and the information-based sampling strategy known as Simple margin. We improve on both sides: we propose an alternative sampling strategy based on mixed-integer quadratic programming and a linear separation method inspired by an algorithm for convex optimization in the oracle model. We conduct experiments on the pure knapsack problem and on a college study plan problem from the literature to show how different linear separation methods and sampling strategies influence the quality of the results in terms of objective value.


title: Peas-in-a-Pod Across the Radius Valley: Rocky Systems are Less Uniform in Mass but More Uniform in Size and Spacing

abstract: The ubiquity of "peas-in-a-pod" architectural patterns and the existence of the radius valley each present a striking population-level trend for planets with $R_{p} \leq 4 R_{\oplus}$ that serves to place powerful constraints on the formation and evolution of these subgiant worlds. As it has yet to be determined whether the strength of this peas-in-a-pod uniformity differs on either side of the radius valley, we separately assess the architectures of systems containing only small ($R_{p} \leq 1.6 R_{\oplus}$), rocky planets from those harboring only intermediate-size ($1.6 R_{\oplus} < R_{p} \leq 4 R_{\oplus}$), volatile-rich worlds to perform a novel statistical comparison of intra-system planetary uniformity across compositionally distinct regimes. We find that, compared to their volatile-rich counterparts, rocky systems are less uniform in mass ($2.6\sigma$), but more uniform in size ($4.0\sigma$) and spacing ($3.0\sigma$). We provide further statistical validation for these results, demonstrating that they are not substantially influenced by the presence of mean motion resonances, low-mass host stars, alternative bulk compositional assumptions, sample size effects, or detection biases. We also obtain tentative evidence ($>2 \sigma$ significance) that the enhanced size uniformity of rocky systems is dominated by the presence of super-Earths ($1 R_{\oplus} \leq R_{p} \leq 1.6 R_{\oplus}$), while their enhanced mass diversity is driven by the presence of sub-Earth ($R_{p} < 1 R_{\oplus}$) worlds.


title: Large Language Models Can Self-Correct with Minimal Effort

abstract: Intrinsic self-correct was a method that instructed large language models (LLMs) to verify and correct their responses without external feedback. Unfortunately, the study concluded that the LLMs could not self-correct reasoning yet. We find that a simple yet effective verification method can unleash inherent capabilities of the LLMs. That is to mask a key condition in the question, add the current response to construct a verification question, and predict the condition to verify the response. The condition can be an entity in an open-domain question or a numeric value in a math question, which requires minimal effort (via prompting) to identify. We propose an iterative verify-then-correct framework to progressively identify and correct (probably) false responses, named ProCo. We conduct experiments on three reasoning tasks. On average, ProCo, with GPT-3.5-Turbo as the backend LLM, yields $+6.8$ exact match on four open-domain question answering datasets, $+14.1$ accuracy on three arithmetic reasoning datasets, and $+9.6$ accuracy on a commonsense reasoning dataset, compared to Self-Correct.


title: A Survey on Vision-Language-Action Models for Embodied AI

abstract: Deep learning has demonstrated remarkable success across many domains, including computer vision, natural language processing, and reinforcement learning. Representative artificial neural networks in these fields span convolutional neural networks, Transformers, and deep Q-networks. Built upon unimodal neural networks, numerous multi-modal models have been introduced to address a range of tasks such as visual question answering, image captioning, and speech recognition. The rise of instruction-following robotic policies in embodied AI has spurred the development of a novel category of multi-modal models known as vision-language-action models (VLAs). Their multi-modality capability has become a foundational element in robot learning. Various methods have been proposed to enhance traits such as versatility, dexterity, and generalizability. Some models focus on refining specific components through pretraining. Others aim to develop control policies adept at predicting low-level actions. Certain VLAs serve as high-level task planners capable of decomposing long-horizon tasks into executable subtasks. Over the past few years, a myriad of VLAs have emerged, reflecting the rapid advancement of embodied AI. Therefore, it is imperative to capture the evolving landscape through a comprehensive survey.


title: Attending to Topological Spaces: The Cellular Transformer

abstract: Topological Deep Learning seeks to enhance the predictive performance of neural network models by harnessing topological structures in input data. Topological neural networks operate on spaces such as cell complexes and hypergraphs, that can be seen as generalizations of graphs. In this work, we introduce the Cellular Transformer (CT), a novel architecture that generalizes graph-based transformers to cell complexes. First, we propose a new formulation of the usual self- and cross-attention mechanisms, tailored to leverage incidence relations in cell complexes, e.g., edge-face and node-edge relations. Additionally, we propose a set of topological positional encodings specifically designed for cell complexes. By transforming three graph datasets into cell complex datasets, our experiments reveal that CT not only achieves state-of-the-art performance, but it does so without the need for more complex enhancements such as virtual nodes, in-domain structural encodings, or graph rewiring.


title: Abrupt X-to-O-wave structural field transition in presence of anomalous dispersion

abstract: All linear, propagation-invariant, paraxial pulsed beams are spatiotemporally X-shaped (conical waves) in absence of group-velocity dispersion (GVD), or in presence of normal GVD. It is known, however, that such conical waves become O-shaped in presence of anomalous GVD, resulting in a field profile that is circularly symmetric in space and time. To date, experiments generating conical waves in which the wavelength of a high-energy pump laser is tuned across the zero-dispersion wavelength of a nonlinear medium have not revealed the expected X-to-O-wave structural field transition. We report here unambiguous observation of a fixed-wavelength X-to-O-wave structural field transition occurring in linear dispersion-free wave packets in the anomalous GVD regime -- without needing to change the sign or magnitude of the GVD. Instead, by tuning the group velocity of a space-time wave packet (STWP) across a threshold value that we call the `escape velocity', we observe an abrupt transition in the STWP from an O-shaped to an X-shaped spatiotemporal profile. This transition is associated with an abrupt change in the associated spatiotemporal spectrum of the STWP: from closed elliptical spatiotemporal spectra below the escape velocity to open hyperbolic spectra above it. These results may furnish new opportunities for engineering the phase-matching conditions in nonlinear and quantum optics.


title: Newton Informed Neural Operator for Computing Multiple Solutions of Nonlinear Partials Differential Equations

abstract: Solving nonlinear partial differential equations (PDEs) with multiple solutions using neural networks has found widespread applications in various fields such as physics, biology, and engineering. However, classical neural network methods for solving nonlinear PDEs, such as Physics-Informed Neural Networks (PINN), Deep Ritz methods, and DeepONet, often encounter challenges when confronted with the presence of multiple solutions inherent in the nonlinear problem. These methods may encounter ill-posedness issues. In this paper, we propose a novel approach called the Newton Informed Neural Operator, which builds upon existing neural network techniques to tackle nonlinearities. Our method combines classical Newton methods, addressing well-posed problems, and efficiently learns multiple solutions in a single learning process while requiring fewer supervised data points compared to existing neural network methods.


title: Impact of gauge fixing precision on the continuum limit of non-local quark-bilinear lattice operators

abstract: We analyze the gauge fixing precision dependence of some non-local quark-blinear lattice operators interesting in computing parton physics for several measurements, using 5 lattice spacings ranging from 0.032 fm to 0.121 fm. Our results show that gauge dependent non-local measurements are significantly more sensitive to the precision of gauge fixing than anticipated. The impact of imprecise gauge fixing is significant for fine lattices and long distances. For instance, even with the typically defined precision of Landau gauge fixing of $10^{-8}$, the deviation caused by imprecise gauge fixing can reach 12 percent, when calculating the trace of Wilson lines at 1.2 fm with a lattice spacing of approximately 0.03 fm. Similar behavior has been observed in $\xi$ gauge and Coulomb gauge as well. For both quasi PDFs and quasi TMD-PDFs operators renormalized using the RI/MOM scheme, convergence for different lattice spacings at long distance is only observed when the precision of Landau gauge fixing is sufficiently high. To describe these findings quantitatively, we propose an empirical formula to estimate the required precision.


title: A continuous perspective on the inertial corrected primal-dual proximal splitting

abstract: We give a continuous perspective on the Inertial Corrected Primal-Dual Proximal Splitting (IC-PDPS) proposed by Valkonen ({\it SIAM J. Optim.}, 30(2): 1391--1420, 2020) for solving saddle-point problems. The algorithm possesses nonergodic convergence rate and admits a tight preconditioned proximal point formulation which involves both inertia and additional correction. Based on new understandings on the relation between the discrete step size and rescaling effect, we rebuild IC-PDPS as a semi-implicit Euler scheme with respect to its iterative sequences and integrated parameters. This leads to two novel second-order ordinary differential equation (ODE) models that are equivalent under proper time transformation, and also provides an alternative interpretation from the continuous point of view. Besides, we present the convergence analysis of the Lagrangian gap along the continuous trajectory by using proper Lyapunov functions.


title: Automatic Differentiation is Essential in Training Neural Networks for Solving Differential Equations

abstract: Neural network-based approaches have recently shown significant promise in solving partial differential equations (PDEs) in science and engineering, especially in scenarios featuring complex domains or the incorporation of empirical data. One advantage of the neural network method for PDEs lies in its automatic differentiation (AD), which necessitates only the sample points themselves, unlike traditional finite difference (FD) approximations that require nearby local points to compute derivatives. In this paper, we quantitatively demonstrate the advantage of AD in training neural networks. The concept of truncated entropy is introduced to characterize the training property. Specifically, through comprehensive experimental and theoretical analyses conducted on random feature models and two-layer neural networks, we discover that the defined truncated entropy serves as a reliable metric for quantifying the residual loss of random feature models and the training speed of neural networks for both AD and FD methods. Our experimental and theoretical analyses demonstrate that, from a training perspective, AD outperforms FD in solving partial differential equations.


title: Water Management Considerations for a Self-Sustaining Moonbase

abstract: The most pragmatic first step in the all-but-inevitable 3rd-millennium V\"olkerwanderung of humanity throughout the Solar System is the establishment of a permanent human presence on the Moon. This research examines: 1. the human, agricultural, and technical water needs of a 100-person, 500 m x 100 m x 6 m self-sustaining lunar colony; 2. choosing a strategic location for the moonbase; 3. a heat drill model by which the needed lunar water ice could be sublimated; and 4. the robust water treatment and recovery infrastructure and water management personnel that would be needed for a self-sustaining moonbase.

categories: physics.soc-ph astro-ph.EP astro-ph.IM physics.pop-ph


title: Enhancing Image Layout Control with Loss-Guided Diffusion Models

abstract: Diffusion models are a powerful class of generative models capable of producing high-quality images from pure noise. In particular, conditional diffusion models allow one to specify the contents of the desired image using a simple text prompt. Conditioning on a text prompt alone, however, does not allow for fine-grained control over the composition and layout of the final image, which instead depends closely on the initial noise distribution. While most methods which introduce spatial constraints (e.g., bounding boxes) require fine-tuning, a smaller and more recent subset of these methods are training-free. They are applicable whenever the prompt influences the model through an attention mechanism, and generally fall into one of two categories. The first entails modifying the cross-attention maps of specific tokens directly to enhance the signal in certain regions of the image. The second works by defining a loss function over the cross-attention maps, and using the gradient of this loss to guide the latent. While previous work explores these as alternative strategies, we provide an interpretation for these methods which highlights their complimentary features, and demonstrate that it is possible to obtain superior performance when both methods are used in concert.


title: Synthesis and characterization of space-time light sheets: A tutorial

abstract: Space-time wave packets (STWPs) are a new class of pulsed optical beams with many unique and intriguing attributes, including propagation invariance and tunable group velocity in linear optical media. STWPs are a form of spatiotemporally structured light, so their synthesis poses challenges that are not shared by conventional monochromatic structured light fields. We present here a detailed description of the synthesis of STWPs that are localized along one transverse dimension and uniform along the other; i.e., space-time light sheets. We also describe the main characterization schemes needed for benchmarking the unique properties of space-time light sheets.


title: Online Self-Preferring Language Models

abstract: Aligning with human preference datasets has been critical to the success of large language models (LLMs). Reinforcement learning from human feedback (RLHF) employs a costly reward model to provide feedback for on-policy sampling responses. Recently, offline methods that directly fit responses with binary preferences in the dataset have emerged as alternatives. However, existing methods do not explicitly model preference strength information, which is crucial for distinguishing different response pairs. To overcome this limitation, we propose Online Self-Preferring (OSP) language models to learn from self-generated response pairs and self-judged preference strengths. For each prompt and corresponding self-generated responses, we introduce a ranked pairing method to construct multiple response pairs with preference strength information. We then propose the soft-preference cross-entropy loss to leverage such information. Empirically, we demonstrate that leveraging preference strength is crucial for avoiding overfitting and enhancing alignment performance. OSP achieves state-of-the-art alignment performance across various metrics in two widely used human preference datasets. OSP is parameter-efficient and more robust than the dominant online method, RLHF when limited offline data are available and generalizing to out-of-domain tasks. Moreover, OSP language models established by LLMs with proficiency in self-preferring can efficiently self-improve without external supervision.


title: On the Identifying Power of Monotonicity for Average Treatment Effects

abstract: In the context of a binary outcome, treatment, and instrument, Balke and Pearl (1993, 1997) establish that adding monotonicity to the instrument exogeneity assumption does not decrease the identified sets for average potential outcomes and average treatment effect parameters when those assumptions are consistent with the distribution of the observable data. We show that the same results hold in the broader context of multi-valued outcome, treatment, and instrument. An important example of such a setting is a multi-arm randomized controlled trial with noncompliance.


title: Distributed Speculative Inference of Large Language Models

abstract: Accelerating the inference of large language models (LLMs) is an important challenge in artificial intelligence. This paper introduces distributed speculative inference (DSI), a novel distributed inference algorithm that is provably faster than speculative inference (SI) [leviathan2023fast, chen2023accelerating, miao2023specinfer] and traditional autoregressive inference (non-SI). Like other SI algorithms, DSI works on frozen LLMs, requiring no training or architectural modifications, and it preserves the target distribution. Prior studies on SI have demonstrated empirical speedups (compared to non-SI) but require a fast and accurate drafter LLM. In practice, off-the-shelf LLMs often do not have matching drafters that are sufficiently fast and accurate. We show a gap: SI gets slower than non-SI when using slower or less accurate drafters. We close this gap by proving that DSI is faster than both SI and non-SI given any drafters. By orchestrating multiple instances of the target and drafters, DSI is not only faster than SI but also supports LLMs that cannot be accelerated with SI. Our simulations show speedups of off-the-shelf LLMs in realistic settings: DSI is 1.29-1.92x faster than SI.


title: Nearly Tight Black-Box Auditing of Differentially Private Machine Learning

abstract: This paper presents a nearly tight audit of the Differentially Private Stochastic Gradient Descent (DP-SGD) algorithm in the black-box model. Our auditing procedure empirically estimates the privacy leakage from DP-SGD using membership inference attacks; unlike prior work, the estimates are appreciably close to the theoretical DP bounds. The main intuition is to craft worst-case initial model parameters, as DP-SGD's privacy analysis is agnostic to the choice of the initial model parameters. For models trained with theoretical $\varepsilon=10.0$ on MNIST and CIFAR-10, our auditing procedure yields empirical estimates of $7.21$ and $6.95$, respectively, on 1,000-record samples and $6.48$ and $4.96$ on the full datasets. By contrast, previous work achieved tight audits only in stronger (i.e., less realistic) white-box models that allow the adversary to access the model's inner parameters and insert arbitrary gradients. Our auditing procedure can be used to detect bugs and DP violations more easily and offers valuable insight into how the privacy analysis of DP-SGD can be further improved.


title: Towards Feature Engineering with Human and AI's Knowledge: Understanding Data Science Practitioners' Perceptions in Human&AI-Assisted Feature Engineering Design

abstract: As AI technology continues to advance, the importance of human-AI collaboration becomes increasingly evident, with numerous studies exploring its potential in various fields. One vital field is data science, including feature engineering (FE), where both human ingenuity and AI capabilities play pivotal roles. Despite the existence of AI-generated recommendations for FE, there remains a limited understanding of how to effectively integrate and utilize humans' and AI's knowledge. To address this gap, we design a readily-usable prototype, human\&AI-assisted FE in Jupyter notebooks. It harnesses the strengths of humans and AI to provide feature suggestions to users, seamlessly integrating these recommendations into practical workflows. Using the prototype as a research probe, we conducted an exploratory study to gain valuable insights into data science practitioners' perceptions, usage patterns, and their potential needs when presented with feature suggestions from both humans and AI. Through qualitative analysis, we discovered that the Creator of the feature (i.e., AI or human) significantly influences users' feature selection, and the semantic clarity of the suggested feature greatly impacts its adoption rate. Furthermore, our findings indicate that users perceive both differences and complementarity between features generated by humans and those generated by AI. Lastly, based on our study results, we derived a set of design recommendations for future human&AI FE design. Our findings show the collaborative potential between humans and AI in the field of FE.


title: Deep Learning for Protein-Ligand Docking: Are We There Yet?

abstract: The effects of ligand binding on protein structures and their in vivo functions carry numerous implications for modern biomedical research and biotechnology development efforts such as drug discovery. Although several deep learning (DL) methods and benchmarks designed for protein-ligand docking have recently been introduced, to date no prior works have systematically studied the behavior of docking methods within the practical context of (1) using predicted (apo) protein structures for docking (e.g., for broad applicability); (2) docking multiple ligands concurrently to a given target protein (e.g., for enzyme design); and (3) having no prior knowledge of binding pockets (e.g., for pocket generalization). To enable a deeper understanding of docking methods' real-world utility, we introduce PoseBench, the first comprehensive benchmark for practical protein-ligand docking. PoseBench enables researchers to rigorously and systematically evaluate DL docking methods for apo-to-holo protein-ligand docking and protein-ligand structure generation using both single and multi-ligand benchmark datasets, the latter of which we introduce for the first time to the DL community. Empirically, using PoseBench, we find that all recent DL docking methods but one fail to generalize to multi-ligand protein targets and also that template-based docking algorithms perform equally well or better for multi-ligand docking as recent single-ligand DL docking methods, suggesting areas of improvement for future work. Code, data, tutorials, and benchmark results are available at https://github.com/BioinfoMachineLearning/PoseBench.


title: Effect of boundary roughness on the attenuation of specular phonon reflection in graphene

abstract: The reduced phonon specularity $p$ from boundary roughness scattering plays a major role in the lower thermal conductivity in semiconducting and insulating nanowires and films. Although the well-known Ziman formula $p=\exp(-4\sigma^{2}q_{x}^{2})$, where $\sigma$ and $q_{x}$ denote the root-mean-square boundary roughness and the normal component of the incident phonon wave vector, respectively, and its variants are commonly used in the literature to estimate how roughness attenuates $p$, their validity and accuracy remain poorly understood, especially when the effects of mode conversion cannot be ignored. In this paper, we investigate the accuracy and validity of the more general Ogilvy formula, from which the Ziman formula is derived, by comparing its predictions to the $p$ values computed from Atomistic Green's Function (AGF) simulations for an ensemble of rough boundaries in single-layer graphene. The effects of phonon dispersion, incident angle, polarization, mode conversion, and correlation length are analyzed. Our results suggest that the Ogilvy formula is remarkably accurate for $0<q_{x}<\frac{\pi}{4\sigma}$ when the lateral correlation length $L$ is large or the phonon is at normal incidence. At large $q_{x}$ in the short-wavelength limit, the $q_{x}$-dependence of $p$ becomes significantly weaker. In the large-$L$ limit, the numerical results suggest the existence of a minimum $p$ for short-wavelength phonons, given by $p\sim p_{0}\exp(-\pi^{2}/4)$, where $p_{0}$ is the baseline specularity for the ideal boundary.


title: Regularity-Conforming Neural Networks (ReCoNNs) for solving Partial Differential Equations

abstract: Whilst the Universal Approximation Theorem guarantees the existence of approximations to Sobolev functions -- the natural function spaces for PDEs -- by Neural Networks (NNs) of sufficient size, low-regularity solutions may lead to poor approximations in practice. For example, classical fully-connected feed-forward NNs fail to approximate continuous functions whose gradient is discontinuous when employing strong formulations like in Physics Informed Neural Networks (PINNs). In this article, we propose the use of regularity-conforming neural networks, where a priori information on the regularity of solutions to PDEs can be employed to construct proper architectures. We illustrate the potential of such architectures via a two-dimensional (2D) transmission problem, where the solution may admit discontinuities in the gradient across interfaces, as well as power-like singularities at certain points. In particular, we formulate the weak transmission problem in a PINNs-like strong formulation with interface and continuity conditions. Such architectures are partially explainable; discontinuities are explicitly described, allowing the introduction of novel terms into the loss function. We demonstrate via several model problems in one and two dimensions the advantages of using regularity-conforming architectures in contrast to classical architectures. The ideas presented in this article easily extend to problems in higher dimensions.


title: Improving Generalization of Deep Neural Networks by Optimum Shifting

abstract: Recent studies showed that the generalization of neural networks is correlated with the sharpness of the loss landscape, and flat minima suggests a better generalization ability than sharp minima. In this paper, we propose a novel method called \emph{optimum shifting}, which changes the parameters of a neural network from a sharp minimum to a flatter one while maintaining the same training loss value. Our method is based on the observation that when the input and output of a neural network are fixed, the matrix multiplications within the network can be treated as systems of under-determined linear equations, enabling adjustment of parameters in the solution space, which can be simply accomplished by solving a constrained optimization problem. Furthermore, we introduce a practical stochastic optimum shifting technique utilizing the Neural Collapse theory to reduce computational costs and provide more degrees of freedom for optimum shifting. Extensive experiments (including classification and detection) with various deep neural network architectures on benchmark datasets demonstrate the effectiveness of our method.


title: The stochastic Landau--Lifshitz--Baryakhtar equation: Global solution and invariant measure

abstract: The Landau--Lifshitz--Baryakhtar (LLBar) equation perturbed by a space-dependent noise is a system of fourth order stochastic PDEs which models the evolution of magnetic spin fields in ferromagnetic materials at elevated temperatures, taking into account longitudinal damping, long-range interactions, and noise-induced phenomena at high temperatures. In this paper, we show the existence of a martingale solution (which is analytically strong) to the stochastic LLBar equation posed in a bounded domain $\mathscr{D}\subset \mathbb{R}^d$, where $d=1,2,3$. We also prove pathwise uniqueness of the solution, which implies the existence of a unique probabilistically strong solution. Finally, we show the Feller property of the Markov semigroup associated with the strong solution, which implies the existence of invariant measures.


title: Multi-modality Regional Alignment Network for Covid X-Ray Survival Prediction and Report Generation

abstract: In response to the worldwide COVID-19 pandemic, advanced automated technologies have emerged as valuable tools to aid healthcare professionals in managing an increased workload by improving radiology report generation and prognostic analysis. This study proposes Multi-modality Regional Alignment Network (MRANet), an explainable model for radiology report generation and survival prediction that focuses on high-risk regions. By learning spatial correlation in the detector, MRANet visually grounds region-specific descriptions, providing robust anatomical regions with a completion strategy. The visual features of each region are embedded using a novel survival attention mechanism, offering spatially and risk-aware features for sentence encoding while maintaining global coherence across tasks. A cross LLMs alignment is employed to enhance the image-to-text transfer process, resulting in sentences rich with clinical detail and improved explainability for radiologist. Multi-center experiments validate both MRANet's overall performance and each module's composition within the model, encouraging further advancements in radiology report generation research emphasizing clinical interpretation and trustworthiness in AI models applied to medical studies. The code is available at https://github.com/zzs95/MRANet.


title: Offline Reinforcement Learning from Datasets with Structured Non-Stationarity

abstract: Current Reinforcement Learning (RL) is often limited by the large amount of data needed to learn a successful policy. Offline RL aims to solve this issue by using transitions collected by a different behavior policy. We address a novel Offline RL problem setting in which, while collecting the dataset, the transition and reward functions gradually change between episodes but stay constant within each episode. We propose a method based on Contrastive Predictive Coding that identifies this non-stationarity in the offline dataset, accounts for it when training a policy, and predicts it during evaluation. We analyze our proposed method and show that it performs well in simple continuous control tasks and challenging, high-dimensional locomotion tasks. We show that our method often achieves the oracle performance and performs better than baselines.


title: Configuring Data Augmentations to Reduce Variance Shift in Positional Embedding of Vision Transformers

abstract: Vision transformers (ViTs) have demonstrated remarkable performance in a variety of vision tasks. Despite their promising capabilities, training a ViT requires a large amount of diverse data. Several studies empirically found that using rich data augmentations, such as Mixup, Cutmix, and random erasing, is critical to the successful training of ViTs. Now, the use of rich data augmentations has become a standard practice in the current state. However, we report a vulnerability to this practice: Certain data augmentations such as Mixup cause a variance shift in the positional embedding of ViT, which has been a hidden factor that degrades the performance of ViT during the test phase. We claim that achieving a stable effect from positional embedding requires a specific condition on the image, which is often broken for the current data augmentation methods. We provide a detailed analysis of this problem as well as the correct configuration for these data augmentations to remove the side effects of variance shift. Experiments showed that adopting our guidelines improves the performance of ViTs compared with the current configuration of data augmentations.


title: Learning Multimodal Confidence for Intention Recognition in Human-Robot Interaction

abstract: The rapid development of collaborative robotics has provided a new possibility of helping the elderly who has difficulties in daily life, allowing robots to operate according to specific intentions. However, efficient human-robot cooperation requires natural, accurate and reliable intention recognition in shared environments. The current paramount challenge for this is reducing the uncertainty of multimodal fused intention to be recognized and reasoning adaptively a more reliable result despite current interactive condition. In this work we propose a novel learning-based multimodal fusion framework Batch Multimodal Confidence Learning for Opinion Pool (BMCLOP). Our approach combines Bayesian multimodal fusion method and batch confidence learning algorithm to improve accuracy, uncertainty reduction and success rate given the interactive condition. In particular, the generic and practical multimodal intention recognition framework can be easily extended further. Our desired assistive scenarios consider three modalities gestures, speech and gaze, all of which produce categorical distributions over all the finite intentions. The proposed method is validated with a six-DoF robot through extensive experiments and exhibits high performance compared to baselines.


title: Knowledge Localization: Mission Not Accomplished? Enter Query Localization!

abstract: Large language models (LLMs) store extensive factual knowledge, but the mechanisms behind how they store and express this knowledge remain unclear. The Knowledge Neuron (KN) thesis is a prominent theory for explaining these mechanisms. This theory is based on the knowledge localization (KL) assumption, which suggests that a fact can be localized to a few knowledge storage units, namely knowledge neurons. However, this assumption may be overly strong regarding knowledge storage and neglects knowledge expression mechanisms. Thus, we re-examine the KL assumption and confirm the existence of facts that do not adhere to it from both statistical and knowledge modification perspectives. Furthermore, we propose the Query Localization (QL) assumption. (1) Query-KN Mapping: The localization results are associated with the query rather than the fact. (2) Dynamic KN Selection: The attention module contributes to the selection of KNs for answering a query. Based on this, we further propose the Consistency-Aware KN modification method, which improves the performance of knowledge modification. We conduct 39 sets of experiments, along with additional visualization experiments, to rigorously validate our conclusions.


title: Microwave Quantum Illumination with Optical Memory and Single-Mode Phase-Conjugate Receiver

abstract: Microwave quantum illumination with entangled pairs of microwave signal and optical idler modes, can achieve the sub-optimal performance with joint measurement of the signal and idler modes. Here, we first propose a testbed of microwave quantum illumination with an optical memory which is simulated with a delay line in the idler mode. It provides how much an input two-mode squeezing is necessary to compensate the loss of the optical memory, while maintaining quantum advantage over coherent state. When the memory is lossy, the input two-mode squeezing has to be higher through high cooperativity in the optical mode. Under the testbed, we propose a single-mode phase conjugate receiver that consists of a low-reflectivity beam splitter, an electro-optomechanical phase conjugator, and a photon number resolving detector. The performance of the newly proposed receiver approaches the maximum quantum advantage for local measurement. Furthermore, the quantum advantage is obtained even with an on-off detection while being robust against the loss of the memory.


title: PuTR: A Pure Transformer for Decoupled and Online Multi-Object Tracking

abstract: Recent advances in Multi-Object Tracking (MOT) have achieved remarkable success in short-term association within the decoupled tracking-by-detection online paradigm. However, long-term tracking still remains a challenging task. Although graph-based approaches can address this issue by modeling trajectories as a graph in the decoupled manner, their non-online nature poses obstacles for real-time applications. In this paper, we demonstrate that the trajectory graph is a directed acyclic graph, which can be represented by an object sequence arranged by frame and a binary adjacency matrix. It is a coincidence that the binary matrix matches the attention mask in the Transformer, and the object sequence serves exactly as a natural input sequence. Intuitively, we propose that a pure Transformer can naturally unify short- and long-term associations in a decoupled and online manner. Our experiments show that a classic Transformer architecture naturally suits the association problem and achieves a strong baseline compared to existing foundational methods across four datasets: DanceTrack, SportsMOT, MOT17, and MOT20, as well as superior generalizability in domain shift. Moreover, the decoupled property also enables efficient training and inference. This work pioneers a promising Transformer-based approach for the MOT task, and provides code to facilitate further research. https://github.com/chongweiliu/PuTR


title: DendroPy 5: a mature Python library for phylogenetic computing

abstract: Contemporary bioinformatics has seen in profound new visibility into the composition, structure, and history of the natural world around us. Arguably, the central pillar of bioinformatics is phylogenetics -- the study of hereditary relatedness among organisms. Insight from phylogenetic analysis has touched nearly every corner of biology. Examples range across natural history, population genetics and phylogeography, conservation biology, public health, medicine, in vivo and in silico experimental evolution, application-oriented evolutionary algorithms, and beyond. High-throughput genetic and phenotypic data has realized groundbreaking results, in large part, through conjunction with open-source software used to process and analyze it. Indeed, the preceding decades have ushered in a flourishing ecosystem of bioinformatics software applications and libraries. Over the course of its nearly fifteen-year history, the DendroPy library for phylogenetic computation in Python has established a generalist niche in serving the bioinformatics community. Here, we report on the recent major release of the library, DendroPy version 5. The software release represents a major milestone in transitioning the library to a sustainable long-term development and maintenance trajectory. As such, this work positions DendroPy to continue fulfilling a key supporting role in phyloinformatics infrastructure.


title: One-shot Active Learning Based on Lewis Weight Sampling for Multiple Deep Models

abstract: Active learning (AL) for multiple target models aims to reduce labeled data querying while effectively training multiple models concurrently. Existing AL algorithms often rely on iterative model training, which can be computationally expensive, particularly for deep models. In this paper, we propose a one-shot AL method to address this challenge, which performs all label queries without repeated model training. Specifically, we extract different representations of the same dataset using distinct network backbones, and actively learn the linear prediction layer on each representation via an $\ell_p$-regression formulation. The regression problems are solved approximately by sampling and reweighting the unlabeled instances based on their maximum Lewis weights across the representations. An upper bound on the number of samples needed is provided with a rigorous analysis for $p\in [1, +\infty)$. Experimental results on 11 benchmarks show that our one-shot approach achieves competitive performances with the state-of-the-art AL methods for multiple target models.


title: Modeling Other Players with Bayesian Beliefs for Games with Incomplete Information

abstract: Bayesian games model interactive decision-making where players have incomplete information -- e.g., regarding payoffs and private data on players' strategies and preferences -- and must actively reason and update their belief models (with regard to such information) using observation and interaction history. Existing work on counterfactual regret minimization have shown great success for games with complete or imperfect information, but not for Bayesian games. To this end, we introduced a new CFR algorithm: Bayesian-CFR and analyze its regret bound with respect to Bayesian Nash Equilibria in Bayesian games. First, we present a method for updating the posterior distribution of beliefs about the game and other players' types. The method uses a kernel-density estimate and is shown to converge to the true distribution. Second, we define Bayesian regret and present a Bayesian-CFR minimization algorithm for computing the Bayesian Nash equilibrium. Finally, we extend this new approach to other existing algorithms, such as Bayesian-CFR+ and Deep Bayesian CFR. Experimental results show that our proposed solutions significantly outperform existing methods in classical Texas Hold'em games.


title: Equations for the overlaps of a SIC

abstract: We give a holomorphic quartic polynomial in the overlap variables whose zeros on the torus are precisely the Weyl-Heisenberg SICs (symmetric informationally complete positive operator valued measures). By way of comparison, all the other known systems of equations that determine a Weyl-Heisenberg SIC involve variables and their complex conjugates. We also give a related interesting result about the powers of the projective Fourier transform of the group G = Z d x Z d .


title: Mixture of Experts Meets Prompt-Based Continual Learning

abstract: Exploiting the power of pre-trained models, prompt-based approaches stand out compared to other continual learning solutions in effectively preventing catastrophic forgetting, even with very few learnable parameters and without the need for a memory buffer. While existing prompt-based continual learning methods excel in leveraging prompts for state-of-the-art performance, they often lack a theoretical explanation for the effectiveness of prompting. This paper conducts a theoretical analysis to unravel how prompts bestow such advantages in continual learning, thus offering a new perspective on prompt design. We first show that the attention block of pre-trained models like Vision Transformers inherently encodes a special mixture of experts architecture, characterized by linear experts and quadratic gating score functions. This realization drives us to provide a novel view on prefix tuning, reframing it as the addition of new task-specific experts, thereby inspiring the design of a novel gating mechanism termed Non-linear Residual Gates (NoRGa). Through the incorporation of non-linear activation and residual connection, NoRGa enhances continual learning performance while preserving parameter efficiency. The effectiveness of NoRGa is substantiated both theoretically and empirically across diverse benchmarks and pretraining paradigms.


title: ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluation

abstract: Large Language Models (LLMs) can elicit unintended and even harmful content when misaligned with human values, posing severe risks to users and society. To mitigate these risks, current evaluation benchmarks predominantly employ expert-designed contextual scenarios to assess how well LLMs align with human values. However, the labor-intensive nature of these benchmarks limits their test scope, hindering their ability to generalize to the extensive variety of open-world use cases and identify rare but crucial long-tail risks. Additionally, these static tests fail to adapt to the rapid evolution of LLMs, making it hard to evaluate timely alignment issues. To address these challenges, we propose ALI-Agent, an evaluation framework that leverages the autonomous abilities of LLM-powered agents to conduct in-depth and adaptive alignment assessments. ALI-Agent operates through two principal stages: Emulation and Refinement. During the Emulation stage, ALI-Agent automates the generation of realistic test scenarios. In the Refinement stage, it iteratively refines the scenarios to probe long-tail risks. Specifically, ALI-Agent incorporates a memory module to guide test scenario generation, a tool-using module to reduce human labor in tasks such as evaluating feedback from target LLMs, and an action module to refine tests. Extensive experiments across three aspects of human values--stereotypes, morality, and legality--demonstrate that ALI-Agent, as a general evaluation framework, effectively identifies model misalignment. Systematic analysis also validates that the generated test scenarios represent meaningful use cases, as well as integrate enhanced measures to probe long-tail risks. Our code is available at https://github.com/SophieZheng998/ALI-Agent.git


title: The Disappearance of Timestep Embedding in Modern Time-Dependent Neural Networks

abstract: Dynamical systems are often time-varying, whose modeling requires a function that evolves with respect to time. Recent studies such as the neural ordinary differential equation proposed a time-dependent neural network, which provides a neural network varying with respect to time. However, we claim that the architectural choice to build a time-dependent neural network significantly affects its time-awareness but still lacks sufficient validation in its current states. In this study, we conduct an in-depth analysis of the architecture of modern time-dependent neural networks. Here, we report a vulnerability of vanishing timestep embedding, which disables the time-awareness of a time-dependent neural network. Furthermore, we find that this vulnerability can also be observed in diffusion models because they employ a similar architecture that incorporates timestep embedding to discriminate between different timesteps during a diffusion process. Our analysis provides a detailed description of this phenomenon as well as several solutions to address the root cause. Through experiments on neural ordinary differential equations and diffusion models, we observed that ensuring alive time-awareness via proposed solutions boosted their performance, which implies that their current implementations lack sufficient time-dependency.


title: Non-unique solutions for electron MHD

abstract: We consider the electron magnetohydrodynamics (MHD) equation on the 3D torus $\mathbb T^3$. For a given smooth vector field $H$ with zero mean and zero divergence, we can construct a weak solution $B$ to the electron MHD in the space $L^\gamma_tW^{1,p}_x$ for appropriate $(\gamma, p)$ such that $B$ is arbitrarily close to $H$ in this space. The parameters $\gamma$ and $p$ depend on the resistivity. As a consequence, non-uniqueness of weak solutions is obtained for the electron MHD with hyper-resistivity. In particular, non-Leray-Hopf solutions can be constructed. As a byproduct, we also show the existence of weak solutions to the electron MHD without resistivity.


title: Transformers for Image-Goal Navigation

abstract: Visual perception and navigation have emerged as major focus areas in the field of embodied artificial intelligence. We consider the task of image-goal navigation, where an agent is tasked to navigate to a goal specified by an image, relying only on images from an onboard camera. This task is particularly challenging since it demands robust scene understanding, goal-oriented planning and long-horizon navigation. Most existing approaches typically learn navigation policies reliant on recurrent neural networks trained via online reinforcement learning. However, training such policies requires substantial computational resources and time, and performance of these models is not reliable on long-horizon navigation. In this work, we present a generative Transformer based model that jointly models image goals, camera observations and the robot's past actions to predict future actions. We use state-of-the-art perception models and navigation policies to learn robust goal conditioned policies without the need for real-time interaction with the environment. Our model demonstrates capability in capturing and associating visual information across long time horizons, helping in effective navigation. NOTE: This work was submitted as part of a Master's Capstone Project and must be treated as such. This is still an early work in progress and not the final version.


title: AlignGPT: Multi-modal Large Language Models with Adaptive Alignment Capability

abstract: Multimodal Large Language Models (MLLMs) are widely regarded as crucial in the exploration of Artificial General Intelligence (AGI). The core of MLLMs lies in their capability to achieve cross-modal alignment. To attain this goal, current MLLMs typically follow a two-phase training paradigm: the pre-training phase and the instruction-tuning phase. Despite their success, there are shortcomings in the modeling of alignment capabilities within these models. Firstly, during the pre-training phase, the model usually assumes that all image-text pairs are uniformly aligned, but in fact the degree of alignment between different image-text pairs is inconsistent. Secondly, the instructions currently used for finetuning incorporate a variety of tasks, different tasks's instructions usually require different levels of alignment capabilities, but previous MLLMs overlook these differentiated alignment needs. To tackle these issues, we propose a new multimodal large language model AlignGPT. In the pre-training stage, instead of treating all image-text pairs equally, we assign different levels of alignment capabilities to different image-text pairs. Then, in the instruction-tuning phase, we adaptively combine these different levels of alignment capabilities to meet the dynamic alignment needs of different instructions. Extensive experimental results show that our model achieves competitive performance on 12 benchmarks.


title: High-probability complexity guarantees for nonconvex minimax problems

abstract: Stochastic smooth nonconvex minimax problems are prevalent in machine learning, e.g., GAN training, fair classification, and distributionally robust learning. Stochastic gradient descent ascent (GDA)-type methods are popular in practice due to their simplicity and single-loop nature. However, there is a significant gap between the theory and practice regarding high-probability complexity guarantees for these methods on stochastic nonconvex minimax problems. Existing high-probability bounds for GDA-type single-loop methods only apply to convex/concave minimax problems and to particular non-monotone variational inequality problems under some restrictive assumptions. In this work, we address this gap by providing the first high-probability complexity guarantees for nonconvex/PL minimax problems corresponding to a smooth function that satisfies the PL-condition in the dual variable. Specifically, we show that when the stochastic gradients are light-tailed, the smoothed alternating GDA method can compute an $\varepsilon$-stationary point within $O(\frac{\ell \kappa^2 \delta^2}{\varepsilon^4} + \frac{\kappa}{\varepsilon^2}(\ell+\delta^2\log({1}/{\bar{q}})))$ stochastic gradient calls with probability at least $1-\bar{q}$ for any $\bar{q}\in(0,1)$, where $\mu$ is the PL constant, $\ell$ is the Lipschitz constant of the gradient, $\kappa=\ell/\mu$ is the condition number, and $\delta^2$ denotes a bound on the variance of stochastic gradients. We also present numerical results on a nonconvex/PL problem with synthetic data and on distributionally robust optimization problems with real data, illustrating our theoretical findings.


title: Statistical Advantages of Perturbing Cosine Router in Sparse Mixture of Experts

abstract: The cosine router in sparse Mixture of Experts (MoE) has recently emerged as an attractive alternative to the conventional linear router. Indeed, the cosine router demonstrates favorable performance in image and language tasks and exhibits better ability to mitigate the representation collapse issue, which often leads to parameter redundancy and limited representation potentials. Despite its empirical success, a comprehensive analysis of the cosine router in sparse MoE has been lacking. Considering the least square estimation of the cosine routing sparse MoE, we demonstrate that due to the intrinsic interaction of the model parameters in the cosine router via some partial differential equations, regardless of the structures of the experts, the estimation rates of experts and model parameters can be as slow as $\mathcal{O}(1/\log^{\tau}(n))$ where $\tau > 0$ is some constant and $n$ is the sample size. Surprisingly, these pessimistic non-polynomial convergence rates can be circumvented by the widely used technique in practice to stabilize the cosine router -- simply adding noises to the $\mathbb{L}_{2}$ norms in the cosine router, which we refer to as \textit{perturbed cosine router}. Under the strongly identifiable settings of the expert functions, we prove that the estimation rates for both the experts and model parameters under the perturbed cosine routing sparse MoE are significantly improved to polynomial rates. Finally, we conduct extensive simulation studies in both synthetic and real data settings to empirically validate our theoretical results.


title: Text-to-Model: Text-Conditioned Neural Network Diffusion for Train-Once-for-All Personalization

abstract: Generative artificial intelligence (GenAI) has made significant progress in understanding world knowledge and generating content from human languages across various modalities, like text-to-text large language models, text-to-image stable diffusion, and text-to-video Sora. While in this paper, we investigate the capability of GenAI for text-to-model generation, to see whether GenAI can comprehend hyper-level knowledge embedded within AI itself parameters. Specifically, we study a practical scenario termed train-once-for-all personalization, aiming to generate personalized models for diverse end-users and tasks using text prompts. Inspired by the recent emergence of neural network diffusion, we present Tina, a text-conditioned neural network diffusion for train-once-for-all personalization. Tina leverages a diffusion transformer model conditioned on task descriptions embedded using a CLIP model. Despite the astronomical number of potential personalized tasks (e.g., $1.73\times10^{13}$), by our design, Tina demonstrates remarkable in-distribution and out-of-distribution generalization even trained on small datasets ($\sim 1000$). We further verify whether and how \Tina understands world knowledge by analyzing its capabilities under zero-shot/few-shot image prompts, different numbers of personalized classes, prompts of natural language descriptions, and predicting unseen entities.


title: Automated Loss function Search for Class-imbalanced Node Classification

abstract: Class-imbalanced node classification tasks are prevalent in real-world scenarios. Due to the uneven distribution of nodes across different classes, learning high-quality node representations remains a challenging endeavor. The engineering of loss functions has shown promising potential in addressing this issue. It involves the meticulous design of loss functions, utilizing information about the quantities of nodes in different categories and the network's topology to learn unbiased node representations. However, the design of these loss functions heavily relies on human expert knowledge and exhibits limited adaptability to specific target tasks. In this paper, we introduce a high-performance, flexible, and generalizable automated loss function search framework to tackle this challenge. Across 15 combinations of graph neural networks and datasets, our framework achieves a significant improvement in performance compared to state-of-the-art methods. Additionally, we observe that homophily in graph-structured data significantly contributes to the transferability of the proposed framework.


title: Sharp convergence rate on Schr\"{o}dinger type operators

abstract: For Schr\"{o}dinger type operators in one dimension, we consider the relationship between the convergence rate and the regularity for initial data. By establishing the associated frequency-localized maximal estimates, we prove sharp results up to the endpoints. The optimal range for the wave operator in all dimensions is also obtained.


title: Learning Geospatial Region Embedding with Heterogeneous Graph

abstract: Learning effective geospatial embeddings is crucial for a series of geospatial applications such as city analytics and earth monitoring. However, learning comprehensive region representations presents two significant challenges: first, the deficiency of effective intra-region feature representation; and second, the difficulty of learning from intricate inter-region dependencies. In this paper, we present GeoHG, an effective heterogeneous graph structure for learning comprehensive region embeddings for various downstream tasks. Specifically, we tailor satellite image representation learning through geo-entity segmentation and point-of-interest (POI) integration for expressive intra-regional features. Furthermore, GeoHG unifies informative spatial interdependencies and socio-environmental attributes into a powerful heterogeneous graph to encourage explicit modeling of higher-order inter-regional relationships. The intra-regional features and inter-regional correlations are seamlessly integrated by a model-agnostic graph learning framework for diverse downstream tasks. Extensive experiments demonstrate the effectiveness of GeoHG in geo-prediction tasks compared to existing methods, even under extreme data scarcity (with just 5% of training data). With interpretable region representations, GeoHG exhibits strong generalization capabilities across regions. We will release code and data upon paper notification.


title: Efficient Multitask Dense Predictor via Binarization

abstract: Multi-task learning for dense prediction has emerged as a pivotal area in computer vision, enabling simultaneous processing of diverse yet interrelated pixel-wise prediction tasks. However, the substantial computational demands of state-of-the-art (SoTA) models often limit their widespread deployment. This paper addresses this challenge by introducing network binarization to compress resource-intensive multi-task dense predictors. Specifically, our goal is to significantly accelerate multi-task dense prediction models via Binary Neural Networks (BNNs) while maintaining and even improving model performance at the same time. To reach this goal, we propose a Binary Multi-task Dense Predictor, Bi-MTDP, and several variants of Bi-MTDP, in which a multi-task dense predictor is constructed via specified binarized modules. Our systematical analysis of this predictor reveals that performance drop from binarization is primarily caused by severe information degradation. To address this issue, we introduce a deep information bottleneck layer that enforces representations for downstream tasks satisfying Gaussian distribution in forward propagation. Moreover, we introduce a knowledge distillation mechanism to correct the direction of information flow in backward propagation. Intriguingly, one variant of Bi-MTDP outperforms full-precision (FP) multi-task dense prediction SoTAs, ARTC (CNN-based) and InvPT (ViT-Based). This result indicates that Bi-MTDP is not merely a naive trade-off between performance and efficiency, but is rather a benefit of the redundant information flow thanks to the multi-task architecture. Code is available at https://github.com/42Shawn/BiMTDP.


title: RET-CLIP: A Retinal Image Foundation Model Pre-trained with Clinical Diagnostic Reports

abstract: The Vision-Language Foundation model is increasingly investigated in the fields of computer vision and natural language processing, yet its exploration in ophthalmology and broader medical applications remains limited. The challenge is the lack of labeled data for the training of foundation model. To handle this issue, a CLIP-style retinal image foundation model is developed in this paper. Our foundation model, RET-CLIP, is specifically trained on a dataset of 193,865 patients to extract general features of color fundus photographs (CFPs), employing a tripartite optimization strategy to focus on left eye, right eye, and patient level to reflect real-world clinical scenarios. Extensive experiments demonstrate that RET-CLIP outperforms existing benchmarks across eight diverse datasets spanning four critical diagnostic categories: diabetic retinopathy, glaucoma, multiple disease diagnosis, and multi-label classification of multiple diseases, which demonstrate the performance and generality of our foundation model. The sourse code and pre-trained model are available at https://github.com/sStonemason/RET-CLIP.


title: High transparency induced superconductivity in field effect two-dimensional electron gases in undoped InAs/AlGaSb surface quantum wells

abstract: We report on transport characteristics of field effect two-dimensional electron gases (2DEG) in 24 nm wide indium arsenide surface quantum wells. High quality single-subband magnetotransport with clear quantized integer quantum Hall plateaus are observed to filling factor $\nu=2$ in magnetic fields of up to B = 18 T, at electron densities up to 8$\times 10^{11}$ /cm$^2$. Peak mobility is 11,000 cm$^2$/Vs at 2$\times 10^{12}$ /cm$^2$. Large Rashba spin-orbit coefficients up to 124 meV$\cdot$\r{A} are obtained through weak anti-localization (WAL) measurements. Proximitized superconductivity is demonstrated in Nb-based superconductor-normal-superconductor (SNS) junctions, yielding 78$-$99% interface transparencies from superconducting contacts fabricated ex-situ (post-growth), using two commonly-used experimental techniques for measuring transparencies. These transparencies are on a par with those reported for epitaxially-grown superconductors. These SNS junctions show characteristic voltages $I_c R_{\text{N}}$ up to 870 $\mu$V and critical current densities up to 9.6 $\mu$A/$\mu$m, among the largest values reported for Nb-InAs SNS devices.