title: Hermite coordinate interpolation kernels: application to image zooming

abstract: A number of basic image processing tasks, such as any geometric transformation require interpolation at subpixel image values. In this work we utilize the multidimensional coordinate Hermite spline interpolation defined on non-equal spaced, rectilinear grids and apply it to a very common image processing task, image zooming. Since Hermite interpolation utilizes function values, as well as partial derivative values, it is natural to apply it to image processing tasks as a special case of equi-spaced grid, using numerical approximations of the image partial derivatives at each pixel. Furthermore, the task of image interpolation requires the calculation of image values at positions with nono-zero fractional part. Thus, any spline interpolation can be written as convolution with an appropriate kernel. In this context we generate the Hermite kernels according to the derived $n-$dimensional interpolant of Theorem 2 in [1]. We show that despite the increased complexity of the interpolant, once the kernels are constructed, the Hermite spline interpolation can be applied to images as efficiently as any other less complicated method. Finally, we perform illustrative numerical examples to showcase the applicability and high accuracy of the proposed Hermite kernels for image zooming, compared to other interpolation methods, both traditional convolution-based, as well as employing deep learning, in terms of PSNR, as well as SSIM error metrics. The proposed Hermite spline kernels outperform all other methods in the majority of the test images, in experiments using many cascaded repetitions of the zoom operation. Interesting conclusions can be drawn considering all methods under comparison.


title: ADAPT to Robustify Prompt Tuning Vision Transformers

abstract: The performance of deep models, including Vision Transformers, is known to be vulnerable to adversarial attacks. Many existing defenses against these attacks, such as adversarial training, rely on full-model fine-tuning to induce robustness in the models. These defenses require storing a copy of the entire model, that can have billions of parameters, for each task. At the same time, parameter-efficient prompt tuning is used to adapt large transformer-based models to downstream tasks without the need to save large copies. In this paper, we examine parameter-efficient prompt tuning of Vision Transformers for downstream tasks under the lens of robustness. We show that previous adversarial defense methods, when applied to the prompt tuning paradigm, suffer from gradient obfuscation and are vulnerable to adaptive attacks. We introduce ADAPT, a novel framework for performing adaptive adversarial training in the prompt tuning paradigm. Our method achieves competitive robust accuracy of ~40% w.r.t. SOTA robustness methods using full-model fine-tuning, by tuning only ~1% of the number of parameters.


title: Robust inference of cooperative behaviour of multiple ion channels in voltage-clamp recordings

abstract: Recent experimental studies have shed light on the intriguing possibility that ion channels exhibit cooperative behaviour. However, a comprehensive understanding of such cooperativity remains elusive, primarily due to limitations in measuring separately the response of each channel. Rather, only the superimposed channel response can be observed, challenging existing data analysis methods. To address this gap, we propose IDC (Idealisation, Discretisation, and Cooperativity inference), a robust statistical data analysis methodology that requires only voltage-clamp current recordings of an ensemble of ion channels. The framework of IDC enables us to integrate recent advancements in idealisation techniques and coupled Markov models. Further, in the cooperativity inference phase of IDC, we introduce a minimum distance estimator and establish its statistical guarantee in the form of asymptotic consistency. We demonstrate the effectiveness and robustness of IDC through extensive simulation studies. As an application, we investigate gramicidin D channels. Our findings reveal that these channels act independently, even at varying applied voltages during voltage-clamp experiments. An implementation of IDC is available from GitLab.


title: Towards Robots That Know When They Need Help: Affordance-Based Uncertainty for Large Language Model Planners

abstract: Large language models (LLMs) showcase many desirable traits for intelligent and helpful robots. However, they are also known to hallucinate predictions. This issue is exacerbated in consumer robotics where LLM hallucinations may result in robots confidently executing plans that are contrary to user goals, relying more frequently on human assistance, or preventing the robot from asking for help at all. In this work, we present LAP, a novel approach for utilizing off-the-shelf LLM's, alongside scene and object Affordances, in robotic Planners that minimize harmful hallucinations and know when to ask for help. Our key finding is that calculating and leveraging a scene affordance score, a measure of whether a given action is possible in the provided scene, helps to mitigate hallucinations in LLM predictions and better align the LLM's confidence measure with the probability of success. We specifically propose and test three different affordance scores, which can be used independently or in tandem to improve performance across different use cases. The most successful of these individual scores involves prompting an LLM to determine if a given action is possible and safe in the given scene and uses the LLM's response to compute the score. Through experiments in both simulation and the real world, on tasks with a variety of ambiguities, we show that LAP significantly increases success rate and decreases the amount of human intervention required relative to prior art. For example, in our real-world testing paradigm, LAP decreases the human help rate of previous methods by over 33% at a success rate of 70%.


title: DecentNeRFs: Decentralized Neural Radiance Fields from Crowdsourced Images

abstract: Neural radiance fields (NeRFs) show potential for transforming images captured worldwide into immersive 3D visual experiences. However, most of this captured visual data remains siloed in our camera rolls as these images contain personal details. Even if made public, the problem of learning 3D representations of billions of scenes captured daily in a centralized manner is computationally intractable. Our approach, DecentNeRF, is the first attempt at decentralized, crowd-sourced NeRFs that require $\sim 10^4\times$ less server computing for a scene than a centralized approach. Instead of sending the raw data, our approach requires users to send a 3D representation, distributing the high computation cost of training centralized NeRFs between the users. It learns photorealistic scene representations by decomposing users' 3D views into personal and global NeRFs and a novel optimally weighted aggregation of only the latter. We validate the advantage of our approach to learn NeRFs with photorealism and minimal server computation cost on structured synthetic and real-world photo tourism datasets. We further analyze how secure aggregation of global NeRFs in DecentNeRF minimizes the undesired reconstruction of personal content by the server.


title: The separating Noether number of abelian groups of rank two

abstract: The exact value of the separating Noether number of an arbitrary finite abelian group of rank two is determined. This is done by a detailed study of the monoid of zero-sum sequences over the group.


title: Black holes, conformal symmetry, and fundamental fields

abstract: Cosmic censorship protects the outside world from black hole singularities and paves the way for assigning entropy to gravity at the event horizons. We point out a tension between cosmic censorship and the quantum backreacted geometry of Schwarzschild black holes, induced by vacuum polarization and driven by the conformal anomaly. A similar tension appears for the Weyl curvature hypothesis at the Big Bang singularity. We argue that the requirement of exact conformal symmetry resolves both conflicts and has major implications for constraining the set of fundamental constituents of the Standard Model.


title: Superconducting transition temperatures of pure vanadium and vanadium-titanium alloys in the presence of dynamical electronic correlations

abstract: Ordinary superconductors are widely assumed insensitive to small concentrations of random nonmagnetic impurities, whereas strong disorder suppresses superconductivity, ultimately leading to a superconductor-insulator transition. In between these limiting cases, a most fascinating regime may emerge where disorder enhances superconductivity. This effect is discussed here for the $\beta$-phase of vanadium-titanium alloys. Disorder is modeled using the coherent potential approximation while local electronic interactions are treated using dynamical mean-field theory. The McMillan formula is employed to estimate the superconducting transition temperature, showing a maximum at a Ti concentration of around $0.33$ for a local Coulomb interaction $U$ in the range of $2$ to $3$ eV. Our calculations quantitatively agree with the experimentally observed concentration dependent increase of $T_c$, and its maximal value of about $20\%$.


title: Quadratic Point Estimate Method for Probabilistic Moments Computation

abstract: This paper presents in detail the originally developed Quadratic Point Estimate Method (QPEM), aimed at efficiently and accurately computing the first four output moments of probabilistic distributions, using 2n^2+1 sample (or sigma) points, with n, the number of input random variables. The proposed QPEM particularly offers an effective, superior, and practical alternative to existing sampling and quadrature methods for low- and moderately-high-dimensional problems. Detailed theoretical derivations are provided proving that the proposed method can achieve a fifth or higher-order accuracy for symmetric input distributions. Various numerical examples, from simple polynomial functions to nonlinear finite element analyses with random field representations, support the theoretical findings and further showcase the validity, efficiency, and applicability of the QPEM, from low- to high-dimensional problems.

categories: math.NA cs.NA math.PR math.ST stat.CO stat.ME stat.TH


title: Diversity-Aware Agnostic Ensemble of Sharpness Minimizers

abstract: There has long been plenty of theoretical and empirical evidence supporting the success of ensemble learning. Deep ensembles in particular take advantage of training randomness and expressivity of individual neural networks to gain prediction diversity, ultimately leading to better generalization, robustness and uncertainty estimation. In respect of generalization, it is found that pursuing wider local minima result in models being more robust to shifts between training and testing sets. A natural research question arises out of these two approaches as to whether a boost in generalization ability can be achieved if ensemble learning and loss sharpness minimization are integrated. Our work investigates this connection and proposes DASH - a learning algorithm that promotes diversity and flatness within deep ensembles. More concretely, DASH encourages base learners to move divergently towards low-loss regions of minimal sharpness. We provide a theoretical backbone for our method along with extensive empirical evidence demonstrating an improvement in ensemble generalizability.


title: Machine learning predicts extreme events in ultrashort pulse lasers

abstract: In this paper we present a nonlinear autoregressive neural network with a hidden layer of 50 neurons, three delays and one output layer that accurately is capable of predict the appearence of extreme events in a Kerr lens mode locking Ti:Sapphire laser with ultrashort pulses. Extreme events are produced in the context of a chaotic atractor and with chirped pulses. The prediction of this neural network works well with experimental and theoretical time series of amplitude of laser pulses. When fed with experimental time series we have 95.45\% of hits and 6.67\% of false positives while using theoretical time series the network predicts 100\% of extreme events but the false positive rise to 23.33\%.


title: Depth-guided NeRF Training via Earth Mover's Distance

abstract: Neural Radiance Fields (NeRFs) are trained to minimize the rendering loss of predicted viewpoints. However, the photometric loss often does not provide enough information to disambiguate between different possible geometries yielding the same image. Previous work has thus incorporated depth supervision during NeRF training, leveraging dense predictions from pre-trained depth networks as pseudo-ground truth. While these depth priors are assumed to be perfect once filtered for noise, in practice, their accuracy is more challenging to capture. This work proposes a novel approach to uncertainty in depth priors for NeRF supervision. Instead of using custom-trained depth or uncertainty priors, we use off-the-shelf pretrained diffusion models to predict depth and capture uncertainty during the denoising process. Because we know that depth priors are prone to errors, we propose to supervise the ray termination distance distribution with Earth Mover's Distance instead of enforcing the rendered depth to replicate the depth prior exactly through L2-loss. Our depth-guided NeRF outperforms all baselines on standard depth metrics by a large margin while maintaining performance on photometric measures.


title: Condensed-Phase Quantum Chemistry

abstract: Molecular quantum chemistry has seen enormous progress in the last few decades thanks to the more advanced and sophisticated numerical techniques and computing power. Following the recent interest in extending these capabilities to condensed-phase problems, we summarize basic knowledge of condensed-phase quantum chemistry for ones with experience in molecular quantum chemistry. We highlight recent efforts in this direction, including solving the electron repulsion integrals bottleneck and implementing hybrid density functional theory and wavefunction methods, and lattice dynamics for periodic systems within atom-centered basis sets. Many computational techniques presented here are inspired by the extensive method developments rooted in quantum chemistry. In this Focus Article, we selectively focus on the computational techniques rooted in molecular quantum chemistry, emphasize some challenges, and point out open questions. We hope our perspectives will encourage researchers to pursue this exciting and promising research avenue.


title: CaDRE: Controllable and Diverse Generation of Safety-Critical Driving Scenarios using Real-World Trajectories

abstract: Simulation is an indispensable tool in the development and testing of autonomous vehicles (AVs), offering an efficient and safe alternative to road testing by allowing the exploration of a wide range of scenarios. Despite its advantages, a significant challenge within simulation-based testing is the generation of safety-critical scenarios, which are essential to ensure that AVs can handle rare but potentially fatal situations. This paper addresses this challenge by introducing a novel generative framework, CaDRE, which is specifically designed for generating diverse and controllable safety-critical scenarios using real-world trajectories. Our approach optimizes for both the quality and diversity of scenarios by employing a unique formulation and algorithm that integrates real-world data, domain knowledge, and black-box optimization techniques. We validate the effectiveness of our framework through extensive testing in three representative types of traffic scenarios. The results demonstrate superior performance in generating diverse and high-quality scenarios with greater sample efficiency than existing reinforcement learning and sampling-based methods.


title: At least one in a dozen stars exhibits evidence of planetary ingestion

abstract: Stellar chemical compositions can be altered by ingestion of planetary material and/or planet formation which removes refractory material from the proto-stellar disc. These "planet signatures" appear as correlations between elemental abundance differences and the dust condensation temperature. Detecting these planet signatures, however, is challenging due to unknown occurrence rates, small amplitudes, and heterogeneous star samples with large differences in stellar ages, and therefore stars born together (i.e., co-natal) with identical compositions can facilitate such detections. While previous spectroscopic studies were limited to small number of binary stars, the Gaia satellite provides new opportunities for detecting stellar chemical signatures of planets among co-moving pairs of stars confirmed to be co-natal. Here we report high-precision chemical abundances for a homogeneous sample of 91 co-natal pairs of stars with a well-defined selection function and identify at least seven new instances of planetary ingestion, corresponding to an occurrence rate of 8%. An independent Bayesian indicator is deployed, which can effectively disentangle the planet signatures from other factors, such as random abundance variation and atomic diffusion. Our study provides new evidence of planet signatures and facilitates a deeper understanding of the star-planet-chemistry connection by providing new observational constraints on the mechanisms of planet engulfment, formation and evolution.


title: Modeling RIS from Electromagnetic Principles to Communication Systems--Part II: System-Level Simulation, Ray Tracing, and Measurement

abstract: In this paper, we systematically study the electromagnetic (EM) and communication aspects of an RIS through EM simulations, system-level and ray-tracing simulations, and finally measurements. We simulate a nearly perfect, lossless RIS, and a realistic lossy anomalous reflector (AR) in different ray tracers and analyze the large-scale fading of simple RIS-assisted links. We also compare the results with continuous and quantized unit cell reflection phases with one to four-bit resolutions. Finally, we perform over-the-air communication link measurements in an indoor setting with a manufactured sample of a wide-angle AR. The EM, system-level, and ray-tracing simulation results show good agreement with the measurement results. It is proved that the introduced macroscopic model of RIS from the EM aspects is consistent with our proposed communication models, both for an ideal RIS and a realistic AR.


title: The Riemannian Penrose Inequality with Matter Density

abstract: Riemannian Penrose Inequalities are precise geometric statements that imply that the total mass of a zero second fundamental form slice of a spacetime is at least the mass contributed by the black holes, assuming that the spacetime has nonnegative matter density everywhere. In this paper, we remove this last assumption, and prove stronger statements that the total mass is at least the mass contributed by the black holes, plus a contribution coming from the matter density along the slice. We use the first author's conformal flow to achieve this, combined with Stern's harmonic level set techniques in the first case, and spinors in the second case. We then compare these new results to results previously known from Huisken-Ilmanen's inverse mean curvature flow techniques.


title: Stability for a multi-frequency inverse random source problem

abstract: We present stability estimates for the inverse source problem of the stochastic Helmholtz equation in two and three dimensions by either near-field or far-field data. The random source is assumed to be a microlocally isotropic generalized Gaussian random function. For the direct problem, by exploring the regularity of the Green function, we demonstrate that the direct problem admits a unique bounded solution with an explicit integral representation, which enhances the existing regularity result. For the case using near-field data, the analysis of the inverse problem employs microlocal analysis to achieve an estimate for the Fourier transform of the micro-correlation strength by the near-field correlation data and a high-frequency tail. The stability follows by showing the analyticity of the data and applying a novel analytic continuation principle. The stability estimate by far-field data is derived by investigating the correlation of the far-field data. The stability estimate consists of the Lipschitz type data discrepancy and the logarithmic stability. The latter decreases as the upper bound of the frequency increases, which exhibits the phenomenon of increasing stability.


title: From Representational Harms to Quality-of-Service Harms: A Case Study on Llama 2 Safety Safeguards

abstract: Recent progress in large language models (LLMs) has led to their widespread adoption in various domains. However, these advancements have also introduced additional safety risks and raised concerns regarding their detrimental impact on already marginalized populations. Despite growing mitigation efforts to develop safety safeguards, such as supervised safety-oriented fine-tuning and leveraging safe reinforcement learning from human feedback, multiple concerns regarding the safety and ingrained biases in these models remain. Furthermore, previous work has demonstrated that models optimized for safety often display exaggerated safety behaviors, such as a tendency to refrain from responding to certain requests as a precautionary measure. As such, a clear trade-off between the helpfulness and safety of these models has been documented in the literature. In this paper, we further investigate the effectiveness of safety measures by evaluating models on already mitigated biases. Using the case of Llama 2 as an example, we illustrate how LLMs' safety responses can still encode harmful assumptions. To do so, we create a set of non-toxic prompts, which we then use to evaluate Llama models. Through our new taxonomy of LLMs responses to users, we observe that the safety/helpfulness trade-offs are more pronounced for certain demographic groups which can lead to quality-of-service harms for marginalized populations.


title: Nellie: Automated organelle segmentation, tracking, and hierarchical feature extraction in 2D/3D live-cell microscopy

abstract: The analysis of dynamic organelles remains a formidable challenge, though key to understanding biological processes. We introduce Nellie, an automated and unbiased pipeline for segmentation, tracking, and feature extraction of diverse intracellular structures. Nellie adapts to image metadata, eliminating user input. Nellie's preprocessing pipeline enhances structural contrast on multiple intracellular scales allowing for robust hierarchical segmentation of sub-organellar regions. Internal motion capture markers are generated and tracked via a radius-adaptive pattern matching scheme, and used as guides for sub-voxel flow interpolation. Nellie extracts a plethora of features at multiple hierarchical levels for deep and customizable analysis. Nellie features a Napari-based GUI that allows for code-free operation and visualization, while its modular open-source codebase invites customization by experienced users. We demonstrate Nellie's wide variety of use cases with two examples: unmixing multiple organelles from a single channel using feature-based classification and training an unsupervised graph autoencoder on mitochondrial multi-mesh graphs to quantify latent space embedding changes following ionomycin treatment.


title: Leveraging advances in machine learning for the robust classification and interpretation of networks

abstract: The ability to simulate realistic networks based on empirical data is an important task across scientific disciplines, from epidemiology to computer science. Often simulation approaches involve selecting a suitable network generative model such as Erd\"os-R\'enyi or small-world. However, few tools are available to quantify if a particular generative model is suitable for capturing a given network structure or organization. We utilize advances in interpretable machine learning to classify simulated networks by our generative models based on various network attributes, using both primary features and their interactions. Our study underscores the significance of specific network features and their interactions in distinguishing generative models, comprehending complex network structures, and the formation of real-world networks.


title: The Schwarzschild Black Hole from Perturbation Theory to all Orders

abstract: Applying the quantum field theoretic perturbiner approach to Einstein gravity, we compute the metric of a Schwarzschild black hole order by order in perturbation theory. Using recursion, this calculation can be carried out in de Donder gauge to all orders in Newton's constant. The result is a geometric series which is convergent outside a disk of finite radius, and it agrees within its region of convergence with the known de Donder gauge metric of a Schwarzschild black hole. It thus provides a first all-order perturbative computation in Einstein gravity with a matter source, and this series converges to the known non-perturbative expression in the expected range of convergence.


title: Transmission across a ribbon containing a square PT impurity

abstract: We study the spectrum and transmission coefficient of plane waves propagating along square ribbons of varying widths, containing a square-shaped, PT-symmetric impurity region. We start with a zero-width ribbon (1D chain) and place a PT symmetric dimer. The spectrum is computed numerically and the instability gain is computed as a function of the gain/loss dimer strength. The transmission coefficient is obtained in closed form and examined as a function of wavevector and the gain/loss parameter. Next, we study a ribbon in a narrow ladder configuration containing a square PT impurity. As before, we compute the instability gain numerically and the transmission coefficient in closed form for the two possible input modes. Finally, we repeat the calculations for a wider ladder ribbon containing a Lieb-like impurity in a PT configuration. For all cases and transmission channels, we obtain transmission divergences in wavevector-gain/loss parameter space, whose number increases with the width of the ribbon


title: Self-Attention Based Semantic Decomposition in Vector Symbolic Architectures

abstract: Vector Symbolic Architectures (VSAs) have emerged as a novel framework for enabling interpretable machine learning algorithms equipped with the ability to reason and explain their decision processes. The basic idea is to represent discrete information through high dimensional random vectors. Complex data structures can be built up with operations over vectors such as the "binding" operation involving element-wise vector multiplication, which associates data together. The reverse task of decomposing the associated elements is a combinatorially hard task, with an exponentially large search space. The main algorithm for performing this search is the resonator network, inspired by Hopfield network-based memory search operations. In this work, we introduce a new variant of the resonator network, based on self-attention based update rules in the iterative search problem. This update rule, based on the Hopfield network with log-sum-exp energy function and norm-bounded states, is shown to substantially improve the performance and rate of convergence. As a result, our algorithm enables a larger capacity for associative memory, enabling applications in many tasks like perception based pattern recognition, scene decomposition, and object reasoning. We substantiate our algorithm with a thorough evaluation and comparisons to baselines.


title: Diffusion Model for Data-Driven Black-Box Optimization

abstract: Generative AI has redefined artificial intelligence, enabling the creation of innovative content and customized solutions that drive business practices into a new era of efficiency and creativity. In this paper, we focus on diffusion models, a powerful generative AI technology, and investigate their potential for black-box optimization over complex structured variables. Consider the practical scenario where one wants to optimize some structured design in a high-dimensional space, based on massive unlabeled data (representing design variables) and a small labeled dataset. We study two practical types of labels: 1) noisy measurements of a real-valued reward function and 2) human preference based on pairwise comparisons. The goal is to generate new designs that are near-optimal and preserve the designed latent structures. Our proposed method reformulates the design optimization problem into a conditional sampling problem, which allows us to leverage the power of diffusion models for modeling complex distributions. In particular, we propose a reward-directed conditional diffusion model, to be trained on the mixed data, for sampling a near-optimal solution conditioned on high predicted rewards. Theoretically, we establish sub-optimality error bounds for the generated designs. The sub-optimality gap nearly matches the optimal guarantee in off-policy bandits, demonstrating the efficiency of reward-directed diffusion models for black-box optimization. Moreover, when the data admits a low-dimensional latent subspace structure, our model efficiently generates high-fidelity designs that closely respect the latent structure. We provide empirical experiments validating our model in decision-making and content-creation tasks.


title: Elevating Software Quality in Agile Environments: The Role of Testing Professionals in Unit Testing

abstract: Testing is an essential quality activity in the software development process. Usually, a software system is tested on several levels, starting with unit testing that checks the smallest parts of the code until acceptance testing, which is focused on the validations with the end-user. Historically, unit testing has been the domain of developers, who are responsible for ensuring the accuracy of their code. However, in agile environments, testing professionals play an integral role in various quality improvement initiatives throughout each development cycle. This paper explores the participation of test engineers in unit testing within an industrial context, employing a survey-based research methodology. Our findings demonstrate that testing professionals have the potential to strengthen unit testing by collaborating with developers to craft thorough test cases and fostering a culture of mutual learning and cooperation, ultimately contributing to increasing the overall quality of software projects.


title: A Contact Model based on Denoising Diffusion to Learn Variable Impedance Control for Contact-rich Manipulation

abstract: In this paper, a novel approach is proposed for learning robot control in contact-rich tasks such as wiping, by developing Diffusion Contact Model (DCM). Previous methods of learning such tasks relied on impedance control with time-varying stiffness tuning by performing Bayesian optimization by trial-and-error with robots. The proposed approach aims to reduce the cost of robot operation by predicting the robot contact trajectories from the variable stiffness inputs and using neural models. However, contact dynamics are inherently highly nonlinear, and their simulation requires iterative computations such as convex optimization. Moreover, approximating such computations by using finite-layer neural models is difficult. To overcome these limitations, the proposed DCM used the denoising diffusion models that could simulate the complex dynamics via iterative computations of multi-step denoising, thus improving the prediction accuracy. Stiffness tuning experiments conducted in simulated and real environments showed that the DCM achieved comparable performance to a conventional robot-based optimization method while reducing the number of robot trials.


title: On Equilibrium Determinacy in Overlapping Generations Models with Money

abstract: This paper provides a detailed analysis of the local determinacy of monetary and non-monetary steady states in Tirole (1985)'s classical two-period overlapping generations model with capital and production. We show that the sufficient condition for local determinacy in endowment economies provided by Scheinkman (1980) does not generalize to models with production: there are robust examples with arbitrary utility functions in which the non-monetary steady state is locally determinate or indeterminate. In contrast, the monetary steady state is locally determinate under fairly weak conditions.


title: Information-theoretic derivation of energy and speed bounds

abstract: Information-theoretic insights have proven fruitful in many areas of quantum physics. But can the fundamental dynamics of quantum systems be derived from purely information-theoretic principles, without resorting to Hilbert space structures such as unitary evolution and self-adjoint observables? Here we provide a model where the dynamics originates from a condition of informational non-equilibrium, the deviation of the system's state from a reference state associated to a field of identically prepared systems. Combining this idea with three basic information-theoretic principles, we derive a notion of energy that captures the main features of energy in quantum theory: it is observable, bounded from below, invariant under time-evolution, in one-to-one correspondence with the generator of the dynamics, and quantitatively related to the speed of state changes. Our results provide an information-theoretic reconstruction of the Mandelstam-Tamm bound on the speed of quantum evolutions, establishing a bridge between dynamical and information-theoretic notions.


title: Simplex slicing: an asymptotically-sharp lower bound

abstract: We show that for the regular n-simplex, the 1-codimensional central slice that's parallel to a facet will achieve the minimum area (up to a 1-o(1) factor) among all 1-codimensional central slices, thus improving the previous best known lower bound (Brzezinski 2013) by a factor of $\frac{2\sqrt{3}}{e} \approx 1.27$. In addition to the standard technique of interpreting geometric problems as problems about probability distributions and standard Fourier-analytic techniques, we rely on a new idea, mainly \emph{changing the contour of integration} of a meromorphic function.


title: Modeling the Label Distributions for Weakly-Supervised Semantic Segmentation

abstract: Weakly-Supervised Semantic Segmentation (WSSS) aims to train segmentation models by weak labels, which is receiving significant attention due to its low annotation cost. Existing approaches focus on generating pseudo labels for supervision while largely ignoring to leverage the inherent semantic correlation among different pseudo labels. We observe that pseudo-labeled pixels that are close to each other in the feature space are more likely to share the same class, and those closer to the distribution centers tend to have higher confidence. Motivated by this, we propose to model the underlying label distributions and employ cross-label constraints to generate more accurate pseudo labels. In this paper, we develop a unified WSSS framework named Adaptive Gaussian Mixtures Model, which leverages a GMM to model the label distributions. Specifically, we calculate the feature distribution centers of pseudo-labeled pixels and build the GMM by measuring the distance between the centers and each pseudo-labeled pixel. Then, we introduce an Online Expectation-Maximization (OEM) algorithm and a novel maximization loss to optimize the GMM adaptively, aiming to learn more discriminative decision boundaries between different class-wise Gaussian mixtures. Based on the label distributions, we leverage the GMM to generate high-quality pseudo labels for more reliable supervision. Our framework is capable of solving different forms of weak labels: image-level labels, points, scribbles, blocks, and bounding-boxes. Extensive experiments on PASCAL, COCO, Cityscapes, and ADE20K datasets demonstrate that our framework can effectively provide more reliable supervision and outperform the state-of-the-art methods under all settings. Code will be available at https://github.com/Luffy03/AGMM-SASS.


title: Non-preservation of $\alpha$-concavity for the porous medium equation in higher dimensions

abstract: In this short note, we prove that $\alpha$-concavity of the pressure is not preserved for the porous medium equation in dimensions $n=3$ and higher for any $\alpha\in [0,1]\backslash \{\frac{1}{2}\}$. Together with the result of Chau-Weinkove for $n=2$, this fully resolves an open problem posed by V\'asquez on whether pressure concavity is preserved in general for the porous medium equation.


title: Pullback operators on Bargmann spaces

abstract: We characterize boundedness and compactness of pullback operators under holomorphic maps between Bargmann spaces of entire holomorphic functions with quadratic strictly plurisubharmonic exponential weights, extending a result of Carswell-MacCluer-Schuster obtained in the case of the radial quadratic weight. We also show that the pullback operator between Bargmann spaces is compact precisely when it is of trace class, with sub-exponentially decaying singular values.


title: Hilbert's Irreducibility Theorem for Linear Differential Operators

abstract: We prove a differential analogue of Hilbert's irreducibility theorem. Let $\mathcal{L}$ be a linear differential operator with coefficients in $C(\mathbb{X})(x)$ that is irreducible over $\overline{C(\mathbb{X})}(x)$, where $\mathbb{X}$ is an irreducible affine algebraic variety over an algebraically closed field $C$ of characteristic zero. We show that the set of $c\in \mathbb{X}(C)$ such that the specialized operator $\mathcal{L}^c$ of $\mathcal{L}$ remains irreducible over $C(x)$ is Zariski dense in $\mathbb{X}(C)$.


title: Extended Spreading of Saline Droplets upon Impact on a Frosty Surface

abstract: Understanding the solidification dynamics of impacted water droplets is fundamental and crucial for applications, especially with the presence of frost and salt. Here, we experimentally investigate the spreading and freezing dynamics of saline droplets upon impact on a cold, frosty surface. Our findings demonstrate that the frost and salt can lead to an extended spreading of impacted droplets under specific conditions. In addition to the well-known 1/2 inertial spreading scaling law for droplets impact on a cold substrate, we observe a distinct transition to a 1/10 scaling law dominated by the capillary-viscous relation at low impacting velocities. We formulate the onset criterion for this extended spreading regime and derive a scaling dependence that captures the droplet's arrested diameter over various supercooling temperatures, by incorporating the effect of impact inertia, partial-wetting behavior and salinity. Based on the analysis, a unified model is proposed for predicting the droplet arrested diameter over a wide range of impact velocity and salinity.


title: BFT-PoLoc: A Byzantine Fortified Trigonometric Proof of Location Protocol using Internet Delays

abstract: Internet platforms depend on accurately determining the geographical locations of online users to deliver targeted services (e.g., advertising). The advent of decentralized platforms (blockchains) emphasizes the importance of geographically distributed nodes, making the validation of locations more crucial. In these decentralized settings, mutually non-trusting participants need to {\em prove} their locations to each other. The incentives for claiming desired location include decentralization properties (validators of a blockchain), explicit rewards for improving coverage (physical infrastructure blockchains) and regulatory compliance -- and entice participants towards prevaricating their true location malicious via VPNs, tampering with internet delays, or compromising other parties (challengers) to misrepresent their location. Traditional delay-based geolocation methods focus on reducing the noise in measurements and are very vulnerable to wilful divergences from prescribed protocol. In this paper we use Internet delay measurements to securely prove the location of IP addresses while being immune to a large fraction of Byzantine actions. Our core methods are to endow Internet telemetry tools (e.g., ping) with cryptographic primitives (signatures and hash functions) together with Byzantine resistant data inferences subject to Euclidean geometric constraints. We introduce two new networking protocols, robust against Byzantine actions: Proof of Internet Geometry (PoIG) converts delay measurements into precise distance estimates across the Internet; Proof of Location (PoLoc) enables accurate and efficient multilateration of a specific IP address. The key algorithmic innovations are in conducting ``Byzantine fortified trigonometry" (BFT) inferences of data, endowing low rank matrix completion methods with Byzantine resistance.


title: Design, construction, and operation of a 1-ton Water-based Liquid scintillator detector at Brookhaven National Laboratory

abstract: Water-based liquid scintillators (WbLS) are attractive neutrino detector materials because they allow us to tune the ratio of the Cherenkov and scintillation signals. Using WbLS large-scale neutrino experiments can benefit from both directional reconstruction and enhanced low-energy efficiency. Furthermore, broadening the science capability of such materials by metal doping may be better suited for water based liquid scintillators. We recently constructed and commissioned a 1-ton WbLS detector with good photosensor coverage and a capable data acquisition system. We intend to use this flexible detector system as a testbed for WbLS R&D. In this paper we give an overview of the 1-ton system and provide some early analysis results.


title: Perturbative Approach to Time-Dependent Quantum Solitons

abstract: Recently we have introduced a lightweight, perturbative approach to quantum solitons. Thus far, our approach has been largely limited to configurations consisting of a single soliton plus a finite number of mesons, whose classical limit is an isolated stationary or rigidly moving soliton. In this paper, with an eye to soliton collisions and oscillons, we generalize this approach to quantum states whose classical limits are genuinely time-dependent. More precisely, we use a unitary operator, inspired by the coherent state approach to solitons, to factor out the nonperturbative part of the state, which includes the classical motion. The solution for the quantum state and its evolution is then reduced to an entirely perturbative problem.


title: Technical Report: Competition Solution For BetterMixture

abstract: In the era of flourishing large-scale models, the challenge of selecting and optimizing datasets from the vast and complex sea of data, to enhance the performance of large language models within the constraints of limited computational resources, has become paramount. This paper details our solution for the BetterMixture challenge, which focuses on the fine-tuning data mixing for large language models. Our approach, which secured third place, incorporates data deduplication, low-level and high-level quality filtering, and diversity selection. The foundation of our solution is Ke-Data-Juicer, an extension of Data-Juicer, demonstrating its robust capabilities in handling and optimizing data for large language models.


title: Distributionally Robust Hospital Capacity Expansion Planning under Stochastic and Correlated Patient Demand

abstract: This paper investigates the optimal locations and capacities of hospital expansion facilities under uncertain future patient demands, considering both spatial and temporal correlations. We propose a novel two-stage distributionally robust optimization (DRO) model that integrates a Spatio-Temporal Neural Network (STNN). Specifically, we develop an STNN model that predicts future hospital occupancy levels considering spatial and temporal patterns in time-series datasets over a network of hospitals. The predictions of the STNN model are then used in the construction of the ambiguity set of the DRO model. To address computational challenges associated with two-stage DRO, we employ the linear-decision-rules technique to derive a tractable mixed-integer linear programming approximation. Extensive computational experiments conducted on real-world data demonstrate the superiority of the STNN model in minimizing forecast errors. Compared to neural network models built for each individual hospital, the proposed STNN model achieves a 53% improvement in average RMSE. Furthermore, the results demonstrate the value of incorporating spatiotemporal dependencies of demand uncertainty in the DRO model, as evidenced by out-of-sample analysis conducted with both ground truth data and under perfect information scenarios.


title: AMCO: Adaptive Multimodal Coupling of Vision and Proprioception for Quadruped Robot Navigation in Outdoor Environments

abstract: We present AMCO, a novel navigation method for quadruped robots that adaptively combines vision-based and proprioception-based perception capabilities. Our approach uses three cost maps: general knowledge map; traversability history map; and current proprioception map; which are derived from a robot's vision and proprioception data, and couples them to obtain a coupled traversability cost map for navigation. The general knowledge map encodes terrains semantically segmented from visual sensing, and represents a terrain's typically expected traversability. The traversability history map encodes the robot's recent proprioceptive measurements on a terrain and its semantic segmentation as a cost map. Further, the robot's present proprioceptive measurement is encoded as a cost map in the current proprioception map. As the general knowledge map and traversability history map rely on semantic segmentation, we evaluate the reliability of the visual sensory data by estimating the brightness and motion blur of input RGB images and accordingly combine the three cost maps to obtain the coupled traversability cost map used for navigation. Leveraging this adaptive coupling, the robot can depend on the most reliable input modality available. Finally, we present a novel planner that selects appropriate gaits and velocities for traversing challenging outdoor environments using the coupled traversability cost map. We demonstrate AMCO's navigation performance in different real-world outdoor environments and observe 10.8%-34.9% reduction w.r.t. two stability metrics, and up to 50% improvement in terms of success rate compared to current navigation methods.


title: Safety-Aware Reinforcement Learning for Electric Vehicle Charging Station Management in Distribution Network

abstract: The increasing integration of electric vehicles (EVs) into the grid can pose a significant risk to the distribution system operation in the absence of coordination. In response to the need for effective coordination of EVs within the distribution network, this paper presents a safety-aware reinforcement learning (RL) algorithm designed to manage EV charging stations while ensuring the satisfaction of system constraints. Unlike existing methods, our proposed algorithm does not rely on explicit penalties for constraint violations, eliminating the need for penalty coefficient tuning. Furthermore, managing EV charging stations is further complicated by multiple uncertainties, notably the variability in solar energy generation and energy prices. To address this challenge, we develop an off-policy RL algorithm to efficiently utilize data to learn patterns in such uncertain environments. Our algorithm also incorporates a maximum entropy framework to enhance the RL algorithm's exploratory process, preventing convergence to local optimal solutions. Simulation results demonstrate that our algorithm outperforms traditional RL algorithms in managing EV charging in the distribution network.


title: Graph Attention Network-based Block Propagation with Optimal AoI and Reputation in Web 3.0

abstract: Web 3.0 is recognized as a pioneering paradigm that empowers users to securely oversee data without reliance on a centralized authority. Blockchains, as a core technology to realize Web 3.0, can facilitate decentralized and transparent data management. Nevertheless, the evolution of blockchain-enabled Web 3.0 is still in its nascent phase, grappling with challenges such as ensuring efficiency and reliability to enhance block propagation performance. In this paper, we design a Graph Attention Network (GAT)-based reliable block propagation optimization framework for blockchain-enabled Web 3.0. We first innovatively apply a data-freshness metric called age of block to measure block propagation efficiency in public blockchains. To achieve the reliability of block propagation, we introduce a reputation mechanism based on the subjective logic model, including the local and recommended opinions to calculate the miner reputation value. Moreover, considering that the GAT possesses the excellent ability to process graph-structured data, we utilize the GAT with reinforcement learning to obtain the optimal block propagation trajectory. Numerical results demonstrate that the proposed scheme exhibits the most outstanding block propagation efficiency and reliability compared with traditional routing mechanisms.


title: Beyond Skeletons: Integrative Latent Mapping for Coherent 4D Sequence Generation

abstract: Directly learning to model 4D content, including shape, color and motion, is challenging. Existing methods depend on skeleton-based motion control and offer limited continuity in detail. To address this, we propose a novel framework that generates coherent 4D sequences with animation of 3D shapes under given conditions with dynamic evolution of shape and color over time through integrative latent mapping. We first employ an integrative latent unified representation to encode shape and color information of each detailed 3D geometry frame. The proposed skeleton-free latent 4D sequence joint representation allows us to leverage diffusion models in a low-dimensional space to control the generation of 4D sequences. Finally, temporally coherent 4D sequences are generated conforming well to the input images and text prompts. Extensive experiments on the ShapeNet, 3DBiCar and DeformingThings4D datasets for several tasks demonstrate that our method effectively learns to generate quality 3D shapes with color and 4D mesh animations, improving over the current state-of-the-art. Source code will be released.


title: Exotic magnetism in perovskite KOsO3

abstract: A new perovskite KOsO3 has been stabilized under high-pressure and high temperature conditions. It is cubic at 500 K (Pm-3m) and undergoes subsequent phase transitions to tetragonal at 320 K (P4/mmm) and rhombohedral (R-3m) at 230 K as shown from refining synchrotron X-ray powder diffraction (SXRD) data. The larger orbital overlap integral and the extended wavefunction of 5d electrons in the perovskite KOsO3 allow to explore physics from the regime where Mott and Hund's rule couplings dominate to the state where the multiple interactions are on equal footing. We demonstrate an exotic magnetic ordering phase found by neutron powder diffraction along with physical properties via a suite of measurements including magnetic and transport properties, differential scanning calorimetry, and specific heat, which provide comprehensive information for a system at the crossover from localized to itinerant electronic behavior.


title: SumTra: A Differentiable Pipeline for Few-Shot Cross-Lingual Summarization

abstract: Cross-lingual summarization (XLS) generates summaries in a language different from that of the input documents (e.g., English to Spanish), allowing speakers of the target language to gain a concise view of their content. In the present day, the predominant approach to this task is to take a performing, pretrained multilingual language model (LM) and fine-tune it for XLS on the language pairs of interest. However, the scarcity of fine-tuning samples makes this approach challenging in some cases. For this reason, in this paper we propose revisiting the summarize-and-translate pipeline, where the summarization and translation tasks are performed in a sequence. This approach allows reusing the many, publicly-available resources for monolingual summarization and translation, obtaining a very competitive zero-shot performance. In addition, the proposed pipeline is completely differentiable end-to-end, allowing it to take advantage of few-shot fine-tuning, where available. Experiments over two contemporary and widely adopted XLS datasets (CrossSum and WikiLingua) have shown the remarkable zero-shot performance of the proposed approach, and also its strong few-shot performance compared to an equivalent multilingual LM baseline, that the proposed approach has been able to outperform in many languages with only 10% of the fine-tuning samples.


title: Tackling Noisy Labels with Network Parameter Additive Decomposition

abstract: Given data with noisy labels, over-parameterized deep networks suffer overfitting mislabeled data, resulting in poor generalization. The memorization effect of deep networks shows that although the networks have the ability to memorize all noisy data, they would first memorize clean training data, and then gradually memorize mislabeled training data. A simple and effective method that exploits the memorization effect to combat noisy labels is early stopping. However, early stopping cannot distinguish the memorization of clean data and mislabeled data, resulting in the network still inevitably overfitting mislabeled data in the early training stage.In this paper, to decouple the memorization of clean data and mislabeled data, and further reduce the side effect of mislabeled data, we perform additive decomposition on network parameters. Namely, all parameters are additively decomposed into two groups, i.e., parameters $\mathbf{w}$ are decomposed as $\mathbf{w}=\bm{\sigma}+\bm{\gamma}$. Afterward, the parameters $\bm{\sigma}$ are considered to memorize clean data, while the parameters $\bm{\gamma}$ are considered to memorize mislabeled data. Benefiting from the memorization effect, the updates of the parameters $\bm{\sigma}$ are encouraged to fully memorize clean data in early training, and then discouraged with the increase of training epochs to reduce interference of mislabeled data. The updates of the parameters $\bm{\gamma}$ are the opposite. In testing, only the parameters $\bm{\sigma}$ are employed to enhance generalization. Extensive experiments on both simulated and real-world benchmarks confirm the superior performance of our method.


title: Improving Legal Case Retrieval with Brain Signals

abstract: The tasks of legal case retrieval have received growing attention from the IR community in the last decade. Relevance feedback techniques with implicit user feedback (e.g., clicks) have been demonstrated to be effective in traditional search tasks (e.g., Web search). In legal case retrieval, however, collecting relevance feedback faces a couple of challenges that are difficult to resolve under existing feedback paradigms. First, legal case retrieval is a complex task as users often need to understand the relationship between legal cases in detail to correctly judge their relevance. Traditional feedback signal such as clicks is too coarse to use as they do not reflect any fine-grained relevance information. Second, legal case documents are usually long, users often need even tens of minutes to read and understand them. Simple behavior signal such as clicks and eye-tracking fixations can hardly be useful when users almost click and examine every part of the document. In this paper, we explore the possibility of solving the feedback problem in legal case retrieval with brain signal. Recent advances in brain signal processing have shown that human emotional can be collected in fine grains through Brain-Machine Interfaces (BMI) without interrupting the users in their tasks. Therefore, we propose a framework for legal case retrieval that uses EEG signal to optimize retrieval results. We collected and create a legal case retrieval dataset with users EEG signal and propose several methods to extract effective EEG features for relevance feedback. Our proposed features achieve a 71% accuracy for feedback prediction with an SVM-RFE model, and our proposed ranking method that takes into account the diverse needs of users can significantly improve user satisfaction for legal case retrieval. Experiment results show that re-ranked result list make user more satisfied.


title: A Comparative Study of Machine Learning Models Predicting Energetics of Interacting Defects

abstract: Interacting defect systems are ubiquitous in materials under realistic scenarios, yet gaining an atomic-level understanding of these systems from a computational perspective is challenging - it often demands substantial resources due to the necessity of employing supercell calculations. While machine learning techniques have shown potential in accelerating materials simulations, their application to systems involving interacting defects remains relatively rare. In this work, we present a comparative study of three different methods to predict the free energy change of systems with interacting defects. We leveraging a limited dataset from Density Functional Theory(DFT) calculations to assess the performance models using materials descriptors, graph neural networks and cluster expansion. Our findings indicate that the cluster expansion model can achieve precise energetics predictions even with this limited dataset. Furthermore, with synthetic data generate from cluster expansion model at near-DFT levels, we obtained enlarged dataset to assess the demands on data for training accurate prediction models using graph neural networks for systems featuring interacting defects. A brief discussion of the computational cost for each method is provided at the end. This research provide a preliminary evaluation of applying machine learning techniques in imperfect surface systems.


title: Instruction Multi-Constraint Molecular Generation Using a Teacher-Student Large Language Model

abstract: While various models and computational tools have been proposed for structure and property analysis of molecules, generating molecules that conform to all desired structures and properties remains a challenge. Here, we introduce a multi-constraint molecular generation large language model, TSMMG, which, akin to a student, incorporates knowledge from various small models and tools, namely, the 'teachers'. To train TSMMG, we construct a large set of text-molecule pairs by extracting molecular knowledge from these 'teachers', enabling it to generate novel molecules that conform to the descriptions through various text prompts. We experimentally show that TSMMG remarkably performs in generating molecules meeting complex, natural language-described property requirements across two-, three-, and four-constraint tasks, with an average molecular validity of over 99% and success ratio of 88.08%, 65.27%, and 61.44%, respectively. The model also exhibits adaptability through zero-shot testing, creating molecules that satisfy combinations of properties that have not been encountered. It can comprehend text inputs with various language styles, extending beyond the confines of outlined prompts, as confirmed through empirical validation. Additionally, the knowledge distillation feature of TSMMG contributes to the continuous enhancement of small models, while the innovative approach to dataset construction effectively addresses the issues of data scarcity and quality, which positions TSMMG as a promising tool in the domains of drug discovery and materials science. Code is available at https://github.com/HHW-zhou/TSMMG.


title: Federated reinforcement learning for robot motion planning with zero-shot generalization

abstract: This paper considers the problem of learning a control policy for robot motion planning with zero-shot generalization, i.e., no data collection and policy adaptation is needed when the learned policy is deployed in new environments. We develop a federated reinforcement learning framework that enables collaborative learning of multiple learners and a central server, i.e., the Cloud, without sharing their raw data. In each iteration, each learner uploads its local control policy and the corresponding estimated normalized arrival time to the Cloud, which then computes the global optimum among the learners and broadcasts the optimal policy to the learners. Each learner then selects between its local control policy and that from the Cloud for next iteration. The proposed framework leverages on the derived zero-shot generalization guarantees on arrival time and safety. Theoretical guarantees on almost-sure convergence, almost consensus, Pareto improvement and optimality gap are also provided. Monte Carlo simulation is conducted to evaluate the proposed framework.


title: Divide-Conquer Transformer Learning for Predicting Electric Vehicle Charging Events Using Smart Meter Data

abstract: Predicting electric vehicle (EV) charging events is crucial for load scheduling and energy management, promoting seamless transportation electrification and decarbonization. While prior studies have focused on EV charging demand prediction, primarily for public charging stations using historical charging data, home charging prediction is equally essential. However, existing prediction methods may not be suitable due to the unavailability of or limited access to home charging data. To address this research gap, inspired by the concept of non-intrusive load monitoring (NILM), we develop a home charging prediction method using historical smart meter data. Different from NILM detecting EV charging that has already occurred, our method provides predictive information of future EV charging occurrences, thus enhancing its utility for charging management. Specifically, our method, leverages a self-attention mechanism-based transformer model, employing a ``divide-conquer'' strategy, to process historical meter data to effectively and learn EV charging representation for charging occurrence prediction. Our method enables prediction at one-minute interval hour-ahead. Experimental results demonstrate the effectiveness of our method, achieving consistently high accuracy of over 96.81\% across different prediction time spans. Notably, our method achieves high prediction performance solely using smart meter data, making it a practical and suitable solution for grid operators.


title: FedNMUT -- Federated Noisy Model Update Tracking Convergence Analysis

abstract: A novel Decentralized Noisy Model Update Tracking Federated Learning algorithm (FedNMUT) is proposed that is tailored to function efficiently in the presence of noisy communication channels that reflect imperfect information exchange. This algorithm uses gradient tracking to minimize the impact of data heterogeneity while minimizing communication overhead. The proposed algorithm incorporates noise into its parameters to mimic the conditions of noisy communication channels, thereby enabling consensus among clients through a communication graph topology in such challenging environments. FedNMUT prioritizes parameter sharing and noise incorporation to increase the resilience of decentralized learning systems against noisy communications. Theoretical results for the smooth non-convex objective function are provided by us, and it is shown that the $\epsilon-$stationary solution is achieved by our algorithm at the rate of $\mathcal{O}\left(\frac{1}{\sqrt{T}}\right)$, where $T$ is the total number of communication rounds. Additionally, via empirical validation, we demonstrated that the performance of FedNMUT is superior to the existing state-of-the-art methods and conventional parameter-mixing approaches in dealing with imperfect information sharing. This proves the capability of the proposed algorithm to counteract the negative effects of communication noise in a decentralized learning framework.


title: Mora: Enabling Generalist Video Generation via A Multi-Agent Framework

abstract: Sora is the first large-scale generalist video generation model that garnered significant attention across society. Since its launch by OpenAI in February 2024, no other video generation models have paralleled {Sora}'s performance or its capacity to support a broad spectrum of video generation tasks. Additionally, there are only a few fully published video generation models, with the majority being closed-source. To address this gap, this paper proposes a new multi-agent framework Mora, which incorporates several advanced visual AI agents to replicate generalist video generation demonstrated by Sora. In particular, Mora can utilize multiple visual agents and successfully mimic Sora's video generation capabilities in various tasks, such as (1) text-to-video generation, (2) text-conditional image-to-video generation, (3) extend generated videos, (4) video-to-video editing, (5) connect videos and (6) simulate digital worlds. Our extensive experimental results show that Mora achieves performance that is proximate to that of Sora in various tasks. However, there exists an obvious performance gap between our work and Sora when assessed holistically. In summary, we hope this project can guide the future trajectory of video generation through collaborative AI agents.


title: A Unified and General Framework for Continual Learning

abstract: Continual Learning (CL) focuses on learning from dynamic and changing data distributions while retaining previously acquired knowledge. Various methods have been developed to address the challenge of catastrophic forgetting, including regularization-based, Bayesian-based, and memory-replay-based techniques. However, these methods lack a unified framework and common terminology for describing their approaches. This research aims to bridge this gap by introducing a comprehensive and overarching framework that encompasses and reconciles these existing methodologies. Notably, this new framework is capable of encompassing established CL approaches as special instances within a unified and general optimization objective. An intriguing finding is that despite their diverse origins, these methods share common mathematical structures. This observation highlights the compatibility of these seemingly distinct techniques, revealing their interconnectedness through a shared underlying optimization objective. Moreover, the proposed general framework introduces an innovative concept called refresh learning, specifically designed to enhance the CL performance. This novel approach draws inspiration from neuroscience, where the human brain often sheds outdated information to improve the retention of crucial knowledge and facilitate the acquisition of new information. In essence, refresh learning operates by initially unlearning current data and subsequently relearning it. It serves as a versatile plug-in that seamlessly integrates with existing CL methods, offering an adaptable and effective enhancement to the learning process. Extensive experiments on CL benchmarks and theoretical analysis demonstrate the effectiveness of the proposed refresh learning. Code is available at \url{https://github.com/joey-wang123/CL-refresh-learning}.


title: Facilitating Pornographic Text Detection for Open-Domain Dialogue Systems via Knowledge Distillation of Large Language Models

abstract: Pornographic content occurring in human-machine interaction dialogues can cause severe side effects for users in open-domain dialogue systems. However, research on detecting pornographic language within human-machine interaction dialogues is an important subject that is rarely studied. To advance in this direction, we introduce CensorChat, a dialogue monitoring dataset aimed at detecting whether the dialogue session contains pornographic content. To this end, we collect real-life human-machine interaction dialogues in the wild and break them down into single utterances and single-turn dialogues, with the last utterance spoken by the chatbot. We propose utilizing knowledge distillation of large language models to annotate the dataset. Specifically, first, the raw dataset is annotated by four open-source large language models, with the majority vote determining the label. Second, we use ChatGPT to update the empty label from the first step. Third, to ensure the quality of the validation and test sets, we utilize GPT-4 for label calibration. If the current label does not match the one generated by GPT-4, we employ a self-criticism strategy to verify its correctness. Finally, to facilitate the detection of pornographic text, we develop a series of text classifiers using a pseudo-labeled dataset. Detailed data analysis demonstrates that leveraging knowledge distillation techniques with large language models provides a practical and cost-efficient method for developing pornographic text detectors.


title: A Rule-Compliance Path Planner for Lane-Merge Scenarios Based on Responsibility-Sensitive Safety

abstract: Lane merging is one of the critical tasks for self-driving cars, and how to perform lane-merge maneuvers effectively and safely has become one of the important standards in measuring the capability of autonomous driving systems. However, due to the ambiguity in driving intentions and right-of-way issues, the lane merging process in autonomous driving remains deficient in terms of maintaining or ceding the right-of-way and attributing liability, which could result in protracted durations for merging and problems such as trajectory oscillation. Hence, we present a rule-compliance path planner (RCPP) for lane-merge scenarios, which initially employs the extended responsibility-sensitive safety (RSS) to elucidate the right-of-way, followed by the potential field-based sigmoid planner for path generation. In the simulation, we have validated the efficacy of the proposed algorithm. The algorithm demonstrated superior performance over previous approaches in aspects such as merging time (Saved 72.3%), path length (reduced 53.4%), and eliminating the trajectory oscillation.


title: Frequency-aware convolution for sound event detection

abstract: In sound event detection (SED), convolution neural networks (CNNs) are widely used to extract time-frequency patterns from the input spectrogram. However, features extracted by CNN can be insensitive to the shift of time-frequency patterns along the frequency axis. To address this issue, frequency dynamic convolution (FDY) has been proposed, which applies different kernels to different frequency components. Compared to the vannila CNN, FDY requires several times more parameters. In this paper, a more efficient solution named frequency-aware convolution (FAC) is proposed. In FAC, frequency-positional information is encoded in a vector and added to the input spectrogram. To match the amplitude of input, the encoding vector is scaled adaptively and channel-independently. Experiments are carried out in the context of DCASE 2022 task 4, and the results demonstrate that FAC can achieve comparable performance to that of FDY with only 515 additional parameters, while FDY requires 8.02 million additional parameters. The ablation study shows that scaling the encoding vector adaptively and channel-independently is critical to the performance of FAC.


title: Document Author Classification Using Parsed Language Structure

abstract: Over the years there has been ongoing interest in detecting authorship of a text based on statistical properties of the text, such as by using occurrence rates of noncontextual words. In previous work, these techniques have been used, for example, to determine authorship of all of \emph{The Federalist Papers}. Such methods may be useful in more modern times to detect fake or AI authorship. Progress in statistical natural language parsers introduces the possibility of using grammatical structure to detect authorship. In this paper we explore a new possibility for detecting authorship using grammatical structural information extracted using a statistical natural language parser. This paper provides a proof of concept, testing author classification based on grammatical structure on a set of "proof texts," The Federalist Papers and Sanditon which have been as test cases in previous authorship detection studies. Several features extracted from the statistical natural language parser were explored: all subtrees of some depth from any level; rooted subtrees of some depth, part of speech, and part of speech by level in the parse tree. It was found to be helpful to project the features into a lower dimensional space. Statistical experiments on these documents demonstrate that information from a statistical parser can, in fact, assist in distinguishing authors.


title: Onset and offset weighted loss function for sound event detection

abstract: In a typical sound event detection (SED) system, the existence of a sound event is detected at a frame level, and consecutive frames with the same event detected are combined as one sound event. The median filter is applied as a post-processing step to remove detection errors as much as possible. However, detection errors occurring around the onset and offset of a sound event are beyond the capacity of the median filter. To address this issue, an onset and offset weighted binary cross-entropy (OWBCE) loss function is proposed in this paper, which trains the DNN model to be more robust on frames around (a) onsets and offsets. Experiments are carried out in the context of DCASE 2022 task 4. Results show that OWBCE outperforms BCE when different models are considered. For a basic CRNN, relative improvements of 6.43% in event-F1, 1.96% in PSDS1, and 2.43% in PSDS2 can be achieved by OWBCE.


title: Network-Aware Value Stacking of Community Battery via Asynchronous Distributed Optimization

abstract: Community battery systems have been widely deployed to provide services to the grid. Unlike a single battery storage system in the community, coordinating multiple community batteries can further unlock their value, enhancing the viability of community battery solutions. However, the centralized control of community batteries relies on the full information of the system, which is less practical and may even lead to privacy leakage. In this paper, we formulate a value-stacking optimization problem for community batteries to interact with local solar, buildings, and the grid, within distribution network constraints. We then propose a distributed algorithm using asynchronous distributed alternate direction method of multipliers (ADMM) to solve the problem. Our algorithm is robust to communication latency between community batteries and the grid while preserving the operational privacy. The simulation results demonstrate the convergence of our proposed asynchronous distributed ADMM algorithm. We also evaluate the electricity cost and the contribution of each value stream in the value-stacking problem for community batteries using real-world data.


title: Bayesian Nonparametric Trees for Principal Causal Effects

abstract: Principal stratification analysis evaluates how causal effects of a treatment on a primary outcome vary across strata of units defined by their treatment effect on some intermediate quantity. This endeavor is substantially challenged when the intermediate variable is continuously scaled and there are infinitely many basic principal strata. We employ a Bayesian nonparametric approach to flexibly evaluate treatment effects across flexibly-modeled principal strata. The approach uses Bayesian Causal Forests (BCF) to simultaneously specify two Bayesian Additive Regression Tree models; one for the principal stratum membership and one for the outcome, conditional on principal strata. We show how the capability of BCF for capturing treatment effect heterogeneity is particularly relevant for assessing how treatment effects vary across the surface defined by continuously-scaled principal strata, in addition to other benefits relating to targeted selection and regularization-induced confounding. The capabilities of the proposed approach are illustrated with a simulation study, and the methodology is deployed to investigate how causal effects of power plant emissions control technologies on ambient particulate pollution vary as a function of the technologies' impact on sulfur dioxide emissions.


title: Arcee's MergeKit: A Toolkit for Merging Large Language Models

abstract: The rapid expansion of the open-source language model landscape presents an opportunity to merge the competencies of these model checkpoints by combining their parameters. Advances in transfer learning, the process of fine-tuning pretrained models for specific tasks, has resulted in the development of vast amounts of task-specific models, typically specialized in individual tasks and unable to utilize each other's strengths. Model merging facilitates the creation of multitask models without the need for additional training, offering a promising avenue for enhancing model performance and versatility. By preserving the intrinsic capabilities of the original models, model merging addresses complex challenges in AI - including the difficulties of catastrophic forgetting and multitask learning. To support this expanding area of research, we introduce MergeKit, a comprehensive, open-source library designed to facilitate the application of model merging strategies. MergeKit offers an extensible framework to efficiently merge models on any hardware, providing utility to researchers and practitioners. To date, thousands of models have been merged by the open-source community, leading to the creation of some of the worlds most powerful open-source model checkpoints, as assessed by the Open LLM Leaderboard. The library is accessible at https://github.com/arcee-ai/MergeKit.


title: SAMCT: Segment Any CT Allowing Labor-Free Task-Indicator Prompts

abstract: Segment anything model (SAM), a foundation model with superior versatility and generalization across diverse segmentation tasks, has attracted widespread attention in medical imaging. However, it has been proved that SAM would encounter severe performance degradation due to the lack of medical knowledge in training and local feature encoding. Though several SAM-based models have been proposed for tuning SAM in medical imaging, they still suffer from insufficient feature extraction and highly rely on high-quality prompts. In this paper, we construct a large CT dataset consisting of 1.1M CT images and 5M masks from public datasets and propose a powerful foundation model SAMCT allowing labor-free prompts. Specifically, based on SAM, SAMCT is further equipped with a U-shaped CNN image encoder, a cross-branch interaction module, and a task-indicator prompt encoder. The U-shaped CNN image encoder works in parallel with the ViT image encoder in SAM to supplement local features. Cross-branch interaction enhances the feature expression capability of the CNN image encoder and the ViT image encoder by exchanging global perception and local features from one to the other. The task-indicator prompt encoder is a plug-and-play component to effortlessly encode task-related indicators into prompt embeddings. In this way, SAMCT can work in an automatic manner in addition to the semi-automatic interactive strategy in SAM. Extensive experiments demonstrate the superiority of SAMCT against the state-of-the-art task-specific and SAM-based medical foundation models on various tasks. The code, data, and models are released at https://github.com/xianlin7/SAMCT.


title: Creative and Correct: Requesting Diverse Code Solutions from AI Foundation Models

abstract: AI foundation models have the capability to produce a wide array of responses to a single prompt, a feature that is highly beneficial in software engineering to generate diverse code solutions. However, this advantage introduces a significant trade-off between diversity and correctness. In software engineering tasks, diversity is key to exploring design spaces and fostering creativity, but the practical value of these solutions is heavily dependent on their correctness. Our study systematically investigates this trade-off using experiments with HumanEval tasks, exploring various parameter settings and prompting strategies. We assess the diversity of code solutions using similarity metrics from the code clone community. The study identifies combinations of parameters and strategies that strike an optimal balance between diversity and correctness, situated on the Pareto front of this trade-off space. These findings offer valuable insights for software engineers on how to effectively use AI foundation models to generate code solutions that are diverse and accurate.


title: A Bayesian Approach for Selecting Relevant External Data (BASE): Application to a study of Long-Term Outcomes in a Hemophilia Gene Therapy Trial

abstract: Gene therapies aim to address the root causes of diseases, particularly those stemming from rare genetic defects that can be life-threatening or severely debilitating. While there has been notable progress in the development of gene therapies in recent years, understanding their long-term effectiveness remains challenging due to a lack of data on long-term outcomes, especially during the early stages of their introduction to the market. To address the critical question of estimating long-term efficacy without waiting for the completion of lengthy clinical trials, we propose a novel Bayesian framework. This framework selects pertinent data from external sources, often early-phase clinical trials with more comprehensive longitudinal efficacy data that could lead to an improved inference of the long-term efficacy outcome. We apply this methodology to predict the long-term factor IX (FIX) levels of HEMGENIX (etranacogene dezaparvovec), the first FDA-approved gene therapy to treat adults with severe Hemophilia B, in a phase 3 study. Our application showcases the capability of the framework to estimate the 5-year FIX levels following HEMGENIX therapy, demonstrating sustained FIX levels induced by HEMGENIX infusion. Additionally, we provide theoretical insights into the methodology by establishing its posterior convergence properties.


title: Self-Supervised Class-Agnostic Motion Prediction with Spatial and Temporal Consistency Regularizations

abstract: The perception of motion behavior in a dynamic environment holds significant importance for autonomous driving systems, wherein class-agnostic motion prediction methods directly predict the motion of the entire point cloud. While most existing methods rely on fully-supervised learning, the manual labeling of point cloud data is laborious and time-consuming. Therefore, several annotation-efficient methods have been proposed to address this challenge. Although effective, these methods rely on weak annotations or additional multi-modal data like images, and the potential benefits inherent in the point cloud sequence are still underexplored. To this end, we explore the feasibility of self-supervised motion prediction with only unlabeled LiDAR point clouds. Initially, we employ an optimal transport solver to establish coarse correspondences between current and future point clouds as the coarse pseudo motion labels. Training models directly using such coarse labels leads to noticeable spatial and temporal prediction inconsistencies. To mitigate these issues, we introduce three simple spatial and temporal regularization losses, which facilitate the self-supervised training process effectively. Experimental results demonstrate the significant superiority of our approach over the state-of-the-art self-supervised methods.


title: Tuning body shape and stiffness to mitigate water-entry forces

abstract: High-speed water entry of projectiles and diving systems induces high forces and jerk to the entering bodies due to the development of large hydrodynamic pressure. Previous research has shown separately that the peak forces can be reduced by improving the aerodynamic shape of the head (nose) or, recently, by introducing a spring element between the head and body. This study seeks to understand whether the aerodynamic shape or spring stiffness coupling is most important for force reduction by combining both in one study. The experiment combines the nose cone aerodynamics and spring stiffness with a rear body and examines the forces acting on the nose and body. Three parameters are varied: the nose angle, spring stiffness, and impact velocity. An unsteady semi-analytical formulation is developed to estimate the water entry forces and coupled body dynamics. We find that the peak force reduction due to the spring is highest when the slamming force is most significant, particularly at higher impact velocities and with blunter nose angles. The spring coupling enables periodic fluctuations between the kinetic and potential energy throughout the duration of impact, which can be tuned by varying the stiffness. These findings can allow engineers to control the dynamic response of water entry.


title: SC-Tune: Unleashing Self-Consistent Referential Comprehension in Large Vision Language Models

abstract: Recent trends in Large Vision Language Models (LVLMs) research have been increasingly focusing on advancing beyond general image understanding towards more nuanced, object-level referential comprehension. In this paper, we present and delve into the self-consistency capability of LVLMs, a crucial aspect that reflects the models' ability to both generate informative captions for specific objects and subsequently utilize these captions to accurately re-identify the objects in a closed-loop process. This capability significantly mirrors the precision and reliability of fine-grained visual-language understanding. Our findings reveal that the self-consistency level of existing LVLMs falls short of expectations, posing limitations on their practical applicability and potential. To address this gap, we introduce a novel fine-tuning paradigm named Self-Consistency Tuning (SC-Tune). It features the synergistic learning of a cyclic describer-locator system. This paradigm is not only data-efficient but also exhibits generalizability across multiple LVLMs. Through extensive experiments, we demonstrate that SC-Tune significantly elevates performance across a spectrum of object-level vision-language benchmarks and maintains competitive or improved performance on image-level vision-language benchmarks. Both our model and code will be publicly available at https://github.com/ivattyue/SC-Tune.


title: Impact of Spanwise Rotation on Flow Separation and Recovery Behind a Bulge in Channel Flows

abstract: Direct numerical simulations of spanwise-rotating turbulent channel flow with a parabolic bump on the bottom wall are employed to investigate the effects of rotation on flow separation. Four rotation rates of $Ro_b := 2\Omega H/U_b = \pm 0.42, \; \pm 1.0$ are compared with the non-rotating scenario. The mild adverse pressure gradient induced by the lee side of the bump allows for a variable pressure-induced separation. The separation region is reduced (increased) when the bump is on the anti-cyclonic (cyclonic) side of the channel, compared with the non-rotating separation. The total drag is reduced in all rotating cases. Through several mechanisms, rotation alters the onset of separation, reattachment, and wake recovery. The mean momentum deficit is found to be the key. A physical interpretation of the ratio between the system rotation and mean shear vorticity, $S:=\Omega/\Omega_s$, provides the mechanisms regarding stability thresholds of $S=-0.5$ and $-1$. The rotation effects are explained accordingly with reference to the dynamics of several flow structures. For anti-cyclonic separation, particularly, the interaction between the Taylor-G\"ortler vortices and hairpin vortices of wall-bounded turbulence is proven to be responsible for the breakdown of the separating shear layer. A generalized argument is made regarding the essential role of near-wall deceleration and resultant ejection of enhanced hairpin vortices in destabilizing an anti-cyclonic flow. This mechanism is anticipated to have broad impacts on other applications in analogy to rotating shear flows, such as thermal convection and boundary layers over concave walls.


title: LHS 1140 b is a potentially habitable water world

abstract: LHS 1140 b is a small planet orbiting in the habitable zone of its M4.5V dwarf host. Recent mass and radius constraints have indicated that it has either a thick H$_2$-rich atmosphere or substantial water by mass. Here we present a transmission spectrum of LHS 1140 b between 1.7 and 5.2 $\mu$m, obtained using the NIRSpec instrument on JWST. By combining spectral retrievals and self-consistent atmospheric models, we show that the transmission spectrum is inconsistent with H$_2$-rich atmospheres with varied size and metallicity, leaving a water world as the remaining scenario to explain the planet's low density. Specifically, a H$_2$-rich atmosphere would result in prominent spectral features of CH$_4$ or CO$_2$ on this planet, but they are not seen in the transmission spectrum. Instead, the data favors a high-mean-molecular-weight atmosphere (possibly N$_2$-dominated with H$_2$O and CO$_2$) with a modest confidence. Forming the planet by accreting C- and N-bearing ices could naturally give rise to a CO$_2$- or N$_2$-dominated atmosphere, and if the planet evolves to or has the climate-stabilizing mechanism to maintain a moderate-size CO$_2$/N$_2$-dominated atmosphere, the planet could have liquid-water oceans. Our models suggest CO$_2$ absorption features with an expected signal of 20 ppm at 4.2 $\mu$m. As the existence of an atmosphere on TRAPPIST-1 planets is uncertain, LHS 1140 b may well present the best current opportunity to detect and characterize a habitable world.


title: Enhancing Security in Multi-Robot Systems through Co-Observation Planning, Reachability Analysis, and Network Flow

abstract: This paper addresses security challenges in multi-robot systems (MRS) where adversaries may compromise robot control, risking unauthorized access to forbidden areas. We propose a novel multi-robot optimal planning algorithm that integrates mutual observations and introduces reachability constraints for enhanced security. This ensures that, even with adversarial movements, compromised robots cannot breach forbidden regions without missing scheduled co-observations. The reachability constraint uses ellipsoidal over-approximation for efficient intersection checking and gradient computation. To enhance system resilience and tackle feasibility challenges, we also introduce sub-teams. These cohesive units replace individual robot assignments along each route, enabling redundant robots to deviate for co-observations across different trajectories, securing multiple sub-teams without requiring modifications. We formulate the cross-trajectory co-observation plan by solving a network flow coverage problem on the checkpoint graph generated from the original unsecured MRS trajectories, providing the same security guarantees against plan-deviation attacks. We demonstrate the effectiveness and robustness of our proposed algorithm, which significantly strengthens the security of multi-robot systems in the face of adversarial threats.


title: Dynamic Information Dissemination Model Incorporating Non-Adjacent Node Interaction

abstract: Describing the dynamics of information dissemination within social networks poses a formidable challenge. Despite multiple endeavors aimed at addressing this issue, only a limited number of studies have effectively replicated and forecasted the evolving course of information dissemination. In this paper, we propose a novel model, DM-NAI, which not only considers the information transfer between adjacent users but also takes into account the information transfer between non-adjacent users to comprehensively depict the information dissemination process. Extensive experiments are conducted on six datasets to predict the information dissemination range and the dissemination trend of the social network. The experimental results demonstrate an average prediction accuracy range of 94.62% to 96.71%, respectively, significantly outperforming state-of-the-art solutions. This finding illustrates that considering information transmission between non-adjacent users helps DM-NAI achieve more accurate information dissemination predictions.


title: Unifews: Unified Entry-Wise Sparsification for Efficient Graph Neural Network

abstract: Graph Neural Networks (GNNs) have shown promising performance in various graph learning tasks, but at the cost of resource-intensive computations. The primary overhead of GNN update stems from graph propagation and weight transformation, both involving operations on graph-scale matrices. Previous studies attempt to reduce the computational budget by leveraging graph-level or network-level sparsification techniques, resulting in downsized graph or weights. In this work, we propose Unifews, which unifies the two operations in an entry-wise manner considering individual matrix elements, and conducts joint edge-weight sparsification to enhance learning efficiency. The entry-wise design of Unifews enables adaptive compression across GNN layers with progressively increased sparsity, and is applicable to a variety of architectural designs with on-the-fly operation simplification. Theoretically, we establish a novel framework to characterize sparsified GNN learning in view of a graph optimization process, and prove that Unifews effectively approximates the learning objective with bounded error and reduced computational load. We conduct extensive experiments to evaluate the performance of our method in diverse settings. Unifews is advantageous in jointly removing more than 90% of edges and weight entries with comparable or better accuracy than baseline models. The sparsification offers remarkable efficiency improvements including 10-20x matrix operation reduction and up to 100x acceleration in graph propagation time for the largest graph at the billion-edge scale.


title: AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models

abstract: We present a novel Parameter-Efficient Fine-Tuning (PEFT) method, dubbed as Adaptive Freezing of Low Rank Adaptation (AFLoRA). Specifically, for each pre-trained frozen weight tensor, we add a parallel path of trainable low-rank matrices, namely a down-projection and an up-projection matrix, each of which is followed by a feature transformation vector. Based on a novel freezing score, we the incrementally freeze these projection matrices during fine-tuning to reduce the computation and alleviate over-fitting. Our experimental results demonstrate that we can achieve state-of-the-art performance with an average improvement of up to $0.85\%$ as evaluated on GLUE benchmark while yeilding up to $9.5\times$ fewer average trainable parameters. While compared in terms of runtime, AFLoRA can yield up to $1.86\times$ improvement as opposed to similar PEFT alternatives. Besides the practical utility of our approach, we provide insights on the trainability requirements of LoRA paths at different modules and the freezing schedule for the different projection matrices. Code will be released.


title: Canonical Descriptors for Periodic Lattice Truss Materials

abstract: For decades, aspects of the topological architecture, and of the mechanical as well as other physical behaviors of periodic lattice truss materials (PLTMs) have been massively studied. Their approximate infinite design space presents a double-edged sword, implying on one hand dramatic designability in fulfilling the requirement of various performance, but on the other hand unexpected intractability in determining the best candidate with tailoring properties. In recent years, the development of additive manufacturing and artificial intelligence spurs an explosion in the methods exploring the design space and searching its boundaries. However, regrettably, a normative description with sufficient information of PLTMs applying to machine learning has not yet been constructed, which confines the inverse design to some discrete and small scrutinized space. In the current paper, we develop a system of canonical descriptors for PLTMs, encoding not only the geometrical configurations but also mechanical properties into matrix forms to establish good quantitative correlations between structures and mechanical behaviors. The system mainly consists of the geometry matrix for the lattice node configuration, density, stretching and bending stiffness matrices for the lattice strut properties, as well as packing matrix for the principal periodic orientation. All these matrices are theoretically derived based on the intrinsic nature of PLTMs, leading to concise descriptions and sufficient information. The characteristics, including the completeness and uniqueness, of the descriptors are analyzed. In addition, we discuss how the current system of descriptors can be applied to the database construction and material discovery, and indicate the possible open problems.