title: Generate-then-Ground in Retrieval-Augmented Generation for Multi-hop Question Answering

abstract: Multi-Hop Question Answering (MHQA) tasks present a significant challenge for large language models (LLMs) due to the intensive knowledge required. Current solutions, like Retrieval-Augmented Generation, typically retrieve potential documents from an external corpus to read an answer. However, the performance of this retrieve-then-read paradigm is constrained by the retriever and the inevitable noise in the retrieved documents. To mitigate these challenges, we introduce a novel generate-then-ground (GenGround) framework, synergizing the parametric knowledge of LLMs and external documents to solve a multi-hop question. GenGround empowers LLMs to alternate two phases until the final answer is derived: (1) formulate a simpler, single-hop question and directly generate the answer; (2) ground the question-answer pair in retrieved documents, amending any wrong predictions in the answer. We also propose an instructional grounding distillation method to generalize our method into smaller models. Extensive experiments conducted on four datasets illustrate the superiority of our method.


title: CCAT: Detector Noise Limited Performance of the RFSoC-based Readout Electronics for mm/sub-mm/far-IR KIDs

abstract: The Fred Young Submillimeter Telescope (FYST), on Cerro Chajnantor in the Atacama desert of Chile, will conduct wide-field and small deep-field surveys of the sky with more than 100,000 detectors on the Prime-Cam instrument. Kinetic inductance detectors (KIDs) were chosen as the primary sensor technology for their high density focal plane packing. Additionally, they benefit from low cost, ease of fabrication, and simplified cryogenic readout, which are all beneficial for successful deployment at scale. The cryogenic multiplexing complexity is pulled out of the cryostat and is instead pushed into the digital signal processing of the room temperature electronics. Using the Xilinx Radio Frequency System on a Chip (RFSoC), a highly multiplexed KID readout was developed for the first light Prime-Cam and commissioning Mod-Cam instruments. We report on the performance of the RFSoC-based readout with multiple detector arrays in various cryogenic setups. Specifically we demonstrate detector noise limited performance of the RFSoC-based readout under the expected optical loading conditions.


title: New Candidates for Organic-rich Regions on Ceres

abstract: We explore the spatial distribution of organics on Ceres using the visible and near-infrared data collected by the Dawn mission. We employ a spectral mixture analysis (SMA) approach to map organic materials within the Ernutet crater at the highest available spatial resolution revealing a discontinuous, granular distribution and a possible correlation with an ancient crater on which Ernutet has been superimposed. The SMA technique also helps us identify 11 new areas as potential sites for organics. These regions are predominantly located within craters or along their walls, resembling the distribution pattern observed in Ernutet, which implies a possible geological link with materials exposed from beneath the surface. In one of these candidate regions situated in the Yalode quadrangle, we detected the characteristic 3.4-micron absorption band in the infrared spectrum, indicative of organics and carbonates. By combining the spatial resolution of the Framing Camera data with the spectral resolution of the Visual and Infrared Imaging Spectrometer using SMA, we investigated the distribution of the 3.4-micron band in this quadrangle. The absorption pattern correlates with the Yalode/Urvara smooth material unit, which formed after significant impacts on Ceres. The association of organic-rich materials with complex and multiple large-impact events supports for an endogenous origin for the organics on Ceres.


title: Talking the Talk Does Not Entail Walking the Walk: On the Limits of Large Language Models in Lexical Entailment Recognition

abstract: Verbs form the backbone of language, providing the structure and meaning to sentences. Yet, their intricate semantic nuances pose a longstanding challenge. Understanding verb relations through the concept of lexical entailment is crucial for comprehending sentence meanings and grasping verb dynamics. This work investigates the capabilities of eight Large Language Models in recognizing lexical entailment relations among verbs through differently devised prompting strategies and zero-/few-shot settings over verb pairs from two lexical databases, namely WordNet and HyperLex. Our findings unveil that the models can tackle the lexical entailment recognition task with moderately good performance, although at varying degree of effectiveness and under different conditions. Also, utilizing few-shot prompting can enhance the models' performance. However, perfectly solving the task arises as an unmet challenge for all examined LLMs, which raises an emergence for further research developments on this topic.


title: Pentaquark molecular states with hidden bottom and double strangeness

abstract: We investigate the meson-baryon interaction in coupled channels with the quantum numbers of the pentaquarks $P_{bss}$ and $P_{bsss}$. The interaction is derived from an extension of the local hidden gauge approach to the heavy quark sector, which has demonstrated accurate results compared to experiments involving $\Omega_{b}$, $\Xi_{b}$ states, and pentaquarks $P_{c}$ and $P_{cs}$. In our study, we identify several molecular states with small decay widths within the chosen set of coupled channels. The spin-parity ($J^P$) of these states is as follows: $J^P={\frac{1}{2}}^-$ for pseudoscalar-baryon (${\frac{1}{2}}^+$) coupled channels, $J^P={\frac{3}{2}}^-$ for pseudoscalar-baryon (${\frac{3}{2}}^+$) coupled channels, $J^P={\frac{1}{2}}^-$ and ${\frac{3}{2}}^-$ for vector-baryon (${\frac{1}{2}}^+$) coupled channels, and $J^P={\frac{1}{2}}^-$, ${\frac{3}{2}}^-$, ${\frac{5}{2}}^-$ for vector-baryon (${\frac{3}{2}}^+$) coupled channels. We search for the poles of these states and evaluate their couplings to the different channels.


title: SelfReg-UNet: Self-Regularized UNet for Medical Image Segmentation

abstract: Since its introduction, UNet has been leading a variety of medical image segmentation tasks. Although numerous follow-up studies have also been dedicated to improving the performance of standard UNet, few have conducted in-depth analyses of the underlying interest pattern of UNet in medical image segmentation. In this paper, we explore the patterns learned in a UNet and observe two important factors that potentially affect its performance: (i) irrelative feature learned caused by asymmetric supervision; (ii) feature redundancy in the feature map. To this end, we propose to balance the supervision between encoder and decoder and reduce the redundant information in the UNet. Specifically, we use the feature map that contains the most semantic information (i.e., the last layer of the decoder) to provide additional supervision to other blocks to provide additional supervision and reduce feature redundancy by leveraging feature distillation. The proposed method can be easily integrated into existing UNet architecture in a plug-and-play fashion with negligible computational cost. The experimental results suggest that the proposed method consistently improves the performance of standard UNets on four medical image segmentation datasets. The code is available at \url{https://github.com/ChongQingNoSubway/SelfReg-UNet}


title: Understanding the variability of helium abundance in the solar corona using three-fluid modeling and UV observations

abstract: The variability of helium abundance in the solar corona and the solar wind is an important signature of solar activity, solar cycle, solar wind sources, as well as coronal heating processes. Motivated by recently reported remote sensing UV imaging observations by Helium Resonance Scattering in the Corona and Heliosphere (HERSCHEL) payload sounding rocket of helium abundance in inner corona on 14-Sep-2009 near solar minimum, we present the results of the first three-dimensional three-fluid (electrons, protons, alpha particles) model of tilted coronal streamer belt and slow solar wind that illustrate the various processes leading to helium abundance differentiation and variability. We find good qualitative agreement between the three-fluid model and the coronal helium abundances variability reported from UV observations of streamers, providing insight on the effects of the physical processes, such as heating, gravitational settling and inter-species Coulomb friction in the out-flowing solar wind that produce the observed features. The study impacts our understanding of the origins of the slow solar wind.


title: Safely Learning with Private Data: A Federated Learning Framework for Large Language Model

abstract: Private data, being larger and quality-higher than public data, can greatly improve large language models (LLM). However, due to privacy concerns, this data is often dispersed in multiple silos, making its secure utilization for LLM training a challenge. Federated learning (FL) is an ideal solution for training models with distributed private data, but traditional frameworks like FedAvg are unsuitable for LLM due to their high computational demands on clients. An alternative, split learning, offloads most training parameters to the server while training embedding and output layers locally, making it more suitable for LLM. Nonetheless, it faces significant challenges in security and efficiency. Firstly, the gradients of embeddings are prone to attacks, leading to potential reverse engineering of private data. Furthermore, the server's limitation of handle only one client's training request at a time hinders parallel training, severely impacting training efficiency. In this paper, we propose a Federated Learning framework for LLM, named FL-GLM, which prevents data leakage caused by both server-side and peer-client attacks while improving training efficiency. Specifically, we first place the input block and output block on local client to prevent embedding gradient attacks from server. Secondly, we employ key-encryption during client-server communication to prevent reverse engineering attacks from peer-clients. Lastly, we employ optimization methods like client-batching or server-hierarchical, adopting different acceleration methods based on the actual computational capabilities of the server. Experimental results on NLU and generation tasks demonstrate that FL-GLM achieves comparable metrics to centralized chatGLM model, validating the effectiveness of our federated learning framework.


title: Initial imprint effect on dynamic mitigation of plasma instability

abstract: We proposed a dynamic mitigation method for plasma instabilities based on a phase control to mitigate plasma instabilities and to smooth plasma non-uniformities [e.g. Phys. Plasmas, 19(2012)024503]. In plasmas, perturbation phase would be unknown in general, and instability growth rate is discussed. However, if the perturbation is introduced by, for example, an illumination non-uniformity of an input energy driver beam, the perturbation phase would be defined by the driver illumination non-uniformity itself. When the driver axis is controlled by its axis oscillation or wobbling motion, the perturbation phase would be known and controlled. By the superimposition of the growing phase-controlled perturbations, the overall plasma instability growth is mitigated. The dynamic mitigation method is effective to mitigate growths of various plasma instabilities. At the same time, it was found that the phase of the growing perturbations mitigated would be still defined by the initial imprint. In this paper, the initial imprint effect is focused on the dynamic mitigation mechanism in plasmas. The results in this paper demonstrate that the initial imprint effect is reduced by a pulse shaping of the oscillating or wobbling perturbation.


title: Decoding Matters: Addressing Amplification Bias and Homogeneity Issue for LLM-based Recommendation

abstract: Adapting Large Language Models (LLMs) for recommendation requires careful consideration of the decoding process, given the inherent differences between generating items and natural language. Existing approaches often directly apply LLMs' original decoding methods. However, we find these methods encounter significant challenges: 1) amplification bias -- where standard length normalization inflates scores for items containing tokens with generation probabilities close to 1 (termed ghost tokens), and 2) homogeneity issue -- generating multiple similar or repetitive items for a user. To tackle these challenges, we introduce a new decoding approach named Debiasing-Diversifying Decoding (D3). D3 disables length normalization for ghost tokens to alleviate amplification bias, and it incorporates a text-free assistant model to encourage tokens less frequently generated by LLMs for counteracting recommendation homogeneity. Extensive experiments on real-world datasets demonstrate the method's effectiveness in enhancing accuracy and diversity.


title: IDentity with Locality: An ideal hash for gene sequence search

abstract: Gene sequence search is a fundamental operation in computational genomics. Due to the petabyte scale of genome archives, most gene search systems now use hashing-based data structures such as Bloom Filters (BF). The state-of-the-art systems such as Compact bit-slicing signature index (COBS) and Repeated And Merged Bloom filters (RAMBO) use BF with Random Hash (RH) functions for gene representation and identification. The standard recipe is to cast the gene search problem as a sequence of membership problems testing if each subsequent gene substring (called kmer) of Q is present in the set of kmers of the entire gene database D. We observe that RH functions, which are crucial to the memory and the computational advantage of BF, are also detrimental to the system performance of gene-search systems. While subsequent kmers being queried are likely very similar, RH, oblivious to any similarity, uniformly distributes the kmers to different parts of potentially large BF, thus triggering excessive cache misses and causing system slowdown. We propose a novel hash function called the Identity with Locality (IDL) hash family, which co-locates the keys close in input space without causing collisions. This approach ensures both cache locality and key preservation. IDL functions can be a drop-in replacement for RH functions and help improve the performance of information retrieval systems. We give a simple but practical construction of IDL function families and show that replacing the RH with IDL functions reduces cache misses by a factor of 5x, thus improving query and indexing times of SOTA methods such as COBS and RAMBO by factors up to 2x without compromising their quality. We also provide a theoretical analysis of the false positive rate of BF with IDL functions. Our hash function is the first study that bridges Locality Sensitive Hash (LSH) and RH to obtain cache efficiency.


title: Zero-one laws for events with positional symmetries

abstract: We use an information-theoretic argument due to O'Connell (2000) to prove that every sufficiently symmetric event concerning a countably infinite family of independent and identically distributed random variables is deterministic (i.e., has a probability of either 0 or 1). The i.i.d. condition can be relaxed. This result encompasses the Hewitt-Savage zero-one law and the ergodicity of the Bernoulli process, but also applies to other scenarios such as infinite random graphs and simple renormalization processes.


title: GIEBench: Towards Holistic Evaluation of Group Identity-based Empathy for Large Language Models

abstract: As large language models (LLMs) continue to develop and gain widespread application, the ability of LLMs to exhibit empathy towards diverse group identities and understand their perspectives is increasingly recognized as critical. Most existing benchmarks for empathy evaluation of LLMs focus primarily on universal human emotions, such as sadness and pain, often overlooking the context of individuals' group identities. To address this gap, we introduce GIEBench, a comprehensive benchmark that includes 11 identity dimensions, covering 97 group identities with a total of 999 single-choice questions related to specific group identities. GIEBench is designed to evaluate the empathy of LLMs when presented with specific group identities such as gender, age, occupation, and race, emphasizing their ability to respond from the standpoint of the identified group. This supports the ongoing development of empathetic LLM applications tailored to users with different identities. Our evaluation of 23 LLMs revealed that while these LLMs understand different identity standpoints, they fail to consistently exhibit equal empathy across these identities without explicit instructions to adopt those perspectives. This highlights the need for improved alignment of LLMs with diverse values to better accommodate the multifaceted nature of human identities. Our datasets are available at https://github.com/GIEBench/GIEBench.


title: Enhancing reliability in prediction intervals using point forecasters: Heteroscedastic Quantile Regression and Width-Adaptive Conformal Inference

abstract: Building prediction intervals for time series forecasting problems presents a complex challenge, particularly when relying solely on point predictors, a common scenario for practitioners in the industry. While research has primarily focused on achieving increasingly efficient valid intervals, we argue that, when evaluating a set of intervals, traditional measures alone are insufficient. There are additional crucial characteristics: the intervals must vary in length, with this variation directly linked to the difficulty of the prediction, and the coverage of the interval must remain independent of the difficulty of the prediction for practical utility. We propose the Heteroscedastic Quantile Regression (HQR) model and the Width-Adaptive Conformal Inference (WACI) method, providing theoretical coverage guarantees, to overcome those issues, respectively. The methodologies are evaluated in the context of Electricity Price Forecasting and Wind Power Forecasting, representing complex scenarios in time series forecasting. The results demonstrate that HQR and WACI not only improve or achieve typical measures of validity and efficiency but also successfully fulfil the commonly ignored mentioned characteristics.


title: An effective subgradient algorithm via Mifflin's line search for nonsmooth nonconvex multiobjective optimization

abstract: We propose a descent subgradient algorithm for unconstrained nonsmooth nonconvex multiobjective optimization problems. To find a descent direction, we present an iterative process that efficiently approximates the Goldstein subdifferential of each objective function. To this end, we develop a new variant of Mifflin's line search in which the subgradients are arbitrary and its finite convergence is proved under a semismooth assumption. To reduce the number of subgradient evaluations, we employ a backtracking line search that identifies the objectives requiring an improvement in the current approximation of the Goldstein subdifferential. Meanwhile, for the remaining objectives, new subgradients are not computed. Unlike bundle-type methods, the proposed approach can handle nonconvexity without the need for algorithmic adjustments. Moreover, the quadratic subproblems have a simple structure, and hence the method is easy to implement. We analyze the global convergence of the proposed method and prove that any accumulation point of the generated sequence satisfies a necessary Pareto optimality condition. Furthermore, our convergence analysis addresses a theoretical challenge in a recently developed subgradient method. Through numerical experiments, we observe the practical capability of the proposed method and evaluate its efficiency when applied to a diverse range of nonsmooth test problems.


title: Inference of neutron-star properties with unified crust-core equations of state for parameter estimation

abstract: Relating different global neutron-star (NS) properties, such as tidal deformability and radius, or mass and radius, requires an equation of state (EoS). Determining the NS EoS is therefore not only the science goal of a variety of observational projects, but it also enters in the analysis process; for example, to predict a NS radius from a measured tidal deformability via gravitational waves (GW) during the inspiral of a binary NS merger. To this aim, it is important to estimate the theoretical uncertainties on the EoS, one of which is the possible bias coming from an inconsistent treatment of the low-density region; that is, the use of a so called non-unified NS crust. We propose a numerical tool allowing the user to consistently match a nuclear-physics informed crust to an arbitrary high-density EoS describing the core of the star. We introduce an inversion procedure of the EoS close to saturation density that allows users to extract nuclear-matter parameters and extend the EoS to lower densities in a consistent way. For the treatment of inhomogeneous matter in the crust, a standard approach based on the compressible liquid-drop (CLD) model approach was used in our work. A Bayesian analysis using a parametric agnostic EoS representation in the high-density region is also presented in order to quantify the uncertainties induced by an inconsistent treatment of the crust. We show that the use of a fixed, realistic-but-inconsistent model for the crust causes small but avoidable errors in the estimation of global NS properties and leads to an underestimation of the uncertainties in the inference of NS properties. Our results highlight the importance of employing a consistent EoS in inference schemes. The numerical tool that we developed to reconstruct such a thermodynamically consistent EoS, CUTER, has been tested and validated for use by the astrophysical community.


title: Maximum Flow is Fair: A Network Flow Approach to Committee Voting

abstract: In the committee voting setting, a subset of $k$ alternatives is selected based on the preferences of voters. In this paper, our goal is to efficiently compute ex-ante fair probability distributions (or lotteries) over committees. Since it is not known whether a lottery satisfying the desirable fairness property of fractional core is polynomial-time computable, we introduce a new axiom called group resource proportionality (GRP), which strengthens other fairness notions in the literature. We characterize our fairness axiom by a correspondence with max flows on a network formulation of committee voting. Using the connection to flow networks revealed by this characterization, we then introduce voting rules which achieve fairness in conjunction with other desirable properties. The redistributive utilitarian rule satisfies ex-ante efficiency in addition to our fairness axiom. We also give a voting rule which maximizes social welfare subject to fairness by reducing to a minimum-cost maximum-flow problem. Lastly, we show our fairness property can be obtained in tandem with strong ex-post fairness properties -- an approach known as best-of-both-worlds fairness. We strengthen existing best-or-both-worlds fairness results in committee voting and resolve an open question posed by Aziz et al. (2023). These findings follow from an auxiliary result which may prove useful in obtaining best-of-both-worlds type results in future research on committee voting.


title: Can we say a cat is a cat? Understanding the challenges in annotating physiological signal-based emotion data

abstract: Artificial Intelligence (AI) algorithms, trained on emotion data extracted from physiological signals, provide a promising approach to monitoring emotions, affect, and mental well-being. However, the field encounters challenges because there is a lack of effective methods for collecting high-quality data in everyday settings that genuinely reflect changes in emotion or affect. This paper presents a position discussion on the current technique of annotating physiological signal-based emotion data. Our discourse underscores the importance of adopting a nuanced understanding of annotation processes, paving the way for a more insightful exploration of the intricate relationship between physiological signals and human emotions.


title: MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression

abstract: Sparse attention can effectively mitigate the significant memory and throughput demands of Large Language Models (LLMs) in long contexts. Existing methods typically employ a uniform sparse attention mask, applying the same sparse pattern across different attention heads and input lengths. However, this uniform approach fails to capture the diverse attention patterns inherent in LLMs, ignoring their distinct accuracy-latency trade-offs. To address this challenge, we propose the Mixture of Attention (MoA), which automatically tailors distinct sparse attention configurations to different heads and layers. MoA constructs and navigates a search space of various attention patterns and their scaling rules relative to input sequence lengths. It profiles the model, evaluates potential configurations, and pinpoints the optimal sparse attention compression plan. MoA adapts to varying input sizes, revealing that some attention heads expand their focus to accommodate longer sequences, while other heads consistently concentrate on fixed-length local contexts. Experiments show that MoA increases the effective context length by $3.9\times$ with the same average attention span, boosting retrieval accuracy by $1.5-7.1\times$ over the uniform-attention baseline across Vicuna-7B, Vicuna-13B, and Llama3-8B models. Moreover, MoA narrows the capability gaps between sparse and dense models, reducing the maximum relative performance drop from $9\%-36\%$ to within $5\%$ across two long-context understanding benchmarks. MoA achieves a $1.2-1.4\times$ GPU memory reduction and boosts decode throughput by $5.5-6.7 \times$ for 7B and 13B dense models on a single GPU, with minimal impact on performance.


title: Towards Dynamic Resource Allocation and Client Scheduling in Hierarchical Federated Learning: A Two-Phase Deep Reinforcement Learning Approach

abstract: Federated learning (FL) is a viable technique to train a shared machine learning model without sharing data. Hierarchical FL (HFL) system has yet to be studied regrading its multiple levels of energy, computation, communication, and client scheduling, especially when it comes to clients relying on energy harvesting to power their operations. This paper presents a new two-phase deep deterministic policy gradient (DDPG) framework, referred to as ``TP-DDPG'', to balance online the learning delay and model accuracy of an FL process in an energy harvesting-powered HFL system. The key idea is that we divide optimization decisions into two groups, and employ DDPG to learn one group in the first phase, while interpreting the other group as part of the environment to provide rewards for training the DDPG in the second phase. Specifically, the DDPG learns the selection of participating clients, and their CPU configurations and the transmission powers. A new straggler-aware client association and bandwidth allocation (SCABA) algorithm efficiently optimizes the other decisions and evaluates the reward for the DDPG. Experiments demonstrate that with substantially reduced number of learnable parameters, the TP-DDPG can quickly converge to effective polices that can shorten the training time of HFL by 39.4% compared to its benchmarks, when the required test accuracy of HFL is 0.9.


title: Computational Model for Parsing Expression Grammars

abstract: We present a computational model for Parsing Expression Grammars (PEGs). The predecessor of PEGs top-down parsing languages (TDPLs) were discovered by A. Birman and J. Ullman in the 1960-s, B. Ford showed in 2004 that both formalisms recognize the same class named Parsing Expression Languages (PELs). A. Birman and J. Ullman established such important properties like TDPLs generate any DCFL and some non-context-free languages like $a^nb^nc^n$, a linear-time parsing algorithm was constructed as well. But since this parsing algorithm was impractical in the 60-s TDPLs were abandoned and then upgraded by B. Ford to PEGs, so the parsing algorithm was improved (from the practical point of view) as well. Now PEGs are actively used in compilers (eg., Python replaced LL(1)-parser with a PEG one) so as for text processing as well. In this paper, we present a computational model for PEG, obtain structural properties of PELs, namely proof that PELs are closed over left concatenation with Boolean closure of regular closure of DCFLs, and present an extension of the PELs class based on the extension of our computational model. Our model is an upgrade of deterministic pushdown automata (DPDA) such that during the pop of a symbol it is allowed to return the head to the position of the push of the symbol. We provide a linear-time simulation algorithm for the 2-way version of this model, which is similar to the S. Cook famous linear-time simulation algorithm of 2-way DPDA.


title: FC3DNet: A Fully Connected Encoder-Decoder for Efficient Demoir'eing

abstract: Moir\'e patterns are commonly seen when taking photos of screens. Camera devices usually have limited hardware performance but take high-resolution photos. However, users are sensitive to the photo processing time, which presents a hardly considered challenge of efficiency for demoir\'eing methods. To balance the network speed and quality of results, we propose a \textbf{F}ully \textbf{C}onnected en\textbf{C}oder-de\textbf{C}oder based \textbf{D}emoir\'eing \textbf{Net}work (FC3DNet). FC3DNet utilizes features with multiple scales in each stage of the decoder for comprehensive information, which contains long-range patterns as well as various local moir\'e styles that both are crucial aspects in demoir\'eing. Besides, to make full use of multiple features, we design a Multi-Feature Multi-Attention Fusion (MFMAF) module to weigh the importance of each feature and compress them for efficiency. These designs enable our network to achieve performance comparable to state-of-the-art (SOTA) methods in real-world datasets while utilizing only a fraction of parameters, FLOPs, and runtime.


title: Cooperative bots exhibit nuanced effects on cooperation across strategic frameworks

abstract: The positive impact of cooperative bots on cooperation within evolutionary game theory is well documented; however, existing studies have predominantly used discrete strategic frameworks, focusing on deterministic actions with a fixed probability of one. This paper extends the investigation to continuous and mixed strategic approaches. Continuous strategies employ intermediate probabilities to convey varying degrees of cooperation and focus on expected payoffs. In contrast, mixed strategies calculate immediate payoffs from actions chosen at a given moment within these probabilities. Using the prisoner's dilemma game, this study examines the effects of cooperative bots on human cooperation within hybrid populations of human players and simple bots, across both well-mixed and structured populations. Our findings reveal that cooperative bots significantly enhance cooperation in both population types across these strategic approaches under weak imitation scenarios, where players are less concerned with material gains. However, under strong imitation scenarios, while cooperative bots do not alter the defective equilibrium in well-mixed populations, they have varied impacts in structured populations across these strategic approaches. Specifically, they disrupt cooperation under discrete and continuous strategies but facilitate it under mixed strategies. These results highlight the nuanced effects of cooperative bots within different strategic frameworks and underscore the need for careful deployment, as their effectiveness is highly sensitive to how humans update their actions and their chosen strategic approach.


title: Random walk in slowly changing environments

abstract: A Random Walk in Changing Environment (RWCE) is a weighted random walk on a locally finite, connected graph $G$ with random, time-dependent edge-weights. This includes self-interacting random walks, where the edge-weights depend on the history of the process. In general, even the basic question of recurrence or transience for RWCEs is difficult, especially when the underlying graph contains cycles. In this note, we derive a condition for recurrence or transience that is too restrictive for classical RWCEs but instead works for any graph $G.$ Namely, we show that any bounded RWCE on $G$ with "slowly" changing edge-weights inherits the recurrence or transience of the initial weighted graph.


title: Polarimetry of $\gamma$-Rays Converting to $e^+e^-$ Pairs with Silicon-Pixel-Based Telescopes

abstract: There are serious prospects that the next $\gamma$-ray space mission could use a telescope based on silicon pixel detectors. I characterize the potential of such active targets for polarimetry with gamma-ray conversions to pairs and find it excellent both in terms of selection efficiency and of effective polarization asymmetry.


title: Demonstrating the Efficacy of Kolmogorov-Arnold Networks in Vision Tasks

abstract: In the realm of deep learning, the Kolmogorov-Arnold Network (KAN) has emerged as a potential alternative to multilayer projections (MLPs). However, its applicability to vision tasks has not been extensively validated. In our study, we demonstrated the effectiveness of KAN for vision tasks through multiple trials on the MNIST, CIFAR10, and CIFAR100 datasets, using a training batch size of 32. Our results showed that while KAN outperformed the original MLP-Mixer on CIFAR10 and CIFAR100, it performed slightly worse than the state-of-the-art ResNet-18. These findings suggest that KAN holds significant promise for vision tasks, and further modifications could enhance its performance in future evaluations.Our contributions are threefold: first, we showcase the efficiency of KAN-based algorithms for visual tasks; second, we provide extensive empirical assessments across various vision benchmarks, comparing KAN's performance with MLP-Mixer, CNNs, and Vision Transformers (ViT); and third, we pioneer the use of natural KAN layers in visual tasks, addressing a gap in previous research. This paper lays the foundation for future studies on KANs, highlighting their potential as a reliable alternative for image classification tasks.


title: LLM2FEA: Discover Novel Designs with Generative Evolutionary Multitasking

abstract: The rapid research and development of generative artificial intelligence has enabled the generation of high-quality images, text, and 3D models from text prompts. This advancement impels an inquiry into whether these models can be leveraged to create digital artifacts for both creative and engineering applications. Drawing on innovative designs from other domains may be one answer to this question, much like the historical practice of ``bionics", where humans have sought inspiration from nature's exemplary designs. This raises the intriguing possibility of using generative models to simultaneously tackle design tasks across multiple domains, facilitating cross-domain learning and resulting in a series of innovative design solutions. In this paper, we propose LLM2FEA as the first attempt to discover novel designs in generative models by transferring knowledge across multiple domains. By utilizing a multi-factorial evolutionary algorithm (MFEA) to drive a large language model, LLM2FEA integrates knowledge from various fields to generate prompts that guide the generative model in discovering novel and practical objects. Experimental results in the context of 3D aerodynamic design verify the discovery capabilities of the proposed LLM2FEA. The designs generated by LLM2FEA not only satisfy practicality requirements to a certain degree but also feature novel and aesthetically pleasing shapes, demonstrating the potential applications of LLM2FEA in discovery tasks.


title: Genus non-increasing totally positive unknotting number

abstract: The genus non-increasing totally positive unknotting number is the minimum number of crossing changes that transform a knot into the unknot, such that all the crossing changes are positive-to-negative crossing changes that do not increase the genus. We show that the genus non-increasing totally positive unknotting number can be arbitrary large for genus one knots.


title: Wasserstein convergence of \v{C}ech persistence diagrams for samplings of submanifolds

abstract: \v{C}ech Persistence diagrams (PDs) are topological descriptors routinely used to capture the geometry of complex datasets. They are commonly compared using the Wasserstein distances $OT_{p}$; however, the extent to which PDs are stable with respect to these metrics remains poorly understood. We partially close this gap by focusing on the case where datasets are sampled on an $m$-dimensional submanifold of $\mathbb{R}^{d}$. Under this manifold hypothesis, we show that convergence with respect to the $OT_{p}$ metric happens exactly when $p\gt m$. We also provide improvements upon the bottleneck stability theorem in this case and prove new laws of large numbers for the total $\alpha$-persistence of PDs. Finally, we show how these theoretical findings shed new light on the behavior of the feature maps on the space of PDs that are used in ML-oriented applications of Topological Data Analysis.


title: Extending GPU-Accelerated Gaussian Integrals in the TeraChem Software Package to f Type Orbitals: Implementation and Applications

abstract: The increasing availability of GPUs for scientific computing has prompted interest in accelerating quantum chemical calculations through their use. The complexity of integral kernels for high angular momentum basis functions however often limits the utility of GPU implementations with large basis sets or for metal containing systems. In this work, we report implementation of $f$ function support in the GPU-accelerated TeraChem software package through the development of efficient kernels for the evaluation of Hamiltonian integrals. The resulting code is very efficient, and we demonstrate this with density functional theory calculations on increasingly large alkanes as well as coupled cluster singles and doubles calculations on water clusters. Preliminary investigations into Ni(I) catalysis and the photochemistry of MnH(CH$_3$) are also carried out. Overall, our GPU-accelerated software appears to be well-suited for fast simulation of large transition metal containing systems.


title: Quadratic spectral concentration of characteristic functions

abstract: It is known that the inequality \begin{align*}\int_{-W/2}^{W/2}|\widehat{f}(\xi)|^2d\xi\leq \int_{-W/2}^{W/2}|\widehat{|f|^*}(\xi)|^2d\xi \end{align*} between the quadratic spectral concentration of a function and that of its decreasing rearrangement holds for any function $f\in L^2,\;|\text{supp} f|=T,$ if and only if the product $WT$ does not exceed the critical value $\approx 0.884$. We show that by restricting ourselves to characteristic functions we can enlarge this range up to $WT\leq 4/3$. Besides, we establish various properties of minimizers of the difference $\int_{-W/2}^{W/2}|\widehat{\chi_A^*}(\xi)|^2d\xi-\int_{-W/2}^{W/2}|\widehat{\chi_A}(\xi)|^2d\xi$ over sets $A$ of finite measure and prove that this difference is non-negative for all $W,T>0$ if $A$ is the union of two intervals. As a corollary, we obtain a sharp (up to a constant) estimate for the $L_2$-norms of non-harmonic trigonometric polynomials with alternating coefficients $\pm 1$.


title: Social learning with complex contagion

abstract: We introduce a mathematical model that combines the concepts of complex contagion with payoff-biased imitation, to describe how social behaviors spread through a population. Traditional models of social learning by imitation are based on simple contagion -- where an individual may imitate a more successful neighbor following a single interaction. Our framework generalizes this process to incorporate complex contagion, which requires multiple exposures before an individual considers adopting a different behavior. We formulate this as a discrete time and state stochastic process in a finite population, and we derive its continuum limit as an ordinary differential equation that generalizes the replicator equation, the most widely used dynamical model in evolutionary game theory. When applied to linear frequency-dependent games, our social learning with complex contagion produces qualitatively different outcomes than traditional imitation dynamics: it can shift the Prisoner's Dilemma from a unique all-defector equilibrium to either a stable mixture of cooperators and defectors in the population, or a bistable system; it changes the Snowdrift game from a single to a bistable equilibrium; and it can alter the Coordination game from bistability at the boundaries to two internal equilibria. The long-term outcome depends on the balance between the complexity of the contagion process and the strength of selection that biases imitation towards more successful types. Our analysis intercalates the fields of evolutionary game theory with complex contagions, and it provides a synthetic framework that describes more realistic forms of behavioral change in social systems.

categories: physics.soc-ph cs.MA cs.NE nlin.AO q-bio.PE


title: Routes to a building or a room suited to the specific needs of users

abstract: OPALE is a multi-service mobile app for users of the Claude Bernard Lyon 1 University campuses, combining the discovery of practical, cultural and scientific resources with useful everyday functions. Points of interest are geolocated on a map of the campus and can be reached via an itinerary. But an itinerary to a building is not always enough to find your way around. The orientation problem often lies in the last few metres: between the entrance to the building and the room you are looking for. In this article we present OPALE, and the approach we have adopted to solve this problem of orienting users in a building. We will also show how we take into account the specific characteristics of users, particularly those with physical or cognitive disabilities.


title: DiPEx: Dispersing Prompt Expansion for Class-Agnostic Object Detection

abstract: Class-agnostic object detection (OD) can be a cornerstone or a bottleneck for many downstream vision tasks. Despite considerable advancements in bottom-up and multi-object discovery methods that leverage basic visual cues to identify salient objects, consistently achieving a high recall rate remains difficult due to the diversity of object types and their contextual complexity. In this work, we investigate using vision-language models (VLMs) to enhance object detection via a self-supervised prompt learning strategy. Our initial findings indicate that manually crafted text queries often result in undetected objects, primarily because detection confidence diminishes when the query words exhibit semantic overlap. To address this, we propose a Dispersing Prompt Expansion (DiPEx) approach. DiPEx progressively learns to expand a set of distinct, non-overlapping hyperspherical prompts to enhance recall rates, thereby improving performance in downstream tasks such as out-of-distribution OD. Specifically, DiPEx initiates the process by self-training generic parent prompts and selecting the one with the highest semantic uncertainty for further expansion. The resulting child prompts are expected to inherit semantics from their parent prompts while capturing more fine-grained semantics. We apply dispersion losses to ensure high inter-class discrepancy among child prompts while preserving semantic consistency between parent-child prompt pairs. To prevent excessive growth of the prompt sets, we utilize the maximum angular coverage (MAC) of the semantic space as a criterion for early termination. We demonstrate the effectiveness of DiPEx through extensive class-agnostic OD and OOD-OD experiments on MS-COCO and LVIS, surpassing other prompting methods by up to 20.1% in AR and achieving a 21.3% AP improvement over SAM. The code is available at https://github.com/jason-lim26/DiPEx.


title: Extraction of 3D trajectories of mandibular condyles from 2D real-time MRI

abstract: Computing the trajectories of mandibular condyles directly from MRI could provide a comprehensive examination, allowing for the extraction of both anatomical and kinematic details. This study aimed to investigate the feasibility of extracting 3D condylar trajectories from 2D real-time MRI and to assess their precision.Twenty healthy subjects underwent real-time MRI while opening and closing their jaws. One axial and two sagittal slices were segmented using a U-Net-based algorithm. The centers of mass of the resulting masks were projected onto the coordinate system based on anatomical markers and temporally adjusted using a common projection. The quality of the computed trajectories was evaluated using metrics designed to estimate movement reproducibility, head motion, and slice placement symmetry.The segmentation of the axial slices demonstrated good-to-excellent quality; however, the segmentation of the sagittal slices required some fine-tuning. The movement reproducibility was acceptable for most cases; nevertheless, head motion displaced the trajectories by 1 mm on average. The difference in the superior-inferior coordinate of the condyles in the closed jaw position was 1.7 mm on average.Despite limitations in precision, real-time MRI enables the extraction of condylar trajectories with sufficient accuracy for evaluating clinically relevant parameters such as condyle displacement, trajectories aspect, and symmetry.


title: Brane-Like Solutions and Other Non-Supersymmetric Vacua

abstract: After recasting the standard charged and uncharged brane profiles in the harmonic gauge, we explore solutions with the same isometries where the potentials $V = T \,e^{\gamma\,\phi}$ of ten-dimensional non-supersymmetric strings are taken into account. Combining a detailed catalog of the possible asymptotics with some numerical results suggests that these spherically symmetric backgrounds terminate at singularities within finite proper distances.


title: Gaussian-Informed Continuum for Physical Property Identification and Simulation

abstract: This paper studies the problem of estimating physical properties (system identification) through visual observations. To facilitate geometry-aware guidance in physical property estimation, we introduce a novel hybrid framework that leverages 3D Gaussian representation to not only capture explicit shapes but also enable the simulated continuum to deduce implicit shapes during training. We propose a new dynamic 3D Gaussian framework based on motion factorization to recover the object as 3D Gaussian point sets across different time states. Furthermore, we develop a coarse-to-fine filling strategy to generate the density fields of the object from the Gaussian reconstruction, allowing for the extraction of object continuums along with their surfaces and the integration of Gaussian attributes into these continuums. In addition to the extracted object surfaces, the Gaussian-informed continuum also enables the rendering of object masks during simulations, serving as implicit shape guidance for physical property estimation. Extensive experimental evaluations demonstrate that our pipeline achieves state-of-the-art performance across multiple benchmarks and metrics. Additionally, we illustrate the effectiveness of the proposed method through real-world demonstrations, showcasing its practical utility. Our project page is at https://jukgei.github.io/project/gic.


title: Autonomous Agents for Collaborative Task under Information Asymmetry

abstract: Large Language Model Multi-Agent Systems (LLM-MAS) have achieved great progress in solving complex tasks. It performs communication among agents within the system to collaboratively solve tasks, under the premise of shared information. However, when agents' communication is leveraged to enhance human cooperation, a new challenge arises due to information asymmetry, since each agent can only access the information of its human user. Previous MAS struggle to complete tasks under this condition. To address this, we propose a new MAS paradigm termed iAgents, which denotes Informative Multi-Agent Systems. In iAgents, the human social network is mirrored in the agent network, where agents proactively exchange human information necessary for task resolution, thereby overcoming information asymmetry. iAgents employs a novel agent reasoning mechanism, InfoNav, to navigate agents' communication towards effective information exchange. Together with InfoNav, iAgents organizes human information in a mixed memory to provide agents with accurate and comprehensive information for exchange. Additionally, we introduce InformativeBench, the first benchmark tailored for evaluating LLM agents' task-solving ability under information asymmetry. Experimental results show that iAgents can collaborate within a social network of 140 individuals and 588 relationships, autonomously communicate over 30 turns, and retrieve information from nearly 70,000 messages to complete tasks within 3 minutes.


title: Efficient Graph Similarity Computation with Alignment Regularization

abstract: We consider the graph similarity computation (GSC) task based on graph edit distance (GED) estimation. State-of-the-art methods treat GSC as a learning-based prediction task using Graph Neural Networks (GNNs). To capture fine-grained interactions between pair-wise graphs, these methods mostly contain a node-level matching module in the end-to-end learning pipeline, which causes high computational costs in both the training and inference stages. We show that the expensive node-to-node matching module is not necessary for GSC, and high-quality learning can be attained with a simple yet powerful regularization technique, which we call the Alignment Regularization (AReg). In the training stage, the AReg term imposes a node-graph correspondence constraint on the GNN encoder. In the inference stage, the graph-level representations learned by the GNN encoder are directly used to compute the similarity score without using AReg again to speed up inference. We further propose a multi-scale GED discriminator to enhance the expressive ability of the learned representations. Extensive experiments on real-world datasets demonstrate the effectiveness, efficiency and transferability of our approach.


title: An independence of the MIN principle from the PHP principle

abstract: The minimization principle MIN($\prec$) studied in bounded arithmetic in [Chiari, M. and Kraj\'i\v{c}ek, J. Witnessing Functions in Bounded Arithmetic and Search Problems, Journal of Symbolic Logic, 63(3):1095-1115, 1998] says that a strict linear ordering $\prec$ on any finite interval $[0,\dots,n)$ has the minimal element. We shall prove that bounded arithmetic theory $T^1_2(\prec)$ augmented by instances of the pigeonhole principle for all $\Delta^b_1(\prec)$ formulas does not prove MIN($\prec$).


title: Multi-beam Training for Near-field Communications in High-frequency Bands

abstract: In this paper, we study efficient multi-beam training design for near-field communications to reduce the beam training overhead of conventional single-beam training methods. In particular, the array-division based multi-beam training method, which is widely used in far-field communications, cannot be directly applied to the near-field scenario, since different sub-arrays may observe different user angles and there exist coverage holes in the angular domain. To address these issues, we first devise a new near-field multi-beam codebook by sparsely activating a portion of antennas to form a sparse linear array (SLA), hence generating multiple beams simultaneously by effective exploiting the near-field grating-lobs. Next, a two-stage near-field beam training method is proposed, for which several candidate user locations are identified firstly based on multi-beam sweeping over time, followed by the second stage to further determine the true user location with a small number of single-beam sweeping. Finally, numerical results show that our proposed multi-beam training method significantly reduces the beam training overhead of conventional single-beam training methods, yet achieving comparable rate performance in data transmission.


title: On the set of non radiative solutions for the energy critical wave equation

abstract: Non radiative solutions of the energy critical non linear wave equation are global solutions $u$ that furthermore have vanishing asymptotic energy outside the lightcone at both $t \to \pm \infty$: \[ \lim_{t \to \pm \infty} \| \nabla_{t,x} u(t) \|_{L^2(|x| \ge |t|+R)} = 0, \] for some $R \gt 0$. They were shown to play an important role in the analysis of long time dynamics of solutions, in particular regarding the soliton resolution: we refer to the seminal works of Duyckaerts, Kenig and Merle, see \cite{DKM:23} and the references therein. We show that the set of non radiative solutions which are small in the energy space is a manifold whose tangent space at $0$ is given by non radiative solutions to the linear equation (described in \cite{CL24}). We also construct nonlinear solutions with an arbitrary prescribed radiation field.


title: ASTERIX: Module for modelling the water flow on vegetated hillslopes

abstract: The paper presents ASTERIX, an open source software for numerical integration of an extended Saint-Venant system of equations used as a mathematical tool to model the water flow from laboratory up to large-scale spatial domains applying physically-based principles of fluid mechanics. Many in-situ observations have shown that the plant cover plays a key roll in controlling the hydrological flux at a catchment scale. The plant roots facilitate the infiltration processes, the canopy intercept some proportion of rain, and plant stems slow down the flow. In case of heavy rains, the infiltration and interception processes cease in a short time, the remaining rainfall gives rise to the Hortonian overland flow and the flash flood is thus initiated. In this context, the following problem is also addressed in the article: how do the gradient of soil surface and the density of the plant cover influence the water dynamics in the Hortonian flow? The mathematical model and ASTERIX were kept as simple as possible in order to be accessible to a wide range of stakeholders interested in understanding the complex processes behind the water flow on hillslopes covered by plants. The software is written in C programming language and it is free under GNU license. It was tested on a series of benchmark problems, laboratory experiments, and theoretical problems; and the results have shown a good agreement with the theoretical or measured data.


title: Learning Autonomous Race Driving with Action Mapping Reinforcement Learning

abstract: Autonomous race driving poses a complex control challenge as vehicles must be operated at the edge of their handling limits to reduce lap times while respecting physical and safety constraints. This paper presents a novel reinforcement learning (RL)-based approach, incorporating the action mapping (AM) mechanism to manage state-dependent input constraints arising from limited tire-road friction. A numerical approximation method is proposed to implement AM, addressing the complex dynamics associated with the friction constraints. The AM mechanism also allows the learned driving policy to be generalized to different friction conditions. Experimental results in our developed race simulator demonstrate that the proposed AM-RL approach achieves superior lap times and better success rates compared to the conventional RL-based approaches. The generalization capability of driving policy with AM is also validated in the experiments.


title: Modelling Legislative Systems into Property Graphs to Enable Advanced Pattern Detection

abstract: Legislative systems face growing complexity due to the ever-increasing number of laws and intricate interdependencies between them. Traditional methods of storing and analyzing legal systems, mainly based on RDF, struggle with this complexity, hindering efficient knowledge discovery, as required by domain experts. In this paper, we propose to model legislation into a property graph, where edges represent citations, modifications, and abrogations between laws and their articles or attachments, both represented as nodes and edges with properties. As a practical use case, we implement the model in the Italian legislative system. First, we describe our approach to extracting knowledge from legal texts. To this aim, we leverage the recently internationally adopted XML law standard, Akoma Ntoso, to parse and identify entities, relationships and properties. Next, we describe the model and the schema implemented using Neo4j, the market-leading graph database management system. The schema is designed to capture the structure and hierarchy of laws, together with their interdependencies. We show how such a property graph enables an efficient answer to complex and relevant queries previously impractical on raw text. By leveraging other implementations of the Akoma Ntoso standard and the proposed property graph approach, we are confident that this work will facilitate a comprehensive comparison of legislative systems and their complexities.


title: On the growth of the parameters of approximating ReLU neural networks

abstract: This work focuses on the analysis of fully connected feed forward ReLU neural networks as they approximate a given, smooth function. In contrast to conventionally studied universal approximation properties under increasing architectures, e.g., in terms of width or depth of the networks, we are concerned with the asymptotic growth of the parameters of approximating networks. Such results are of interest, e.g., for error analysis or consistency results for neural network training. The main result of our work is that, for a ReLU architecture with state of the art approximation error, the realizing parameters grow at most polynomially. The obtained rate with respect to a normalized network size is compared to existing results and is shown to be superior in most cases, in particular for high dimensional input.


title: O -Acetylated sugars in the gas phase: stability, migration, positional isomers and conformation

abstract: O-acetylations are functional modifications which can be found on different hydroxyl groups of glycans and which contribute to the fine tuning of their biological activity. Localizing the acetyl modifications is notoriously challenging in glycoanalysis, in particular because of their mobility: loss or migration of the acetyl group may occur through the analytical workflow. Whereas migration conditions in the condensed phase have been rationalized, little is known about the suitability of Mass Spectrometry to retain and resolve the structure of O-acetylated glycan isomers. Here we use the resolving power of infrared ion spectroscopy in combination with ab initio calculation to assess the structure of O-acetylated monosaccharide ions in the gaseous environment of a mass analyzer. N-acetyl glucosamines were synthetized with an O-acetyl group in positions 3 or 6, respectively. The protonated ions produced by electrospray ionization were observed by mass spectrometry and their vibrational fingerprints were recorded in the 3 $\mu$m range by IRMPD spectroscopy (InfraRed Multiple Photon Dissociation). Experimentally, the isomers show distinctive IR fingerprints. Additionally, ab initio calculations confirm the position of the O-acetylation and resolve their gas phase conformation. These findings demonstrate that the position of O-acetyl groups is retained through the transfer from solution to the gas phase, and can be identified by IRMPD spectroscopy.


title: Towards Retrieval Augmented Generation over Large Video Libraries

abstract: Video content creators need efficient tools to repurpose content, a task that often requires complex manual or automated searches. Crafting a new video from large video libraries remains a challenge. In this paper we introduce the task of Video Library Question Answering (VLQA) through an interoperable architecture that applies Retrieval Augmented Generation (RAG) to video libraries. We propose a system that uses large language models (LLMs) to generate search queries, retrieving relevant video moments indexed by speech and visual metadata. An answer generation module then integrates user queries with this metadata to produce responses with specific video timestamps. This approach shows promise in multimedia content retrieval, and AI-assisted video content creation.


title: RIS-aided MIMO Beamforming: Piece-Wise Near-field Channel Model

abstract: This paper proposes a joint active and passive beamforming design for reconfigurable intelligent surface (RIS)-aided wireless communication systems, adopting a piece-wise near-field channel model. While a traditional near-field channel model, applied without any approximations, offers higher modeling accuracy than a far-field model, it renders the system design more sensitive to channel estimation errors (CEEs). As a remedy, we propose to adopt a piece-wise near-field channel model that leverages the advantages of the near-field approach while enhancing its robustness against CEEs. Our study analyzes the impact of different channel models, including the traditional near-field, the proposed piece-wise near-field and far-field channel models, on the interference distribution caused by CEEs and model mismatches. Subsequently, by treating the interference as noise, we formulate a joint active and passive beamforming design problem to maximize the spectral efficiency (SE). The formulated problem is then recast as a mean squared error (MSE) minimization problem and a suboptimal algorithm is developed to iteratively update the active and passive beamforming strategies. Simulation results demonstrate that adopting the piece-wise near-field channel model leads to an improved SE compared to both the near-field and far-field models in the presence of CEEs. Furthermore, the proposed piece-wise near-field model achieves a good trade-off between modeling accuracy and system's degrees of freedom (DoF).


title: Rapid IRMPD analysis for glycomics

abstract: Infrared vibrational spectroscopy in the gas phase has emerged as a powerful tool to determine complex molecular structures with Angstrom accuracy. Among the different approaches IRMPD (InfraRed Multiple Photon Dissociation), which requires the use of an intense pulsed tunable laser in the IR domain, has been broadly applied to the study of complex (bio)molecules. Recently, it also emerged as a highly relevant approach for analytical purposes especially in the field of glycomics in which structural analysis is still a tremendous challenge. This opens the perspective to develop new analytical tools allowing for the determination of molecular structures with atomic precision, and to address advanced questions in the field. However, IRMPD experiments require either non commercial equipments and long acquisition time which limits the data output. Here we show that it is possible to improve the IRMPD performances by optimizing the combination between a LTQ XL mass spectrometer and a high repetition tunable laser Firefly. Two orders of magnitude are gained with this approach compared to usual experiments ultimately leading to a completely resolved spectrum acquired in less than one minute. These results open the way to many new applications in glycomics with the possibility to include IRMPD in complex analytical workflows.


title: Brightearth roads: Towards fully automatic road network extraction from satellite imagery

abstract: The modern road network topology comprises intricately designed structures that introduce complexity when automatically reconstructing road networks. While open resources like OpenStreetMap (OSM) offer road networks with well-defined topology, they may not always be up to date worldwide. In this paper, we propose a fully automated pipeline for extracting road networks from very-high-resolution (VHR) satellite imagery. Our approach directly generates road line-strings that are seamlessly connected and precisely positioned. The process involves three key modules: a CNN-based neural network for road segmentation, a graph optimization algorithm to convert road predictions into vector line-strings, and a machine learning model for classifying road materials. Compared to OSM data, our results demonstrate significant potential for providing the latest road layouts and precise positions of road segments.


title: Computations regarding the torsion homology of Oeljeklaus-Toma manifolds

abstract: This article investigates the torsion homology behaviour in towers of Oeljeklaus-Toma (OT) manifolds. This adapts an idea of Silver and Williams from knot theory to OT-manifolds and extends it to higher degree homology groups.In the case of surfaces, i.e. Inoue surfaces of type $S^{0}$, the torsion grows exponentially in both $H_1$ and $H_2$ according to a parameters which already plays a role in Inoue's classical paper. This motivates running example calculations in all homological degrees.


title: Global solutions and uniform convergence stability for compressible Navier-Stokes equations with Oldroyd-type constitutive law

abstract: We consider one dimensional isentropic compressible Navier-Stokes equations with Oldroyd-type constitutive law. By establishing uniform a priori estimates (with respect to re-laxation time), we show global existence of smooth solutions with small initial data. Moreover,we get global-in-time convergence of the system towards the classical isentropic compressible Navier-Stokes equations.


title: A q-analogue of delta-matroids and related concepts

abstract: We define and study q-delta-matroids, and q-g-matroids. These objects are analogues, for finite-dimensional vector spaces over finite fields, of delta-matroids and g-matroids arising from finite sets. We compare axiomatic descriptions with definitions by means of strong maps of q-matroids.


title: Complex Lagrangian minimal surfaces, bi-complex Higgs bundles and $\mathrm{SL}(3,\mathbb{C})$-quasi-Fuchsian representations

abstract: In this paper we introduce complex minimal Lagrangian surfaces in the bi-complex hyperbolic space and study their relation with representations in $\mathrm{SL}(3,\mathbb{C})$. Our theory generalizes at the same time minimal Lagrangian surfaces in the complex hyperbolic plane, hyperbolic affine spheres in $\mathbb{R}^3$, and Bers embeddings in the holomorphic space form $\mathbb{CP}^1 \times \mathbb{CP}^1 \setminus \Delta$. If these surfaces are equivariant under representations in $\mathrm{SL}(3,\mathbb{C})$, our approach generalizes the study of almost $\mathbb{R}$-Fuchsian representations in $\mathrm{SU}(2,1)$, Hitchin representations in $\mathrm{SL}(3,\mathbb{R})$, and quasi-Fuchsian representations in $\mathrm{SL}(2,\mathbb{C})$. Moreover, we give a parameterization of $\mathrm{SL}(3,\mathbb{C})$-quasi-Fuchsian representations by an open set in the product of two copies of the bundle of holomorphic cubic differentials over the Teichm\"uller space of $S$, from which we deduce that this space of representations is endowed with a bi-complex structure. In the process, we introduce bi-complex Higgs bundles as a new tool for studying representations into semisimple complex Lie groups.


title: Probing topological phase transitions in Aubry-Andre-Harper model via high-harmonic generation

abstract: We study the high-harmonic generation in the Aubry-Andre-Harper (AAH) model. The modulating phase of the AAH model is used as a control parameter while preserving the chiral symmetry hosting the zero-energy edge states. The harmonic yield in a particular energy range exhibits a strong dependence on the control parameter with clear separation of the region of topologically trivial and nontrivial phases of the system. The threshold for the harmonic yield is found to serve as an all-optical tool for detecting topological phases. We extended our study with broken chiral symmetry by including the onsite potential. The introduction of the onsite potential lifts the degeneracy in the edge states, which affects the harmonic enhancement. Furthermore, it is also observed that the system's onsite strength can control the HHG yield.


title: LiCS: Navigation using Learned-imitation on Cluttered Space

abstract: In this letter, we propose a robust and fast navigation system in a narrow indoor environment for UGV (Unmanned Ground Vehicle) using 2D LiDAR and odometry. We used behavior cloning with Transformer neural network to learn the optimization-based baseline algorithm. We inject Gaussian noise during expert demonstration to increase the robustness of learned policy. We evaluate the performance of LiCS using both simulation and hardware experiments. It outperforms all other baselines in terms of navigation performance and can maintain its robust performance even on highly cluttered environments. During the hardware experiments, LiCS can maintain safe navigation at maximum speed of $1.5\ m/s$.


title: Identifying impurities in a silicon substrate by using a superconducting flux qubit

abstract: A bismuth-doped silicon substrate was analyzed by using a magnetometer based on a superconducting flux qubit. The temperature dependence of the magnetization indicates that the silicon substrate contains at least two signal sources, intentionally doped bismuth spins and a spin 1/2 system with a ratio of 0.873 to 0.127. In combination with a conventional electron spin resonance spectrometer, a candidate origin of the spin 1/2 system was identified as a dangling bond on the silicon surface. In addition, the spin sensitivity of the magnetometer was also estimated to be 12 spins/$\sqrt{\mathrm{Hz}}$ by using optimized dispersive readout.


title: CEASEFIRE: An AI-powered system for combatting illicit firearms trafficking

abstract: Modern technologies have led illicit firearms trafficking to partially merge with cybercrime, while simultaneously permitting its off-line aspects to become more sophisticated. Law enforcement officers face difficult challenges that require hi-tech solutions. This article presents a real-world system, powered by advanced Artificial Intelligence, for facilitating them in their everyday work.


title: Characterization of sets of finite local and non local perimeter via non local heat equation

abstract: In this paper we provide a characterization of sets of finite local and non local perimeter via a $\Gamma-$convergence result. As an application we give a proof of the isoperimetric inequality, both in the local and in the non local case.


title: An Idiosyncrasy of Time-discretization in Reinforcement Learning

abstract: Many reinforcement learning algorithms are built on an assumption that an agent interacts with an environment over fixed-duration, discrete time steps. However, physical systems are continuous in time, requiring a choice of time-discretization granularity when digitally controlling them. Furthermore, such systems do not wait for decisions to be made before advancing the environment state, necessitating the study of how the choice of discretization may affect a reinforcement learning algorithm. In this work, we consider the relationship between the definitions of the continuous-time and discrete-time returns. Specifically, we acknowledge an idiosyncrasy with naively applying a discrete-time algorithm to a discretized continuous-time environment, and note how a simple modification can better align the return definitions. This observation is of practical consideration when dealing with environments where time-discretization granularity is a choice, or situations where such granularity is inherently stochastic.


title: ESC-Eval: Evaluating Emotion Support Conversations in Large Language Models

abstract: Emotion Support Conversation (ESC) is a crucial application, which aims to reduce human stress, offer emotional guidance, and ultimately enhance human mental and physical well-being. With the advancement of Large Language Models (LLMs), many researchers have employed LLMs as the ESC models. However, the evaluation of these LLM-based ESCs remains uncertain. Inspired by the awesome development of role-playing agents, we propose an ESC Evaluation framework (ESC-Eval), which uses a role-playing agent to interact with ESC models, followed by a manual evaluation of the interactive dialogues. In detail, we first re-organize 2,801 role-playing cards from seven existing datasets to define the roles of the role-playing agent. Second, we train a specific role-playing model called ESC-Role which behaves more like a confused person than GPT-4. Third, through ESC-Role and organized role cards, we systematically conduct experiments using 14 LLMs as the ESC models, including general AI-assistant LLMs (ChatGPT) and ESC-oriented LLMs (ExTES-Llama). We conduct comprehensive human annotations on interactive multi-turn dialogues of different ESC models. The results show that ESC-oriented LLMs exhibit superior ESC abilities compared to general AI-assistant LLMs, but there is still a gap behind human performance. Moreover, to automate the scoring process for future ESC models, we developed ESC-RANK, which trained on the annotated data, achieving a scoring performance surpassing 35 points of GPT-4. Our data and code are available at https://github.com/haidequanbu/ESC-Eval.


title: Deep Imbalanced Regression to Estimate Vascular Age from PPG Data: a Novel Digital Biomarker for Cardiovascular Health

abstract: Photoplethysmography (PPG) is emerging as a crucial tool for monitoring human hemodynamics, with recent studies highlighting its potential in assessing vascular aging through deep learning. However, real-world age distributions are often imbalanced, posing significant challenges for deep learning models. In this paper, we introduce a novel, simple, and effective loss function named the Dist Loss to address deep imbalanced regression tasks. We trained a one-dimensional convolutional neural network (Net1D) incorporating the Dist Loss on the extensive UK Biobank dataset (n=502,389) to estimate vascular age from PPG signals and validate its efficacy in characterizing cardiovascular health. The model's performance was validated on a 40% held-out test set, achieving state-of-the-art results, especially in regions with small sample sizes. Furthermore, we divided the population into three subgroups based on the difference between predicted vascular age and chronological age: less than -10 years, between -10 and 10 years, and greater than 10 years. We analyzed the relationship between predicted vascular age and several cardiovascular events over a follow-up period of up to 10 years, including death, coronary heart disease, and heart failure. Our results indicate that the predicted vascular age has significant potential to reflect an individual's cardiovascular health status. Our code will be available at https://github.com/Ngk03/AI-vascular-age.


title: A Unified Framework for Synthesizing Multisequence Brain MRI via Hybrid Fusion

abstract: Multisequence Magnetic Resonance Imaging (MRI) provides a reliable diagnosis in clinical applications through complementary information within sequences. However, in practice, the absence of certain MR sequences is a common problem that can lead to inconsistent analysis results. In this work, we propose a novel unified framework for synthesizing multisequence MR images, called Hybrid Fusion GAN (HF-GAN). We introduce a hybrid fusion encoder designed to ensure the disentangled extraction of complementary and modality-specific information, along with a channel attention-based feature fusion module that integrates the features into a common latent space handling the complexity from combinations of accessible MR sequences. Common feature representations are transformed into a target latent space via the modality infuser to synthesize missing MR sequences. We have performed experiments on multisequence brain MRI datasets from healthy individuals and patients diagnosed with brain tumors. Experimental results show that our method outperforms state-of-the-art methods in both quantitative and qualitative comparisons. In addition, a detailed analysis of our framework demonstrates the superiority of our designed modules and their effectiveness for use in data imputation tasks.


title: ICLEval: Evaluating In-Context Learning Ability of Large Language Models

abstract: In-Context Learning (ICL) is a critical capability of Large Language Models (LLMs) as it empowers them to comprehend and reason across interconnected inputs. Evaluating the ICL ability of LLMs can enhance their utilization and deepen our understanding of how this ability is acquired at the training stage. However, existing evaluation frameworks primarily focus on language abilities and knowledge, often overlooking the assessment of ICL ability. In this work, we introduce the ICLEval benchmark to evaluate the ICL abilities of LLMs, which encompasses two key sub-abilities: exact copying and rule learning. Through the ICLEval benchmark, we demonstrate that ICL ability is universally present in different LLMs, and model size is not the sole determinant of ICL efficacy. Surprisingly, we observe that ICL abilities, particularly copying, develop early in the pretraining process and stabilize afterward. Our source codes and benchmark are released at https://github.com/yiye3/ICLEval.


title: Unlocking the Global Synergies in Low-Rank Adapters

abstract: Low-rank Adaption (LoRA) has been the de-facto parameter-efficient fine-tuning technique for large language models. We present HeteroLoRA, a light-weight search algorithm that leverages zero-cost proxies to allocate the limited LoRA trainable parameters across the model for better fine-tuned performance. In addition to the allocation for the standard LoRA-adapted models, we also demonstrate the efficacy of HeteroLoRA by performing the allocation in a more challenging search space that includes LoRA modules and LoRA-adapted shortcut connections. Experiments show that HeteroLoRA enables improvements in model performance given the same parameter budge. For example, on MRPC, we see an improvement of 1.6% in accuracy with similar training parameter budget. We will open-source our algorithm once the paper is accepted.


title: Plunging region emission in the X-ray binary MAXI J0637$-$430

abstract: On the second of November 2019 the black hole X-ray binary MAXI J0637$-$430 went into outburst, at the start of which it was observed in a thermal ``disc-dominated'' state. High photon energy (extending above 10 keV) observations taken by the NuSTAR telescope reveal that this thermal spectrum can not be fit by conventional two-component (disc plus corona) approaches which ignore disc emission sourced from within the plunging region of the black hole's spacetime. Instead, these models require a third ``additional'' thermal component to reproduce the data. Using new disc solutions which extend classical models into the plunging region we show that this ``additional'' thermal emission can be explained self-consistently with photons emitted from the accretion flow at radii within the innermost stable circular orbit of the black hole. This represents the second low mass X-ray binary, after MAXI J1820+070, with a detection of plunging region emission, suggesting that signatures of this highly relativistic region may well be widespread but not previously widely appreciated. To allow for a detection of the plunging region, the black hole in MAXI J0637$-$430 must be at most moderately spinning, and we constrain the spin to be $a_\bullet < 0.86$ at 99.9$\%$ confidence. We finish by discussing the observational requirements for the robust detection of this region.


title: Skip and Skip: Segmenting Medical Images with Prompts

abstract: Most medical image lesion segmentation methods rely on hand-crafted accurate annotations of the original image for supervised learning. Recently, a series of weakly supervised or unsupervised methods have been proposed to reduce the dependence on pixel-level annotations. However, these methods are essentially based on pixel-level annotation, ignoring the image-level diagnostic results of the current massive medical images. In this paper, we propose a dual U-shaped two-stage framework that utilizes image-level labels to prompt the segmentation. In the first stage, we pre-train a classification network with image-level labels, which is used to obtain the hierarchical pyramid features and guide the learning of downstream branches. In the second stage, we feed the hierarchical features obtained from the classification branch into the downstream branch through short-skip and long-skip and get the lesion masks under the supervised learning of pixel-level labels. Experiments show that our framework achieves better results than networks simply using pixel-level annotations.


title: Uncovering the origin of interface stress enhancement and compressive-to-tensile stress transition in immiscible nanomultilayers

abstract: The intrinsic stress in nanomultilayers (NMLs) is typically dominated by interface stress, which is particularly high in immiscible Cu/W NMLs. Here, atomistic simulations with a chemically-accurate neural network potential reveal the role of interfacial intermixing and metastable phase formation on the interface stress levels. These results rationalize an experimentally-reported compressive- to-tensile transition as a function of NML deposition conditions and the extremely high interface stresses under some conditions.


title: On the planar Taylor-Couette system and related exterior problems

abstract: We consider the planar Taylor-Couette system for the steady motion of a viscous incompressible fluid in the region between two concentric disks, the inner one being at rest and the outer one rotating with constant angular speed. We study the uniqueness and multiplicity of solutions to the forced system in different classes. For any angular velocity we prove that the classical Taylor-Couette flow is the unique smooth solution displaying rotational symmetry. Instead, we show that infinitely many solutions arise, even for arbitrarily small angular velocities, in a larger, class of \textit{incomplete} solutions that we introduce. By prescribing the transversal flux, unique solvability of the Taylor-Couette system is recovered among rotationally invariant incomplete solutions. Finally, we study the behavior of these solutions as the radius of the outer disk goes to infinity, connecting our results with the celebrated Stokes paradox.


title: On $p$-adic Transference Theorem

abstract: Dual lattice is an important concept of Euclidean lattices. In 2024, Deng gave the definition to the concept of the dual lattice of a $p$-adic lattice from the duality theory of locally compact abelian groups. He also proved some important properties of the dual lattice of $p$-adic lattices, which can be viewed as $p$-adic analogues of the famous Minkowski's first, second theorems and transference theorems for Euclidean lattices. However, he only proved the lower bound of the transference theorems for $p$-adic lattices. The upper bound is left as an open question. In this paper, we prove the upper bound of the transference theorems for $p$-adic lattices. We then prove that the dual basis of an orthogonal basis is also an orthogonal basis with respect to the maximum norm.


title: Contextual Interaction via Primitive-based Adversarial Training For Compositional Zero-shot Learning

abstract: Compositional Zero-shot Learning (CZSL) aims to identify novel compositions via known attribute-object pairs. The primary challenge in CZSL tasks lies in the significant discrepancies introduced by the complex interaction between the visual primitives of attribute and object, consequently decreasing the classification performance towards novel compositions. Previous remarkable works primarily addressed this issue by focusing on disentangling strategy or utilizing object-based conditional probabilities to constrain the selection space of attributes. Unfortunately, few studies have explored the problem from the perspective of modeling the mechanism of visual primitive interactions. Inspired by the success of vanilla adversarial learning in Cross-Domain Few-Shot Learning, we take a step further and devise a model-agnostic and Primitive-Based Adversarial training (PBadv) method to deal with this problem. Besides, the latest studies highlight the weakness of the perception of hard compositions even under data-balanced conditions. To this end, we propose a novel over-sampling strategy with object-similarity guidance to augment target compositional training data. We performed detailed quantitative analysis and retrieval experiments on well-established datasets, such as UT-Zappos50K, MIT-States, and C-GQA, to validate the effectiveness of our proposed method, and the state-of-the-art (SOTA) performance demonstrates the superiority of our approach. The code is available at https://github.com/lisuyi/PBadv_czsl.


title: Optimised Grouped-Query Attention Mechanism for Transformers

abstract: Grouped-query attention (GQA) has been widely adopted in LLMs to mitigate the complexity of multi-head attention (MHA). To transform an MHA to a GQA, neighbour queries in MHA are evenly split into groups where each group shares the value and key layers. In this work, we propose AsymGQA, an activation-informed approach to asymmetrically grouping an MHA to a GQA for better model performance. Our AsymGQA outperforms the GQA within the same model size budget. For example, AsymGQA LLaMA-2-7B has an accuracy increase of 7.5% on MMLU compared to neighbour grouping. Our approach addresses the GQA's trade-off problem between model performance and hardware efficiency.


title: VividDreamer: Towards High-Fidelity and Efficient Text-to-3D Generation

abstract: Text-to-3D generation aims to create 3D assets from text-to-image diffusion models. However, existing methods face an inherent bottleneck in generation quality because the widely-used objectives such as Score Distillation Sampling (SDS) inappropriately omit U-Net jacobians for swift generation, leading to significant bias compared to the "true" gradient obtained by full denoising sampling. This bias brings inconsistent updating direction, resulting in implausible 3D generation e.g., color deviation, Janus problem, and semantically inconsistent details). In this work, we propose Pose-dependent Consistency Distillation Sampling (PCDS), a novel yet efficient objective for diffusion-based 3D generation tasks. Specifically, PCDS builds the pose-dependent consistency function within diffusion trajectories, allowing to approximate true gradients through minimal sampling steps (1-3). Compared to SDS, PCDS can acquire a more accurate updating direction with the same sampling time (1 sampling step), while enabling few-step (2-3) sampling to trade compute for higher generation quality. For efficient generation, we propose a coarse-to-fine optimization strategy, which first utilizes 1-step PCDS to create the basic structure of 3D objects, and then gradually increases PCDS steps to generate fine-grained details. Extensive experiments demonstrate that our approach outperforms the state-of-the-art in generation quality and training efficiency, conspicuously alleviating the implausible 3D generation issues caused by the deviated updating direction. Moreover, it can be simply applied to many 3D generative applications to yield impressive 3D assets, please see our project page: https://narcissusex.github.io/VividDreamer.


title: Energy-Aware Random Access Networks: Connection-Based versus Packet-Based

abstract: Characterizing and comparing the optimal energy efficiency in energy-aware machine-to-machine (M2M) random access networks remains a challenge due to the distributed nature of the access behavior of nodes. To address this issue, this letter focuses on the energy efficiency limits of two typical random access schemes, i.e., connection-based Aloha and packet-based Aloha, based on which we conducted a performance comparison. Specifically, by integrating limited energy constraints and network throughput, the lifetime throughput can be derived, and further optimized with a guarantee of targeted lifetime via selecting the transmission probability. Then we present a comparative study on the optimal lifetime throughput of packet-based Aloha and connection-based Aloha to characterize criteria for beneficial connection establishment.


title: AIGC-Chain: A Blockchain-Enabled Full Lifecycle Recording System for AIGC Product Copyright Management

abstract: As artificial intelligence technology becomes increasingly prevalent, Artificial Intelligence Generated Content (AIGC) is being adopted across various sectors. Although AIGC is playing an increasingly significant role in business and culture, questions surrounding its copyright have sparked widespread debate. The current legal framework for copyright and intellectual property is grounded in the concept of human authorship, but in the creation of AIGC, human creators primarily provide conceptual ideas, with AI independently responsible for the expressive elements. This disconnect creates complexity and difficulty in determining copyright ownership under existing laws. Consequently, it is imperative to reassess the intellectual contributions of all parties involved in the creation of AIGC to ensure a fair allocation of copyright ownership. To address this challenge, we introduce AIGC-Chain, a blockchain-enabled full lifecycle recording system designed to manage the copyright of AIGC products. It is engineered to meticulously document the entire lifecycle of AIGC products, providing a transparent and dependable platform for copyright management. Furthermore, we propose a copyright tracing method based on an Indistinguishable Bloom Filter, named IBFT, which enhances the efficiency of blockchain transaction queries and significantly reduces the risk of fraudulent copyright claims for AIGC products. In this way, auditors can analyze the copyright of AIGC products by reviewing all relevant information retrieved from the blockchain.


title: Magnon-mediated quantum gates for superconducting qubits

abstract: We propose a hybrid quantum system consisting of a magnetic particle inductively coupled to two superconducting transmon qubits, where qubit-qubit interactions are mediated via magnons. We show that the system can be tuned into three different regimes of effective qubit-qubit interactions, namely a transverse ($XX + YY$), a longitudinal ($ZZ$) and a non-trivial $ZX$ interaction. In addition, we show that an enhanced coupling can be achieved by employing an ellipsoidal magnet, carrying anisotropic magnetic fluctuations. We propose a scheme for realizing two-qubit gates, and simulate their performance under realistic experimental conditions. We find that iSWAP and CZ gates can be performed in this setup with an average fidelity $\gtrsim 99 \% $ , while an iCNOT gate can be applied with an average fidelity $\gtrsim 88 \%$. Our proposed hybrid circuit architecture offers an alternative platform for realizing two-qubit gates between superconducting qubits and could be employed for constructing qubit networks using magnons as mediators.


title: A blindness property of the Min-Sum decoding for the toric code

abstract: Kitaev's toric code is one of the most prominent models for fault-tolerant quantum computation, currently regarded as the leading solution for connectivity constrained quantum technologies. Significant effort has been recently devoted to improving the error correction performance of the toric code under message-passing decoding, a class of low-complexity, iterative decoding algorithms that play a central role in both theory and practice of classical low-density parity-check codes. Here, we provide a theoretical analysis of the toric code under min-sum (MS) decoding, a message-passing decoding algorithm known to solve the maximum-likelihood decoding problem in a localized manner, for codes defined by acyclic graphs. Our analysis reveals an intrinsic limitation of the toric code, which confines the propagation of local information during the message-passing process. We show that if the unsatisfied checks of an error syndrome are at distance greater or equal to 5 from each other, then the MS decoding is locally blind: the qubits in the direct neighborhood of an unsatisfied check are never aware of any other unsatisfied checks, except their direct neighbor. Moreover, we show that degeneracy is not the only cause of decoding failures for errors of weight at least 4, that is, the MS non-degenerate decoding radius is equal to 3, for any toric code of distance greater or equal to 9. Finally, complementing our theoretical analysis, we present a pre-processing method of practical relevance. The proposed method, referred to as stabiliser-blowup, has linear complexity and allows correcting all (degenerate) errors of weight up to 3, providing quadratic improvement in the logical error rate performance, as compared to MS only.


title: Uni-Mol2: Exploring Molecular Pretraining Model at Scale

abstract: In recent years, pretraining models have made significant advancements in the fields of natural language processing (NLP), computer vision (CV), and life sciences. The significant advancements in NLP and CV are predominantly driven by the expansion of model parameters and data size, a phenomenon now recognized as the scaling laws. However, research exploring scaling law in molecular pretraining models remains unexplored. In this work, we present Uni-Mol2 , an innovative molecular pretraining model that leverages a two-track transformer to effectively integrate features at the atomic level, graph level, and geometry structure level. Along with this, we systematically investigate the scaling law within molecular pretraining models, characterizing the power-law correlations between validation loss and model size, dataset size, and computational resources. Consequently, we successfully scale Uni-Mol2 to 1.1 billion parameters through pretraining on 800 million conformations, making it the largest molecular pretraining model to date. Extensive experiments show consistent improvement in the downstream tasks as the model size grows. The Uni-Mol2 with 1.1B parameters also outperforms existing methods, achieving an average 27% improvement on the QM9 and 14% on COMPAS-1D dataset.


title: Prescribed nonlinearity helps in an anisotropic Calder\'on-type problem

abstract: In this paper I consider the inverse boundary value problem for a quasilinear, anisotropic, elliptic equation of the form $\nabla\cdot(\gamma\nabla u+|\nabla u|^{p-2}\nabla u)=0$, where $\gamma$ is a smooth, matrix valued, function with a uniform lower bound. I show that boundary Dirichlet and Neumann data for this equation, in the form of a Dirichlet-to-Neumann map, determine the coefficient matrix uniquely, in dimension 3 and higher. This stands in contrast to the classical linear anisotropic Calder\'on problem where there is a known obstruction to uniqueness due to the invariance of the boundary data under transformations of the equation via any boundary fixing diffeomorphism.


title: Domain Adaptation of Llama3-70B-Instruct through Continual Pre-Training and Model Merging: A Comprehensive Evaluation

abstract: We conducted extensive experiments on domain adaptation of the Meta-Llama-3-70B-Instruct model on SEC data, exploring its performance on both general and domain-specific benchmarks. Our focus included continual pre-training (CPT) and model merging, aiming to enhance the model's domain-specific capabilities while mitigating catastrophic forgetting. Through this study, we evaluated the impact of integrating financial regulatory data into a robust language model and examined the effectiveness of our model merging techniques in preserving and improving the model's instructive abilities. The model is accessible at hugging face: https://huggingface.co/arcee-ai/Llama-3-SEC-Base, arcee-ai/Llama-3-SEC-Base. This is an intermediate checkpoint of our final model, which has seen 20B tokens so far. The full model is still in the process of training. This is a preprint technical report with thorough evaluations to understand the entire process.


title: A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems

abstract: Retrieval Augmented Generation (RAG) represents a significant advancement in artificial intelligence combining a retrieval phase with a generative phase, with the latter typically being powered by large language models (LLMs). The current common practices in RAG involve using "instructed" LLMs, which are fine-tuned with supervised training to enhance their ability to follow instructions and are aligned with human preferences using state-of-the-art techniques. Contrary to popular belief, our study demonstrates that base models outperform their instructed counterparts in RAG tasks by 20% on average under our experimental settings. This finding challenges the prevailing assumptions about the superiority of instructed LLMs in RAG applications. Further investigations reveal a more nuanced situation, questioning fundamental aspects of RAG and suggesting the need for broader discussions on the topic; or, as Fromm would have it, "Seldom is a glance at the statistics enough to understand the meaning of the figures".