title: STAR: Swarm Technology for Aerial Robotics Research

abstract: In recent years, the field of aerial robotics has witnessed significant progress, finding applications in diverse domains, including post-disaster search and rescue operations. Despite these strides, the prohibitive acquisition costs associated with deploying physical multi-UAV systems have posed challenges, impeding their widespread utilization in research endeavors. To overcome these challenges, we present STAR (Swarm Technology for Aerial Robotics Research), a framework developed explicitly to improve the accessibility of aerial swarm research experiments. Our framework introduces a swarm architecture based on the Crazyflie, a low-cost, open-source, palm-sized aerial platform, well suited for experimental swarm algorithms. To augment cost-effectiveness and mitigate the limitations of employing low-cost robots in experiments, we propose a landmark-based localization module leveraging fiducial markers. This module, also serving as a target detection module, enhances the adaptability and versatility of the framework. Additionally, collision and obstacle avoidance are implemented through velocity obstacles. The presented work strives to bridge the gap between theoretical advances and tangible implementations, thus fostering progress in the field.


title: CAVE: Controllable Authorship Verification Explanations

abstract: Authorship Verification (AV) (do two documents have the same author?) is essential for many sensitive real-life applications. AV is often used in proprietary domains that require a private, offline model, making SOTA online models like ChatGPT undesirable. Other SOTA systems use methods, e.g. Siamese Networks, that are uninterpretable, and hence cannot be trusted in high-stakes applications. In this work, we take the first step to address the above challenges with our model CAVE (Controllable Authorship Verification Explanations): CAVE generates free-text AV explanations that are controlled to be 1) structured (can be decomposed into sub-explanations with respect to relevant linguistic features), and 2) easily verified for explanation-label consistency (via intermediate labels in sub-explanations). In this work, we train a Llama-3-8B as CAVE; since there are no human-written corpora for AV explanations, we sample silver-standard explanations from GPT-4-TURBO and distill them into a pretrained Llama-3-8B. Results on three difficult AV datasets IMdB2, Blog-Auth, and FanFiction show that CAVE generates high quality explanations (as measured by automatic and human evaluation) as well as competitive task accuracies.


title: Faster Computation of Stabilizer Extent

abstract: Characterization of nonstabilizerness is fruitful due to its application in gate synthesis and classical simulation. In particular, the resource monotone called the $\textit{stabilizer extent}$ is indispensable to estimate the simulation cost using the rank-based simulators, one of the state-of-the-art simulators of Clifford+$T$ circuits. In this work, we propose fast numerical algorithms to compute the stabilizer extent. Our algorithm utilizes the Column Generation method, which iteratively updates the subset of pure stabilizer states used for calculation. This subset is selected based on the overlaps between all stabilizer states and a target state. Upon updating the subset, we make use of a newly proposed subroutine for calculating the $\textit{stabilizer fidelity}$ that (i) achieves the linear time complexity with respect to the number of stabilizer states, (ii) super-exponentially reduces the space complexity by in-place calculation, and (iii) prunes unnecessary states for the computation. As a result, our algorithm can compute the stabilizer fidelity and the stabilizer extent for Haar random pure states up to $n=9$ qubits, which naively requires a memory of 305 EiB. We further show that our algorithm runs even faster when the target state vector is real. The optimization problem size is reduced so that we can compute the case of $n=10$ qubits in 4.7 hours.


title: Computational Approaches to the Detection of Lesser-Known Rhetorical Figures: A Systematic Survey and Research Challenges

abstract: Rhetorical figures play a major role in our everyday communication as they make text more interesting, more memorable, or more persuasive. Therefore, it is important to computationally detect rhetorical figures to fully understand the meaning of a text. We provide a comprehensive overview of computational approaches to lesser-known rhetorical figures. We explore the linguistic and computational perspectives on rhetorical figures, emphasizing their significance for the domain of Natural Language Processing. We present different figures in detail, delving into datasets, definitions, rhetorical functions, and detection approaches. We identified challenges such as dataset scarcity, language limitations, and reliance on rule-based methods.


title: Decentralized and Centralized IDD Schemes for Cell-Free Networks

abstract: In this paper, we propose iterative interference cancellation schemes with access points selection (APs-Sel) for cell-free massive multiple-input multiple-output (CF-mMIMO) systems. Closed-form expressions for centralized and decentralized linear minimum mean square error (LMMSE) receive filters with APs-Sel are derived assuming imperfect channel state information (CSI). Furthermore, we develop a list-based detector based on LMMSE receive filters that exploits interference cancellation and the constellation points. A message-passing-based iterative detection and decoding (IDD) scheme that employs low-density parity-check (LDPC) codes is then developed. Moreover, log-likelihood ratio (LLR) refinement strategies based on censoring and a linear combination of local LLRs are proposed to improve the network performance. We compare the cases with centralized and decentralized processing in terms of bit error rate (BER) performance, complexity, and signaling under perfect CSI (PCSI) and imperfect CSI (ICSI) and verify the superiority of the distributed architecture with LLR refinements.


title: Unveiling Cognitive Constraints in Language Production: Extracting and Validating the Active Ego Network of Words

abstract: The "ego network of words" model captures structural properties in language production associated with cognitive constraints. While previous research focused on the layer-based structure and its semantic properties, this paper argues that an essential element, the concept of an active network, is missing. The active part of the ego network of words only includes words that are regularly used by individuals, akin to the ego networks in the social domain, where the active part includes relationships regularly nurtured by individuals and hence demanding cognitive effort. In this work, we define a methodology for extracting the active part of the ego network of words and validate it using interview transcripts and tweets. The robustness of our method to varying input data sizes and temporal stability is demonstrated. We also demonstrate that without the active network concept (and a tool for properly extracting the active network from data), the "ego network of words" model is not able to properly estimate the cognitive effort involved and it becomes vulnerable to the amount of data considered (leading to the disappearance of the layered structure in large datasets). Our results are well-aligned with prior analyses of the ego network of words, where the limitation of the data collected led automatically (and implicitly) to approximately consider the active part of the network only. Moreover, the validation on the transcripts dataset (MediaSum) highlights the generalizability of the model across diverse domains and the ingrained cognitive constraints in language usage.


title: Design criteria for zero-strain behavior in iron-based fluorides of tungsten bronze type as cathode materials for alkali ion batteries -- a first-principles study

abstract: Mechanical stresses and strains in the microstructure of cathode materials evolving during charge/discharge cycles can reduce the long-term stability of intercalation-type alkali-metal-ion batteries. In this context, crystalline compounds exhibiting zero-strain (ZS) behavior are of particular interest. Near-ZS sodiation was experimentally measured in the \textit{tetragonal tungsten bronze} (TTB) type compound Na$_x$FeF$\mathrm{_3}$. Using a first-principles method based on density functional theory, we investigate the potential of iron-based fluoride compounds with tungsten bronze (TB) structures as ZS cathode materials. Simulations were conducted to study the intercalation of the alkali metal ions Li$\mathrm{^+}$, Na$\mathrm{^+}$, and K$\mathrm{^+}$ into the TTB and two related TB structures of the cubic perovskite (PTB) and hexagonal (HTB) types. We describe compensating local volume effects that can explain the experimentally measured low volume change of Na$_x$FeF$\mathrm{_3}$. We discuss the structural and chemical prerequisites of the host lattice for ZS insertion mechanism for alkali ions in TB structures and present a qualitative descriptor to predict the local volume change, that provides a way for faster screening and discovery of novel ZS battery materials.


title: Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentation

abstract: Segmenting text into sentences plays an early and crucial role in many NLP systems. This is commonly achieved by using rule-based or statistical methods relying on lexical features such as punctuation. Although some recent works no longer exclusively rely on punctuation, we find that no prior method achieves all of (i) robustness to missing punctuation, (ii) effective adaptability to new domains, and (iii) high efficiency. We introduce a new model - Segment any Text (SaT) - to solve this problem. To enhance robustness, we propose a new pretraining scheme that ensures less reliance on punctuation. To address adaptability, we introduce an extra stage of parameter-efficient fine-tuning, establishing state-of-the-art performance in distinct domains such as verses from lyrics and legal documents. Along the way, we introduce architectural modifications that result in a threefold gain in speed over the previous state of the art and solve spurious reliance on context far in the future. Finally, we introduce a variant of our model with fine-tuning on a diverse, multilingual mixture of sentence-segmented data, acting as a drop-in replacement and enhancement for existing segmentation tools. Overall, our contributions provide a universal approach for segmenting any text. Our method outperforms all baselines - including strong LLMs - across 8 corpora spanning diverse domains and languages, especially in practically relevant situations where text is poorly formatted. Our models and code, including documentation, are available at https://huggingface.co/segment-any-text under the MIT license.


title: Multi-Robot Collaborative Localization and Planning with Inter-Ranging

abstract: Robots often use feature-based image tracking to identify their position in their surrounding environment; however, feature-based image tracking is prone to errors in low-textured and poorly lit environments. Specifically, we investigate a scenario where robots are tasked with exploring the surface of the Moon and are required to have an accurate estimate of their position to be able to correctly geotag scientific measurements. To reduce localization error, we complement traditional feature-based image tracking with ultra-wideband (UWB) distance measurements between the robots. The robots use an advanced mesh-ranging protocol that allows them to continuously share distance measurements amongst each other rather than relying on the common "anchor" and "tag" UWB architecture. We develop a decentralized multi-robot coordination algorithm that actively plans paths based on measurement line-of-sight vectors amongst all robots to minimize collective localization error. We then demonstrate the emergent behavior of the proposed multi-robot coordination algorithm both in simulation and hardware to lower a geometry-based uncertainty metric and reduce localization error.


title: Partial classification of spectrum maximizing products for pairs of $2\times2$ matrices

abstract: Experiments suggest that typical finite families of square matrices admit spectrum maximizing products (SMPs), that is, products that attain the joint spectral radius (JSR). Furthermore, those SMPs are often combinatorially "simple." In this paper, we consider pairs of real $2 \times 2$ matrices. We identify regions in the space of such pairs where SMPs are guaranteed to exist and to have a simple structure. We also identify another region where SMPs may fail to exist (in fact, this region includes all known counterexamples to the finiteness conjecture), but nevertheless a Sturmian maximizing measure exists. Though our results apply to a large chunk of the space of pairs of $2 \times 2$ matrices, including for instance all pairs of non-negative matrices, they leave out certain "wild" regions where more complicated behavior is possible.


title: A Comprehensive Review of Emerging Approaches in Machine Learning for De Novo PROTAC Design

abstract: Targeted protein degradation (TPD) is a rapidly growing field in modern drug discovery that aims to regulate the intracellular levels of proteins by harnessing the cell's innate degradation pathways to selectively target and degrade disease-related proteins. This strategy creates new opportunities for therapeutic intervention in cases where occupancy-based inhibitors have not been successful. Proteolysis-targeting chimeras (PROTACs) are at the heart of TPD strategies, which leverage the ubiquitin-proteasome system for the selective targeting and proteasomal degradation of pathogenic proteins. As the field evolves, it becomes increasingly apparent that the traditional methodologies for designing such complex molecules have limitations. This has led to the use of machine learning (ML) and generative modeling to improve and accelerate the development process. In this review, we explore the impact of ML on de novo PROTAC design $-$ an aspect of molecular design that has not been comprehensively reviewed despite its significance. We delve into the distinct characteristics of PROTAC linker design, underscoring the complexities required to create effective bifunctional molecules capable of TPD. We then examine how ML in the context of fragment-based drug design (FBDD), honed in the realm of small-molecule drug discovery, is paving the way for PROTAC linker design. Our review provides a critical evaluation of the limitations inherent in applying this method to the complex field of PROTAC development. Moreover, we review existing ML works applied to PROTAC design, highlighting pioneering efforts and, importantly, the limitations these studies face. By offering insights into the current state of PROTAC development and the integral role of ML in PROTAC design, we aim to provide valuable perspectives for researchers in their pursuit of better design strategies for this new modality.


title: Enhancement of Opto-Electro-Mechanical Entanglement through Three-Level Atoms

abstract: We address the dynamical bipartite entanglement in an opto-electro-mechanical system that involves a three-level atom. The system consists of a degenerate three-level atom, a mechanical resonator, an optical cavity, and a microwave cavity. By utilizing the linearization approximation and nonlinear quantum-Langevin equations, the dynamics of the system are analyzed, and the bipartite entanglement is evaluated using the logarithmic negativity. The research findings indicate that the entanglement between each subsystem increases with the atom injection rate, suggesting that a higher atom injection rate leads to enhanced information transmission between the subsystems. Additionally, it is observed that the correlation between subsystems increases with an increase in the coupling rate. Moreover, the study demonstrates that the correlation between each subsystem decreases as temperature rises. The results highlight the positive impact of three-level atoms on the bipartite entanglement in an opto-electro-mechanical system. Consequently, such electro-optomechanical systems can offer a framework for optomechanical information transfer.


title: Repulsive Score Distillation for Diverse Sampling of Diffusion Models

abstract: Score distillation sampling has been pivotal for integrating diffusion models into generation of complex visuals. Despite impressive results it suffers from mode collapse and lack of diversity. To cope with this challenge, we leverage the gradient flow interpretation of score distillation to propose Repulsive Score Distillation (RSD). In particular, we propose a variational framework based on repulsion of an ensemble of particles that promotes diversity. Using a variational approximation that incorporates a coupling among particles, the repulsion appears as a simple regularization that allows interaction of particles based on their relative pairwise similarity, measured e.g., via radial basis kernels. We design RSD for both unconstrained and constrained sampling scenarios. For constrained sampling we focus on inverse problems in the latent space that leads to an augmented variational formulation, that strikes a good balance between compute, quality and diversity. Our extensive experiments for text-to-image generation, and inverse problems demonstrate that RSD achieves a superior trade-off between diversity and quality compared with state-of-the-art alternatives.


title: Deterministic generation of concatenated graph codes from quantum emitters

abstract: Photon loss is the dominant noise mechanism in photonic quantum technologies. Designing fault-tolerant schemes with high tolerance to loss is thus a central challenge in scaling photonic quantum information processors. Concatenation of a fault-tolerant construction with a code able to efficiently correct loss is a promising approach to achieve this, but practical ways to implement code concatenation with photons have been lacking. We propose schemes for generating concatenated graph codes using multi-photon emission from two quantum emitters or a single quantum emitter coupled to a memory; capabilities available in several photonic platforms. We show that these schemes enable fault-tolerant fusion-based quantum computation in practical regimes with high photon loss and standard fusion gates without the need for auxiliary photons.


title: A locking-free isogeometric thin shell formulation based on higher order accurate local strain projection via approximate dual splines

abstract: We present a novel isogeometric discretization approach for the Kirchhoff-Love shell formulation based on the Hellinger-Reissner variational principle. For mitigating membrane locking, we discretize the independent strains with spline basis functions that are one degree lower than those used for the displacements. To enable computationally efficient condensation of the independent strains, we first discretize the variations of the independent strains with approximate dual splines to obtain a projection matrix that is close to a diagonal matrix. We then diagonalize this strain projection matrix via row-sum lumping. The combination of approximate dual test functions with row-sum lumping enables the direct condensation of the independent strain fields at the quadrature point level, while maintaining higher-order accuracy at optimal rates of convergence. We illustrate the numerical properties and the performance of our approach through numerical benchmarks, including a curved Euler-Bernoulli beam and the examples of the shell obstacle course.


title: Quantum and classical symmetries

abstract: We suggest a somewhat non-standard view on a set of curious, paradoxical from the standpoint of simple classical physics and everyday experience phenomena. There are the quantisation (discrete set of values) of the observables (e.g., energy, momentum, angular momentum); forbidden simultaneous measurements of the observables in the most cases (e.g., of a coordinate and momentum, of angular momentum projections on difference axis); counter-intuitive relations on the simultaneously measurable quantities (e.g., the famous expression for the square momentum $l(l+1)$ with the maximal projection $l$). These and other paradoxes are traditionally related to "purely quantum" phenomenon, i.e., having no analogue in the "classical world" ones. However, there are deep analogies between classical and "quantum" worlds, as soon as the quantum technique is applied to the classical phenomenon. We follow these analogies with the examples of relatively simple and well known models of classical physics, such as a simplified model of light transition through the media, a system of electric charges close to each other and far from the observer; the specific of motion in the Coulomb/Newtonian field. This text can be considered as a mini-course addressed to higher school and undergraduate students who are interested in basics of quantum mechanics, but are not yet ready for systematic study of standard courses. The text may be also useful to those who supervise such students.


title: Link Prediction with Untrained Message Passing Layers

abstract: Message passing neural networks (MPNNs) operate on graphs by exchanging information between neigbouring nodes. MPNNs have been successfully applied to various node-, edge-, and graph-level tasks in areas like molecular science, computer vision, natural language processing, and combinatorial optimization. However, most MPNNs require training on large amounts of labeled data, which can be costly and time-consuming. In this work, we explore the use of various untrained message passing layers in graph neural networks, i.e. variants of popular message passing architecture where we remove all trainable parameters that are used to transform node features in the message passing step. Focusing on link prediction, we find that untrained message passing layers can lead to competitive and even superior performance compared to fully trained MPNNs, especially in the presence of high-dimensional features. We provide a theoretical analysis of untrained message passing by relating the inner products of features implicitly produced by untrained message passing layers to path-based topological node similarity measures. As such, untrained message passing architectures can be viewed as a highly efficient and interpretable approach to link prediction.


title: Commutative families in DIM algebra, integrable many-body systems and $q,t$ matrix models

abstract: We extend our consideration of commutative subalgebras (rays) in different representations of the $W_{1+\infty}$ algebra to the elliptic Hall algebra (or, equivalently, to the Ding-Iohara-Miki (DIM) algebra $U_{q,t}(\widehat{\widehat{\mathfrak{gl}}}_1)$). Its advantage is that it possesses the Miki automorphism, which makes all commutative rays equivalent. Integrable systems associated with these rays become finite-difference and, apart from the trigonometric Ruijsenaars system not too much familiar. We concentrate on the simplest many-body and Fock representations, and derive explicit formulas for all generators of the elliptic Hall algebra $e_{n,m}$. In the one-body representation, they differ just by normalization from $z^nq^{m\hat D}$ of the $W_{1+\infty}$ Lie algebra, and, in the $N$-body case, they are non-trivially generalized to monomials of the Cherednik operators with action restricted to symmetric polynomials. In the Fock representation, the resulting operators are expressed through auxiliary polynomials of $n$ variables, which define weights in the residues formulas. We also discuss $q,t$-deformation of matrix models associated with constructed commutative subalgebras.


title: Coding schemes in neural networks learning classification tasks

abstract: Neural networks posses the crucial ability to generate meaningful representations of task-dependent features. Indeed, with appropriate scaling, supervised learning in neural networks can result in strong, task-dependent feature learning. However, the nature of the emergent representations, which we call the `coding scheme', is still unclear. To understand the emergent coding scheme, we investigate fully-connected, wide neural networks learning classification tasks using the Bayesian framework where learning shapes the posterior distribution of the network weights. Consistent with previous findings, our analysis of the feature learning regime (also known as `non-lazy', `rich', or `mean-field' regime) shows that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity. In linear networks, an analog coding scheme of the task emerges. Despite the strong representations, the mean predictor is identical to the lazy case. In nonlinear networks, spontaneous symmetry breaking leads to either redundant or sparse coding schemes. Our findings highlight how network properties such as scaling of weights and neuronal nonlinearity can profoundly influence the emergent representations.

categories: cs.LG cond-mat.dis-nn cond-mat.stat-mech stat.ML


title: Scaling Laws for Linear Complexity Language Models

abstract: The interest in linear complexity models for large language models is on the rise, although their scaling capacity remains uncertain. In this study, we present the scaling laws for linear complexity language models to establish a foundation for their scalability. Specifically, we examine the scaling behaviors of three efficient linear architectures. These include TNL, a linear attention model with data-independent decay; HGRN2, a linear RNN with data-dependent decay; and cosFormer2, a linear attention model without decay. We also include LLaMA as a baseline architecture for softmax attention for comparison. These models were trained with six variants, ranging from 70M to 7B parameters on a 300B-token corpus, and evaluated with a total of 1,376 intermediate checkpoints on various downstream tasks. These tasks include validation loss, commonsense reasoning, and information retrieval and generation. The study reveals that existing linear complexity language models exhibit similar scaling capabilities as conventional transformer-based models while also demonstrating superior linguistic proficiency and knowledge retention.


title: Theory Driven Evolution of the Weak Mixing Angle

abstract: We present the first purely theoretical calculation of the weak mixing angle in the MS scheme at low energies by combining results from lattice QCD with perturbation theory. We discuss its correlation with the hadronic contribution to the anomalous magnetic moment of the muon and to the energy dependence of the electromagnetic coupling. We also compare the results with calculations using cross-section data as input. Implications for the Standard Model prediction of the mass of the W boson are also discussed.


title: Stationary and Sparse Denoising Approach for Corticomuscular Causality Estimation

abstract: Objective: Cortico-muscular communication patterns are instrumental in understanding movement control. Estimating significant causal relationships between motor cortex electroencephalogram (EEG) and surface electromyogram (sEMG) from concurrently active muscles presents a formidable challenge since the relevant processes underlying muscle control are typically weak in comparison to measurement noise and background activities. Methodology: In this paper, a novel framework is proposed to simultaneously estimate the order of the autoregressive model of cortico-muscular interactions along with the parameters while enforcing stationarity condition in a convex program to ensure global optimality. The proposed method is further extended to a non-convex program to account for the presence of measurement noise in the recorded signals by introducing a wavelet sparsity assumption on the excitation noise in the model. Results: The proposed methodology is validated using both simulated data and neurophysiological signals. In case of simulated data, the performance of the proposed methods has been compared with the benchmark approaches in terms of order identification, computational efficiency, and goodness of fit in relation to various noise levels. In case of physiological signals our proposed methods are compared against the state-of-the-art approaches in terms of the ability to detect Granger causality. Significance: The proposed methods are shown to be effective in handling stationarity and measurement noise assumptions, revealing significant causal interactions from brain to muscles and vice versa.


title: Measuring the matter fluctuations in the Local Universe with the ALFALFA catalog

abstract: The standard model of cosmology describes the matter fluctuations through the matter power spectrum, where $\sigma_{8} \equiv \sigma_{8,0} \equiv \sigma_{8}(z = 0)$, defined at the scale of $8\,h^{-1}$\,Mpc, acts as a normalisation parameter. Currently, there is a notable discrepancy in the reported values of $\sigma_{8}$ obtained from cosmic microwave background and large scale structure data analyses, which indicates a $\sim 3\,\sigma$ tension between these measurements. This study quantifies matter fluctuations in the Local Universe using HI extragalactic sources mapped by the ALFALFA survey to test the standard cosmological model under extreme conditions in the highly non-linear Local Universe, $z \approx 0$, quantifying the amplitude of the matter fluctuations there, $\sigma_8$. Our work directly measures $\sigma_{8}$ using the ALFALFA data, where 3D distances were obtained without assuming $H_0$, resulting in a robust model-independent analysis.


title: Task Oriented In-Domain Data Augmentation

abstract: Large Language Models (LLMs) have shown superior performance in various applications and fields. To achieve better performance on specialized domains such as law and advertisement, LLMs are often continue pre-trained on in-domain data. However, existing approaches suffer from two major issues. First, in-domain data are scarce compared with general domain-agnostic data. Second, data used for continual pre-training are not task-aware, such that they may not be helpful to downstream applications. We propose TRAIT, a task-oriented in-domain data augmentation framework. Our framework is divided into two parts: in-domain data selection and task-oriented synthetic passage generation. The data selection strategy identifies and selects a large amount of in-domain data from general corpora, and thus significantly enriches domain knowledge in the continual pre-training data. The synthetic passages contain guidance on how to use domain knowledge to answer questions about downstream tasks. By training on such passages, the model aligns with the need of downstream applications. We adapt LLMs to two domains: advertisement and math. On average, TRAIT improves LLM performance by 8% in the advertisement domain and 7.5% in the math domain.


title: Geometry-Aware Score Distillation via 3D Consistent Noising and Gradient Consistency Modeling

abstract: Score distillation sampling (SDS), the methodology in which the score from pretrained 2D diffusion models is distilled into 3D representation, has recently brought significant advancements in text-to-3D generation task. However, this approach is still confronted with critical geometric inconsistency problems such as the Janus problem. Starting from a hypothesis that such inconsistency problems may be induced by multiview inconsistencies between 2D scores predicted from various viewpoints, we introduce GSD, a simple and general plug-and-play framework for incorporating 3D consistency and therefore geometry awareness into the SDS process. Our methodology is composed of three components: 3D consistent noising, designed to produce 3D consistent noise maps that perfectly follow the standard Gaussian distribution, geometry-based gradient warping for identifying correspondences between predicted gradients of different viewpoints, and novel gradient consistency loss to optimize the scene geometry toward producing more consistent gradients. We demonstrate that our method significantly improves performance, successfully addressing the geometric inconsistency problems in text-to-3D generation task with minimal computation cost and being compatible with existing score distillation-based models. Our project page is available at https://ku-cvlab.github.io/GSD/.


title: Public Constitutional AI

abstract: We are increasingly subjected to the power of AI authorities. As AI decisions become inescapable, entering domains such as healthcare, education, and law, we must confront a vital question: how can we ensure AI systems have the legitimacy necessary for effective governance? This essay argues that to secure AI legitimacy, we need methods that engage the public in designing and constraining AI systems, ensuring these technologies reflect the community's shared values. Constitutional AI, proposed by Anthropic, represents a step towards this goal, offering a model for democratic control of AI. However, while Constitutional AI's commitment to hardcoding explicit principles into AI models enhances transparency and accountability, it falls short in two crucial aspects: addressing the opacity of individual AI decisions and fostering genuine democratic legitimacy. To overcome these limitations, this essay proposes "Public Constitutional AI." This approach envisions a participatory process where diverse stakeholders, including ordinary citizens, deliberate on the principles guiding AI development. The resulting "AI Constitution" would carry the legitimacy of popular authorship, grounding AI governance in the public will. Furthermore, the essay proposes "AI Courts" to develop "AI case law," providing concrete examples for operationalizing constitutional principles in AI training. This evolving combination of constitutional principles and case law aims to make AI governance more responsive to public values. By grounding AI governance in deliberative democratic processes, Public Constitutional AI offers a path to imbue automated authorities with genuine democratic legitimacy, addressing the unique challenges posed by increasingly powerful AI systems while ensuring their alignment with the public interest.


title: Expected Runtime Comparisons Between Breadth-First Search and Constant-Depth Restarting Random Walks

abstract: When greedy search algorithms encounter a local minima or plateau, the search typically devolves into a breadth-first search (BrFS), or a local search technique is used in an attempt to find a way out. In this work, we formally analyze the performance of BrFS and constant-depth restarting random walks (RRW) -- two methods often used for finding exits to a plateau/local minima -- to better understand when each is best suited. In particular, we formally derive the expected runtime for BrFS in the case of a uniformly distributed set of goals at a given goal depth. We then prove RRW will be faster than BrFS on trees if there are enough goals at that goal depth. We refer to this threshold as the crossover point. Our bound shows that the crossover point grows linearly with the branching factor of the tree, the goal depth, and the error in the random walk depth, while the size of the tree grows exponentially in branching factor and goal depth. Finally, we discuss the practical implications and applicability of this bound.


title: Learning Interpretable Fair Representations

abstract: Numerous approaches have been recently proposed for learning fair representations that mitigate unfair outcomes in prediction tasks. A key motivation for these methods is that the representations can be used by third parties with unknown objectives. However, because current fair representations are generally not interpretable, the third party cannot use these fair representations for exploration, or to obtain any additional insights, besides the pre-contracted prediction tasks. Thus, to increase data utility beyond prediction tasks, we argue that the representations need to be fair, yet interpretable. We propose a general framework for learning interpretable fair representations by introducing an interpretable "prior knowledge" during the representation learning process. We implement this idea and conduct experiments with ColorMNIST and Dsprite datasets. The results indicate that in addition to being interpretable, our representations attain slightly higher accuracy and fairer outcomes in a downstream classification task compared to state-of-the-art fair representations.


title: Phenomenological Aspects of Lee-Wick QED

abstract: We study some phenomenological aspects of Lee-Wick (LW) QED. In particular, we show that LW QED implies charge dequantization and a flavor-dependent LW scale. We study the implications of the Weak Gravity Conjecture (WGC) in LW QED and calculate the modified electric force and potential and use the former to reformulate the WGC in LW QED. We also calculate the photon self-energy and the Uehling potential in LW QED. We show that bounds on milli-charged particles from matter neutrality experiments and from Cavendish-type experiments set stringent limits on the LW scale of fermions and of the photon.


title: Anomalies in mirror symmetry enriched topological orders

abstract: Two-dimensional mirror symmetry-enriched topological orders can be studied using the folding approach: it can be folded along the mirror axis and turned into a bilayer system on which the mirror symmetry acts as a $\mathbb Z_2$ layer-exchange symmetry. How mirror-symmetry enriches the topological order is then encoded at the mirror axis, which is a gapped boundary of the folded bilayer system. Based on anyon-condensation theory, we classify possible $\mathbb Z_2$-symmetric gapped boundaries of the folded system. In particular, we derive an $H^2$ obstruction function, which corresponds to an $H^3$ obstruction for topological orders enriched by the time-reversal symmetry instead of mirror symmetry. We demonstrate that states with a nontrivial $H^2$ obstruction function can be constructed on the surface of a three-dimensional mirror SET order.


title: Demystifying the Effect of Receptive Field Size in U-Net Models for Medical Image Segmentation

abstract: Medical image segmentation is a critical task in healthcare applications, and U-Nets have demonstrated promising results. This work delves into the understudied aspect of receptive field (RF) size and its impact on the U-Net and Attention U-Net architectures. This work explores several critical elements including the relationship between RF size, characteristics of the region of interest, and model performance, as well as the balance between RF size and computational costs for U-Net and Attention U-Net methods for different datasets. This work also proposes a mathematical notation for representing the theoretical receptive field (TRF) of a given layer in a network and proposes two new metrics - effective receptive field (ERF) rate and the Object rate to quantify the fraction of significantly contributing pixels within the ERF against the TRF area and assessing the relative size of the segmentation object compared to the TRF size respectively. The results demonstrate that there exists an optimal TRF size that successfully strikes a balance between capturing a wider global context and maintaining computational efficiency, thereby optimizing model performance. Interestingly, a distinct correlation is observed between the data complexity and the required TRF size; segmentation based solely on contrast achieved peak performance even with smaller TRF sizes, whereas more complex segmentation tasks necessitated larger TRFs. Attention U-Net models consistently outperformed their U-Net counterparts, highlighting the value of attention mechanisms regardless of TRF size. These novel insights present an invaluable resource for developing more efficient U-Net-based architectures for medical imaging and pave the way for future exploration. A tool is also developed that calculates the TRF for a U-Net (and Attention U-Net) model, and also suggest an appropriate TRF size for a given model and dataset.


title: North-PHASE: Studying Periodicity, Hot Spots, Accretion Stability and Early Evolution in young stars in the northern hemisphere

abstract: We present the overview and first results from the North-PHASE Legacy Survey, which follows six young clusters for five years, using the 2 deg$^2$ FoV of the JAST80 telescope from the Javalambre Observatory (Spain). North-PHASE investigates stellar variability on timescales from days to years for thousands of young stars distributed over entire clusters. This allows us to find new YSO, characterise accretion and study inner disk evolution within the cluster context. Each region (Tr37, CepOB3, IC5070, IC348, NGC2264, and NGC1333) is observed in six filters (SDSS griz, u band, and J0660, which covers H$\alpha$), detecting cluster members as well as field variable stars. Tr37 is used to prove feasibility and optimise the variability analysis techniques. In Tr37, variability reveals 50 new YSO, most of them proper motion outliers. North-PHASE independently confirms the youth of astrometric members, efficiently distinguishes accreting and non-accreting stars, reveals the extent of the cluster populations along Tr37/IC1396 bright rims, and detects variability resulting from rotation, dips, and irregular bursts. The proper motion outliers unveil a more complex star formation history than inferred from Gaia alone, and variability highlights previously hidden proper motion deviations in the surrounding clouds. We also find that non-YSO variables identified by North-PHASE cover a different variability parameter space and include long-period variables, eclipsing binaries, RR Lyr, and $\delta$ Scuti stars. These early results also emphasize the power of variability to complete the picture of star formation where it is missed by astrometry.


title: Velocity-vorticity-pressure mixed formulation for the Kelvin-Voigt-Brinkman-Forchheimer equations

abstract: In this paper, we propose and analyze a mixed formulation for the Kelvin-Voigt-Brinkman-Forchheimer equations for unsteady viscoelastic flows in porous media. Besides the velocity and pressure, our approach introduces the vorticity as a further unknown. Consequently, we obtain a three-field mixed variational formulation, where the aforementioned variables are the main unknowns of the system. We establish the existence and uniqueness of a solution for the weak formulation, and derive the corresponding stability bounds, employing a fixed-point strategy, along with monotone operators theory and Schauder theorem. Afterwards, we introduce a semidiscrete continuous-in-time approximation based on stable Stokes elements for the velocity and pressure, and continuous piecewise polynomial spaces for the vorticity. Additionally, employing backward Euler time discretization, we introduce a fully discrete finite element scheme. We prove well-posedness, derive stability bounds, and establish the corresponding error estimates for both schemes. We provide several numerical results verifying the theoretical rates of convergence and illustrating the performance and flexibility of the method for a range of domain configurations and model parameters.


title: Tuning a Cascaded Online Feedback Optimization Controller for Provision of Distributed Flexibility

abstract: Coordinating a high number of flexibility providing units (e.g. to provide ancillary services for the transmission system) across various grid layers requires new control concepts. A flexibility request at a point of common coupling can be met by utilizing a cascaded control structure based on online feedback optimization. In this paper the influence of the parameterization of the individual controllers on the performance of the hierarchical flexibility provision is studied on a three-level test system. The results show a high interdependency between the choice of control parameters of one controller and the behavior of other controllers as well as a significant impact on the accuracy and speed of flexibility provision. With a careful tuning, a cascaded structure based on online feedback optimization can achieve efficient vertical coordination of flexibility providing units.


title: Cycles in graphs and in hypergraphs (in Russian)

abstract: In this expository paper we present some ideas of algebraic topology in a language accessible to non-specialists in the area. A $1$-cycle in a graph is a set $C$ of edges such that every vertex is contained in an even number of edges from $C$. It is easy to check that the sum (modulo $2$) of $1$-cycles is a $1$-cycle. We start from the following problems: to find $\bullet$ the number of all $1$-cycles in a given graph; $\bullet$ a small number of $1$-cycles in a given graph such that any $1$-cycle is the sum of some of them. We consider generalizations (of these problems) to graphs with symmetry, to $2$-cycles in $2$-dimensional hypergraphs, and to certain configuration spaces of graphs (namely, to the square and the deleted square).


title: Collective preparation of large quantum registers with high fidelity

abstract: We report on the preparation of a large quantum register of 5612 qubits, with the unprecedented high global fidelity of $F\simeq 0.9956$. This was achieved by applying an improved cooperative quantum information erasure (CQIE) protocol [Buffoni, L. and Campisi, M., Quantum 7, 961 (2023)] to a programmable network of superconducting qubits featuring a high connectivity. At variance with the standard method based on the individual reset of each qubit in parallel, here the quantum register is treated as a whole, thus avoiding the well-known orthogonality catastrophe wehereby even an extremely high individual reset fidelity $f$ results in vanishing global fidelities $F=f^N$ with growing number $N$ of qubits.


title: Probabilistic Subgoal Representations for Hierarchical Reinforcement learning

abstract: In goal-conditioned hierarchical reinforcement learning (HRL), a high-level policy specifies a subgoal for the low-level policy to reach. Effective HRL hinges on a suitable subgoal represen tation function, abstracting state space into latent subgoal space and inducing varied low-level behaviors. Existing methods adopt a subgoal representation that provides a deterministic mapping from state space to latent subgoal space. Instead, this paper utilizes Gaussian Processes (GPs) for the first probabilistic subgoal representation. Our method employs a GP prior on the latent subgoal space to learn a posterior distribution over the subgoal representation functions while exploiting the long-range correlation in the state space through learnable kernels. This enables an adaptive memory that integrates long-range subgoal information from prior planning steps allowing to cope with stochastic uncertainties. Furthermore, we propose a novel learning objective to facilitate the simultaneous learning of probabilistic subgoal representations and policies within a unified framework. In experiments, our approach outperforms state-of-the-art baselines in standard benchmarks but also in environments with stochastic elements and under diverse reward conditions. Additionally, our model shows promising capabilities in transferring low-level policies across different tasks.


title: CausalFormer: An Interpretable Transformer for Temporal Causal Discovery

abstract: Temporal causal discovery is a crucial task aimed at uncovering the causal relations within time series data. The latest temporal causal discovery methods usually train deep learning models on prediction tasks to uncover the causality between time series. They capture causal relations by analyzing the parameters of some components of the trained models, e.g., attention weights and convolution weights. However, this is an incomplete mapping process from the model parameters to the causality and fails to investigate the other components, e.g., fully connected layers and activation functions, that are also significant for causal discovery. To facilitate the utilization of the whole deep learning models in temporal causal discovery, we proposed an interpretable transformer-based causal discovery model termed CausalFormer, which consists of the causality-aware transformer and the decomposition-based causality detector. The causality-aware transformer learns the causal representation of time series data using a prediction task with the designed multi-kernel causal convolution which aggregates each input time series along the temporal dimension under the temporal priority constraint. Then, the decomposition-based causality detector interprets the global structure of the trained causality-aware transformer with the proposed regression relevance propagation to identify potential causal relations and finally construct the causal graph. Experiments on synthetic, simulated, and real datasets demonstrate the state-of-the-art performance of CausalFormer on discovering temporal causality. Our code is available at https://github.com/lingbai-kong/CausalFormer.


title: Evidence of directional structural superlubricity and L\'evy flights in a van der Waals heterostructure

abstract: Structural superlubricity is a special frictionless contact in which two crystals are in incommensurate arrangement such that relative in-plane translation is associated with vanishing energy barrier crossing. So far, it has been realized in multilayer graphene and other van der Waals two-dimensional crystals with hexagonal or triangular crystalline symmetries, leading to isotropic frictionless contacts. Directional structural superlubricity, to date unrealized in two-dimensional systems, is possible when the reciprocal lattices of the two crystals coincide in one direction only. Here, we evidence directional structural superlubricity a $\alpha$-bismuthene/graphite van der Waals system, manifested by spontaneous hopping of the islands over hundreds of nanometres at room temperature, resolved by low-energy electron microscopy and supported by registry simulations. Statistical analysis of individual and collective $\alpha$-bismuthene islands populations reveal a heavy-tailed distribution of the hopping lengths and sticking times indicative of L\'evy flight dynamics, largely unobserved in massive condensed-matter systems.


title: Portrait3D: 3D Head Generation from Single In-the-wild Portrait Image

abstract: While recent works have achieved great success on one-shot 3D common object generation, high quality and fidelity 3D head generation from a single image remains a great challenge. Previous text-based methods for generating 3D heads were limited by text descriptions and image-based methods struggled to produce high-quality head geometry. To handle this challenging problem, we propose a novel framework, Portrait3D, to generate high-quality 3D heads while preserving their identities. Our work incorporates the identity information of the portrait image into three parts: 1) geometry initialization, 2) geometry sculpting, and 3) texture generation stages. Given a reference portrait image, we first align the identity features with text features to realize ID-aware guidance enhancement, which contains the control signals representing the face information. We then use the canny map, ID features of the portrait image, and a pre-trained text-to-normal/depth diffusion model to generate ID-aware geometry supervision, and 3D-GAN inversion is employed to generate ID-aware geometry initialization. Furthermore, with the ability to inject identity information into 3D head generation, we use ID-aware guidance to calculate ID-aware Score Distillation (ISD) for geometry sculpting. For texture generation, we adopt the ID Consistent Texture Inpainting and Refinement which progressively expands the view for texture inpainting to obtain an initialization UV texture map. We then use the id-aware guidance to provide image-level supervision for noisy multi-view images to obtain a refined texture map. Extensive experiments demonstrate that we can generate high-quality 3D heads with accurate geometry and texture from single in-the-wild portrait images. The project page is at https://jinkun-hao.github.io/Portrait3D/.


title: Generalized Modal Analysis in Power System with High CIG Penetration: Concept and Quantitative Assessment

abstract: This paper presents a Generalized Modal Analysis (GMA) concept for the small-signal stability analysis of power systems with high penetration of Converter-Interfaced Generation (CIG). GMA quantitatively assesses interactions between various elements in the power system, offering intuitive and transparent physical interpretations. The method's versatility in selecting physical quantities at different input and output ports makes it broadly applicable. Based on the concept of GMA, the study further defines interaction quantification indices by selecting voltage ports, examining the impact of grid disturbances on power sources and the support from the power sources to the grid at connection points. Numerical simulations on modified 14-bus and 68-bus systems validate GMA's effectiveness in capturing the coupling of the dynamic characteristics between grid elements. This research provides a theoretical foundation and analytical framework for future analyses of power system stability with diverse power sources.


title: Ab Initio Molecular Dynamics calculations on NO oxidation over oxygen functionalized Highly Oriented Pyrolytic Graphite

abstract: The oxidation of NO molecules on epoxy-functionalized highly oriented pyrolytic graphite, thermalized at 300 K, was studied by means of ab initio molecular dynamics (AIMD) calculations. Four collision energies and two different orientations were analyzed where the reaction, adsorption, and scattering probabilities were computed. Our results reveal that NO$_2$ formation can occur even at the lowest collision energy investigated (0.025 eV), approximately equivalent to room temperature (300 K), which agrees qualitatively with the experimental results. This underscores the influence of dynamics on the NO oxidation process, since this oxidation barrier was previously theoretically estimated to be about 0.1 eV at 0 K, which is four times higher than our lowest collision energy. Additionally, we obtained angular and energy distributions of the products under selected simulation conditions. Scattered NO molecules show low specular reflection, lose half of their initial translational energy, and remain vibrationally cold with minimal rotational excitation. Furthermore, a statistical analysis of all reactive trajectories, focusing on configurations at specific reaction moments, elucidated the structural requirements for the reaction to occur under dynamic conditions. Finally, this study demonstrates the potential of oxygen-doped carbon surfaces for the conversion of NO to NO$_2$.


title: ShanghaiTech Mapping Robot is All You Need: Robot System for Collecting Universal Ground Vehicle Datasets

abstract: This paper presents the ShanghaiTech Mapping Robot, a state-of-the-art unmanned ground vehicle (UGV) designed for collecting comprehensive multi-sensor datasets to support research in robotics, computer vision, and autonomous driving. The robot is equipped with a wide array of sensors including RGB cameras, RGB-D cameras, event-based cameras, IR cameras, LiDARs, mmWave radars, IMUs, ultrasonic range finders, and a GNSS RTK receiver. The sensor suite is integrated onto a specially designed mechanical structure with a centralized power system and a synchronization mechanism to ensure spatial and temporal alignment of the sensor data. A 16-node on-board computing cluster handles sensor control, data collection, and storage. We describe the hardware and software architecture of the robot in detail and discuss the calibration procedures for the various sensors. The capabilities of the platform are demonstrated through an extensive dataset collected in diverse real-world environments. To facilitate research, we make the dataset publicly available along with the associated robot sensor calibration data. Performance evaluations on a set of standard perception and localization tasks showcase the potential of the dataset to support developments in Robot Autonomy.


title: AutoDetect: Towards a Unified Framework for Automated Weakness Detection in Large Language Models

abstract: Although Large Language Models (LLMs) are becoming increasingly powerful, they still exhibit significant but subtle weaknesses, such as mistakes in instruction-following or coding tasks. As these unexpected errors could lead to severe consequences in practical deployments, it is crucial to investigate the limitations within LLMs systematically. Traditional benchmarking approaches cannot thoroughly pinpoint specific model deficiencies, while manual inspections are costly and not scalable. In this paper, we introduce a unified framework, AutoDetect, to automatically expose weaknesses in LLMs across various tasks. Inspired by the educational assessment process that measures students' learning outcomes, AutoDetect consists of three LLM-powered agents: Examiner, Questioner, and Assessor. The collaboration among these three agents is designed to realize comprehensive and in-depth weakness identification. Our framework demonstrates significant success in uncovering flaws, with an identification success rate exceeding 30% in prominent models such as ChatGPT and Claude. More importantly, these identified weaknesses can guide specific model improvements, proving more effective than untargeted data augmentation methods like Self-Instruct. Our approach has led to substantial enhancements in popular LLMs, including the Llama series and Mistral-7b, boosting their performance by over 10% across several benchmarks. Code and data are publicly available at https://github.com/thu-coai/AutoDetect.


title: GC-Bench: A Benchmark Framework for Graph Condensation with New Insights

abstract: Graph condensation (GC) is an emerging technique designed to learn a significantly smaller graph that retains the essential information of the original graph. This condensed graph has shown promise in accelerating graph neural networks while preserving performance comparable to those achieved with the original, larger graphs. Additionally, this technique facilitates downstream applications such as neural architecture search and enhances our understanding of redundancy in large graphs. Despite the rapid development of GC methods, a systematic evaluation framework remains absent, which is necessary to clarify the critical designs for particular evaluative aspects. Furthermore, several meaningful questions have not been investigated, such as whether GC inherently preserves certain graph properties and offers robustness even without targeted design efforts. In this paper, we introduce GC-Bench, a comprehensive framework to evaluate recent GC methods across multiple dimensions and to generate new insights. Our experimental findings provide a deeper insights into the GC process and the characteristics of condensed graphs, guiding future efforts in enhancing performance and exploring new applications. Our code is available at \url{https://github.com/Emory-Melody/GraphSlim/tree/main/benchmark}.


title: One-Class Learning with Adaptive Centroid Shift for Audio Deepfake Detection

abstract: As speech synthesis systems continue to make remarkable advances in recent years, the importance of robust deepfake detection systems that perform well in unseen systems has grown. In this paper, we propose a novel adaptive centroid shift (ACS) method that updates the centroid representation by continually shifting as the weighted average of bonafide representations. Our approach uses only bonafide samples to define their centroid, which can yield a specialized centroid for one-class learning. Integrating our ACS with one-class learning gathers bonafide representations into a single cluster, forming well-separated embeddings robust to unseen spoofing attacks. Our proposed method achieves an equal error rate (EER) of 2.19% on the ASVspoof 2021 deepfake dataset, outperforming all existing systems. Furthermore, the t-SNE visualization illustrates that our method effectively maps the bonafide embeddings into a single cluster and successfully disentangles the bonafide and spoof classes.


title: Probing Yu-Shiba-Rusinov state via quantum noise and $\Delta_T$ noise

abstract: Recent attention has been drawn to temperature gradient generated $\Delta_T$ noise at vanishing charge current. This study delves into examining the properties of spin-polarised $\Delta T$ noise in conjunction with $\Delta_T$-shot noise, $\Delta_T$-thermal noise, and quantum noise (again both shot and thermal noise) in a one-dimensional (1D) structure comprising metal/spin-flipper/metal/insulator/superconductor junction to probe Yu-Shiba-Rusinov (YSR) bound states. YSR bound states, which are localized states within the superconducting gap of a superconductor are induced by a magnetic impurity acting as a spin-flipper. A YSR bound state should be distinguished from a Majorana bound state (MBS), which too can occur due to interaction with magnetic impurities, e.g., magnetic adatoms on superconductors, and this can lead to false positives in detecting MBS. Clarifying this by providing a unique signature for the YSR-bound state is the main aim of this work. In this paper, we show that YSR bound states can be effectively probed using quantum noise and the recently discovered $\Delta_T$ noise, with a focus on especially spin transport. We see that the spin $\Delta_T$ noise is a superior tool compared to the charge $\Delta_T$ noise as a probe for YSR bound states. Additionally, our analysis of quantum noise reveals that similar to $\Delta_T$ noise, spin quantum noise is more effective than charge quantum noise in detecting YSR bound states.


title: A Boot-Strapping Technique to Design Dense Output Formulae for Modified Patankar-Runge-Kutta Methods

abstract: In this work modified Patankar-Runge-Kutta (MPRK) schemes up to order four are considered and equipped with a dense output formula of appropriate accuracy. Since these time integrators are conservative and positivity preserving for any time step size, we impose the same requirements on the corresponding dense output formula. In particular, we discover that there is an explicit first order formula. However, to develop a boot-strapping technique we propose to use implicit formulae which naturally fit into the framework of MPRK schemes. In particular, if lower order MPRK schemes are used to construct methods of higher order, the same can be donw with the dense output formulae we propose in this work. We explicitly construct formulae up to order three and demonstrate how to generalize this approach as long as the underlying Runge-Kutta method possesses a dense output formulae of appropriate accuracy. We also note that even though linear systems have to be solved to compute an approximation for intermediate points in time using these higher order dense output formulae, the overall computational effort is reduced compared to using the scheme with a smaller step size.


title: Backstepping control for the sterile mosquitoes technique: stabilization of extinction equilibrium

abstract: The control of a mosquito population using the sterile insect technique is considered. Building on a model-based approach, where the control input is the release rate of sterilized males, we propose a non-negative backstepping control law capable of globally stabilizing the extinction equilibrium of the system. A simulation study supports and validates the theoretical findings, showing the efficacy of the approach both on a reduced model, used for control design, and on a complete model of the mosquito population dynamics.


title: Dynamic Probability Logic: Decidability & Computability

abstract: In this article, the decidability and computability issues of dynamic probability logic (DPL) are addressed. Firstly, a proof system $\mathcal{H}_{DPL}$ is introduced for DPL and shown that it is weakly complete. Furthermore, this logic has the finite model property and so is decidable. Secondly, a strongly complete proof system HDPL is presented for DPL and proved that its canonical model is a computable structure.


title: Spatially Structured Regression for Non-conformable Spaces: Integrating Pathology Imaging and Genomics Data in Cancer

abstract: The spatial composition and cellular heterogeneity of the tumor microenvironment plays a critical role in cancer development and progression. High-definition pathology imaging of tumor biopsies provide a high-resolution view of the spatial organization of different types of cells. This allows for systematic assessment of intra- and inter-patient spatial cellular interactions and heterogeneity by integrating accompanying patient-level genomics data. However, joint modeling across tumor biopsies presents unique challenges due to non-conformability (lack of a common spatial domain across biopsies) as well as high-dimensionality. To address this problem, we propose the Dual random effect and main effect selection model for Spatially structured regression model (DreameSpase). DreameSpase employs a Bayesian variable selection framework that facilitates the assessment of spatial heterogeneity with respect to covariates both within (through fixed effects) and between spaces (through spatial random effects) for non-conformable spatial domains. We demonstrate the efficacy of DreameSpase via simulations and integrative analyses of pathology imaging and gene expression data obtained from $335$ melanoma biopsies. Our findings confirm several existing relationships, e.g. neutrophil genes being associated with both inter- and intra-patient spatial heterogeneity, as well as discovering novel associations. We also provide freely available and computationally efficient software for implementing DreameSpase.


title: Venturing into Uncharted Waters: The Navigation Compass from Transformer to Mamba

abstract: Transformer, a deep neural network architecture, has long dominated the field of natural language processing and beyond. Nevertheless, the recent introduction of Mamba challenges its supremacy, sparks considerable interest among researchers, and gives rise to a series of Mamba-based models that have exhibited notable potential. This survey paper orchestrates a comprehensive discussion, diving into essential research dimensions, covering: (i) the functioning of the Mamba mechanism and its foundation on the principles of structured state space models; (ii) the proposed improvements and the integration of Mamba with various networks, exploring its potential as a substitute for Transformers; (iii) the combination of Transformers and Mamba to compensate for each other's shortcomings. We have also made efforts to interpret Mamba and Transformer in the framework of kernel functions, allowing for a comparison of their mathematical nature within a unified context. Our paper encompasses the vast majority of improvements related to Mamba to date.


title: Efficient machine learning for motion sensing for lighting applications

abstract: The use of machine learning for building a classifier in signal processing for motion sensing presents unique challenges. This paper proposes a novel method that effectively addresses the combination of skewed data sets and optimization requirements. By utilizing a customized loss function and a product of probability models, our approach achieves a fully automated and efficient machine learning process. Additionally, our resulting probability models offer reduced complexity, making them ideal for embedded applications. Our method offers a promising solution for motion sensing applications that require accurate and efficient classification.


title: {\mu}-Net: A Deep Learning-Based Architecture for {\mu}-CT Segmentation

abstract: X-ray computed microtomography ({\mu}-CT) is a non-destructive technique that can generate high-resolution 3D images of the internal anatomy of medical and biological samples. These images enable clinicians to examine internal anatomy and gain insights into the disease or anatomical morphology. However, extracting relevant information from 3D images requires semantic segmentation of the regions of interest, which is usually done manually and results time-consuming and tedious. In this work, we propose a novel framework that uses a convolutional neural network (CNN) to automatically segment the full morphology of the heart of Carassius auratus. The framework employs an optimized 2D CNN architecture that can infer a 3D segmentation of the sample, avoiding the high computational cost of a 3D CNN architecture. We tackle the challenges of handling large and high-resoluted image data (over a thousand pixels in each dimension) and a small training database (only three samples) by proposing a standard protocol for data normalization and processing. Moreover, we investigate how the noise, contrast, and spatial resolution of the sample and the training of the architecture are affected by the reconstruction technique, which depends on the number of input images. Experiments show that our framework significantly reduces the time required to segment new samples, allowing a faster microtomography analysis of the Carassius auratus heart shape. Furthermore, our framework can work with any bio-image (biological and medical) from {\mu}-CT with high-resolution and small dataset size


title: The host of GRB 171205A in 3D -- A resolved multiwavelength study of a rare grand-design spiral GRB host

abstract: Long GRB hosts at z<1 are usually low-mass, low metallicity star-forming galaxies. Here we present the until now most detailed, spatially resolved study of the host of GRB 171205A, a grand-design barred spiral galaxy at z=0.036. Our analysis includes MUSE integral field spectroscopy, complemented by high spatial resolution UV/VIS HST imaging and CO(1-0) and HI 21cm data. The GRB is located in a small star-forming region in a spiral arm of the galaxy at a deprojected distance of ~ 8 kpc from the center. The galaxy shows a smooth negative metallicity gradient and the metallicity at the GRB site is half solar, slightly below the mean metallicity at the corresponding distance from the center. Star formation in this galaxy is concentrated in a few HII regions between 5-7 kpc from the center and at the end of the bar, inwards of the GRB region, however, the HII region hosting the GRB is in the top 10% of regions with highest specific star-formation rate. The stellar population at the GRB site has a very young component (< 5 Myr) contributing a significant part of the light. Ionized and molecular gas show only minor deviations at the end of the bar. A parallel study found an asymmetric HI distribution and some additional gas near the position of the GRB, which might explain the star-forming region of the GRB site. Our study shows that long GRBs can occur in many types of star-forming galaxies, however, the actual GRB sites consistently have low metallicity, high star formation and a young population. Furthermore, gas inflow or interactions triggering the star formation producing the GRB progenitor might not be evident in ionized or even molecular gas but only in HI.


title: HI and CO spectroscopy of the unusual host of GRB 171205A: A grand design spiral galaxy with a distorted HI field

abstract: GRBs produced by the collapse of massive stars are usually found near the most prominent star-forming regions of star-forming galaxies. GRB 171205A happened in the outskirts of a spiral galaxy, a peculiar location in an atypical GRB host. In this paper we present a highly-resolved study of the molecular gas of this host, with CO(1-0) observations from ALMA. We compare with GMRT atomic HI observations, and with data at other wavelengths to provide a broad-band view of the galaxy. The ALMA observations have a spatial resolution of 0.2" and a spectral resolution of 10 km/s, observed when the afterglow had a flux density of ~53 mJy. This allowed a molecular study both in emission and absorption. The HI observations allowed to study the host galaxy and its extended environment. The CO emission shows an undisturbed spiral structure with a central bar, and no significant emission at the location of the GRB. Our CO spectrum does not reveal any CO absorption, with a column density limit of < 10^15 cm^-2. This argues against the progenitor forming in a massive molecular cloud. The molecular gas traces the galaxy arms with higher concentration in the regions dominated by dust. The HI gas does not follow the stellar light or the molecular gas and is concentrated in two blobs, with no emission towards the centre of the galaxy, and is slightly displaced towards the southwest of the galaxy, where the GRB exploded. Within the extended neighbourhood of the host galaxy, we identify another prominent HI source at the same redshift, at a projected distance of 188 kpc. Our observations show that the progenitor of this GRB is not associated to a massive molecular cloud, but more likely related to low-metallicity atomic gas. The distortion in the HI gas field is indicator of an odd environment that could have triggered star formation and could be linked to a past interaction with the companion galaxy.


title: Higher differentiability for the fractional $p$-Laplacian

abstract: In this work, we study the higher differentiability of solutions to the inhomogeneous fractional $p$-Laplace equation under different regularity assumptions on the data. In the superquadratic case, we extend and sharpen several previous results, while in the subquadratic regime our results constitute completely novel developments even in the homogeneous case. In particular, in the local limit our results are consistent with well-known higher differentiability results for the standard inhomogeneous $p$-Laplace equation. All of our main results remain valid in the vectorial context of fractional $p$-Laplace systems.


title: CausalMMM: Learning Causal Structure for Marketing Mix Modeling

abstract: In online advertising, marketing mix modeling (MMM) is employed to predict the gross merchandise volume (GMV) of brand shops and help decision-makers to adjust the budget allocation of various advertising channels. Traditional MMM methods leveraging regression techniques can fail in handling the complexity of marketing. Although some efforts try to encode the causal structures for better prediction, they have the strict restriction that causal structures are prior-known and unchangeable. In this paper, we define a new causal MMM problem that automatically discovers the interpretable causal structures from data and yields better GMV predictions. To achieve causal MMM, two essential challenges should be addressed: (1) Causal Heterogeneity. The causal structures of different kinds of shops vary a lot. (2) Marketing Response Patterns. Various marketing response patterns i.e., carryover effect and shape effect, have been validated in practice. We argue that causal MMM needs dynamically discover specific causal structures for different shops and the predictions should comply with the prior known marketing response patterns. Thus, we propose CausalMMM that integrates Granger causality in a variational inference framework to measure the causal relationships between different channels and predict the GMV with the regularization of both temporal and saturation marketing response patterns. Extensive experiments show that CausalMMM can not only achieve superior performance of causal structure learning on synthetic datasets with improvements of 5.7%\sim 7.1%, but also enhance the GMV prediction results on a representative E-commerce platform.


title: Optimizing Search Strategies: A Study of Two-Pointer Linear Search Implementation

abstract: This report investigates three fundamental search algorithms: Linear Search, Binary Search, and Two Pointer Search. Linear Search checks each element sequentially, Binary Search divides the search space in half, and Two Pointer Search uses two pointers to scan from both ends of a sorted list. We compare these algorithms in terms of time complexity, space complexity, and practical performance. Our findings demonstrate that while Linear Search is straightforward, it is inefficient for large datasets. Binary Search is efficient for sorted data but requires an initial sorting step. The Two Pointer Search, combining elements of both methods, offers a practical balance of simplicity and efficiency. We propose a novel implementation of the Two Pointer Search algorithm and validate its performance through comprehensive testing on various hardware configurations. The results indicate that our proposed algorithm significantly improves search efficiency, making it suitable for both sorted and unsorted datasets. Future work will extend this algorithm to more complex data structures and real-world applications.


title: Convolutional neural network for Lyman break galaxies classification and redshift regression in DESI (Dark Energy Spectroscopic Instrument)

abstract: DESI is a groundbreaking international project to observe more than 40 million quasars and galaxies over a 5-year period to create a 3D map of the sky. This map will enable us to probe multiple aspects of cosmology, from dark energy to neutrino mass. We are focusing here on one type of object observed by DESI, the Lyman Break Galaxies (LBGs). The aim is to use their spectra to determine whether they are indeed LBGs, and if so, to determine their distance from the Earth using a phenomenon called redshift. This will enable us to place these galaxies on the DESI 3D map. The aim is therefore to develop a convolutional neural network (CNN) inspired by QuasarNET (See arXiv:1808.09955), performing simultaneously a classification (LBG type or not) and a regression task (determine the redshift of the LBGs). Initially, data augmentation techniques such as shifting the spectra in wavelengths, adding noise to the spectra, or adding synthetic spectra were used to increase the model training dataset from 3,019 data to over 66,000. In a second phase, modifications to the QuasarNET architecture, notably through transfer learning and hyperparameter tuning with Bayesian optimization, boosted model performance. Gains of up to 26% were achieved on the Purity/Efficiency curve, which is used to evaluate model performance, particularly in areas with interesting redshifts, at low (around 2) and high (around 4) redshifts. The best model obtained an average score of 94%, compared with 75% for the initial model.


title: On the boundedness of Dunkl multipliers

abstract: In this article we use Littlewood-Paley-Stein theory to prove two versions of Dunkl multiplier theorem when the multiplier $ m $ satisfies a modified H\"ormander condition. When $ m $ is radial we give a simple proof of a known result. For general $ m $ we prove that the Dunkl multiplier operator takes radial functions in $ L^p $ boundedly into $ L^p $ for all $ p \geq 2.$


title: CLIMATELI: Evaluating Entity Linking on Climate Change Data

abstract: Climate Change (CC) is a pressing topic of global importance, attracting increasing attention across research fields, from social sciences to Natural Language Processing (NLP). CC is also discussed in various settings and communication platforms, from academic publications to social media forums. Understanding who and what is mentioned in such data is a first critical step to gaining new insights into CC. We present CLIMATELI (CLIMATe Entity LInking), the first manually annotated CC dataset that links 3,087 entity spans to Wikipedia. Using CLIMATELI (CLIMATe Entity LInking), we evaluate existing entity linking (EL) systems on the CC topic across various genres and propose automated filtering methods for CC entities. We find that the performance of EL models notably lags behind humans at both token and entity levels. Testing within the scope of retaining or excluding non-nominal and/or non-CC entities particularly impacts the models' performances.


title: The diameter of random Schreier graphs

abstract: We give a combinatorial proof of the following theorem. Let $G$ be any finite group acting transitively on a set of cardinality $n$. If $S \subseteq G$ is a random set of size $k$, with $k \geq (\log n)^{1+\varepsilon}$ for some $\varepsilon >0$, then the diameter of the corresponding Schreier graph is $O(\log_k n)$ with high probability. Except for the implicit constant, this result is the best possible.


title: Scheduling with Obligatory Tests

abstract: Motivated by settings such as medical treatments or aircraft maintenance, we consider a scheduling problem with jobs that consist of two operations, a test and a processing part. The time required to execute the test is known in advance while the time required to execute the processing part becomes known only upon completion of the test. We use competitive analysis to study algorithms for minimizing the sum of completion times for $n$ given jobs on a single machine. As our main result, we prove using a novel analysis technique that the natural $1$-SORT algorithm has competitive ratio at most 1.861. For the special case of uniform test times, we show that a simple threshold-based algorithm has competitive ratio at most 1.585. We also prove a lower bound that shows that no deterministic algorithm can be better than $\sqrt{2}$-competitive even in the case of uniform test times.


title: The $L^p$ Poisson-Neumann problem and its relation to the Neumann problem

abstract: We introduce the $L^p$ Poisson-Neumann problem for an uniformly elliptic operator $L=-\rm{div }A\nabla$ in divergence form in a bounded 1-sided Chord Arc Domain $\Omega$, which considers solutions to $Lu=h-\rm{div}\vec{F}$ in $\Omega$ with zero Neumann data on the boundary for $h$ and $\vec F$ in some tent spaces. We give different characterizations of solvability of the $L^p$ Poisson-Neumann problem and its weaker variants, and in particular, we show that solvability of the weak $L^p$ Poisson-Neumann probelm is equivalent to a weak reverse H\"older inequality. We show that the Poisson-Neumman problem is closely related to the $L^p$ Neumann problem, whose solvability is a long-standing open problem. We are able to improve the extrapolation of the $L^p$ Neumann problem from Kenig and Pipher by obtaining an extrapolation result on the Poisson-Neumann problem.


title: Structure of geodesics for Finsler metrics arising from Riemannian g.o. metrics

abstract: Homogeneous geodesics of homogeneous Finsler metrics derived from two or more Riemannian geodesic orbit metrics are investigated. For a broad newly defined family of positively related Riemannian geodesic orbit metrics, geodesic lemma is proved and it is shown that the derived Finsler metrics have also geodesic orbit property. These Finsler metrics belong to the newly defined class of the $\alpha_i$-type metrics which includes in particular the $(\alpha_1,\alpha_2)$ metrics. Geodesic graph for the sphere ${\mathrm{S}}^7={\mathrm{Sp(2)}}{\mathrm{U}}(1)/{\mathrm{Sp(1)}{\mathrm{diag}}{\mathrm{U}}(1)}$ with geodesic orbit Finsler metrics of the new type $(\alpha_1,\alpha_2,\alpha_3)$, arising from two or more Riemannian geodesic orbit metrics, is analyzed in detail. This type of metrics on $S^7$ is one of the missing cases in a previously published classification of geodesic orbit metrics on spheres.


title: A Digital Human Model for Symptom Progression of Vestibular Motion Sickness based on Subjective Vertical Conflict Theory

abstract: Digital human models of motion sickness have been actively developed, among which models based on subjective vertical conflict (SVC) theory are the most actively studied. These models facilitate the prediction of motion sickness in various scenarios such as riding in a car. Most SVC theory models predict the motion sickness incidence (MSI), which is defined as the percentage of people who would vomit with the given specific motion stimulus. However, no model has been developed to describe milder forms of discomfort or specific symptoms of motion sickness, even though predicting milder symptoms is important for applications in automobiles and daily use vehicles. Therefore, the purpose of this study was to build a computational model of symptom progression of vestibular motion sickness based on SVC theory. We focused on a model of vestibular motion sickness with six degrees-of-freedom (6DoF) head motions. The model was developed by updating the output part of the state-of-the-art SVC model, termed the 6DoF-SVC (IN1) model, from MSI to the MIsery SCale (MISC), which is a subjective rating scale for symptom progression. We conducted an experiment to measure the progression of motion sickness during a straight fore-aft motion. It was demonstrated that our proposed method, with the parameters of the output parts optimized by the experimental results, fits well with the observed MISC.


title: Inducing Group Fairness in LLM-Based Decisions

abstract: Prompting Large Language Models (LLMs) has created new and interesting means for classifying textual data. While evaluating and remediating group fairness is a well-studied problem in classifier fairness literature, some classical approaches (e.g., regularization) do not carry over, and some new opportunities arise (e.g., prompt-based remediation). We measure fairness of LLM-based classifiers on a toxicity classification task, and empirically show that prompt-based classifiers may lead to unfair decisions. We introduce several remediation techniques and benchmark their fairness and performance trade-offs. We hope our work encourages more research on group fairness in LLM-based classifiers.


title: Agent-Driven Automatic Software Improvement

abstract: With software maintenance accounting for 50% of the cost of developing software, enhancing code quality and reliability has become more critical than ever. In response to this challenge, this doctoral research proposal aims to explore innovative solutions by focusing on the deployment of agents powered by Large Language Models (LLMs) to perform software maintenance tasks. The iterative nature of agents, which allows for continuous learning and adaptation, can help surpass common challenges in code generation. One distinct challenge is the last-mile problems, errors at the final stage of producing functionally and contextually relevant code. Furthermore, this project aims to surpass the inherent limitations of current LLMs in source code through a collaborative framework where agents can correct and learn from each other's errors. We aim to use the iterative feedback in these systems to further fine-tune the LLMs underlying the agents, becoming better aligned to the task of automated software improvement. Our main goal is to achieve a leap forward in the field of automatic software improvement by developing new tools and frameworks that can enhance the efficiency and reliability of software development.


title: Learning the boundary-to-domain mapping using Lifting Product Fourier Neural Operators for partial differential equations

abstract: Neural operators such as the Fourier Neural Operator (FNO) have been shown to provide resolution-independent deep learning models that can learn mappings between function spaces. For example, an initial condition can be mapped to the solution of a partial differential equation (PDE) at a future time-step using a neural operator. Despite the popularity of neural operators, their use to predict solution functions over a domain given only data over the boundary (such as a spatially varying Dirichlet boundary condition) remains unexplored. In this paper, we refer to such problems as boundary-to-domain problems; they have a wide range of applications in areas such as fluid mechanics, solid mechanics, heat transfer etc. We present a novel FNO-based architecture, named Lifting Product FNO (or LP-FNO) which can map arbitrary boundary functions defined on the lower-dimensional boundary to a solution in the entire domain. Specifically, two FNOs defined on the lower-dimensional boundary are lifted into the higher dimensional domain using our proposed lifting product layer. We demonstrate the efficacy and resolution independence of the proposed LP-FNO for the 2D Poisson equation.


title: Extracting thin film structures of energy materials using transformers

abstract: Neutron-Transformer Reflectometry and Advanced Computation Engine (N-TRACE ), a neural network model using transformer architecture, is introduced for neutron reflectometry data analysis. It offers fast, accurate initial parameter estimations and efficient refinements, improving efficiency and precision for real-time data analysis of lithium-mediated nitrogen reduction for electrochemical ammonia synthesis, with relevance to other chemical transformations and batteries. Despite limitations in generalizing across systems, it shows promises for the use of transformers as the basis for models that could replace trial-and-error approaches to modeling reflectometry data.


title: Unveiling Activity-Travel Patterns through Topological Data Analysis

abstract: In the context of rapid urbanization, understanding the patterns of urban residents' activities and mobility is crucial for optimizing transportation systems and enhancing urban management efficiency. This study addresses the limitations of traditional travel analysis methods in handling high-dimensional and large-scale spatiotemporal data by incorporating Topological Data Analysis (TDA) techniques, specifically using persistent homology. This method allows for the extraction of information from the topological structure of data, enabling the effective identification and analysis of complex spatiotemporal behavior patterns without reducing the data's dimensionality. We utilized mobile signaling data from a community in Shenzhen, which includes detailed geographic and temporal information, providing an ideal sample for analyzing urban residents' behavior patterns. Using our pattern mining framework, we successfully identified five main patterns of residents' activities and travel, revealing daily behavioral habits and reflecting the activity heterogeneity among residents with different socio-economic attributes. These findings not only assist urban planners in better session design but also provide new characteristics for predictive mobility models.


title: Adversarial Contrastive Decoding: Boosting Safety Alignment of Large Language Models via Opposite Prompt Optimization

abstract: With the widespread application of Large Language Models (LLMs), it has become a significant concern to ensure their safety and prevent harmful responses. While current safe-alignment methods based on instruction fine-tuning and Reinforcement Learning from Human Feedback (RLHF) can effectively reduce harmful responses from LLMs, they often require high-quality datasets and heavy computational overhead during model training. Another way to align language models is to modify the logit of tokens in model outputs without heavy training. Recent studies have shown that contrastive decoding can enhance the performance of language models by reducing the likelihood of confused tokens. However, these methods require the manual selection of contrastive models or instruction templates. To this end, we propose Adversarial Contrastive Decoding (ACD), an optimization-based framework to generate two opposite system prompts for prompt-based contrastive decoding. ACD only needs to apply a lightweight prompt tuning on a rather small anchor dataset (< 3 min for each model) without training the target model. Experiments conducted on extensive models and benchmarks demonstrate that the proposed method achieves much better safety performance than previous model training-free decoding methods without sacrificing its original generation ability.


title: Lone Pair Induced 1D Character and Weak Cation-anion Interactions: Two Ingredients for Low Thermal Conductivity in Mixed-anion Metal Chalcohalides

abstract: Mixed-anion compounds, which incorporate multiple types of anions into materials, displays tailored crystal structures and physical/chemical properties, garnering immense interests in various applications such as batteries, catalysis, photovoltaics, and thermoelectrics. However, detailed studies regarding correlations between crystal structure, chemical bonding, and thermal/vibrational properties are rare for these compounds, which limits the exploration of mixed-anion compounds for associated thermal applications. In this work, we investigate the lattice dynamics and thermal transport properties of the metal chalcohalides, CuBiSCl2. A high-purity polycrystalline CuBiSCl2 sample, successfully synthesized via modified solid-state synthetic method, exhibits a low lattice thermal conductivity of 0.9-0.6 W m-1 K-1 from 300 to 573 K. By combining various experimental techniques including 3D electron diffraction with theoretical calculations, we elucidate the origin of low lattice thermal conductivity in CuBiSCl2. The stereo-chemical activity of the 6s2 lone pair of Bi3+ favors an asymmetric environment with neighboring anions involving both short and long bond lengths. This particularity often implies weak bonding, low structure dimensionality, and strong anharmonicity, leading to low lattice thermal conductivity. In addition, the strong two-fold linear S-Cu-S coordination with weak Cu -- Cl interactions induces large anisotropic vibration of Cu or structural disorder, which enables strong phonon-phonon scattering and decreases lattice thermal conductivity. The investigations into lattice dynamics and thermal transport properties of CuBiSCl2 broadens the scope of the existing mixed-anion compounds suitable for the associated thermal applications, offering a new avenue for the search of low thermal conductivity materials in low-cost mixed-anion compounds.


title: Bandits with Preference Feedback: A Stackelberg Game Perspective

abstract: Bandits with preference feedback present a powerful tool for optimizing unknown target functions when only pairwise comparisons are allowed instead of direct value queries. This model allows for incorporating human feedback into online inference and optimization and has been employed in systems for fine-tuning large language models. The problem is well understood in simplified settings with linear target functions or over finite small domains that limit practical interest. Taking the next step, we consider infinite domains and nonlinear (kernelized) rewards. In this setting, selecting a pair of actions is quite challenging and requires balancing exploration and exploitation at two levels: within the pair, and along the iterations of the algorithm. We propose MAXMINLCB, which emulates this trade-off as a zero-sum Stackelberg game, and chooses action pairs that are informative and yield favorable rewards. MAXMINLCB consistently outperforms existing algorithms and satisfies an anytime-valid rate-optimal regret guarantee. This is due to our novel preference-based confidence sequences for kernelized logistic estimators.


title: The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources

abstract: Foundation model development attracts a rapidly expanding body of contributors, scientists, and applications. To help shape responsible development practices, we introduce the Foundation Model Development Cheatsheet: a growing collection of 250+ tools and resources spanning text, vision, and speech modalities. We draw on a large body of prior work to survey resources (e.g. software, documentation, frameworks, guides, and practical tools) that support informed data selection, processing, and understanding, precise and limitation-aware artifact documentation, efficient model training, advance awareness of the environmental impact from training, careful model evaluation of capabilities, risks, and claims, as well as responsible model release, licensing and deployment practices. We hope this curated collection of resources helps guide more responsible development. The process of curating this list, enabled us to review the AI development ecosystem, revealing what tools are critically missing, misused, or over-used in existing practices. We find that (i) tools for data sourcing, model evaluation, and monitoring are critically under-serving ethical and real-world needs, (ii) evaluations for model safety, capabilities, and environmental impact all lack reproducibility and transparency, (iii) text and particularly English-centric analyses continue to dominate over multilingual and multi-modal analyses, and (iv) evaluation of systems, rather than just models, is needed so that capabilities and impact are assessed in context.


title: Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers

abstract: Accommodating long sequences efficiently in autoregressive Transformers, especially within an extended context window, poses significant challenges due to the quadratic computational complexity and substantial KV memory requirements inherent in self-attention mechanisms. In this work, we introduce SPARSEK Attention, a novel sparse attention mechanism designed to overcome these computational and memory obstacles while maintaining performance. Our approach integrates a scoring network and a differentiable top-k mask operator, SPARSEK, to select a constant number of KV pairs for each query, thereby enabling gradient-based optimization. As a result, SPARSEK Attention offers linear time complexity and constant memory footprint during generation. Experimental results reveal that SPARSEK Attention outperforms previous sparse attention methods and provides significant speed improvements during both training and inference, particularly in language modeling and downstream tasks. Furthermore, our method can be seamlessly integrated into pre-trained Large Language Models (LLMs) with minimal fine-tuning, offering a practical solution for effectively managing long-range dependencies in diverse applications.