title: Availing non-Markovian dynamics in effective negative temperature-based transient quantum Otto engines

abstract: We demonstrate that the efficiency of effective negative temperature-based quantum Otto engines, already known to outperform their traditional counterparts operating with positive-temperature thermal reservoirs, can be further improved by terminating the isochoric strokes before the working substance reaches perfect equilibrium with its environment. Our investigation encompasses both Markovian and non-Markovian dynamics during these finite-time isochoric processes while considering a weak coupling between the working substance and the reservoirs. We assess the performance of these engines as they undergo a transition from the Markovian to the non-Markovian regime using two figures of merit: maximum achievable efficiency at a certain finite time during the isochoric heating stroke, and overall performance of the engine over an extended period during the transient phase of this stroke. We show that the maximum efficiency increases with the increase of non-Markovianity. However, the overall engine performance decreases as non-Markovianity increases. Additionally, we discover the existence of effective negative temperature-based necessarily transient quantum Otto engines. These engines operate within an extended operational domain, reaching into temperature ranges where conventional effective negative temperature-based quantum Otto engines, which rely on perfect thermalization during the isochoric strokes, are unable to function. Furthermore, this extended operational domain of an effective negative temperature-based necessarily transient quantum Otto engine increases as non-Markovianity becomes more pronounced.


title: AccEq-DRT: Planning Demand-Responsive Transit to reduce inequality of accessibility

abstract: Accessibility measures how well a location is connected to surrounding opportunities. We focus on accessibility provided by Public Transit (PT). There is an evident inequality in the distribution of accessibility between city centers or close to main transportation corridors and suburbs. In the latter, poor PT service leads to a chronic car-dependency. Demand-Responsive Transit (DRT) is better suited for low-density areas than conventional fixed-route PT. However, its potential to tackle accessibility inequality has not yet been exploited. On the contrary, planning DRT without care to inequality (as in the methods proposed so far) can further improve the accessibility gap in urban areas. To the best of our knowledge this paper is the first to propose a DRT planning strategy, which we call AccEq-DRT, aimed at reducing accessibility inequality, while ensuring overall efficiency. To this aim, we combine a graph representation of conventional PT and a Continuous Approximation (CA) model of DRT. The two are combined in the same multi-layer graph, on which we compute accessibility. We then devise a scoring function to estimate the need of each area for an improvement, appropriately weighting population density and accessibility. Finally, we provide a bilevel optimization method, where the upper level is a heuristic to allocate DRT buses, guided by the scoring function, and the lower level performs traffic assignment. Numerical results in a simplified model of Montreal show that inequality, measured with the Atkinson index, is reduced by up to 34\%. Keywords: DRT Public, Transportation, Accessibility, Continuous Approximation, Network Design


title: Toward a Plug-and-Play Vision-Based Grasping Module for Robotics

abstract: Despite recent advancements in AI for robotics, grasping remains a partially solved challenge, hindered by the lack of benchmarks and reproducibility constraints. This paper introduces a vision-based grasping framework that can easily be transferred across multiple manipulators. Leveraging Quality-Diversity (QD) algorithms, the framework generates diverse repertoires of open-loop grasping trajectories, enhancing adaptability while maintaining a diversity of grasps. This framework addresses two main issues: the lack of an off-the-shelf vision module for detecting object pose and the generalization of QD trajectories to the whole robot operational space. The proposed solution combines multiple vision modules for 6DoF object detection and tracking while rigidly transforming QD-generated trajectories into the object frame. Experiments on a Franka Research 3 arm and a UR5 arm with a SIH Schunk hand demonstrate comparable performance when the real scene aligns with the simulation used for grasp generation. This work represents a significant stride toward building a reliable vision-based grasping module transferable to new platforms, while being adaptable to diverse scenarios without further training iterations.


title: Evidence of pair production of longitudinally polarised vector bosons and study of CP properties in $ZZ \to 4\ell$ events with the ATLAS detector at $\sqrt{s} = 13$ TeV

abstract: A study of the polarisation and CP properties in $ZZ$ production is presented. The used data set corresponds to an integrated luminosity of 140 fb$^{-1}$ of proton-proton collisions at a centre-of-mass energy of $13$ TeV recorded by the ATLAS detector at the Large Hadron Collider. The $ZZ$ candidate events are reconstructed using two same-flavour opposite-charge electron or muon pairs. The production of two longitudinally polarised $Z$ bosons is measured with a significance of 4.3 standard deviations, and its cross-section is measured in a fiducial phase space to be $2.45 \pm 0.60$ fb, consistent with the next-to-leading-order Standard Model prediction. The inclusive differential cross-section as a function of a CP-sensitive angular observable is also measured. The results are used to constrain anomalous CP-odd neutral triple gauge couplings.


title: The long-range interacting Fermi polaron

abstract: We construct the simplest density functional for the problem of a single impurity interacting with a Fermi gas via a long--ranged potential using the Thomas--Fermi approach. We find that the Fermi polaron is fully bosonized in two dimensions, as the model results in a suitable Landau--Pekar functional known from the Bose polaron problem which describes a self--interacting impurity. In other dimensions, the impurity self--interacts with an infinite number of its own images, and no bosonization occurs. We discuss applications of our theory for the $2d$ exciton--polaron and the ionic polaron problem and compute the effective mass for these cases, finding a self--trapping transition with order depending on the dimensionality.


title: Fair Feature Importance Scores for Interpreting Tree-Based Methods and Surrogates

abstract: Across various sectors such as healthcare, criminal justice, national security, finance, and technology, large-scale machine learning (ML) and artificial intelligence (AI) systems are being deployed to make critical data-driven decisions. Many have asked if we can and should trust these ML systems to be making these decisions. Two critical components are prerequisites for trust in ML systems: interpretability, or the ability to understand why the ML system makes the decisions it does, and fairness, which ensures that ML systems do not exhibit bias against certain individuals or groups. Both interpretability and fairness are important and have separately received abundant attention in the ML literature, but so far, there have been very few methods developed to directly interpret models with regard to their fairness. In this paper, we focus on arguably the most popular type of ML interpretation: feature importance scores. Inspired by the use of decision trees in knowledge distillation, we propose to leverage trees as interpretable surrogates for complex black-box ML models. Specifically, we develop a novel fair feature importance score for trees that can be used to interpret how each feature contributes to fairness or bias in trees, tree-based ensembles, or tree-based surrogates of any complex ML system. Like the popular mean decrease in impurity for trees, our Fair Feature Importance Score is defined based on the mean decrease (or increase) in group bias. Through simulations as well as real examples on benchmark fairness datasets, we demonstrate that our Fair Feature Importance Score offers valid interpretations for both tree-based ensembles and tree-based surrogates of other ML systems.


title: An In-Context Learning Agent for Formal Theorem-Proving

abstract: We present an in-context learning agent for formal theorem-proving in environments like Lean and Coq. Current state-of-the-art models for the problem are finetuned on environment-specific proof data. By contrast, our approach, called COPRA, repeatedly asks a high-capacity, general-purpose large language model (GPT-4) to propose tactic applications from within a stateful backtracking search. Proposed tactics are executed in the underlying proof environment. Feedback from the execution is used to build the prompt for the next model query, along with selected information from the search history and lemmas retrieved from an external database. We evaluate our implementation of COPRA on the miniF2F benchmark for Lean and a set of Coq tasks from the CompCert project. On these benchmarks, COPRA significantly outperforms few-shot invocations of GPT-4. It also compares favorably against finetuning-based approaches, outperforming REPROVER, a state-of-the-art finetuned approach for Lean, in terms of the pass@1 metric. Our code and data are available at https://github.com/trishullab/copra.


title: Integrating Transformations in Probabilistic Circuits

abstract: This study addresses the predictive limitation of probabilistic circuits and introduces transformations as a remedy to overcome it. We demonstrate this limitation in robotic scenarios. We motivate that independent component analysis is a sound tool to preserve the independence properties of probabilistic circuits. Our approach is an extension of joint probability trees, which are model-free deterministic circuits. By doing so, it is demonstrated that the proposed approach is able to achieve higher likelihoods while using fewer parameters compared to the joint probability trees on seven benchmark data sets as well as on real robot data. Furthermore, we discuss how to integrate transformations into tree-based learning routines. Finally, we argue that exact inference with transformed quantile parameterized distributions is not tractable. However, our approach allows for efficient sampling and approximate inference.


title: Computation of viscoelastic shear shock waves using finite volume schemes with artificial compressibility

abstract: The formation of shear shock waves in the brain has been proposed as one of the plausible explanations for deep intracranial injuries. In fact, such singular solutions emerge naturally in soft viscoelastic tissues under dynamic loading conditions. To improve our understanding of the mechanical processes at hand, the development of dedicated computational models is needed. The present study concerns three-dimensional numerical models of incompressible viscoelastic solids whose motion is analysed by means of shock-capturing finite volume methods. More specifically, we focus on the use of the artificial compressibility method, a technique that has been frequently employed in computational fluid dynamics. The material behaviour is deduced from the Fung--Simo quasi-linear viscoelasiticity theory (QLV) where the elastic response is of Yeoh type. We analyse the accuracy of the method and demonstrate its applicability for the study of nonlinear wave propagation in soft solids. The numerical results cover accuracy tests, shock formation and wave focusing.


title: Mapping the DeFi Crime Landscape: An Evidence-based Picture

abstract: Over the past years, decentralized finance (DeFi) has been the target of numerous profit-driven crimes. However, until now, the full prevalence and cumulative impact of these crimes have not been assessed. This study provides a first comprehensive assessment of profit-driven crimes targeting the DeFi sector. To achieve this, we collected data on 1155 crime events from 2017 to 2022. Of these, 1050 were related to the DeFi industry and 105 to the centralized finance (CeFi) industry. Focusing on the former, a taxonomy was developed to clarify the similarities and differences among these crimes. All events were mapped onto the DeFi stack to assess the impacted technical layers, and the financial damages were quantified to gauge their scale. The findings show that the entire cryptoasset industry has suffered a minimum loss of US$30B, with two thirds related to centralized finance (CeFi) and one third to DeFi. Focusing solely on the latter, the results highlight that during an attack, a DeFi actor (an entity developing a DeFi technology) can serve as a direct target, as a perpetrator, or as an intermediary. The findings show that DeFi actors are the first victims of crimes targeting the DeFi industry: 52% of crime events targeted them, primarily due to technical vulnerabilities at the protocol layer, and these events accounted for 83% of all recorded financial damages. On the other hand, in 40% of crime events, DeFi actors were themselves malicious perpetrators, predominantly misusing contracts at the cryptoasset layer (e.g., rug pull scams). However, these events accounted for only 17% of all financial damages. The study's findings offer a preliminary assessment of the size and scope of crime events within the DeFi sector and highlight the vulnerable position of DeFi actors in the ecosystem.


title: Asymptotically free sketched ridge ensembles: Risks, cross-validation, and tuning

abstract: We employ random matrix theory to establish consistency of generalized cross validation (GCV) for estimating prediction risks of sketched ridge regression ensembles, enabling efficient and consistent tuning of regularization and sketching parameters. Our results hold for a broad class of asymptotically free sketches under very mild data assumptions. For squared prediction risk, we provide a decomposition into an unsketched equivalent implicit ridge bias and a sketching-based variance, and prove that the risk can be globally optimized by only tuning sketch size in infinite ensembles. For general subquadratic prediction risk functionals, we extend GCV to construct consistent risk estimators, and thereby obtain distributional convergence of the GCV-corrected predictions in Wasserstein-2 metric. This in particular allows construction of prediction intervals with asymptotically correct coverage conditional on the training data. We also propose an "ensemble trick" whereby the risk for unsketched ridge regression can be efficiently estimated via GCV using small sketched ridge ensembles. We empirically validate our theoretical results using both synthetic and real large-scale datasets with practical sketches including CountSketch and subsampled randomized discrete cosine transforms.


title: Transferring speech-generic and depression-specific knowledge for Alzheimer's disease detection

abstract: The detection of Alzheimer's disease (AD) from spontaneous speech has attracted increasing attention while the sparsity of training data remains an important issue. This paper handles the issue by knowledge transfer, specifically from both speech-generic and depression-specific knowledge. The paper first studies sequential knowledge transfer from generic foundation models pretrained on large amounts of speech and text data. A block-wise analysis is performed for AD diagnosis based on the representations extracted from different intermediate blocks of different foundation models. Apart from the knowledge from speech-generic representations, this paper also proposes to simultaneously transfer the knowledge from a speech depression detection task based on the high comorbidity rates of depression and AD. A parallel knowledge transfer framework is studied that jointly learns the information shared between these two tasks. Experimental results show that the proposed method improves AD and depression detection, and produces a state-of-the-art F1 score of 0.928 for AD diagnosis on the commonly used ADReSSo dataset.


title: Stationary trajectories in Minkowski spacetimes

abstract: We determine the conjugacy classes of the Poincar\'e group $\mathrm{ISO}^+(n,1)$ and apply this to classify the stationary trajectories of Minkowski spacetimes in terms of timelike Killing vectors. Stationary trajectories are the orbits of timelike Killing vectors and, equivalently, the solutions to Frenet-Serret equations with constant curvature coefficients. We extend the $3+1$ Minkowski spacetime Frenet-Serret equations due to Letaw to Minkowski spacetimes of arbitrary dimension. We present the explicit families of stationary trajectories in $4+1$ Minkowski spacetime.


title: SwimXYZ: A large-scale dataset of synthetic swimming motions and videos

abstract: Technologies play an increasingly important role in sports and become a real competitive advantage for the athletes who benefit from it. Among them, the use of motion capture is developing in various sports to optimize sporting gestures. Unfortunately, traditional motion capture systems are expensive and constraining. Recently developed computer vision-based approaches also struggle in certain sports, like swimming, due to the aquatic environment. One of the reasons for the gap in performance is the lack of labeled datasets with swimming videos. In an attempt to address this issue, we introduce SwimXYZ, a synthetic dataset of swimming motions and videos. SwimXYZ contains 3.4 million frames annotated with ground truth 2D and 3D joints, as well as 240 sequences of swimming motions in the SMPL parameters format. In addition to making this dataset publicly available, we present use cases for SwimXYZ in swimming stroke clustering and 2D pose estimation.


title: Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversion

abstract: Transformer models can face practical limitations due to their high computational requirements. At the same time, such models exhibit significant activation sparsity, which can be leveraged to reduce the inference cost by converting parts of the network into equivalent Mixture-of-Experts (MoE) layers. Despite the crucial role played by activation sparsity, its impact on this process remains unexplored. In particular, we show that the efficiency of the conversion can be significantly enhanced by a proper regularization of the activation sparsity of the base model. Moreover, motivated by the high variance of the number of activated neurons for different inputs, we introduce a more effective dynamic-k expert selection rule that adjusts the number of executed experts on a per-token basis. Finally, we extend this approach to multi-head attention projections, which results in additional savings compared to only converting the FFN blocks. The proposed method, Dense to Dynamic-$k$ Mixture-of-Experts (D2DMoE), outperforms existing approaches on common NLP and vision tasks, allowing us to save up to 60% of inference cost without significantly affecting model performance.


title: Einsteinian gravitational concepts throughout secondary school

abstract: Einstein's theory of relativity is largely thought of as one of the most important discoveries of the 20$^{th}$ century and continues to pass observational tests over 100 years later. Yet, it is Newtonian gravity, a 350 year old formalism proven to be less accurate than relativity, which is taught in schools. It has been shown that Einsteinian gravitational concepts can be well understood by students in both primary and secondary education. In this paper, a cross-section of students from Yr 7-13 enrolled in an English secondary school took part in an intervention designed to introduce the idea of gravity from spacetime curvature. The overall aim of this work is to assess the viability of including relativity in the secondary curriculum and to ascertain which year this material would be best placed in. We determine that all year groups where able to appreciate the effects of curvature to some extent. Visual demonstrations aided conceptual understanding at Yr 7-8 level, but this does not have a strong effect on their ideas around the source of the gravitational force. Participants in Yr 9-13 were able to understand concepts beyond those introduced in the demonstrations. However, a deeper understanding of curvature as the source of the gravitational force is not seen until years 12 & 13. We find that those in Yr 13 have the best overall understanding of the concepts introduced during our intervention.


title: Amortizing intractable inference in large language models

abstract: Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This limits tractable querying of this knowledge to start-to-end autoregressive sampling. However, many tasks of interest -- including sequence continuation, infilling, and other forms of constrained generation -- involve sampling from intractable posterior distributions. We address this limitation by using amortized Bayesian inference to sample from these intractable posteriors. Such amortization is algorithmically achieved by fine-tuning LLMs via diversity-seeking reinforcement learning algorithms: generative flow networks (GFlowNets). We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training and reward-maximizing policy optimization. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem and demonstrate that our approach enables data-efficient adaptation of LLMs to tasks that require multi-step rationalization and tool use.


title: Enhanced Backpressure Routing Using Wireless Link Features

abstract: Backpressure (BP) routing is a well-established framework for distributed routing and scheduling in wireless multi-hop networks. However, the basic BP scheme suffers from poor end-to-end delay due to the drawbacks of slow startup, random walk, and the last packet problem. Biased BP with shortest path awareness can address the first two drawbacks, and sojourn time-based backlog metrics were proposed for the last packet problem. Furthermore, these BP variations require no additional signaling overhead in each time step compared to the basic BP. In this work, we further address three long-standing challenges associated with the aforementioned low-cost BP variations, including optimal scaling of the biases, bias maintenance under mobility, and incorporating sojourn time awareness into biased BP. Our analysis and experimental results show that proper scaling of biases can be achieved with the help of common link features, which can effectively reduce end-to-end delay of BP by mitigating the random walk of packets under low-to-medium traffic, including the last packet scenario. In addition, our low-overhead bias maintenance scheme is shown to be effective under mobility, and our bio-inspired sojourn time-aware backlog metric is demonstrated to be more efficient and effective for the last packet problem than existing approaches when incorporated into biased BP.


title: Fundamental groups of highly symmetrical curves and Fermat line arrangments

abstract: We showcase a computation of the fundamental group of $\mathbb{CP}^2 - \mathcal{C}$ when $\mathcal{C}$ is a curve admitting a lot of symmetries. In particular, let $\mathcal{C}$ denote the Fermat line arrangement in $\mathbb{CP}^2$ defined by the vanishing locus of homogeneous polynomial $(x^n-y^n)(y^n-z^n)(z^n-x^n)$. In this article, we compute the fundamental group $\pi_1(\mathbb{CP}^2-\mathcal{C})$ of complement of this line arrangement in the complex projective plane. We show that this group is semi-direct product of $G$ and $F_n$, i.e., $\pi_1(\mathbb{CP}^2-\mathcal{C}, \overline{\epsilon}) = G \rtimes F_{n}$, where $G$ and $F_n$ is defined in 4.3, and 1.2 respectively.


title: Swordfish: A Framework for Evaluating Deep Neural Network-based Basecalling using Computation-In-Memory with Non-Ideal Memristors

abstract: Basecalling, an essential step in many genome analysis studies, relies on large Deep Neural Networks (DNNs) to achieve high accuracy. Unfortunately, these DNNs are computationally slow and inefficient, leading to considerable delays and resource constraints in the sequence analysis process. A Computation-In-Memory (CIM) architecture using memristors can significantly accelerate the performance of DNNs. However, inherent device non-idealities and architectural limitations of such designs can greatly degrade the basecalling accuracy, which is critical for accurate genome analysis. To facilitate the adoption of memristor-based CIM designs for basecalling, it is important to (1) conduct a comprehensive analysis of potential CIM architectures and (2) develop effective strategies for mitigating the possible adverse effects of inherent device non-idealities and architectural limitations. This paper proposes Swordfish, a novel hardware/software co-design framework that can effectively address the two aforementioned issues. Swordfish incorporates seven circuit and device restrictions or non-idealities from characterized real memristor-based chips. Swordfish leverages various hardware/software co-design solutions to mitigate the basecalling accuracy loss due to such non-idealities. To demonstrate the effectiveness of Swordfish, we take Bonito, the state-of-the-art (i.e., accurate and fast), open-source basecaller as a case study. Our experimental results using Sword-fish show that a CIM architecture can realistically accelerate Bonito for a wide range of real datasets by an average of 25.7x, with an accuracy loss of 6.01%.


title: A Marketplace Price Anomaly Detection System at Scale

abstract: Online marketplaces execute large volume of price updates that are initiated by individual marketplace sellers each day on the platform. This price democratization comes with increasing challenges with data quality. Lack of centralized guardrails that are available for a traditional online retailer causes a higher likelihood for inaccurate prices to get published on the website, leading to poor customer experience and potential for revenue loss. We present MoatPlus (Masked Optimal Anchors using Trees, Proximity-based Labeling and Unsupervised Statistical-features), a scalable price anomaly detection framework for a growing marketplace platform. The goal is to leverage proximity and historical price trends from unsupervised statistical features to generate an upper price bound. We build an ensemble of models to detect irregularities in price-based features, exclude irregular features and use optimized weighting scheme to build a reliable price bound in real-time pricing pipeline. We observed that our approach improves precise anchor coverage by up to 46.6% in high-vulnerability item subsets


title: A Core Calculus for Documents

abstract: Passive documents and active programs now widely comingle. Document languages include Turing-complete programming elements, and programming languages include sophisticated document notations. However, there are no formal foundations that model these languages. This matters because the interaction between document and program can be subtle and error-prone. In this paper we describe several such problems, then taxonomize and formalize document languages as levels of a document calculus. We employ the calculus as a foundation for implementing complex features such as reactivity, as well as for proving theorems about the boundary of content and computation. We intend for the document calculus to provide a theoretical basis for new document languages, and to assist designers in cleaning up the unsavory corners of existing languages.


title: MBTFNet: Multi-Band Temporal-Frequency Neural Network For Singing Voice Enhancement

abstract: A typical neural speech enhancement (SE) approach mainly handles speech and noise mixtures, which is not optimal for singing voice enhancement scenarios. Music source separation (MSS) models treat vocals and various accompaniment components equally, which may reduce performance compared to the model that only considers vocal enhancement. In this paper, we propose a novel multi-band temporal-frequency neural network (MBTFNet) for singing voice enhancement, which particularly removes background music, noise and even backing vocals from singing recordings. MBTFNet combines inter and intra-band modeling for better processing of full-band signals. Dual-path modeling are introduced to expand the receptive field of the model. We propose an implicit personalized enhancement (IPE) stage based on signal-to-noise ratio (SNR) estimation, which further improves the performance of MBTFNet. Experiments show that our proposed model significantly outperforms several state-of-the-art SE and MSS models.


title: First Adiabatic Invariant and the Brightness Temperature of Relativistic Jets

abstract: Assuming that the first adiabatic invariant for radiating particles in relativistic jets is conserved, we investigate the change in brightness temperature along the jet axis. We show that in this case the observed break in the dependence of the brightness temperature on the distance to the central engine can be explained.


title: Rapid transform optimisation strategy for decoherence-protected quantum register in diamond

abstract: Decoherence-protected spins associated with nitrogen-vacancy color centers in diamond possess remarkable long coherence time, which make them one of the most promising and robust quantum registers. The current demand is to explore practical rapid control strategies for preparing and manipulating the such register. Our work provides all-microwave control strategies optimized using multiple optimization methods to significantly reduce the processing time by $80\%$ with a set of smooth near-zero-endpoints control fields that are shown to be experimentally realizable. Furthermore, we optimize and analyze the robustness of these strategies under frequency and amplitude imperfections of the control fields, during which process we use only $16$ samples to give a fair estimation of the robustness map with $2500$ pixels. Overall, we provide a ready-to-implement recipe to facilitate high-performance information processing via decoherence-protected quantum register for future quantum technology applications.


title: Rotation Curve Fitting Model

abstract: One key piece of evidence for dark matter is the flat rotation curve problem: the disagreement between measured galactic rotation curves and their luminous mass. A novel solution to this problem is presented here. A model of relativistic frame effects on Doppler shifts due to the slightly curved frames of an emitting galaxy and the Milky Way is derived. This model predicts observed Doppler shifted spectra (in excess of the luminous mass) based only on the observed luminous matter profile and one free model parameter. Fits to the 175 galaxies reported in the SPARC database of galactic rotation profiles and accurate photometry measurements are compared between this novel model and dark matter and MOND (RAR) models. We find on the SPARC sample of 175 galaxies; that MOND-RAR has an average reduced chisquare of $\chi^2_r = 4.22$ for 175 galaxies fitted, the isothermal dark matter model has $\chi^2_r = 1.90$ for 165 galaxies fitted, and the new model we present has $\chi^2_r = 2.39$ for 172 galaxies fitted. Implications of this model are discussed.


title: Confronting Reward Model Overoptimization with Constrained RLHF

abstract: Large language models are typically aligned with human preferences by optimizing $\textit{reward models}$ (RMs) fitted to human feedback. However, human preferences are multi-faceted, and it is increasingly common to derive reward from a composition of simpler reward models which each capture a different aspect of language quality. This itself presents a challenge, as it is difficult to appropriately weight these component RMs when combining them. Compounding this difficulty, because any RM is only a proxy for human evaluation, this process is vulnerable to $\textit{overoptimization}$, wherein past a certain point, accumulating higher reward is associated with worse human ratings. In this paper, we perform, to our knowledge, the first study on overoptimization in composite RMs, showing that correlation between component RMs has a significant effect on the locations of these points. We then introduce an approach to solve this issue using constrained reinforcement learning as a means of preventing the agent from exceeding each RM's threshold of usefulness. Our method addresses the problem of weighting component RMs by learning dynamic weights, naturally expressed by Lagrange multipliers. As a result, each RM stays within the range at which it is an effective proxy, improving evaluation performance. Finally, we introduce an adaptive method using gradient-free optimization to identify and optimize towards these points during a single run.


title: Rock anisotropy promotes hydraulic fracture containment at depth

abstract: We report laboratory experiments and numerical simulations demonstrating that the anisotropic characteristics of rocks play a major role in the elongation of hydraulic fractures propagating in a plane perpendicular to bedding. Transverse anisotropy leads to larger hydraulic fracture extension in the parallel-to-bedding/divider direction compared to the perpendicular-to-bedding/arrester direction. This directly promotes vertical containment of hydraulic fractures in most sedimentary basins worldwide even in the absence of any favorable in-situ stress contrasts or other material heterogeneities. More importantly, the ratio of the energy dissipated in fluid viscous flow in the fracture to the energy dissipated in the creation of new surfaces is found to play a critical role on fracture elongation, with fracture-energy dominated hydraulic fractures being the most elongated while the viscous dominated ones remain more circular. These results open the door to a better engineering and control of hydraulic fractures containment at depth in view of the competition between material anisotropy and injection parameters (fluid viscosity and rate of injection).


title: Epitaxial Y1Ba2Cu3O7 thin films on CeO2 buffer layers on sapphire substrates

abstract: Pulsed laser deposition has been used to deposit Y1Ba2Cu3O7 layer on CeO2 buffer layers on (1102) sapphire. Both layers are epitaxial with the (110) direction of the CeO2 layer aligned with the <2021> direction of the sapphire substrate. The c-axis Y1Ba2Cu3O7 layer has its <100> direction aligned with the <110> direction of the CeO2. Cross-sectional transmission electron microscopy shows the epitaxy to be coherent and the interfaces to be abrupt at an atomic level. The best films have a critical current of 9 e6 A/cm2 at 4.2 K and lower microwave surface resistance than copper at 77 K and at a frequency of 31 GHz.


title: Near-linear Time Dispersion of Mobile Agents

abstract: Consider that there are $k\le n$ agents in a simple, connected, and undirected graph $G=(V,E)$ with $n$ nodes and $m$ edges. The goal of the dispersion problem is to move these $k$ agents to distinct nodes. Agents can communicate only when they are at the same node, and no other means of communication such as whiteboards are available. We assume that the agents operate synchronously. We consider two scenarios: when all agents are initially located at any single node (rooted setting) and when they are initially distributed over any one or more nodes (general setting). Kshemkalyani and Sharma presented a dispersion algorithm for the general setting, which uses $O(m_k)$ time and $\log(k+\delta)$ bits of memory per agent [OPODIS 2021]. Here, $m_k$ is the maximum number of edges in any induced subgraph of $G$ with $k$ nodes, and $\delta$ is the maximum degree of $G$. This algorithm is the fastest in the literature, as no algorithm with $o(m_k)$ time has been discovered even for the rooted setting. In this paper, we present faster algorithms for both the rooted and general settings. First, we present an algorithm for the rooted setting that solves the dispersion problem in $O(k\log \min(k,\delta))=O(k\log k)$ time using $O(\log \delta)$ bits of memory per agent. Next, we propose an algorithm for the general setting that achieves dispersion in $O(k (\log k)\cdot (\log \min(k,\delta))=O(k \log^2 k)$ time using $O(\log (k+\delta))$ bits. Finally, for the rooted setting, we give a time-optimal, i.e.,$O(k)$-time, algorithm with $O(\delta)$ bits of space per agent.


title: Fock bundles and Hitchin components

abstract: We introduce the concept of a Fock bundle, a smooth principal bundle over a surface equipped with a special kind of adjoint-valued 1-form, as a new tool for studying character varieties of surface groups. Although similar to Higgs bundles, the crucial difference is that no complex structure is fixed on the underlying surface. Fock bundles are the gauge-theoretic realization of higher complex structures. We construct a canonical connection to a Fock bundle equipped with compatible symmetric pairing and hermitian structure. The space of flat Fock bundles maps to the character variety of the split real form. Determining the hermitian structure such that this connection is flat gives a non-linear PDE similar to Hitchin's equation. We explicitly construct solutions for Fock bundles in the Fuchsian locus. Ellipticity of the relevant linear operator provides a map from a neighborhood of the Fuchsian locus in the space of higher complex structures modulo higher diffeomorphisms to a neighborhood of the Fuchsian locus in the Hitchin component.


title: Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference

abstract: Latent Diffusion models (LDMs) have achieved remarkable results in synthesizing high-resolution images. However, the iterative sampling process is computationally intensive and leads to slow generation. Inspired by Consistency Models (song et al.), we propose Latent Consistency Models (LCMs), enabling swift inference with minimal steps on any pre-trained LDMs, including Stable Diffusion (rombach et al). Viewing the guided reverse diffusion process as solving an augmented probability flow ODE (PF-ODE), LCMs are designed to directly predict the solution of such ODE in latent space, mitigating the need for numerous iterations and allowing rapid, high-fidelity sampling. Efficiently distilled from pre-trained classifier-free guided diffusion models, a high-quality 768 x 768 2~4-step LCM takes only 32 A100 GPU hours for training. Furthermore, we introduce Latent Consistency Fine-tuning (LCF), a novel method that is tailored for fine-tuning LCMs on customized image datasets. Evaluation on the LAION-5B-Aesthetics dataset demonstrates that LCMs achieve state-of-the-art text-to-image generation performance with few-step inference. Project Page: https://latent-consistency-models.github.io/


title: Tuning topological superconductivity within the $t$-$J$-$U$ model of twisted bilayer cuprates

abstract: We carry out a theoretical study of unconventional superconductivity in twisted bilayer cuprates as a function of electron density and layer twist angle. The bilayer $t$-$J$-$U$ model is employed and analyzed within the framework of a generalized variational wave function approach in the statistically-consistent Gutzwiller formulation. The constructed phase diagram encompasses both gapless $d$-wave state (reflecting the pairing symmetry of untwisted copper-oxides) and gapped $d+\mathrm{e}^{i\varphi}d$ phase that breaks spontaneously time-reversal-symmetry (TRS) and is characterized by nontrivial Chern number. We find that $d+\mathrm{e}^{i\varphi}d$ state occupies a non-convex butterfly-shaped region in the doping vs. twist-angle plane, and demonstrate the presence of previously unreported reentrant TRS-breaking phase on the underdoped side of the phase diagram. This circumstance supports the emergence of topological superconductivity for fine-tuned twist angles away from $45^\circ$. Our analysis of the microscopically derived Landau free energy functional points toward sensitivity of the superconducting order parameter to small perturbations close to the topological state boundary.


title: 2$b$ or not 2$b$: on the rejection of $g\to b\bar b$ jets

abstract: Motivated by new physics models which lead to final states containing a high multiplicity of bottom and top quarks, we develop a tagging strategy to suppress reducible and non-reducible multi-jet backgrounds. The idea takes advantage of the properties of light parton showers and of the gluon fragmentation into heavy quarks to reject jets that do not originate from a bottom quark.


title: Hermes: Unlocking Security Analysis of Cellular Network Protocols by Synthesizing Finite State Machines from Natural Language Specifications

abstract: In this paper, we present Hermes, an end-to-end framework to automatically generate formal representations from natural language cellular specifications. We first develop a neural constituency parser, NEUTREX, to process transition-relevant texts and extract transition components (i.e., states, conditions, and actions). We also design a domain-specific language to translate these transition components to logical formulas by leveraging dependency parse trees. Finally, we compile these logical formulas to generate transitions and create the formal model as finite state machines. To demonstrate the effectiveness of Hermes, we evaluate it on 4G NAS, 5G NAS, and 5G RRC specifications and obtain an overall accuracy of 81-87%, which is a substantial improvement over the state-of-the-art. Our security analysis of the extracted models uncovers 3 new vulnerabilities and identifies 19 previous attacks in 4G and 5G specifications, and 7 deviations in commercial 4G basebands.


title: Supernova Remnants in the Irregular Galaxy NGC4449

abstract: The nearby irregular galaxy NGC4449 has a star formation rate of about 0.4 solar masses/yr and should host of order 70 SNRs younger than 20,000 years, a typical age for SNRs expanding into to an ISM with unit density to reach the radiative phase. We have carried out an optical imaging and spectroscopic survey in an attempt to identify these SNRs. This task is challenging because diffuse gas with elevated ratios of [SII]:H-alpha is omnipresent in NGC4449, {causing confusion when using this common diagnostic for SNRs. Using narrow-band interference-filter images, we first identified 49 objects that have elevated [SII]:H-alpha ratios compared to nearby HII regions. Using Gemini-N and GMOS, we then obtained high-resolution spectra of 30 of these SNR candidates, 25 of which have [SII]:H-alpha ratios greater than 0.5. Of these, 15 nebulae are almost certainly SNRs, based on a combination of characteristics: higher [OI]:H-alpha ratios and broader line widths than observed from HII regions. The remainder are good candidates as well, but need additional confirmation. Surprisingly, despite having superior imaging and spectroscopic data sets to examine, we are unable to confirm most of the candidates suggested by Leonidaki (2013). While NGC4449 is likely an extreme case because of the high surface brightness and elevated [SII]:H-alpha ratio of diffuse gas, it highlights the need for sensitive high-resolution optical spectroscopy, or high spatial resolution radio or X-ray observations that can ensure accurate SNR identifications in external galaxies.


title: Matter-wave collimation to picokelvin energies with scattering length and potential shape control

abstract: The sensitivity of atom interferometers depends on their ability to realize long pulse separation times and prevent loss of contrast by limiting the expansion of the atomic ensemble within the interferometer beam through matter-wave collimation. Here we investigate the impact of atomic interactions on collimation by applying a lensing protocol to a $^{39}$K Bose-Einstein condensate at different scattering lengths. Tailoring interactions, we measure energies corresponding to $340 \pm 12$ pK in one direction. Our results are supported by an accurate simulation, which allows us to extrapolate a 2D ballistic expansion energy of $438 \pm 77$ pK. Based on our findings we propose an advanced scenario, which enables 3D expansion energies below $16$ pK by implementing an additional pulsed delta-kick. Our results pave the way to realize ensembles with more than $1\times10^5$ atoms and 3D energies in the two-digit pK range in typical dipole trap setups without the need for micro-gravity or long baseline environments.


title: Context-aware Trace Contracts

abstract: The behavior of concurrent, asynchronous procedures depends in general on the call context, because of the global protocol that governs scheduling. This context cannot be specified with the state-based Hoare-style contracts common in deductive verification. Recent work generalized state-based to trace contracts, which permit to specify the internal behavior of a procedure, such as calls or state changes, but not its call context. In this article we propose a program logic of context-aware trace contracts for specifying global behavior of asynchronous programs. We also provide a sound proof system that addresses two challenges: To observe the program state not merely at the end points of a procedure, we introduce the novel concept of an observation quantifier. And to combat combinatorial explosion of possible call sequences of procedures, we transfer Liskov's principle of behavioral subtyping to the analysis of asynchronous procedures.


title: On the Torsion Congruence for Zeta Functions of Totally Real Fields

abstract: In this note, we study the special values for zeta functions of totally real fields using the Shintani's cone decomposition. We prove certain congruence between the special values for zeta functions under the prime degree field extension. This congruence implies the `torsion congruence' proved by Ritter-Weiss which is crucial in the proof of the noncommutative Iwasawa main conjecture for totally real fields.


title: Branching fractional Brownian motion: discrete approximations and maximal displacement

abstract: We construct and study branching fractional Brownian motion with Hurst parameter $H\in(1/2,1)$. The construction relies on a generalization of the discrete approximation of fractional Brownian motion (Hammond and Sheffield, Probability Theory and Related Fields, 2013) to power law P\'olya urns indexed by trees. We show that the first order of the speed of branching fractional Brownian motion with Hurst parameter $H$ is $ct^{H+1/2}$ where $c$ is explicit and only depends on the Hurst parameter. A notion of "branching property" for processes with memory emerges naturally from our construction.


title: A CMB lensing analysis of the extended mass distribution of clusters

abstract: The aim of this work is to study the anisotropic weak lensing signal associated with the mass distribution of massive clusters of galaxies using the Cosmic Microwave Background (CMB) data. For this purpose, we stack patches of the Planck Collaboration 2018 CMB lensing convergence map centered on SDSS DR8 redMaPPer clusters within the redshift range [0.4, 0.5]. We obtain mean radial profiles of the convergence parameter k finding strong signals at scales as large as 40 Mpc/h. By orienting the clusters along their major axis defined through the galaxy member distribution, we find a significant difference between the parallel and perpendicular oriented convergence profiles. The amplitude of the profile along the parallel direction is about 50% larger than that along the perpendicular direction, indicating that the clusters are well aligned with the surrounding mass distribution. From a model with an anisotropic surface mass density, we obtain a suitable agreement for both mass and ellipticities of clusters compared to results derived from weak lensing shear estimates, finding strong evidence of the correlation between the galaxy cluster member distribution and the large--scale mass distribution.


title: A chromatic vanishing result for TR

abstract: In this note, we establish a vanishing result for telescopically localized $\mathrm{TR}$. More precisely, we prove that $T(k)$-local $\mathrm{TR}$ vanishes on connective $L_n^{p,f}$-acyclic $\mathbf{E}_1$-rings for every $1 \leq k \leq n$ and deduce consequences for connective Morava K-theory and the Thom spectra $y(n)$. The proof relies on the relationship between $\mathrm{TR}$ and the spectrum of curves on K-theory together with fact that algebraic K-theory preserves infinite products of additive $\infty$-categories which was recently established by C\'{o}rdova Fedeli.


title: Evolution of High Throughput Satellite Systems: Vision, Requirements, and Key Technologies

abstract: High throughput satellites (HTS), with their digital payload technology, are expected to play a key role as enablers of the upcoming 6G networks. HTS are mainly designed to provide higher data rates and capacities. Fueled by technological advancements including beamforming, advanced modulation techniques, reconfigurable phased array technologies, and electronically steerable antennas, HTS have emerged as a fundamental component for future network generation. This paper offers a comprehensive state-of-the-art of HTS systems, with a focus on standardization, patents, channel multiple access techniques, routing, load balancing, and the role of software-defined networking (SDN). In addition, we provide a vision for next-satellite systems that we named as extremely-HTS (EHTS) toward autonomous satellites supported by the main requirements and key technologies expected for these systems. The EHTS system will be designed such that it maximizes spectrum reuse and data rates, and flexibly steers the capacity to satisfy user demand. We introduce a novel architecture for future regenerative payloads while summarizing the challenges imposed by this architecture.


title: Experimental Designs for Heteroskedastic Variance

abstract: Most linear experimental design problems assume homogeneous variance although heteroskedastic noise is present in many realistic settings. Let a learner have access to a finite set of measurement vectors $\mathcal{X}\subset \mathbb{R}^d$ that can be probed to receive noisy linear responses of the form $y=x^{\top}\theta^{\ast}+\eta$. Here $\theta^{\ast}\in \mathbb{R}^d$ is an unknown parameter vector, and $\eta$ is independent mean-zero $\sigma_x^2$-sub-Gaussian noise defined by a flexible heteroskedastic variance model, $\sigma_x^2 = x^{\top}\Sigma^{\ast}x$. Assuming that $\Sigma^{\ast}\in \mathbb{R}^{d\times d}$ is an unknown matrix, we propose, analyze and empirically evaluate a novel design for uniformly bounding estimation error of the variance parameters, $\sigma_x^2$. We demonstrate the benefits of this method with two adaptive experimental design problems under heteroskedastic noise, fixed confidence transductive best-arm identification and level-set identification and prove the first instance-dependent lower bounds in these settings. Lastly, we construct near-optimal algorithms and demonstrate the large improvements in sample complexity gained from accounting for heteroskedastic variance in these designs empirically.


title: On a Hierarchy of Spectral Invariants for Graphs

abstract: We consider a hierarchy of graph invariants that naturally extends the spectral invariants defined by F\"urer (Lin. Alg. Appl. 2010) based on the angles formed by the set of standard basis vectors and their projections onto eigenspaces of the adjacency matrix. We provide a purely combinatorial characterization of this hierarchy in terms of the walk counts. This allows us to give a complete answer to F\"urer's question about the strength of his invariants in distinguishing non-isomorphic graphs in comparison to the 2-dimensional Weisfeiler-Leman algorithm, extending the recent work of Rattan and Seppelt (SODA 2023). As another application of the characterization, we prove that almost all graphs are determined up to isomorphism in terms of the spectrum and the angles, which is of interest in view of the long-standing open problem whether almost all graphs are determined by their eigenvalues alone. Finally, we describe the exact relationship between the hierarchy and the Weisfeiler-Leman algorithms for small dimensions, as also some other important spectral characteristics of a graph such as the generalized and the main spectra.


title: Anomalous dimensions from the $\mathcal{N}=4$ SYM hexagon

abstract: We consider the correlator $\langle \mathcal{L} \mathcal{K} \tilde{ \mathcal{K}} \rangle $ of the Lagrange operator of $\mathcal{N}=4$ super Yang-Mills theory and two conjugate two-excitation operators in an $su(2)$ sector. We recover the planar one-loop anomalous dimension of the renormalised operators from this hexagon computation.


title: Fuzzy VC Combinatorics and Distality in Continuous Logic

abstract: Model theory, machine learning, and combinatorics each have generalizations of VC-dimension for fuzzy and real-valued versions of set systems. These different dimensions define a unique notion of a VC-class for both fuzzy sets and real-valued functions. We study these VC-classes, obtaining generalizations of certain combinatorial results from the discrete case. These include appropriate generalizations of $\varepsilon$-nets, the fractional Helly property and the $(p,q)$-theorem. We then apply these results to continuous logic. We prove that NIP for metric structures is equivalent to an appropriate generalization of honest definitions, which we use to study externally definable predicates and the Shelah expansion. We then examine distal metric structures, providing several equivalent characterizations, in terms of indiscernible sequences, distal types, strong honest definitions, and distal cell decompositions.


title: Spin-Mediated Direct Photon Scattering by Plasmons in BiTeI

abstract: We use polarization resolved Raman spectroscopy to demonstrate that for a 3D giant Rashba system the bulk plasmon collective mode can directly couple to the Raman response even in the long wavelength $\mathbf q \rightarrow 0$ limit. Although conventional theory predicts the plasmon spectral weight to be suppressed as the square of its quasi-momentum and thus negligibly weak in the Raman spectra, we observe a sharp in-gap plasmon mode in the Raman spectrum of BiTeI below the Rashba continuum. This coupling, in a polar system with spin-orbit coupling, occurs without assistance from phonons when the incoming photon excitation is resonant with Rashba-split intermediate states. We discuss the distinctive features of BiTeI's giant Rashba system band structure that enable the direct observation of plasmon in Raman scattering.


title: Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference

abstract: We propose a method to improve the efficiency and accuracy of amortized Bayesian inference by leveraging universal symmetries in the joint probabilistic model of parameters and data. In a nutshell, we invert Bayes' theorem and estimate the marginal likelihood based on approximate representations of the joint model. Upon perfect approximation, the marginal likelihood is constant across all parameter values by definition. However, errors in approximate inference lead to undesirable variance in the marginal likelihood estimates across different parameter values. We penalize violations of this symmetry with a \textit{self-consistency loss} which significantly improves the quality of approximate inference in low data regimes and can be used to augment the training of popular neural density estimators. We apply our method to a number of synthetic problems and realistic scientific models, discovering notable advantages in the context of both neural posterior and likelihood approximation.


title: Interpolating Parametrized Quantum Circuits using Blackbox Queries

abstract: This article focuses on developing classical surrogates for parametrized quantum circuits using interpolation via (trigonometric) polynomials. We develop two algorithms for the construction of such surrogates and prove performance guarantees. The constructions are based on circuit evaluations which are blackbox in the sense that no structural specifics of the circuits are exploited. While acknowledging the limitations of the blackbox approach compared to whitebox evaluations, which exploit specific circuit properties, we demonstrate scenarios in which the blackbox approach might prove beneficial. Sample applications include but are not restricted to the approximation of VQEs and the alleviaton of the barren plateau problem.


title: Cloning, deleting, and hiding in modal quantum theory

abstract: We examine the toy model of modal quantum theory (MQT), an analogue of actual quantum theory based on finite fields. In particular, we investigate how several essential ``no-go'' results (for cloning, deleting and hiding processes) work in MQT. Cloning and deleting are still forbidden in MQT, though the details of these results are somewhat different in the new context. However, the information of a modal qubit can be completely hidden in the correlations between two entangled modal qubits.


title: Analysis and Algorithmic Construction of Self-Assembled DNA Complexes

abstract: DNA self-assembly is an important tool that has a wide range of applications such as building nanostructures, the transport of target virotherapies, and nano-circuitry. Tools from graph theory can be used to encode the biological process of DNA self-assembly. The principle component of this process is to examine collections of branched junction molecules, called pots, and study the types of structures that can be constructed. We restrict our attention to pots which contain one set of complementary cohesive-ends, i.e. a single bond-edge type, and we identify the types and sizes of structures that can be built from such a pot. In particular, we show a dependence between the order of graphs in the output of the pot and the number of arms on the corresponding tiles. Furthermore, we provide two algorithms which will construct complete complexes for a pot with a single bond-edge type.


title: Improving Stability in Simultaneous Speech Translation: A Revision-Controllable Decoding Approach

abstract: Simultaneous Speech-to-Text translation serves a critical role in real-time crosslingual communication. Despite the advancements in recent years, challenges remain in achieving stability in the translation process, a concern primarily manifested in the flickering of partial results. In this paper, we propose a novel revision-controllable method designed to address this issue. Our method introduces an allowed revision window within the beam search pruning process to screen out candidate translations likely to cause extensive revisions, leading to a substantial reduction in flickering and, crucially, providing the capability to completely eliminate flickering. The experiments demonstrate the proposed method can significantly improve the decoding stability without compromising substantially on the translation quality.


title: On the Embedding Collapse when Scaling up Recommendation Models

abstract: Recent advances in foundation models have led to a promising trend of developing large recommendation models to leverage vast amounts of available data. Still, mainstream models remain embarrassingly small in size and na\"ive enlarging does not lead to sufficient performance gain, suggesting a deficiency in the model scalability. In this paper, we identify the embedding collapse phenomenon as the inhibition of scalability, wherein the embedding matrix tends to occupy a low-dimensional subspace. Through empirical and theoretical analysis, we demonstrate a \emph{two-sided effect} of feature interaction specific to recommendation models. On the one hand, interacting with collapsed embeddings restricts embedding learning and exacerbates the collapse issue. On the other hand, interaction is crucial in mitigating the fitting of spurious features as a scalability guarantee. Based on our analysis, we propose a simple yet effective multi-embedding design incorporating embedding-set-specific interaction modules to learn embedding sets with large diversity and thus reduce collapse. Extensive experiments demonstrate that this proposed design provides consistent scalability and effective collapse mitigation for various recommendation models. Code is available at this repository: https://github.com/thuml/Multi-Embedding.


title: Neighbour Sum Patterns : Chessboards to Toroidal Worlds

abstract: We say that a chessboard filled with integer entries satisfies the neighbour-sum property if the number appearing on each cell is the sum of entries in its neighbouring cells, where neighbours are cells sharing a common edge or vertex. We show that an $n\times n$ chessboard satisfies this property if and only if $n\equiv 5\pmod 6$. Existence of solutions is further investigated of rectangular, toroidal boards, as well as on Neumann neighbourhoods, including a nice connection to discrete harmonic functions. Construction of solutions on infinite boards are also presented. Finally, answers to three dimensional analogues of these boards are explored using properties of cyclotomic polynomials and relevant ideas conjectured.


title: An algorithm to decide if an outer automorphism is geometric

abstract: An outer automorphism of a free group is geometric if it can be represented by a homeomorphism of a compact surface. Bestvina and Handel gave an algorithmic characterization of geometric irreducible outer automorphisms using relative train tracks in 1995. The general case of detecting geometric outer automorphisms remained open, with a few partial results appearing subsequently. In this paper we give a complete resolution to the problem: an algorithm that can decide if a general outer automorphism is geometric. The algorithm is constructive and produces a realizing surface homeomorphism if one exists. We make use of advances in train-track theory, in conjunction with the Guirardel core of tree actions and Nielsen-Thurston theory for surfaces.


title: The impact of high-dimensional phase space correlations on the beam dynamics in a linear accelerator

abstract: Hadron beams develop intensity-dependent transverse-longitudinal correlations within radio-frequency quadrupole (RFQ) accelerating structures. These correlations are only visible in six-dimensional phase space and are destroyed by reconstructions from low-dimensional projections. In this work, we estimate the effect of artificial decorrelation on the beam dynamics in the Spallation Neutron Source (SNS) linac and Beam Test Facility (BTF). We show that the evolution of a realistic initial distribution and its decorrelated twin converge during the early acceleration stages; thus, low-dimensional projections are probably sufficient for detailed predictions in high-power linacs.


title: Seeding the CGM: How Satellites Populate the Cold Phase of Milky Way Halos

abstract: The origin of the cold phase in the CGM is a highly debated question. We investigate the contribution of satellite galaxies to the cold gas budget in the circumgalactic medium (CGM)of a Milky Way-like host galaxy. We perform controlled experiments with three different satellite mass distributions and identify several mechanisms by which satellites can add cold gas to the CGM, including ram pressure stripping and induced cooling in the mixing layer of the stripped cold gas. These two mechanisms contribute a comparable amount of cold gas to the host CGM. We find that the less massive satellites ($\leq 10^9 M_\odot$) not only lose all of their cold gas in a short period ($\sim$ 0.5-1 Gyr), but their stripped cold clouds also mix with the hot CGM gas and get heated up quickly. However, stellar feedback from these less massive satellites can hugely alter the fate of their stripped gas. Feedback speeds up the destruction of the stripped cold clouds from these satellites by making them more diffuse with more surface area. On the other hand, the more massive satellites (LMC or SMC-like $\sim 10^{10} M_\odot$) can add cold gas to the total gas budget of the host CGM for several Gyrs.


title: Resummed spinning waveforms from five-point amplitudes

abstract: We compute the classical tree-level five-point amplitude for the two-to-two scattering of spinning celestial objects with the emission of a graviton. Using this five-point amplitude, we then turn to the computation of the leading-order time-domain gravitational waveform. The method we describe is suitable for arbitrary values of classical spin of Kerr black holes and does not require any expansion in powers of the spin. In this paper we illustrate it in the simpler case of the scattering of one Kerr and one Schwarzschild black hole. An important ingredient of our calculation is a novel form of the Compton amplitude with spinning particles including contact terms derived from matching to black-hole perturbation theory calculations. This ensures that our waveform is valid up to at least fourth order in the spin. Our method can be applied immediately to generate improved waveforms once higher-order contact terms in the Compton amplitude become available. Finally, we show the formula for the gravitational memory to all orders in the spin, which is in agreement with our results.


title: Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models

abstract: While language models (LMs) have shown potential across a range of decision-making tasks, their reliance on simple acting processes limits their broad deployment as autonomous agents. In this paper, we introduce Language Agent Tree Search (LATS) -- the first general framework that synergizes the capabilities of LMs in reasoning, acting, and planning. By leveraging the in-context learning ability of LMs, we integrate Monte Carlo Tree Search into LATS to enable LMs as agents, along with LM-powered value functions and self-reflections for proficient exploration and enhanced decision-making. A key feature of our approach is the incorporation of an environment for external feedback, which offers a more deliberate and adaptive problem-solving mechanism that surpasses the constraints of existing techniques. Our experimental evaluation across diverse domains, including programming, interactive question-answering (QA), web navigation, and math, validates the effectiveness and generality of LATS in decision-making while maintaining competitive or improved reasoning performance. Notably, LATS achieves state-of-the-art pass@1 accuracy (92.7%) for programming on HumanEval with GPT-4 and demonstrates gradient-free performance (average score of 75.9) comparable to gradient-based fine-tuning for web navigation on WebShop with GPT-3.5. Code can be found at https://github.com/lapisrocks/LanguageAgentTreeSearch


title: Policy-Gradient Training of Language Models for Ranking

abstract: Text retrieval plays a crucial role in incorporating factual knowledge for decision making into language processing pipelines, ranging from chat-based web search to question answering systems. Current state-of-the-art text retrieval models leverage pre-trained large language models (LLMs) to achieve competitive performance, but training LLM-based retrievers via typical contrastive losses requires intricate heuristics, including selecting hard negatives and using additional supervision as learning signals. This reliance on heuristics stems from the fact that the contrastive loss itself is heuristic and does not directly optimize the downstream metrics of decision quality at the end of the processing pipeline. To address this issue, we introduce Neural PG-RANK, a novel training algorithm that learns to rank by instantiating a LLM as a Plackett-Luce ranking policy. Neural PG-RANK provides a principled method for end-to-end training of retrieval models as part of larger decision systems via policy gradient, with little reliance on complex heuristics, and it effectively unifies the training objective with downstream decision-making quality. We conduct extensive experiments on various text retrieval benchmarks. The results demonstrate that when the training objective aligns with the evaluation setup, Neural PG-RANK yields remarkable in-domain performance improvement, with substantial out-of-domain generalization to some critical datasets employed in downstream question answering tasks.


title: RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation

abstract: Retrieving documents and prepending them in-context at inference time improves performance of language model (LMs) on a wide range of tasks. However, these documents, often spanning hundreds of words, make inference substantially more expensive. We propose compressing the retrieved documents into textual summaries prior to in-context integration. This not only reduces the computational costs but also relieves the burden of LMs to identify relevant information in long retrieved documents. We present two compressors -- an extractive compressor which selects useful sentences from retrieved documents and an abstractive compressor which generates summaries by synthesizing information from multiple documents. Both compressors are trained to improve LMs' performance on end tasks when the generated summaries are prepended to the LMs' input, while keeping the summary concise.If the retrieved documents are irrelevant to the input or offer no additional information to LM, our compressor can return an empty string, implementing selective augmentation.We evaluate our approach on language modeling task and open domain question answering task. We achieve a compression rate of as low as 6% with minimal loss in performance for both tasks, significantly outperforming the off-the-shelf summarization models. We show that our compressors trained for one LM can transfer to other LMs on the language modeling task and provide summaries largely faithful to the retrieved documents.


title: Metal-Optic Nanophotonic Modulators in Standard CMOS Technology

abstract: Integrating nanophotonics with electronics promises revolutionary applications, from LiDAR to holographic displays. Although silicon photonics is maturing, realizing active nanophotonics in the ubiquitous bulk CMOS processes remains challenging. We introduce a fabless approach to embed active nanophotonics in bulk CMOS by co-designing the back-end-of-line metal layers for optical functionality. Using a 65nm CMOS process, we create plasmonic liquid crystal modulators with switching speeds 100x faster than commercial technologies. This zero-change nanophotonics method could equip mass-produced chips with optical communications, sensing and imaging. Embedding nanophotonics in the dominant electronics platform democratizes nanofabrication, spawning technologies from chip-scale LiDAR to holographic light-field displays.


title: Evidence for Spatial Separation of Galactic Dust Populations

abstract: We present an implementation of a Bayesian mixture model using Hamiltonian Monte Carlo (HMC) techniques to search for spatial separation of Galactic dust populations. Utilizing intensity measurements from Planck High Frequency Instrument (HFI), we apply this model to high-latitude Galactic dust emission. Our analysis reveals a strong preference for a spatially-varying two-population dust model over a one-population dust model, when the latter must capture the total variance in the sky. Each dust population is well characterized by a single-component spectral energy distribution (SED) and accommodates small variations. These populations could signify two distinct components, or may originate from a one-component model with different temperatures resulting in different SED scalings. While no spatial information is built into the likelihood, our investigation unveils large-scale spatially coherent structures with high significance, pointing to a physical origin for the observed spatial variation. These results are robust to our choice of likelihood and of input data. Furthermore, this spatially varying two-population model is the most favored from Bayesian evidence calculations. Incorporating IRAS 100 $\mu$m to constrain the Wein-side of the blackbody function, we find the dust populations differ at the 2.5$\sigma$ level in the spectral index ($\beta_d$) vs. temperature ($T_d$) plane. The presence of multiple dust populations has implications for component separation techniques frequently employed in the recovery of the cosmic microwave background.


title: Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RL

abstract: The divergence of the Q-value estimation has been a prominent issue in offline RL, where the agent has no access to real dynamics. Traditional beliefs attribute this instability to querying out-of-distribution actions when bootstrapping value targets. Though this issue can be alleviated with policy constraints or conservative Q estimation, a theoretical understanding of the underlying mechanism causing the divergence has been absent. In this work, we aim to thoroughly comprehend this mechanism and attain an improved solution. We first identify a fundamental pattern, self-excitation, as the primary cause of Q-value estimation divergence in offline RL. Then, we propose a novel Self-Excite Eigenvalue Measure (SEEM) metric based on Neural Tangent Kernel (NTK) to measure the evolving property of Q-network at training, which provides an intriguing explanation of the emergence of divergence. For the first time, our theory can reliably decide whether the training will diverge at an early stage, and even predict the order of the growth for the estimated Q-value, the model's norm, and the crashing step when an SGD optimizer is used. The experiments demonstrate perfect alignment with this theoretic analysis. Building on our insights, we propose to resolve divergence from a novel perspective, namely improving the model's architecture for better extrapolating behavior. Through extensive empirical studies, we identify LayerNorm as a good solution to effectively avoid divergence without introducing detrimental bias, leading to superior performance. Experimental results prove that it can still work in some most challenging settings, i.e. using only 1 transitions of the dataset, where all previous methods fail. Moreover, it can be easily plugged into modern offline RL methods and achieve SOTA results on many challenging tasks. We also give unique insights into its effectiveness.


title: FedConv: Enhancing Convolutional Neural Networks for Handling Data Heterogeneity in Federated Learning

abstract: Federated learning (FL) is an emerging paradigm in machine learning, where a shared model is collaboratively learned using data from multiple devices to mitigate the risk of data leakage. While recent studies posit that Vision Transformer (ViT) outperforms Convolutional Neural Networks (CNNs) in addressing data heterogeneity in FL, the specific architectural components that underpin this advantage have yet to be elucidated. In this paper, we systematically investigate the impact of different architectural elements, such as activation functions and normalization layers, on the performance within heterogeneous FL. Through rigorous empirical analyses, we are able to offer the first-of-its-kind general guidance on micro-architecture design principles for heterogeneous FL. Intriguingly, our findings indicate that with strategic architectural modifications, pure CNNs can achieve a level of robustness that either matches or even exceeds that of ViTs when handling heterogeneous data clients in FL. Additionally, our approach is compatible with existing FL techniques and delivers state-of-the-art solutions across a broad spectrum of FL benchmarks. The code is publicly available at https://github.com/UCSC-VLAA/FedConv


title: Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets

abstract: Offline policy learning is aimed at learning decision-making policies using existing datasets of trajectories without collecting additional data. The primary motivation for using reinforcement learning (RL) instead of supervised learning techniques such as behavior cloning is to find a policy that achieves a higher average return than the trajectories constituting the dataset. However, we empirically find that when a dataset is dominated by suboptimal trajectories, state-of-the-art offline RL algorithms do not substantially improve over the average return of trajectories in the dataset. We argue this is due to an assumption made by current offline RL algorithms of staying close to the trajectories in the dataset. If the dataset primarily consists of sub-optimal trajectories, this assumption forces the policy to mimic the suboptimal actions. We overcome this issue by proposing a sampling strategy that enables the policy to only be constrained to ``good data" rather than all actions in the dataset (i.e., uniform sampling). We present a realization of the sampling strategy and an algorithm that can be used as a plug-and-play module in standard offline RL algorithms. Our evaluation demonstrates significant performance gains in 72 imbalanced datasets, D4RL dataset, and across three different offline RL algorithms. Code is available at https://github.com/Improbable-AI/dw-offline-rl.


title: CIFAR-10-Warehouse: Broad and More Realistic Testbeds in Model Generalization Analysis

abstract: Analyzing model performance in various unseen environments is a critical research problem in the machine learning community. To study this problem, it is important to construct a testbed with out-of-distribution test sets that have broad coverage of environmental discrepancies. However, existing testbeds typically either have a small number of domains or are synthesized by image corruptions, hindering algorithm design that demonstrates real-world effectiveness. In this paper, we introduce CIFAR-10-Warehouse, consisting of 180 datasets collected by prompting image search engines and diffusion models in various ways. Generally sized between 300 and 8,000 images, the datasets contain natural images, cartoons, certain colors, or objects that do not naturally appear. With CIFAR-10-W, we aim to enhance the evaluation and deepen the understanding of two generalization tasks: domain generalization and model accuracy prediction in various out-of-distribution environments. We conduct extensive benchmarking and comparison experiments and show that CIFAR-10-W offers new and interesting insights inherent to these tasks. We also discuss other fields that would benefit from CIFAR-10-W.


title: Why Do We Need Weight Decay in Modern Deep Learning?

abstract: Weight decay is a broadly used technique for training state-of-the-art deep networks, including large language models. Despite its widespread usage, its role remains poorly understood. In this work, we highlight that the role of weight decay in modern deep learning is different from its regularization effect studied in classical learning theory. For overparameterized deep networks, we show how weight decay modifies the optimization dynamics enhancing the ever-present implicit regularization of SGD via the loss stabilization mechanism. In contrast, for underparameterized large language models trained with nearly online SGD, we describe how weight decay balances the bias-variance tradeoff in stochastic optimization leading to lower training loss. Moreover, we show that weight decay also prevents sudden loss divergences for bfloat16 mixed-precision training which is a crucial tool for LLM training. Overall, we present a unifying perspective from ResNets on vision tasks to LLMs: weight decay is never useful as an explicit regularizer but instead changes the training dynamics in a desirable way. Our code is available at https://github.com/tml-epfl/why-weight-decay.


title: Alice Benchmarks: Connecting Real World Re-Identification with the Synthetic

abstract: For object re-identification (re-ID), learning from synthetic data has become a promising strategy to cheaply acquire large-scale annotated datasets and effective models, with few privacy concerns. Many interesting research problems arise from this strategy, e.g., how to reduce the domain gap between synthetic source and real-world target. To facilitate developing more new approaches in learning from synthetic data, we introduce the Alice benchmarks, large-scale datasets providing benchmarks as well as evaluation protocols to the research community. Within the Alice benchmarks, two object re-ID tasks are offered: person and vehicle re-ID. We collected and annotated two challenging real-world target datasets: AlicePerson and AliceVehicle, captured under various illuminations, image resolutions, etc. As an important feature of our real target, the clusterability of its training set is not manually guaranteed to make it closer to a real domain adaptation test scenario. Correspondingly, we reuse existing PersonX and VehicleX as synthetic source domains. The primary goal is to train models from synthetic data that can work effectively in the real world. In this paper, we detail the settings of Alice benchmarks, provide an analysis of existing commonly-used domain adaptation methods, and discuss some interesting future directions. An online server has been set up for the community to evaluate methods conveniently and fairly. Datasets and the online server details are available at https://sites.google.com/view/alice-benchmarks.


title: Diffusion Random Feature Model

abstract: Diffusion probabilistic models have been successfully used to generate data from noise. However, most diffusion models are computationally expensive and difficult to interpret with a lack of theoretical justification. Random feature models on the other hand have gained popularity due to their interpretability but their application to complex machine learning tasks remains limited. In this work, we present a diffusion model-inspired deep random feature model that is interpretable and gives comparable numerical results to a fully connected neural network having the same number of trainable parameters. Specifically, we extend existing results for random features and derive generalization bounds between the distribution of sampled data and the true distribution using properties of score matching. We validate our findings by generating samples on the fashion MNIST dataset and instrumental audio data.


title: Functional Interpolation for Relative Positions Improves Long Context Transformers

abstract: Preventing the performance decay of Transformers on inputs longer than those used for training has been an important challenge in extending the context length of these models. Though the Transformer architecture has fundamentally no limits on the input sequence lengths it can process, the choice of position encoding used during training can limit the performance of these models on longer inputs. We propose a novel functional relative position encoding with progressive interpolation, FIRE, to improve Transformer generalization to longer contexts. We theoretically prove that this can represent some of the popular relative position encodings, such as T5's RPE, Alibi, and Kerple. We next empirically show that FIRE models have better generalization to longer contexts on both zero-shot language modeling and long text benchmarks.


title: Microscopic origin of the quantum Mpemba effect in integrable systems

abstract: The highly complicated nature of far from equilibrium systems can lead to a complete breakdown of the physical intuition developed in equilibrium. A famous example of this is the Mpemba effect, which states that non-equilibrium states may relax faster when they are further from equilibrium or, put another way, hot water can freeze faster than warm water. Despite possessing a storied history, the precise criteria and mechanisms underpinning this phenomenon are still not known. Here we study a quantum version of the Mpemba effect that takes place in closed many body systems with a U(1) conserved charge: in certain cases a more asymmetric initial configuration relaxes and restores the symmetry faster than a more symmetric one. In contrast to the classical case, we establish the criteria for this to occur in arbitrary integrable quantum systems using the recently introduced entanglement asymmetry. We describe the quantum Mpemba effect in such systems and relate properties of the initial state, specifically its charge fluctuations, to the criteria for its occurrence. These criteria are expounded using exact analytic and numerical techniques in several examples, a free fermion model, the Rule 54 cellular automaton, and the Lieb-Liniger model.


title: BrainSCUBA: Fine-Grained Natural Language Captions of Visual Cortex Selectivity

abstract: Understanding the functional organization of higher visual cortex is a central focus in neuroscience. Past studies have primarily mapped the visual and semantic selectivity of neural populations using hand-selected stimuli, which may potentially bias results towards pre-existing hypotheses of visual cortex functionality. Moving beyond conventional approaches, we introduce a data-driven method that generates natural language descriptions for images predicted to maximally activate individual voxels of interest. Our method -- Semantic Captioning Using Brain Alignments ("BrainSCUBA") -- builds upon the rich embedding space learned by a contrastive vision-language model and utilizes a pre-trained large language model to generate interpretable captions. We validate our method through fine-grained voxel-level captioning across higher-order visual regions. We further perform text-conditioned image synthesis with the captions, and show that our images are semantically coherent and yield high predicted activations. Finally, to demonstrate how our method enables scientific discovery, we perform exploratory investigations on the distribution of "person" representations in the brain, and discover fine-grained semantic selectivity in body-selective areas. Unlike earlier studies that decode text, our method derives voxel-wise captions of semantic selectivity. Our results show that BrainSCUBA is a promising means for understanding functional preferences in the brain, and provides motivation for further hypothesis-driven investigation of visual cortex.


title: Kinetic equations and level-set approach for simulating solid-state microstructure evolutions at the mesoscopic scale: state of the art, limitations, and prospects

abstract: For over three decades, the front-capturing level-set method has demonstrated its prowess for the simulation, at the mesoscopic scale, of numerous mechanisms in the context of microstructure evolution occurring during complex thermomechanical paths. This review delves into the foundations of this numerical framework, charting its evolution concerning polycrystalline materials, examining its recent advancements, scrutinizing its current shortcomings, and exploring future possibilities. Special attention will be given to the context of hot metal forming processes. In this context, this article also aims to reintroduce, as simply as possible, the kinetic equations related to the grain boundary migration.


title: Qualitative and quantitative evaluation of a methodology for the Digital Twin creation of brownfield production systems

abstract: The Digital Twin is a well-known concept of industry 4.0 and is the cyber part of a cyber-physical production system providing several benefits such as virtual commissioning or predictive maintenance. The existing production systems are lacking a Digital Twin which has to be created manually in a time-consuming and error-prone process. Therefore, methods to create digital models of existing production systems and their relations between them were developed. This paper presents the implementation of the methodology for the creation of multi-disciplinary relations and a quantitative and qualitative evaluation of the benefits of the methodology.


title: The BELSAR dataset: Mono- and bistatic full-pol L-band SAR for agriculture and hydrology

abstract: The BELSAR dataset is a unique collection of high-resolution airborne mono- and bistatic fully-polarimetric synthetic aperture radar (SAR) data in L-band, alongside concurrent measurements of vegetation and soil bio-geophysical variables measured in maize and winter wheat fields during the summer of 2018 in Belgium. This innovative dataset, the collection of which was funded by the European Space Agency (ESA), helps addressing the lack of publicly-accessible experimental datasets combining multistatic SAR and in situ measurements. As such, it offers an opportunity to advance the development of SAR remote sensing science and applications for agricultural monitoring and hydrology. This paper aims to facilitate its adoption and exploration by offering comprehensive documentation and integrating its multiple data sources into a unified, analysis-ready dataset.


title: Stability Analysis of Non-Linear Classifiers using Gene Regulatory Neural Network for Biological AI

abstract: The Gene Regulatory Network (GRN) of biological cells governs a number of key functionalities that enables them to adapt and survive through different environmental conditions. Close observation of the GRN shows that the structure and operational principles resembles an Artificial Neural Network (ANN), which can pave the way for the development of Biological Artificial Intelligence. In particular, a gene's transcription and translation process resembles a sigmoidal-like property based on transcription factor inputs. In this paper, we develop a mathematical model of gene-perceptron using a dual-layered transcription-translation chemical reaction model, enabling us to transform a GRN into a Gene Regulatory Neural Network (GRNN). We perform stability analysis for each gene-perceptron within the fully-connected GRNN sub network to determine temporal as well as stable concentration outputs that will result in reliable computing performance. We focus on a non-linear classifier application for the GRNN, where we analyzed generic multi-layer GRNNs as well as E.Coli GRNN that is derived from trans-omic experimental data. Our analysis found that varying the parameters of the chemical reactions can allow us shift the boundaries of the classification region, laying the platform for programmable GRNNs that suit diverse application requirements.


title: Red Teaming Generative AI/NLP, the BB84 quantum cryptography protocol and the NIST-approved Quantum-Resistant Cryptographic Algorithms

abstract: In the contemporary digital age, Quantum Computing and Artificial Intelligence (AI) convergence is reshaping the cyber landscape, introducing unprecedented opportunities and potential vulnerabilities.This research, conducted over five years, delves into the cybersecurity implications of this convergence, with a particular focus on AI/Natural Language Processing (NLP) models and quantum cryptographic protocols, notably the BB84 method and specific NIST-approved algorithms. Utilising Python and C++ as primary computational tools, the study employs a "red teaming" approach, simulating potential cyber-attacks to assess the robustness of quantum security measures. Preliminary research over 12 months laid the groundwork, which this study seeks to expand upon, aiming to translate theoretical insights into actionable, real-world cybersecurity solutions. Located at the University of Oxford's technology precinct, the research benefits from state-of-the-art infrastructure and a rich collaborative environment. The study's overarching goal is to ensure that as the digital world transitions to quantum-enhanced operations, it remains resilient against AI-driven cyber threats. The research aims to foster a safer, quantum-ready digital future through iterative testing, feedback integration, and continuous improvement. The findings are intended for broad dissemination, ensuring that the knowledge benefits academia and the global community, emphasising the responsible and secure harnessing of quantum technology.


title: Research Funding in the Middle East and North Africa: Analyses of Acknowledgments in Scientific Publications indexed in the Web of Science (2008-2021)

abstract: Funding acknowledgments are important objects of study in the context of science funding. This study uses a mixed-methods approach to analyze the funding acknowledgments found in 2.3 million scientific publications published between 2008 and 2021 by authors affiliated with research institutions located in the Middle Eastern and North Africa (MENA). The aim is to identify the major funders, assess their contribution to national scientific publications, and gain insights into the funding mechanism in relation to collaboration and publication. Publication data from the Web of Science is examined to provide key insights about funding activities. Saudi Arabia and Qatar lead the region, as about half of their publications include acknowledgments to funding sources. Most MENA countries exhibit strong linkages with foreign agencies, mainly due to a high level of international collaborations. The distinction between domestic and international publications reveals some differences in terms of funding structures. For instance, Turkey and Iran are dominated by one or two major funders whereas a few other countries like Saudi Arabia showcase multiple funders. Iran and Kuwait are examples of countries where research is mainly funded by domestic funders. The government and academic sectors mainly fund scientific research in MENA whereas the industry sector plays little or no role in terms of research funding. Lastly, the qualitative analyses provide more context into the complex funding mechanism. The findings of this study contribute to a better understanding of the funding structure in MENA countries and provide insights to funders and research managers to evaluate the funding landscape.


title: Generative AI in the Construction Industry: Opportunities & Challenges

abstract: In the last decade, despite rapid advancements in artificial intelligence (AI) transforming many industry practices, construction largely lags in adoption. Recently, the emergence and rapid adoption of advanced large language models (LLM) like OpenAI's GPT, Google's PaLM, and Meta's Llama have shown great potential and sparked considerable global interest. However, the current surge lacks a study investigating the opportunities and challenges of implementing Generative AI (GenAI) in the construction sector, creating a critical knowledge gap for researchers and practitioners. This underlines the necessity to explore the prospects and complexities of GenAI integration. Bridging this gap is fundamental to optimizing GenAI's early-stage adoption within the construction sector. Given GenAI's unprecedented capabilities to generate human-like content based on learning from existing content, we reflect on two guiding questions: What will the future bring for GenAI in the construction industry? What are the potential opportunities and challenges in implementing GenAI in the construction industry? This study delves into reflected perception in literature, analyzes the industry perception using programming-based word cloud and frequency analysis, and integrates authors' opinions to answer these questions. This paper recommends a conceptual GenAI implementation framework, provides practical recommendations, summarizes future research questions, and builds foundational literature to foster subsequent research expansion in GenAI within the construction and its allied architecture & engineering domains.


title: Avalanche Prediction and Dynamics using Temperature Variance , Grain Size Variance and Flow Regimes

abstract: We investigate the effects of temperature variance, grain size variation, flow regimes, and the use of Support Vector Machines (SVMs) in avalanche studies. The temperature variance experiments involved ice single crystals and polycrystals, revealing that the scale-free pattern of avalanche sizes remains consistent regardless of temperature. The dynamics of dislocations in polycrystals were found to be independent of stress level and temperature. The Material Point Method (MPM) was used to explore snow avalanche behavior and identify flow regimes. The MPM accurately represented various flow patterns of snow avalanches, although challenges remained in capturing powder clouds. SVMs were employed for avalanche forecasting, using meteorological and snowpack variables as input features. The selected features provided insights into snowfall characteristics, snow accumulation, rain interaction, snowdrift patterns, cloud dynamics, snowpack mechanics, and temperature distribution within the snowpack. The findings contribute to a better understanding of avalanche dynamics and offer potential improvements in avalanche trend predictions.


title: NetDiffus: Network Traffic Generation by Diffusion Models through Time-Series Imaging

abstract: Network data analytics are now at the core of almost every networking solution. Nonetheless, limited access to networking data has been an enduring challenge due to many reasons including complexity of modern networks, commercial sensitivity, privacy and regulatory constraints. In this work, we explore how to leverage recent advancements in Diffusion Models (DM) to generate synthetic network traffic data. We develop an end-to-end framework - NetDiffus that first converts one-dimensional time-series network traffic into two-dimensional images, and then synthesizes representative images for the original data. We demonstrate that NetDiffus outperforms the state-of-the-art traffic generation methods based on Generative Adversarial Networks (GANs) by providing 66.4% increase in fidelity of the generated data and 18.1% increase in downstream machine learning tasks. We evaluate NetDiffus on seven diverse traffic traces and show that utilizing synthetic data significantly improves traffic fingerprinting, anomaly detection and traffic classification.