title: Modeling Large-Scale Walking and Cycling Networks: A Machine Learning Approach Using Mobile Phone and Crowdsourced Data

abstract: Walking and cycling are known to bring substantial health, environmental, and economic advantages. However, the development of evidence-based active transportation planning and policies has been impeded by significant data limitations, such as biases in crowdsourced data and representativeness issues of mobile phone data. In this study, we develop and apply a machine learning based modeling approach for estimating daily walking and cycling volumes across a large-scale regional network in New South Wales, Australia that includes 188,999 walking links and 114,885 cycling links. The modeling methodology leverages crowdsourced and mobile phone data as well as a range of other datasets on population, land use, topography, climate, etc. The study discusses the unique challenges and limitations related to all three aspects of model training, testing, and inference given the large geographical extent of the modeled networks and relative scarcity of observed walking and cycling count data. The study also proposes a new technique to identify model estimate outliers and to mitigate their impact. Overall, the study provides a valuable resource for transportation modelers, policymakers and urban planners seeking to enhance active transportation infrastructure planning and policies with advanced emerging data-driven modeling methodologies.


title: CT respiratory motion synthesis using joint supervised and adversarial learning

abstract: Objective: Four-dimensional computed tomography (4DCT) imaging consists in reconstructing a CT acquisition into multiple phases to track internal organ and tumor motion. It is commonly used in radiotherapy treatment planning to establish planning target volumes. However, 4DCT increases protocol complexity, may not align with patient breathing during treatment, and lead to higher radiation delivery. Approach: In this study, we propose a deep synthesis method to generate pseudo respiratory CT phases from static images for motion-aware treatment planning. The model produces patient-specific deformation vector fields (DVFs) by conditioning synthesis on external patient surface-based estimation, mimicking respiratory monitoring devices. A key methodological contribution is to encourage DVF realism through supervised DVF training while using an adversarial term jointly not only on the warped image but also on the magnitude of the DVF itself. This way, we avoid excessive smoothness typically obtained through deep unsupervised learning, and encourage correlations with the respiratory amplitude. Main results: Performance is evaluated using real 4DCT acquisitions with smaller tumor volumes than previously reported. Results demonstrate for the first time that the generated pseudo-respiratory CT phases can capture organ and tumor motion with similar accuracy to repeated 4DCT scans of the same patient. Mean inter-scans tumor center-of-mass distances and Dice similarity coefficients were $1.97$mm and $0.63$, respectively, for real 4DCT phases and $2.35$mm and $0.71$ for synthetic phases, and compares favorably to a state-of-the-art technique (RMSim).


title: Sequential Synthetic Difference in Differences

abstract: We study the estimation of treatment effects of a binary policy in environments with a staggered treatment rollout. We propose a new estimator -- Sequential Synthetic Difference in Difference (Sequential SDiD) -- and establish its theoretical properties in a linear model with interactive fixed effects. Our estimator is based on sequentially applying the original SDiD estimator proposed in Arkhangelsky et al. (2021) to appropriately aggregated data. To establish the theoretical properties of our method, we compare it to an infeasible OLS estimator based on the knowledge of the subspaces spanned by the interactive fixed effects. We show that this OLS estimator has a sequential representation and use this result to show that it is asymptotically equivalent to the Sequential SDiD estimator. This result implies the asymptotic normality of our estimator along with corresponding efficiency guarantees. The method developed in this paper presents a natural alternative to the conventional DiD strategies in staggered adoption designs.


title: Individual Text Corpora Predict Openness, Interests, Knowledge and Level of Education

abstract: Here we examine whether the personality dimension of openness to experience can be predicted from the individual google search history. By web scraping, individual text corpora (ICs) were generated from 214 participants with a mean number of 5 million word tokens. We trained word2vec models and used the similarities of each IC to label words, which were derived from a lexical approach of personality. These IC-label-word similarities were utilized as predictive features in neural models. For training and validation, we relied on 179 participants and held out a test sample of 35 participants. A grid search with varying number of predictive features, hidden units and boost factor was performed. As model selection criterion, we used R2 in the validation samples penalized by the absolute R2 difference between training and validation. The selected neural model explained 35% of the openness variance in the test sample, while an ensemble model with the same architecture often provided slightly more stable predictions for intellectual interests, knowledge in humanities and level of education. Finally, a learning curve analysis suggested that around 500 training participants are required for generalizable predictions. We discuss ICs as a complement or replacement of survey-based psychodiagnostics.


title: Uncovering Bias in Large Vision-Language Models with Counterfactuals

abstract: With the advent of Large Language Models (LLMs) possessing increasingly impressive capabilities, a number of Large Vision-Language Models (LVLMs) have been proposed to augment LLMs with visual inputs. Such models condition generated text on both an input image and a text prompt, enabling a variety of use cases such as visual question answering and multimodal chat. While prior studies have examined the social biases contained in text generated by LLMs, this topic has been relatively unexplored in LVLMs. Examining social biases in LVLMs is particularly challenging due to the confounding contributions of bias induced by information contained across the text and visual modalities. To address this challenging problem, we conduct a large-scale study of text generated by different LVLMs under counterfactual changes to input images. Specifically, we present LVLMs with identical open-ended text prompts while conditioning on images from different counterfactual sets, where each set contains images which are largely identical in their depiction of a common subject (e.g., a doctor), but vary only in terms of intersectional social attributes (e.g., race and gender). We comprehensively evaluate the text produced by different LVLMs under this counterfactual generation setting and find that social attributes such as race, gender, and physical characteristics depicted in input images can significantly influence toxicity and the generation of competency-associated words.


title: Electrical double layer and capacitance of TiO2 electrolyte interfaces from first principles simulations

abstract: The electrical double layer (EDL) at aqueous solution-metal oxide interfaces critically affects many fundamental processes in electrochemistry, geology and biology, yet understanding its microscopic structure is challenging for both theory and experiments. Here we employ ab initio-based machine learning potentials including long-range electrostatics in large-scale atomistic simulations of the EDL at the TiO2-electrolyte interface. Our simulations provide a molecular-scale picture of the EDL that demonstrates the limitations of standard mean-field models. We further develop a method to accurately calculate the electrostatic potential drop at the interface. The computed capacitance originating from the adsorbed charges and the potential drop agrees with experiments, supporting the reliability of our description of the EDL. The larger interfacial capacitance of basic relative to acidic solutions originates from the higher affinity of the cations for the oxide surface and gives rise to distinct charging mechanisms on negative and positive surfaces.


title: Multi-Level Neural Scene Graphs for Dynamic Urban Environments

abstract: We estimate the radiance field of large-scale dynamic areas from multiple vehicle captures under varying environmental conditions. Previous works in this domain are either restricted to static environments, do not scale to more than a single short video, or struggle to separately represent dynamic object instances. To this end, we present a novel, decomposable radiance field approach for dynamic urban environments. We propose a multi-level neural scene graph representation that scales to thousands of images from dozens of sequences with hundreds of fast-moving objects. To enable efficient training and rendering of our representation, we develop a fast composite ray sampling and rendering scheme. To test our approach in urban driving scenarios, we introduce a new, novel view synthesis benchmark. We show that our approach outperforms prior art by a significant margin on both established and our proposed benchmark while being faster in training and rendering.


title: Gravitational lensing by an ellipsoidal Navarro--Frenk--White dark-matter halo: an analytic solution and its properties

abstract: The analysis of gravitational lensing by galaxies and galaxy clusters typically relies on ellipsoidal lens models to describe the deflection of light by the involved dark-matter halos. These models are most often based on the isothermal density profile -- not an optimal description of the halo, but easy to use because it leads to an analytic deflection-angle formula. Dark-matter halos are better described by the Navarro--Frenk--White (NFW) density profile. We set out to study lensing by a general triaxial ellipsoidal NFW halo, with the aim of providing an analytic model that would be more consistent with the current understanding of dark-matter halos. We computed the conversion between the properties of a triaxial ellipsoidal lens model and its elliptical surface-density profile. In the case of the NFW lens model, its angular scale is defined by the projected scale semi-major axis of the halo, while its lensing regime depends on two parameters: the projected eccentricity $e$ and the convergence parameter $\kappa_\text{s}$. We employed the Bourassa \& Kantowski formalism to compute the complex scattering function of the model, which yields the deflection-angle components when separated into its real and imaginary parts. We present the obtained closed-form expressions for the deflection-angle components, valid for an arbitrary eccentricity of the surface-density profile. We use them to compute and describe the lensing properties of the model, including: the shear, its components, and the phase; critical curves, caustics, and the parameter-space mapping of their geometries; the deformations and orientations of images. The analytically solved ellipsoidal NFW lens model is available for implementation in gravitational lensing software. The techniques introduced here such as the image-plane analysis can prove to be useful for understanding the properties of other lens models as well.


title: Dynamic Pedestrian Traffic Assignment with Link Transmission Model for Bidirectional Sidewalk Networks

abstract: Planning assessment of the urban walking infrastructure requires appropriate methodologies that can capture the time-dependent and unique microscopic characteristics of bidirectional pedestrian flow. In this paper, we develop a simulation-based dynamic pedestrian traffic assignment (DPTA) model specifically formulated for walking networks (e.g. sidewalks) with bidirectional links. The model consists of a dynamic user equilibrium (DUE) based route choice and a link transmission model (LTM) for network loading. The formulated DUE adopts a pedestrian volume delay function (pVDF) taking into account the properties of bidirectional pedestrian streams such as self-organization. The adopted LTM uses a three-dimensional triangular bidirectional fundamental diagram as well as a generalized first-order node model. The applicability and validity of the model is demonstrated in hypothetical small networks as well as a real-world large-scale network of sidewalks in Sydney. The model successfully replicates formation and propagation of shockwaves in walking corridors and networks due to bidirectional effects.


title: No Risk, No Reward: Towards An Automated Measure of Psychological Safety from Online Communication

abstract: The data created from virtual communication platforms presents the opportunity to explore automated measures for monitoring team performance. In this work, we explore one important characteristic of successful teams - Psychological Safety - or the belief that a team is safe for interpersonal risk-taking. To move towards an automated measure of this phenomenon, we derive virtual communication characteristics and message keywords related to elements of Psychological Safety from the literature. Using a mixed methods approach, we investigate whether these characteristics are present in the Slack messages from two design teams - one high in Psychological Safety, and one low. We find that some usage characteristics, such as replies, reactions, and user mentions, might be promising metrics to indicate higher levels of Psychological Safety, while simple keyword searches may not be nuanced enough. We present the first step towards the automated detection of this important, yet complex, team characteristic.


title: Universal Bovine Identification via Depth Data and Deep Metric Learning

abstract: This paper proposes and evaluates, for the first time, a top-down (dorsal view), depth-only deep learning system for accurately identifying individual cattle and provides associated code, datasets, and training weights for immediate reproducibility. An increase in herd size skews the cow-to-human ratio at the farm and makes the manual monitoring of individuals more challenging. Therefore, real-time cattle identification is essential for the farms and a crucial step towards precision livestock farming. Underpinned by our previous work, this paper introduces a deep-metric learning method for cattle identification using depth data from an off-the-shelf 3D camera. The method relies on CNN and MLP backbones that learn well-generalised embedding spaces from the body shape to differentiate individuals -- requiring neither species-specific coat patterns nor close-up muzzle prints for operation. The network embeddings are clustered using a simple algorithm such as $k$-NN for highly accurate identification, thus eliminating the need to retrain the network for enrolling new individuals. We evaluate two backbone architectures, ResNet, as previously used to identify Holstein Friesians using RGB images, and PointNet, which is specialised to operate on 3D point clouds. We also present CowDepth2023, a new dataset containing 21,490 synchronised colour-depth image pairs of 99 cows, to evaluate the backbones. Both ResNet and PointNet architectures, which consume depth maps and point clouds, respectively, led to high accuracy that is on par with the coat pattern-based backbone.


title: Comparing Hyper-optimized Machine Learning Models for Predicting Efficiency Degradation in Organic Solar Cells

abstract: This work presents a set of optimal machine learning (ML) models to represent the temporal degradation suffered by the power conversion efficiency (PCE) of polymeric organic solar cells (OSCs) with a multilayer structure ITO/PEDOT:PSS/P3HT:PCBM/Al. To that aim, we generated a database with 996 entries, which includes up to 7 variables regarding both the manufacturing process and environmental conditions for more than 180 days. Then, we relied on a software framework that brings together a conglomeration of automated ML protocols that execute sequentially against our database by simply command-line interface. This easily permits hyper-optimizing and randomizing seeds of the ML models through exhaustive benchmarking so that optimal models are obtained. The accuracy achieved reaches values of the coefficient determination (R2) widely exceeding 0.90, whereas the root mean squared error (RMSE), sum of squared error (SSE), and mean absolute error (MAE)>1% of the target value, the PCE. Additionally, we contribute with validated models able to screen the behavior of OSCs never seen in the database. In that case, R2~0.96-0.97 and RMSE~1%, thus confirming the reliability of the proposal to predict. For comparative purposes, classical Bayesian regression fitting based on non-linear mean squares (LMS) are also presented, which only perform sufficiently for univariate cases of single OSCs. Hence they fail to outperform the breadth of the capabilities shown by the ML models. Finally, thanks to the standardized results offered by the ML framework, we study the dependencies between the variables of the dataset and their implications for the optimal performance and stability of the OSCs. Reproducibility is ensured by a standardized report altogether with the dataset, which are publicly available at Github.


title: On the weak-fragmentability index of some Lipschitz-free spaces

abstract: We show the existence of Lipschitz-free spaces verifying the Point of Continuity Property with arbitrarily high weak-fragmentability index. For this purpose, we use a generalized construction of the countably branching diamond graphs. As a consequence, we deduce that to be Lipschitz-universal for countable complete metric spaces, a separable complete metric space cannot be purely 1-unrectifiable. Another corollary is the existence of an uncountable family of pairwise non-isomorphic Lipschitz-free spaces over purely 1-unrectifiable metric spaces. Some results on compact reduction are also obtained.


title: Compact moduli of marked noncommutative cubic surfaces

abstract: We introduce a compact moduli scheme of marked noncommutative cubic surfaces as the GIT moduli scheme of relations of a quiver associated with a full strong exceptional collection on a cubic surface. It is a toric variety containing the configuration space of six points on a plane in general position as a locally closed subvariety, and birationally parametrizes a class of AS-regular $\mathbb{Z}$-algebras.


title: The LSCD Benchmark: a Testbed for Diachronic Word Meaning Tasks

abstract: Lexical Semantic Change Detection (LSCD) is a complex, lemma-level task, which is usually operationalized based on two subsequently applied usage-level tasks: First, Word-in-Context (WiC) labels are derived for pairs of usages. Then, these labels are represented in a graph on which Word Sense Induction (WSI) is applied to derive sense clusters. Finally, LSCD labels are derived by comparing sense clusters over time. This modularity is reflected in most LSCD datasets and models. It also leads to a large heterogeneity in modeling options and task definitions, which is exacerbated by a variety of dataset versions, preprocessing options and evaluation metrics. This heterogeneity makes it difficult to evaluate models under comparable conditions, to choose optimal model combinations or to reproduce results. Hence, we provide a benchmark repository standardizing LSCD evaluation. Through transparent implementation results become easily reproducible and by standardization different components can be freely combined. The repository reflects the task's modularity by allowing model evaluation for WiC, WSI and LSCD. This allows for careful evaluation of increasingly complex model components providing new ways of model optimization.


title: Quantum Transition Probabilities

abstract: Transition probabilities are an important and useful tool in quantum mechanics. However, in their present form, they are limited in scope and only apply to pure quantum states. In this article we extend their applicability to mixed states and to transitions between quantum effects. We also present their dependence on a measured operation or instrument. We begin by defining our concepts on a general quantum effect algebra. These concepts are illustrated using Holevo operations and instruments. We then present transition probabilities in the special case of the Hilbert space formulation of quantum mechanics. We show that for pure states and particular types of operations the transition probabilities reduce to their usual form. We give examples in terms of L\"uders operations and instruments.


title: Beyond Suspension: A Two-phase Methodology for Concluding Sports Leagues

abstract: Problem definition: Professional sports leagues may be suspended due to various reasons such as the recent COVID-19 pandemic. A critical question the league must address when re-opening is how to appropriately select a subset of the remaining games to conclude the season in a shortened time frame. Academic/practical relevance: Despite the rich literature on scheduling an entire season starting from a blank slate, concluding an existing season is quite different. Our approach attempts to achieve team rankings similar to that which would have resulted had the season been played out in full. Methodology: We propose a data-driven model which exploits predictive and prescriptive analytics to produce a schedule for the remainder of the season comprised of a subset of originally-scheduled games. Our model introduces novel rankings-based objectives within a stochastic optimization model, whose parameters are first estimated using a predictive model. We introduce a deterministic equivalent reformulation along with a tailored Frank-Wolfe algorithm to efficiently solve our problem, as well as a robust counterpart based on min-max regret. Results: We present simulation-based numerical experiments from previous National Basketball Association (NBA) seasons 2004--2019, and show that our models are computationally efficient, outperform a greedy benchmark that approximates a non-rankings-based scheduling policy, and produce interpretable results. Managerial implications: Our data-driven decision-making framework may be used to produce a shortened season with 25-50\% fewer games while still producing an end-of-season ranking similar to that of the full season, had it been played.


title: Multi-Region Transfer Learning for Segmentation of Crop Field Boundaries in Satellite Images with Limited Labels

abstract: The goal of field boundary delineation is to predict the polygonal boundaries and interiors of individual crop fields in overhead remotely sensed images (e.g., from satellites or drones). Automatic delineation of field boundaries is a necessary task for many real-world use cases in agriculture, such as estimating cultivated area in a region or predicting end-of-season yield in a field. Field boundary delineation can be framed as an instance segmentation problem, but presents unique research challenges compared to traditional computer vision datasets used for instance segmentation. The practical applicability of previous work is also limited by the assumption that a sufficiently-large labeled dataset is available where field boundary delineation models will be applied, which is not the reality for most regions (especially under-resourced regions such as Sub-Saharan Africa). We present an approach for segmentation of crop field boundaries in satellite images in regions lacking labeled data that uses multi-region transfer learning to adapt model weights for the target region. We show that our approach outperforms existing methods and that multi-region transfer learning substantially boosts performance for multiple model architectures. Our implementation and datasets are publicly available to enable use of the approach by end-users and serve as a benchmark for future work.


title: Differentiable nuclear deexcitation simulation for low energy neutrino physics

abstract: Neutrino-nucleus interactions play an important role in present and future neutrino experiments. The accurate simulation of these interactions at low energies ($<$100 MeV) is crucial for the detection and study of supernova, solar and atmospheric neutrinos. In particular, the reconstruction of the incoming neutrino properties depends on the ability to measure the products from the deexcitation of the final state nucleus after the initial neutrino-nucleus scattering reaction. A realistic nuclear deexcitation model that can correctly manage the theoretical uncertainties in the process is key to determine the response of a detector to low energy neutrinos and estimate the overall systematic uncertainties that affect it. Automatic differentiation frameworks like PyTorch or JAX, widely used in the field of Machine Learning, provide us the tools to compute exact gradients of the simulation outputs with respect to the model parameters. Such differentiable simulators can be applied in simulator tuning to match observed data and forward modeling to efficiently infer the impact of parameter distributions to the physics output, or for parameter inference acting as a likelihood estimator. This paradigm can pave the way to a fully differentiable analysis of the whole simulation chain or direct integration with preexisting Machine Learning tools. As a proof of concept, in this work we implement a fully differentiable nuclear deexcitation simulation based on a simplified version of the Hauser-Feshbach statistical emission model.


title: Oscillations in the Dark?

abstract: The main aim of this work is to use a model-independent approach, along with late-time observational probes, to reconstruct the dark energy (DE) equation of state $w_{\rm DE}(z)$. Our analysis showed that, for a late time universe, $w_{\rm DE}$ deviates from being a constant but in contrast exhibits an oscillatory behavior, hence both quintessence ($w_{\rm DE}> -1$) and phantom ($w_{\rm DE} < -1$) regimes are equally allowed. In order to portray this oscillatory behavior, we explored various parametrizations for the equation of state and identified the closest approximation based on the goodness of fit with the data and the Bayesian evidence analysis. Our findings indicated that while all considered oscillating DE parametrizations provided a better fit to the data, compared to the cosmological constant, they are penalized in the Bayesian evidence analysis due to the additional free parameters. Overall, the present article demonstrates that in the low redshift regime, the equation of state of the DE prefers to be dynamical and oscillating. We anticipate that future cosmological probes will take a stand in this direction.


title: On the Sobolev boundedness of vector fields on compact Riemannian manifolds

abstract: We analyse the sharpness of the Sobolev order for vector fields on compact Riemannian manifolds. As an application, we then prove ill-posedness for a certain class of Cauchy problems.


title: Shared Hardships Strengthen Bonds: Negative Shocks, Embeddedness and Employee Retention

abstract: Unexpected events -- "shocks" -- are the motive force in explaining changes in embeddedness and retention within the unfolding model of labor turnover. Substantial research effort has examined strategies for insulating valued employees from adverse shocks. However, this paper provides empirical evidence that unambiguously negative shocks can increase employee retention when underlying firm and employee incentives with respect to these shocks are aligned. Using survival analysis on a unique data set of 466,236 communication records and 45,873 employment spells from 21 trucking companies, we show how equipment-related shocks tend to increase the duration of employment. Equipment shocks also generate paradoxically positive sentiments that demonstrate an increase in employees' affective commitment to the firm. Our results highlight the important moderating role aligned incentives have in how shocks ultimately translate into retention. Shared hardships strengthen bonds in employment as in other areas.


title: Word Ladders: A Mobile Application for Semantic Data Collection

abstract: Word Ladders is a free mobile application for Android and iOS, developed for collecting linguistic data, specifically lists of words related to each other through semantic relations of categorical inclusion, within the Abstraction project (ERC-2021-STG-101039777). We hereby provide an overview of Word Ladders, explaining its game logic, motivation and expected results and applications to nlp tasks as well as to the investigation of cognitive scientific open questions


title: On Inherent Adversarial Robustness of Active Vision Systems

abstract: Current Deep Neural Networks are vulnerable to adversarial examples, which alter their predictions by adding carefully crafted noise. Since human eyes are robust to such inputs, it is possible that the vulnerability stems from the standard way of processing inputs in one shot by processing every pixel with the same importance. In contrast, neuroscience suggests that the human vision system can differentiate salient features by (1) switching between multiple fixation points (saccades) and (2) processing the surrounding with a non-uniform external resolution (foveation). In this work, we advocate that the integration of such active vision mechanisms into current deep learning systems can offer robustness benefits. Specifically, we empirically demonstrate the inherent robustness of two active vision methods - GFNet and FALcon - under a black box threat model. By learning and inferencing based on downsampled glimpses obtained from multiple distinct fixation points within an input, we show that these active methods achieve (2-3) times greater robustness compared to a standard passive convolutional network under state-of-the-art adversarial attacks. More importantly, we provide illustrative and interpretable visualization analysis that demonstrates how performing inference from distinct fixation points makes active vision methods less vulnerable to malicious inputs.


title: A Sequential Quadratic Programming Approach to the Solution of Open-Loop Generalized Nash Equilibria for Autonomous Racing

abstract: Dynamic games can be an effective approach for modeling interactive behavior between multiple competitive agents in autonomous racing and they provide a theoretical framework for simultaneous prediction and control in such scenarios. In this work, we propose DG-SQP, a numerical method for the solution of local generalized Nash equilibria (GNE) for open-loop general-sum dynamic games for agents with nonlinear dynamics and constraints. In particular, we formulate a sequential quadratic programming (SQP) approach which requires only the solution of a single convex quadratic program at each iteration. The three key elements of the method are a non-monotonic line search for solving the associated KKT equations, a merit function to handle zero sum costs, and a decaying regularization scheme for SQP step selection. We show that our method achieves linear convergence in the neighborhood of local GNE and demonstrate the effectiveness of the approach in the context of head-to-head car racing, where we show significant improvement in solver success rate when comparing against the state-of-the-art PATH solver for dynamic games. An implementation of our solver can be found at https://github.com/zhu-edward/DGSQP.


title: Portfolio management using graph centralities: Review and comparison

abstract: We investigate an application of network centrality measures to portfolio optimization, by generalizing the method in [Pozzi, Di Matteo and Aste, \emph{Spread of risks across financial markets: better to invest in the peripheries}, Scientific Reports 3:1665, 2013], that however had significant limitations with respect to the state of the art in network theory. In this paper, we systematically compare many possible variants of the originally proposed method on S\&P 500 stocks. We use daily data from twenty-seven years as training set and their following year as test set. We thus select the best network-based methods according to different viewpoints including for instance the highest Sharpe Ratio and the highest expected return. We give emphasis in new centrality measures and we also conduct a thorough analysis, which reveals significantly stronger results compared to those with more traditional methods. According to our analysis, this graph-theoretical approach to investment can be used successfully by investors with different investment profiles leading to high risk-adjusted returns.


title: DataAgent: Evaluating Large Language Models' Ability to Answer Zero-Shot, Natural Language Queries

abstract: Conventional processes for analyzing datasets and extracting meaningful information are often time-consuming and laborious. Previous work has identified manual, repetitive coding and data collection as major obstacles that hinder data scientists from undertaking more nuanced labor and high-level projects. To combat this, we evaluated OpenAI's GPT-3.5 as a "Language Data Scientist" (LDS) that can extrapolate key findings, including correlations and basic information, from a given dataset. The model was tested on a diverse set of benchmark datasets to evaluate its performance across multiple standards, including data science code-generation based tasks involving libraries such as NumPy, Pandas, Scikit-Learn, and TensorFlow, and was broadly successful in correctly answering a given data science query related to the benchmark dataset. The LDS used various novel prompt engineering techniques to effectively answer a given question, including Chain-of-Thought reinforcement and SayCan prompt engineering. Our findings demonstrate great potential for leveraging Large Language Models for low-level, zero-shot data analysis.


title: GPTA: Generative Prompt Tuning Assistant for Synergistic Downstream Neural Network Enhancement with LLMs

abstract: This study introduces GPTA, a Large Language Model assistance training framework, that enhances the training of downstream task models via prefix prompt. By minimizing data exposure to LLM, the framework addresses the security and legal challenges of applying LLM in downstream task model training. GPTA utilizes a new synergistic training approach, optimizing the downstream models with parameter gradients and LLMs with the novel ``dialogue gradient''. The framework not only demonstrates significant improvements in model performance across six NLP benchmark datasets, but also reduces overfitting in low-resource scenarios effectively. The detailed analyses further validate that our pioneer framework provides a cost-efficient and adaptive method for downstream task model training with LLM support.


title: GuaranTEE: Towards Attestable and Private ML with CCA

abstract: Machine-learning (ML) models are increasingly being deployed on edge devices to provide a variety of services. However, their deployment is accompanied by challenges in model privacy and auditability. Model providers want to ensure that (i) their proprietary models are not exposed to third parties; and (ii) be able to get attestations that their genuine models are operating on edge devices in accordance with the service agreement with the user. Existing measures to address these challenges have been hindered by issues such as high overheads and limited capability (processing/secure memory) on edge devices. In this work, we propose GuaranTEE, a framework to provide attestable private machine learning on the edge. GuaranTEE uses Confidential Computing Architecture (CCA), Arm's latest architectural extension that allows for the creation and deployment of dynamic Trusted Execution Environments (TEEs) within which models can be executed. We evaluate CCA's feasibility to deploy ML models by developing, evaluating, and openly releasing a prototype. We also suggest improvements to CCA to facilitate its use in protecting the entire ML deployment pipeline on edge devices.


title: Optimal Blackjack Strategy Recommender: A Comprehensive Study on Computer Vision Integration for Enhanced Gameplay

abstract: This research project investigates the application of several computer vision techniques for playing card detection and recognition in the context of the popular casino game, blackjack. The primary objective is to develop a robust system that is capable of detecting and accurately classifying playing cards in real-time, and displaying the optimal move recommendation based on the given image of the current game. The proposed methodology involves using K-Means for image segmentation, card reprojection and feature extraction, training of the KNN classifier using a labeled dataset, and integration of the detection system into a Blackjack Basic Strategy recommendation algorithm. Further, the study aims to observe the effectiveness of this approach in detecting various card designs under different lighting conditions and occlusions. Overall, the project examines the potential benefits of incorporating computer vision techniques, with a specific focus on card detection, into commonly played games aiming to enhance player decision-making and optimize strategic outcomes. The results obtained from our experimental evaluations with models developed under considerable time constraints, highlight the potential for practical implementation in real-world casino environments and across other similarly structured games.


title: Tools and Tasks in Sensemaking: A Visual Accessibility Perspective

abstract: Our previous interview study explores the needs and uses of diagrammatic information by the Blind and Low Vision (BLV) community, resulting in a framework called the Ladder of Diagram Access. The framework outlines five levels of information access when interacting with a diagram. In this paper, we connect this framework to include the global activity of sensemaking and discuss its (in)accessibility to the BLV demographic. We also discuss the integration of this framework into the sensemaking process and explore the current sensemaking practices and strategies employed by the BLV community, the challenges they face at different levels of the ladder, and potential solutions to enhance inclusivity towards a data-driven workforce.


title: Impact of Helium II resonant absorption on the UVB modeled in three dimensions

abstract: We implement a treatment of Helium {\sc ii} absorption and re-emission into the \TD cosmological simulations to study its impact on the metagalactic ultraviolet background (UVB) in three dimensions. By comparing simulations with and without He {\sc ii} reprocessing, we show that it weakens the mean UVB by $\sim$3 dex from $z = 10$ to $z = 5$ between 3.5 and 4 Ryd, where the He {\sc ii} Lyman-series resonance occurs. In overdense regions, the overall UVB amplitude is higher and the impact of \HeII reprocessing is weaker, qualitatively indicating an early start to \HeII reionization near galaxies. Comparing our simulations to two popular one-dimensional UV models, we find good agreement up to 3 Ryd at $z = 5$. At higher energies, our simulation shows significantly greater He {\sc ii} absorption because it accounts for He {\sc ii} arising both in diffuse regions and in Lyman limit systems. By contrast, the comparison models account only for He {\sc ii} in Lyman limit systems, which are subdominant prior to the completion of He {\sc ii} reionization. The H {\sc i} and He {\sc ii} reionization histories are nearly unaffected by He {\sc ii} reprocessing although the cosmic star formation rate density is altered by up to $4\%$. The cosmic mass density of C {\sc iv} is reduced by $\sim$2 dex when He {\sc ii} is accounted for while Si {\sc iv}, C {\sc ii}, Mg {\sc ii}, Si {\sc ii}, and O {\sc i} are unaffected.


title: A coarse Tutte polynomial for hypermaps

abstract: We give an analogue of the Tutte polynomial for hypermaps. This polynomial can be defined as either a sum over subhypermaps, or recursively through deletion-contraction reductions where the terminal forms consist of isolated vertices. Our Tutte polynomial extends the classical Tutte polynomial of a graph as well as the Tutte polynomial of an embedded graph (i.e., the ribbon graph polynomial), and it is a specialization of the transition polynomial via a medial map transformation. We give hypermap duality and partial duality identities for our polynomial, as well as some evaluations, and examine relations between our polynomial and other hypermap polynomials.


title: Multiple-policy Evaluation via Density Estimation

abstract: We study the multiple-policy evaluation problem where we are given a set of $K$ policies and the goal is to evaluate their performance (expected total reward over a fixed horizon) to an accuracy $\epsilon$ with probability at least $1-\delta$. We propose an algorithm named $\mathrm{CAESAR}$ for this problem. Our approach is based on computing an approximate optimal offline sampling distribution and using the data sampled from it to perform the simultaneous estimation of the policy values. $\mathrm{CAESAR}$ has two phases. In the first we produce coarse estimates of the visitation distributions of the target policies at a low order sample complexity rate that scales with $\tilde{O}(\frac{1}{\epsilon})$. In the second phase, we approximate the optimal offline sampling distribution and compute the importance weighting ratios for all target policies by minimizing a step-wise quadratic loss function inspired by the DualDICE \cite{nachum2019dualdice} objective. Up to low order and logarithmic terms $\mathrm{CAESAR}$ achieves a sample complexity $\tilde{O}\left(\frac{H^4}{\epsilon^2}\sum_{h=1}^H\max_{k\in[K]}\sum_{s,a}\frac{(d_h^{\pi^k}(s,a))^2}{\mu^*_h(s,a)}\right)$, where $d^{\pi}$ is the visitation distribution of policy $\pi$, $\mu^*$ is the optimal sampling distribution, and $H$ is the horizon.


title: Combined Static Analysis and Machine Learning Prediction for Application Debloating

abstract: Software debloating can effectively thwart certain code reuse attacks by reducing attack surfaces to break gadget chains. Approaches based on static analysis enable a reduced set of functions reachable at a callsite for execution by leveraging static properties of the callgraph. This achieves low runtime overhead, but the function set is conservatively computed, negatively affecting reduction. In contrast, approaches based on machine learning (ML) have much better precision and can sharply reduce function sets, leading to significant improvement in attack surface. Nevertheless, mispredictions occur in ML-based approaches. These cause overheads, and worse, there is no clear way to distinguish between mispredictions and actual attacks. In this work, we contend that a software debloating approach that incorporates ML-based predictions at runtime is realistic in a whole application setting, and that it can achieve significant attack surface reductions beyond the state of the art. We develop a framework, Predictive Debloat with Static Guarantees (PDSG). PDSG is fully sound and works on application source code. At runtime it predicts the dynamic callee set emanating from a callsite, and to resolve mispredictions, it employs a lightweight audit based on static invariants of call chains. We deduce the invariants offline and assert that they hold at runtime when there is a misprediction. To the best of our knowledge, it achieves the highest gadget reductions among similar techniques on SPEC CPU 2017, reducing 82.5% of the total gadgets on average. It triggers misprediction checks on only 3.8% of the total predictions invoked at runtime, and it leverages Datalog to verify dynamic call sequences conform to the static call relations. It has an overhead of 8.9%, which makes the scheme attractive for practical deployments.


title: Astrophysical systematics in Kinematic Lensing: quantifying an Intrinsic Alignment analog

abstract: Kinematic lensing (KL) is a new weak lensing technique that reduces shape noise for disk galaxies by including spectroscopically measured galaxy kinematics in addition to photometrically measured galaxy shapes. Since KL utilizes the Tully-Fisher relation, any correlation of this relation with the local environment may bias the cosmological interpretation. For the first time, we explore such a Tully-Fisher environmental dependence (TED) effect as a potential astrophysical systematic for KL. Our derivation of the TED systematic can be described in a similar analytical form as intrinsic alignment for traditional weak lensing. We demonstrate analytically that TED only impacts KL if intrinsic aligment for disk galaxies is non-zero. We further use IllustrisTNG simulations to quantify the TED effect. Our two-point correlation measurements do not yield any additional coherent signals that would indicate a systematic bias on KL, within the uncertainties set by the simulation volume.


title: Extending the self-discharge time of Dicke quantum batteries using molecular triplets

abstract: Quantum batteries, quantum systems for energy storage, have gained interest due to their potential scalable charging power density. A quantum battery proposal based on the Dicke model has been explored using organic microcavities, which enable a cavity-enhanced energy transfer process called superabsorption. However, energy storage lifetime in these devices is limited by fast radiative emission losses, worsened by superradiance. Here, we demonstrate a promising approach to extend the energy storage lifetime of Dicke quantum batteries using molecular triplet states. We examine a type of multi-layer microcavities where an active absorption layer transfers energy to the molecular triplets of a storage layer, identifying two regimes based on exciton-polariton resonances. We tested one of these mechanisms by fabricating and characterising five devices across a triplet-polariton resonance. We conclude by discussing potential optimisation outlooks for this class of devices.


title: An Efficient Sparse Identification Algorithm For Stochastic Systems With General Observation Sequences

abstract: This paper studies the sparse identification problem of unknown sparse parameter vectors in stochastic dynamic systems. Firstly, a novel sparse identification algorithm is proposed, which can generate sparse estimates based on least squares estimation by adaptively adjusting the threshold. Secondly, under a possibly weakest non-persistent excited condition, we prove that the proposed algorithm can correctly identify the zero and nonzero elements of the sparse parameter vector using a finite number of observations, and further estimates of the nonzero elements almost surely converge to the true values. Compared with the related works, e.g., LASSO, our method only requires the weakest assumptions and does not require solving additional optimization problems. Besides, our theoretical results do not require any statistical assumptions on the regression signals, including independence or stationarity, which makes our results promising for application to stochastic feedback systems. Thirdly, the number of finite observations that guarantee the convergence of the zero-element set of unknown sparse parameters of the Hammerstein system is derived for the first time. Finally, numerical simulations are provided, demonstrating the effectiveness of the proposed method. Since there is no additional optimization problem, i.e., no additional numerical error, the proposed algorithm performs much better than other related algorithms.


title: Managing power balance and reserve feasibility in the AC unit commitment problem

abstract: Incorporating the AC power flow equations into unit commitment models has the potential to avoid costly corrective actions required by less accurate power flow approximations. However, research on unit commitment with AC power flow constraints has been limited to a few relatively small test networks. This work investigates large-scale AC unit commitment problems for the day-ahead market and develops decomposition algorithms capable of obtaining high-quality solutions at industry-relevant scales. The results illustrate that a simple algorithm that only seeks to satisfy unit commitment, reserve, and AC power balance constraints can obtain surprisingly high-quality solutions to this AC unit commitment problem. However, a naive strategy that prioritizes reserve feasibility leads to AC infeasibility, motivating the need to design heuristics that can effectively balance reserve and AC feasibility. Finally, this work explores a parallel decomposition strategy that allows the proposed algorithm to obtain feasible solutions on large cases within the two hour time limit required by typical day-ahead market operations.


title: Angular analysis of $B \to K^* e^+ e^-$ in the low-$q^2$ region with new electron identification at Belle

abstract: We perform an angular analysis of the $B\to K^* e^+ e^-$ decay for the dielectron mass squared, $q^2$, range of $0.0008$ to $1.1200 ~\text{GeV}^2 /c^4$ using the full Belle data set in the $K^{*0} \to K^+ \pi^-$ and $K^{*+} \to K_S^0 \pi^+$ channels, incorporating new methods of electron identification to improve the statistical power of the data set. This analysis is sensitive to contributions from right-handed currents from physics beyond the Standard Model by constraining the Wilson coefficients $\mathcal{C}_7^{(\prime)}$. We perform a fit to the $B\to K^* e^+ e^-$ differential decay rate and measure the imaginary component of the transversality amplitude to be $A_T^{\rm Im} = -1.27 \pm 0.52 \pm 0.12$, and the $K^*$ transverse asymmetry to be $A_T^{(2)} = 0.52 \pm 0.53 \pm 0.11$. The resulting constraints on the value of $\mathcal{C}_7^{\prime}$ are consistent with the Standard Model within a $2\sigma$ confidence interval.


title: Reaction dynamics with qubit-efficient momentum-space mapping

abstract: Description of quantum many-body dynamics is extremely challenging on classical computers, as it can involve many degrees of freedom. On the other hand, the time evolution of quantum states is a natural application for quantum computers, which are designed to efficiently perform unitary transformations. In this paper we study quantum algorithms for response functions, relevant for describing different reactions governed by linear response. We consider a qubit-efficient mapping on a lattice, which can be efficiently performed using momentum-space basis states. We analyze the advantages and disadvantages of this approach, focusing on the nuclear two-body system and a typical response function relevant for electron scattering as an example. We investigate ground-state preparation, controlled time evolution and the required measurements. We examine circuit depth and the hardware noise level required to interpret the signal.


title: No-Regret Learning for Stackelberg Equilibrium Computation in Newsvendor Pricing Games

abstract: We introduce the application of online learning in a Stackelberg game pertaining to a system with two learning agents in a dyadic exchange network, consisting of a supplier and retailer, specifically where the parameters of the demand function are unknown. In this game, the supplier is the first-moving leader, and must determine the optimal wholesale price of the product. Subsequently, the retailer who is the follower, must determine both the optimal procurement amount and selling price of the product. In the perfect information setting, this is known as the classical price-setting Newsvendor problem, and we prove the existence of a unique Stackelberg equilibrium when extending this to a two-player pricing game. In the framework of online learning, the parameters of the reward function for both the follower and leader must be learned, under the assumption that the follower will best respond with optimism under uncertainty. A novel algorithm based on contextual linear bandits with a measurable uncertainty set is used to provide a confidence bound on the parameters of the stochastic demand. Consequently, optimal finite time regret bounds on the Stackelberg regret, along with convergence guarantees to an approximate Stackelberg equilibrium, are provided.


title: A PPO-based DRL Auto-Tuning Nonlinear PID Drone Controller for Robust Autonomous Flights

abstract: This project aims to revolutionize drone flight control by implementing a nonlinear Deep Reinforcement Learning (DRL) agent as a replacement for traditional linear Proportional Integral Derivative (PID) controllers. The primary objective is to seamlessly transition drones between manual and autonomous modes, enhancing responsiveness and stability. We utilize the Proximal Policy Optimization (PPO) reinforcement learning strategy within the Gazebo simulator to train the DRL agent. Adding a $20,000 indoor Vicon tracking system offers <1mm positioning accuracy, which significantly improves autonomous flight precision. To navigate the drone in the shortest collision-free trajectory, we also build a 3 dimensional A* path planner and implement it into the real flight successfully.


title: Conceptual and Unbiased Reasoning in Language Models

abstract: Conceptual reasoning, the ability to reason in abstract and high-level perspectives, is key to generalization in human cognition. However, limited study has been done on large language models' capability to perform conceptual reasoning. In this work, we bridge this gap and propose a novel conceptualization framework that forces models to perform conceptual reasoning on abstract questions and generate solutions in a verifiable symbolic space. Using this framework as an analytical tool, we show that existing large language models fall short on conceptual reasoning, dropping 9% to 28% on various benchmarks compared to direct inference methods. We then discuss how models can improve since high-level abstract reasoning is key to unbiased and generalizable decision-making. We propose two techniques to add trustworthy induction signals by generating familiar questions with similar underlying reasoning paths and asking models to perform self-refinement. Experiments show that our proposed techniques improve models' conceptual reasoning performance by 8% to 11%, achieving a more robust reasoning system that relies less on inductive biases.


title: Random Reed-Solomon Codes are List Recoverable with Optimal List Size

abstract: We prove that Reed-Solomon (RS) codes with random evaluation points are list recoverable up to capacity with optimal output list size, for any input list size. Namely, given an input list size $\ell$, a designated rate $R$, and any $\varepsilon > 0$, we show that a random RS code is list recoverable from $1-R-\varepsilon$ fraction of errors with output list size $L = O(\ell/\varepsilon)$, for field size $q=\exp(\ell,1/\varepsilon) \cdot n^2$. In particular, this shows that random RS codes are list recoverable beyond the "list recovery Johnson bound". Such a result was not even known for arbitrary random linear codes. Our technique follows and extends the recent line of work on list decoding of random RS codes, specifically the works of Brakensiek, Gopi, and Makam (STOC 2023), and of Guo and Zhang (FOCS 2023).


title: Causal Inference for Human-Language Model Collaboration

abstract: In this paper, we examine the collaborative dynamics between humans and language models (LMs), where the interactions typically involve LMs proposing text segments and humans editing or responding to these proposals. Productive engagement with LMs in such scenarios necessitates that humans discern effective text-based interaction strategies, such as editing and response styles, from historical human-LM interactions. This objective is inherently causal, driven by the counterfactual `what-if' question: how would the outcome of collaboration change if humans employed a different text editing/refinement strategy? A key challenge in answering this causal inference question is formulating an appropriate causal estimand: the conventional average treatment effect (ATE) estimand is inapplicable to text-based treatments due to their high dimensionality. To address this concern, we introduce a new causal estimand -- Incremental Stylistic Effect (ISE) -- which characterizes the average impact of infinitesimally shifting a text towards a specific style, such as increasing formality. We establish the conditions for the non-parametric identification of ISE. Building on this, we develop CausalCollab, an algorithm designed to estimate the ISE of various interaction strategies in dynamic human-LM collaborations. Our empirical investigations across three distinct human-LM collaboration scenarios reveal that CausalCollab effectively reduces confounding and significantly improves counterfactual estimation over a set of competitive baselines.


title: Discrete Natural Evolution Strategies

abstract: Natural evolution strategies are a class of approximate-gradient black-box optimizers that have been successfully used for continuous parameter spaces. In this paper, we derive NES algorithms for discrete parameter spaces and demonstrate their effectiveness in tasks involving discrete parameters.


title: EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge Graphs

abstract: Narrative reasoning relies on the understanding of eventualities in story contexts, which requires a wealth of background world knowledge. To help machines leverage such knowledge, existing solutions can be categorized into two groups. Some focus on implicitly modeling eventuality knowledge by pretraining language models (LMs) with eventuality-aware objectives. However, this approach breaks down knowledge structures and lacks interpretability. Others explicitly collect world knowledge of eventualities into structured eventuality-centric knowledge graphs (KGs). However, existing research on leveraging these knowledge sources for free-texts is limited. In this work, we propose an initial comprehensive framework called EventGround, which aims to tackle the problem of grounding free-texts to eventuality-centric KGs for contextualized narrative reasoning. We identify two critical problems in this direction: the event representation and sparsity problems. We provide simple yet effective parsing and partial information extraction methods to tackle these problems. Experimental results demonstrate that our approach consistently outperforms baseline models when combined with graph neural network (GNN) or large language model (LLM) based graph reasoning models. Our framework, incorporating grounded knowledge, achieves state-of-the-art performance while providing interpretable evidence.


title: Socially Aware Robot Navigation through Scoring Using Vision-Language Models

abstract: We propose VLM-Social-Nav, a novel Vision-Language Model (VLM) based navigation approach to compute a robot's trajectory in human-centered environments. Our goal is to make real-time decisions on robot actions that are socially compliant with human expectations. We utilize a perception model to detect important social entities and prompt a VLM to generate guidance for socially compliant robot behavior. VLM-Social-Nav uses a VLM-based scoring module that computes a cost term that ensures socially appropriate and effective robot actions generated by the underlying planner. Our overall approach reduces reliance on large datasets (for training) and enhances adaptability in decision-making. In practice, it results in improved socially compliant navigation in human-shared environments. We demonstrate and evaluate our system in four different real-world social navigation scenarios with a Turtlebot robot. We observe at least 36.37% improvement in average success rate and 20.00% improvement in average collision rate in the four social navigation scenarios. The user study score shows that VLM-Social-Nav generates the most socially compliant navigation behavior.


title: Multi-Conditional Ranking with Large Language Models

abstract: Utilizing large language models (LLMs) to rank a set of items has become a common approach in recommendation and retrieval systems. Typically, these systems focus on ordering a substantial number of documents in a monotonic order based on a given query. However, real-world scenarios often present a different challenge: ranking a comparatively smaller set of items, but according to a variety of diverse and occasionally conflicting conditions. In this paper, we define and explore the task of multi-conditional ranking by introducing MCRank, a benchmark tailored for assessing multi-conditional ranking across various item types and conditions. Our analysis of LLMs using MCRank indicates a significant decrease in performance as the number and complexity of items and conditions grow. To overcome this limitation, we propose a novel decomposed reasoning method, consisting of EXtracting and Sorting the conditions, and then Iterativly Ranking the items (EXSIR). Our extensive experiments show that this decomposed reasoning method enhances LLMs' performance significantly, achieving up to a 12% improvement over existing LLMs. We also provide a detailed analysis of LLMs performance across various condition categories, and examine the effectiveness of decomposition step. Furthermore, we compare our method with existing approaches such as Chain-of-Thought and an encoder-type ranking model, demonstrating the superiority of our approach and complexity of MCR task. We released our dataset and code.


title: Cost-sensitive computational adequacy of higher-order recursion in synthetic domain theory

abstract: We study a cost-aware programming language for higher-order recursion dubbed $\textbf{PCF}_\mathsf{cost}$ in the setting of synthetic domain theory (SDT). Our main contribution relates the denotational cost semantics of $\textbf{PCF}_\mathsf{cost}$ to its computational cost semantics, a new kind of dynamic semantics for program execution that serves as a mathematically natural alternative to operational semantics in SDT. In particular we prove an internal, cost-sensitive version of Plotkin's computational adequacy theorem, giving a precise correspondence between the denotational and computational semantics for complete programs at base type. The constructions and proofs of this paper take place in the internal dependent type theory of an SDT topos extended by a phase distinction in the sense of Sterling and Harper. By controlling the interpretation of cost structure via the phase distinction in the denotational semantics, we show that $\textbf{PCF}_\mathsf{cost}$ programs also evince a noninterference property of cost and behavior. We verify the axioms of the type theory by means of a model construction based on relative sheaf models of SDT.


title: Injecting New Knowledge into Large Language Models via Supervised Fine-Tuning

abstract: In recent years, Large Language Models (LLMs) have shown remarkable performance in generating human-like text, proving to be a valuable asset across various applications. However, adapting these models to incorporate new, out-of-domain knowledge remains a challenge, particularly for facts and events that occur after the model's knowledge cutoff date. This paper investigates the effectiveness of Supervised Fine-Tuning (SFT) as a method for knowledge injection in LLMs, specifically focusing on the domain of recent sporting events. We compare different dataset generation strategies -- token-based and fact-based scaling -- to create training data that helps the model learn new information. Our experiments on GPT-4 demonstrate that while token-based scaling can lead to improvements in Q&A accuracy, it may not provide uniform coverage of new knowledge. Fact-based scaling, on the other hand, offers a more systematic approach to ensure even coverage across all facts. We present a novel dataset generation process that leads to more effective knowledge ingestion through SFT, and our results show considerable performance improvements in Q&A tasks related to out-of-domain knowledge. This study contributes to the understanding of domain adaptation for LLMs and highlights the potential of SFT in enhancing the factuality of LLM responses in specific knowledge domains.


title: Ferromagnetic state with large magnetic moments realized in epitaxially strained Sr3Ru2O7 films

abstract: Technical advancement of oxide molecular beam epitaxy (MBE) has opened new avenues for studying various quantum transport phenomena in correlated transition-metal oxides, as exemplified by the exotic superconductivity of Sr$_2$RuO$_4$ and quantum oscillations of SrRuO$_3$. On the other hand, film research of another Ruddlesden-Popper strontium ruthenate Sr$_3$Ru$_2$O$_7$ which exhibits a unique quantum phase related to metamagnetism in bulk systems did not progress well. Here we report the fabrication of high-quality Sr$_3$Ru$_2$O$_7$ thin films by oxide MBE and the observation of a strain-induced ferromagnetic ground state. The change in magnetic exchange coupling evaluated by first-principles calculations indicates a systematic relation between the compression of the $c$-axis length and induced ferromagnetism. Giant epitaxial strain in high-quality films will be a key to a comprehensive understanding of the magnetism in Ruddlesden-Popper strontium ruthenates Sr$_{n+1}$Ru$_n$O$_{3n+1}$, which sensitively depends on the ratio of in-plane to out-of-plane Ru-Ru distances.


title: Minimizing the Profligacy of Searches with Reset

abstract: We introduce the profligacy of a search process as a competition between its expected cost and the probability of finding the target. The arbiter of the competition is a parameter $\lambda$ that represents how much a searcher invests into increasing the chance of success. Minimizing the profligacy with respect to the search strategy specifies the optimal search. We show that in the case of diffusion with stochastic resetting, the amount of resetting in the optimal strategy has a highly nontrivial dependence on model parameters resulting in classical continuous transitions, discontinuous transitions and tricritical points as well as non-standard discontinuous transitions exhibiting re-entrant behavior and overhangs.


title: Is Factuality Decoding a Free Lunch for LLMs? Evaluation on Knowledge Editing Benchmark

abstract: The rapid development of large language models (LLMs) enables them to convey factual knowledge in a more human-like fashion. Extensive efforts have been made to reduce factual hallucinations by modifying LLMs with factuality decoding. However, they also pose risks of hindering knowledge updates, as they make models overly confident in known facts. In this work, we first revisite the current factuality decoding methods and verified their effectiveness in enhancing factual accuracy. Subsequently, we conduct further evaluation of several strong factuality decoding methods on the knowledge editing benchmark. All these decoding methods significantly diminish the performance of llama2 models compared to their original decoding, with the largest decrease being a staggering 81.3\%. This further indicates that the current existing decoding methods still cannot perfectly address the factual hallucinations, as they overlook the importance of preserving the flexibility for knowledge editing. Therefore, our work suggests that research into factual alignment should simultaneously focus on the effectiveness of knowledge editing.


title: Rationale-based Opinion Summarization

abstract: Opinion summarization aims to generate concise summaries that present popular opinions of a large group of reviews. However, these summaries can be too generic and lack supporting details. To address these issues, we propose a new paradigm for summarizing reviews, rationale-based opinion summarization. Rationale-based opinion summaries output the representative opinions as well as one or more corresponding rationales. To extract good rationales, we define four desirable properties: relatedness, specificity, popularity, and diversity and present a Gibbs-sampling-based method to extract rationales. Overall, we propose RATION, an unsupervised extractive system that has two components: an Opinion Extractor (to extract representative opinions) and Rationales Extractor (to extract corresponding rationales). We conduct automatic and human evaluations to show that rationales extracted by RATION have the proposed properties and its summaries are more useful than conventional summaries. The implementation of our work is available at https://github.com/leehaoyuan/RATION.


title: Functional-Edged Network Modeling

abstract: Contrasts with existing works which all consider nodes as functions and use edges to represent the relationships between different functions. We target at network modeling whose edges are functional data and transform the adjacency matrix into a functional adjacency tensor, introducing an additional dimension dedicated to function representation. Tucker functional decomposition is used for the functional adjacency tensor, and to further consider the community between nodes, we regularize the basis matrices to be symmetrical. Furthermore, to deal with irregular observations of the functional edges, we conduct model inference to solve a tensor completion problem. It is optimized by a Riemann conjugate gradient descent method. Besides these, we also derive several theorems to show the desirable properties of the functional edged network model. Finally, we evaluate the efficacy of our proposed model using simulation data and real metro system data from Hong Kong and Singapore.


title: Complete universal scaling in first-order phase transitions

abstract: Phase transitions and critical phenomena are among the most intriguing phenomena in nature and society. They are classified as first-order phase transitions (FOPTs) and continuous ones. While the latter show marvelous phenomena of scaling and universality, whether the former behaves similarly is a long-standing controversial issue. Here we definitely demonstrate complete universal scaling in field driven FOPTs for Langevin equations in both zero and two spatial dimensions by rescaling all parameters and subtracting extra contributions with singular dimensions from an effective temperature and a special field according to an effective theory that possesses a fixed point of a seemingly un-physical imaginary value. This offers a perspective different from the usual nucleation and growth but conforming to continuous phase transitions to study FOPTs.


title: Partially-Observable Sequential Change-Point Detection for Autocorrelated Data via Upper Confidence Region

abstract: Sequential change point detection for multivariate autocorrelated data is a very common problem in practice. However, when the sensing resources are limited, only a subset of variables from the multivariate system can be observed at each sensing time point. This raises the problem of partially observable multi-sensor sequential change point detection. For it, we propose a detection scheme called adaptive upper confidence region with state space model (AUCRSS). It models multivariate time series via a state space model (SSM), and uses an adaptive sampling policy for efficient change point detection and localization. A partially-observable Kalman filter algorithm is developed for online inference of SSM, and accordingly, a change point detection scheme based on a generalized likelihood ratio test is developed. How its detection power relates to the adaptive sampling strategy is analyzed. Meanwhile, by treating the detection power as a reward, its connection with the online combinatorial multi-armed bandit (CMAB) problem is formulated and an adaptive upper confidence region algorithm is proposed for adaptive sampling policy design. Theoretical analysis of the asymptotic average detection delay is performed, and thorough numerical studies with synthetic data and real-world data are conducted to demonstrate the effectiveness of our method.


title: Robust Learning for Optimal Dynamic Treatment Regimes with Observational Data

abstract: Many public policies and medical interventions involve dynamics in their treatment assignments, where treatments are sequentially assigned to the same individuals across multiple stages, and the effect of treatment at each stage is usually heterogeneous with respect to the history of prior treatments and associated characteristics. We study statistical learning of optimal dynamic treatment regimes (DTRs) that guide the optimal treatment assignment for each individual at each stage based on the individual's history. We propose a step-wise doubly-robust approach to learn the optimal DTR using observational data under the assumption of sequential ignorability. The approach solves the sequential treatment assignment problem through backward induction, where, at each step, we combine estimators of propensity scores and action-value functions (Q-functions) to construct augmented inverse probability weighting estimators of values of policies for each stage. The approach consistently estimates the optimal DTR if either a propensity score or Q-function for each stage is consistently estimated. Furthermore, the resulting DTR can achieve the optimal convergence rate $n^{-1/2}$ of regret under mild conditions on the convergence rate for estimators of the nuisance parameters.


title: Positivity preservers over finite fields

abstract: We resolve an algebraic version of Schoenberg's celebrated theorem [Duke Math. J., 1942] characterizing entrywise matrix transforms that preserve positive definiteness. Compared to the classical real and complex settings, we consider matrices with entries in a finite field and obtain a complete characterization of such preservers for matrices of a fixed dimension. When the dimension of the matrices is at least $3$, we prove that, surprisingly, the positivity preservers are precisely the positive multiples of the field's automorphisms. Our work makes crucial use of the well-known character-sum bound due to Weil, and of a result of Carlitz [Proc. Amer. Math. Soc., 1960] that provides a characterization of the automorphisms of Paley graphs.


title: Decoding rotating neutron stars: Role of the symmetry energy slope

abstract: In December 2023, the Fermi LAT Catalog announced the discovery of 33 new millisecond pulsars. Motivated by that, in this work, I study how different values of the symmetry energy slope $(L)$ affect the properties of static and slowly rotating neutron stars. For fixed values of angular velocity, I study how the slope influences the increase of the maximum mass, the radii of the canonical 1.4 solar mass, its eccentricity, as well the same quantities for the 2.01 $M_\odot$ stars. I show that different slope values cause different variations not only the absolute quantities but also in relative ones. Indeed, different slope values predict different values for the eccentricity, which does not depend on the absolute value of the neutron stars' radii. Therefore, this quantity can be a powerful tool to constrain the symmetry energy slope.


title: Classification and Clustering of Sentence-Level Embeddings of Scientific Articles Generated by Contrastive Learning

abstract: Scientific articles are long text documents organized into sections, each describing aspects of the research. Analyzing scientific production has become progressively challenging due to the increase in the number of available articles. Within this scenario, our approach consisted of fine-tuning transformer language models to generate sentence-level embeddings from scientific articles, considering the following labels: background, objective, methods, results, and conclusion. We trained our models on three datasets with contrastive learning. Two datasets are from the article's abstracts in the computer science and medical domains. Also, we introduce PMC-Sents-FULL, a novel dataset of sentences extracted from the full texts of medical articles. We compare the fine-tuned and baseline models in clustering and classification tasks to evaluate our approach. On average, clustering agreement measures values were five times higher. For the classification measures, in the best-case scenario, we had an average improvement in F1-micro of 30.73\%. Results show that fine-tuning sentence transformers with contrastive learning and using the generated embeddings in downstream tasks is a feasible approach to sentence classification in scientific articles. Our experiment codes are available on GitHub.


title: Heterogeneous Contrastive Learning for Foundation Models and Beyond

abstract: In the era of big data and Artificial Intelligence, an emerging paradigm is to utilize contrastive self-supervised learning to model large-scale heterogeneous data. Many existing foundation models benefit from the generalization capability of contrastive self-supervised learning by learning compact and high-quality representations without relying on any label information. Amidst the explosive advancements in foundation models across multiple domains, including natural language processing and computer vision, a thorough survey on heterogeneous contrastive learning for the foundation model is urgently needed. In response, this survey critically evaluates the current landscape of heterogeneous contrastive learning for foundation models, highlighting the open challenges and future trends of contrastive learning. In particular, we first present how the recent advanced contrastive learning-based methods deal with view heterogeneity and how contrastive learning is applied to train and fine-tune the multi-view foundation models. Then, we move to contrastive learning methods for task heterogeneity, including pretraining tasks and downstream tasks, and show how different tasks are combined with contrastive learning loss for different purposes. Finally, we conclude this survey by discussing the open challenges and shedding light on the future directions of contrastive learning.


title: Design as Desired: Utilizing Visual Question Answering for Multimodal Pre-training

abstract: Multimodal pre-training demonstrates its potential in the medical domain, which learns medical visual representations from paired medical reports. However, many pre-training tasks require extra annotations from clinicians, and most of them fail to explicitly guide the model to learn the desired features of different pathologies. To the best of our knowledge, we are the first to utilize Visual Question Answering (VQA) for multimodal pre-training to guide the framework focusing on targeted pathological features. In this work, we leverage descriptions in medical reports to design multi-granular question-answer pairs associated with different diseases, which assist the framework in pre-training without requiring extra annotations from experts. We also propose a novel pre-training framework with a quasi-textual feature transformer, a module designed to transform visual features into a quasi-textual space closer to the textual domain via a contrastive learning strategy. This narrows the vision-language gap and facilitates modality alignment. Our framework is applied to four downstream tasks: report generation, classification, segmentation, and detection across five datasets. Extensive experiments demonstrate the superiority of our framework compared to other state-of-the-art methods. Our code will be released upon acceptance.


title: A Survey of using Large Language Models for Generating Infrastructure as Code

abstract: Infrastructure as Code (IaC) is a revolutionary approach which has gained significant prominence in the Industry. IaC manages and provisions IT infrastructure using machine-readable code by enabling automation, consistency across the environments, reproducibility, version control, error reduction and enhancement in scalability. However, IaC orchestration is often a painstaking effort which requires specialised skills as well as a lot of manual effort. Automation of IaC is a necessity in the present conditions of the Industry and in this survey, we study the feasibility of applying Large Language Models (LLM) to address this problem. LLMs are large neural network-based models which have demonstrated significant language processing abilities and shown to be capable of following a range of instructions within a broad scope. Recently, they have also been adapted for code understanding and generation tasks successfully, which makes them a promising choice for the automatic generation of IaC configurations. In this survey, we delve into the details of IaC, usage of IaC in different platforms, their challenges, LLMs in terms of code-generation aspects and the importance of LLMs in IaC along with our own experiments. Finally, we conclude by presenting the challenges in this area and highlighting the scope for future research.


title: InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning

abstract: Continual learning requires the model to learn multiple tasks sequentially. In continual learning, the model should possess the ability to maintain its performance on old tasks (stability) and the ability to adapt to new tasks continuously (plasticity). Recently, parameter-efficient fine-tuning (PEFT), which involves freezing a pre-trained model and injecting a small number of learnable parameters to adapt to downstream tasks, has gained increasing popularity in continual learning. Although existing continual learning methods based on PEFT have demonstrated superior performance compared to those not based on PEFT, most of them do not consider how to eliminate the interference of the new task on the old tasks, which inhibits the model from making a good trade-off between stability and plasticity. In this work, we propose a new PEFT method, called interference-free low-rank adaptation (InfLoRA), for continual learning. InfLoRA injects a small number of parameters to reparameterize the pre-trained weights and shows that fine-tuning these injected parameters is equivalent to fine-tuning the pre-trained weights within a subspace. Furthermore, InfLoRA designs this subspace to eliminate the interference of the new task on the old tasks, making a good trade-off between stability and plasticity. Experimental results show that InfLoRA outperforms existing state-of-the-art continual learning methods on multiple datasets.


title: Rotating-modulated Higher-Order Topological States in a Split-ring Photonic Insulator

abstract: The emerging field of topology has brought device effects to a new level. Higher-order topological insulators (HOTIs) go beyond traditional descriptions of bulk-edge correspondence, broadening the understanding of topologically insulating phases. In this paper, a second-order split-ring photonic crystal (SSPC) with zero-dimensional (0D) corner states and one-dimensional (1D) edge states is proposed. Based on the coupling strength determined by the opening direction between the split-rings, the electronic transition strength of the electronic system is imitated, and the topological trivial and non-trivial transformation of the topological two-dimensional (2D) SSH model are realized by using the rotating split-ring lattice. Theory and simulation find that SSPC has non-trivial topological edge states that can be quantified by bulk polarization. As the opening direction of the split-rings gradually changes within one period, there will be transitions between four different topological polarizations of the lowest energy bands, which can be conveniently used to achieve transitions between different topological phases. Our research can be extended to higher dimensions and broaden research paths for higher-order photonic topological insulators and semimetals.


title: Latent Watermark: Inject and Detect Watermarks in Latent Diffusion Space

abstract: Watermarking is a tool for actively identifying and attributing the images generated by latent diffusion models. Existing methods face the dilemma of watermark robustness and image quality. The reason for this dilemma is that watermark detection is performed in pixel space, implying an intrinsic link between image quality and watermark robustness. In this paper, we highlight that an effective solution to the problem is to both inject and detect watermarks in latent space, and propose Latent Watermark (LW) with a progressive training strategy. Experiments show that compared to the recently proposed methods such as StegaStamp, StableSignature, RoSteALS and TreeRing, LW not only surpasses them in terms of robustness but also offers superior image quality. When we inject 64-bit messages, LW can achieve an identification performance close to 100% and an attribution performance above 97% under 9 single-attack scenarios and one all-attack scenario. Our code will be available on GitHub.


title: Attention-based Shape-Deformation Networks for Artifact-Free Geometry Reconstruction of Lumbar Spine from MR Images

abstract: Lumbar disc degeneration, a progressive structural wear and tear of lumbar intervertebral disc, is regarded as an essential role on low back pain, a significant global health concern. Automated lumbar spine geometry reconstruction from MR images will enable fast measurement of medical parameters to evaluate the lumbar status, in order to determine a suitable treatment. Existing image segmentation-based techniques often generate erroneous segments or unstructured point clouds, unsuitable for medical parameter measurement. In this work, we present $\textit{UNet-DeformSA}$ and $\textit{TransDeformer}$: novel attention-based deep neural networks that reconstruct the geometry of the lumbar spine with high spatial accuracy and mesh correspondence across patients, and we also present a variant of $\textit{TransDeformer}$ for error estimation. Specially, we devise new attention modules with a new attention formula, which integrate image features and tokenized contour features to predict the displacements of the points on a shape template without the need for image segmentation. The deformed template reveals the lumbar spine geometry in an image. Experiment results show that our networks generate artifact-free geometry outputs, and the variant of $\textit{TransDeformer}$ can predict the errors of a reconstructed geometry. Our code is available at https://github.com/linchenq/TransDeformer-Mesh.


title: Efficient Automatic Tuning for Data-driven Model Predictive Control via Meta-Learning

abstract: AutoMPC is a Python package that automates and optimizes data-driven model predictive control. However, it can be computationally expensive and unstable when exploring large search spaces using pure Bayesian Optimization (BO). To address these issues, this paper proposes to employ a meta-learning approach called Portfolio that improves AutoMPC's efficiency and stability by warmstarting BO. Portfolio optimizes initial designs for BO using a diverse set of configurations from previous tasks and stabilizes the tuning process by fixing initial configurations instead of selecting them randomly. Experimental results demonstrate that Portfolio outperforms the pure BO in finding desirable solutions for AutoMPC within limited computational resources on 11 nonlinear control simulation benchmarks and 1 physical underwater soft robot dataset.


title: On a stability of higher level Coxeter unipotent representations

abstract: Let $\mathbb{G}$ be a connected reductive group over $\mathcal{O}$, a complete discrete valuation ring with finite residue field $\mathbb{F}_q$. Let $R_{T_r,U_r}^{\theta}$ be a level $r$ Deligne--Lusztig representation of $\mathbb{G}(\mathcal{O})$. We show that, if $q$ is not small, the Coxeter unipotent $R_{T_r,U_r}^1$ degenerates to the $r=1$ case. For $\mathbb{G}=\mathrm{GL}_2$ (or $\mathrm{SL}_2$), as an application we give the dimensions and decompositions of all Coxeter $R_{T_r,U_r}^{\theta}$. For general $\mathbb{G}$ we state a conjectural sign formula for $R_{T_r,U_r}^{\theta}$.


title: Grid Diffusion Models for Text-to-Video Generation

abstract: Recent advances in the diffusion models have significantly improved text-to-image generation. However, generating videos from text is a more challenging task than generating images from text, due to the much larger dataset and higher computational cost required. Most existing video generation methods use either a 3D U-Net architecture that considers the temporal dimension or autoregressive generation. These methods require large datasets and are limited in terms of computational costs compared to text-to-image generation. To tackle these challenges, we propose a simple but effective novel grid diffusion for text-to-video generation without temporal dimension in architecture and a large text-video paired dataset. We can generate a high-quality video using a fixed amount of GPU memory regardless of the number of frames by representing the video as a grid image. Additionally, since our method reduces the dimensions of the video to the dimensions of the image, various image-based methods can be applied to videos, such as text-guided video manipulation from image manipulation. Our proposed method outperforms the existing methods in both quantitative and qualitative evaluations, demonstrating the suitability of our model for real-world video generation.


title: Information Security and Privacy in the Digital World: Some Selected Topics

abstract: In the era of generative artificial intelligence and the Internet of Things, while there is explosive growth in the volume of data and the associated need for processing, analysis, and storage, several new challenges are faced in identifying spurious and fake information and protecting the privacy of sensitive data. This has led to an increasing demand for more robust and resilient schemes for authentication, integrity protection, encryption, non-repudiation, and privacy-preservation of data. The chapters in this book present some of the state-of-the-art research works in the field of cryptography and security in computing and communications.


title: Enhancing Content-based Recommendation via Large Language Model

abstract: In real-world applications, users express different behaviors when they interact with different items, including implicit click/like interactions, and explicit comments/reviews interactions. Nevertheless, almost all recommender works are focused on how to describe user preferences by the implicit click/like interactions, to find the synergy of people. For the content-based explicit comments/reviews interactions, some works attempt to utilize them to mine the semantic knowledge to enhance recommender models. However, they still neglect the following two points: (1) The content semantic is a universal world knowledge; how do we extract the multi-aspect semantic information to empower different domains? (2) The user/item ID feature is a fundamental element for recommender models; how do we align the ID and content semantic feature space? In this paper, we propose a `plugin' semantic knowledge transferring method \textbf{LoID}, which includes two major components: (1) LoRA-based large language model pretraining to extract multi-aspect semantic information; (2) ID-based contrastive objective to align their feature spaces. We conduct extensive experiments with SOTA baselines on real-world datasets, the detailed results demonstrating significant improvements of our method LoID.


title: Joint Pedestrian Trajectory Prediction through Posterior Sampling

abstract: Joint pedestrian trajectory prediction has long grappled with the inherent unpredictability of human behaviors. Recent investigations employing variants of conditional diffusion models in trajectory prediction have exhibited notable success. Nevertheless, the heavy dependence on accurate historical data results in their vulnerability to noise disturbances and data incompleteness. To improve the robustness and reliability, we introduce the Guided Full Trajectory Diffuser (GFTD), a novel diffusion model framework that captures the joint full (historical and future) trajectory distribution. By learning from the full trajectory, GFTD can recover the noisy and missing data, hence improving the robustness. In addition, GFTD can adapt to data imperfections without additional training requirements, leveraging posterior sampling for reliable prediction and controllable generation. Our approach not only simplifies the prediction process but also enhances generalizability in scenarios with noise and incomplete inputs. Through rigorous experimental evaluation, GFTD exhibits superior performance in both trajectory prediction and controllable generation.


title: Flattening a trapped atomic gas using a programmable optical potential in a feedback loop

abstract: We present a method for producing a flat, large-area Fermi gas of $^6$Li with a uniform area density. The method uses a programmable optical potential within a feedback loop to flatten the in-plane trapping potential for atoms. The optical potential is generated using a laser beam, whose intensity profile is adjusted by a spatial light modulator and optimized through measurements of the density distribution of the sample. The resulting planar sample exhibits a uniform area density within a region of about 480 $\mu$m in diameter and the standard deviation of the trap bottom potential is estimated to be $\approx k_B \times$ 6.1 nK, which is less than 20$\%$ of the transverse confinement energy. We discuss a dimensional crossover toward 2D regime by reducing the number of atoms in the planar trap, including the effect of the spatial variation of the transverse trapping frequency in the large-area sample.


title: On the Small Jumps of L\'evy Processes and the Multivariate Dickman Distribution

abstract: We develop a Dickman approximation to the small jumps of multivariate L\'evy processes and related stochastic integral processes. Further, we show that the multivariate Dickman distribution is the unique distribution satisfying a certain stability property and that a related stochastic process is self-similar. Along the way we develop applications to non-Gaussian OU-Processes, to the class of generalized multivariate gamma distributions, and to the popular Barndorff-Nielsen and Shephard stochastic volatility model.


title: Collapse in Noncommutative Geometry and Spectral Continuity

abstract: If two compact quantum metric spaces are close in the metric sense, then how similar are they, as noncommutative spaces? In the classical realm of Riemannian geometry, informally, if two manifolds are close in the Gromov-Hausdorff distance, and belong to a class of manifolds with bounded curvature and diameter, then the spectra of their Laplacian or Dirac operators are also close under many scenari. Of particular interest is the case where a sequence of manifolds converge for the Gromov-Hausdorff distance to a manifold of lower dimension, and the question of the continuity, in some sense, of the spectra of geometrically relevant operators. In this paper, we initiate the study of the continuity of spectra and other properties of metric spectral triples under collapse in the noncommutative realm. As a first step in this study, we work with collapse for the spectral propinquity, an analogue of the Gromov-Hausdorff distance for spectral triples introduced by the second author, i.e. a form of metric for differential structures. Inspired by results from collapse in Riemannian geometry, we begin with the study of spectral triples which decompose, in some sense, in a vertical and a horizontal direction, and we collapse these spectral triples along the vertical direction. We obtain convergence results, and by the work of the second author, we conclude continuity results for the spectra of the Dirac operators of these spectral triples. Examples include collapse of product of spectral triples with one Abelian factor, $U(1)$ principal bundles over Riemannian spin manifolds, and noncommutative principal bundles, including C*-crossed-products and other noncommutative bundles.


title: Cosmological mass of the photon related to Stueckelberg and Higgs mechanisms

abstract: I consider electro-weak (EW) masses and interactions generated for photons by vacuum expectation values of Stueckelberg and Higgs fields. I provide a prescription to relate their parametric values to a cosmological range derived from a fundamental Heisenberg uncertainty principle and Einstein-de Sitter cosmological constant and horizon. This yields qualitative connections between microscopic ranges acquired by $W^{\pm}$ or $Z^0$ gauge Bosons and the cosmological scale and minimal mass acquired by $g$-photons. I apply that procedure to an established Stueckelberg-Higgs mechanism, while I consider a similar procedure for a pair of Higgs fields that may spontaneously break all U(1)xSU(2) gauge invariances. My estimates of photon masses and their additional parity-breaking interactions with leptons and neutrinos may be detectable in suitable accelerator experiments. Their effects may also be observable astronomically through massive $g$-photon condensates that may contribute to dark matter and dark energy.


title: DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference

abstract: Given the increasing demand for tree-structured interactions with LLMs, we introduce DeFT (Decoding with Flash Tree-Attention), an IO-aware tree attention algorithm tailored for tree-structured inference. Unlike traditional sequence-based decoding, tree-structured decoding better accommodates modern task requirements, including self-consistency, few-shot prompting, multi-step reasoning, and multi-model/head coordination. However, existing sequence-based inference systems are ill-suited for tree-structured decoding, resulting in redundancy in computation, memory footprints, and memory access, thereby undermining inference efficiency. To address this challenge, DeFT maintains memory-efficient attention calculation with low memory footprints through two key stages: (1) QKV Preparation: We propose a KV-Guided Grouping Strategy with Tree Split to intelligently group QKV, optimizing GPU resource utilization while minimizing memory reads/writes for KV cache between GPU global memory and on-chip shared memory; (2)Attention Calculation: We compute partial attention of each QKV group in a fused kernel and employ a Tree-topology-aware Global Reduction strategy to obtain final attention. By reducing 73-99% KV cache IO and nearly 100% IO for partial results during attention calculation (e.g., Softmax), DeFT achieves up to 2.52/3.82x speedup in the end-to-end/attention latency across three practical tree-based workloads: namely, few-shot prompting, multi-step reasoning, and speculative decoding, over state-of-the-art attention algorithms.