title: Changing human's impression of empathy from agent by verbalizing agent's position

abstract: As anthropomorphic agents (AI and robots) are increasingly used in society, empathy and trust between people and agents are becoming increasingly important. A better understanding of agents by people will help to improve the problems caused by the future use of agents in society. In the past, there has been a focus on the importance of self-disclosure and the relationship between agents and humans in their interactions. In this study, we focused on the attributes of self-disclosure and the relationship between agents and people. An experiment was conducted to investigate hypotheses on trust and empathy with agents through six attributes of self-disclosure (opinions and attitudes, hobbies, work, money, personality, and body) and through competitive and cooperative relationships before a robotic agent performs a joint task. The experiment consisted of two between-participant factors: six levels of self-disclosure attributes and two levels of relationship with the agent. The results showed that the two factors had no effect on trust in the agent, but there was statistical significance for the attribute of self-disclosure regarding a person's empathy toward the agent. In addition, statistical significance was found regarding the agent's ability to empathize with a person as perceived by the person only in the case where the type of relationship, competitive or cooperative, was presented. The results of this study could lead to an effective method for building relationships with agents, which are increasingly used in society.


title: No time to derive: unraveling total time derivatives in in-in perturbation theory

abstract: The in-in formalism provides a way to systematically organize the calculation of primordial correlation functions. Although its theoretical foundations are now firmly settled, the treatment of total time derivative interactions, incorrectly trivialized as ``boundary terms'', has been the subject of intense discussions and conceptual mistakes. In this work, we demystify the use of total time derivatives -- as well as terms proportional to the linear equations of motion -- and show that they can lead to artificially large contributions cancelling at different orders of the in-in operator formalism. We discuss the treatment of total time derivative interactions in the Lagrangian path integral formulation of the in-in perturbation theory, and we showcase the importance of interaction terms proportional to linear equations of motion. We then provide a new route to the calculation of primordial correlation functions, which avoids the generation of total time derivatives, by working directly at the level of the full Hamiltonian in terms of phase-space variables. Instead of integrating by parts, we perform canonical transformations to simplify interactions. We explain how to retrieve correlation functions of the initial phase-space variables from the knowledge of the ones after canonical transformations, and we provide diagrammatic rules to systematically classify all possible contributions. As an important first application, we find the explicit sizes of Hamiltonian cubic interactions in single-field inflation with canonical kinetic terms and for any background evolution [...]. Our results are important for performing complete calculations of exchange diagrams in inflation [...]. Being already written in a form amenable to characterize quantum properties of primordial fluctuations, they also promise to shed light on the non-linear dynamics of quantum states during inflation.


title: Visibility-Aware Keypoint Localization for 6DoF Object Pose Estimation

abstract: Localizing predefined 3D keypoints in a 2D image is an effective way to establish 3D-2D correspondences for 6DoF object pose estimation. However, unreliable localization results of invisible keypoints degrade the quality of correspondences. In this paper, we address this issue by localizing the important keypoints in terms of visibility. Since keypoint visibility information is currently missing in dataset collection process, we propose an efficient way to generate binary visibility labels from available object-level annotations, for keypoints of both asymmetric objects and symmetric objects. We further derive real-valued visibility-aware importance from binary labels based on PageRank algorithm. Taking advantage of the flexibility of our visibility-aware importance, we construct VAPO (Visibility-Aware POse estimator) by integrating the visibility-aware importance with a state-of-the-art pose estimation algorithm, along with additional positional encoding. Extensive experiments are conducted on popular pose estimation benchmarks including Linemod, Linemod-Occlusion, and YCB-V. The results show that, VAPO improves both the keypoint correspondences and final estimated poses, and clearly achieves state-of-the-art performances.


title: Moir\'e-modulated band gap and van Hove singularities in twisted bilayer germanene

abstract: Twisting bilayers of two-dimensional topological insulators has the potential to create unique quantum states of matter. Here, we successfully synthesized a twisted bilayer of germanene on Ge2Pt(101) with a 21.8$^o$ degrees twist angle, corresponding to a commensurate ($\sqrt{7} \times \sqrt{7}$) structure. Using scanning tunneling microscopy and spectroscopy, we unraveled the structural and electronic properties of this configuration, revealing a moir\'e-modulated band gap and a well-defined edge state. This band gap opens at AB/BA stacked sites and closes at AA stacked sites, a phenomenon attributed to the electric field induced by the scanning tunneling microscopy tip. Our study further revealed two van Hove singularities at -0.8 eV and +1.04 eV, resulting in a Fermi velocity of $(8 \pm 1) \times 10^5$ m/s. Our tight-binding results uncover a unique quantum state, where the topological properties could be regulated through an electric field, potentially triggering two topological phase transitions.


title: Looking Together $\neq$ Seeing the Same Thing: Understanding Surgeons' Visual Needs During Intra-operative Coordination and Instruction

abstract: Shared gaze visualizations have been found to enhance collaboration and communication outcomes in diverse HCI scenarios including computer supported collaborative work and learning contexts. Given the importance of gaze in surgery operations, especially when a surgeon trainer and trainee need to coordinate their actions, research on the use of gaze to facilitate intra-operative coordination and instruction has been limited and shows mixed implications. We performed a field observation of 8 surgeries and an interview study with 14 surgeons to understand their visual needs during operations, informing ways to leverage and augment gaze to enhance intra-operative coordination and instruction. We found that trainees have varying needs in receiving visual guidance which are often unfulfilled by the trainers' instructions. It is critical for surgeons to control the timing of the gaze-based visualizations and effectively interpret gaze data. We suggest overlay technologies, e.g., gaze-based summaries and depth sensing, to augment raw gaze in support of surgical coordination and instruction.


title: The Era of Semantic Decoding

abstract: Recent work demonstrated great promise in the idea of orchestrating collaborations between LLMs, human input, and various tools to address the inherent limitations of LLMs. We propose a novel perspective called semantic decoding, which frames these collaborative processes as optimization procedures in semantic space. Specifically, we conceptualize LLMs as semantic processors that manipulate meaningful pieces of information that we call semantic tokens (known thoughts). LLMs are among a large pool of other semantic processors, including humans and tools, such as search engines or code executors. Collectively, semantic processors engage in dynamic exchanges of semantic tokens to progressively construct high-utility outputs. We refer to these orchestrated interactions among semantic processors, optimizing and searching in semantic space, as semantic decoding algorithms. This concept draws a direct parallel to the well-studied problem of syntactic decoding, which involves crafting algorithms to best exploit auto-regressive language models for extracting high-utility sequences of syntactic tokens. By focusing on the semantic level and disregarding syntactic details, we gain a fresh perspective on the engineering of AI systems, enabling us to imagine systems with much greater complexity and capabilities. In this position paper, we formalize the transition from syntactic to semantic tokens as well as the analogy between syntactic and semantic decoding. Subsequently, we explore the possibilities of optimizing within the space of semantic tokens via semantic decoding algorithms. We conclude with a list of research opportunities and questions arising from this fresh perspective. The semantic decoding perspective offers a powerful abstraction for search and optimization directly in the space of meaningful concepts, with semantic tokens as the fundamental units of a new type of computation.


title: Evaluating the impact of instrumental variables in propensity score models using synthetic and negative control experiments

abstract: In pharmacoepidemiology research, instrumental variables (IVs) are variables that strongly predict treatment but have no causal effect on the outcome of interest except through the treatment. There remain concerns about the inclusion of IVs in propensity score (PS) models amplifying estimation bias and reducing precision. Some PS modeling approaches attempt to address the potential effects of IVs, including selecting only covariates for the PS model that are strongly associated to the outcome of interest, thus screening out IVs. We conduct a study utilizing simulations and negative control experiments to evaluate the effect of IVs on PS model performance and to uncover best PS practices for real-world studies. We find that simulated IVs have a weak effect on bias and precision in both simulations and negative control experiments based on real-world data. In simulation experiments, PS methods that utilize outcome data, including the high-dimensional propensity score, produce the least estimation bias. However, in real-world settings underlying causal structures are unknown, and negative control experiments can illustrate a PS model's ability to minimize systematic bias. We find that large-scale, regularized regression based PS models in this case provide the most centered negative control distributions, suggesting superior performance in real-world scenarios.


title: On K\"{o}the's normality question for locally finite-dimensional central division algebras

abstract: This paper considers K\"{o}the's question of whether every associative locally finite-dimensional (abbr., LFD) central division algebra $R$ over a field $K$ is a normally locally finite (abbr., NLF) algebra over $K$, that is, whether every nonempty finite subset $Y$ of $R$ is contained in a finite-dimensional central $K$-subalgebra $\mathcal{R} _{Y}$ of $R$. It shows that the answer to the posed question is negative if $K$ is a purely transcendental extension of infinite transcendence degree over an algebraically closed field $k$. On the other hand, central division LFD-algebras over $K$ turn out to be NLF in the following special cases: (i) $K$ is a finitely-generated extension of a finite or a pseudo-algebraically closed field; (ii) $K$ is a higher-dimensional local field with a finite last residue field.


title: A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Science

abstract: This paper explores the use of large language models (LLMs) to score and explain short-answer assessments in K-12 science. While existing methods can score more structured math and computer science assessments, they often do not provide explanations for the scores. Our study focuses on employing GPT-4 for automated assessment in middle school Earth Science, combining few-shot and active learning with chain-of-thought reasoning. Using a human-in-the-loop approach, we successfully score and provide meaningful explanations for formative assessment responses. A systematic analysis of our method's pros and cons sheds light on the potential for human-in-the-loop techniques to enhance automated grading for open-ended science assessments.


title: A survey on Concept-based Approaches For Model Improvement

abstract: The focus of recent research has shifted from merely improving the metrics based performance of Deep Neural Networks (DNNs) to DNNs which are more interpretable to humans. The field of eXplainable Artificial Intelligence (XAI) has observed various techniques, including saliency-based and concept-based approaches. These approaches explain the model's decisions in simple human understandable terms called Concepts. Concepts are known to be the thinking ground of humans}. Explanations in terms of concepts enable detecting spurious correlations, inherent biases, or clever-hans. With the advent of concept-based explanations, a range of concept representation methods and automatic concept discovery algorithms have been introduced. Some recent works also use concepts for model improvement in terms of interpretability and generalization. We provide a systematic review and taxonomy of various concept representations and their discovery algorithms in DNNs, specifically in vision. We also provide details on concept-based model improvement literature marking the first comprehensive survey of these methods.


title: Global Solutions to the 3D Half-Wave Maps Equation with Angular Regularity

abstract: The half-wave maps equation is a nonlocal geometric equation arising in the continuum dynamics of Haldane-Shashtry and Calogero-Moser spin systems. In high dimensions $n\geq4$, global wellposedness for data which is small in the critical Besov space $\dot{B}^{n/2}_{2,1}$ is known since the works of Krieger, Sire and Kiesenhofer [13,9]. There is a major obstruction in extending these results to three dimensions due to the loss of the crucial $L^2_tL^\infty_x$ Strichartz estimate. In this work, we make progress on this case by proving that the equation is "weakly" globally well-posed (in the sense of Tao [28]) for initial data which is not only small in $\dot{B}^{3/2}_{2,1}$ but also possesses some angular regularity and weighted decay of derivatives. We use Sterbenz's improved Strichartz estimates in conjunction with certain commuting vector fields to develop trilinear estimates in weighted Strichartz spaces which avoid the use of the $L^2_tL^\infty_x$ endpoint.


title: Pressure-induced enhancement of superconductivity in a non-centrosymmetric compound LaPtGe

abstract: We report a pressure-induced enhancement of the superconducting transition temperature (Tc) in a non-centrosymmetric (NCS) compound, LaPtGe. With pressure, till 3 GPa, we observed a modest enhancement of the Tc with a rate of 0.071 K/GPa. However, above this pressure, the rate showed a ~2.5 times increase, 0.183 K/GPa. We observed a Tc of 3.94 K at 6 GPa, the highest pressure value used in our transport study. Synchrotron high-pressure x-ray powder diffraction (HP-XRPD) measurements do not reveal any structural phase transition in this system in this pressure range. However, it showed an apparent change of slope in the pressure dependence of lattice parameters above and below 3 GPa. Pressure dependence of the unit-cell volume also followed a distinct trend below and above 3 GPa, with the Birch-Murnaghan equation of state fit providing a bulk modulus (B0) value of ~144 and ~162 GPa, respectively, for two pressure regions. Further, the magnetotransport measurement under pressure up to 2.45 GPa reveals the enhancement of the upper critical field (Hc2(0)) from 0.7 T (0 GPa) to 0.92 T (2.45 GPa). In addition, the upward curvature in Hc2(T) becomes stronger with increasing pressure, suggesting a change of the underlying Fermi surface topology with pressure. The signature for the inducible Hc2(T) with pressure, the distinct enhancement of Tc around 3 GPa, and the noticeable change in lattice parameters around 3 GPa suggests the possibility of multi-gap superconductivity in LaPtGe similar to identical structure NCS compound, LaPtSi. The enhancement of Tc by pressure can be correlated with the possible underlying lattice modulation by compression and the change in the density of states at the Fermi level. Also, the bare change in pressure-dependent activation energy U0 up to 2.45 GPa calculated using the Arrhenius relation clearly shows that the shift in Tc does not arise from grain boundaries.


title: On the group cohomology of groups of the form $\mathbb{Z}^n\rtimes \mathbb{Z}/m$ with $m$ free of squares

abstract: We provide an explicit computation of the cohomology groups (with untwisted coefficients) of semidirect products of the form $\mathbb{Z}^n\rtimes \mathbb{Z}/m$ with $m$ free of squares, by means of formulas that only depend on $n$, $m$ and the action of $\mathbb{Z}/m$ on $\mathbb{Z}^n$. We want to highlight the fact that we are not impossing any conditions on the $\mathbb{Z}/m$-action on $\mathbb{Z}^n$, and as far as we know our formulas are the first in the literature in this generality. This generalizes previous computations of L\"uck-Davis and Adem-Ge-Pan-Petrosyan. In order to show that our formulas are usable, we develop a concrete example of the form $\mathbb{Z}^5\rtimes \mathbb{Z}/6$ where its cohomology groups are described in full detail.


title: Robust estimations from distribution structures: II. Central Moments

abstract: In descriptive statistics, $U$-statistics arise naturally in producing minimum-variance unbiased estimators. In 1984, Serfling considered the distribution formed by evaluating the kernel of the $U$-statistics and proposed generalized $L$-statistics which includes Hodges-Lehamnn estimator and Bickel-Lehmann spread as special cases. However, the structures of the kernel distributions remain unclear. In 1954, Hodges and Lehmann demonstrated that if $X$ and $Y$ are independently sampled from the same unimodal distribution, $X-Y$ will exhibit symmetrical unimodality with its peak centered at zero. Building upon this foundational work, the current study delves into the structure of the kernel distribution. It is shown that the $\mathbf{k}$th central moment kernel distributions ($\mathbf{k}>2$) derived from a unimodal distribution exhibit location invariance and is also nearly unimodal with the mode and median close to zero. This article provides an approach to study the general structure of kernel distributions.


title: Trivial source character tables of Frobenius groups of type $(C_p \times C_p) \rtimes H$

abstract: Let $p$ be a prime number. We compute the trivial source character tables of finite Frobenius groups $G$ with an abelian Frobenius complement $H$ and an elementary abelian Frobenius kernel of order $p^2$. More precisely, we deal with all infinite families of such groups which occur in the two extremal cases for the fusion of $p$-subgroups: the case in which there exists exactly one $G$-conjugacy class of non-trivial cyclic $p$-subgroups, and the case in which there exist exactly $p+1$ distinct $G$-conjugacy classes of non-trivial cyclic $p$-subgroups.


title: Implicit Style-Content Separation using B-LoRA

abstract: Image stylization involves manipulating the visual appearance and texture (style) of an image while preserving its underlying objects, structures, and concepts (content). The separation of style and content is essential for manipulating the image's style independently from its content, ensuring a harmonious and visually pleasing result. Achieving this separation requires a deep understanding of both the visual and semantic characteristics of images, often necessitating the training of specialized models or employing heavy optimization. In this paper, we introduce B-LoRA, a method that leverages LoRA (Low-Rank Adaptation) to implicitly separate the style and content components of a single image, facilitating various image stylization tasks. By analyzing the architecture of SDXL combined with LoRA, we find that jointly learning the LoRA weights of two specific blocks (referred to as B-LoRAs) achieves style-content separation that cannot be achieved by training each B-LoRA independently. Consolidating the training into only two blocks and separating style and content allows for significantly improving style manipulation and overcoming overfitting issues often associated with model fine-tuning. Once trained, the two B-LoRAs can be used as independent components to allow various image stylization tasks, including image style transfer, text-based image stylization, consistent style generation, and style-content mixing.


title: A Transfer Learning Causal Approach to Evaluate Racial/Ethnic and Geographic Variation in Outcomes Following Congenital Heart Surgery

abstract: Congenital heart defects (CHD) are the most prevalent birth defects in the United States and surgical outcomes vary considerably across the country. The outcomes of treatment for CHD differ for specific patient subgroups, with non-Hispanic Black and Hispanic populations experiencing higher rates of mortality and morbidity. A valid comparison of outcomes within racial/ethnic subgroups is difficult given large differences in case-mix and small subgroup sizes. We propose a causal inference framework for outcome assessment and leverage advances in transfer learning to incorporate data from both target and source populations to help estimate causal effects while accounting for different sources of risk factor and outcome differences across populations. Using the Society of Thoracic Surgeons' Congenital Heart Surgery Database (STS-CHSD), we focus on a national cohort of patients undergoing the Norwood operation from 2016-2022 to assess operative mortality and morbidity outcomes across U.S. geographic regions by race/ethnicity. We find racial and ethnic outcome differences after controlling for potential confounding factors. While geography does not have a causal effect on outcomes for non-Hispanic Caucasian patients, non-Hispanic Black patients experience wide variability in outcomes with estimated 30-day mortality ranging from 5.9% (standard error 2.2%) to 21.6% (4.4%) across U.S. regions.


title: A subtraction scheme for processes involving fragmentation functions at NLO

abstract: We present a novel subtraction method to remove the soft and collinear divergences at next-to-leading order for processes involving an arbitrary number of fragmentation functions, where this method acts directly in the hadronic centre-of-mass frame. We provide the analytical formulae of the subtraction terms in the general case where all the final state partons fragment to hadrons and for the two special cases when one of the partons of the final state does not fragment, i.e. it is a photon or involved in a jet.


title: Generating Photon Pairs in a Hybrid Si-BTO Platform

abstract: Here we show photon pair generation from ring resonator and waveguide structures in a hybrid silicon-BTO on insulator platform with a pulsed pump. Our analysis of single photon and coincidence generation rates show that Spontaneous Four-Wave Mixing is comparable to that expected from SOI devices of similar characteristics and find a $\gamma_{eff}$ of (14.7 $\pm$ 1.3) and (2.0 $\pm$ 0.3) MHz/mW$^{2}$ for ring resonator and waveguide structures respectively.


title: Fully Evaluated Left-Sequential Logics

abstract: We consider a family of two-valued "fully evaluated left-sequential logics" (FELs), of which Free FEL (defined by Staudt in 2012) is most distinguishing (weakest) and immune to atomic side effects. Next is Memorising FEL, in which evaluations of subexpressions are memorised. The following stronger logic is Conditional FEL (inspired by Guzm\'an and Squier's Conditional logic, 1990). The strongest FEL is static FEL, a sequential version of propositional logic. We use evaluation trees as a simple, intuitive semantics and provide complete axiomatisations for closed terms (left-sequential propositional expressions). For each FEL except Static FEL, we also define its three-valued version, with a constant U for "undefinedness" and again provide complete, independent aziomatisations, each one containing two additional axioms for U on top of the axiomatisations of the two-valued case. In this setting, the strongest FEL is equivalent to Bochvar's strict logic.


title: Application of the VMM ASIC for SiPM-based calorimetry

abstract: Highly integrated multichannel readout electronics is crucial in contemporary particle physics experiments. A novel silicon photomultiplier readout system based on the VMM3a ASIC was developed, for the first time exploiting this chip for calorimetric purposes. To extend the dynamic range the signal from each SiPM channel was processed by two electronics channels with different gain. A fully operational prototype system with 256 SiPM readout channels allowed the collection of data from a prototype of the ALICE Forward Hadron Calorimeter (FoCal-H). The design and the test beam results using high energy hadron beams are presented and discussed, confirming the applicability of VMM3a-based solutions for energy measurements in a high rate environment.


title: RAmBLA: A Framework for Evaluating the Reliability of LLMs as Assistants in the Biomedical Domain

abstract: Large Language Models (LLMs) increasingly support applications in a wide range of domains, some with potential high societal impact such as biomedicine, yet their reliability in realistic use cases is under-researched. In this work we introduce the Reliability AssesMent for Biomedical LLM Assistants (RAmBLA) framework and evaluate whether four state-of-the-art foundation LLMs can serve as reliable assistants in the biomedical domain. We identify prompt robustness, high recall, and a lack of hallucinations as necessary criteria for this use case. We design shortform tasks and tasks requiring LLM freeform responses mimicking real-world user interactions. We evaluate LLM performance using semantic similarity with a ground truth response, through an evaluator LLM.


title: On the minimization of the Willmore energy under a constraint on total mean curvature and area

abstract: Motivated by a model for lipid bilayer cell membranes, we study the minimization of the Willmore functional in the class of oriented closed surfaces with prescribed total mean curvature, prescribed area, and prescribed genus. Adapting methods previously developed by Keller-Mondino-Rivi\`ere, Bauer-Kuwert, and Ndiaye-Sch\"atzle, we prove existence of smooth minimizers for a large class of constraints. Moreover, we analyze the asymptotic behaviour of the energy profile close to the unit sphere and consider the total mean curvature of axisymmetric surfaces.


title: Tomographic redshift dipole: Testing the cosmological principle

abstract: The cosmological principle posits that the universe is statistically homogeneous and isotropic on large scales, implying all matter shares the same rest frame. This principle suggests that velocity estimates of our motion from various sources should agree with the cosmic microwave background (CMB) dipole's inferred velocity of 370 km/s. Yet, for over two decades, analyses of radio galaxy and quasar catalogs have found velocities at odds with the CMB dipole, with tensions up to 5$\sigma$. In a blind analysis of BOSS and eBOSS spectroscopic data from galaxies and quasars across $0.2<z<2.2$, we applied a novel dipole estimator for a tomographic approach, robustly correcting biases and quantifying uncertainties with realistic mock catalogs. Our findings with eBOSS data ($0.6<z<2.2$), indicating a velocity of $195^{+99}_{-81}$ km/s, demonstrate a $2\sigma$ agreement with the CMB dipole when considering the full 3D vector distribution and a 3-to-6$\sigma$ tension with previous number count studies. This result supports the cosmological principle, emphasizing the consistency of our motion with the CMB across vast cosmic distances. On the other hand, the BOSS data revealed potential unmodeled systematics; the estimator could not be minimized using the LOWZ set ($0.2<z<0.4$), and the CMASS set ($0.4<z<0.6$) presented results that pointed towards the southern hemisphere, conflicting with the CMB dipole. Addressing the disparities with earlier number count analyses and understanding possible systematics in spectroscopic measurements will be essential to further validate the cosmological principle.


title: Global, robust and comparable digital carbon assets

abstract: Carbon credits purchased in the voluntary carbon market allow unavoidable emissions, such as from international flights for essential travel, to be offset by an equivalent climate benefit, such as avoiding emissions from tropical deforestation. However, many concerns regarding the credibility of these offsetting claims have been raised. Moreover, the credit market is manual, therefore inefficient and unscalable, and non-fungible, therefore illiquid. To address these issues, we propose an efficient digital methodology that combines remote sensing data, modern econometric techniques, and on-chain certification and trading to create a new digital carbon asset (the PACT stablecoin) against which carbon offsetting claims can be transparently verified. PACT stablecoins are produced as outputs from a reproducible computational pipeline for estimating the climate benefits of carbon offset projects that not only quantifies the CO2 emissions involved, but also allows for similar credits to be pooled based on their co-benefits such as biodiversity and jurisdictional attributes, increasing liquidity through fungibility within pools. We implement and evaluate the PACT carbon stablecoin on the Tezos blockchain, which is designed to facilitate low-cost transactions while minimizing environmental impact. Our implementation includes a contract for a registry for tracking issuance, ownership, and retirement of credits, and a custodian contract to bridge on-chain and off-chain transactions. Our work brings scale and trust to the voluntary carbon market by providing a transparent, scalable, and efficient framework for high integrity carbon credit transactions.


title: Large Language Models for Multi-Choice Question Classification of Medical Subjects

abstract: The aim of this paper is to evaluate whether large language models trained on multi-choice question data can be used to discriminate between medical subjects. This is an important and challenging task for automatic question answering. To achieve this goal, we train deep neural networks for multi-class classification of questions into the inferred medical subjects. Using our Multi-Question (MQ) Sequence-BERT method, we outperform the state-of-the-art results on the MedMCQA dataset with an accuracy of 0.68 and 0.60 on their development and test sets, respectively. In this sense, we show the capability of AI and LLMs in particular for multi-classification tasks in the Healthcare domain.


title: Co-Optimization of Environment and Policies for Decentralized Multi-Agent Navigation

abstract: This work views the multi-agent system and its surrounding environment as a co-evolving system, where the behavior of one affects the other. The goal is to take both agent actions and environment configurations as decision variables, and optimize these two components in a coordinated manner to improve some measure of interest. Towards this end, we consider the problem of decentralized multi-agent navigation in cluttered environments. By introducing two sub-objectives of multi-agent navigation and environment optimization, we propose an $\textit{agent-environment co-optimization}$ problem and develop a $\textit{coordinated algorithm}$ that alternates between these sub-objectives to search for an optimal synthesis of agent actions and obstacle configurations in the environment; ultimately, improving the navigation performance. Due to the challenge of explicitly modeling the relation between agents, environment and performance, we leverage policy gradient to formulate a model-free learning mechanism within the coordinated framework. A formal convergence analysis shows that our coordinated algorithm tracks the local minimum trajectory of an associated time-varying non-convex optimization problem. Extensive numerical results corroborate theoretical findings and show the benefits of co-optimization over baselines. Interestingly, the results also indicate that optimized environment configurations are able to offer structural guidance that is key to de-conflicting agents in motion.


title: Dynamical importance and network perturbations

abstract: The leading eigenvalue $\lambda$ of the adjacency matrix of a graph exerts much influence on the behavior of dynamical processes on that graph. It is thus relevant to relate notions of the importance (specifically, centrality measures) of network structures to $\lambda$ and its associated eigenvector. We study a previously derived measure of edge importance known as "dynamical importance", which estimates how much $\lambda$ changes when one removes an edge from a graph or adds an edge to it. We examine the accuracy of this estimate for different network structures and compare it to the true change in $\lambda$ after an edge removal or edge addition. We then derive a first-order approximation of the change in the leading eigenvector. We also consider the effects of edge additions on Kuramoto dynamics on networks, and we express the Kuramoto order parameter in terms of dynamical importance. Through our analysis and computational experiments, we find that studying dynamical importance can improve understanding of the relationship between network perturbations and dynamical processes on networks.


title: Slow decay of correlations for generic mixing automorphisms

abstract: Let $\psi(n)\to +0$ and a square-integrable function $f$ be non-zero, then for the typical mixing automorphism $T$ the set $\{n:\, |(T^nf,f) |>\psi( n)\}$ is infinite. The mildly mixing automorphisms $T$ do not have convergences of non-zero averages $\frac 1 {k_n} \sum_{i=1}^{k_n}T^if (x)$ with the rate of $o\left(\frac 1 {k_n}\right)$.


title: On four-manifolds without 1- and 3-handles

abstract: We note that infinitely many irreducible, closed, simply connected 4-manifolds, with prescribed signature and spin type, admit perfect Morse functions, i.e. they can be given handle decompositions without 1- and 3-handles. In particular, there are many such 4-manifolds homeomorphic but not diffeomorphic to the standard 4-manifolds # m (S^2 x S^2) and # n (CP^2 # -CP^2), respectively, which answers Problem 4.91 on Kirby's 1997 list.


title: An Analysis of Linear Time Series Forecasting Models

abstract: Despite their simplicity, linear models perform well at time series forecasting, even when pitted against deeper and more expensive models. A number of variations to the linear model have been proposed, often including some form of feature normalisation that improves model generalisation. In this paper we analyse the sets of functions expressible using these linear model architectures. In so doing we show that several popular variants of linear models for time series forecasting are equivalent and functionally indistinguishable from standard, unconstrained linear regression. We characterise the model classes for each linear variant. We demonstrate that each model can be reinterpreted as unconstrained linear regression over a suitably augmented feature set, and therefore admit closed-form solutions when using a mean-squared loss function. We provide experimental evidence that the models under inspection learn nearly identical solutions, and finally demonstrate that the simpler closed form solutions are superior forecasters across 72% of test settings.


title: Exponential Networks for Linear Partitions

abstract: Previous work has given proof and evidence that BPS states in local Calabi-Yau 3-folds can be described and counted by exponential networks on the punctured plane, with the help of a suitable non-abelianization map to the mirror curve. This provides an appealing elementary depiction of moduli of special Lagrangian submanifolds, but so far only a handful of examples have been successfully worked out in detail. In this note, we exhibit an explicit correspondence between torus fixed points of the Hilbert scheme of points on $\mathbb C^2\subset\mathbb C^3$ and anomaly free exponential networks attached to the quadratically framed pair of pants. This description realizes an interesting, and seemingly novel, "age decomposition" of linear partitions. We also provide further details about the networks' perspective on the full D-brane moduli space.


title: ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy

abstract: Language agents have demonstrated autonomous decision-making abilities by reasoning with foundation models. Recently, efforts have been made to train language agents for performance improvement, with multi-step reasoning and action trajectories as the training data. However, collecting such trajectories still requires considerable human effort, by either artificial annotation or implementations of diverse prompting frameworks. In this work, we propose A$^3$T, a framework that enables the Autonomous Annotation of Agent Trajectories in the style of ReAct. The central role is an ActRe prompting agent, which explains the reason for an arbitrary action. When randomly sampling an external action, the ReAct-style agent could query the ActRe agent with the action to obtain its textual rationales. Novel trajectories are then synthesized by prepending the posterior reasoning from ActRe to the sampled action. In this way, the ReAct-style agent executes multiple trajectories for the failed tasks, and selects the successful ones to supplement its failed trajectory for contrastive self-training. Realized by policy gradient methods with binarized rewards, the contrastive self-training with accumulated trajectories facilitates a closed loop for multiple rounds of language agent self-improvement. We conduct experiments using QLoRA fine-tuning with the open-sourced Mistral-7B-Instruct-v0.2. In AlfWorld, the agent trained with A$^3$T obtains a 1-shot success rate of 96%, and 100% success with 4 iterative rounds. In WebShop, the 1-shot performance of the A$^3$T agent matches human average, and 4 rounds of iterative refinement lead to the performance approaching human experts. A$^3$T agents significantly outperform existing techniques, including prompting with GPT-4, advanced agent frameworks, and fully fine-tuned LLMs.


title: When faster rotation is harmful: the competition of alliances with inner blocking mechanism

abstract: Competitors in an intransitive loop of dominance can form a defensive alliance against an external species. The vitality of this super-structure, however, is jeopardized if we modify the original rock-scissors-paper-like rule and allow that the vicinity of a predator blocks stochastically the invasion success of its neighboring prey towards a third actor. To explore the potential consequences of this multi-point interaction we introduce a minimal model where two three-member alliances are fighting but one of them suffers from this inner blocking mechanism. We demonstrate that this weakness can be compensated by a faster inner rotation which is in agreement with previous findings. This broadly valid principle, however, is not always true here because the increase of rotation speed could be harmful and results in series of reentrant phase transitions on the parameter plane. This unexpected behavior can be explained by the relation of the blocked triplet and a neutral pair formed by a triplet member with an external species. Our results provide novel aspects to the fundamental laws which determine the evolutionary process in multi-strategy ecological systems.

categories: q-bio.PE cond-mat.stat-mech nlin.PS physics.soc-ph


title: New variants of arithmetic quantum ergodicity

abstract: We establish two new variants of arithmetic quantum ergodicity. The first is for self-dual $\mathrm{GL}_2$ Hecke-Maass newforms over $\mathbb{Q}$ as the level and Laplace eigenvalue vary jointly. The second is a nonsplit analogue wherein almost all restrictions of Hilbert (respectively Bianchi) Hecke-Maass cusp forms to the modular surface dissipate as their Laplace eigenvalues grow.


title: Envisioning the Next-Generation AI Coding Assistants: Insights & Proposals

abstract: As a research-product hybrid group in AI for Software Engineering (AI4SE), we present four key takeaways from our experience developing in-IDE AI coding assistants. AI coding assistants should set clear expectations for usage, integrate with advanced IDE capabilities and existing extensions, use extendable backend designs, and collect app data responsibly for downstream analyses. We propose open questions and challenges that academia and industry should address to realize the vision of next-generation AI coding assistants.


title: Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery and Algebraic Equilibrium Proof

abstract: Adversarial inverse reinforcement learning (AIRL) stands as a cornerstone approach in imitation learning, yet it faces criticisms from prior studies. In this paper, we rethink AIRL and respond to these criticisms. Criticism 1 lies in Inadequate Policy Imitation. We show that substituting the built-in algorithm with soft actor-critic (SAC) during policy updating (requires multi-iterations) significantly enhances the efficiency of policy imitation. Criticism 2 lies in Limited Performance in Transferable Reward Recovery Despite SAC Integration. While we find that SAC indeed exhibits a significant improvement in policy imitation, it introduces drawbacks to transferable reward recovery. We prove that the SAC algorithm itself is not feasible to disentangle the reward function comprehensively during the AIRL training process, and propose a hybrid framework, PPO-AIRL + SAC, for a satisfactory transfer effect. Criticism 3 lies in Unsatisfactory Proof from the Perspective of Potential Equilibrium. We reanalyze it from an algebraic theory perspective.


title: VXP: Voxel-Cross-Pixel Large-scale Image-LiDAR Place Recognition

abstract: Recent works on the global place recognition treat the task as a retrieval problem, where an off-the-shelf global descriptor is commonly designed in image-based and LiDAR-based modalities. However, it is non-trivial to perform accurate image-LiDAR global place recognition since extracting consistent and robust global descriptors from different domains (2D images and 3D point clouds) is challenging. To address this issue, we propose a novel Voxel-Cross-Pixel (VXP) approach, which establishes voxel and pixel correspondences in a self-supervised manner and brings them into a shared feature space. Specifically, VXP is trained in a two-stage manner that first explicitly exploits local feature correspondences and enforces similarity of global descriptors. Extensive experiments on the three benchmarks (Oxford RobotCar, ViViD++ and KITTI) demonstrate our method surpasses the state-of-the-art cross-modal retrieval by a large margin.


title: Mutation of signed valued quivers and presentations of simple complex Lie algebras

abstract: We introduce a signed variant of (valued) quivers and a mutation rule that generalizes the classical Fomin-Zelevinsky mutation of quivers. To any signed valued quiver we associate a matrix that is a signed analogue of the Cartan counterpart appearing in the theory of cluster algebras. From this matrix, we construct a Lie algebra via a "Serre-like" presentation. In the mutation Dynkin case, we define root systems using the signed Cartan counterpart and show compatibility with mutation of roots as defined by Parsons. Using results from Barot-Rivera and P\'erez-Rivera, we show that mutation equivalent signed quivers yield isomorphic Lie algebras, giving presentations of simple complex Lie algebras.


title: Shadows and Properties of Spin-Induced Scalarized Black Holes with and without a Ricci Coupling

abstract: In this work, we explore the properties and shadows of spin-induced scalarized black holes, as well as investigate how a Ricci coupling influences them. Our findings reveal significant deviations from the Kerr metric in terms of the location and geodesic frequencies of the innermost stable circular orbit and light ring, with the former exhibiting more pronounced disparities. The shadows of scalarized black holes exhibit relatively minor deviations when compared to those of Kerr black holes with the same mass and spin. Overall, the presence of a Ricci coupling is observed to mitigate deviations from the Kerr metric.


title: Extended Reality for Enhanced Human-Robot Collaboration: a Human-in-the-Loop Approach

abstract: The rise of automation has provided an opportunity to achieve higher efficiency in manufacturing processes, yet it often compromises the flexibility required to promptly respond to evolving market needs and meet the demand for customization. Human-robot collaboration attempts to tackle these challenges by combining the strength and precision of machines with human ingenuity and perceptual understanding. In this paper, we conceptualize and propose an implementation framework for an autonomous, machine learning-based manipulator that incorporates human-in-the-loop principles and leverages Extended Reality (XR) to facilitate intuitive communication and programming between humans and robots. Furthermore, the conceptual framework foresees human involvement directly in the robot learning process, resulting in higher adaptability and task generalization. The paper highlights key technologies enabling the proposed framework, emphasizing the importance of developing the digital ecosystem as a whole. Additionally, we review the existent implementation approaches of XR in human-robot collaboration, showcasing diverse perspectives and methodologies. The challenges and future outlooks are discussed, delving into the major obstacles and potential research avenues of XR for more natural human-robot interaction and integration in the industrial landscape.


title: PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model

abstract: PSALM is a powerful extension of the Large Multi-modal Model (LMM) to address the segmentation task challenges. To overcome the limitation of the LMM being limited to textual output, PSALM incorporates a mask decoder and a well-designed input schema to handle a variety of segmentation tasks. This schema includes images, task instructions, conditional prompts, and mask tokens, which enable the model to generate and classify segmentation masks effectively. The flexible design of PSALM supports joint training across multiple datasets and tasks, leading to improved performance and task generalization. PSALM achieves superior results on several benchmarks, such as RefCOCO/RefCOCO+/RefCOCOg, COCO Panoptic Segmentation, and COCO-Interactive, and further exhibits zero-shot capabilities on unseen tasks, such as open-vocabulary segmentation, generalized referring expression segmentation and video object segmentation, making a significant step towards a GPT moment in computer vision. Through extensive experiments, PSALM demonstrates its potential to transform the domain of image segmentation, leveraging the robust visual understanding capabilities of LMMs as seen in natural language processing. Code and models are available at https://github.com/zamling/PSALM.


title: MyVLM: Personalizing VLMs for User-Specific Queries

abstract: Recent large-scale vision-language models (VLMs) have demonstrated remarkable capabilities in understanding and generating textual descriptions for visual content. However, these models lack an understanding of user-specific concepts. In this work, we take a first step toward the personalization of VLMs, enabling them to learn and reason over user-provided concepts. For example, we explore whether these models can learn to recognize you in an image and communicate what you are doing, tailoring the model to reflect your personal experiences and relationships. To effectively recognize a variety of user-specific concepts, we augment the VLM with external concept heads that function as toggles for the model, enabling the VLM to identify the presence of specific target concepts in a given image. Having recognized the concept, we learn a new concept embedding in the intermediate feature space of the VLM. This embedding is tasked with guiding the language model to naturally integrate the target concept in its generated response. We apply our technique to BLIP-2 and LLaVA for personalized image captioning and further show its applicability for personalized visual question-answering. Our experiments demonstrate our ability to generalize to unseen images of learned concepts while preserving the model behavior on unrelated inputs.


title: Wall-Crossing Effects on Quiver BPS Algebras

abstract: BPS states in supersymmetric theories can admit additional algebro-geometric structures in their spectra, described as quiver Yangian algebras. Equivariant fixed points on the quiver variety are interpreted as vectors populating a representation module, and matrix elements for the generators are then defined as Duistermaat-Heckman integrals in the vicinity of these points. The well-known wall-crossing phenomena are that the fixed point spectrum establishes a dependence on the stability (Fayet-Illiopolous) parameters $\zeta$, jumping abruptly across the walls of marginal stability, which divide the $\zeta$-space into a collection of stability chambers -- ``phases'' of the theory. The standard construction of the quiver Yangian algebra relies heavily on the molten crystal model, valid in a sole cyclic chamber where all the $\zeta$-parameters have the same sign. We propose to lift this restriction and investigate the effects of the wall-crossing phenomena on the quiver Yangian algebra and its representations -- starting with the example of affine super-Yangian $\mathsf{Y}(\widehat{\mathfrak{gl}}_{1|1})$. In addition to the molten crystal construction more general atomic structures appear, in other non-cyclic phases (chambers of the $\zeta$-space). We call them glasses and also divide in a few different classes. For some of the new phases we manage to associate an algebraic structure again as a representation of the same affine Yangian $\mathsf{Y}(\widehat{\mathfrak{gl}}_{1|1})$. This observation supports an earlier conjecture that the BPS algebraic structures can be considered as new wall-crossing invariants.

categories: hep-th math-ph math.AG math.MP math.QA math.RT


title: Miscibility-Immiscibility transition of strongly interacting bosonic mixtures in optical lattices

abstract: Interaction plays key role in the mixing properties of a multi-component system. The miscibility-immiscibility transition (MIT) in a weakly interacting mixture of Bose gases is predominantly determined by the strengths of the intra and inter-component two-body contact interactions. On the other hand, in the strongly interacting regime interaction induced processes become relevant. Despite previous studies on bosonic mixtures in optical lattices, the effects of the interaction induced processes on the MIT remains unexplored. In this work, we investigate the MIT in the strongly interacting phases of two-component bosonic mixture trapped in a homogeneous two-dimensional square optical lattice. Particularly we examine the transition when both the components are in superfluid (SF), one-body staggered superfluid (OSSF) or supersolid (SS) phases. Our study prevails that, similar to the contact interactions, the MIT can be influenced by competing intra and inter-component density induced tunnelings and off-site interactions. To probe the MIT in the strongly interacting regime, we study the extended version of the Bose-Hubbard model with the density induced tunneling and nearest-neighbouring interaction terms, and focus in the regime where the hopping processes are considerably weaker than the on-site interaction. We solve this model through site-decoupling mean-field theory with Gutzwiller ansatz and characterize the miscibility through the site-wise co-existence of the two-component across the lattice. Our study contributes to the better understanding of miscibility properties of multi-component systems in the strongly interacting regime.


title: ReNoise: Real Image Inversion Through Iterative Noising

abstract: Recent advancements in text-guided diffusion models have unlocked powerful image manipulation capabilities. However, applying these methods to real images necessitates the inversion of the images into the domain of the pretrained diffusion model. Achieving faithful inversion remains a challenge, particularly for more recent models trained to generate images with a small number of denoising steps. In this work, we introduce an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations. Building on reversing the diffusion sampling process, our method employs an iterative renoising mechanism at each inversion sampling step. This mechanism refines the approximation of a predicted point along the forward diffusion trajectory, by iteratively applying the pretrained diffusion model, and averaging these predictions. We evaluate the performance of our ReNoise technique using various sampling algorithms and models, including recent accelerated diffusion models. Through comprehensive evaluations and comparisons, we show its effectiveness in terms of both accuracy and speed. Furthermore, we confirm that our method preserves editability by demonstrating text-driven image editing on real images.


title: Alfv\'en Pulse Driven Spicule-like Jets in the Presence of Thermal Conduction and Ion-Neutral Collision in Two-Fluid Regime

abstract: We present the formation of quasi-periodic cool spicule-like jets in the solar atmosphere using 2.5-D numerical simulation in two-fluid regime (ions+neutrals) under the presence of thermal conduction and ion-neutral collision. The non-linear, impulsive Alfv\'enic perturbations at the top of the photosphere trigger field aligned magnetoacoustic perturbations due to ponderomotive force. The transport of energy from Alfv\'en pulse to such vertical velocity perturbations due to ponderomotive force is considered as an initial trigger mechanism. Thereafter, these velocity perturbations steepen into the shocks followed by quasi-periodic rise and fall of the cool jets transporting mass in the overlying corona.


title: An explicit parity theorem for multiple zeta values via multitangent functions

abstract: We give an explicit formula for the well-known parity result for multiple zeta values as an application of the multitangent functions.


title: SDP Synthesis of Maximum Coverage Trees for Probabilistic Planning under Control Constraints

abstract: The paper presents Maximal Covariance Backward Reachable Trees (MAXCOVAR BRT), which is a multi-query algorithm for planning of dynamic systems under stochastic motion uncertainty and constraints on the control input with explicit coverage guarantees. In contrast to existing roadmap-based probabilistic planning methods that sample belief nodes randomly and draw edges between them \cite{csbrm_tro2024}, under control constraints, the reachability of belief nodes needs to be explicitly established and is determined by checking the feasibility of a non-convex program. Moreover, there is no explicit consideration of coverage of the roadmap while adding nodes and edges during the construction procedure for the existing methods. Our contribution is a novel optimization formulation to add nodes and construct the corresponding edge controllers such that the generated roadmap results in provably maximal coverage under control constraints as compared to any other method of adding nodes and edges. We characterize formally the notion of coverage of a roadmap in this stochastic domain via introduction of the h-$\operatorname{BRS}$ (Backward Reachable Set of Distributions) of a tree of distributions under control constraints, and also support our method with extensive simulations on a 6 DoF model.


title: The Elements of Differentiable Programming

abstract: Artificial intelligence has recently experienced remarkable advances, fueled by large models, vast datasets, accelerated hardware, and, last but not least, the transformative power of differentiable programming. This new programming paradigm enables end-to-end differentiation of complex computer programs (including those with control flows and data structures), making gradient-based optimization of program parameters possible. As an emerging paradigm, differentiable programming builds upon several areas of computer science and applied mathematics, including automatic differentiation, graphical models, optimization and statistics. This book presents a comprehensive review of the fundamental concepts useful for differentiable programming. We adopt two main perspectives, that of optimization and that of probability, with clear analogies between the two. Differentiable programming is not merely the differentiation of programs, but also the thoughtful design of programs intended for differentiation. By making programs differentiable, we inherently introduce probability distributions over their execution, providing a means to quantify the uncertainty associated with program outputs.


title: Polynomial-Time Classical Simulation of Noisy IQP Circuits with Constant Depth

abstract: Sampling from the output distributions of quantum computations comprising only commuting gates, known as instantaneous quantum polynomial (IQP) computations, is believed to be intractable for classical computers, and hence this task has become a leading candidate for testing the capabilities of quantum devices. Here we demonstrate that for an arbitrary IQP circuit undergoing dephasing or depolarizing noise, whose depth is greater than a critical $O(1)$ threshold, the output distribution can be efficiently sampled by a classical computer. Unlike other simulation algorithms for quantum supremacy tasks, we do not require assumptions on the circuit's architecture, on anti-concentration properties, nor do we require $\Omega(\log(n))$ circuit depth. We take advantage of the fact that IQP circuits have deep sections of diagonal gates, which allows the noise to build up predictably and induce a large-scale breakdown of entanglement within the circuit. Our results suggest that quantum supremacy experiments based on IQP circuits may be more susceptible to classical simulation than previously thought.


title: Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey

abstract: Large models represent a groundbreaking advancement in multiple application fields, enabling remarkable achievements across various tasks. However, their unprecedented scale comes with significant computational costs. These models, often consisting of billions of parameters, require vast amounts of computational resources for execution. Especially, the expansive scale and computational demands pose considerable challenges when customizing them for particular downstream tasks, particularly over the hardware platforms constrained by computational capabilities. Parameter Efficient Fine-Tuning (PEFT) provides a practical solution by efficiently adapt the large models over the various downstream tasks. In particular, PEFT refers to the process of adjusting the parameters of a pre-trained large models to adapt it to a specific task while minimizing the number of additional parameters introduced or computational resources required. This approach is particularly important when dealing with large language models with high parameter counts, as fine-tuning these models from scratch can be computationally expensive and resource-intensive, posing considerable challenges in the supporting system platform design. In this survey, we present comprehensive studies of various PEFT algorithms, examining their performance and computational overhead. Moreover, we provide an overview of applications developed using different PEFT algorithms and discuss common techniques employed to mitigate computation costs for PEFT. In addition to the algorithmic perspective, we overview various real-world system designs to investigate the implementation costs associated with different PEFT algorithms. This survey serves as an indispensable resource for researchers aiming to understand both the PEFT algorithm and its system implementation, offering detailed insights into recent advancements and practical applications.


title: Fast and accurate log-determinant approximations

abstract: We consider the problem of estimating log-determinants of large, sparse, positive definite matrices. A key focus of our algorithm is to reduce computational cost, and it is based on sparse approximate inverses. The algorithm can be implemented to be adaptive, and it uses graph spline approximation to improve accuracy. We illustrate our approach on classes of large sparse matrices.


title: T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergy

abstract: We present T-Rex2, a highly practical model for open-set object detection. Previous open-set object detection methods relying on text prompts effectively encapsulate the abstract concept of common objects, but struggle with rare or complex object representation due to data scarcity and descriptive limitations. Conversely, visual prompts excel in depicting novel objects through concrete visual examples, but fall short in conveying the abstract concept of objects as effectively as text prompts. Recognizing the complementary strengths and weaknesses of both text and visual prompts, we introduce T-Rex2 that synergizes both prompts within a single model through contrastive learning. T-Rex2 accepts inputs in diverse formats, including text prompts, visual prompts, and the combination of both, so that it can handle different scenarios by switching between the two prompt modalities. Comprehensive experiments demonstrate that T-Rex2 exhibits remarkable zero-shot object detection capabilities across a wide spectrum of scenarios. We show that text prompts and visual prompts can benefit from each other within the synergy, which is essential to cover massive and complicated real-world scenarios and pave the way towards generic object detection. Model API is now available at \url{https://github.com/IDEA-Research/T-Rex}.


title: Explorative Inbetweening of Time and Space

abstract: We introduce bounded generation as a generalized task to control video generation to synthesize arbitrary camera and subject motion based only on a given start and end frame. Our objective is to fully leverage the inherent generalization capability of an image-to-video model without additional training or fine-tuning of the original model. This is achieved through the proposed new sampling strategy, which we call Time Reversal Fusion, that fuses the temporally forward and backward denoising paths conditioned on the start and end frame, respectively. The fused path results in a video that smoothly connects the two frames, generating inbetweening of faithful subject motion, novel views of static scenes, and seamless video looping when the two bounding frames are identical. We curate a diverse evaluation dataset of image pairs and compare against the closest existing methods. We find that Time Reversal Fusion outperforms related work on all subtasks, exhibiting the ability to generate complex motions and 3D-consistent views guided by bounded frames. See project page at https://time-reversal.github.io.


title: Observing the Galactic Underworld: Predicting photometry and astrometry from compact remnant microlensing events

abstract: Isolated black holes (BHs) and neutron stars (NSs) are largely undetectable across the electromagnetic spectrum. For this reason, our only real prospect of observing these isolated compact remnants is via microlensing; a feat recently performed for the first time. However, characterisation of the microlensing events caused by BHs and NSs is still in its infancy. In this work, we perform N-body simulations to explore the frequency and physical characteristics of microlensing events across the entire sky. Our simulations find that every year we can expect $88_{-6}^{+6}$ BH, $6.8_{-1.6}^{+1.7}$ NS and $20^{+30}_{-20}$ stellar microlensing events which cause an astrometric shift larger than 2~mas. Similarly, we can expect $21_{-3}^{+3}$ BH, $18_{-3}^{+3}$ NS and $7500_{-500}^{+500}$ stellar microlensing events which cause a bump magnitude larger than 1~mag. Leveraging a more comprehensive dynamical model than prior work, we predict the fraction of microlensing events caused by BHs as a function of Einstein time to be smaller than previously thought. Comparison of our microlensing simulations to events in Gaia finds good agreement. Finally, we predict that in the combination of Gaia and GaiaNIR data there will be $14700_{-900}^{+600}$ BH and $1600_{-200}^{+300}$ NS events creating a centroid shift larger than 1~mas and $330_{-120}^{+100}$ BH and $310_{-100}^{+110}$ NS events causing bump magnitudes $> 1$. Of these, $<10$ BH and $5_{-5}^{+10}$ NS events should be detectable using current analysis techniques. These results inform future astrometric mission design, such as GaiaNIR, as they indicate that, compared to stellar events, there are fewer observable BH events than previously thought.


title: DreamReward: Text-to-3D Generation with Human Preference

abstract: 3D content creation from text prompts has shown remarkable success recently. However, current text-to-3D methods often generate 3D results that do not align well with human preferences. In this paper, we present a comprehensive framework, coined DreamReward, to learn and improve text-to-3D models from human preference feedback. To begin with, we collect 25k expert comparisons based on a systematic annotation pipeline including rating and ranking. Then, we build Reward3D -- the first general-purpose text-to-3D human preference reward model to effectively encode human preferences. Building upon the 3D reward model, we finally perform theoretical analysis and present the Reward3D Feedback Learning (DreamFL), a direct tuning algorithm to optimize the multi-view diffusion models with a redefined scorer. Grounded by theoretical proof and extensive experiment comparisons, our DreamReward successfully generates high-fidelity and 3D consistent results with significant boosts in prompt alignment with human intention. Our results demonstrate the great potential for learning from human feedback to improve text-to-3D models.


title: AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation

abstract: In the image acquisition process, various forms of degradation, including noise, haze, and rain, are frequently introduced. These degradations typically arise from the inherent limitations of cameras or unfavorable ambient conditions. To recover clean images from degraded versions, numerous specialized restoration methods have been developed, each targeting a specific type of degradation. Recently, all-in-one algorithms have garnered significant attention by addressing different types of degradations within a single model without requiring prior information of the input degradation type. However, these methods purely operate in the spatial domain and do not delve into the distinct frequency variations inherent to different degradation types. To address this gap, we propose an adaptive all-in-one image restoration network based on frequency mining and modulation. Our approach is motivated by the observation that different degradation types impact the image content on different frequency subbands, thereby requiring different treatments for each restoration task. Specifically, we first mine low- and high-frequency information from the input features, guided by the adaptively decoupled spectra of the degraded image. The extracted features are then modulated by a bidirectional operator to facilitate interactions between different frequency components. Finally, the modulated features are merged into the original input for a progressively guided restoration. With this approach, the model achieves adaptive reconstruction by accentuating the informative frequency subbands according to different input degradations. Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance on different image restoration tasks, including denoising, dehazing, deraining, motion deblurring, and low-light image enhancement. Our code is available at https://github.com/c-yn/AdaIR.


title: Search for protostellar jets with UWISH2 in the molecular cloud complexes Vulpecula and IRDC G53.2

abstract: Jets and outflows are the early signposts of stellar birth. Using the UKIRT Wide Field Infrared Survey for H2 (UWISH2) at 2.12 micron, 127 outflows are identified in molecular cloud complexes Vulpecula OB1 and IRDC G53.2 covering 12 square degrees of the Galactic plane. Using multi-wavelength datasets, from 1.2 to 70 micron, 79 young stellar objects (YSOs) are proposed as potential driving sources, where, $\sim$ 79% are likely Class 0/I protostars, 17% are Class II YSOs and the remaining 4% are Class III YSOs. The outflows are characterized in terms of their length, flux, luminosity and knot-spacing. The identified outflows have a median lobe length of 0.22 pc and 0.17 pc for outflows in Vulpecula OB1 and IRDC G53.2, respectively. Our analysis, from the knot spacing, reveals a typical ejection frequency of $\sim$ 1.2 kyr suggesting an intermediate type between the FU-Ori and EX-Ori type of eruptions in both cloud complexes. Furthermore, the physical parameters of the driving sources are obtained by performing radiative transfer modelling to the observed spectral energy distributions (SEDs), which suggest that the outflows are driven by intermediate mass stars. Various observed trends between the outflow properties and the corresponding driving sources, and various interesting outflows and star forming sites, including sites of triggered star formation and protocluster forming clump with clusters of jets, are discussed. The obtained results and the identified jet-bearing protostellar sample will pave the way to understand many aspects of outflows with future high-resolution observations.


title: Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning

abstract: Self-supervised representation learning has been highly promising for histopathology image analysis with numerous approaches leveraging their patient-slide-patch hierarchy to learn better representations. In this paper, we explore how the combination of domain specific natural language information with such hierarchical visual representations can benefit rich representation learning for medical image tasks. Building on automated language description generation for features visible in histopathology images, we present a novel language-tied self-supervised learning framework, Hierarchical Language-tied Self-Supervision (HLSS) for histopathology images. We explore contrastive objectives and granular language description based text alignment at multiple hierarchies to inject language modality information into the visual representations. Our resulting model achieves state-of-the-art performance on two medical imaging benchmarks, OpenSRH and TCGA datasets. Our framework also provides better interpretability with our language aligned representation space. Code is available at https://github.com/Hasindri/HLSS.


title: Videoshop: Localized Semantic Video Editing with Noise-Extrapolated Diffusion Inversion

abstract: We introduce Videoshop, a training-free video editing algorithm for localized semantic edits. Videoshop allows users to use any editing software, including Photoshop and generative inpainting, to modify the first frame; it automatically propagates those changes, with semantic, spatial, and temporally consistent motion, to the remaining frames. Unlike existing methods that enable edits only through imprecise textual instructions, Videoshop allows users to add or remove objects, semantically change objects, insert stock photos into videos, etc. with fine-grained control over locations and appearance. We achieve this through image-based video editing by inverting latents with noise extrapolation, from which we generate videos conditioned on the edited image. Videoshop produces higher quality edits against 6 baselines on 2 editing benchmarks using 10 evaluation metrics.


title: Simulation-Based Inference of the sky-averaged 21-cm signal from CD-EoR with REACH

abstract: The redshifted 21-cm signal from the Cosmic Dawn and Epoch of Reionization carries invaluable information about the cosmology and astrophysics of the early Universe. Analyzing the data from a sky-averaged 21-cm signal experiment typically involves navigating through an intricate parameter space to accurately address various factors such as foregrounds, beam uncertainties, ionospheric distortions, and receiver noise for the search of the cosmological 21-cm signal. The traditional likelihood-based sampling methods for modeling these effects could become computationally demanding for such highly complex models, which makes it infeasible to include physically motivated 21-cm signal models in the analysis. Moreover, the inference with these traditional methods is driven by the assumed functional form of the likelihood function. This work demonstrates how Simulation-Based Inference through Truncated Marginal Neural Ratio Estimation (TMNRE) can naturally handle these issues at a significantly reduced computational cost than the likelihood-based methods. We estimate the posterior distribution on our model parameters with TMNRE for simulated mock observations, composed of beam-weighted foregrounds, physically motivated 21-cm signal, and radiometric noise. We find that maximizing the information content by simultaneously analyzing the data from multiple time slices and antennas significantly improves the parameter constraints and leads to a better exploration of the cosmological signal. We discuss the application of TMNRE for the current configuration of the REACH experiment and demonstrate how it can be utilized to explore potential avenues for REACH. The method presented here can be easily generalized for any sky-averaged 21-cm signal experiment.


title: ClusteringSDF: Self-Organized Neural Implicit Surfaces for 3D Decomposition

abstract: 3D decomposition/segmentation still remains a challenge as large-scale 3D annotated data is not readily available. Contemporary approaches typically leverage 2D machine-generated segments, integrating them for 3D consistency. While the majority of these methods are based on NeRFs, they face a potential weakness that the instance/semantic embedding features derive from independent MLPs, thus preventing the segmentation network from learning the geometric details of the objects directly through radiance and density. In this paper, we propose ClusteringSDF, a novel approach to achieve both segmentation and reconstruction in 3D via the neural implicit surface representation, specifically Signal Distance Function (SDF), where the segmentation rendering is directly integrated with the volume rendering of neural implicit surfaces. Although based on ObjectSDF++, ClusteringSDF no longer requires the ground-truth segments for supervision while maintaining the capability of reconstructing individual object surfaces, but purely with the noisy and inconsistent labels from pre-trained models.As the core of ClusteringSDF, we introduce a high-efficient clustering mechanism for lifting the 2D labels to 3D and the experimental results on the challenging scenes from ScanNet and Replica datasets show that ClusteringSDF can achieve competitive performance compared against the state-of-the-art with significantly reduced training time.


title: From Local Spin Nematicity to Altermagnets: Footprints of Band Topology

abstract: Altermagnets are crystallographic rotational symmetry breaking spin-ordered states, possessing a net zero magnetization despite manifesting Kramers non-degenerate bands. Here, we show that momentum-independent local spin nematic orders in monolayer, Bernal bilayer and rhombohedral trilayer graphene give rise to $p$-wave, $d$-wave and $f$-wave altermagnets, respectively, thereby inheriting topology of linear, quadratic and cubic free fermion band dispersions that are also described in terms of angular momentum $\ell=1,\; 2$ and $3$ harmonics in the reciprocal space. The same conclusions also hold inside a spin-triplet nematic superconductor, featuring Majorana altermagnets. Altogether, these findings highlight the importance of electronic band structure in identifying such exotic magnetic orders in quantum materials. We depict the effects of in-plane magnetic fields on altermagnets, and propose novel spin-disordered alter-valleymagnets in these systems.


title: GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation

abstract: We introduce GRM, a large-scale reconstructor capable of recovering a 3D asset from sparse-view images in around 0.1s. GRM is a feed-forward transformer-based model that efficiently incorporates multi-view information to translate the input pixels into pixel-aligned Gaussians, which are unprojected to create a set of densely distributed 3D Gaussians representing a scene. Together, our transformer architecture and the use of 3D Gaussians unlock a scalable and efficient reconstruction framework. Extensive experimental results demonstrate the superiority of our method over alternatives regarding both reconstruction quality and efficiency. We also showcase the potential of GRM in generative tasks, i.e., text-to-3D and image-to-3D, by integrating it with existing multi-view diffusion models. Our project website is at: https://justimyhxu.github.io/projects/grm/.


title: Language Repository for Long Video Understanding

abstract: Language has become a prominent modality in computer vision with the rise of multi-modal LLMs. Despite supporting long context-lengths, their effectiveness in handling long-term information gradually declines with input length. This becomes critical, especially in applications such as long-form video understanding. In this paper, we introduce a Language Repository (LangRepo) for LLMs, that maintains concise and structured information as an interpretable (i.e., all-textual) representation. Our repository is updated iteratively based on multi-scale video chunks. We introduce write and read operations that focus on pruning redundancies in text, and extracting information at various temporal scales. The proposed framework is evaluated on zero-shot visual question-answering benchmarks including EgoSchema, NExT-QA, IntentQA and NExT-GQA, showing state-of-the-art performance at its scale. Our code is available at https://github.com/kkahatapitiya/LangRepo.


title: Simplified Diffusion Schr\"odinger Bridge

abstract: This paper introduces a novel theoretical simplification of the Diffusion Schr\"odinger Bridge (DSB) that facilitates its unification with Score-based Generative Models (SGMs), addressing the limitations of DSB in complex data generation and enabling faster convergence and enhanced performance. By employing SGMs as an initial solution for DSB, our approach capitalizes on the strengths of both frameworks, ensuring a more efficient training process and improving the performance of SGM. We also propose a reparameterization technique that, despite theoretical approximations, practically improves the network's fitting capabilities. Our extensive experimental evaluations confirm the effectiveness of the simplified DSB, demonstrating its significant improvements. We believe the contributions of this work pave the way for advanced generative modeling. The code is available at https://github.com/checkcrab/SDSB.


title: MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?

abstract: The remarkable progress of Multi-modal Large Language Models (MLLMs) has garnered unparalleled attention, due to their superior performance in visual contexts. However, their capabilities in visual math problem-solving remain insufficiently evaluated and understood. We investigate current benchmarks to incorporate excessive visual content within textual questions, which potentially assist MLLMs in deducing answers without truly interpreting the input diagrams. To this end, we introduce MathVerse, an all-around visual math benchmark designed for an equitable and in-depth evaluation of MLLMs. We meticulously collect 2,612 high-quality, multi-subject math problems with diagrams from publicly available sources. Each problem is then transformed by human annotators into six distinct versions, each offering varying degrees of information content in multi-modality, contributing to 15K test samples in total. This approach allows MathVerse to comprehensively assess whether and how much MLLMs can truly understand the visual diagrams for mathematical reasoning. In addition, we propose a Chain-of-Thought (CoT) evaluation strategy for a fine-grained assessment of the output answers. Rather than naively judging True or False, we employ GPT-4(V) to adaptively extract crucial reasoning steps, and then score each step with detailed error analysis, which can reveal the intermediate CoT reasoning quality by MLLMs. We hope the MathVerse benchmark may provide unique insights to guide the future development of MLLMs. Project page: https://mathverse-cuhk.github.io


title: LiFT: A Surprisingly Simple Lightweight Feature Transform for Dense ViT Descriptors

abstract: We present a simple self-supervised method to enhance the performance of ViT features for dense downstream tasks. Our Lightweight Feature Transform (LiFT) is a straightforward and compact postprocessing network that can be applied to enhance the features of any pre-trained ViT backbone. LiFT is fast and easy to train with a self-supervised objective, and it boosts the density of ViT features for minimal extra inference cost. Furthermore, we demonstrate that LiFT can be applied with approaches that use additional task-specific downstream modules, as we integrate LiFT with ViTDet for COCO detection and segmentation. Despite the simplicity of LiFT, we find that it is not simply learning a more complex version of bilinear interpolation. Instead, our LiFT training protocol leads to several desirable emergent properties that benefit ViT features in dense downstream tasks. This includes greater scale invariance for features, and better object boundary maps. By simply training LiFT for a few epochs, we show improved performance on keypoint correspondence, detection, segmentation, and object discovery tasks. Overall, LiFT provides an easy way to unlock the benefits of denser feature arrays for a fraction of the computational cost. For more details, refer to our project page at https://www.cs.umd.edu/~sakshams/LiFT/.


title: ODTFormer: Efficient Obstacle Detection and Tracking with Stereo Cameras Based on Transformer

abstract: Obstacle detection and tracking represent a critical component in robot autonomous navigation. In this paper, we propose ODTFormer, a Transformer-based model to address both obstacle detection and tracking problems. For the detection task, our approach leverages deformable attention to construct a 3D cost volume, which is decoded progressively in the form of voxel occupancy grids. We further track the obstacles by matching the voxels between consecutive frames. The entire model can be optimized in an end-to-end manner. Through extensive experiments on DrivingStereo and KITTI benchmarks, our model achieves state-of-the-art performance in the obstacle detection task. We also report comparable accuracy to state-of-the-art obstacle tracking models while requiring only a fraction of their computation cost, typically ten-fold to twenty-fold less. The code and model weights will be publicly released.


title: MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images

abstract: We propose MVSplat, an efficient feed-forward 3D Gaussian Splatting model learned from sparse multi-view images. To accurately localize the Gaussian centers, we propose to build a cost volume representation via plane sweeping in the 3D space, where the cross-view feature similarities stored in the cost volume can provide valuable geometry cues to the estimation of depth. We learn the Gaussian primitives' opacities, covariances, and spherical harmonics coefficients jointly with the Gaussian centers while only relying on photometric supervision. We demonstrate the importance of the cost volume representation in learning feed-forward Gaussian Splatting models via extensive experimental evaluations. On the large-scale RealEstate10K and ACID benchmarks, our model achieves state-of-the-art performance with the fastest feed-forward inference speed (22 fps). Compared to the latest state-of-the-art method pixelSplat, our model uses $10\times $ fewer parameters and infers more than $2\times$ faster while providing higher appearance and geometry quality as well as better cross-dataset generalization.


title: Zero-Shot Multi-Object Shape Completion

abstract: We present a 3D shape completion method that recovers the complete geometry of multiple objects in complex scenes from a single RGB-D image. Despite notable advancements in single object 3D shape completion, high-quality reconstructions in highly cluttered real-world multi-object scenes remains a challenge. To address this issue, we propose OctMAE, an architecture that leverages an Octree U-Net and a latent 3D MAE to achieve high-quality and near real-time multi-object shape completion through both local and global geometric reasoning. Because a na\"ive 3D MAE can be computationally intractable and memory intensive even in the latent space, we introduce a novel occlusion masking strategy and adopt 3D rotary embeddings, which significantly improves the runtime and shape completion quality. To generalize to a wide range of objects in diverse scenes, we create a large-scale photorealistic dataset, featuring a diverse set of 12K 3D object models from the Objaverse dataset which are rendered in multi-object scenes with physics-based positioning. Our method outperforms the current state-of-the-art on both synthetic and real-world datasets and demonstrates a strong zero-shot capability.


title: Physics-based signal analysis of genome sequences: GenomeBits overview

abstract: A comprehensive overview of the recent physics-inspired genome analysis tool, GenomeBits, is presented. This is based on traditional signal processing methods such as Discrete Fourier Transform (DFT). GenomeBits can be used to extract underlying genomics features from the distribution of nucleotides, and can be further used to analyze the mutation patterns in viral genomes. Examples of the main GenomeBits findings outlining the intrinsic signal organization of genomics sequences for different SARS-CoV-2 variants along the pandemic years 2020-2022 and Monkeypox cases in 2021 are presented to show the usefulness of GenomeBits. GenomeBits results for DFT of SARS-CoV-2 genomes in different geographical regions are discussed together with the GenomeBits analysis of complete genome sequences for the first coronavirus variants reported: Alpha, Beta, Gamma, Epsilon and Eta. Interesting features of the Delta and Omicron variants in the form of a unique "order-disorder" transition are uncovered from these samples as well as from their cumulative distribution function and scatter plots. This class of transitions might reveal the cumulative outcome of mutations on the spike protein. A salient feature of GenomeBits is the mapping of the nucleotide bases (A,T,C,G) into an alternating spin-like numerical sequence via a series having binary (0,1) indicators for each A,T,C,G. This leads to derive a set of statistical distribution curves. Furthermore, the quantum-based extension of the GenomeBits model to an analogous probability measure is shown to identify properties of genome sequences as wavefunctions via a superposition of states. An association of the integral of the GenomeBits coding and a binding-like energy can in principle also be established. The relevance of these different results in Bioinformatics is analyzed.


title: Study on quantum thermalization from thermal initial states in a superconducting quantum computer

abstract: Quantum thermalization in contemporary quantum devices, in particular quantum computers, has recently attracted significant theoretical interest. Unusual thermalization processes, such as the Quantum Mpemba Effect (QME), have been explored theoretically. However, there is a shortage of experimental results due to the difficulty in preparing thermal states. In this paper, we propose a method to address this challenge. Moreover, we experimentally validate our approach using IBM quantum devices, providing results for unusal relaxation in equidistant quenches as predicted for the IBM qubit. We also assess the formalism introduced for the QME, obtaining results consistent with the theoretical predictions. This demonstration underscores that our method can streamline the investigation of thermal states and thermalization in quantum physics.


title: Predicting root numbers with neural networks

abstract: We report on two machine learning experiments in search of statistical relationships between Dirichlet coefficients and root numbers or analytic ranks of certain low-degree $L$-functions. The first experiment is to construct interpretable models based on murmurations, a recently discovered correlation between Dirichlet coefficients and root numbers. We show experimentally that these models achieve high accuracy by learning a combination of Mestre-Nagao type heuristics and murmurations, noting that the relative importance of these features varies with degree. The second experiment is to search for a low-complexity statistic of Dirichlet coefficients that can be used to predict root numbers in polynomial time. We give experimental evidence and provide heuristics that suggest this can not be done with standard machine learning techniques.


title: Hyperbolic Jacobsthal Spinor Sequences and Their Mathematical Properties

abstract: In this study, novel Hyperbolic spinor sequences of Jacobsthal, Jacobsthal-Lucas and Jacobsthal polynomial, which have not been studied before, are defined by investigating the relationship between spinors, which are important mathematical objects used in physics and mathematics, and split Jacobsthal and split Jacobsthal-Lucas quaternions, which are extensions of the known Jacobsthal and Jacobsthal-Lucas numbers to quaternion algebra. The recurrence relations of sequences whose members are Hyperbolic Jacobsthal, Jacobsthal-Lucas and Jacobsthal polynomial spinors are described. Additionally, certain properties of these spinors, such as the generator function and Binet formula, are presented and some identities resulting from these spinors are obtained.


title: Born With a Silver Spoon? Investigating Socioeconomic Bias in Large Language Models

abstract: Socioeconomic bias in society exacerbates disparities, influencing access to opportunities and resources based on individuals' economic and social backgrounds. This pervasive issue perpetuates systemic inequalities, hindering the pursuit of inclusive progress as a society. In this paper, we investigate the presence of socioeconomic bias, if any, in large language models. To this end, we introduce a novel dataset SilverSpoon, consisting of 3000 samples that illustrate hypothetical scenarios that involve underprivileged people performing ethically ambiguous actions due to their circumstances, and ask whether the action is ethically justified. Further, this dataset has a dual-labeling scheme and has been annotated by people belonging to both ends of the socioeconomic spectrum. Using SilverSpoon, we evaluate the degree of socioeconomic bias expressed in large language models and the variation of this degree as a function of model size. We also perform qualitative analysis to analyze the nature of this bias. Our analysis reveals that while humans disagree on which situations require empathy toward the underprivileged, most large language models are unable to empathize with the socioeconomically underprivileged regardless of the situation. To foster further research in this domain, we make SilverSpoon and our evaluation harness publicly available.


title: Bounds for moments of quadratic Dirichlet character sums

abstract: We establish upper bounds for moments of smoothed quadratic Dirichlet character sums under the generalized Riemann hypothesis, confirming a conjecture of M. Jutila.


title: AI Sustainability in Practice Part One: Foundations for Sustainable AI Projects

abstract: Sustainable AI projects are continuously responsive to the transformative effects as well as short-, medium-, and long-term impacts on individuals and society that the design, development, and deployment of AI technologies may have. Projects, which centre AI Sustainability, ensure that values-led, collaborative, and anticipatory reflection both guides the assessment of potential social and ethical impacts and steers responsible innovation practices. This workbook is the first part of a pair that provides the concepts and tools needed to put AI Sustainability into practice. It introduces the SUM Values, which help AI project teams to assess the potential societal impacts and ethical permissibility of their projects. It then presents a Stakeholder Engagement Process (SEP), which provides tools to facilitate proportionate engagement of and input from stakeholders with an emphasis on equitable and meaningful participation and positionality awareness.


title: AI Fairness in Practice

abstract: Reaching consensus on a commonly accepted definition of AI Fairness has long been a central challenge in AI ethics and governance. There is a broad spectrum of views across society on what the concept of fairness means and how it should best be put to practice. In this workbook, we tackle this challenge by exploring how a context-based and society-centred approach to understanding AI Fairness can help project teams better identify, mitigate, and manage the many ways that unfair bias and discrimination can crop up across the AI project workflow. We begin by exploring how, despite the plurality of understandings about the meaning of fairness, priorities of equality and non-discrimination have come to constitute the broadly accepted core of its application as a practical principle. We focus on how these priorities manifest in the form of equal protection from direct and indirect discrimination and from discriminatory harassment. These elements form ethical and legal criteria based upon which instances of unfair bias and discrimination can be identified and mitigated across the AI project workflow. We then take a deeper dive into how the different contexts of the AI project lifecycle give rise to different fairness concerns. This allows us to identify several types of AI Fairness (Data Fairness, Application Fairness, Model Design and Development Fairness, Metric-Based Fairness, System Implementation Fairness, and Ecosystem Fairness) that form the basis of a multi-lens approach to bias identification, mitigation, and management. Building on this, we discuss how to put the principle of AI Fairness into practice across the AI project workflow through Bias Self-Assessment and Bias Risk Management as well as through the documentation of metric-based fairness criteria in a Fairness Position Statement.


title: SimGrade: Using Code Similarity Measures for More Accurate Human Grading

abstract: While the use of programming problems on exams is a common form of summative assessment in CS courses, grading such exam problems can be a difficult and inconsistent process. Through an analysis of historical grading patterns we show that inaccurate and inconsistent grading of free-response programming problems is widespread in CS1 courses. These inconsistencies necessitate the development of methods to ensure more fairer and more accurate grading. In subsequent analysis of this historical exam data we demonstrate that graders are able to more accurately assign a score to a student submission when they have previously seen another submission similar to it. As a result, we hypothesize that we can improve exam grading accuracy by ensuring that each submission that a grader sees is similar to at least one submission they have previously seen. We propose several algorithms for (1) assigning student submissions to graders, and (2) ordering submissions to maximize the probability that a grader has previously seen a similar solution, leveraging distributed representations of student code in order to measure similarity between submissions. Finally, we demonstrate in simulation that these algorithms achieve higher grading accuracy than the current standard random assignment process used for grading.


title: Personalized Programming Guidance based on Deep Programming Learning Style Capturing

abstract: With the rapid development of big data and AI technology, programming is in high demand and has become an essential skill for students. Meanwhile, researchers also focus on boosting the online judging system's guidance ability to reduce students' dropout rates. Previous studies mainly targeted at enhancing learner engagement on online platforms by providing personalized recommendations. However, two significant challenges still need to be addressed in programming: C1) how to recognize complex programming behaviors; C2) how to capture intrinsic learning patterns that align with the actual learning process. To fill these gaps, in this paper, we propose a novel model called Programming Exercise Recommender with Learning Style (PERS), which simulates learners' intricate programming behaviors. Specifically, since programming is an iterative and trial-and-error process, we first introduce a positional encoding and a differentiating module to capture the changes of consecutive code submissions (which addresses C1). To better profile programming behaviors, we extend the Felder-Silverman learning style model, a classical pedagogical theory, to perceive intrinsic programming patterns. Based on this, we align three latent vectors to record and update programming ability, processing style, and understanding style, respectively (which addresses C2). We perform extensive experiments on two real-world datasets to verify the rationality of modeling programming learning styles and the effectiveness of PERS for personalized programming guidance.