title: RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content

abstract: Large Language Models (LLMs) are trained on vast amounts of data, most of which is automatically scraped from the internet. This data includes encyclopedic documents that harbor a vast amount of general knowledge (e.g., Wikipedia) but also potentially overlap with benchmark datasets used for evaluating LLMs. Consequently, evaluating models on test splits that might have leaked into the training set is prone to misleading conclusions. To foster sound evaluation of language models, we introduce a new test dataset named RepLiQA, suited for question-answering and topic retrieval tasks. RepLiQA is a collection of five splits of test sets, four of which have not been released to the internet or exposed to LLM APIs prior to this publication. Each sample in RepLiQA comprises (1) a reference document crafted by a human annotator and depicting an imaginary scenario (e.g., a news article) absent from the internet; (2) a question about the document's topic; (3) a ground-truth answer derived directly from the information in the document; and (4) the paragraph extracted from the reference document containing the answer. As such, accurate answers can only be generated if a model can find relevant content within the provided document. We run a large-scale benchmark comprising several state-of-the-art LLMs to uncover differences in performance across models of various types and sizes in a context-conditional language modeling setting. Released splits of RepLiQA can be found here: https://huggingface.co/datasets/ServiceNow/repliqa.


title: Unified analysis of algorithms for equilibrium, non-equilibrium, and hysteresis models of phase transition in permafrost

abstract: In this paper we consider a nonlinear partial differential equation describing heat flow with ice-water phase transition in permafrost soils. Such models and their numerical approximations have been well explored in the applications literature. In this paper we describe a new direction in which the allow relaxation and hysteresis of the phase transition which introduce additional nonlinear terms and complications for the analysis. We present numerical algorithms as well as analysis of the well-posedness and convergence of the fully implicit iterative schemes. The analysis we propose handles the equilibrium, non-equilibrium, and hysteresis cases in a unified way. We also illustrate with numerical examples for a model ODE and PDE.


title: How Do Large Language Models Acquire Factual Knowledge During Pretraining?

abstract: Despite the recent observation that large language models (LLMs) can store substantial factual knowledge, there is a limited understanding of the mechanisms of how they acquire factual knowledge through pretraining. This work addresses this gap by studying how LLMs acquire factual knowledge during pretraining. The findings reveal several important insights into the dynamics of factual knowledge acquisition during pretraining. First, counterintuitively, we observe that pretraining on more data shows no significant improvement in the model's capability to acquire and maintain factual knowledge. Next, there is a power-law relationship between training steps and forgetting of memorization and generalization of factual knowledge, and LLMs trained with duplicated training data exhibit faster forgetting. Third, training LLMs with larger batch sizes can enhance the models' robustness to forgetting. Overall, our observations suggest that factual knowledge acquisition in LLM pretraining occurs by progressively increasing the probability of factual knowledge presented in the pretraining data at each step. However, this increase is diluted by subsequent forgetting. Based on this interpretation, we demonstrate that we can provide plausible explanations for recently observed behaviors of LLMs, such as the poor performance of LLMs on long-tail knowledge and the benefits of deduplicating the pretraining corpus.


title: Stochastic Neural Network Symmetrisation in Markov Categories

abstract: We consider the problem of symmetrising a neural network along a group homomorphism: given a homomorphism $\varphi : H \to G$, we would like a procedure that converts $H$-equivariant neural networks into $G$-equivariant ones. We formulate this in terms of Markov categories, which allows us to consider neural networks whose outputs may be stochastic, but with measure-theoretic details abstracted away. We obtain a flexible, compositional, and generic framework for symmetrisation that relies on minimal assumptions about the structure of the group and the underlying neural network architecture. Our approach recovers existing methods for deterministic symmetrisation as special cases, and extends directly to provide a novel methodology for stochastic symmetrisation also. Beyond this, we believe our findings also demonstrate the utility of Markov categories for addressing problems in machine learning in a conceptual yet mathematically rigorous way.


title: LLARVA: Vision-Action Instruction Tuning Enhances Robot Learning

abstract: In recent years, instruction-tuned Large Multimodal Models (LMMs) have been successful at several tasks, including image captioning and visual question answering; yet leveraging these models remains an open question for robotics. Prior LMMs for robotics applications have been extensively trained on language and action data, but their ability to generalize in different settings has often been less than desired. To address this, we introduce LLARVA, a model trained with a novel instruction tuning method that leverages structured prompts to unify a range of robotic learning tasks, scenarios, and environments. Additionally, we show that predicting intermediate 2-D representations, which we refer to as "visual traces", can help further align vision and action spaces for robot learning. We generate 8.5M image-visual trace pairs from the Open X-Embodiment dataset in order to pre-train our model, and we evaluate on 12 different tasks in the RLBench simulator as well as a physical Franka Emika Panda 7-DoF robot. Our experiments yield strong performance, demonstrating that LLARVA - using 2-D and language representations - performs well compared to several contemporary baselines, and can generalize across various robot environments and configurations.


title: VideoLLM-online: Online Video Large Language Model for Streaming Video

abstract: Recent Large Language Models have been enhanced with vision capabilities, enabling them to comprehend images, videos, and interleaved vision-language content. However, the learning methods of these large multimodal models typically treat videos as predetermined clips, making them less effective and efficient at handling streaming video inputs. In this paper, we propose a novel Learning-In-Video-Stream (LIVE) framework, which enables temporally aligned, long-context, and real-time conversation within a continuous video stream. Our LIVE framework comprises comprehensive approaches to achieve video streaming dialogue, encompassing: (1) a training objective designed to perform language modeling for continuous streaming inputs, (2) a data generation scheme that converts offline temporal annotations into a streaming dialogue format, and (3) an optimized inference pipeline to speed up the model responses in real-world video streams. With our LIVE framework, we built VideoLLM-online model upon Llama-2/Llama-3 and demonstrate its significant advantages in processing streaming videos. For instance, on average, our model can support streaming dialogue in a 5-minute video clip at over 10 FPS on an A100 GPU. Moreover, it also showcases state-of-the-art performance on public offline video benchmarks, such as recognition, captioning, and forecasting. The code, model, data, and demo have been made available at https://showlab.github.io/videollm-online.


title: Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Level

abstract: Direct Preference Optimization (DPO), a standard method for aligning language models with human preferences, is traditionally applied to offline preferences. Recent studies show that DPO benefits from iterative training with online preferences labeled by a trained reward model. In this work, we identify a pitfall of vanilla iterative DPO - improved response quality can lead to increased verbosity. To address this, we introduce iterative length-regularized DPO (iLR-DPO) to penalize response length. Our empirical results show that iLR-DPO can enhance a 7B model to perform on par with GPT-4 without increasing verbosity. Specifically, our 7B model achieves a $50.5\%$ length-controlled win rate against $\texttt{GPT-4 Preview}$ on AlpacaEval 2.0, and excels across standard benchmarks including MT-Bench, Arena-Hard and OpenLLM Leaderboard. These results demonstrate the effectiveness of iterative DPO in aligning language models with human feedback.


title: Embodied Instruction Following in Unknown Environments

abstract: Enabling embodied agents to complete complex human instructions from natural language is crucial to autonomous systems in household services. Conventional methods can only accomplish human instructions in the known environment where all interactive objects are provided to the embodied agent, and directly deploying the existing approaches for the unknown environment usually generates infeasible plans that manipulate non-existing objects. On the contrary, we propose an embodied instruction following (EIF) method for complex tasks in the unknown environment, where the agent efficiently explores the unknown environment to generate feasible plans with existing objects to accomplish abstract instructions. Specifically, we build a hierarchical embodied instruction following framework including the high-level task planner and the low-level exploration controller with multimodal large language models. We then construct a semantic representation map of the scene with dynamic region attention to demonstrate the known visual clues, where the goal of task planning and scene exploration is aligned for human instruction. For the task planner, we generate the feasible step-by-step plans for human goal accomplishment according to the task completion process and the known visual clues. For the exploration controller, the optimal navigation or object interaction policy is predicted based on the generated step-wise plans and the known visual clues. The experimental results demonstrate that our method can achieve 45.09% success rate in 204 complex human instructions such as making breakfast and tidying rooms in large house-level scenes.


title: MegaScenes: Scene-Level View Synthesis at Scale

abstract: Scene-level novel view synthesis (NVS) is fundamental to many vision and graphics applications. Recently, pose-conditioned diffusion models have led to significant progress by extracting 3D information from 2D foundation models, but these methods are limited by the lack of scene-level training data. Common dataset choices either consist of isolated objects (Objaverse), or of object-centric scenes with limited pose distributions (DTU, CO3D). In this paper, we create a large-scale scene-level dataset from Internet photo collections, called MegaScenes, which contains over 100K structure from motion (SfM) reconstructions from around the world. Internet photos represent a scalable data source but come with challenges such as lighting and transient objects. We address these issues to further create a subset suitable for the task of NVS. Additionally, we analyze failure cases of state-of-the-art NVS methods and significantly improve generation consistency. Through extensive experiments, we validate the effectiveness of both our dataset and method on generating in-the-wild scenes. For details on the dataset and code, see our project page at https://megascenes.github.io .


title: Composing Object Relations and Attributes for Image-Text Matching

abstract: We study the visual semantic embedding problem for image-text matching. Most existing work utilizes a tailored cross-attention mechanism to perform local alignment across the two image and text modalities. This is computationally expensive, even though it is more powerful than the unimodal dual-encoder approach. This work introduces a dual-encoder image-text matching model, leveraging a scene graph to represent captions with nodes for objects and attributes interconnected by relational edges. Utilizing a graph attention network, our model efficiently encodes object-attribute and object-object semantic relations, resulting in a robust and fast-performing system. Representing caption as a scene graph offers the ability to utilize the strong relational inductive bias of graph neural networks to learn object-attribute and object-object relations effectively. To train the model, we propose losses that align the image and caption both at the holistic level (image-caption) and the local level (image-object entity), which we show is key to the success of the model. Our model is termed Composition model for Object Relations and Attributes, CORA. Experimental results on two prominent image-text retrieval benchmarks, Flickr30K and MSCOCO, demonstrate that CORA outperforms existing state-of-the-art computationally expensive cross-attention methods regarding recall score while achieving fast computation speed of the dual encoder.


title: Simple matrix expressions for the curvatures of Grassmannian

abstract: We show that modeling a Grassmannian as symmetric orthogonal matrices $\operatorname{Gr}(k,\mathbb{R}^n) \cong\{Q \in \mathbb{R}^{n \times n} : Q^{\scriptscriptstyle\mathsf{T}} Q = I, \; Q^{\scriptscriptstyle\mathsf{T}} = Q,\; \operatorname{tr}(Q)=2k - n\}$ yields exceedingly simple matrix formulas for various curvatures and curvature-related quantities, both intrinsic and extrinsic. These include Riemann, Ricci, Jacobi, sectional, scalar, mean, principal, and Gaussian curvatures; Schouten, Weyl, Cotton, Bach, Pleba\'nski, cocurvature, nonmetricity, and torsion tensors; first, second, and third fundamental forms; Gauss and Weingarten maps; and upper and lower delta invariants. We will derive explicit, simple expressions for the aforementioned quantities in terms of standard matrix operations that are stably computable with numerical linear algebra. Many of these aforementioned quantities have never before been presented for the Grassmannian.


title: A precision benchmark suite for nuclear reactor point kinetics equations via converged accelerated Taylor series (cats)

abstract: Extreme benchmarks of ten or more places for the point kinetics equations for time dependent nuclear reactor power transients are rare. Therefore, to establish an extreme benchmark, we will employ a Taylor series with continuous analytical continuation (CAC) to solve the ordinary differential equations of point kinetics including feedback. Non-linear Wynn-epsilon convergence acceleration confirms the highly precise solutions for neutron and precursor densities. Through adaptive partitioning of time intervals, the proposed Converged Accelerated Taylor Series, or CATS algorithm [1] in double precision, automatically performs successive mesh refinement to obtain high precision initial conditions for each sub-interval, with the intent to reduce propagation error. Confirmation of ten to twelve places comes from comparison to the BEFD (Backward Euler Finite Difference) algorithm [2] in quadruple precision also developed by the author. We report benchmark results for common cases found in the literature including step, ramp, zigzag and sinusoidal prescribed reactivity insertions and insertions with non-linear adiabatic Doppler feedback. We also establish a suite of new prescribed reactivity insertions and insertions with feedback, based on reactivities with Taylor series as suggested by the CATS algorithm.


title: On Efficient Language and Vision Assistants for Visually-Situated Natural Language Understanding: What Matters in Reading and Reasoning

abstract: Recent advancements in language and vision assistants have showcased impressive capabilities but suffer from a lack of transparency, limiting broader research and reproducibility. While open-source models handle general image tasks effectively, they face challenges with the high computational demands of complex visually-situated text understanding. Such tasks often require increased token inputs and large vision modules to harness high-resolution information. Striking a balance between model size and data importance remains an open question. This study aims to redefine the design of vision-language models by identifying key components and creating efficient models with constrained inference costs. By strategically formulating datasets, optimizing vision modules, and enhancing supervision techniques, we achieve significant improvements in inference throughput while maintaining high performance. Extensive experiments across models ranging from 160M to 13B parameters offer insights into model optimization. We will fully open-source our codebase, models, and datasets at https://github.com/naver-ai/elva .


title: Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation

abstract: We introduce Infinigen Indoors, a Blender-based procedural generator of photorealistic indoor scenes. It builds upon the existing Infinigen system, which focuses on natural scenes, but expands its coverage to indoor scenes by introducing a diverse library of procedural indoor assets, including furniture, architecture elements, appliances, and other day-to-day objects. It also introduces a constraint-based arrangement system, which consists of a domain-specific language for expressing diverse constraints on scene composition, and a solver that generates scene compositions that maximally satisfy the constraints. We provide an export tool that allows the generated 3D objects and scenes to be directly used for training embodied agents in real-time simulators such as Omniverse and Unreal. Infinigen Indoors is open-sourced under the BSD license. Please visit https://infinigen.org for code and videos.


title: Spectral Introspection Identifies Group Training Dynamics in Deep Neural Networks for Neuroimaging

abstract: Neural networks, whice have had a profound effect on how researchers study complex phenomena, do so through a complex, nonlinear mathematical structure which can be difficult for human researchers to interpret. This obstacle can be especially salient when researchers want to better understand the emergence of particular model behaviors such as bias, overfitting, overparametrization, and more. In Neuroimaging, the understanding of how such phenomena emerge is fundamental to preventing and informing users of the potential risks involved in practice. In this work, we present a novel introspection framework for Deep Learning on Neuroimaging data, which exploits the natural structure of gradient computations via the singular value decomposition of gradient components during reverse-mode auto-differentiation. Unlike post-hoc introspection techniques, which require fully-trained models for evaluation, our method allows for the study of training dynamics on the fly, and even more interestingly, allow for the decomposition of gradients based on which samples belong to particular groups of interest. We demonstrate how the gradient spectra for several common deep learning models differ between schizophrenia and control participants from the COBRE study, and illustrate how these trajectories may reveal specific training dynamics helpful for further analysis.


title: Limit theorems for extrema of Airy processes

abstract: We establish limit theorems for the maxima and minima of Airy$_1$ and Airy$_2$ processes (denoted by $\mathcal{A}_1(\cdot)$ and $\mathcal{A}_2(\cdot)$ respectively) over growing intervals. In particular, we identify the finite non-zero constants that are the almost sure limits of $(\log t)^{-2/3}\max_{0\le s \le t} \mathcal{A}_{i}(s)$ and $(\log t)^{-1/3}\min_{0\le s \le t} \mathcal{A}_{i}(s)$ for $i=1,2$. This complements and extends the results of (Pu, 2023), where the question for the maxima was considered and the order of growth was identified for both $\mathcal{A}_1$ and $\mathcal{A}_2$ and the constant was identified for $\mathcal{A}_1$. Our approach is different from that of (Pu, 2023); instead of complicated formulae for multi-point distributions, we rely on the well-known convergence of passage time profiles in planar exponential last passage percolation started from different initial conditions to $\mathcal{A}_1$ and $\mathcal{A}_2$, together with the recently developed sharp one-point estimates in (Baslingker et al., 2024) for the point-to-point and point-to-line passage times in exponential LPP and a combination of old and new results on the geometry of the LPP landscape.


title: WPO: Enhancing RLHF with Weighted Preference Optimization

abstract: Reinforcement learning from human feedback (RLHF) is a promising solution to align large language models (LLMs) more closely with human values. Off-policy preference optimization, where the preference data is obtained from other models, is widely adopted due to its cost efficiency and scalability. However, off-policy preference optimization often suffers from a distributional gap between the policy used for data collection and the target policy, leading to suboptimal optimization. In this paper, we propose a novel strategy to mitigate this problem by simulating on-policy learning with off-policy preference data. Our Weighted Preference Optimization (WPO) method adapts off-policy data to resemble on-policy data more closely by reweighting preference pairs according to their probability under the current policy. This method not only addresses the distributional gap problem but also enhances the optimization process without incurring additional costs. We validate our method on instruction following benchmarks including Alpaca Eval 2 and MT-bench. WPO not only outperforms Direct Preference Optimization (DPO) by up to 5.6% on Alpaca Eval 2 but also establishes a remarkable length-controlled winning rate against GPT-4-turbo of 48.6% based on Llama-3-8B-Instruct, making it the strongest 8B model on the leaderboard. We will release the code and models at https://github.com/wzhouad/WPO.


title: Learning sum of diverse features: computational hardness and efficient gradient-based training for ridge combinations

abstract: We study the computational and sample complexity of learning a target function $f_*:\mathbb{R}^d\to\mathbb{R}$ with additive structure, that is, $f_*(x) = \frac{1}{\sqrt{M}}\sum_{m=1}^M f_m(\langle x, v_m\rangle)$, where $f_1,f_2,...,f_M:\mathbb{R}\to\mathbb{R}$ are nonlinear link functions of single-index models (ridge functions) with diverse and near-orthogonal index features $\{v_m\}_{m=1}^M$, and the number of additive tasks $M$ grows with the dimensionality $M\asymp d^\gamma$ for $\gamma\ge 0$. This problem setting is motivated by the classical additive model literature, the recent representation learning theory of two-layer neural network, and large-scale pretraining where the model simultaneously acquires a large number of "skills" that are often localized in distinct parts of the trained network. We prove that a large subset of polynomial $f_*$ can be efficiently learned by gradient descent training of a two-layer neural network, with a polynomial statistical and computational complexity that depends on the number of tasks $M$ and the information exponent of $f_m$, despite the unknown link function and $M$ growing with the dimensionality. We complement this learnability guarantee with computational hardness result by establishing statistical query (SQ) lower bounds for both the correlational SQ and full SQ algorithms.


title: Flow Shadowing: A Method to Detect Multiple Flow Headings using an Array of Densely Packed Whisker-inspired Sensors

abstract: Understanding airflow around a drone is critical for performing advanced maneuvers while maintaining flight stability. Recent research has worked to understand this flow by employing 2D and 3D flow sensors to measure flow from a single source like wind or the drone's relative motion. Our current work advances flow detection by introducing a strategy to distinguish between two flow sources applied simultaneously from different directions. By densely packing an array of flow sensors (or whiskers), we alter the path of airflow as it moves through the array. We have named this technique ``flow shadowing'' because we take advantage of the fact that a downstream whisker shadowed (or occluded) by an upstream whisker receives less incident flow. We show that this relationship is predictable for two whiskers based on the percent of occlusion. We then show that a 2x2 spatial array of whiskers responds asymmetrically when multiple flow sources from different headings are applied to the array. This asymmetry is direction-dependent, allowing us to predict the headings of flow from two different sources, like wind and a drone's relative motion.


title: Language Modeling with Editable External Knowledge

abstract: When the world changes, so does the text that humans write about it. How do we build language models that can be easily updated to reflect these changes? One popular approach is retrieval-augmented generation, in which new documents are inserted into a knowledge base and retrieved during prediction for downstream tasks. Most prior work on these systems have focused on improving behavior during prediction through better retrieval or reasoning. This paper introduces ERASE, which instead improves model behavior when new documents are acquired, by incrementally deleting or rewriting other entries in the knowledge base each time a document is added. In two new benchmark datasets evaluating models' ability to answer questions about a stream of news articles or conversations, ERASE improves accuracy relative to conventional retrieval-augmented generation by 7-13% (Mixtral-8x7B) and 6-10% (Llama-3-8B) absolute. Code and data are available at https://github.com/belindal/ERASE


title: Exploring the Role of Large Language Models in Prompt Encoding for Diffusion Models

abstract: Large language models (LLMs) based on decoder-only transformers have demonstrated superior text understanding capabilities compared to CLIP and T5-series models. However, the paradigm for utilizing current advanced LLMs in text-to-image diffusion models remains to be explored. We observed an unusual phenomenon: directly using a large language model as the prompt encoder significantly degrades the prompt-following ability in image generation. We identified two main obstacles behind this issue. One is the misalignment between the next token prediction training in LLM and the requirement for discriminative prompt features in diffusion models. The other is the intrinsic positional bias introduced by the decoder-only architecture. To deal with this issue, we propose a novel framework to fully harness the capabilities of LLMs. Through the carefully designed usage guidance, we effectively enhance the text representation capability for prompt encoding and eliminate its inherent positional bias. This allows us to integrate state-of-the-art LLMs into the text-to-image generation model flexibly. Furthermore, we also provide an effective manner to fuse multiple LLMs into our framework. Considering the excellent performance and scaling capabilities demonstrated by the transformer architecture, we further design an LLM-Infused Diffusion Transformer (LI-DiT) based on the framework. We conduct extensive experiments to validate LI-DiT across model size and data size. Benefiting from the inherent ability of the LLMs and our innovative designs, the prompt understanding performance of LI-DiT easily surpasses state-of-the-art open-source models as well as mainstream closed-source commercial models including Stable Diffusion 3, DALL-E 3, and Midjourney V6. The powerful LI-DiT-10B will be available through the online platform and API after further optimization and security checks.


title: Unveiling Encoder-Free Vision-Language Models

abstract: Existing vision-language models (VLMs) mostly rely on vision encoders to extract visual features followed by large language models (LLMs) for visual-language tasks. However, the vision encoders set a strong inductive bias in abstracting visual representation, e.g., resolution, aspect ratio, and semantic priors, which could impede the flexibility and efficiency of the VLMs. Training pure VLMs that accept the seamless vision and language inputs, i.e., without vision encoders, remains challenging and rarely explored. Empirical observations reveal that direct training without encoders results in slow convergence and large performance gaps. In this work, we bridge the gap between encoder-based and encoder-free models, and present a simple yet effective training recipe towards pure VLMs. Specifically, we unveil the key aspects of training encoder-free VLMs efficiently via thorough experiments: (1) Bridging vision-language representation inside one unified decoder; (2) Enhancing visual recognition capability via extra supervision. With these strategies, we launch EVE, an encoder-free vision-language model that can be trained and forwarded efficiently. Notably, solely utilizing 35M publicly accessible data, EVE can impressively rival the encoder-based VLMs of similar capacities across multiple vision-language benchmarks. It significantly outperforms the counterpart Fuyu-8B with mysterious training procedures and undisclosed training data. We believe that EVE provides a transparent and efficient route for developing a pure decoder-only architecture across modalities. Our code and models are publicly available at: https://github.com/baaivision/EVE.


title: MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs

abstract: Generating natural and meaningful responses to communicate with multi-modal human inputs is a fundamental capability of Large Vision-Language Models(LVLMs). While current open-source LVLMs demonstrate promising performance in simplified scenarios such as single-turn single-image input, they fall short in real-world conversation scenarios such as following instructions in a long context history with multi-turn and multi-images. Existing LVLM benchmarks primarily focus on single-choice questions or short-form responses, which do not adequately assess the capabilities of LVLMs in real-world human-AI interaction applications. Therefore, we introduce MMDU, a comprehensive benchmark, and MMDU-45k, a large-scale instruction tuning dataset, designed to evaluate and improve LVLMs' abilities in multi-turn and multi-image conversations. We employ the clustering algorithm to ffnd the relevant images and textual descriptions from the open-source Wikipedia and construct the question-answer pairs by human annotators with the assistance of the GPT-4o model. MMDU has a maximum of 18k image+text tokens, 20 images, and 27 turns, which is at least 5x longer than previous benchmarks and poses challenges to current LVLMs. Our in-depth analysis of 15 representative LVLMs using MMDU reveals that open-source LVLMs lag behind closed-source counterparts due to limited conversational instruction tuning data. We demonstrate that ffne-tuning open-source LVLMs on MMDU-45k signiffcantly address this gap, generating longer and more accurate conversations, and improving scores on MMDU and existing benchmarks (MMStar: +1.1%, MathVista: +1.5%, ChartQA:+1.2%). Our contributions pave the way for bridging the gap between current LVLM models and real-world application demands. This project is available at https://github.com/Liuziyu77/MMDU.


title: Perfect chiral quantum routing

abstract: Routing classical and quantum information is a fundamental task for most quantum information technologies and processes. Here, we consider information encoded in the position of a quantum walker on a graph, and design an optimal structure to achieve perfect quantum routing exploiting chirality and weighting of the edges. The topology, termed the {\em Lily Graph}, enables perfect (i.e., with fidelity one) and robust routing of classical (localized) or quantum (superposition) states of the walker to $n$ different, orthogonal, spatial regions of the graph, corresponding to the $n$ possible outputs of the device. The routing time is independent of the input signal and the number of outputs, making our scheme a robust and scalable solution for quantum networks.


title: OoDIS: Anomaly Instance Segmentation Benchmark

abstract: Autonomous vehicles require a precise understanding of their environment to navigate safely. Reliable identification of unknown objects, especially those that are absent during training, such as wild animals, is critical due to their potential to cause serious accidents. Significant progress in semantic segmentation of anomalies has been driven by the availability of out-of-distribution (OOD) benchmarks. However, a comprehensive understanding of scene dynamics requires the segmentation of individual objects, and thus the segmentation of instances is essential. Development in this area has been lagging, largely due to the lack of dedicated benchmarks. To address this gap, we have extended the most commonly used anomaly segmentation benchmarks to include the instance segmentation task. Our evaluation of anomaly instance segmentation methods shows that this challenge remains an unsolved problem. The benchmark website and the competition page can be found at: https://vision.rwth-aachen.de/oodis .


title: RetinaGS: Scalable Training for Dense Scene Rendering with Billion-Scale 3D Gaussians

abstract: In this work, we explore the possibility of training high-parameter 3D Gaussian splatting (3DGS) models on large-scale, high-resolution datasets. We design a general model parallel training method for 3DGS, named RetinaGS, which uses a proper rendering equation and can be applied to any scene and arbitrary distribution of Gaussian primitives. It enables us to explore the scaling behavior of 3DGS in terms of primitive numbers and training resolutions that were difficult to explore before and surpass previous state-of-the-art reconstruction quality. We observe a clear positive trend of increasing visual quality when increasing primitive numbers with our method. We also demonstrate the first attempt at training a 3DGS model with more than one billion primitives on the full MatrixCity dataset that attains a promising visual quality.


title: Scaling the Codebook Size of VQGAN to 100,000 with a Utilization Rate of 99%

abstract: In the realm of image quantization exemplified by VQGAN, the process encodes images into discrete tokens drawn from a codebook with a predefined size. Recent advancements, particularly with LLAMA 3, reveal that enlarging the codebook significantly enhances model performance. However, VQGAN and its derivatives, such as VQGAN-FC (Factorized Codes) and VQGAN-EMA, continue to grapple with challenges related to expanding the codebook size and enhancing codebook utilization. For instance, VQGAN-FC is restricted to learning a codebook with a maximum size of 16,384, maintaining a typically low utilization rate of less than 12% on ImageNet. In this work, we propose a novel image quantization model named VQGAN-LC (Large Codebook), which extends the codebook size to 100,000, achieving an utilization rate exceeding 99%. Unlike previous methods that optimize each codebook entry, our approach begins with a codebook initialized with 100,000 features extracted by a pre-trained vision encoder. Optimization then focuses on training a projector that aligns the entire codebook with the feature distributions of the encoder in VQGAN-LC. We demonstrate the superior performance of our model over its counterparts across a variety of tasks, including image reconstruction, image classification, auto-regressive image generation using GPT, and image creation with diffusion- and flow-based generative models. Code and models are available at https://github.com/zh460045050/VQGAN-LC.


title: Autoregressive Image Generation without Vector Quantization

abstract: Conventional wisdom holds that autoregressive models for image generation are typically accompanied by vector-quantized tokens. We observe that while a discrete-valued space can facilitate representing a categorical distribution, it is not a necessity for autoregressive modeling. In this work, we propose to model the per-token probability distribution using a diffusion procedure, which allows us to apply autoregressive models in a continuous-valued space. Rather than using categorical cross-entropy loss, we define a Diffusion Loss function to model the per-token probability. This approach eliminates the need for discrete-valued tokenizers. We evaluate its effectiveness across a wide range of cases, including standard autoregressive models and generalized masked autoregressive (MAR) variants. By removing vector quantization, our image generator achieves strong results while enjoying the speed advantage of sequence modeling. We hope this work will motivate the use of autoregressive generation in other continuous-valued domains and applications.


title: mDPO: Conditional Preference Optimization for Multimodal Large Language Models

abstract: Direct preference optimization (DPO) has shown to be an effective method for large language model (LLM) alignment. Recent works have attempted to apply DPO to multimodal scenarios but have found it challenging to achieve consistent improvement. Through a comparative experiment, we identify the unconditional preference problem in multimodal preference optimization, where the model overlooks the image condition. To address this problem, we propose mDPO, a multimodal DPO objective that prevents the over-prioritization of language-only preferences by also optimizing image preference. Moreover, we introduce a reward anchor that forces the reward to be positive for chosen responses, thereby avoiding the decrease in their likelihood -- an intrinsic problem of relative preference optimization. Experiments on two multimodal LLMs of different sizes and three widely used benchmarks demonstrate that mDPO effectively addresses the unconditional preference problem in multimodal preference optimization and significantly improves model performance, particularly in reducing hallucination.


title: LLaNA: Large Language and NeRF Assistant

abstract: Multimodal Large Language Models (MLLMs) have demonstrated an excellent understanding of images and 3D data. However, both modalities have shortcomings in holistically capturing the appearance and geometry of objects. Meanwhile, Neural Radiance Fields (NeRFs), which encode information within the weights of a simple Multi-Layer Perceptron (MLP), have emerged as an increasingly widespread modality that simultaneously encodes the geometry and photorealistic appearance of objects. This paper investigates the feasibility and effectiveness of ingesting NeRF into MLLM. We create LLaNA, the first general-purpose NeRF-language assistant capable of performing new tasks such as NeRF captioning and Q\&A. Notably, our method directly processes the weights of the NeRF's MLP to extract information about the represented objects without the need to render images or materialize 3D data structures. Moreover, we build a dataset of NeRFs with text annotations for various NeRF-language tasks with no human intervention. Based on this dataset, we develop a benchmark to evaluate the NeRF understanding capability of our method. Results show that processing NeRF weights performs favourably against extracting 2D or 3D representations from NeRFs.


title: The algebraic classification of nilpotent bicommutative algebras

abstract: This paper is devoted to the complete algebraic classification of complex 5-dimensional nilpotent bicommutative algebras.


title: The Ethics of AI in Education

abstract: The transition of Artificial Intelligence (AI) from a lab-based science to live human contexts brings into sharp focus many historic, socio-cultural biases, inequalities, and moral dilemmas. Many questions that have been raised regarding the broader ethics of AI are also relevant for AI in Education (AIED). AIED raises further specific challenges related to the impact of its technologies on users, how such technologies might be used to reinforce or alter the way that we learn and teach, and what we, as a society and individuals, value as outcomes of education. This chapter discusses key ethical dimensions of AI and contextualises them within AIED design and engineering practices to draw connections between the AIED systems we build, the questions about human learning and development we ask, the ethics of the pedagogies we use, and the considerations of values that we promote in and through AIED within a wider socio-technical system.


title: Explanation Hacking: The perils of algorithmic recourse

abstract: We argue that the trend toward providing users with feasible and actionable explanations of AI decisions, known as recourse explanations, comes with ethical downsides. Specifically, we argue that recourse explanations face several conceptual pitfalls and can lead to problematic explanation hacking, which undermines their ethical status. As an alternative, we advocate that explanations of AI decisions should aim at understanding.


title: Prompting the E-Brushes: Users as Authors in Generative AI

abstract: Since its introduction in 2022, Generative AI has significantly impacted the art world, from winning state art fairs to creating complex videos from simple prompts. Amid this renaissance, a pivotal issue emerges: should users of Generative AI be recognized as authors eligible for copyright protection? The Copyright Office, in its March 2023 Guidance, argues against this notion. By comparing the prompts to clients' instructions for commissioned art, the Office denies users authorship due to their limited role in the creative process. This Article challenges this viewpoint and advocates for the recognition of Generative AI users who incorporate these tools into their creative endeavors. It argues that the current policy fails to consider the intricate and dynamic interaction between Generative AI users and the models, where users actively influence the output through a process of adjustment, refinement, selection, and arrangement. Rather than dismissing the contributions generated by AI, this Article suggests a simplified and streamlined registration process that acknowledges the role of AI in creation. This approach not only aligns with the constitutional goal of promoting the progress of science and useful arts but also encourages public engagement in the creative process, which contributes to the pool of training data for AI. Moreover, it advocates for a flexible framework that evolves alongside technological advancements while ensuring safety and public interest. In conclusion, by examining text-to-image generators and addressing misconceptions about Generative AI and user interaction, this Article calls for a regulatory framework that adapts to technological developments and safeguards public interests


title: Decoding the Digital Fine Print: Navigating the potholes in Terms of service/ use of GenAI tools against the emerging need for Transparent and Trustworthy Tech Futures

abstract: The research investigates the crucial role of clear and intelligible terms of service in cultivating user trust and facilitating informed decision-making in the context of AI, in specific GenAI. It highlights the obstacles presented by complex legal terminology and detailed fine print, which impede genuine user consent and recourse, particularly during instances of algorithmic malfunctions, hazards, damages, or inequities, while stressing the necessity of employing machine-readable terms for effective service licensing. The increasing reliance on General Artificial Intelligence (GenAI) tools necessitates transparent, comprehensible, and standardized terms of use, which facilitate informed decision-making while fostering trust among stakeholders. Despite recent efforts promoting transparency via system and model cards, existing documentation frequently falls short of providing adequate disclosures, leaving users ill-equipped to evaluate potential risks and harms. To address this gap, this research examines key considerations necessary in terms of use or terms of service for Generative AI tools, drawing insights from multiple studies. Subsequently, this research evaluates whether the terms of use or terms of service of prominent Generative AI tools against the identified considerations. Findings indicate inconsistencies and variability in document quality, signaling a pressing demand for uniformity in disclosure practices. Consequently, this study advocates for robust, enforceable standards ensuring complete and intelligible disclosures prior to the release of GenAI tools, thereby empowering end-users to make well-informed choices and enhancing overall accountability in the field.


title: Lifecycle of a sub-metered tertiary multi-use (GreEn-ER) building's open energy data: from resource mobilisation to data re-usability

abstract: The proliferation of sensors in buildings has given us access to more data than before. To shepherd this rise in data, many open data lifecycles have been proposed over the past decade. However, many of the proposed lifecycles do not reflect the necessary complexity in the built environment. In this paper, we present a new open data lifecycle model: Open Energy Data Lifecycle (OPENDAL). OPENDAL builds on the key themes in more popular lifecycles and looks to extend them by better accounting for the information flows between cycles and the interactions between stakeholders around the data. These elements are included in the lifecycle in a bid to increase the reuse of published datasets. In addition, we apply the lifecycle model to the datasets from the GreEn-ER building, a mixed-use education building in France. Different use cases of these datasets are highlighted and discussed as a way to incentivise the use of data by other individuals.


title: Integrating behavior analysis with machine learning to predict online learning performance: A scientometric review and empirical study

abstract: The interest in predicting online learning performance using ML algorithms has been steadily increasing. We first conducted a scientometric analysis to provide a systematic review of research in this area. The findings show that most existing studies apply the ML methods without considering learning behavior patterns, which may compromise the prediction accuracy and precision of the ML methods. This study proposes an integration framework that blends learning behavior analysis with ML algorithms to enhance the prediction accuracy of students' online learning performance. Specifically, the framework identifies distinct learning patterns among students by employing clustering analysis and implements various ML algorithms to predict performance within each pattern. For demonstration, the integration framework is applied to a real dataset from edX and distinguishes two learning patterns, as in, low autonomy students and motivated students. The results show that the framework yields nearly perfect prediction performance for autonomous students and satisfactory performance for motivated students. Additionally, this study compares the prediction performance of the integration framework to that of directly applying ML methods without learning behavior analysis using comprehensive evaluation metrics. The results consistently demonstrate the superiority of the integration framework over the direct approach, particularly when integrated with the best-performing XGBoosting method. Moreover, the framework significantly improves prediction accuracy for the motivated students and for the worst-performing random forest method. This study also evaluates the importance of various learning behaviors within each pattern using LightGBM with SHAP values. The implications of the integration framework and the results for online education practice and future research are discussed.


title: Model Of Information System Towards Harmonized Industry And Computer Science

abstract: The aim of attending an educational institution is learning, which in turn is sought after for the reason of independence of thoughts, ideologies as well as physical and material independence. This physical and material independence is gotten from working in the industry, that is, being a part of the independent working population of the country. There needs to be a way by which students upon graduation can easily adapt to the real world with necessary skills and knowledge required. This problem has been a challenge in some computer science departments, which after effects known after the student begins to work in an industry. The objectives of this project include: Designing a web based chat application for the industry and computer science department, Develop a web based chat application for the industry and computer science and Evaluate the web based chat application for the industry and computer science department. Waterfall system development lifecycle is used in establishing a system project plan, because it gives an overall list of processes and sub-processes required in developing a system. The descriptive research method applied in this project is documentary analysis of previous articles. The result of the project is the design, software a web-based chat application that aids communication between the industry and the computer science department and the evaluation of the system. The application is able to store this information which can be decided to be used later. Awareness of the software to companies and universities, implementation of the suggestions made by the industry in the computer science curriculum, use of this software in universities across Nigeria and use of this not just in the computer science field but in other field of study


title: What do we know about Computing Education in Africa? A Systematic Review of Computing Education Research Literature

abstract: Noticeably, Africa is underrepresented in the computing education research (CER) community. However, there has been some effort from the researchers in the region to contribute to the growing need for computing for all. To understand the body of works that emerged from the global south region and their area of focus in computing education, we conducted a systematic review of the literature. This research investigates the prominent CER journals and conferences to discern the kind of research that has been published and how much contribution they have made to the growing field. Of the 68 selected studies, 45 papers were from South Africa. The prominent aspect of computing in the literature is programming, which accounts for 43%. We identified open areas for research in the context and discussed the implication of our findings for the development of CER in Africa.


title: Closed-loop Teaching via Demonstrations to Improve Policy Transparency

abstract: Demonstrations are a powerful way of increasing the transparency of AI policies. Though informative demonstrations may be selected a priori through the machine teaching paradigm, student learning may deviate from the preselected curriculum in situ. This paper thus explores augmenting a curriculum with a closed-loop teaching framework inspired by principles from the education literature, such as the zone of proximal development and the testing effect. We utilize tests accordingly to close to the loop and maintain a novel particle filter model of human beliefs throughout the learning process, allowing us to provide demonstrations that are targeted to the human's current understanding in real time. A user study finds that our proposed closed-loop teaching framework reduces the regret in human test responses by 43% over a baseline.


title: GUARD-D-LLM: An LLM-Based Risk Assessment Engine for the Downstream uses of LLMs

abstract: Amidst escalating concerns about the detriments inflicted by AI systems, risk management assumes paramount importance, notably for high-risk applications as demanded by the European Union AI Act. Guidelines provided by ISO and NIST aim to govern AI risk management; however, practical implementations remain scarce in scholarly works. Addressing this void, our research explores risks emanating from downstream uses of large language models (LLMs), synthesizing a taxonomy grounded in earlier research. Building upon this foundation, we introduce a novel LLM-based risk assessment engine (GUARD-D-LLM: Guided Understanding and Assessment for Risk Detection for Downstream use of LLMs) designed to pinpoint and rank threats relevant to specific use cases derived from text-based user inputs. Integrating thirty intelligent agents, this innovative approach identifies bespoke risks, gauges their severity, offers targeted suggestions for mitigation, and facilitates risk-aware development. The paper also documents the limitations of such an approach along with way forward suggestions to augment experts in such risk assessment thereby leveraging GUARD-D-LLM in identifying risks early on and enabling early mitigations. This paper and its associated code serve as a valuable resource for developers seeking to mitigate risks associated with LLM-based applications.


title: Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation

abstract: Amidst the growing interest in developing task-autonomous AI for automated mental health care, this paper addresses the ethical and practical challenges associated with the issue and proposes a structured framework that delineates levels of autonomy, outlines ethical requirements, and defines beneficial default behaviors for AI agents in the context of mental health support. We also evaluate ten state-of-the-art language models using 16 mental health-related questions designed to reflect various mental health conditions, such as psychosis, mania, depression, suicidal thoughts, and homicidal tendencies. The question design and response evaluations were conducted by mental health clinicians (M.D.s). We find that existing language models are insufficient to match the standard provided by human professionals who can navigate nuances and appreciate context. This is due to a range of issues, including overly cautious or sycophantic responses and the absence of necessary safeguards. Alarmingly, we find that most of the tested models could cause harm if accessed in mental health emergencies, failing to protect users and potentially exacerbating existing symptoms. We explore solutions to enhance the safety of current models. Before the release of increasingly task-autonomous AI systems in mental health, it is crucial to ensure that these models can reliably detect and manage symptoms of common psychiatric disorders to prevent harm to users. This involves aligning with the ethical framework and default behaviors outlined in our study. We contend that model developers are responsible for refining their systems per these guidelines to safeguard against the risks posed by current AI technologies to user mental health and safety. Trigger warning: Contains and discusses examples of sensitive mental health topics, including suicide and self-harm.


title: Decoding the Sociotechnical Dimensions of Digital Misinformation: A Comprehensive Literature Review

abstract: This paper presents a systematic literature review in Computer Science that provide an overview of the initiatives related to digital misinformation. This is an exploratory study that covers research from 1993 to 2020, focusing on the investigation of the phenomenon of misinformation. The review consists of 788 studies from SCOPUS, IEEE, and ACM digital libraries, synthesizing the primary research directions and sociotechnical challenges. These challenges are classified into Physical, Empirical, Syntactic, Semantic, Pragmatic, and Social dimensions, drawing from Organizational Semiotics. The mapping identifies issues related to the concept of misinformation, highlights deficiencies in mitigation strategies, discusses challenges in approaching stakeholders, and unveils various sociotechnical aspects relevant to understanding and mitigating the harmful effects of digital misinformation. As contributions, this study present a novel categorization of mitigation strategies, a sociotechnical taxonomy for classifying types of false information and elaborate on the inter-relation of sociotechnical aspects and their impacts.


title: Attributions toward Artificial Agents in a modified Moral Turing Test

abstract: Advances in artificial intelligence (AI) raise important questions about whether people view moral evaluations by AI systems similarly to human-generated moral evaluations. We conducted a modified Moral Turing Test (m-MTT), inspired by Allen and colleagues' (2000) proposal, by asking people to distinguish real human moral evaluations from those made by a popular advanced AI language model: GPT-4. A representative sample of 299 U.S. adults first rated the quality of moral evaluations when blinded to their source. Remarkably, they rated the AI's moral reasoning as superior in quality to humans' along almost all dimensions, including virtuousness, intelligence, and trustworthiness, consistent with passing what Allen and colleagues call the comparative MTT. Next, when tasked with identifying the source of each evaluation (human or computer), people performed significantly above chance levels. Although the AI did not pass this test, this was not because of its inferior moral reasoning but, potentially, its perceived superiority, among other possible explanations. The emergence of language models capable of producing moral responses perceived as superior in quality to humans' raises concerns that people may uncritically accept potentially harmful moral guidance from AI. This possibility highlights the need for safeguards around generative language models in matters of morality.


title: Law and the Emerging Political Economy of Algorithmic Audits

abstract: For almost a decade now, scholarship in and beyond the ACM FAccT community has been focusing on novel and innovative ways and methodologies to audit the functioning of algorithmic systems. Over the years, this research idea and technical project has matured enough to become a regulatory mandate. Today, the Digital Services Act (DSA) and the Online Safety Act (OSA) have established the framework within which technology corporations and (traditional) auditors will develop the `practice' of algorithmic auditing thereby presaging how this `ecosystem' will develop. In this paper, we systematically review the auditing provisions in the DSA and the OSA in light of observations from the emerging industry of algorithmic auditing. Who is likely to occupy this space? What are some political and ethical tensions that are likely to arise? How are the mandates of `independent auditing' or `the evaluation of the societal context of an algorithmic function' likely to play out in practice? By shaping the picture of the emerging political economy of algorithmic auditing, we draw attention to strategies and cultures of traditional auditors that risk eroding important regulatory pillars of the DSA and the OSA. Importantly, we warn that ambitious research ideas and technical projects of/for algorithmic auditing may end up crashed by the standardising grip of traditional auditors and/or diluted within a complex web of (sub-)contractual arrangements, diverse portfolios, and tight timelines.


title: Google's Chrome Antitrust Paradox

abstract: This article delves into Google's dominance of the browser market, highlighting how Google's Chrome browser is playing a critical role in asserting Google's dominance in other markets. While Google perpetuates the perception that Google Chrome is a neutral platform built on open-source technologies, we argue that Chrome is instrumental in Google's strategy to reinforce its dominance in online advertising, publishing, and the browser market itself. Our examination of Google's strategic acquisitions, anti-competitive practices, and the implementation of so-called "privacy controls," shows that Chrome is far from a neutral gateway to the web. Rather, it serves as a key tool for Google to maintain and extend its market power, often to the detriment of competition and innovation. We examine how Chrome not only bolsters Google's position in advertising and publishing through practices such as coercion, and self-preferencing, it also helps leverage its advertising clout to engage in a "pay-to-play" paradigm, which serves as a cornerstone in Google's larger strategy of market control. We also discuss potential regulatory interventions and remedies, drawing on historical antitrust precedents. We propose a triad of solutions motivated from our analysis of Google's abuse of Chrome: behavioral remedies targeting specific anti-competitive practices, structural remedies involving an internal separation of Google's divisions, and divestment of Chrome from Google. Despite Chrome's dominance and its critical role in Google's ecosystem, it has escaped antitrust scrutiny -- a gap our article aims to bridge. Addressing this gap is instrumental to solve current market imbalances and future challenges brought on by increasingly hegemonizing technology firms, ensuring a competitive digital environment that nurtures innovation and safeguards consumer interests.


title: AI Royalties -- an IP Framework to Compensate Artists & IP Holders for AI-Generated Content

abstract: This article investigates how AI-generated content can disrupt central revenue streams of the creative industries, in particular the collection of dividends from intellectual property (IP) rights. It reviews the IP and copyright questions related to the input and output of generative AI systems. A systematic method is proposed to assess whether AI-generated outputs, especially images, infringe previous copyrights, using a similarity metric (CLIP) between images against historical copyright rulings. An examination (economic and technical feasibility) of previously proposed compensation frameworks reveals their financial implications for creatives and IP holders. Lastly, we propose a novel IP framework for compensation of artists and IP holders based on their published "licensed AIs" as a new medium and asset from which to collect AI royalties.


title: Student Perspectives on Using a Large Language Model (LLM) for an Assignment on Professional Ethics

abstract: The advent of Large Language Models (LLMs) started a serious discussion among educators on how LLMs would affect, e.g., curricula, assessments, and students' competencies. Generative AI and LLMs also raised ethical questions and concerns for computing educators and professionals. This experience report presents an assignment within a course on professional competencies, including some related to ethics, that computing master's students need in their careers. For the assignment, student groups discussed the ethical process by Lennerfors et al. by analyzing a case: a fictional researcher considers whether to attend the real CHI 2024 conference in Hawaii. The tasks were (1) to participate in in-class discussions on the case, (2) to use an LLM of their choice as a discussion partner for said case, and (3) to document both discussions, reflecting on their use of the LLM. Students reported positive experiences with the LLM as a way to increase their knowledge and understanding, although some identified limitations. The LLM provided a wider set of options for action in the studied case, including unfeasible ones. The LLM would not select a course of action, so students had to choose themselves, which they saw as coherent. From the educators' perspective, there is a need for more instruction for students using LLMs: some students did not perceive the tools as such but rather as an authoritative knowledge base. Therefore, this work has implications for educators considering the use of LLMs as discussion partners or tools to practice critical thinking, especially in computing ethics education.


title: "Sora is Incredible and Scary": Emerging Governance Challenges of Text-to-Video Generative AI Models

abstract: Text-to-video generative AI models such as Sora OpenAI have the potential to disrupt multiple industries. In this paper, we report a qualitative social media analysis aiming to uncover people's perceived impact of and concerns about Sora's integration. We collected and analyzed comments (N=292) under popular posts about Sora-generated videos, comparison between Sora videos and Midjourney images, and artists' complaints about copyright infringement by Generative AI. We found that people were most concerned about Sora's impact on content creation-related industries. Emerging governance challenges included the for-profit nature of OpenAI, the blurred boundaries between real and fake content, human autonomy, data privacy, copyright issues, and environmental impact. Potential regulatory solutions proposed by people included law-enforced labeling of AI content and AI literacy education for the public. Based on the findings, we discuss the importance of gauging people's tech perceptions early and propose policy recommendations to regulate Sora before its public release.


title: An ethical study of generative AI from the Actor-Network Theory perspective

abstract: The widespread use of Generative Artificial Intelligence in the innovation and generation of communication content is mainly due to its exceptional creative ability, operational efficiency, and compatibility with diverse industries. Nevertheless, this has also sparked ethical problems, such as unauthorized access to data, biased decision-making by algorithms, and criminal use of generated content. In order to tackle the security vulnerabilities linked to Generative Artificial Intelligence, we analyze ChatGPT as a case study within the framework of Actor-Network Theory. We have discovered a total of nine actors, including both human and non-human creatures. We examine the actors and processes of translation involved in the ethical issues related to ChatGPT and analyze the key players involved in the emergence of moral issues. The objective is to explore the origins of the ethical issues that arise with Generative Artificial Intelligence and provide a particular perspective on the governance of Generative Artificial Intelligence.


title: Digital Transformation of Education, Systems Approach and Applied Research

abstract: This article proposes the construction of a systemic model of digital education as part of research applied to public policy (French Ministry of Education). Considering the digital domain in its pervasiveness, it highlights the importance of a complex approach to understanding the transformation of practices. As an applied research modality, we present digital theme groups (GTnum). The methodological approach combines a reflexive posture informed by research contributions, conceptual choices centered on digital humanities and the systems approach, participatory research and open science via the Hypotheses ''Education, digital and research'' notebook. As a result, our modeling is centered on a ''digital environment'' and six units of action put to the test via the GTnum themes. We interpret these results through a comparison with other systemic frameworks, an application to the axes of digital transformation in academies, a prospective reflection with the development of generative AI and perspectives for participatory research. Finally, the article discusses the limits and contributions of this approach: variability in the understanding of the issues at stake and in the integration of research contributions, as well as avenues for anticipating a new digital configuration with the place of AI.


title: Matrix-Free Jacobian Chaining

abstract: The efficient computation of Jacobians represents a fundamental challenge in computational science and engineering. Large-scale modular numerical simulation programs can be regarded as sequences of evaluations of in our case differentiable subprograms with corresponding elemental Jacobians. The latter are typically not available. Tangent and adjoint versions of the individual subprograms are assumed to be given as results of algorithmic differentiation instead. The classical (Jacobian) Matrix Chain Product problem is reformulated in terms of matrix-free Jacobian-matrix (tangents) and matrix-Jacobian products (adjoints), subject to limited memory for storing information required by latter. All numerical results can be reproduced using an open-source reference implementation.


title: The Transformation Risk-Benefit Model of Artificial Intelligence: Balancing Risks and Benefits Through Practical Solutions and Use Cases

abstract: This paper summarizes the most cogent advantages and risks associated with Artificial Intelligence from an in-depth review of the literature. Then the authors synthesize the salient risk-related models currently being used in AI, technology and business-related scenarios. Next, in view of an updated context of AI along with theories and models reviewed and expanded constructs, the writers propose a new framework called "The Transformation Risk-Benefit Model of Artificial Intelligence" to address the increasing fears and levels of AI risk. Using the model characteristics, the article emphasizes practical and innovative solutions where benefits outweigh risks and three use cases in healthcare, climate change/environment and cyber security to illustrate unique interplay of principles, dimensions and processes of this powerful AI transformational model.


title: Gender Differences in Class Participation in Online versus In-Person Core CS Courses

abstract: The COVID-19 pandemic significantly altered how post-secondary students receive their education. Namely, the transition from an in-person to an online class format changed how students interact with their instructors and their classmates. In this paper, we use student participation scores from two core computer science classes across ten in-person and three online quarters at a public research university to analyze whether the shift to primarily asynchronous online learning has impacted the gender gap in student participation scores and students' attitudes towards themselves and their peers. We observe a shift on the online class forum: in in-person classes, males score higher on average and dominate the top scores while in online classes, male and female students participate at approximately the same rate classwide. To understand what might be driving changes in participation behavior, we analyze survey responses from over a quarter of the students enrolled in the online classes. While we find that students of both genders tend to compare themselves to their peers less when classes are online, we also find that this trend is much more accentuated for females than males. This data suggests that observed female participation habits in typical in-person classes are not inherent gender differences, but rather, a product of the environment. Therefore, it is critical the community investigates the root causes of these behavioral differences, and experiments with ways to mitigate them, before we soon return to an in-person format.


title: Artificial Intelligence in Everyday Life 2.0: Educating University Students from Different Majors

abstract: With the surge in data-centric AI and its increasing capabilities, AI applications have become a part of our everyday lives. However, misunderstandings regarding their capabilities, limitations, and associated advantages and disadvantages are widespread. Consequently, in the university setting, there is a crucial need to educate not only computer science majors but also students from various disciplines about AI. In this experience report, we present an overview of an introductory course that we offered to students coming from different majors. Moreover, we discuss the assignments and quizzes of the course, which provided students with a firsthand experience of AI processes and insights into their learning patterns. Additionally, we provide a summary of the course evaluation, as well as students' performance. Finally, we present insights gained from teaching this course and elaborate on our future plans.


title: Toward an Artist-Centred AI

abstract: Awareness about the immense impact that artificial intelligence (AI) might have or already has made on the social, economic, political, and cultural realities of our world has become part of the mainstream public discourse. Attributes such as ethical, responsible, or explainable emerge as associative and descriptive nominal references in guidelines that influence perspectives on AI application and development. This paper contextualizes the notions of suitability and desirability of principles, practices, and tools related to the use of AI in the arts. The result is a framework drafted as a set of atomic attributes that summarize the values of AI deemed important for artistic creativity. It was composed by examining the challenges that AI poses to art production, distribution, consumption, and monetization. Considering the differentiating potentials of AI and taking a perspective aside from the purely technical ontology, we argue that artistically pertinent AI should be unexpected, diversified, affordant, and evolvable.


title: Not as Simple as It Looked: Are We Concluding for Biased Arrest Practices?

abstract: This study examines racial disparities in violent arrest outcomes, challenging conventional methods through a nuanced analysis of Cincinnati Police Department data. Acknowledging the intricate nature of racial disparity, the study categorizes explanations into types of place, types of person, and a combination of both, emphasizing the impact of neighborhood characteristics on crime distribution and police deployment. By introducing alternative scenarios, such as spuriousness, directed policing, and the geo-concentration of racial groups, the study underscores the complexity of racial disparity calculations. Employing a case study approach, the analysis of violent arrest outcomes reveals approximately 40 percent of the observed variation attributed to neighborhood-level characteristics, with concentrated disadvantage neutralizing the influence of race on arrest rates. Contrary to expectations, the study challenges the notion of unintentional racism, suggesting that neighborhood factors play a more significant role than the racial composition in explaining arrests. Policymakers are urged to focus on comprehensive community development initiatives addressing socioeconomic inequalities and support the development of robust racial disparity indices. The study calls for nuanced explorations of unintentional racism and future research addressing potential limitations, aiming to enhance understanding of the complexities surrounding racial disparities in arrests.


title: Ethical Framework for Responsible Foundational Models in Medical Imaging

abstract: Foundational models (FMs) have tremendous potential to revolutionize medical imaging. However, their deployment in real-world clinical settings demands extensive ethical considerations. This paper aims to highlight the ethical concerns related to FMs and propose a framework to guide their responsible development and implementation within medicine. We meticulously examine ethical issues such as privacy of patient data, bias mitigation, algorithmic transparency, explainability and accountability. The proposed framework is designed to prioritize patient welfare, mitigate potential risks, and foster trust in AI-assisted healthcare.


title: Earth ECS using Modified Energy Budget Methods and Trend Analyses v.1SS

abstract: Earth global and regional effective thermal conductance G(eff) (in (W/m^2)/C and often labeled lambda in climate research) and the related Equilibrium Climate Sensitivity (ECS) are evaluated by applying a modified version of the Energy Budget method, and using data only after 1970. By removing Periodic Interfering temperature components (using a novel PIR process) and applying high frequency filtering, an extraordinarily near linear temperature response is revealed, enhancing accurate G(eff) calculation and avoiding the pre-1970 aerosol forcing and ocean energy per area (E*) absorption uncertainties. A formal/empirical method is used to determine more reliable values of Q(t)=d[E*(Ocean.energy)]/dt . Using NOAA data, and after PIR, it is shown that: 1) The Energy Budget Method can be realistically applied to the Ocean and Land regions independently, 2) the effective volcanic forcing is <= 1/5 the IPCC AR5 estimate, 3) the "historical" 1980-2020 ECS(eff) values for Global, global Ocean, and global Land regions values are <= 2.16, 1.69, 2.96 C/2xCO2 respectively; where the updated IPCC AR5 orthodox independent global Forcing value of 0.4 (W/m^2)/Decade and F/2xCO2= 3.7 W/m^2 were used. The Global average ECS(true) value of <= 2.10 C is 70% of the IPCC AR6 ECS estimate of 3.0 C, but 127% of the ECS(eff) value reported by Lewis (1.66 C) . The estimated oceans average TCR/ECS ratio = 0.71 and the global average TCR/ECS ratio = 0.83, and ECS(land)/ECS(Ocean) = 1.78 . [Results using HADCRUT temperature data instead are similar, but 6% "cooler" over land, and 8% "warmer" over oceans.] A simplified physically realistic formal/empirical Coarse 2-D Global Climate Model is derived wherein variation of Geff(t) until equilibrium (i.e. "pattern effects") are proven to be negligible or "cooling", using these Methods. And so it is likely ECS(true) <= ECS(eff).


title: Sisteme Hibride de Invatare Automata si Aplicatii

abstract: In this paper, a deep neural network approach and a neuro-symbolic one are proposed for classification and regression. The neuro-symbolic predictive models based on Logic Tensor Networks are capable of discriminating and in the same time of explaining the characterization of bad connections, called alerts or attacks, and of normal connections. The proposed hybrid systems incorporate both the ability of deep neural networks to improve on their own through experience and the interpretability of the results provided by symbolic artificial intelligence approach. To justify the need for shifting towards hybrid systems, explanation, implementation, and comparison of the dense neural network and the neuro-symbolic network is performed in detail. For the comparison to be relevant, the same datasets were used in training and the metrics resulted have been compared. A review of the resulted metrics shows that while both methods have similar precision in their predictive models, with Logic Tensor Networks being also possible to have interactive accuracy and deductive reasoning over data. Other advantages and disadvantages such as overfitting mitigation and scalability issues are also further discussed.


title: Generative AI Voting: Fair Collective Choice is Resilient to LLM Biases and Inconsistencies

abstract: Scaling up deliberative and voting participation is a longstanding endeavor -- a cornerstone for direct democracy and legitimate collective choice. Recent breakthroughs in generative artificial intelligence (AI) and large language models (LLMs) provide unprecedented opportunities, but also alerting risks for digital democracy. AI personal assistants can overcome cognitive bandwidth limitations of humans, providing decision support capabilities or even direct AI representation of human voters at large scale. However, the quality of this representation and what underlying biases manifest when delegating collective decision making to LLMs is an alarming and timely challenge to tackle. By rigorously emulating with high realism more than >50K LLM voting personas in 81 real-world voting elections, we show that different LLMs (GPT 3, GPT 3.5, and Llama2) come with biases and significant inconsistencies in complex preferential ballot formats, compared to simpler and more consistent majoritarian elections. Strikingly, fair voting aggregation methods, such as equal shares, prove to be a win-win: fairer voting outcomes for humans with fairer AI representation. This novel underlying relationship proves paramount for democratic resilience in progressives scenarios with low voters turnout and voter fatigue supported by AI representatives: abstained voters are mitigated by recovering highly representative voting outcomes that are fairer. These insights provide remarkable foundations for science, policymakers and citizens in explaining and mitigating AI risks in democratic innovations.


title: The EarlyBird Gets the WORM: Heuristically Accelerating EarlyBird Convergence

abstract: The Lottery Ticket hypothesis proposes that ideal sparse subnetworks called lottery tickets exist in the untrained dense network. The Early Bird hypothesis proposes an efficient algorithm to find these winning lottery tickets in convolutional neural networks using the novel concept of distance between subnetworks to detect convergence in the subnetworks of a model. However, this approach overlooks unchanging groups of unimportant neurons near the end of the search. We propose WORM, a method that exploits these static groups by truncating their gradients, forcing the model to rely on other neurons. Experiments show WORM achieves faster ticket identification training and uses fewer FLOPs, despite the additional computational overhead. Additionally WORM pruned models lose less accuracy during pruning and recover accuracy faster, improving the robustness of the model. Furthermore, WORM is also able to generalize the Early Bird hypothesis reasonably well to larger models such as transformers, displaying its flexibility to adapt to various architectures.


title: Logic-Based Explainability: Past, Present & Future

abstract: In recent years, the impact of machine learning (ML) and artificial intelligence (AI) in society has been absolutely remarkable. This impact is expected to continue in the foreseeable future. However,the adoption of AI/ML is also a cause of grave concern. The operation of the most advances AI/ML models is often beyond the grasp of human decision makers. As a result, decisions that impact humans may not be understood and may lack rigorous validation. Explainable AI (XAI) is concerned with providing human decision-makers with understandable explanations for the predictions made by ML models. As a result, XAI is a cornerstone of trustworthy AI. Despite its strategic importance, most work on XAI lacks rigor, and so its use in high-risk or safety-critical domains serves to foster distrust instead of contributing to build much-needed trust. Logic-based XAI has recently emerged as a rigorous alternative to those other non-rigorous methods of XAI. This paper provides a technical survey of logic-based XAI, its origins, the current topics of research, and emerging future topics of research. The paper also highlights the many myths that pervade non-rigorous approaches for XAI.


title: On the theory of pseudo-balayage on a locally compact space and its applications

abstract: Under suitable requirements on a kernel on a locally compact space, we develop a theory of inner (outer) pseudo-balayage of quite general signed Radon measures (not necessarily of finite energy) onto quite general sets (not necessarily closed). Such investigations were initiated in Fuglede's work (Anal. Math., 2016), which was, however, mainly concerned with the outer pseudo-balayage of positive measures of finite energy, whereas treatment of signed measures of infinite energy requires essentially new methods and approaches. To illustrate this, it is enough to note that in the particular case of kernels satisfying the balayage principle, the pseudo-balayage of positive measures coincides with their balayage, whereas for signed measures, these two concepts are drastically different. The theory thereby established enables us to provide necessary and sufficient conditions for the solvability of the inner Gauss variational problem, a point of interest for a number of recent researches in constructive function theory. This study covers many interesting kernels in classical and modern potential theory, which looks promising for further applications.


title: ChatPCG: Large Language Model-Driven Reward Design for Procedural Content Generation

abstract: Driven by the rapid growth of machine learning, recent advances in game artificial intelligence (AI) have significantly impacted productivity across various gaming genres. Reward design plays a pivotal role in training game AI models, wherein researchers implement concepts of specific reward functions. However, despite the presence of AI, the reward design process predominantly remains in the domain of human experts, as it is heavily reliant on their creativity and engineering skills. Therefore, this paper proposes ChatPCG, a large language model (LLM)-driven reward design framework.It leverages human-level insights, coupled with game expertise, to generate rewards tailored to specific game features automatically. Moreover, ChatPCG is integrated with deep reinforcement learning, demonstrating its potential for multiplayer game content generation tasks. The results suggest that the proposed LLM exhibits the capability to comprehend game mechanics and content generation tasks, enabling tailored content generation for a specified game. This study not only highlights the potential for improving accessibility in content generation but also aims to streamline the game AI development process.


title: Edge Quasi $\lambda$-distance-balanced Graphs in Metric Space

abstract: In a graph $A$, the measure $|M_g^A(f)|=m_g^A(f)$ for each arbitrary edge $f=gh$ counts the edges in $A$ closer to $g$ than $h$. $A$ is termed an edge quasi-$\lambda$-distance-balanced graph in a metric space (abbreviated as $EQDBG$), where a rational number ($>1$) is assigned to each edge $f=gh$ such that $m_g^A(f)=\lambda^{\pm1}m_h^A(f)$. This paper introduces and discusses these graph concepts, providing essential examples and construction methods. The study examines how every $EQDBG$ is a bipartite graph and calculates the edge-Szeged index for such graphs. Additionally, it explores their properties in Cartesian and lexicographic products. Lastly, the concept is extended to nicely edge distance-balanced and strongly edge distance-balanced graphs revealing significant outcomes.


title: Solar Power Prediction Using Satellite Data in Different Parts of Nepal

abstract: Due to the unavailability of solar irradiance data for many potential sites of Nepal, the paper proposes predicting solar irradiance based on alternative meteorological parameters. The study focuses on five distinct regions in Nepal and utilizes a dataset spanning almost ten years, obtained from CERES SYN1deg and MERRA-2. Machine learning models such as Random Forest, XGBoost, K-Nearest Neighbors, and deep learning models like LSTM and ANN-MLP are employed and evaluated for their performance. The results indicate high accuracy in predicting solar irradiance, with R-squared(R2) scores close to unity for both train and test datasets. The impact of parameter integration on model performance is analyzed, revealing the significance of various parameters in enhancing predictive accuracy. Each model demonstrates strong performance across all parameters, consistently achieving MAE values below 6, RMSE values under 10, MBE within |2|, and nearly unity R2 values. Upon removal of various solar parameters such as "Solar_Irradiance_Clear_Sky", "UVA", etc. from the datasets, the model's performance is significantly affected. This exclusion leads to considerable increases in MAE, reaching up to 82, RMSE up to 135, and MBE up to |7|. Among the models, KNN displays the weakest performance, with an R2 of 0.7582546. Conversely, ANN exhibits the strongest performance, boasting an R2 value of 0.9245877. Hence, the study concludes that Artificial Neural Network (ANN) performs exceptionally well, showcasing its versatility even under sparse data parameter conditions.


title: The special unitary groups $SU(2n)$ as framed manifolds

abstract: Let $[SU(2n),\mathscr{L}]$ denote the bordism class of $SU(2n)$ $(n\ge 2)$ equipped with the left invariant framing $\mathscr{L}$. Then it is well known that $e_\mathbb{C}([SU(2n), \mathscr{L}])=0$ in $\mathbb{O}/\mathbb{Z}$ where $e_\mathbb{C}$ denotes the complex Adams $e$-invariant. In this note we show that replacing $\mathscr{L}$ by the twisted framing by a specific map it can be transformed into a generator of $\mathrm{Im} \, e_\mathbb{C}$. In addition to that we also show that the same procedure affords an analogous result for a quotient of $SU(2n+1)$ by a circle subgroup which inherits a canonical framing from $SU(2n+1)$ in the usual way.


title: Experimental verification of the optimal fingerprint method for detecting climate change

abstract: The optimal fingerprint method serves as a potent approach for detecting and attributing climate change. However, its experimental validation encounters challenges due to the intricate nature of climate systems. Here, we experimentally examine the optimal fingerprint method simulated by a precisely controlled magnetic resonance system of spins. The spin dynamic under an applied deterministic driving field and a noise field is utilized to emulate the complex climate system with external forcing and internal variability. Our experimental results affirm the theoretical prediction regarding the existence of an optimal detection direction which maximizes the signal-to-noise ratio, thereby validating the optimal fingerprint method. This work offers direct empirical verification of the optimal fingerprint method, crucial for comprehending climate change and its societal impacts.


title: Knowledge Return Oriented Prompting (KROP)

abstract: Many Large Language Models (LLMs) and LLM-powered apps deployed today use some form of prompt filter or alignment to protect their integrity. However, these measures aren't foolproof. This paper introduces KROP, a prompt injection technique capable of obfuscating prompt injection attacks, rendering them virtually undetectable to most of these security measures.


title: A non-autonomous framework for climate change and extreme weather events increase in a stochastic energy balance model

abstract: We develop a three-timescale framework for modelling climate change and introduce a space-heterogeneous one-dimensional energy balance model. This model, addressing temperature fluctuations from rising carbon dioxide levels and the super-greenhouse effect in tropical regions, fits within the setting of stochastic reaction-diffusion equations. Our results show how both mean and variance of temperature increase, without the system going through a bifurcation point. This study aims to advance the conceptual understanding of the extreme weather events frequency increase due to climate change.


title: Applications of Explainable artificial intelligence in Earth system science

abstract: In recent years, artificial intelligence (AI) rapidly accelerated its influence and is expected to promote the development of Earth system science (ESS) if properly harnessed. In application of AI to ESS, a significant hurdle lies in the interpretability conundrum, an inherent problem of black-box nature arising from the complexity of AI algorithms. To address this, explainable AI (XAI) offers a set of powerful tools that make the models more transparent. The purpose of this review is twofold: First, to provide ESS scholars, especially newcomers, with a foundational understanding of XAI, serving as a primer to inspire future research advances; second, to encourage ESS professionals to embrace the benefits of AI, free from preconceived biases due to its lack of interpretability. We begin with elucidating the concept of XAI, along with typical methods. We then delve into a review of XAI applications in the ESS literature, highlighting the important role that XAI has played in facilitating communication with AI model decisions, improving model diagnosis, and uncovering scientific insights. We identify four significant challenges that XAI faces within the ESS, and propose solutions. Furthermore, we provide a comprehensive illustration of multifaceted perspectives. Given the unique challenges in ESS, an interpretable hybrid approach that seamlessly integrates AI with domain-specific knowledge appears to be a promising way to enhance the utility of AI in ESS. A visionary outlook for ESS envisions a harmonious blend where process-based models govern the known, AI models explore the unknown, and XAI bridges the gap by providing explanations.


title: Data Petri Nets meet Probabilistic Programming (Extended version)

abstract: Probabilistic programming (PP) is a programming paradigm that allows for writing statistical models like ordinary programs, performing simulations by running those programs, and analyzing and refining their statistical behavior using powerful inference engines. This paper takes a step towards leveraging PP for reasoning about data-aware processes. To this end, we present a systematic translation of Data Petri Nets (DPNs) into a model written in a PP language whose features are supported by most PP systems. We show that our translation is sound and provides statistical guarantees for simulating DPNs. Furthermore, we discuss how PP can be used for process mining tasks and report on a prototype implementation of our translation. We also discuss further analysis scenarios that could be easily approached based on the proposed translation and available PP tools.


title: Hierarchical Compression of Text-Rich Graphs via Large Language Models

abstract: Text-rich graphs, prevalent in data mining contexts like e-commerce and academic graphs, consist of nodes with textual features linked by various relations. Traditional graph machine learning models, such as Graph Neural Networks (GNNs), excel in encoding the graph structural information, but have limited capability in handling rich text on graph nodes. Large Language Models (LLMs), noted for their superior text understanding abilities, offer a solution for processing the text in graphs but face integration challenges due to their limitation for encoding graph structures and their computational complexities when dealing with extensive text in large neighborhoods of interconnected nodes. This paper introduces ``Hierarchical Compression'' (HiCom), a novel method to align the capabilities of LLMs with the structure of text-rich graphs. HiCom processes text in a node's neighborhood in a structured manner by organizing the extensive textual information into a more manageable hierarchy and compressing node text step by step. Therefore, HiCom not only preserves the contextual richness of the text but also addresses the computational challenges of LLMs, which presents an advancement in integrating the text processing power of LLMs with the structural complexities of text-rich graphs. Empirical results show that HiCom can outperform both GNNs and LLM backbones for node classification on e-commerce and citation graphs. HiCom is especially effective for nodes from a dense region in a graph, where it achieves a 3.48% average performance improvement on five datasets while being more efficient than LLM backbones.


title: A Trifecta of Modelling Tools: Navigating COMPAS, Shark and Bayesian Inference for Binary Black Hole Model Selection

abstract: Gravitational waves (GWs) have revealed surprising properties of binary black hole (BBH) populations, but there is still mystery surrounding how these compact objects evolve. We apply Bayesian inference and an efficient method to calculate the BBH merger rates in the Shark host galaxies, to determine the combination of COMPAS parameters that outputs a population most like the GW sources from the LVK transient catalogue. For our COMPAS models, we calculate the likelihood with and without the dependence on the predicted number of BBH merger events. We find strong correlations between hyper-parameters governing the specific angular momentum (AM) of mass lost during mass transfer, the mass-loss rates of Wolf--Rayet stars via winds and the chemically homogeneous evolution (CHE) formation channel. We conclude that analysing the marginalised and unmarginalised likelihood is a good indicator of whether the population parameters distribution and number of observed events reflect the LVK data. In doing so, we see that the majority of the models preferred in terms of the population-level parameters of the BBHs greatly overpredict the number of events we should have observed to date. Looking at the smaller number of models which perform well with both likelihoods, we find that those with no CHE, AM loss occurring closer to the donor during the first mass-transfer event, and/or higher rates of mass-loss from Wolf--Rayet winds are generally preferred by current data. We find these conclusions to be robust to our choice of selection criteria.


title: Financial Assets Dependency Prediction Utilizing Spatiotemporal Patterns

abstract: Financial assets exhibit complex dependency structures, which are crucial for investors to create diversified portfolios to mitigate risk in volatile financial markets. To explore the financial asset dependencies dynamics, we propose a novel approach that models the dependencies of assets as an Asset Dependency Matrix (ADM) and treats the ADM sequences as image sequences. This allows us to leverage deep learning-based video prediction methods to capture the spatiotemporal dependencies among assets. However, unlike images where neighboring pixels exhibit explicit spatiotemporal dependencies due to the natural continuity of object movements, assets in ADM do not have a natural order. This poses challenges to organizing the relational assets to reveal better the spatiotemporal dependencies among neighboring assets for ADM forecasting. To tackle the challenges, we propose the Asset Dependency Neural Network (ADNN), which employs the Convolutional Long Short-Term Memory (ConvLSTM) network, a highly successful method for video prediction. ADNN can employ static and dynamic transformation functions to optimize the representations of the ADM. Through extensive experiments, we demonstrate that our proposed framework consistently outperforms the baselines in the ADM prediction and downstream application tasks. This research contributes to understanding and predicting asset dependencies, offering valuable insights for financial market participants.