title: On the Design and Study of an Installation for Office Workers to Amplify Temporal Diversity and Connection to Nature

abstract: We present the design and user study of an installation for office workers, enabling moments of temporal diversity and connection to nature. The installation is a form of creative computing experience that departs from the traditional focus on office technologies for productivity. Drawing on neuroscience insights and the slowing effect of nature sounds on time perception, we created an immersive, slow interaction, generative AI installation that composes an audiovisual space - serving as a perceptual portal into temporal realms beyond the linear rhythm of the office. Our study investigates the lived experiences of 18 office workers, gathered via explicitation interviews, observational notes, and video recordings, analysed through an inductive thematic analysis. Key findings highlight the ephemeral qualities in creative computing experiences using generative AI, its potential to foster contemplative practices, amplify ecological temporalities, and reshape office workers' engagement with their environment. Our design and user study offer research and practical implications for utilising creative computing to enrich office experiences.


title: CDM-MPC: An Integrated Dynamic Planning and Control Framework for Bipedal Robots Jumping

abstract: Performing acrobatic maneuvers like dynamic jumping in bipedal robots presents significant challenges in terms of actuation, motion planning, and control. Traditional approaches to these tasks often simplify dynamics to enhance computational efficiency, potentially overlooking critical factors such as the control of centroidal angular momentum (CAM) and the variability of centroidal composite rigid body inertia (CCRBI). This paper introduces a novel integrated dynamic planning and control framework, termed centroidal dynamics model-based model predictive control (CDM-MPC), designed for robust jumping control that fully considers centroidal momentum and non-constant CCRBI. The framework comprises an optimization-based kinodynamic motion planner and an MPC controller for real-time trajectory tracking and replanning. Additionally, a centroidal momentum-based inverse kinematics (IK) solver and a landing heuristic controller are developed to ensure stability during high-impact landings. The efficacy of the CDM-MPC framework is validated through extensive testing on the full-sized humanoid robot KUAVO in both simulations and experiments.


title: On critical double phase problems in $\mathbb{R}^N$ involving variable exponents

abstract: We establish a Lions-type concentration-compactness principle and its variant at infinity for Musielak-Orlicz-Sobolev spaces associated with a double phase operator with variable exponents. Based on these principles, we demonstrate the existence and concentration of solutions for a class of critical double phase equations of Schr\"odinger type in $\mathbb{R}^N$ involving variable exponents with various types of potentials. Our growth condition is more appropriately suited compared to the existing works.


title: Exploring Ordinality in Text Classification: A Comparative Study of Explicit and Implicit Techniques

abstract: Ordinal Classification (OC) is a widely encountered challenge in Natural Language Processing (NLP), with applications in various domains such as sentiment analysis, rating prediction, and more. Previous approaches to tackle OC have primarily focused on modifying existing or creating novel loss functions that \textbf{explicitly} account for the ordinal nature of labels. However, with the advent of Pretrained Language Models (PLMs), it became possible to tackle ordinality through the \textbf{implicit} semantics of the labels as well. This paper provides a comprehensive theoretical and empirical examination of both these approaches. Furthermore, we also offer strategic recommendations regarding the most effective approach to adopt based on specific settings.


title: Active Exploration for Real-Time Haptic Training

abstract: Tactile perception is important for robotic systems that interact with the world through touch. Touch is an active sense in which tactile measurements depend on the contact properties of an interaction--e.g., velocity, force, acceleration--as well as properties of the sensor and object under test. These dependencies make training tactile perceptual models challenging. Additionally, the effects of limited sensor life and the near-field nature of tactile sensors preclude the practical collection of exhaustive data sets even for fairly simple objects. Active learning provides a mechanism for focusing on only the most informative aspects of an object during data collection. Here we employ an active learning approach that uses a data-driven model's entropy as an uncertainty measure and explore relative to that entropy conditioned on the sensor state variables. Using a coverage-based ergodic controller, we train perceptual models in near-real time. We demonstrate our approach using a biomimentic sensor, exploring "tactile scenes" composed of shapes, textures, and objects. Each learned representation provides a perceptual sensor model for a particular tactile scene. Models trained on actively collected data outperform their randomly collected counterparts in real-time training tests. Additionally, we find that the resulting network entropy maps can be used to identify high salience portions of a tactile scene.


title: AGN energetics and lifetimes from remnant radio galaxies

abstract: The energy coupling efficiency of active galactic nucleus (AGN) outbursts is known to differ significantly with factors including the jet kinetic power, duration of the outburst, and properties of the host galaxy cluster. As such, constraints on their jet power and lifetime functions are crucial to quantify the role of kinetic-mode AGN feedback on the evolution of galaxies since $z \sim 1$. In this work, we address this issue by measuring the energetics of a sample of 79 low-redshift (0.02 $< z <$ 0.2) remnant radio galaxies compiled from large-sky radio surveys - these objects uniquely probe the full duration of an AGN outburst. The jet kinetic power and outburst duration of each remnant are determined using the RAiSE dynamical model based on the surface brightness distribution observed in multi-frequency radio images. We compare the energetics constrained for this sample to those predicted for mock radio source populations - with various intrinsic functions for jet power and lifetime distributions - to correct for sample selection biases imposed on our sample. The intrinsic jet power and lifetime functions that yield a selection-biased mock population most similar to our observed sample are found using Bayesian inference. Our analysis places robust constraints on assumed power-law indices for the intrinsic jet power and lifetime functions: $p(Q)\propto Q^{-1.49\pm0.07}$ and $p(t_{\mathrm{on}})\propto t_{\mathrm{on}}^{-0.97\pm0.12}$ respectively. We discuss the implication of these findings for feedback-regulated accretion and the self-regulating nature of jet activity. The methodology proposed in this work can be extended to active radio galaxies in future studies.


title: Efficient Multi-agent Reinforcement Learning by Planning

abstract: Multi-agent reinforcement learning (MARL) algorithms have accomplished remarkable breakthroughs in solving large-scale decision-making tasks. Nonetheless, most existing MARL algorithms are model-free, limiting sample efficiency and hindering their applicability in more challenging scenarios. In contrast, model-based reinforcement learning (MBRL), particularly algorithms integrating planning, such as MuZero, has demonstrated superhuman performance with limited data in many tasks. Hence, we aim to boost the sample efficiency of MARL by adopting model-based approaches. However, incorporating planning and search methods into multi-agent systems poses significant challenges. The expansive action space of multi-agent systems often necessitates leveraging the nearly-independent property of agents to accelerate learning. To tackle this issue, we propose the MAZero algorithm, which combines a centralized model with Monte Carlo Tree Search (MCTS) for policy search. We design a novel network structure to facilitate distributed execution and parameter sharing. To enhance search efficiency in deterministic environments with sizable action spaces, we introduce two novel techniques: Optimistic Search Lambda (OS($\lambda$)) and Advantage-Weighted Policy Optimization (AWPO). Extensive experiments on the SMAC benchmark demonstrate that MAZero outperforms model-free approaches in terms of sample efficiency and provides comparable or better performance than existing model-based methods in terms of both sample and computational efficiency. Our code is available at https://github.com/liuqh16/MAZero.


title: Tunnelling amplitudes through localised external potentials from Feynman diagram summation

abstract: Currently there is no general theory of quantum tunnelling of a particle through a potential barrier which is compatible with QFT. We present a complete calculation of tunnelling amplitudes for a scalar field for some simple potentials using quantum field-theoretic methods. Using the perturbative S-matrix formalism, starting with the Klein-Gordon Lagrangian, we show that an infinite summation of Feynman diagrams can recover tunnelling amplitudes consistent with relativistic quantum mechanics. While this work does not include many-particle effects arising from a fully quantised QFT, it is necessary to investigate QFT corrections to tunnelling amplitudes.


title: General bounds on the quality of Bayesian coresets

abstract: Bayesian coresets speed up posterior inference in the large-scale data regime by approximating the full-data log-likelihood function with a surrogate log-likelihood based on a small, weighted subset of the data. But while Bayesian coresets and methods for construction are applicable in a wide range of models, existing theoretical analysis of the posterior inferential error incurred by coreset approximations only apply in restrictive settings -- i.e., exponential family models, or models with strong log-concavity and smoothness assumptions. This work presents general upper and lower bounds on the Kullback-Leibler (KL) divergence of coreset approximations that reflect the full range of applicability of Bayesian coresets. The lower bounds require only mild model assumptions typical of Bayesian asymptotic analyses, while the upper bounds require the log-likelihood functions to satisfy a generalized subexponentiality criterion that is weaker than conditions used in earlier work. The lower bounds are applied to obtain fundamental limitations on the quality of coreset approximations, and to provide a theoretical explanation for the previously-observed poor empirical performance of importance sampling-based construction methods. The upper bounds are used to analyze the performance of recent subsample-optimize methods. The flexibility of the theory is demonstrated in validation experiments involving multimodal, unidentifiable, heavy-tailed Bayesian posterior distributions.


title: Structural Nested Mean Models Under Parallel Trends with Interference

abstract: Despite the common occurrence of interference in Difference-in-Differences (DiD) applications, standard DiD methods rely on an assumption that interference is absent, and comparatively little work has considered how to accommodate and learn about spillover effects within a DiD framework. Here, we extend the so-called `DiD-SNMMs' of Shahn et al (2022) to accommodate interference in a time-varying DiD setting. Doing so enables estimation of a richer set of effects than previous DiD approaches. For example, DiD-SNMMs do not assume the absence of spillover effects after direct exposures and can model how effects of direct or indirect (i.e. spillover) exposures depend on past and concurrent (direct or indirect) exposure and covariate history. We consider both cluster and network interference structures an illustrate the methodology in simulations.


title: Formulation and evaluation of ocean dynamics problems as optimization problems for quantum annealing machines

abstract: Recent advancements in quantum computing suggest the potential to revolutionize computational algorithms across various scientific domains including oceanography and atmospheric science. The field is still relatively young and quantum computation is so different from classical computation that suitable frameworks to represent oceanic and atmospheric dynamics are yet to be explored. Quantum annealing, one of the major paradigms, focuses on combinatorial optimization tasks. In this paper, we solve the classical Stommel problem by quantum annealing (QA) and simulated annealing (SA), a classical counterpart of quantum annealing. We cast the linear partial differential equation into an optimization problem by the least-squares method and discretize the cost function in two ways: finite difference and truncated basis expansion. In either case, SA successfully reproduces the expected solution when appropriate parameters are chosen, demonstrating that annealing has the potential. In contrast, QA using the D-Wave quantum annealing machine fails to obtain good solutions for some cases owing to hardware limitations; in particular, the highly limited connectivity graph of the machine limits the size of the solvable problems, at least under currently available algorithms. Either expanding the connectivity graph or improving the graph embedding algorithms would probably be necessary for quantum annealing machines to be usable for oceanic and atmospheric dynamics problems. While this finding emphasizes the need for hardware improvements and enhancements in graph embedding algorithms for practical applications of quantum annealers, the results from simulated annealing suggest its potential to address practical geophysical dynamics problems. As quantum calculation continues to evolve, addressing these challenges may lead to transformative advancements in ocean and atmosphere modeling.


title: Inverse Design of Metal-Organic Frameworks Using Quantum Natural Language Processing

abstract: In this study, we explore the potential of using quantum natural language processing (QNLP) to inverse design metal-organic frameworks (MOFs) with targeted properties. Specifically, by analyzing 150 hypothetical MOF structures consisting of 10 metal nodes and 15 organic ligands, we categorize these structures into four distinct classes for pore volume and $H_{2}$ uptake values. We then compare various QNLP models (i.e. the bag-of-words, DisCoCat (Distributional Compositional Categorical), and sequence-based models) to identify the most effective approach to process the MOF dataset. Using a classical simulator provided by the IBM Qiskit, the bag-of-words model is identified to be the optimum model, achieving validation accuracies of 85.7% and 86.7% for binary classification tasks on pore volume and $H_{2}$ uptake, respectively. Further, we developed multi-class classification models tailored to the probabilistic nature of quantum circuits, with average test accuracies of 88.4% and 80.7% across different classes for pore volume and $H_{2}$ uptake datasets. Finally, the performance of generating MOF with target properties showed accuracies of 93.5% for pore volume and 89% for $H_{2}$ uptake, respectively. Although our investigation covers only a fraction of the vast MOF search space, it marks a promising first step towards using quantum computing for materials design, offering a new perspective through which to explore the complex landscape of MOFs.


title: Reward-Punishment Reinforcement Learning with Maximum Entropy

abstract: We introduce the ``soft Deep MaxPain'' (softDMP) algorithm, which integrates the optimization of long-term policy entropy into reward-punishment reinforcement learning objectives. Our motivation is to facilitate a smoother variation of operators utilized in the updating of action values beyond traditional ``max'' and ``min'' operators, where the goal is enhancing sample efficiency and robustness. We also address two unresolved issues from the previous Deep MaxPain method. Firstly, we investigate how the negated (``flipped'') pain-seeking sub-policy, derived from the punishment action value, collaborates with the ``min'' operator to effectively learn the punishment module and how softDMP's smooth learning operator provides insights into the ``flipping'' trick. Secondly, we tackle the challenge of data collection for learning the punishment module to mitigate inconsistencies arising from the involvement of the ``flipped'' sub-policy (pain-avoidance sub-policy) in the unified behavior policy. We empirically explore the first issue in two discrete Markov Decision Process (MDP) environments, elucidating the crucial advancements of the DMP approach and the necessity for soft treatments on the hard operators. For the second issue, we propose a probabilistic classifier based on the ratio of the pain-seeking sub-policy to the sum of the pain-seeking and goal-reaching sub-policies. This classifier assigns roll-outs to separate replay buffers for updating reward and punishment action-value functions, respectively. Our framework demonstrates superior performance in Turtlebot 3's maze navigation tasks under the ROS Gazebo simulation.


title: Guided Multi-objective Generative AI to Enhance Structure-based Drug Design

abstract: Generative AI has the potential to revolutionize drug discovery. Yet, despite recent advances in machine learning, existing models cannot generate molecules that satisfy all desired physicochemical properties. Herein, we describe IDOLpro, a novel generative chemistry AI combining deep diffusion with multi-objective optimization for structure-based drug design. The latent variables of the diffusion model are guided by differentiable scoring functions to explore uncharted chemical space and generate novel ligands in silico, optimizing a plurality of target physicochemical properties. We demonstrate its effectiveness by generating ligands with optimized binding affinity and synthetic accessibility on two benchmark sets. IDOLpro produces ligands with binding affinities over 10% higher than the next best state-of-the-art on each test set. On a test set of experimental complexes, IDOLpro is the first to surpass the performance of experimentally observed ligands. IDOLpro can accommodate other scoring functions (e.g. ADME-Tox) to accelerate hit-finding, hit-to-lead, and lead optimization for drug discovery.


title: Graviton-photon conversions in Euler-Heisenberg nonlinear electrodynamics

abstract: We study graviton-photon conversions in an environment of the uniform and constant magnetic field considering Euler-K\"ockel-Heisenberg-type nonlinear corrections in electrodynamics. We take the transverse-tracefree gauge for gravitons and employ both the field potential and the electric and magnetic (EM) fields for photons. The nonlinear correction causes parity violating (chiral) graviton propagation equations, which depend on the two treatments for photons. Thus, the medium becomes effectively birefringent for two polarizations of gravitational waves similar to the photon birefringence a characteristic of the nonlinear correction. In the presence of gravity, due to the nontrivial relation between the potential and EM fields, it is important to present the result using the EM fields.


title: Enhanced dissipation and stability of Poiseuille flow for two-dimensional Boussinesq system

abstract: We investigate the nonlinear stability problem for the two-dimensional Boussinesq system around the Poiseuille flow in a finite channel. The system has the characteristic of Navier-slip boundary condition for the velocity and Dirichlet boundary condition for the temperature, with a small viscosity $\nu$ and small thermal diffusion $\mu,$ respectively. More precisely, we prove that if the initial velocity and initial temperature satisfies$$||u_{0}-(1-y^2,0) ||_{H^{\frac{7}{2}+}}\leq c_0\min\left\lbrace \mu,\nu\right\rbrace ^{\frac{2}{3}}$$ and $$ ||\theta_{0}||_{H^1}+|||D_x|^{\frac{1}{8}}\theta_{0}||_{H^1}\leq c_1\min\left\lbrace \mu,\nu\right\rbrace ^{{\frac{31}{24}}}$$ for some small constants $c_0$ and $c_1$ which are both independent of $\mu,\nu$, then we can reach the conclusion that the velocity remains within $O\left( \min\left\lbrace \mu,\nu\right\rbrace ^{\frac{2}{3}}\right) $ of the Poiseuille flow; the temperature remains $O\left( \min\left\lbrace \mu,\nu\right\rbrace ^{\frac{31}{24}}\right) $ of the constant 0, and approaches to 0 as $t\rightarrow\infty.$


title: TinyLLaVA Factory: A Modularized Codebase for Small-scale Large Multimodal Models

abstract: We present TinyLLaVA Factory, an open-source modular codebase for small-scale large multimodal models (LMMs) with a focus on simplicity of code implementations, extensibility of new features, and reproducibility of training results. Following the design philosophy of the factory pattern in software engineering, TinyLLaVA Factory modularizes the entire system into interchangeable components, with each component integrating a suite of cutting-edge models and methods, meanwhile leaving room for extensions to more features. In addition to allowing users to customize their own LMMs, TinyLLaVA Factory provides popular training recipes to let users pretrain and finetune their models with less coding effort. Empirical experiments validate the effectiveness of our codebase. The goal of TinyLLaVA Factory is to assist researchers and practitioners in exploring the wide landscape of designing and training small-scale LMMs with affordable computational resources.


title: The Grandparent Scam: A Systems Perspective Case Study On Elder Fraud And The Concept Of Human Layering

abstract: In April 2024, an 81-year-old Ohio man was charged with murder, assault, and kidnapping. The man believed that he was protecting his family from scammers threatening harm. What he did not realize was that the 61-year-old Uber driver he killed, was also a victim of the same scammers. This case study examines some common variants of the Grandparent Scam from a systems perspective and how weaponization of conscience is used in these scams. Additionally, this study examines the parallels between layering in money laundering and human layering in the execution of these scams.


title: Graviton physics: Quantum field theory of gravitons, graviton noise and gravitational decoherence -- a concise tutorial

abstract: The detection of gravitational waves in 2015 ushered in a new era of gravitational wave astronomy capable of probing into the strong field dynamics of black holes and neutron stars. It has opened up an exciting new window for laboratory and space tests of Einstein's theory of classical general relativity. In recent years there are two interesting proposals aimed at revealing the quantum natures of perturbative gravity: 1) theoretical predictions in how graviton noise from the early universe after the vacuum of the gravitational field was strongly squeezed by inflationary expansion; 2) experimental proposals using the quantum entanglement between two masses each in a superposition state. The first proposal invokes the stochastic properties of quantum fields, the second invokes a key concept of quantum information. An equally basic and interesting idea is to ask whether and how gravity might be responsible for a quantum system becoming classical in appearance, known as gravitational decoherence. Decoherence due to gravity is of special interest because gravity is universal. This is an important issue in macroscopic quantum phenomena. To fully appreciate these exciting developments requires a working knowledge in classical GR, QF theory and QI plus some familiarity with stochastic processes, namely, noise in quantum fields. Traditionally a new researcher may be conversant in one or two of these four subjects: GR, QFT, QI, SP, depending on his/her background. This tutorial attempts to provide the necessary connections between them, helping an engaging reader from any one of these four subjects to leapfrog to the frontier of these interdisciplinary research topics. Here we shall treat the three topics listed in the title, save gravitational entanglement, because its nature and implications proclaimed in relation to quantum gravity still contain many controversial elements.


title: CaseGNN++: Graph Contrastive Learning for Legal Case Retrieval with Graph Augmentation

abstract: Legal case retrieval (LCR) is a specialised information retrieval task that aims to find relevant cases to a given query case. LCR holds pivotal significance in facilitating legal practitioners in finding precedents. Most of existing LCR methods are based on traditional lexical models and language models, which have gained promising performance in retrieval. However, the domain-specific structural information inherent in legal documents is yet to be exploited to further improve the performance. Our previous work CaseGNN successfully harnesses text-attributed graphs and graph neural networks to address the problem of legal structural information neglect. Nonetheless, there remain two aspects for further investigation: (1) The underutilization of rich edge information within text-attributed case graphs limits CaseGNN to generate informative case representation. (2) The inadequacy of labelled data in legal datasets hinders the training of CaseGNN model. In this paper, CaseGNN++, which is extended from CaseGNN, is proposed to simultaneously leverage the edge information and additional label data to discover the latent potential of LCR models. Specifically, an edge feature-based graph attention layer (EUGAT) is proposed to comprehensively update node and edge features during graph modelling, resulting in a full utilisation of structural information of legal cases. Moreover, a novel graph contrastive learning objective with graph augmentation is developed in CaseGNN++ to provide additional training signals, thereby enhancing the legal comprehension capabilities of CaseGNN++ model. Extensive experiments on two benchmark datasets from COLIEE 2022 and COLIEE 2023 demonstrate that CaseGNN++ not only significantly improves CaseGNN but also achieves supreme performance compared to state-of-the-art LCR methods. Code has been released on https://github.com/yanran-tang/CaseGNN.


title: Source Localization by Multidimensional Steered Response Power Mapping with Sparse Bayesian Learning

abstract: We propose an advance Steered Response Power (SRP) method for localizing multiple sources. While conventional SRP performs well in adverse conditions, it remains to struggle in scenarios with closely neighboring sources, resulting in ambiguous SRP maps. We address this issue by applying sparsity optimization in SRP to obtain high-resolution maps. Our approach represents SRP maps as multidimensional matrices to preserve time-frequency information and further improve performance in unfavorable conditions. We use multi-dictionary Sparse Bayesian Learning to localize sources without needing prior knowledge of their quantity. We validate our method through practical experiments with a 16-channel planar microphone array and compare against three other SRP and sparsity-based methods. Our multidimensional SRP approach outperforms conventional SRP and the current state-of-the-art sparse SRP methods for localizing closely spaced sources in a reverberant room.


title: MM-Retinal: Knowledge-Enhanced Foundational Pretraining with Fundus Image-Text Expertise

abstract: Current fundus image analysis models are predominantly built for specific tasks relying on individual datasets. The learning process is usually based on data-driven paradigm without prior knowledge, resulting in poor transferability and generalizability. To address this issue, we propose MM-Retinal, a multi-modal dataset that encompasses high-quality image-text pairs collected from professional fundus diagram books. Moreover, enabled by MM-Retinal, we present a novel Knowledge-enhanced foundational pretraining model which incorporates Fundus Image-Text expertise, called KeepFIT. It is designed with image similarity-guided text revision and mixed training strategy to infuse expert knowledge. Our proposed fundus foundation model achieves state-of-the-art performance across six unseen downstream tasks and holds excellent generalization ability in zero-shot and few-shot scenarios. MM-Retinal and KeepFIT are available at https://github.com/lxirich/MM-Retinal.


title: ViViD: Video Virtual Try-on using Diffusion Models

abstract: Video virtual try-on aims to transfer a clothing item onto the video of a target person. Directly applying the technique of image-based try-on to the video domain in a frame-wise manner will cause temporal-inconsistent outcomes while previous video-based try-on solutions can only generate low visual quality and blurring results. In this work, we present ViViD, a novel framework employing powerful diffusion models to tackle the task of video virtual try-on. Specifically, we design the Garment Encoder to extract fine-grained clothing semantic features, guiding the model to capture garment details and inject them into the target video through the proposed attention feature fusion mechanism. To ensure spatial-temporal consistency, we introduce a lightweight Pose Encoder to encode pose signals, enabling the model to learn the interactions between clothing and human posture and insert hierarchical Temporal Modules into the text-to-image stable diffusion model for more coherent and lifelike video synthesis. Furthermore, we collect a new dataset, which is the largest, with the most diverse types of garments and the highest resolution for the task of video virtual try-on to date. Extensive experiments demonstrate that our approach is able to yield satisfactory video try-on results. The dataset, codes, and weights will be publicly available. Project page: https://becauseimbatman0.github.io/ViViD.


title: Application of time-series quantum generative model to financial data

abstract: Despite proposing a quantum generative model for time series that successfully learns correlated series with multiple Brownian motions, the model has not been adapted and evaluated for financial problems. In this study, a time-series generative model was applied as a quantum generative model to actual financial data. Future data for two correlated time series were generated and compared with classical methods such as long short-term memory and vector autoregression. Furthermore, numerical experiments were performed to complete missing values. Based on the results, we evaluated the practical applications of the time-series quantum generation model. It was observed that fewer parameter values were required compared with the classical method. In addition, the quantum time-series generation model was feasible for both stationary and nonstationary data. These results suggest that several parameters can be applied to various types of time-series data.


title: Constructing vortex functions and basis states of Chern insulators: ideal condition, inequality from index theorem, and coherent-like states on von Neumann lattice

abstract: In the field of fractional Chern insulators, a great deal of effort has been devoted to characterizing Chern bands that exhibit properties similar to the Landau levels. Among them, the concept of the vortex function, which generalizes the complex coordinate used for the symmetric-gauge Landau-level basis, allows for a concise description. In this paper, we develop a theory of constructing the vortex function and basis states of Chern insulators in the tight-binding formalism. In the first half, we consider the optimization process of the vortex function, which minimizes an indicator that measures the difference from the ideal Chern insulators. In particular, we focus on the sublattice position dependence of the vortex function or the quantum geometric tensor. This degree of freedom serves as a discrete analog of the non-uniformity in the spatial metric and magnetic field in a continuous model. In the second half, we construct two types of basis sets for a given vortex function: radially localized basis set and coherent-like basis set. The former basis set is defined as the eigenstates of an analogy of the angular momentum operator. Remarkably, one can always find exact zero mode(s) for this operator, which is explained by the celebrated Atiyah-Singer index theorem. As a byproduct, we propose an inequality rooted in the band topology. We also discuss the subtle differences between our formalism and the previous works about the momentum-space Landau level. The latter basis set generalizes the concept of coherent states on von Neumann lattice. While this basis set is not orthogonal, it is useful to compare the LLL and the given Chern insulator directly in the Brillouin zone. These basis sets are expected to be useful for many-body calculations of fractional Chern insulators.

categories: cond-mat.mes-hall cond-mat.mtrl-sci cond-mat.quant-gas cond-mat.str-el math-ph math.MP


title: Minimal frequently stable is almost automorphic

abstract: We show that a minimal toplogical dynamical system that is frequently stable if and only if it is almost automorphic.


title: Self-Supervised Learning of Visual Servoing for Low-Rigidity Robots Considering Temporal Body Changes

abstract: In this study, we investigate object grasping by visual servoing in a low-rigidity robot. It is difficult for a low-rigidity robot to handle its own body as intended compared to a rigid robot, and calibration between vision and body takes some time. In addition, the robot must constantly adapt to changes in its body, such as the change in camera position and change in joints due to aging. Therefore, we develop a method for a low-rigidity robot to autonomously learn visual servoing of its body. We also develop a mechanism that can adaptively change its visual servoing according to temporal body changes. We apply our method to a low-rigidity 6-axis arm, MyCobot, and confirm its effectiveness by conducting object grasping experiments based on visual servoing.


title: Regularized Entanglement Entropy of Electron-Positron Scattering with a Witness Photon

abstract: Regularized quantum information metrics are calculated for the scattering process $e^-e^+ \rightarrow \gamma,Z\rightarrow \mu^-\mu^+$ that has a witness photon entangled with the initial electron-positron state. Unitarity implies the correct regularization of divergences that appear in both the final density matrix and von Neumann entanglement entropies. The entropies are found to quantify uncertainty or randomness. The variation of information, entanglement entropy, and correlation between the muon's and witness photon's helicities are found to convey equivalent information. The magnitude of the muon's expected helicity rises (falls) as the helicity entropy falls (rises). Area, or the scattering cross section, is a source of entropy for the muon's helicity entropy and momentum entropy. The muon's differential angular entropy distribution is similar to the differential angular cross section distribution, capturing the forward-backward asymmetry at high center of mass energies.


title: Generative AI in Higher Education: A Global Perspective of Institutional Adoption Policies and Guidelines

abstract: Integrating generative AI (GAI) into higher education is crucial for preparing a future generation of GAI-literate students. Yet a thorough understanding of the global institutional adoption policy remains absent, with most of the prior studies focused on the Global North and the promises and challenges of GAI, lacking a theoretical lens. This study utilizes the Diffusion of Innovations Theory to examine GAI adoption strategies in higher education across 40 universities from six global regions. It explores the characteristics of GAI innovation, including compatibility, trialability, and observability, and analyses the communication channels and roles and responsibilities outlined in university policies and guidelines. The findings reveal a proactive approach by universities towards GAI integration, emphasizing academic integrity, teaching and learning enhancement, and equity. Despite a cautious yet optimistic stance, a comprehensive policy framework is needed to evaluate the impacts of GAI integration and establish effective communication strategies that foster broader stakeholder engagement. The study highlights the importance of clear roles and responsibilities among faculty, students, and administrators for successful GAI integration, supporting a collaborative model for navigating the complexities of GAI in education. This study contributes insights for policymakers in crafting detailed strategies for its integration.


title: LSEnet: Lorentz Structural Entropy Neural Network for Deep Graph Clustering

abstract: Graph clustering is a fundamental problem in machine learning. Deep learning methods achieve the state-of-the-art results in recent years, but they still cannot work without predefined cluster numbers. Such limitation motivates us to pose a more challenging problem of graph clustering with unknown cluster number. We propose to address this problem from a fresh perspective of graph information theory (i.e., structural information). In the literature, structural information has not yet been introduced to deep clustering, and its classic definition falls short of discrete formulation and modeling node features. In this work, we first formulate a differentiable structural information (DSI) in the continuous realm, accompanied by several theoretical results. By minimizing DSI, we construct the optimal partitioning tree where densely connected nodes in the graph tend to have the same assignment, revealing the cluster structure. DSI is also theoretically presented as a new graph clustering objective, not requiring the predefined cluster number. Furthermore, we design a neural LSEnet in the Lorentz model of hyperbolic space, where we integrate node features to structural information via manifold-valued graph convolution. Extensive empirical results on real graphs show the superiority of our approach.


title: Counterfactual Explanation-Based Badminton Motion Guidance Generation Using Wearable Sensors

abstract: This study proposes a framework for enhancing the stroke quality of badminton players by generating personalized motion guides, utilizing a multimodal wearable dataset. These guides are based on counterfactual algorithms and aim to reduce the performance gap between novice and expert players. Our approach provides joint-level guidance through visualizable data to assist players in improving their movements without requiring expert knowledge. The method was evaluated against a traditional algorithm using metrics to assess validity, proximity, and plausibility, including arithmetic measures and motion-specific evaluation metrics. Our evaluation demonstrates that the proposed framework can generate motions that maintain the essence of original movements while enhancing stroke quality, providing closer guidance than direct expert motion replication. The results highlight the potential of our approach for creating personalized sports motion guides by generating counterfactual motion guidance for arbitrary input motion samples of badminton strokes.


title: Learning of Balance Controller Considering Changes in Body State for Musculoskeletal Humanoids

abstract: The musculoskeletal humanoid is difficult to modelize due to the flexibility and redundancy of its body, whose state can change over time, and so balance control of its legs is challenging. There are some cases where ordinary PID controls may cause instability. In this study, to solve these problems, we propose a method of learning a correlation model among the joint angle, muscle tension, and muscle length of the ankle and the zero moment point to perform balance control. In addition, information on the changing body state is embedded in the model using parametric bias, and the model estimates and adapts to the current body state by learning this information online. This makes it possible to adapt to changes in upper body posture that are not directly taken into account in the model, since it is difficult to learn the complete dynamics of the whole body considering the amount of data and computation. The model can also adapt to changes in body state, such as the change in footwear and change in the joint origin due to recalibration. The effectiveness of this method is verified by a simulation and by using an actual musculoskeletal humanoid, Musashi.


title: (Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts

abstract: Recent advancements in machine translation (MT) have significantly enhanced translation quality across various domains. However, the translation of literary texts remains a formidable challenge due to their complex language, figurative expressions, and cultural nuances. In this work, we introduce a novel multi-agent framework based on large language models (LLMs) for literary translation, implemented as a company called TransAgents, which mirrors traditional translation publication process by leveraging the collective capabilities of multiple agents, to address the intricate demands of translating literary works. To evaluate the effectiveness of our system, we propose two innovative evaluation strategies: Monolingual Human Preference (MHP) and Bilingual LLM Preference (BLP). MHP assesses translations from the perspective of monolingual readers of the target language, while BLP uses advanced LLMs to compare translations directly with the original texts. Empirical findings indicate that despite lower d-BLEU scores, translations from TransAgents are preferred by both human evaluators and LLMs over human-written references, particularly in genres requiring domain-specific knowledge. We also highlight the strengths and limitations of TransAgents through case studies and suggests directions for future research.


title: The Geometry of Tangent Spaces on Causal Sets

abstract: In this paper, we expand on previous work describing partial derivatives and metric component estimators to define tangent spaces on causal sets. Partial derivative operators are the basis vectors of the tangent space, and the metric defines the inner product. First, we use partial derivatives of the metric components to define the connection and partial derivatives of the connection to define the curvature. Numerical results show that both of these approach the expected values for a flat spacetime as density increases. Then we used the connection to define parallel transport and geodesics.


title: Global stability and period-doubling bifurcations of a discrete Kolmogorov predator-prey model with Ricker-type prey growth

abstract: In this paper, we study the dynamics of a discrete Kolmogorov predator-prey model with Ricker-type prey growth. We give the sufficient and necessary condition to guarantee the existence and uniqueness of the positive fixed point. Using the center manifold theory, we prove that the period-doubling bifurcations can occur at the positive fixed point. Furthermore, our numerical simulations reveal that the model can exhibit cascades of period-doubling bifurcations leading to chaos, which is a significant difference from the behavior of continuous predator-prey models. Despite the complexities of the model dynamics, we are able to provide a criterion for the global stability of the positive fixed point by using a geometric analysis of the nullclines.


title: Dual-sided Peltier Elements for Rapid Thermal Feedback in Wearables

abstract: This paper introduces a motor-driven Peltier device designed to deliver immediate thermal sensations within extended reality (XR) environments. The system incorporates eight motor-driven Peltier elements, facilitating swift transitions between warm and cool sensations by rotating preheated or cooled elements to opposite sides. A multi-layer structure, comprising aluminum and silicone layers, ensures user comfort and safety while maintaining optimal temperatures for thermal stimuli. Time-temperature characteristic analysis demonstrates the system's ability to provide warm and cool sensations efficiently, with a dual-sided lifetime of up to 206 seconds at a 2V input. Our system design is adaptable to various body parts and can be synchronized with corresponding visual stimuli to enhance the immersive sensation of virtual object interaction and information delivery.


title: Online Learning Feedback Control Considering Hysteresis for Musculoskeletal Structures

abstract: While the musculoskeletal humanoid has various biomimetic benefits, its complex modeling is difficult, and many learning control methods have been developed. However, for the actual robot, the hysteresis of its joint angle tracking is still an obstacle, and realizing target posture quickly and accurately has been difficult. Therefore, we develop a feedback control method considering the hysteresis. To solve the problem in feedback controls caused by the closed-link structure of the musculoskeletal body, we update a neural network representing the relationship between the error of joint angles and the change in target muscle lengths online, and realize target joint angles accurately in a few trials. We compare the performance of several configurations with various network structures and loss definitions, and verify the effectiveness of this study on an actual musculoskeletal humanoid, Musashi.


title: Distill-then-prune: An Efficient Compression Framework for Real-time Stereo Matching Network on Edge Devices

abstract: In recent years, numerous real-time stereo matching methods have been introduced, but they often lack accuracy. These methods attempt to improve accuracy by introducing new modules or integrating traditional methods. However, the improvements are only modest. In this paper, we propose a novel strategy by incorporating knowledge distillation and model pruning to overcome the inherent trade-off between speed and accuracy. As a result, we obtained a model that maintains real-time performance while delivering high accuracy on edge devices. Our proposed method involves three key steps. Firstly, we review state-of-the-art methods and design our lightweight model by removing redundant modules from those efficient models through a comparison of their contributions. Next, we leverage the efficient model as the teacher to distill knowledge into the lightweight model. Finally, we systematically prune the lightweight model to obtain the final model. Through extensive experiments conducted on two widely-used benchmarks, Sceneflow and KITTI, we perform ablation studies to analyze the effectiveness of each module and present our state-of-the-art results.


title: Formation of Iron-Helium Compounds under High Pressure

abstract: While helium is a representative noble gas element characterized by its chemical inertness at ambient conditions, recent experiments and calculations argued that a minor amount of helium is incorporated into molten iron from silicate under high pressures. Here we examined the reaction between iron and helium at 8-42 GPa and ~1000-2820 K and found remarkable volume expansion of the Fe lattice, which is attributed to the formations of fcc and distorted hcp iron-helium compounds with x in FeHex up to 0.13 and 0.42, respectively. Upon releasing pressure under room temperature, these fcc and distorted hcp FeHex were still observed while the former lost some helium. In addition, our first-principles calculations indicate that fcc FeHe0.25, with He atoms in the tetrahedral interstitial sites, is dynamically stable in different magnetic states throughout 0-50 GPa. These results support that the Earth's core can be a large reservoir of primordial 3He.


title: FedCAda: Adaptive Client-Side Optimization for Accelerated and Stable Federated Learning

abstract: Federated learning (FL) has emerged as a prominent approach for collaborative training of machine learning models across distributed clients while preserving data privacy. However, the quest to balance acceleration and stability becomes a significant challenge in FL, especially on the client-side. In this paper, we introduce FedCAda, an innovative federated client adaptive algorithm designed to tackle this challenge. FedCAda leverages the Adam algorithm to adjust the correction process of the first moment estimate $m$ and the second moment estimate $v$ on the client-side and aggregate adaptive algorithm parameters on the server-side, aiming to accelerate convergence speed and communication efficiency while ensuring stability and performance. Additionally, we investigate several algorithms incorporating different adjustment functions. This comparative analysis revealed that due to the limited information contained within client models from other clients during the initial stages of federated learning, more substantial constraints need to be imposed on the parameters of the adaptive algorithm. As federated learning progresses and clients gather more global information, FedCAda gradually diminishes the impact on adaptive parameters. These findings provide insights for enhancing the robustness and efficiency of algorithmic improvements. Through extensive experiments on computer vision (CV) and natural language processing (NLP) datasets, we demonstrate that FedCAda outperforms the state-of-the-art methods in terms of adaptability, convergence, stability, and overall performance. This work contributes to adaptive algorithms for federated learning, encouraging further exploration.


title: Nonlinear Lindblad Master Equation and Postselected Skin Effect

abstract: We introduce a non-linear Lindblad master equation to describe the postselection dynamics of open quantum systems described by the Lindblad master equation, which continuously interpolates between the Lindblad master equation and the dynamical equation governed by an effective non-Hermitian Hamiltonian. Within the framework of the non-linear Lindblad master equation, we study a prototypical model and demonstrate the existence of the postselected skin effect with the distribution of a steady state characterized by the accumulation of particles on one side, as long as some quantum jumping terms are discarded by postselction processes. Moreover, we show that the trajectory-averaged entanglement entropy can reflect the different influences from the environment and postselection, and unveil it exhibiting a special distribution with algebraic growth in the short chain and saturation in the long chain induced by the postselected skin effect.


title: Blow-up phenomena and the local well-posedness of a Two-Component b-Family Equations in Besov spaces

abstract: In this paper, we first establish the local well-posednesss of a Two-Component b-Family equations in nonhomogeneous Besov spaces $B^{1+\frac 1 p}_{p,1}$ with $1\leq p<+\infty.$ Then we present a new blow-up result for the Two-Component b-Family equations.


title: Climatic & Anthropogenic Hazards to the Nasca World Heritage: Application of Remote Sensing, AI, and Flood Modelling

abstract: Preservation of the Nasca geoglyphs at the UNESCO World Heritage Site in Peru is urgent as natural and human impact accelerates. More frequent weather extremes such as flashfloods threaten Nasca artifacts. We demonstrate that runoff models based on (sub-)meter scale, LiDAR-derived digital elevation data can highlight AI-detected geoglyphs that are in danger of erosion. We recommend measures of mitigation to protect the famous "lizard", "tree", and "hand" geoglyphs located close by, or even cut by the Pan-American Highway.


title: Method of Filtration in first passage time problems

abstract: Statistics of stochastic processes crucially depends on the imposed boundary condition. In one spatial dimension, for example, first passage time distribution in semi-infinite space (one absorbing boundary) is markedly different from that in a finite interval with two absorbing boundaries. Here we propose a method, which we call method of filtration, that allows us to construct the latter from the knowledge of the former only. We argue that our method is a sort of generalization of the method of image. We also demonstrate that in terms of the performance, it is complementary to another standard method based on the eigenfunction expansion solution of the Fokker-Planck equation.


title: Transfer Learning for CSI-based Positioning with Multi-environment Meta-learning

abstract: Utilizing deep learning (DL) techniques for radio-based positioning of user equipment (UE) through channel state information (CSI) fingerprints has demonstrated significant potential. DL models can extract complex characteristics from the CSI fingerprints of a particular environment and accurately predict the position of a UE. Nonetheless, the effectiveness of the DL model trained on CSI fingerprints is highly dependent on the particular training environment, limiting the trained model's applicability across different environments. This paper proposes a novel DL model structure consisting of two parts, where the first part aims at identifying features that are independent from any specific environment, while the second part combines those features in an environment specific way with the goal of positioning. To train such a two-part model, we propose the multi-environment meta-learning (MEML) approach for the first part to facilitate training across various environments, while the second part of the model is trained solely on data from a specific environment. Our findings indicate that employing the MEML approach for initializing the weights of the DL model for a new unseen environment significantly boosts the accuracy of UE positioning in the new target environment as well the reliability of its uncertainty estimation. This method outperforms traditional transfer learning methods, whether direct transfer learning (DTL) between environments or completely training from scratch with data from a new environment. The proposed approach is verified with real measurements for both line-of-sight (LOS) and non-LOS (NLOS) environments.


title: Systematic Review on Healthcare Systems Engineering utilizing ChatGPT

abstract: This paper presents an analytical framework for conducting academic reviews in the field of Healthcare Systems Engineering, employing ChatGPT, a state-of-the-art tool among recent language models. We utilized 9,809 abstract paragraphs from conference presentations to systematically review the field. The framework comprises distinct analytical processes, each employing tailored prompts and the systematic use of the ChatGPT API. Through this framework, we organized the target field into 11 topic categories and conducted a comprehensive analysis covering quantitative yearly trends and detailed sub-categories. This effort explores the potential for leveraging ChatGPT to alleviate the burden of academic reviews. Furthermore, it provides valuable insights into the dynamic landscape of Healthcare Systems Engineering research.


title: A Rate-Distortion Analysis for Composite Sources Under Subsource-Dependent Fidelity Criteria

abstract: A composite source, consisting of multiple subsources and a memoryless switch, outputs one symbol at a time from the subsource selected by the switch. If some data should be encoded more accurately than other data from an information source, the composite source model is suitable because in this model different distortion constraints can be put on the subsources. In this context, we propose subsource-dependent fidelity criteria for composite sources and use them to formulate a rate-distortion problem. We solve the problem and obtain a single-letter expression for the rate-distortion function. Further rate-distortion analysis characterizes the performance of classify-then-compress (CTC) coding, which is frequently used in practice when subsource-dependent fidelity criteria are considered. Our analysis shows that CTC coding generally has performance loss relative to optimal coding, even if the classification is perfect. We also identify the cause of the performance loss, that is, class labels have to be reproduced in CTC coding. Last but not least, we show that the performance loss is negligible for asymptotically small distortion if CTC coding is appropriately designed and some mild conditions are satisfied.


title: Beyond MLE: Investigating SEARNN for Low-Resourced Neural Machine Translation

abstract: Structured prediction tasks, like machine translation, involve learning functions that map structured inputs to structured outputs. Recurrent Neural Networks (RNNs) have historically been a popular choice for such tasks, including in natural language processing (NLP) applications. However, training RNNs using Maximum Likelihood Estimation (MLE) has its limitations, including exposure bias and a mismatch between training and testing metrics. SEARNN, based on the learning to search (L2S) framework, has been proposed as an alternative to MLE for RNN training. This project explored the potential of SEARNN to improve machine translation for low-resourced African languages -- a challenging task characterized by limited training data availability and the morphological complexity of the languages. Through experiments conducted on translation for English to Igbo, French to \ewe, and French to \ghomala directions, this project evaluated the efficacy of SEARNN over MLE in addressing the unique challenges posed by these languages. With an average BLEU score improvement of $5.4$\% over the MLE objective, we proved that SEARNN is indeed a viable algorithm to effectively train RNNs on machine translation for low-resourced languages.


title: Testing neutrino mass hierarchy under type-II seesaw scenario in $U(1)_X$ at colliders

abstract: The origin of tiny neutrino mass is a long standing unsolved puzzle of the Standard Model (SM) which allows us to consider scenarios beyond the Standard Model (BSM) in a variety of ways. One of them being the gauge extension of the SM could be realized as in the form of an anomaly free, general $U(1)_X$ extension of the SM where an $SU(2)_L$ triplet scalar being charged under $U(1)_X$ gauge group is introduced through a Dirac Yukawa coupling with the SM lepton doublet. Once the triplet scalar generates VEV, light neutrinos could acquire tiny Majorana mass and hence affecting the decay modes of the triplet scalar involving the neutrino oscillation data for different neutrino mass hierarchies. After the breaking of $U(1)_X$ scenarios, a neutral BSM, neutral gauge boson $(Z^\prime)$ acquires mass which interact differently with the left and right handed fermions. Satisfying the recent LHC bounds on the triplet scalar and $Z^\prime$ production, we study the pair production of the triplet scalar at LHC, $e^-e^+$ and $\mu^-\mu^+$ colliders followed by its decay into dominant mode depending on the neutrino mass hierarchy. Generating the SM generic backgrounds, we study the possible signal significance of four lepton final states. We also compare our results with the purely SM gauge mediated triplet scalar pair production followed by four lepton final states which could be significant only in $\mu^- \mu^+$ collider.


title: A Three-Phase Analysis of Synergistic Effects During Co-pyrolysis of Algae and Wood for Biochar Yield Using Machine Learning

abstract: Pyrolysis techniques have served to be a groundbreaking technique for effectively utilising natural and man-made biomass products like plastics, wood, crop residue, fruit peels etc. Recent advancements have shown a greater yield of essential products like biochar, bio-oil and other non-condensable gases by blending different biomasses in a certain ratio. This synergy effect of combining two pyrolytic raw materials i.e co-pyrolysis of algae and wood biomass has been systematically studied and grouped into 3 phases in this research paper-kinetic analysis of co-pyrolysis, correlation among proximate and ultimate analysis with bio-char yield and lastly grouping of different weight ratios based on biochar yield up to a certain percentage. Different ML and DL algorithms have been utilized for regression and classification techniques to give a comprehensive overview of the effect of the synergy of two different biomass materials on biochar yield. For the first phase, the best prediction of biochar yield was obtained by using a decision tree regressor with a perfect MSE score of 0.00, followed by a gradient-boosting regressor. The second phase was analyzed using both ML and DL techniques. Within ML, SVR proved to be the most convenient model with an accuracy score of 0.972 with DNN employed for deep learning technique. Finally, for the third phase, binary classification was applied to biochar yield with and without heating rate for biochar yield percentage above and below 40%. The best technique for ML was Support Vector followed by Random forest while ANN was the most suitable Deep Learning Technique.


title: FeTT: Continual Class Incremental Learning via Feature Transformation Tuning

abstract: Continual learning (CL) aims to extend deep models from static and enclosed environments to dynamic and complex scenarios, enabling systems to continuously acquire new knowledge of novel categories without forgetting previously learned knowledge. Recent CL models have gradually shifted towards the utilization of pre-trained models (PTMs) with parameter-efficient fine-tuning (PEFT) strategies. However, continual fine-tuning still presents a serious challenge of catastrophic forgetting due to the absence of previous task data. Additionally, the fine-tune-then-frozen mechanism suffers from performance limitations due to feature channels suppression and insufficient training data in the first CL task. To this end, this paper proposes feature transformation tuning (FeTT) model to non-parametrically fine-tune backbone features across all tasks, which not only operates independently of CL training data but also smooths feature channels to prevent excessive suppression. Then, the extended ensemble strategy incorporating different PTMs with FeTT model facilitates further performance improvement. We further elaborate on the discussions of the fine-tune-then-frozen paradigm and the FeTT model from the perspectives of discrepancy in class marginal distributions and feature channels. Extensive experiments on CL benchmarks validate the effectiveness of our proposed method.


title: Stereo-Knowledge Distillation from dpMV to Dual Pixels for Light Field Video Reconstruction

abstract: Dual pixels contain disparity cues arising from the defocus blur. This disparity information is useful for many vision tasks ranging from autonomous driving to 3D creative realism. However, directly estimating disparity from dual pixels is less accurate. This work hypothesizes that distilling high-precision dark stereo knowledge, implicitly or explicitly, to efficient dual-pixel student networks enables faithful reconstructions. This dark knowledge distillation should also alleviate stereo-synchronization setup and calibration costs while dramatically increasing parameter and inference time efficiency. We collect the first and largest 3-view dual-pixel video dataset, dpMV, to validate our explicit dark knowledge distillation hypothesis. We show that these methods outperform purely monocular solutions, especially in challenging foreground-background separation regions using faithful guidance from dual pixels. Finally, we demonstrate an unconventional use case unlocked by dpMV and implicit dark knowledge distillation from an ensemble of teachers for Light Field (LF) video reconstruction. Our LF video reconstruction method is the fastest and most temporally consistent to date. It remains competitive in reconstruction fidelity while offering many other essential properties like high parameter efficiency, implicit disocclusion handling, zero-shot cross-dataset transfer, geometrically consistent inference on higher spatial-angular resolutions, and adaptive baseline control. All source code is available at the anonymous repository https://github.com/Aryan-Garg.


title: Time evolution of the von Neumann entropy in open quantum system

abstract: Control of open quantum dynamics is of great interest for realizing quantum technologies. Therefore, it is an important task to quantify and characterize the entropy for open quantum systems under decoherence. In this paper, we study the time evolution of the von Neumann entropy for open quantum systems described by the Lindblad master equation. Note that, in particular, when the decoherence corresponds to the measurement for the observable in the system, the von Neumann entropy tends to monotonically increases as the variance becomes larger. Furthermore, we present a lower bound of the von Neumann entropy in the long-time limit. This lower bound has advantages of being straightforwardly calculated and applicable to a general Markovian open quantum system.


title: Measuring Technical Debt in AI-Based Competition Platforms

abstract: Advances in AI have led to new types of technical debt in software engineering projects. AI-based competition platforms face challenges due to rapid prototyping and a lack of adherence to software engineering principles by participants, resulting in technical debt. Additionally, organizers often lack methods to evaluate platform quality, impacting sustainability and maintainability. In this research, we identify and categorize types of technical debt in AI systems through a scoping review. We develop a questionnaire for assessing technical debt in AI competition platforms, categorizing debt into various types, such as algorithm, architectural, code, configuration, data etc. We introduce Accessibility Debt, specific to AI competition platforms, highlighting challenges participants face due to inadequate platform usability. Our framework for managing technical debt aims to improve the sustainability and effectiveness of these platforms, providing tools for researchers, organizers, and participants.


title: Data quality control system and long-term performance monitor of the LHAASO-KM2A

abstract: The KM2A is the largest sub-array of the Large High Altitude Air Shower Observatory (LHAASO). It consists of 5216 electromagnetic particle detectors (EDs) and 1188 muon detectors (MDs). The data recorded by the EDs and MDs are used to reconstruct primary information of cosmic ray and gamma-ray showers. This information is used for physical analysis in gamma-ray astronomy and cosmic ray physics. To ensure the reliability of the LHAASO-KM2A data, a three-level quality control system has been established. It is used to monitor the status of detector units, stability of reconstructed parameters and the performance of the array based on observations of the Crab Nebula and Moon shadow. This paper will introduce the control system and its application on the LHAASO-KM2A data collected from August 2021 to July 2023. During this period, the pointing and angular resolution of the array were stable. From the observations of the Moon shadow and Crab Nebula, the results achieved using the two methods are consistent with each other. According to the observation of the Crab Nebula at energies from 25 TeV to 100 TeV, the time averaged pointing errors are estimated to be $-0.003^{\circ} \pm 0.005^{\circ}$ and $0.001^{\circ} \pm 0.006^{\circ}$ in the R.A. and Dec directions, respectively.


title: Photometric analysis of the dwarf nova SU UMa based on TESS

abstract: We report the photometric analysis of SU UMa based on the observations of the Transiting Exoplanet Survey Satellite (TESS). TESS has released a large amount of data, which contains the light curves of a complete superoutburst and three normal outbursts of SU UMa. Based on the observations, the evolution of superhumps during the superoutburst was analyzed. By using the O - C method, the three stages were determined, and the superhump period for each stage was calculated. The mass ratio $q=0.137 (1)$ was estimated based on the stage A superhump method. A periodic oscillation with a period of 2.17(9) days was found in the superhump minimum O - C diagram, which is related to the precession of the accretion disk. We investigated the frequency information of quasiperiodic oscillations (QPOs) in the light curves. An ultralong period QPO of 0.18 days at the time of normal outburst and two ultralong periods QPOs of 0.19 days and 0.285 days in quiescence were found. No QPO was found in the superoutburst.


title: Federated Learning with Incomplete Sensing Modalities

abstract: Many mobile sensing applications utilize data from various modalities, including motion and physiological sensors in mobile and wearable devices. Federated Learning (FL) is particularly suitable for these applications thanks to its privacy-preserving feature. However, challenges such as limited battery life, poor network conditions, and sensor malfunctions can restrict the use of all available modalities for local model training. Additionally, existing multimodal FL systems also struggle with scalability and efficiency as the number of modality sources increases. To address these issues, we introduce FLISM, a framework designed to enable multimodal FL with incomplete modalities. FLISM leverages simulation technique to learn robust representations that can handle missing modalities and transfers model knowledge across clients with varying set of modalities. The evaluation results using three real-world datasets and simulations demonstrate FLISM's effective balance between model performance and system efficiency. It shows an average improvement of .067 in F1-score, while also reducing communication (2.69x faster) and computational (2.28x more efficient) overheads compared to existing methods addressing incomplete modalities. Moreover, in simulated scenarios involving tasks with a larger number of modalities, FLISM achieves a significant speedup of 3.23x~85.10x in communication and 3.73x~32.29x in computational efficiency.


title: Adversarially Diversified Rehearsal Memory (ADRM): Mitigating Memory Overfitting Challenge in Continual Learning

abstract: Continual learning focuses on learning non-stationary data distribution without forgetting previous knowledge. Rehearsal-based approaches are commonly used to combat catastrophic forgetting. However, these approaches suffer from a problem called "rehearsal memory overfitting, " where the model becomes too specialized on limited memory samples and loses its ability to generalize effectively. As a result, the effectiveness of the rehearsal memory progressively decays, ultimately resulting in catastrophic forgetting of the learned tasks. We introduce the Adversarially Diversified Rehearsal Memory (ADRM) to address the memory overfitting challenge. This novel method is designed to enrich memory sample diversity and bolster resistance against natural and adversarial noise disruptions. ADRM employs the FGSM attacks to introduce adversarially modified memory samples, achieving two primary objectives: enhancing memory diversity and fostering a robust response to continual feature drifts in memory samples. Our contributions are as follows: Firstly, ADRM addresses overfitting in rehearsal memory by employing FGSM to diversify and increase the complexity of the memory buffer. Secondly, we demonstrate that ADRM mitigates memory overfitting and significantly improves the robustness of CL models, which is crucial for safety-critical applications. Finally, our detailed analysis of features and visualization demonstrates that ADRM mitigates feature drifts in CL memory samples, significantly reducing catastrophic forgetting and resulting in a more resilient CL model. Additionally, our in-depth t-SNE visualizations of feature distribution and the quantification of the feature similarity further enrich our understanding of feature representation in existing CL approaches. Our code is publically available at https://github.com/hikmatkhan/ADRM.


title: Fe2+ partitioning in Al-free pyrolite: consequences for seismic velocities and heterogeneities

abstract: Iron partitioning among the main lower mantle phases, bridgmanite (Bm) and ferropericlase (Fp), has non-monotonic behavior owing to the high-spin to low-spin crossover in ferrous iron (Fe2+) in Fp. Results of previous studies of the iron partitioning coefficient between these phases, $K_D$, still have considerable uncertainty. Here, we investigate the Fe2+ partitioning behavior using well-documented ab initio free energy results plus new updates. Although we focus on Fe2+ only, we describe the effect of this iron spin crossover (ISC) on $K_D$ and of the latter on compositions and seismic velocities in a pyrolitic aggregate. Our results suggest that its velocities are mainly affected by the ISC and less so by the Fe2+ partitioning. In contrast, iron partitioning manifests in thermally induced velocity heterogeneity ratios. Prediction of the seismological parameter $R_{S/P}$ ($\partial \ln V_S/\partial \ln V_P$) including iron partitioning effects resembles quantitatively $R_{S/P}$'s inferred from several tomographic studies down to 2,400 km depth.


title: SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model

abstract: Transformers have revolutionized deep learning across various tasks, including audio representation learning, due to their powerful modeling capabilities. However, they often suffer from quadratic complexity in both GPU memory usage and computational inference time, affecting their efficiency. Recently, state space models (SSMs) like Mamba have emerged as a promising alternative, offering a more efficient approach by avoiding these complexities. Given these advantages, we explore the potential of SSM-based models in audio tasks. In this paper, we introduce Self-Supervised Audio Mamba (SSAMBA), the first self-supervised, attention-free, and SSM-based model for audio representation learning. SSAMBA leverages the bidirectional Mamba to capture complex audio patterns effectively. We incorporate a self-supervised pretraining framework that optimizes both discriminative and generative objectives, enabling the model to learn robust audio representations from large-scale, unlabeled datasets. We evaluated SSAMBA on various tasks such as audio classification, keyword spotting, and speaker identification. Our results demonstrate that SSAMBA outperforms the Self-Supervised Audio Spectrogram Transformer (SSAST) in most tasks. Notably, SSAMBA is approximately 92.7% faster in batch inference speed and 95.4% more memory-efficient than SSAST for the tiny model size with an input token size of 22k. These efficiency gains, combined with superior performance, underscore the effectiveness of SSAMBA's architectural innovation, making it a compelling choice for a wide range of audio processing applications.


title: Braiding Topology of Non-Hermitian Open-Boundary Bands

abstract: There has been much recent interest and progress on topological structures of the non-Hermitian Bloch bands. Here, we study the topological structures of non-Bloch bands of non-Hermitian multi-band quantum systems under open boundary conditions, which has received limited attention in prior studies. Using a continuity criterion and an efficient sub-GBZ algorithm, we establish a homotopic characterization -- braiding topology, e.g., characterized by the band's total vorticity -- for open-boundary bands and sub-GBZs. Such topological identification is robust without topological transition and emergent degenerate points, such as exceptional points. We further analyze the transition's impact on bands and spectral flows, including interesting properties unique to open boundaries, and numerically demonstrate our conclusions with tight-binding model examples. We unveil a crucial insight that open-boundary bands interchange their portions after encountering certain exceptional points. Our results enrich our foundational understanding of topological characterizations for generic non-Hermitian quantum systems.


title: Determination of the N\'eel vector in rutile altermagnets through x-ray magnetic circular dichroism: the case of MnF$_2$

abstract: We present a numerical simulation of the X-ray magnetic circular dichroism (XMCD) at the $L_{2,3}$ edge of Mn in altermagnetic MnF$_2$ using a combination of density functional + exact diagonlization of an atomic model. We explore how the dichroic spectra vary with the light propagation vector and the N\'eel vector. We show how XMCD in rutile structures can be employed to determine the orientation of the N\'eel vector. An exact relationship between the XMCD spectra for different orientation of the N\'eel vector valid in the absence on the valence spin-orbit coupling and core-valence multipole interaction is derived and its approximate validity demonstrated by numerical calculation for the full Hamiltonian.


title: Modified Greenwood statistic and its application for statistical testing

abstract: In this paper, we explore the modified Greenwood statistic, which, in contrast to the classical Greenwood statistic, is properly defined for random samples from any distribution. The classical Greenwood statistic, extensively examined in the existing literature, has found diverse and interesting applications across various domains. Furthermore, numerous modifications to the classical statistic have been proposed. The modified Greenwood statistic, as proposed and discussed in this paper, shares several key properties with its classical counterpart. Emphasizing its stochastic monotonicity within three broad classes of distributions - namely, generalized Pareto, $\alpha-$stable, and Student's t distributions - we advocate for the utilization of the modified Greenwood statistic in testing scenarios. Our exploration encompasses three distinct directions. In the first direction, we employ the modified Greenwood statistic for Gaussian distribution testing. Our empirical results compellingly illustrate that the proposed approach consistently outperforms alternative goodness-of-fit tests documented in the literature, particularly exhibiting superior efficacy for small sample sizes. The second considered problem involves testing the infinite-variance distribution of a given random sample. The last proposition suggests using the modified Greenwood statistic for testing of a given distribution. The presented simulation study strongly supports the efficiency of the proposed approach in the considered problems. Theoretical results and power simulation studies are further validated by real data analysis.


title: Demo Paper: A Game Agents Battle Driven by Free-Form Text Commands Using Code-Generation LLM

abstract: This paper presents a demonstration of our monster battle game, in which the game agents fight in accordance with their player's language commands. The commands were translated into the knowledge expression called behavior branches by a code-generation large language model. This work facilitated the design of the commanding system more easily, enabling the game agent to comprehend more various and continuous commands than rule-based methods. The results of the commanding and translation process were stored in a database on an Amazon Web Services server for more comprehensive validation. This implementation would provide a sufficient evaluation of this ongoing work, and give insights to the industry that they could use this to develop their interactive game agents.


title: PLASMA -- Platform for Service Management in Digital Remote Maintenance Applications

abstract: To support maintenance and servicing of industrial machines, service processes are even today often performed manually and analogously, although supportive technologies such as augmented reality, virtual reality and digital platforms already exist. In many cases, neither technicians on-site nor remote experts have all the essential information and options for suitable actions available. Existing service products and platforms do not cover all the required functions in practice in order to map end-to-end processes. PLASMA is a concept for a Cloud-based remote maintenance platform designed to meet these demands. But for a real-life implementation of PLASMA, security measures are essential as we show in this paper.


title: Improving the Explain-Any-Concept by Introducing Nonlinearity to the Trainable Surrogate Model

abstract: In the evolving field of Explainable AI (XAI), interpreting the decisions of deep neural networks (DNNs) in computer vision tasks is an important process. While pixel-based XAI methods focus on identifying significant pixels, existing concept-based XAI methods use pre-defined or human-annotated concepts. The recently proposed Segment Anything Model (SAM) achieved a significant step forward to prepare automatic concept sets via comprehensive instance segmentation. Building upon this, the Explain Any Concept (EAC) model emerged as a flexible method for explaining DNN decisions. EAC model is based on using a surrogate model which has one trainable linear layer to simulate the target model. In this paper, by introducing an additional nonlinear layer to the original surrogate model, we show that we can improve the performance of the EAC model. We compare our proposed approach to the original EAC model and report improvements obtained on both ImageNet and MS COCO datasets.


title: Sweedler duality for Hom-algebras and Hom-modules

abstract: The construction of Sweedler duality is an important tool in the theory of Hopf algebras over a field, which is a right adjoint to the dual algebra functor. In this paper, we study the Sweedler duality of Hom-algebras and their Hom-modules. We delve into the structure of Hom-coalgebras and derive the linear morphisms associated with them. Additionally, as an application, we present the (right) Hom-(co)module morphisms under the Sweedler duality.


title: Highly charged ions of heavy actinides as sensitive probes for time variation of the fine structure constant

abstract: Highly charged ions of heavy actinides from uranium to einsteinium are studied theoretically to find optical transitions sensitive to the variation of the fine structure constant. A number of promising transitions have been found in ions with ionisation degree $\sim$~10. All these transitions correspond in single-electron approximation to the $6p$ - $5f$ transitions. Many of the transitions are between ground and excited metastable states of the ions which means that they can probably be used as optical clock transitions. Some of the ions have more than one clock transition with different sensitivity to the variation of the fine structure constant $\alpha$. The most promising systems include the Np$^{10+}$, Np$^{9+}$, Pu$^{11+}$, Pu$^{10+}$, Pu$^{9+}$, Pu$^{8+}$, Bk$^{15+}$, Cm$^{12+}$, and Es$^{15+}$ ions.


title: Growth of cosmic perturbations in the modified $f(R,T)$ gravity

abstract: We explore the generalized $f(R,T)$ modified theory of gravity, where the gravitational Lagrangian is a function of Ricci scalar $R$ and the trace of the energy-momentum tensor $T$. We derive modified field equations to the linear order of perturbations in the context of $f(R,T)$ model. We then investigate the growth of perturbations in the context of $f(R,T)$ modified gravity. Primary numerical investigations based on matter power spectra diagrams indicates a structure growth suppression in $f(R,T)$ gravity, which exhibits consistency with local measurements. Also, we notice that matter-geometry interaction in $f(R,T)$ model would results in the specific feature named as "matter acoustic oscillations" appeared in matter power spectra diagrams. Moreover, we put constraints on the cosmological parameters of $f(R,T)$ model, utilizing current observations, chiefly cosmic microwave background (CMB), weak lensing, supernovae, baryon acoustic oscillations (BAO), and redshift-space distortions (RSD) data. Numerical results based on MCMC calculations imply that $f(R,T)$ is a qualified theory of modified gravity in reconciling Planck CMB data with local probes of large scale structures, by reporting lower values for the structure growth parameter $\sigma_8$ compared to the standard model of cosmology.


title: Evaluating and Modeling Social Intelligence: A Comparative Study of Human and AI Capabilities

abstract: Facing the current debate on whether Large Language Models (LLMs) attain near-human intelligence levels (Mitchell & Krakauer, 2023; Bubeck et al., 2023; Kosinski, 2023; Shiffrin & Mitchell, 2023; Ullman, 2023), the current study introduces a benchmark for evaluating social intelligence, one of the most distinctive aspects of human cognition. We developed a comprehensive theoretical framework for social dynamics and introduced two evaluation tasks: Inverse Reasoning (IR) and Inverse Inverse Planning (IIP). Our approach also encompassed a computational model based on recursive Bayesian inference, adept at elucidating diverse human behavioral patterns. Extensive experiments and detailed analyses revealed that humans surpassed the latest GPT models in overall performance, zero-shot learning, one-shot generalization, and adaptability to multi-modalities. Notably, GPT models demonstrated social intelligence only at the most basic order (order = 0), in stark contrast to human social intelligence (order >= 2). Further examination indicated a propensity of LLMs to rely on pattern recognition for shortcuts, casting doubt on their possession of authentic human-level social intelligence. Our codes, dataset, appendix and human data are released at https://github.com/bigai-ai/Evaluate-n-Model-Social-Intelligence.


title: High energy nucleus-nucleus collisions with accounting for Coulomb interaction

abstract: The differential elastic cross sections of $^{12}$C - $^{12}$C, $^{16}$O - $^{16}$O and $^{20}$Ne - $^{20}$Ne nuclei scattering are calculated in the complete Glauber theory with the account of the modification due to Coulomb interaction and form factor effects. The role of the Coulomb interaction is shown to be significant mainly in the diffractive minima. The results of the complete Coulomb calculations are noticeably different from those obtained in the Born approximation.


title: Investigation of superhumps in SU UMa-type dwarf novae based on the observations of TESS

abstract: We report the superhumps analysis of seven SU UMa-type dwarf novae based on the observations of Transiting Exoplanet Survey Satellite (TESS). Superhumps are seen during superoutbursts of SU UMa-type dwarf novae. The month-long data sets of TESS are well suited for studying the variation of superhumps. We selected seven non-eclipsing SU UMa-type dwarf novae with superhumps in which TESS light curves are available and have not yet been studied. The stages A, B, and C of superhumps in superoutbursts were determined by O-C method. The results indicate that not all complete superoutbursts show obvious three stages, such as DT Oct and the second superoutburst of J1730+6247. We calculated the superhump periods for each stage and the mean periods for whole superoutbursts. Taking the stage A superhump method, the mass ratios (M2/M1) were estimated. According to the results, the seven stars are pre-bounce systems with mass ratios ranging from 0.1 to 0.2. By combining the orbital periods and the mean superhump periods, the precession periods were calculated. The results show that the precession periods of the seven SU UMa stars are about 2 days.


title: NeRTCAM: CAM-Based CMOS Implementation of Reference Frames for Neuromorphic Processors

abstract: Neuromorphic architectures mimicking biological neural networks have been proposed as a much more efficient alternative to conventional von Neumann architectures for the exploding compute demands of AI workloads. Recent neuroscience theory on intelligence suggests that Cortical Columns (CCs) are the fundamental compute units in the neocortex and intelligence arises from CC's ability to store, predict and infer information via structured Reference Frames (RFs). Based on this theory, recent works have demonstrated brain-like visual object recognition using software simulation. Our work is the first attempt towards direct CMOS implementation of Reference Frames for building CC-based neuromorphic processors. We propose NeRTCAM (Neuromorphic Reverse Ternary Content Addressable Memory), a CAM-based building block that supports the key operations (store, predict, infer) required to perform inference using RFs. NeRTCAM architecture is presented in detail including its key components. All designs are implemented in SystemVerilog and synthesized in 7nm CMOS, and hardware complexity scaling is evaluated for varying storage sizes. NeRTCAM system for biologically motivated MNIST inference with a storage size of 1024 entries incurs just 0.15 mm^2 area, 400 mW power and 9.18 us critical path latency, demonstrating the feasibility of direct CMOS implementation of CAM-based Reference Frames.


title: Speed of Random Walks in Dirichlet Environment on a Galton-Watson Tree

abstract: This paper deals with a transient random walk in Dirichlet environment, or equivalently a linearly edge reinforced random walk, on a Galton-Watson tree. We compute the stationary distribution of the environment seen from the particle of an edge reinforced random walk. We obtain a formula for the speed and give a necessary and sufficient condition for the walk to have a positive speed under some moment conditions on the offspring distribution of the tree.


title: EPPS: Advanced Polyp Segmentation via Edge Information Injection and Selective Feature Decoupling

abstract: Accurate segmentation of polyps in colonoscopy images is essential for early-stage diagnosis and management of colorectal cancer. Despite advancements in deep learning for polyp segmentation, enduring limitations persist. The edges of polyps are typically ambiguous, making them difficult to discern from the background, and the model performance is often compromised by the influence of irrelevant or unimportant features. To alleviate these challenges, we propose a novel model named Edge-Prioritized Polyp Segmentation (EPPS). Specifically, we incorporate an Edge Mapping Engine (EME) aimed at accurately extracting the edges of polyps. Subsequently, an Edge Information Injector (EII) is devised to augment the mask prediction by injecting the captured edge information into Decoder blocks. Furthermore, we introduce a component called Selective Feature Decoupler (SFD) to suppress the influence of noise and extraneous features on the model. Extensive experiments on 3 widely used polyp segmentation benchmarks demonstrate the superior performance of our method compared with other state-of-the-art approaches.


title: Understanding the ultraspherical spectral method

abstract: The ultraspherical spectral method features high accuracy and fast solution. In this article, we determine the sources of error arising from the ultraspherical spectral method and derive its effective condition number, which explains why its backward error is consistent with a numerical method with bounded condition number. In addition, we show the cause for the Cauchy error to go below the machine epsilon and decay eventually to exact zero, revealing the fact that the Cauchy error can be misleading when used as an indicator of convergence and accuracy. The analysis in this work can be readily extended to other spectral methods, when applicable, and to the solution of PDEs.


title: Alternators For Sequence Modeling

abstract: This paper introduces alternators, a novel family of non-Markovian dynamical models for sequences. An alternator features two neural networks: the observation trajectory network (OTN) and the feature trajectory network (FTN). The OTN and the FTN work in conjunction, alternating between outputting samples in the observation space and some feature space, respectively, over a cycle. The parameters of the OTN and the FTN are not time-dependent and are learned via a minimum cross-entropy criterion over the trajectories. Alternators are versatile. They can be used as dynamical latent-variable generative models or as sequence-to-sequence predictors. When alternators are used as generative models, the FTN produces interpretable low-dimensional latent variables that capture the dynamics governing the observations. When alternators are used as sequence-to-sequence predictors, the FTN learns to predict the observed features. In both cases, the OTN learns to produce sequences that match the data. Alternators can uncover the latent dynamics underlying complex sequential data, accurately forecast and impute missing data, and sample new trajectories. We showcase the capabilities of alternators in three applications. We first used alternators to model the Lorenz equations, often used to describe chaotic behavior. We then applied alternators to Neuroscience, to map brain activity to physical activity. Finally, we applied alternators to Climate Science, focusing on sea-surface temperature forecasting. In all our experiments, we found alternators are stable to train, fast to sample from, yield high-quality generated samples and latent variables, and outperform strong baselines such as neural ODEs and diffusion models in the domains we studied.

categories: stat.ML cs.AI cs.LG cs.NE physics.ao-ph q-bio.NC


title: Jumping Automata Must Pay

abstract: Jumping automata are finite automata that read their input in a non-sequential manner, by allowing a reading head to ``jump'' between positions on the input, consuming a permutation of the input word. We argue that allowing the head to jump should incur some cost. To this end, we propose three quantitative semantics for jumping automata, whereby the jumps of the head in an accepting run define the cost of the run. The three semantics correspond to different interpretations of jumps: the \emph{absolute distance} semantics counts the distance the head jumps, the \emph{reversal} semantics counts the number of times the head changes direction, and the \emph{Hamming distance} measures the number of letter-swaps the run makes. We study these measures, with the main focus being the \emph{boundedness problem}: given a jumping automaton, decide whether its (quantitative) languages is bounded by some given number $k$. We establish the decidability and complexity for this problem under several variants.


title: Rethinking Overlooked Aspects in Vision-Language Models

abstract: Recent advancements in large vision-language models (LVLMs), such as GPT4-V and LLaVA, have been substantial. LLaVA's modular architecture, in particular, offers a blend of simplicity and efficiency. Recent works mainly focus on introducing more pre-training and instruction tuning data to improve model's performance. This paper delves into the often-neglected aspects of data efficiency during pre-training and the selection process for instruction tuning datasets. Our research indicates that merely increasing the size of pre-training data does not guarantee improved performance and may, in fact, lead to its degradation. Furthermore, we have established a pipeline to pinpoint the most efficient instruction tuning (SFT) dataset, implying that not all SFT data utilized in existing studies are necessary. The primary objective of this paper is not to introduce a state-of-the-art model, but rather to serve as a roadmap for future research, aiming to optimize data usage during pre-training and fine-tuning processes to enhance the performance of vision-language models.


title: Lower classes and Chung's LILs of the fractional integrated generalized fractional Brownian motion

abstract: Let $\{X(t)\}_{t\geqslant0}$ be the generalized fractional Brownian motion introduced by Pang and Taqqu (2019): \begin{align*} \{X(t)\}_{t\ge0}\overset{d}{=}&\left\{ \int_{\mathbb R} \left((t-u)_+^{\alpha}-(-u)_+^{\alpha} \right) |u|^{-\gamma/2} B(du) \right\}_{t\ge0}, \end{align*} where $ \gamma\in [0,1), \ \ \alpha\in \left(-\frac12+\frac{\gamma}{2}, \ \frac12+\frac{\gamma}{2} \right)$ are constants. For any $\theta>0$, let \begin{align*} Y(t)=\frac{1}{\Gamma(\theta)}\int_0^t (t-u)^{\theta-1} X(u)du, \quad t\ge 0. \end{align*} Building upon the arguments of Talagrand (1996), we give integral criteria for the lower classes of $Y$ at $t=0$ and at infinity, respectively. As a consequence, we derive its Chung-type laws of the iterated logarithm. In the proofs, the small ball probability estimates play important roles.


title: Evolving Storytelling: Benchmarks and Methods for New Character Customization with Diffusion Models

abstract: Diffusion-based models for story visualization have shown promise in generating content-coherent images for storytelling tasks. However, how to effectively integrate new characters into existing narratives while maintaining character consistency remains an open problem, particularly with limited data. Two major limitations hinder the progress: (1) the absence of a suitable benchmark due to potential character leakage and inconsistent text labeling, and (2) the challenge of distinguishing between new and old characters, leading to ambiguous results. To address these challenges, we introduce the NewEpisode benchmark, comprising refined datasets designed to evaluate generative models' adaptability in generating new stories with fresh characters using just a single example story. The refined dataset involves refined text prompts and eliminates character leakage. Additionally, to mitigate the character confusion of generated results, we propose EpicEvo, a method that customizes a diffusion-based visual story generation model with a single story featuring the new characters seamlessly integrating them into established character dynamics. EpicEvo introduces a novel adversarial character alignment module to align the generated images progressively in the diffusive process, with exemplar images of new characters, while applying knowledge distillation to prevent forgetting of characters and background details. Our evaluation quantitatively demonstrates that EpicEvo outperforms existing baselines on the NewEpisode benchmark, and qualitative studies confirm its superior customization of visual story generation in diffusion models. In summary, EpicEvo provides an effective way to incorporate new characters using only one example story, unlocking new possibilities for applications such as serialized cartoons.