title: Stable and Hurwitz slices, a degree principle and a generalized Grace-Walsh-Szeg\H{o} theorem

abstract: Univariate polynomials are called stable with respect to a circular region $\mathcal{A}$, if all of their roots are in $\mathcal{A}$. We consider the special case where $\mathcal{A}$ is a half-plane and investigate affine slices of the set of stable polynomials. In this setup, we show that an affine slice of codimension $k$ always contains a stable polynomial that possesses at most $2(k+2)$ distinct roots on the boundary and at most $(k+2)$ distinct roots in the interior of $\mathcal{A}$. This result also extends to affine slices of weakly Hurwitz polynomials, i.e. real, univariate, left half-plane stable polynomials. Subsequently, we apply these results to symmetric polynomials and varieties. Here we show that a variety described by polynomials in few multiaffine polynomials has no root in $\mathcal{A}^n$, if and only if it has no root in $\mathcal{A}^n$ with few distinct coordinates. This is at the same time a generalization of the degree principle to stable polynomials and a generalization of Grace-Walsh-Szeg\H{o}'s coincidence theorem.


title: Risk-Sensitive Multi-Agent Reinforcement Learning in Network Aggregative Markov Games

abstract: Classical multi-agent reinforcement learning (MARL) assumes risk neutrality and complete objectivity for agents. However, in settings where agents need to consider or model human economic or social preferences, a notion of risk must be incorporated into the RL optimization problem. This will be of greater importance in MARL where other human or non-human agents are involved, possibly with their own risk-sensitive policies. In this work, we consider risk-sensitive and non-cooperative MARL with cumulative prospect theory (CPT), a non-convex risk measure and a generalization of coherent measures of risk. CPT is capable of explaining loss aversion in humans and their tendency to overestimate/underestimate small/large probabilities. We propose a distributed sampling-based actor-critic (AC) algorithm with CPT risk for network aggregative Markov games (NAMGs), which we call Distributed Nested CPT-AC. Under a set of assumptions, we prove the convergence of the algorithm to a subjective notion of Markov perfect Nash equilibrium in NAMGs. The experimental results show that subjective CPT policies obtained by our algorithm can be different from the risk-neutral ones, and agents with a higher loss aversion are more inclined to socially isolate themselves in an NAMG.


title: Transverse Geometry of Lorentzian foliations with applications to Lorentzian orbifolds

abstract: We prove a transverse diameter theorem in the context of Lorentzian foliations, which can be interpreted as a Hawking--Penrose-type singularity theorem for timelike geodesics transverse to the foliation. In order to develop the necessary machinery we introduce and study a novel causality structure on the leaf space via the transverse Lorentzian geometry on the foliated manifold. We describe the initial rungs of a transverse causal ladder and relate them to their standard counterparts on an underlying foliated spacetime. We show how these results can be interpreted as doing Lorentzian (and more generally semi-Riemannian) geometry on low-regularity spaces that can be realized as leaf spaces of foliations. Accordingly, we discuss how all of these concepts and results apply to Lorentzian orbifolds, insofar as these can be seen as leaf spaces of a specific class of Lorentzian foliations. In particular, we derive an associated Lorentzian timelike diameter theorem on orbifolds.


title: Transition dynamics in the $\Lambda_{\rm s}$CDM model: Implications for bound cosmic structures

abstract: We explore the predictions of $\Lambda_{\rm s}$CDM, a novel framework suggesting a rapid anti-de Sitter (AdS) to de Sitter (dS) vacua transition in the late Universe, on bound cosmic structures. In its simplest version, the cosmological constant, $\Lambda_{\rm s}$, abruptly switches sign from negative to positive, attaining its present-day value at a redshift of ${z_\dagger\sim 2}$. The $\Lambda_{\rm s}$CDM model emerges as a promising solution to major cosmological tensions, particularly the $H_0$ and $S_8$ tensions, as well as other less definite tensions. A key aspect of our investigation is examining the impact of the abrupt $\Lambda_{\rm s}$CDM model on the formation and evolution of bound cosmic structures. We identify three primary influences: (i) the negative cosmological constant (AdS) phase for $z > z_\dagger$, (ii) the abrupt transition marked by a type II (sudden) singularity, leading to an abrupt increase in the universe's expansion rate at $z=z_\dagger$, and (iii) an increased expansion rate in the late universe under a positive cosmological constant for $z < z_\dagger$, compared to $\Lambda$CDM. Utilizing the spherical collapse model, we investigate the non-linear evolution of bound cosmic structures within the $\Lambda_{\rm s}$CDM framework. We find that the virialization process of cosmic structures, and consequently their matter overdensity, varies depending on whether the AdS-dS transition precedes or follows the turnaround. Specifically, structures virialize with either increased or reduced matter overdensity compared to the Planck-$\Lambda$CDM model, contingent on the timing of the transition. Additionally, our results demonstrate that the sudden singularity does not result in the dissociation of bound systems. Despite its profound nature, the singularity exerts only relatively weak effects on such systems, thereby reinforcing the model's viability in this context.


title: A Survey on Detection, Classification, and Tracking of Aerial Threats using Radar and Communications Systems

abstract: The use of unmanned aerial vehicles (UAVs) for a variety of commercial, civilian, and defense applications has increased many folds in recent years. While UAVs are expected to transform future air operations, there are instances where they can be used for malicious purposes. In this context, the detection, classification, and tracking (DCT) of UAVs (DCT-U) for safety and surveillance of national air space is a challenging task when compared to DCT of manned aerial vehicles. In this survey, we discuss the threats and challenges from malicious UAVs and we subsequently study three radio frequency (RF)-based systems for DCT-U. These RF-based systems include radars, communication systems, and RF analyzers. Radar systems are further divided into conventional and modern radar systems, while communication systems can be used for joint communications and sensing (JC&S) in active mode and act as a source of illumination to passive radars for DCT-U. The limitations of the three RF-based systems are also provided. The survey briefly discusses non-RF systems for DCT-U and their limitations. Future directions based on the lessons learned are provided at the end of the survey.


title: Background independent tensor networks

abstract: Conventional holographic tensor networks can be described as toy holographic maps constructed from many small linear maps acting in a spatially local way, all connected together with ``background entanglement'', i.e. links of a fixed state, often the maximally entangled state. However, these constructions fall short of modeling real holographic maps. One reason is that their ``areas'' are trivial, taking the same value for all states, unlike in gravity where the geometry is dynamical. Recently, new constructions have ameliorated this issue by adding degrees of freedom that ``live on the links''. This makes areas non-trivial, equal to the background entanglement piece plus a new positive piece that depends on the state of the link degrees of freedom. Nevertheless, this still has the downside that there is background entanglement, and hence it only models relatively limited code subspaces in which every area has a definite minimum value given by the background entanglement. In this note, we simply point out that a version of these constructions goes one step further: they can be background independent, with no background entanglement in the holographic map. This is advantageous because it allows tensor networks to model holographic maps for larger code subspaces. In addition to pointing this out, we address some subtleties involved in making it work and point out a nice connection it offers to recent discussions of random CFT data.


title: Evaluation of the Real-time El Ni\~no Forecasts by the Climate Network Approach between 2011 and Present

abstract: El Ni\~no episodes are part of the El Ni\~no-Southern Oscillation (ENSO), which is the strongest driver of interannual climate variability, and can trigger extreme weather events and disasters in various parts of the globe. Previously we have described a network approach that allows to forecast El Ni\~no events about 1 year ahead. Here we evaluate the real-time forecasts of this approach between 2011 and 2022. We find that the approach correctly predicted (in 2013 and 2017) the onset of both El Ni\~no periods (2014-2016 and 2018-2019) and generated only 1 false alarm in 2019. In June 2022, the approach correctly forecasted the onset of an El Ni\~no event in 2023. We show how to determine the $p$-value of the 12 real-time forecasts between 2011 and 2022 and find $p\cong 0.005$, this way strongly rejecting the null hypothesis that the same quality of the forecast can be obtained by random guessing. We also discuss how the algorithm can be further improved by reducing the number of false alarms in the network model forecast. When combined with other statistical methods, a more detailed forecast, including the magnitude of the event and its type, can be obtained. For 2024, the method indicates the absence of a new El Ni\~no event.


title: Towards a holistic magnetic braking model -- II: explaining several long-term internal- and surface-spin properties of solar-like stars and the Sun

abstract: We extend our model of magnetic braking (MB) from fully convective M-dwarfs (FCMDs) to explain the surface and internal spin $P_\mathrm{spin}$ evolution of partly convective dwarfs (PCDs) starting from disc-dispersal stage to main-sequence turnoff. In our model, the spin of the core is governed by shear at the core-envelope boundary while the spin of the envelope is governed by MB and shear. We show that (1) the most massive FCMDs experience a stronger spin-down than PCDs and less massive FCMDs; (2) the stalled spin-down and enhanced activity of K-dwarfs and the pileup of G-dwarfs older than a few Gyr are stellar-structure- and MB-dependent, and weakly dependent on core-envelope coupling effects; (3) our empirical expression of the core-envelope convergence time-scale $\tau_\mathrm{converge}(M_\ast,\,P_\mathrm{spin})$ between a few 10 to 100 Myr is strongly dependent on stellar structure and weakly dependent on MB strength and shear, where fast and massive rotators achieve corotation earlier; (4) our estimates of the surface magnetic fields are in general agreement with observations and our wind mass loss evolution explains the weak winds from the solar analog $\pi^1$ UMa; (5) the massive young Sun theory as a solution to the faint young Sun problem, which states that the early Sun was sufficiently more massive to maintain liquid water on Earth when the Sun's luminosity would have been about 30 percent lower, can likely be ruled out because the maximum mass lost by winds from our Sun with our model is about $0.001M_\odot$, an order of magnitude smaller than required to solve the problem with this theory.


title: Efficient Stagewise Pretraining via Progressive Subnetworks

abstract: Recent developments in large language models have sparked interest in efficient pretraining methods. A recent effective paradigm is to perform stage-wise training, where the size of the model is gradually increased over the course of training (e.g. gradual stacking (Reddi et al., 2023)). While the resource and wall-time savings are appealing, it has limitations, particularly the inability to evaluate the full model during earlier stages, and degradation in model quality due to smaller model capacity in the initial stages. In this work, we propose an alternative framework, progressive subnetwork training, that maintains the full model throughout training, but only trains subnetworks within the model in each step. We focus on a simple instantiation of this framework, Random Path Training (RaPTr) that only trains a sub-path of layers in each step, progressively increasing the path lengths in stages. RaPTr achieves better pre-training loss for BERT and UL2 language models while requiring 20-33% fewer FLOPs compared to standard training, and is competitive or better than other efficient training methods. Furthermore, RaPTr shows better downstream performance on UL2, improving QA tasks and SuperGLUE by 1-5% compared to standard training and stacking. Finally, we provide a theoretical basis for RaPTr to justify (a) the increasing complexity of subnetworks in stages, and (b) the stability in loss across stage transitions due to residual connections and layer norm.


title: Triangular phase-shift detector for drone precise vertical landing RF systems

abstract: This paper presents a circuit for precise vertical landing of drones based on a three phase-shifts detection of a single frequency transmitted from the landing point. The circuit can be considered as a new navigation sensor that assists in guidance corrections for landing at a specific point. The circuit has three inputs to which the signal transmitted from an oscillator located at the landing point arrives with different delays. The input signals are combined in pairs in each of the three analog phase detectors, after having passed through 3 dB@90 o hybrid couplers that guarantee a theoretical non-ambiguous phase-shift range of +-90 degree. Each output has a voltage that is proportional to the phase-shift between each of the input signals, which in turn depend on the position relative to the landing point. A simple landing algorithm based on phase-shift values is proposed, which could be integrated into the same flight control platform, thus avoiding the need to add additional processing components. To demonstrate the feasibility of the proposed design, a triangular phase-shift detector prototype has been implemented using commercial devices. Calibration and measurements at 2.46 GHz show a dynamic range of 30 dB and a non-ambiguous detection range of +-80 degree in the worst cases. Those specs let us to track the drone during the landing maneuver in an inverted cone formed by a surface with a +-4.19 m radius at 10m high and the landing point.


title: Cosmic-ray induced ionization rates and non-thermal emissions from nuclei of starburst galaxies

abstract: Cosmic rays are the only agent capable of ionizing the interior of dense molecular clouds and, thus, they are believed to play an essential role in determining the physical and chemical evolution of star-forming regions. In this work, we aim to study cosmic-ray induced ionization rates in starburst environments using non-thermal emissions of cosmic rays from starburst nuclei. To this end, we first revisit cosmic-ray models which could explain data of non-thermal emissions from radio to X-ray and gamma-ray from nuclei of three prototypical starburst galaxies NGC 253, M82, and Arp 220. These models are then applied to predict ionization rates in starburst environments which gives values around $10^{-14}$ s$^{-1}$. Such a high value of the ionization rate, which is 2 to 3 orders of magnitude higher than the typical values found in the Milky Way, is probably due to relatively high rates of supernova explosions occurring within the nuclei of these starburst galaxies. We also discuss in more details the case of NGC 253 where our predicted ionization rate is found to be, in most cases, a few times smaller than the values inferred from molecular line observations of clouds in the starburst nucleus. The general framework provided in this work illustrates how the use of non-thermal emission data could help to provide more insights into ionization rates or, more generally, cosmic-ray impact in starburst environments.


title: GenEFT: Understanding Statics and Dynamics of Model Generalization via Effective Theory

abstract: We present GenEFT: an effective theory framework for shedding light on the statics and dynamics of neural network generalization, and illustrate it with graph learning examples. We first investigate the generalization phase transition as data size increases, comparing experimental results with information-theory-based approximations. We find generalization in a Goldilocks zone where the decoder is neither too weak nor too powerful. We then introduce an effective theory for the dynamics of representation learning, where latent-space representations are modeled as interacting particles (repons), and find that it explains our experimentally observed phase transition between generalization and overfitting as encoder and decoder learning rates are scanned. This highlights the power of physics-inspired effective theories for bridging the gap between theoretical predictions and practice in machine learning.


title: Point-VOS: Pointing Up Video Object Segmentation

abstract: Current state-of-the-art Video Object Segmentation (VOS) methods rely on dense per-object mask annotations both during training and testing. This requires time-consuming and costly video annotation mechanisms. We propose a novel Point-VOS task with a spatio-temporally sparse point-wise annotation scheme that substantially reduces the annotation effort. We apply our annotation scheme to two large-scale video datasets with text descriptions and annotate over 19M points across 133K objects in 32K videos. Based on our annotations, we propose a new Point-VOS benchmark, and a corresponding point-based training mechanism, which we use to establish strong baseline results. We show that existing VOS methods can easily be adapted to leverage our point annotations during training, and can achieve results close to the fully-supervised performance when trained on pseudo-masks generated from these points. In addition, we show that our data can be used to improve models that connect vision and language, by evaluating it on the Video Narrative Grounding (VNG) task. We will make our code and annotations available at https://pointvos.github.io.


title: Consensus-driven Deviated Pursuit for Guaranteed Simultaneous Interception of Moving Targets

abstract: This work proposes a cooperative strategy that employs deviated pursuit guidance to simultaneously intercept a moving (but not manoeuvring) target. As opposed to many existing cooperative guidance strategies which use estimates of time-to-go, based on proportional-navigation guidance, the proposed strategy uses an exact expression for time-to-go to ensure simultaneous interception. The guidance design considers nonlinear engagement kinematics, allowing the proposed strategy to remain effective over a large operating regime. Unlike existing strategies on simultaneous interception that achieve interception at the average value of their initial time-to-go estimates, this work provides flexibility in the choice of impact time. By judiciously choosing the edge weights of the communication network, a weighted consensus in time-to-go can be achieved. It has been shown that by allowing an edge weight to be negative, consensus in time-to-go can even be achieved for an impact time that lies outside the convex hull of the set of initial time-to-go values of the individual interceptors. The bounds on such negative weights have been analysed for some special graphs, using Nyquist criterion. Simulations are provided to vindicate the efficacy of the proposed strategy.

categories: eess.SY cs.MA cs.SY math.DS math.OC nlin.AO


title: Collaborative Control for Geometry-Conditioned PBR Image Generation

abstract: Current 3D content generation approaches build on diffusion models that output RGB images. Modern graphics pipelines, however, require physically-based rendering (PBR) material properties. We propose to model the PBR image distribution directly, avoiding photometric inaccuracies in RGB generation and the inherent ambiguity in extracting PBR from RGB. Existing paradigms for cross-modal fine-tuning are not suited for PBR generation due to both a lack of data and the high dimensionality of the output modalities: we overcome both challenges by retaining a frozen RGB model and tightly linking a newly trained PBR model using a novel cross-network communication paradigm. As the base RGB model is fully frozen, the proposed method does not risk catastrophic forgetting during fine-tuning and remains compatible with techniques such as IPAdapter pretrained for the base RGB model. We validate our design choices, robustness to data sparsity, and compare against existing paradigms with an extensive experimental section.


title: Simple Representations of BPS Algebras: the case of $Y(\widehat{\mathfrak{gl}}_2)$

abstract: BPS algebras are the symmetries of a wide class of brane-inspired models. They are closely related to Yangians -- the peculiar and somewhat sophisticated limit of DIM algebras. Still they possess some simple and explicit representations. We explain here that for $Y(\widehat{\mathfrak{gl}}_r)$ these representations are related to Uglov polynomials, whose families are also labeled by natural $r$. They arise in the limit $\hbar\longrightarrow 0$ from Macdonald polynomials, and generalize the well-known Jack polynomials ($\beta$-deformation of Schur functions), associated with $r=1$. For $r=2$ they approximate Macdonald polynomials with the accuracy $O(\hbar^2)$, so that they are eigenfunctions of {\it two} immediately available commuting operators, arising from the $\hbar$-expansion of the first Macdonald Hamiltonian. These operators have a clear structure, which is easily generalizable, -- what provides a technically simple way to build an explicit representation of Yangian $Y(\widehat{\mathfrak{gl}}_2)$, where $U^{(2)}$ are associated with the states $|\lambda\rangle$, parametrized by chess-colored Young diagrams. An interesting feature of this representation is that the odd time-variables $p_{2n+1}$ can be expressed through mutually commuting operators from Yangian, however even time-variables $p_{2n}$ are inexpressible. Implications to higher $r$ become now straightforward, yet we describe them only in a sketchy way.


title: On the Use of Decision Tree Regression for Predicting Vibration Frequency Response of Handheld Probes

abstract: This article focuses on the prediction of the vibration frequency response of handheld probes. A novel approach that involves machine learning and readily available data from probes was explored. Vibration probes are efficient and affordable devices that provide information about testing airborne sound insulation in building acoustics. However, fixing a probe to a vibrating surface downshifts sensor resonances and underestimates levels. Therefore, the calibration response of the sensor included in a probe differs from the frequency response of that same probe. Simulation techniques of complex mechanical systems may describe this issue, but they include hardly obtainable parameters, ultimately restricting the model. Thus, this study discusses an alternative method, which comprises different parts. Firstly, the vibration frequency responses of 85 probes were measured and labelled according to six features. Then, Linear Regression, Decision Tree Regression and Artificial Neural Networks algorithms were analysed. It was revealed that decision tree regression is the more appropriate technique for this data. The best decision tree models, in terms of scores and model structure, were fine-tuned. Eventually, the final suggested model employs only four out of the six original features. A trade-off solution that involved a simple structure, an interpretable model and accurate predictions was accomplished. It showed a maximum average deviation from test measurements ranging from 0.6 dB in low-frequency to 3 dB in high-frequency while remaining at a low computational load. This research developed an original and reliable prediction tool that provides the vibration frequency response of handheld probes.


title: Horizontal branch structure, age, and chemical composition for very metal-poor extragalactic globular clusters

abstract: This paper presents the results of analysing the integrated light (IL) low-resolution spectra of globular clusters (GCs) in the M31 and Centaurus A groups of galaxies. The sample consists of eight very metal-poor GCs ($\rm [Fe/H]\le -2$ dex) with high signal-to-noise ratio spectra acquired with the telescopes: the 6-m SAO RAS (BTA), the Southern African Large (SALT) and the 6.5-m Magellan (MMT). We study the influence of contribution of the horizontal branch stars on the hydrogen Balmer line profiles in the IL spectra. By modelling the Balmer lines, as well as the metal lines in the observed spectra, we determine the optimum parameters of stellar evolution isochrones and, consequently, the parameters of the atmospheres of the cluster stars. For all the studied GCs, the parameters of horizontal branch stars set by the selected isochrones, the corresponding ages, and carbon abundances are presented for the first time. The abundances of several other elements (Mg, Ca, Ti, Cr, and Mn) were determined for five GCs for the first time. All the studied GCs have blue horizontal branches and are older than 10 Gyr. Their chemical abundances, with the exception of Mg and Mn, are in good agreement with the abundances of stars in the Galactic field. The reasons of low [Mg/Fe] and of high [Mn/Fe] are discussed. Study of the fundamental properties of stellar populations in old globular clusters facilitates a better understanding of the formation processes of their parent galaxies and nucleosynthesis in the early Universe.


title: Mean field control of droplet dynamics with high order finite element computations

abstract: Liquid droplet dynamics are widely used in biological and engineering applications, which contain complex interfacial instabilities and pattern formulation such as droplet merging, splitting, and transport. This paper studies a class of mean field control formulation towards these droplet dynamics. They are used to control and maintain the manipulation of droplets in applications. We first formulate the droplet dynamics as gradient flows of free energies in modified optimal transport metrics with nonlinear mobilities. We then design an optimal control problem for these gradient flows. We lastly apply the primal-dual hybrid gradient algorithm with high-order finite element methods to simulate the proposed mean field control problems. Numerical examples, including droplet formation, bead-up/spreading, transport, and merging/splitting on a two-dimensional spatial domain, demonstrate the effectiveness of the proposed mean field control mechanism.


title: Understanding Social Immunity in Ants: A Markovian Approach to Collective Cleaning Strategies

abstract: Understanding social immunity mechanisms in ant colonies remains crucial to comprehending the evolution of defense strategies in eusocial organisms. This study assumes the absence of the role of memory in the ants' defense strategy, considering that they can make a new exploration of collective cleaning behaviors. We investigate how worker interactions and previous behaviors influence the evolution of social immunity strategies in response to the presence of pathogens and vulnerable colony members. In this context, with the application of Markov transition models it was possible to describe changes in cleaning behavior over time influenced by the presence of vulnerable members and treatment with an entomopathogenic fungus. We found a significant effect on ant behavior when exposed to \it{Metarhizium anisopliae} when a fungus garden fragment and one larva were present. Our findings confirm a link between prophylactic cleaning and the presence of vulnerable members. Remarkably, distinct behaviors and transition times vary among treatments, revealing workers' adaptability to threats. Allogrooming shows adaptive changes when exposed to pathogens, potentially affecting pathogen transmission. In addition, our study elucidates the intricate interaction between internal and external factors shaping worker behavior. The influence of environmental context on decision-making principles emerges, emphasizing the importance of both intrinsic colony organization and external threats. By using a Markov model to understand ant hygiene behavior, we offer insights into social immunity mechanisms in eusocial organisms. Deciphering collective cleaning strategies can aid in understanding disease dynamics and decision-making processes in complex societies.


title: YOLO-CIANNA: Galaxy detection with deep learning in radio data. I. A new YOLO-inspired source detection method applied to the SKAO SDC1

abstract: The upcoming Square Kilometer Array (SKA) will set a new standard regarding data volume generated by an astronomical instrument, which is likely to challenge widely adopted data analysis tools that scale inadequately with the data size. This study aims to develop a new source detection and characterization method for massive radio astronomical datasets by adapting modern deep-learning object detection techniques. These approaches have proved their efficiency on complex computer vision tasks, and we seek to identify their specific strengths and weaknesses when applied to astronomical data. We introduce YOLO-CIANNA, a highly customized deep-learning object detector designed specifically for astronomical datasets. This paper presents the method and describes all the low-level adaptations required to address the specific challenges of radio-astronomical images. We demonstrate this method's capabilities using simulated 2D continuum images from the SKAO SDC1 dataset. Our method outperforms every other published result on the specific SDC1 dataset. Using the SDC1 metric, we improve the challenge-winning score by +139\% and the score of the only other post-challenge participation by +61\%. Our catalog has a detection purity of 94\% while detecting 40 to 60 \% more sources than previous top-score results. The trained model can also be forced to reach 99\% purity in post-process and still detect 10 to 30\% more sources than the other top-score methods. It is also capable of real-time detection, with a peak prediction speed of 500 images of 512x512 pixels per second on a single GPU. YOLO-CIANNA achieves state-of-the-art detection and characterization results on the simulated SDC1 dataset. The method is open source and included in the wider CIANNA framework. We provide scripts to train and apply this method to the SDC1 dataset in the CIANNA repository.


title: On the Convergence of Zeroth-Order Federated Tuning for Large Language Models

abstract: The confluence of Federated Learning (FL) and Large Language Models (LLMs) is ushering in a new era in privacy-preserving natural language processing. However, the intensive memory requirements for fine-tuning LLMs pose significant challenges, especially when deploying on clients with limited computational resources. To circumvent this, we explore the novel integration of Memory-efficient Zeroth-Order Optimization within a federated setting, a synergy we term as FedMeZO. Our study is the first to examine the theoretical underpinnings of FedMeZO in the context of LLMs, tackling key questions regarding the influence of large parameter spaces on optimization behavior, the establishment of convergence properties, and the identification of critical parameters for convergence to inform personalized federated strategies. Our extensive empirical evidence supports the theory, showing that FedMeZO not only converges faster than traditional first-order methods such as FedAvg but also significantly reduces GPU memory usage during training to levels comparable to those during inference. Moreover, the proposed personalized FL strategy that is built upon the theoretical insights to customize the client-wise learning rate can effectively accelerate loss reduction. We hope our work can help to bridge theoretical and practical aspects of federated fine-tuning for LLMs, thereby stimulating further advancements and research in this area.


title: Yamabe metrics on conical manifolds

abstract: We prove existence of Yamabe metrics on singular manifolds with conical points and conical links of Einstein type that include orbifold structures. We deal with metrics of generic type and derive a counterpart of Aubin's classical result. Interestingly, the singular nature of the metric determines a different condition on the dimension, compared to the regular case. We derive asymptotic expansions on the Yamabe quotient by adding a proper and implicit lower-order correction to standard bubbles, whose contribution to the expansion of the quotient can be determined combining the decomposition of symmetric two-tensor fields and Fourier analysis on the conical links.


title: Sharp Rates in Dependent Learning Theory: Avoiding Sample Size Deflation for the Square Loss

abstract: In this work, we study statistical learning with dependent ($\beta$-mixing) data and square loss in a hypothesis class $\mathscr{F}\subset L_{\Psi_p}$ where $\Psi_p$ is the norm $\|f\|_{\Psi_p} \triangleq \sup_{m\geq 1} m^{-1/p} \|f\|_{L^m} $ for some $p\in [2,\infty]$. Our inquiry is motivated by the search for a sharp noise interaction term, or variance proxy, in learning with dependent data. Absent any realizability assumption, typical non-asymptotic results exhibit variance proxies that are deflated multiplicatively by the mixing time of the underlying covariates process. We show that whenever the topologies of $L^2$ and $\Psi_p$ are comparable on our hypothesis class $\mathscr{F}$ -- that is, $\mathscr{F}$ is a weakly sub-Gaussian class: $\|f\|_{\Psi_p} \lesssim \|f\|_{L^2}^\eta$ for some $\eta\in (0,1]$ -- the empirical risk minimizer achieves a rate that only depends on the complexity of the class and second order statistics in its leading term. Our result holds whether the problem is realizable or not and we refer to this as a \emph{near mixing-free rate}, since direct dependence on mixing is relegated to an additive higher order term. We arrive at our result by combining the above notion of a weakly sub-Gaussian class with mixed tail generic chaining. This combination allows us to compute sharp, instance-optimal rates for a wide range of problems. Examples that satisfy our framework include sub-Gaussian linear regression, more general smoothly parameterized function classes, finite hypothesis classes, and bounded smoothness classes.


title: An Interactive Agent Foundation Model

abstract: The development of artificial intelligence systems is transitioning from creating static, task-specific models to dynamic, agent-based systems capable of performing well in a wide range of applications. We propose an Interactive Agent Foundation Model that uses a novel multi-task agent training paradigm for training AI agents across a wide range of domains, datasets, and tasks. Our training paradigm unifies diverse pre-training strategies, including visual masked auto-encoders, language modeling, and next-action prediction, enabling a versatile and adaptable AI framework. We demonstrate the performance of our framework across three separate domains -- Robotics, Gaming AI, and Healthcare. Our model demonstrates its ability to generate meaningful and contextually relevant outputs in each area. The strength of our approach lies in its generality, leveraging a variety of data sources such as robotics sequences, gameplay data, large-scale video datasets, and textual information for effective multimodal and multi-task learning. Our approach provides a promising avenue for developing generalist, action-taking, multimodal systems.


title: WebLINX: Real-World Website Navigation with Multi-Turn Dialogue

abstract: We propose the problem of conversational web navigation, where a digital agent controls a web browser and follows user instructions to solve real-world tasks in a multi-turn dialogue fashion. To support this problem, we introduce WEBLINX - a large-scale benchmark of 100K interactions across 2300 expert demonstrations of conversational web navigation. Our benchmark covers a broad range of patterns on over 150 real-world websites and can be used to train and evaluate agents in diverse scenarios. Due to the magnitude of information present, Large Language Models (LLMs) cannot process entire web pages in real-time. To solve this bottleneck, we design a retrieval-inspired model that efficiently prunes HTML pages by ranking relevant elements. We use the selected elements, along with screenshots and action history, to assess a variety of models for their ability to replicate human behavior when navigating the web. Our experiments span from small text-only to proprietary multimodal LLMs. We find that smaller finetuned decoders surpass the best zero-shot LLMs (including GPT-4V), but also larger finetuned multimodal models which were explicitly pretrained on screenshots. However, all finetuned models struggle to generalize to unseen websites. Our findings highlight the need for large multimodal models that can generalize to novel settings. Our code, data and models are available for research: https://mcgill-nlp.github.io/weblinx


title: Topological Kondo effect with spinful Majorana fermions

abstract: Motivated by the importance of studying topological superconductors beyond the mean-field approximation, we here investigate mesoscopic islands of time reversal invariant topological superconductors (TRITOPS). We characterize the spectrum in the presence of strong order parameter fluctuations in the presence of an arbitrary number of Kramers pairs of Majorana edge states and study the effect of coupling the Coulomb blockaded island to external leads. In the case of an odd fermionic parity on the island, we derive an unconventional Kondo Hamiltonian in which metallic leads couple to both topological Majorana degrees of freedom (which keep track of the parity in different leads) and the overall spin-1/2 in the island. For the simplest case of a single wire (two pairs of Majorana edge states), we demonstrate that anisotropies are irrelevant in the weak coupling renormalization group flow. This permits us to solve the Kondo problem in the vicinity of a Toulouse-like point using Abelian Bosonization. We demonstrate a residual ground state entropy of $\ln(2)$, which is protected by spin-rotation symmetry, but reduced to $\ln(\sqrt{2})$ (as in the spinless topological Kondo effect) by symmetry breaking perturbations. In the symmetric case, we further demonstrate the simultaneous presence of both Fermi liquid and non-Fermi liquid like thermodynamics (depending on the observable) and derive charge and spin transport signatures of the Coulomb blockaded island.


title: Driving Everywhere with Large Language Model Policy Adaptation

abstract: Adapting driving behavior to new environments, customs, and laws is a long-standing problem in autonomous driving, precluding the widespread deployment of autonomous vehicles (AVs). In this paper, we present LLaDA, a simple yet powerful tool that enables human drivers and autonomous vehicles alike to drive everywhere by adapting their tasks and motion plans to traffic rules in new locations. LLaDA achieves this by leveraging the impressive zero-shot generalizability of large language models (LLMs) in interpreting the traffic rules in the local driver handbook. Through an extensive user study, we show that LLaDA's instructions are useful in disambiguating in-the-wild unexpected situations. We also demonstrate LLaDA's ability to adapt AV motion planning policies in real-world datasets; LLaDA outperforms baseline planning approaches on all our metrics. Please check our website for more details: https://boyiliee.github.io/llada.


title: Time Series Diffusion in the Frequency Domain

abstract: Fourier analysis has been an instrumental tool in the development of signal processing. This leads us to wonder whether this framework could similarly benefit generative modelling. In this paper, we explore this question through the scope of time series diffusion models. More specifically, we analyze whether representing time series in the frequency domain is a useful inductive bias for score-based diffusion models. By starting from the canonical SDE formulation of diffusion in the time domain, we show that a dual diffusion process occurs in the frequency domain with an important nuance: Brownian motions are replaced by what we call mirrored Brownian motions, characterized by mirror symmetries among their components. Building on this insight, we show how to adapt the denoising score matching approach to implement diffusion models in the frequency domain. This results in frequency diffusion models, which we compare to canonical time diffusion models. Our empirical evaluation on real-world datasets, covering various domains like healthcare and finance, shows that frequency diffusion models better capture the training distribution than time diffusion models. We explain this observation by showing that time series from these datasets tend to be more localized in the frequency domain than in the time domain, which makes them easier to model in the former case. All our observations point towards impactful synergies between Fourier analysis and diffusion models.


title: Classifying Nodes in Graphs without GNNs

abstract: Graph neural networks (GNNs) are the dominant paradigm for classifying nodes in a graph, but they have several undesirable attributes stemming from their message passing architecture. Recently, distillation methods succeeded in eliminating the use of GNNs at test time but they still require them during training. We perform a careful analysis of the role that GNNs play in distillation methods. This analysis leads us to propose a fully GNN-free approach for node classification, not requiring them at train or test time. Our method consists of three key components: smoothness constraints, pseudo-labeling iterations and neighborhood-label histograms. Our final approach can match the state-of-the-art accuracy on standard popular benchmarks such as citation and co-purchase networks, without training a GNN.


title: SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models

abstract: We propose SPHINX-X, an extensive Multimodality Large Language Model (MLLM) series developed upon SPHINX. To improve the architecture and training efficiency, we modify the SPHINX framework by removing redundant visual encoders, bypassing fully-padded sub-images with skip tokens, and simplifying multi-stage training into a one-stage all-in-one paradigm. To fully unleash the potential of MLLMs, we assemble a comprehensive multi-domain and multimodal dataset covering publicly available resources in language, vision, and vision-language tasks. We further enrich this collection with our curated OCR intensive and Set-of-Mark datasets, extending the diversity and generality. By training over different base LLMs including TinyLlama1.1B, InternLM2-7B, LLaMA2-13B, and Mixtral8x7B, we obtain a spectrum of MLLMs that vary in parameter size and multilingual capabilities. Comprehensive benchmarking reveals a strong correlation between the multi-modal performance with the data and parameter scales. Code and models are released at https://github.com/Alpha-VLLM/LLaMA2-Accessory


title: Closure properties in positively decreasing and related distributions under dependence

abstract: We consider closure properties in the class of positively decreasing distributions. Our results stem from different types of dependence, but each type belongs in the family of asymptotically independent dependence structure. Namely we examine the closure property with respect to minimum, maximum, convolution product and convolution. Furthermore, we take into account some closure properties of the class of generalized subexponential positively decreasing distributions, as also we introduce and study the class of the generalized long-tailed positively decreasing distributions. Also we revisited the (independent) convolution closure problem of subexponentiality, in the case of subexponential positively decreasing class. In some classes we discuss the closedness of randomly stopped sums. In the last section we study the closure property with respect to minimum for two classes of random vectors.


title: InstaGen: Enhancing Object Detection by Training on Synthetic Dataset

abstract: In this paper, we present a novel paradigm to enhance the ability of object detector, e.g., expanding categories or improving detection performance, by training on synthetic dataset generated from diffusion models. Specifically, we integrate an instance-level grounding head into a pre-trained, generative diffusion model, to augment it with the ability of localising instances in the generated images. The grounding head is trained to align the text embedding of category names with the regional visual feature of the diffusion model, using supervision from an off-the-shelf object detector, and a novel self-training scheme on (novel) categories not covered by the detector. We conduct thorough experiments to show that, this enhanced version of diffusion model, termed as InstaGen, can serve as a data synthesizer, to enhance object detectors by training on its generated samples, demonstrating superior performance over existing state-of-the-art methods in open-vocabulary (+4.5 AP) and data-sparse (+1.2 to 5.2 AP) scenarios. Project page with code: https://fcjian.github.io/InstaGen.


title: The Jackson-Richmond 4CT Constant is EXACTLY 10/27

abstract: In their recent claimed computer-free proof of the Four Color Theorem, David Jackson and Bruce Richmond attempted to use sophisticated "asymptotic analysis" to explicitly compute a certain number whose positivity (according to them) implies this famous theorem. While the jury is still out whether their valiant attempt holds water, we prove, in this modest note, that this constant equals exactly 10/27. We also point out that their evaluation of this constant must be erroneous, for two good reasons. Finally, as an encore, we state many similar, but more complicated, results.


title: Uncertainty Awareness of Large Language Models Under Code Distribution Shifts: A Benchmark Study

abstract: Large Language Models (LLMs) have been widely employed in programming language analysis to enhance human productivity. Yet, their reliability can be compromised by various code distribution shifts, leading to inconsistent outputs. While probabilistic methods are known to mitigate such impact through uncertainty calibration and estimation, their efficacy in the language domain remains underexplored compared to their application in image-based tasks. In this work, we first introduce a large-scale benchmark dataset, incorporating three realistic patterns of code distribution shifts at varying intensities. Then we thoroughly investigate state-of-the-art probabilistic methods applied to CodeLlama using these shifted code snippets. We observe that these methods generally improve the uncertainty awareness of CodeLlama, with increased calibration quality and higher uncertainty estimation~(UE) precision. However, our study further reveals varied performance dynamics across different criteria (e.g., calibration error vs misclassification detection) and trade-off between efficacy and efficiency, highlighting necessary methodological selection tailored to specific contexts.


title: Causal Relationship Network of Risk Factors Impacting Workday Loss in Underground Coal Mines

abstract: This study aims to establish the causal relationship network between various factors leading to workday loss in underground coal mines using a novel causal artificial intelligence (AI) method. The analysis utilizes data obtained from the National Institute for Occupational Safety and Health (NIOSH). A total of 101,010 injury records from 3,982 unique underground coal mines spanning the years from 1990 to 2020 were extracted from the NIOSH database. Causal relationships were analyzed and visualized using a novel causal AI method called Grouped Greedy Equivalence Search (GGES). The impact of each variable on workday loss was assessed through intervention do-calculus adjustment (IDA) scores. Model training and validation were performed using the 10-fold cross-validation technique. Performance metrics, including adjacency precision (AP), adjacency recall (AR), arrowhead precision (AHP), and arrowhead recall (AHR), were utilized to evaluate the models. Findings revealed that after 2006, key direct causes of workday loss among mining employees included total mining experience, mean office employees, mean underground employees, county, and total mining experience (years). Total mining experience emerged as the most influential factor, whereas mean employees per mine exhibited the least influence. The analyses emphasized the significant role of total mining experience in determining workday loss. The models achieved optimal performance, with AP, AR, AHP, and AHR values measuring 0.694, 0.653, 0.386, and 0.345, respectively. This study demonstrates the feasibility of utilizing the new GGES method to clarify the causal factors behind the workday loss by analyzing employment demographics and injury records and establish their causal relationship network.


title: Character-based Outfit Generation with Vision-augmented Style Extraction via LLMs

abstract: The outfit generation problem involves recommending a complete outfit to a user based on their interests. Existing approaches focus on recommending items based on anchor items or specific query styles but do not consider customer interests in famous characters from movie, social media, etc. In this paper, we define a new Character-based Outfit Generation (COG) problem, designed to accurately interpret character information and generate complete outfit sets according to customer specifications such as age and gender. To tackle this problem, we propose a novel framework LVA-COG that leverages Large Language Models (LLMs) to extract insights from customer interests (e.g., character information) and employ prompt engineering techniques for accurate understanding of customer preferences. Additionally, we incorporate text-to-image models to enhance the visual understanding and generation (factual or counterfactual) of cohesive outfits. Our framework integrates LLMs with text-to-image models and improves the customer's approach to fashion by generating personalized recommendations. With experiments and case studies, we demonstrate the effectiveness of our solution from multiple dimensions.


title: Cooperative Knowledge Distillation: A Learner Agnostic Approach

abstract: Knowledge distillation is a simple but powerful way to transfer knowledge between a teacher model to a student model. Existing work suffers from at least one of the following key limitations in terms of direction and scope of transfer which restrict its use: all knowledge is transferred from teacher to student regardless of whether or not that knowledge is useful, the student is the only one learning in this exchange, and typically distillation transfers knowledge only from a single teacher to a single student. We formulate a novel form of knowledge distillation in which many models can act as both students and teachers which we call cooperative distillation. The models cooperate as follows: a model (the student) identifies specific deficiencies in it's performance and searches for another model (the teacher) who encodes learned knowledge into instructional virtual instances via counterfactual instance generation. Because different models may have different strengths and weaknesses, all models can act as either students or teachers (cooperation) when appropriate and only distill knowledge in areas specific to their strengths (focus). Since counterfactuals as a paradigm are not tied to any specific algorithm, we can use this method to distill knowledge between learners of different architectures, algorithms, and even feature spaces. We demonstrate that our approach not only outperforms baselines such as transfer learning, self-supervised learning, and multiple knowledge distillation algorithms on several datasets, but it can also be used in settings where the aforementioned techniques cannot.


title: A hybrid IndRNNLSTM approach for real-time anomaly detection in software-defined networks

abstract: Anomaly detection in SDN using data flow prediction is a difficult task. This problem is included in the category of time series and regression problems. Machine learning approaches are challenging in this field due to the manual selection of features. On the other hand, deep learning approaches have important features due to the automatic selection of features. Meanwhile, RNN-based approaches have been used the most. The LSTM and GRU approaches learn dependent entities well; on the other hand, the IndRNN approach learns non-dependent entities in time series. The proposed approach tried to use a combination of IndRNN and LSTM approaches to learn dependent and non-dependent features. Feature selection approaches also provide a suitable view of features for the models; for this purpose, four feature selection models, Filter, Wrapper, Embedded, and Autoencoder were used. The proposed IndRNNLSTM algorithm, in combination with Embedded, was able to achieve MAE=1.22 and RMSE=9.92 on NSL-KDD data.


title: Todyformer: Towards Holistic Dynamic Graph Transformers with Structure-Aware Tokenization

abstract: Temporal Graph Neural Networks have garnered substantial attention for their capacity to model evolving structural and temporal patterns while exhibiting impressive performance. However, it is known that these architectures are encumbered by issues that constrain their performance, such as over-squashing and over-smoothing. Meanwhile, Transformers have demonstrated exceptional computational capacity to effectively address challenges related to long-range dependencies. Consequently, we introduce Todyformer-a novel Transformer-based neural network tailored for dynamic graphs. It unifies the local encoding capacity of Message-Passing Neural Networks (MPNNs) with the global encoding of Transformers through i) a novel patchifying paradigm for dynamic graphs to improve over-squashing, ii) a structure-aware parametric tokenization strategy leveraging MPNNs, iii) a Transformer with temporal positional-encoding to capture long-range dependencies, and iv) an encoding architecture that alternates between local and global contextualization, mitigating over-smoothing in MPNNs. Experimental evaluations on public benchmark datasets demonstrate that Todyformer consistently outperforms the state-of-the-art methods for downstream tasks. Furthermore, we illustrate the underlying aspects of the proposed model in effectively capturing extensive temporal dependencies in dynamic graphs.


title: Eliminating Information Leakage in Hard Concept Bottleneck Models with Supervised, Hierarchical Concept Learning

abstract: Concept Bottleneck Models (CBMs) aim to deliver interpretable and interventionable predictions by bridging features and labels with human-understandable concepts. While recent CBMs show promising potential, they suffer from information leakage, where unintended information beyond the concepts (either when concepts are represented with probabilities or binary states) are leaked to the subsequent label prediction. Consequently, distinct classes are falsely classified via indistinguishable concepts, undermining the interpretation and intervention of CBMs. This paper alleviates the information leakage issue by introducing label supervision in concept predication and constructing a hierarchical concept set. Accordingly, we propose a new paradigm of CBMs, namely SupCBM, which achieves label predication via predicted concepts and a deliberately-designed intervention matrix. SupCBM focuses on concepts that are mostly relevant to the predicted label and only distinguishes classes when different concepts are presented. Our evaluations show that SupCBM outperforms SOTA CBMs over diverse datasets. It also manifests better generality across different backbone models. With proper quantification of information leakage in different CBMs, we demonstrate that SupCBM significantly reduces the information leakage.


title: Unveiling Latent Causal Rules: A Temporal Point Process Approach for Abnormal Event Explanation

abstract: In high-stakes systems such as healthcare, it is critical to understand the causal reasons behind unusual events, such as sudden changes in patient's health. Unveiling the causal reasons helps with quick diagnoses and precise treatment planning. In this paper, we propose an automated method for uncovering "if-then" logic rules to explain observational events. We introduce temporal point processes to model the events of interest, and discover the set of latent rules to explain the occurrence of events. To achieve this, we employ an Expectation-Maximization (EM) algorithm. In the E-step, we calculate the likelihood of each event being explained by each discovered rule. In the M-step, we update both the rule set and model parameters to enhance the likelihood function's lower bound. Notably, we optimize the rule set in a differential manner. Our approach demonstrates accurate performance in both discovering rules and identifying root causes. We showcase its promising results using synthetic and real healthcare datasets.


title: Separable Multi-Concept Erasure from Diffusion Models

abstract: Large-scale diffusion models, known for their impressive image generation capabilities, have raised concerns among researchers regarding social impacts, such as the imitation of copyrighted artistic styles. In response, existing approaches turn to machine unlearning techniques to eliminate unsafe concepts from pre-trained models. However, these methods compromise the generative performance and neglect the coupling among multi-concept erasures, as well as the concept restoration problem. To address these issues, we propose a Separable Multi-concept Eraser (SepME), which mainly includes two parts: the generation of concept-irrelevant representations and the weight decoupling. The former aims to avoid unlearning substantial information that is irrelevant to forgotten concepts. The latter separates optimizable model weights, making each weight increment correspond to a specific concept erasure without affecting generative performance on other concepts. Specifically, the weight increment for erasing a specified concept is formulated as a linear combination of solutions calculated based on other known undesirable concepts. Extensive experiments indicate the efficacy of our approach in eliminating concepts, preserving model performance, and offering flexibility in the erasure or recovery of various concepts.


title: DE$^3$-BERT: Distance-Enhanced Early Exiting for BERT based on Prototypical Networks

abstract: Early exiting has demonstrated its effectiveness in accelerating the inference of pre-trained language models like BERT by dynamically adjusting the number of layers executed. However, most existing early exiting methods only consider local information from an individual test sample to determine their exiting indicators, failing to leverage the global information offered by sample population. This leads to suboptimal estimation of prediction correctness, resulting in erroneous exiting decisions. To bridge the gap, we explore the necessity of effectively combining both local and global information to ensure reliable early exiting during inference. Purposefully, we leverage prototypical networks to learn class prototypes and devise a distance metric between samples and class prototypes. This enables us to utilize global information for estimating the correctness of early predictions. On this basis, we propose a novel Distance-Enhanced Early Exiting framework for BERT (DE$^3$-BERT). DE$^3$-BERT implements a hybrid exiting strategy that supplements classic entropy-based local information with distance-based global information to enhance the estimation of prediction correctness for more reliable early exiting decisions. Extensive experiments on the GLUE benchmark demonstrate that DE$^3$-BERT consistently outperforms state-of-the-art models under different speed-up ratios with minimal storage or computational overhead, yielding a better trade-off between model performance and inference efficiency. Additionally, an in-depth analysis further validates the generality and interpretability of our method.


title: An explainable machine learning-based approach for analyzing customers' online data to identify the importance of product attributes

abstract: Online customer data provides valuable information for product design and marketing research, as it can reveal the preferences of customers. However, analyzing these data using artificial intelligence (AI) for data-driven design is a challenging task due to potential concealed patterns. Moreover, in these research areas, most studies are only limited to finding customers' needs. In this study, we propose a game theory machine learning (ML) method that extracts comprehensive design implications for product development. The method first uses a genetic algorithm to select, rank, and combine product features that can maximize customer satisfaction based on online ratings. Then, we use SHAP (SHapley Additive exPlanations), a game theory method that assigns a value to each feature based on its contribution to the prediction, to provide a guideline for assessing the importance of each feature for the total satisfaction. We apply our method to a real-world dataset of laptops from Kaggle, and derive design implications based on the results. Our approach tackles a major challenge in the field of multi-criteria decision making and can help product designers and marketers, to understand customer preferences better with less data and effort. The proposed method outperforms benchmark methods in terms of relevant performance metrics.


title: SQT -- std $Q$-target

abstract: Std $Q$-target is a conservative, actor-critic, ensemble, $Q$-learning-based algorithm, which is based on a single key $Q$-formula: $Q$-networks standard deviation, which is an "uncertainty penalty", and, serves as a minimalistic solution to the problem of overestimation bias. We implement SQT on top of TD3/TD7 code and test it against the state-of-the-art (SOTA) actor-critic algorithms, DDPG, TD3 and TD7 on seven popular MuJoCo and Bullet tasks. Our results demonstrate SQT's $Q$-target formula superiority over TD3's $Q$-target formula as a conservative solution to overestimation bias in RL, while SQT shows a clear performance advantage on a wide margin over DDPG, TD3, and TD7 on all tasks.


title: MinMaxMin $Q$-learning

abstract: MinMaxMin $Q$-learning is a novel optimistic Actor-Critic algorithm that addresses the problem of overestimation bias ($Q$-estimations are overestimating the real $Q$-values) inherent in conservative RL algorithms. Its core formula relies on the disagreement among $Q$-networks in the form of the min-batch MaxMin $Q$-networks distance which is added to the $Q$-target and used as the priority experience replay sampling-rule. We implement MinMaxMin on top of TD3 and TD7, subjecting it to rigorous testing against state-of-the-art continuous-space algorithms-DDPG, TD3, and TD7-across popular MuJoCo and Bullet environments. The results show a consistent performance improvement of MinMaxMin over DDPG, TD3, and TD7 across all tested tasks.


title: Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques

abstract: The integration of Large Language Models (LLMs) with Graph Representation Learning (GRL) marks a significant evolution in analyzing complex data structures. This collaboration harnesses the sophisticated linguistic capabilities of LLMs to improve the contextual understanding and adaptability of graph models, thereby broadening the scope and potential of GRL. Despite a growing body of research dedicated to integrating LLMs into the graph domain, a comprehensive review that deeply analyzes the core components and operations within these models is notably lacking. Our survey fills this gap by proposing a novel taxonomy that breaks down these models into primary components and operation techniques from a novel technical perspective. We further dissect recent literature into two primary components including knowledge extractors and organizers, and two operation techniques including integration and training stratigies, shedding light on effective model design and training strategies. Additionally, we identify and explore potential future research avenues in this nascent yet underexplored field, proposing paths for continued progress.


title: idMotif: An Interactive Motif Identification in Protein Sequences

abstract: This article introduces idMotif, a visual analytics framework designed to aid domain experts in the identification of motifs within protein sequences. Motifs, short sequences of amino acids, are critical for understanding the distinct functions of proteins. Identifying these motifs is pivotal for predicting diseases or infections. idMotif employs a deep learning-based method for the categorization of protein sequences, enabling the discovery of potential motif candidates within protein groups through local explanations of deep learning model decisions. It offers multiple interactive views for the analysis of protein clusters or groups and their sequences. A case study, complemented by expert feedback, illustrates idMotif's utility in facilitating the analysis and identification of protein sequences and motifs.


title: EasyFS: an Efficient Model-free Feature Selection Framework via Elastic Transformation of Features

abstract: Traditional model-free feature selection methods treat each feature independently while disregarding the interrelationships among features, which leads to relatively poor performance compared with the model-aware methods. To address this challenge, we propose an efficient model-free feature selection framework via elastic expansion and compression of the features, namely EasyFS, to achieve better performance than state-of-the-art model-aware methods while sharing the characters of efficiency and flexibility with the existing model-free methods. In particular, EasyFS expands the feature space by using the random non-linear projection network to achieve the non-linear combinations of the original features, so as to model the interrelationships among the features and discover most correlated features. Meanwhile, a novel redundancy measurement based on the change of coding rate is proposed for efficient filtering of redundant features. Comprehensive experiments on 21 different datasets show that EasyFS outperforms state-of-the art methods up to 10.9\% in the regression tasks and 5.7\% in the classification tasks while saving more than 94\% of the time.


title: A Hyper-Transformer model for Controllable Pareto Front Learning with Split Feasibility Constraints

abstract: Controllable Pareto front learning (CPFL) approximates the Pareto solution set and then locates a Pareto optimal solution with respect to a given reference vector. However, decision-maker objectives were limited to a constraint region in practice, so instead of training on the entire decision space, we only trained on the constraint region. Controllable Pareto front learning with Split Feasibility Constraints (SFC) is a way to find the best Pareto solutions to a split multi-objective optimization problem that meets certain constraints. In the previous study, CPFL used a Hypernetwork model comprising multi-layer perceptron (Hyper-MLP) blocks. With the substantial advancement of transformer architecture in deep learning, transformers can outperform other architectures in various tasks. Therefore, we have developed a hyper-transformer (Hyper-Trans) model for CPFL with SFC. We use the theory of universal approximation for the sequence-to-sequence function to show that the Hyper-Trans model makes MED errors smaller in computational experiments than the Hyper-MLP model.


title: Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting

abstract: Transformers for time series forecasting mainly model time series from limited or fixed scales, making it challenging to capture different characteristics spanning various scales. We propose Pathformer, a multi-scale Transformer with adaptive pathways. It integrates both temporal resolution and temporal distance for multi-scale modeling. Multi-scale division divides the time series into different temporal resolutions using patches of various sizes. Based on the division of each scale, dual attention is performed over these patches to capture global correlations and local details as temporal dependencies. We further enrich the multi-scale Transformer with adaptive pathways, which adaptively adjust the multi-scale modeling process based on the varying temporal dynamics of the input, improving the accuracy and generalization of Pathformer. Extensive experiments on eleven real-world datasets demonstrate that Pathformer not only achieves state-of-the-art performance by surpassing all current models but also exhibits stronger generalization abilities under various transfer scenarios. The code is made available at https://github.com/decisionintelligence/pathformer.


title: Accelerating PDE Data Generation via Differential Operator Action in Solution Space

abstract: Recent advancements in data-driven approaches, such as Neural Operator (NO), have demonstrated their effectiveness in reducing the solving time of Partial Differential Equations (PDEs). However, one major challenge faced by these approaches is the requirement for a large amount of high-precision training data, which needs significant computational costs during the generation process. To address this challenge, we propose a novel PDE dataset generation algorithm, namely Differential Operator Action in Solution space (DiffOAS), which speeds up the data generation process and enhances the precision of the generated data simultaneously. Specifically, DiffOAS obtains a few basic PDE solutions and then combines them to get solutions. It applies differential operators on these solutions, a process we call 'operator action', to efficiently generate precise PDE data points. Theoretical analysis shows that the time complexity of DiffOAS method is one order lower than the existing generation method. Experimental results show that DiffOAS accelerates the generation of large-scale datasets with 10,000 instances by 300 times. Even with just 5% of the generation time, NO trained on the data generated by DiffOAS exhibits comparable performance to that using the existing generation method, which highlights the efficiency of DiffOAS.


title: A comparative study on wearables and single-camera video for upper-limb out-of-thelab activity recognition with different deep learning architectures

abstract: The use of a wide range of computer vision solutions, and more recently high-end Inertial Measurement Units (IMU) have become increasingly popular for assessing human physical activity in clinical and research settings. Nevertheless, to increase the feasibility of patient tracking in out-of-the-lab settings, it is necessary to use a reduced number of devices for movement acquisition. Promising solutions in this context are IMU-based wearables and single camera systems. Additionally, the development of machine learning systems able to recognize and digest clinically relevant data in-the-wild is needed, and therefore determining the ideal input to those is crucial.


title: Nature-Inspired Local Propagation

abstract: The spectacular results achieved in machine learning, including the recent advances in generative AI, rely on large data collections. On the opposite, intelligent processes in nature arises without the need for such collections, but simply by online processing of the environmental information. In particular, natural learning processes rely on mechanisms where data representation and learning are intertwined in such a way to respect spatiotemporal locality. This paper shows that such a feature arises from a pre-algorithmic view of learning that is inspired by related studies in Theoretical Physics. We show that the algorithmic interpretation of the derived "laws of learning", which takes the structure of Hamiltonian equations, reduces to Backpropagation when the speed of propagation goes to infinity. This opens the doors to machine learning studies based on full on-line information processing that are based the replacement of Backpropagation with the proposed spatiotemporal local algorithm.


title: Phase-driven Domain Generalizable Learning for Nonstationary Time Series

abstract: Monitoring and recognizing patterns in continuous sensing data is crucial for many practical applications. These real-world time-series data are often nonstationary, characterized by varying statistical and spectral properties over time. This poses a significant challenge in developing learning models that can effectively generalize across different distributions. In this work, based on our observation that nonstationary statistics are intrinsically linked to the phase information, we propose a time-series learning framework, PhASER. It consists of three novel elements: 1) phase augmentation that diversifies non-stationarity while preserving discriminatory semantics, 2) separate feature encoding by viewing time-varying magnitude and phase as independent modalities, and 3) feature broadcasting by incorporating phase with a novel residual connection for inherent regularization to enhance distribution invariant learning. Upon extensive evaluation on 5 datasets from human activity recognition, sleep-stage classification, and gesture recognition against 10 state-of-the-art baseline methods, we demonstrate that PhASER consistently outperforms the best baselines by an average of 5% and up to 13% in some cases. Moreover, PhASER's principles can be applied broadly to boost the generalization ability of existing time series classification models.


title: Genetic-guided GFlowNets for Sample Efficient Molecular Optimization

abstract: The challenge of discovering new molecules with desired properties is crucial in domains like drug discovery and material design. Recent advances in deep learning-based generative methods have shown promise but face the issue of sample efficiency due to the computational expense of evaluating the reward function. This paper proposes a novel algorithm for sample-efficient molecular optimization by distilling a powerful genetic algorithm into deep generative policy using GFlowNets training, the off-policy method for amortized inference. This approach enables the deep generative policy to learn from domain knowledge, which has been explicitly integrated into the genetic algorithm. Our method achieves state-of-the-art performance in the official molecular optimization benchmark, significantly outperforming previous methods. It also demonstrates effectiveness in designing inhibitors against SARS-CoV-2 with substantially fewer reward calls.


title: EXGC: Bridging Efficiency and Explainability in Graph Condensation

abstract: Graph representation learning on vast datasets, like web data, has made significant strides. However, the associated computational and storage overheads raise concerns. In sight of this, Graph condensation (GCond) has been introduced to distill these large real datasets into a more concise yet information-rich synthetic graph. Despite acceleration efforts, existing GCond methods mainly grapple with efficiency, especially on expansive web data graphs. Hence, in this work, we pinpoint two major inefficiencies of current paradigms: (1) the concurrent updating of a vast parameter set, and (2) pronounced parameter redundancy. To counteract these two limitations correspondingly, we first (1) employ the Mean-Field variational approximation for convergence acceleration, and then (2) propose the objective of Gradient Information Bottleneck (GDIB) to prune redundancy. By incorporating the leading explanation techniques (e.g., GNNExplainer and GSAT) to instantiate the GDIB, our EXGC, the Efficient and eXplainable Graph Condensation method is proposed, which can markedly boost efficiency and inject explainability. Our extensive evaluations across eight datasets underscore EXGC's superiority and relevance. Code is available at https://github.com/MangoKiller/EXGC.


title: Frugal Actor-Critic: Sample Efficient Off-Policy Deep Reinforcement Learning Using Unique Experiences

abstract: Efficient utilization of the replay buffer plays a significant role in the off-policy actor-critic reinforcement learning (RL) algorithms used for model-free control policy synthesis for complex dynamical systems. We propose a method for achieving sample efficiency, which focuses on selecting unique samples and adding them to the replay buffer during the exploration with the goal of reducing the buffer size and maintaining the independent and identically distributed (IID) nature of the samples. Our method is based on selecting an important subset of the set of state variables from the experiences encountered during the initial phase of random exploration, partitioning the state space into a set of abstract states based on the selected important state variables, and finally selecting the experiences with unique state-reward combination by using a kernel density estimator. We formally prove that the off-policy actor-critic algorithm incorporating the proposed method for unique experience accumulation converges faster than the vanilla off-policy actor-critic algorithm. Furthermore, we evaluate our method by comparing it with two state-of-the-art actor-critic RL algorithms on several continuous control benchmarks available in the Gym environment. Experimental results demonstrate that our method achieves a significant reduction in the size of the replay buffer for all the benchmarks while achieving either faster convergent or better reward accumulation compared to the baseline algorithms.


title: A Survey on Transformer Compression

abstract: Transformer plays a vital role in the realms of natural language processing (NLP) and computer vision (CV), specially for constructing large language models (LLM) and large vision models (LVM). Model compression methods reduce the memory and computational cost of Transformer, which is a necessary step to implement large language/vision models on practical devices. Given the unique architecture of Transformer, featuring alternative attention and feedforward neural network (FFN) modules, specific compression techniques are usually required. The efficiency of these compression methods is also paramount, as retraining large models on the entire training dataset is usually impractical. This survey provides a comprehensive review of recent compression methods, with a specific focus on their application to Transformer-based models. The compression methods are primarily categorized into pruning, quantization, knowledge distillation, and efficient architecture design (Mamba, RetNet, RWKV, etc.). In each category, we discuss compression methods for both language and vision tasks, highlighting common underlying principles. Finally, we delve into the relation between various compression methods, and discuss further directions in this domain.


title: Hybrid Neural Representations for Spherical Data

abstract: In this paper, we study hybrid neural representations for spherical data, a domain of increasing relevance in scientific research. In particular, our work focuses on weather and climate data as well as comic microwave background (CMB) data. Although previous studies have delved into coordinate-based neural representations for spherical signals, they often fail to capture the intricate details of highly nonlinear signals. To address this limitation, we introduce a novel approach named Hybrid Neural Representations for Spherical data (HNeR-S). Our main idea is to use spherical feature-grids to obtain positional features which are combined with a multilayer perception to predict the target signal. We consider feature-grids with equirectangular and hierarchical equal area isolatitude pixelization structures that align with weather data and CMB data, respectively. We extensively verify the effectiveness of our HNeR-S for regression, super-resolution, temporal interpolation, and compression tasks.


title: Rethink Model Re-Basin and the Linear Mode Connectivity

abstract: Recent studies suggest that with sufficiently wide models, most SGD solutions can, up to permutation, converge into the same basin. This phenomenon, known as the model re-basin regime, has significant implications for model averaging. However, current re-basin strategies are limited in effectiveness due to a lack of comprehensive understanding of underlying mechanisms. Addressing this gap, our work revisits standard practices and uncovers the frequent inadequacies of existing matching algorithms, which we show can be mitigated through proper re-normalization. By introducing a more direct analytical approach, we expose the interaction between matching algorithms and re-normalization processes. This perspective not only clarifies and refines previous findings but also facilitates novel insights. For instance, it connects the linear mode connectivity to pruning, motivating a lightweight yet effective post-pruning plug-in that can be directly merged with any existing pruning techniques. Our implementation is available at https://github.com/XingyuQu/rethink-re-basin.


title: The last Dance : Robust backdoor attack via diffusion models and bayesian approach

abstract: Diffusion models are state-of-the-art deep learning generative models that are trained on the principle of learning forward and backward diffusion processes via the progressive addition of noise and denoising. In this paper, we aim to fool audio-based DNN models, such as those from the Hugging Face framework, primarily those that focus on audio, in particular transformer-based artificial intelligence models, which are powerful machine learning models that save time and achieve results faster and more efficiently. We demonstrate the feasibility of backdoor attacks (called `BacKBayDiffMod`) on audio transformers derived from Hugging Face, a popular framework in the world of artificial intelligence research. The backdoor attack developed in this paper is based on poisoning model training data uniquely by incorporating backdoor diffusion sampling and a Bayesian approach to the distribution of poisoned data.


title: Federated Learning Priorities Under the European Union Artificial Intelligence Act

abstract: The age of AI regulation is upon us, with the European Union Artificial Intelligence Act (AI Act) leading the way. Our key inquiry is how this will affect Federated Learning (FL), whose starting point of prioritizing data privacy while performing ML fundamentally differs from that of centralized learning. We believe the AI Act and future regulations could be the missing catalyst that pushes FL toward mainstream adoption. However, this can only occur if the FL community reprioritizes its research focus. In our position paper, we perform a first-of-its-kind interdisciplinary analysis (legal and ML) of the impact the AI Act may have on FL and make a series of observations supporting our primary position through quantitative and qualitative analysis. We explore data governance issues and the concern for privacy. We establish new challenges regarding performance and energy efficiency within lifecycle monitoring. Taken together, our analysis suggests there is a sizable opportunity for FL to become a crucial component of AI Act-compliant ML systems and for the new regulation to drive the adoption of FL techniques in general. Most noteworthy are the opportunities to defend against data bias and enhance private and secure computation


title: Breaking Symmetry When Training Transformers

abstract: As we show in this paper, the prediction for output token $n+1$ of Transformer architectures without one of the mechanisms of positional encodings and causal attention is invariant to permutations of input tokens $1, 2, ..., n-1$. Usually, both mechanisms are employed and the symmetry with respect to the input tokens is broken. Recently, it has been shown that one can train Transformers without positional encodings. This must be enabled by the causal attention mechanism. In this paper, we elaborate on the argument that the causal connection mechanism must be responsible for the fact that Transformers are able to model input sequences where the order is important. Vertical "slices" of Transformers are all encouraged to represent the same location $k$ in the input sequence. We hypothesize that residual connections contribute to this phenomenon, and demonstrate evidence for this.


title: Modeling Spatio-temporal Dynamical Systems with Neural Discrete Learning and Levels-of-Experts

abstract: In this paper, we address the issue of modeling and estimating changes in the state of the spatio-temporal dynamical systems based on a sequence of observations like video frames. Traditional numerical simulation systems depend largely on the initial settings and correctness of the constructed partial differential equations (PDEs). Despite recent efforts yielding significant success in discovering data-driven PDEs with neural networks, the limitations posed by singular scenarios and the absence of local insights prevent them from performing effectively in a broader real-world context. To this end, this paper propose the universal expert module -- that is, optical flow estimation component, to capture the evolution laws of general physical processes in a data-driven fashion. To enhance local insight, we painstakingly design a finer-grained physical pipeline, since local characteristics may be influenced by various internal contextual information, which may contradict the macroscopic properties of the whole system. Further, we harness currently popular neural discrete learning to unveil the underlying important features in its latent space, this process better injects interpretability, which can help us obtain a powerful prior over these discrete random variables. We conduct extensive experiments and ablations to demonstrate that the proposed framework achieves large performance margins, compared with the existing SOTA baselines.


title: Are we making much progress? Revisiting chemical reaction yield prediction from an imbalanced regression perspective

abstract: The yield of a chemical reaction quantifies the percentage of the target product formed in relation to the reactants consumed during the chemical reaction. Accurate yield prediction can guide chemists toward selecting high-yield reactions during synthesis planning, offering valuable insights before dedicating time and resources to wet lab experiments. While recent advancements in yield prediction have led to overall performance improvement across the entire yield range, an open challenge remains in enhancing predictions for high-yield reactions, which are of greater concern to chemists. In this paper, we argue that the performance gap in high-yield predictions results from the imbalanced distribution of real-world data skewed towards low-yield reactions, often due to unreacted starting materials and inherent ambiguities in the reaction processes. Despite this data imbalance, existing yield prediction methods continue to treat different yield ranges equally, assuming a balanced training distribution. Through extensive experiments on three real-world yield prediction datasets, we emphasize the urgent need to reframe reaction yield prediction as an imbalanced regression problem. Finally, we demonstrate that incorporating simple cost-sensitive re-weighting methods can significantly enhance the performance of yield prediction models on underrepresented high-yield regions.


title: Gaussian-process-regression-based method for the localization of exceptional points in complex resonance spectra

abstract: Resonances in open quantum systems depending on at least two controllable parameters can show the phenomenon of exceptional points (EPs), where not only the eigenvalues but also the eigenvectors of two or more resonances coalesce. Their exact localization in the parameter space is challenging, in particular in systems, where the computation of the quantum spectra and resonances is numerically very expensive. We introduce an efficient machine learning algorithm to find exceptional points based on Gaussian process regression (GPR). The GPR-model is trained with an initial set of eigenvalue pairs belonging to an EP and used for a first estimation of the EP position via a numerically cheap root search. The estimate is then improved iteratively by adding selected exact eigenvalue pairs as training points to the GPR-model. The GPR-based method is developed and tested on a simple low-dimensional matrix model and then applied to a challenging real physical system, viz., the localization of EPs in the resonance spectra of excitons in cuprous oxide in external electric and magnetic fields. The precise computation of EPs, by taking into account the complete valence band structure and central-cell corrections of the crystal, can be the basis for the experimental observation of EPs in this system.

categories: quant-ph cond-mat.mes-hall cond-mat.str-el cs.LG


title: Blockchain-enabled Clustered and Scalable Federated Learning (BCS-FL) Framework in UAV Networks

abstract: Privacy, scalability, and reliability are significant challenges in unmanned aerial vehicle (UAV) networks as distributed systems, especially when employing machine learning (ML) technologies with substantial data exchange. Recently, the application of federated learning (FL) to UAV networks has improved collaboration, privacy, resilience, and adaptability, making it a promising framework for UAV applications. However, implementing FL for UAV networks introduces drawbacks such as communication overhead, synchronization issues, scalability limitations, and resource constraints. To address these challenges, this paper presents the Blockchain-enabled Clustered and Scalable Federated Learning (BCS-FL) framework for UAV networks. This improves the decentralization, coordination, scalability, and efficiency of FL in large-scale UAV networks. The framework partitions UAV networks into separate clusters, coordinated by cluster head UAVs (CHs), to establish a connected graph. Clustering enables efficient coordination of updates to the ML model. Additionally, hybrid inter-cluster and intra-cluster model aggregation schemes generate the global model after each training round, improving collaboration and knowledge sharing among clusters. The numerical findings illustrate the achievement of convergence while also emphasizing the trade-offs between the effectiveness of training and communication efficiency.


title: RAGE for the Machine: Image Compression with Low-Cost Random Access for Embedded Applications

abstract: We introduce RAGE, an image compression framework that achieves four generally conflicting objectives: 1) good compression for a wide variety of color images, 2) computationally efficient, fast decompression, 3) fast random access of images with pixel-level granularity without the need to decompress the entire image, 4) support for both lossless and lossy compression. To achieve these, we rely on the recent concept of generalized deduplication (GD), which is known to provide efficient lossless (de)compression and fast random access in time-series data, and deliver key expansions suitable for image compression, both lossless and lossy. Using nine different datasets, incl. graphics, logos, natural images, we show that RAGE has similar or better compression ratios to state-of-the-art lossless image compressors, while delivering pixel-level random access capabilities. Tests in an ARM Cortex-M33 platform show seek times between 9.9 and 40.6~ns and average decoding time per pixel between 274 and 1226~ns. Our measurements also show that RAGE's lossy variant, RAGE-Q, outperforms JPEG by several fold in terms of distortion in embedded graphics and has reasonable compression and distortion for natural images.


title: A Deep Learning Approach for Brain Tumor Classification and Segmentation Using a Multiscale Convolutional Neural Network

abstract: In this paper, we present a fully automatic brain tumor segmentation and classification model using a Deep Convolutional Neural Network that includes a multiscale approach. One of the differences of our proposal with respect to previous works is that input images are processed in three spatial scales along different processing pathways. This mechanism is inspired in the inherent operation of the Human Visual System. The proposed neural model can analyze MRI images containing three types of tumors: meningioma, glioma, and pituitary tumor, over sagittal, coronal, and axial views and does not need preprocessing of input images to remove skull or vertebral column parts in advance. The performance of our method on a publicly available MRI image dataset of 3064 slices from 233 patients is compared with previously classical machine learning and deep learning published methods. In the comparison, our method remarkably obtained a tumor classification accuracy of 0.973, higher than the other approaches using the same database.


title: RankSum An unsupervised extractive text summarization based on rank fusion

abstract: In this paper, we propose Ranksum, an approach for extractive text summarization of single documents based on the rank fusion of four multi-dimensional sentence features extracted for each sentence: topic information, semantic content, significant keywords, and position. The Ranksum obtains the sentence saliency rankings corresponding to each feature in an unsupervised way followed by the weighted fusion of the four scores to rank the sentences according to their significance. The scores are generated in completely unsupervised way, and a labeled document set is required to learn the fusion weights. Since we found that the fusion weights can generalize to other datasets, we consider the Ranksum as an unsupervised approach. To determine topic rank, we employ probabilistic topic models whereas semantic information is captured using sentence embeddings. To derive rankings using sentence embeddings, we utilize Siamese networks to produce abstractive sentence representation and then we formulate a novel strategy to arrange them in their order of importance. A graph-based strategy is applied to find the significant keywords and related sentence rankings in the document. We also formulate a sentence novelty measure based on bigrams, trigrams, and sentence embeddings to eliminate redundant sentences from the summary. The ranks of all the sentences computed for each feature are finally fused to get the final score for each sentence in the document. We evaluate our approach on publicly available summarization datasets CNN/DailyMail and DUC 2002. Experimental results show that our approach outperforms other existing state-of-the-art summarization methods.


title: Tool wear monitoring using an online, automatic and low cost system based on local texture

abstract: In this work we propose a new online, low cost and fast approach based on computer vision and machine learning to determine whether cutting tools used in edge profile milling processes are serviceable or disposable based on their wear level. We created a new dataset of 254 images of edge profile cutting heads which is, to the best of our knowledge, the first publicly available dataset with enough quality for this purpose. All the inserts were segmented and their cutting edges were cropped, obtaining 577 images of cutting edges: 301 functional and 276 disposable. The proposed method is based on (1) dividing the cutting edge image in different regions, called Wear Patches (WP), (2) characterising each one as worn or serviceable using texture descriptors based on different variants of Local Binary Patterns (LBP) and (3) determine, based on the state of these WP, if the cutting edge (and, therefore, the tool) is serviceable or disposable. We proposed and assessed five different patch division configurations. The individual WP were classified by a Support Vector Machine (SVM) with an intersection kernel. The best patch division configuration and texture descriptor for the WP achieves an accuracy of 90.26% in the detection of the disposable cutting edges. These results show a very promising opportunity for automatic wear monitoring in edge profile milling processes.


title: Combining shape and contour features to improve tool wear monitoring in milling processes

abstract: In this paper, a new system based on combinations of a shape descriptor and a contour descriptor has been proposed for classifying inserts in milling processes according to their wear level following a computer vision based approach. To describe the wear region shape we have proposed a new descriptor called ShapeFeat and its contour has been characterized using the method BORCHIZ that, to the best of our knowledge, achieves the best performance for tool wear monitoring following a computer vision-based approach. Results show that the combination of BORCHIZ with ShapeFeat using a late fusion method improves the classification performance significantly, obtaining an accuracy of 91.44% in the binary classification (i.e. the classification of the wear as high or low) and 82.90% using three target classes (i.e. classification of the wear as high, medium or low). These results outperform the ones obtained by both descriptors used on their own, which achieve accuracies of 88.70 and 80.67% for two and three classes, respectively, using ShapeFeat and 87.06 and 80.24% with B-ORCHIZ. This study yielded encouraging results for the manufacturing community in order to classify automatically the inserts in terms of their wear for milling processes.