title: VIDiff: Translating Videos via Multi-Modal Instructions with Diffusion Models

abstract: Diffusion models have achieved significant success in image and video generation. This motivates a growing interest in video editing tasks, where videos are edited according to provided text descriptions. However, most existing approaches only focus on video editing for short clips and rely on time-consuming tuning or inference. We are the first to propose Video Instruction Diffusion (VIDiff), a unified foundation model designed for a wide range of video tasks. These tasks encompass both understanding tasks (such as language-guided video object segmentation) and generative tasks (video editing and enhancement). Our model can edit and translate the desired results within seconds based on user instructions. Moreover, we design an iterative auto-regressive method to ensure consistency in editing and enhancing long videos. We provide convincing generative results for diverse input videos and written instructions, both qualitatively and quantitatively. More examples can be found at our website https://ChenHsing.github.io/VIDiff.


title: Dataset Distillation in Large Data Era

abstract: Dataset distillation aims to generate a smaller but representative subset from a large dataset, which allows a model to be trained efficiently, meanwhile evaluating on the original testing data distribution to achieve decent performance. Many prior works have aimed to align with diverse aspects of the original datasets, such as matching the training weight trajectories, gradient, feature/BatchNorm distributions, etc. In this work, we show how to distill various large-scale datasets such as full ImageNet-1K/21K under a conventional input resolution of 224$\times$224 to achieve the best accuracy over all previous approaches, including SRe$^2$L, TESLA and MTT. To achieve this, we introduce a simple yet effective ${\bf C}$urriculum ${\bf D}$ata ${\bf A}$ugmentation ($\texttt{CDA}$) during data synthesis that obtains the accuracy on large-scale ImageNet-1K and 21K with 63.2% under IPC (Images Per Class) 50 and 36.1% under IPC 20, respectively. Finally, we show that, by integrating all our enhancements together, the proposed model beats the current state-of-the-art by more than 4% Top-1 accuracy on ImageNet-1K/21K and for the first time, reduces the gap to its full-data training counterpart to less than absolute 15%. Moreover, this work represents the inaugural success in dataset distillation on larger-scale ImageNet-21K under the standard 224$\times$224 resolution. Our code and distilled ImageNet-21K dataset of 20 IPC, 2K recovery budget are available at https://github.com/VILA-Lab/SRe2L/tree/main/CDA.


title: TrafficMOT: A Challenging Dataset for Multi-Object Tracking in Complex Traffic Scenarios

abstract: Multi-object tracking in traffic videos is a crucial research area, offering immense potential for enhancing traffic monitoring accuracy and promoting road safety measures through the utilisation of advanced machine learning algorithms. However, existing datasets for multi-object tracking in traffic videos often feature limited instances or focus on single classes, which cannot well simulate the challenges encountered in complex traffic scenarios. To address this gap, we introduce TrafficMOT, an extensive dataset designed to encompass diverse traffic situations with complex scenarios. To validate the complexity and challenges presented by TrafficMOT, we conducted comprehensive empirical studies using three different settings: fully-supervised, semi-supervised, and a recent powerful zero-shot foundation model Tracking Anything Model (TAM). The experimental results highlight the inherent complexity of this dataset, emphasising its value in driving advancements in the field of traffic monitoring and multi-object tracking.


title: Just Add $\pi$! Pose Induced Video Transformers for Understanding Activities of Daily Living

abstract: Video transformers have become the de facto standard for human action recognition, yet their exclusive reliance on the RGB modality still limits their adoption in certain domains. One such domain is Activities of Daily Living (ADL), where RGB alone is not sufficient to distinguish between visually similar actions, or actions observed from multiple viewpoints. To facilitate the adoption of video transformers for ADL, we hypothesize that the augmentation of RGB with human pose information, known for its sensitivity to fine-grained motion and multiple viewpoints, is essential. Consequently, we introduce the first Pose Induced Video Transformer: PI-ViT (or $\pi$-ViT), a novel approach that augments the RGB representations learned by video transformers with 2D and 3D pose information. The key elements of $\pi$-ViT are two plug-in modules, 2D Skeleton Induction Module and 3D Skeleton Induction Module, that are responsible for inducing 2D and 3D pose information into the RGB representations. These modules operate by performing pose-aware auxiliary tasks, a design choice that allows $\pi$-ViT to discard the modules during inference. Notably, $\pi$-ViT achieves the state-of-the-art performance on three prominent ADL datasets, encompassing both real-world and large-scale RGB-D datasets, without requiring poses or additional computational overhead at inference.


title: On the maximum black hole mass at solar metallicity

abstract: In high metallicity environments the mass that black holes (BHs) can reach just after core-collapse widely depends on how much mass their progenitor stars lose via winds. On one hand new theoretical and observational insights suggest that early-stage winds should be weaker than what many canonical models prescribe. On the other hand the proximity to the Eddington limit should affect the formation of optically thick envelopes already during the earliest stages of stars with initial masses $M_{\rm ZAMS}\gtrsim 100$ M$_\odot$, hence resulting in higher mass-loss rates during the main sequence. We use the evolutionary codes MESA and Genec to calculate a suite of tracks for massive stars at solar metallicity Z$_\odot=0.014$ which incorporate these changes in our wind mass loss prescription. In our calculations we employ moderate rotation, high overshooting and magnetic angular momentum transport. We find a maximum BH mass $M_{\rm BH, max}=28.3$ M$_\odot$ at Z$_\odot$. The most massive BHs are predicted to form from stars with $M_{\rm ZAMS}\gtrsim 250$ M$_\odot$, with the BH mass directly proportional to its progenitor's $M_{\rm ZAMS}$. We also find in our models that at Z$_\odot$ almost any BH progenitor naturally evolves into a Wolf-Rayet star due to the combined effect of internal mixing and wind mass loss. These results are considerably different from most recent studies regarding the final mass of stars before their collapse into BHs. While we acknowledge the inherent uncertainties in stellar evolution modelling, our study underscores the importance of employing the most up-to-date physics in BH mass predictions.


title: On random pairwise comparisons matrices and their geometry

abstract: We describe a framework for random pairwise comparisons matrices, inspired by selected constructions releted to the so called inconsistency reduction of pairwise comparisons (PC) matrices. In to build up structures on random pairwise comparisons matrices, the set up for (deterministic) PC matrices for non-reciprocal PC matrices is completed. The extension of basic concepts such as inconsistency indexes and geometric mean method are extended to random pairwise comparisons matrices and completed by new notions which seem useful to us. Two procedures for (random) inconsistency reduction are sketched, based on well-known existing objects, and a fiber bundle-like decomposition of random pairwise comparisons is proposed.


title: The Discontinuity Group of a Locally Bounded Homomorphism of a Lie Group into a Lie Group Is Commutative

abstract: We prove that the discontinuity group of every locally bounded homomorphism of a Lie group into a Lie group is not only compact and connected, which is known, but is also commutative.


title: Transport Equation based Physics Informed Neural Network to predict the Yield Strength of Architected Materials

abstract: In this research, the application of the Physics-Informed Neural Network (PINN) model is explored to solve transport equation-based Partial Differential Equations (PDEs). The primary objective is to analyze the impact of different activation functions incorporated within the PINN model on its predictive performance, specifically assessing the Mean Squared Error (MSE) and Mean Absolute Error (MAE). The dataset used in the study consists of a varied set of input parameters related to strut diameter, unit cell size, and the corresponding yield stress values. Through this investigation the aim is to understand the effectiveness of the PINN model and the significance of choosing appropriate activation functions for solving complex PDEs in real-world applications. The outcomes suggest that the choice of activation function may have minimal influence on the model's predictive accuracy for this particular problem. The PINN model showcases exceptional generalization capabilities, indicating its capacity to avoid overfitting with the provided dataset. The research underscores the importance of striking a balance between performance and computational efficiency while selecting an activation function for specific real-world applications. These valuable findings contribute to advancing the understanding and potential adoption of PINN as an effective tool for solving challenging PDEs in diverse scientific and engineering domains.


title: On the variational treatment of a class of double-well oscillators

abstract: We compare the well known Rayleigh-Ritz variational method (RRVM) with a recently proposed approach based on supersymmetric quantum mechanics and the Gram-Schmidt orthogonalization method (SSQMGS). We apply both procedures to a particular class of double-well harmonic oscillators that had been conveniently chosen for the application of the latter approach. The RRVM eigenvalues converge smoothly from above providing much more accurate results with less computational effort. Present results show that the unproved SSQMGS upper bounds do not hold.


title: NumCalc: An open source BEM code for solving acoustic scattering problems

abstract: The calculation of the acoustic field in or around objects is an important task in acoustic engineering. To numerically solve this task, the boundary element method (BEM) is a commonly used method especially for infinite domains. The open source tool Mesh2HRTF and its BEM core NumCalc provide users with a collection of free software for acoustic simulations without the need of having an in-depth knowledge into numerical methods. However, we feel that users should have a basic understanding with respect to the methods behind the software they are using. We are convinced that this basic understanding helps in avoiding common mistakes and also helps to understand the requirements to use the software. To provide this background is the first motivation for this paper. A second motivation for this paper is to demonstrate the accuracy of NumCalc when solving benchmark problems. Thus, users can get an idea about the accuracy they can expect when using NumCalc as well as the memory and CPU requirements of NumCalc. A third motivation for this paper is to give users detailed information about some parts of the actual implementation that are usually not mentioned in literature, e.g., the specific version of the fast multipole method and its clustering process or how to use frequency-dependent admittance boundary conditions.


title: Enhancing ML-Based DoS Attack Detection Through Combinatorial Fusion Analysis

abstract: Mitigating Denial-of-Service (DoS) attacks is vital for online service security and availability. While machine learning (ML) models are used for DoS attack detection, new strategies are needed to enhance their performance. We suggest an innovative method, combinatorial fusion, which combines multiple ML models using advanced algorithms. This includes score and rank combinations, weighted techniques, and diversity strength of scoring systems. Through rigorous evaluations, we demonstrate the effectiveness of this fusion approach, considering metrics like precision, recall, and F1-score. We address the challenge of low-profiled attack classification by fusing models to create a comprehensive solution. Our findings emphasize the potential of this approach to improve DoS attack detection and contribute to stronger defense mechanisms.


title: Space-Time Decomposition of Kalman Filter

abstract: We present an innovative interpretation of Kalman Filter (KF, for short) combining the ideas of Schwarz Domain Decomposition (DD) and Parallel in Time (PinT) approaches. Thereafter we call it DD-KF. In contrast to standard DD approaches which are already incorporated in KF and other state estimation models, implementing a straightforward data parallelism inside the loop over time, DD-KF ab-initio partitions the whole model, including filter equations and dynamic model along both space and time directions/steps. As a consequence, we get local KFs reproducing the original filter at smaller dimensions on local domains. Also, sub problems could be solved in parallel. In order to enforce the matching of local solutions on overlapping regions, and then to achieve the same global solution of KF, local KFs are slightly modified by adding a correction term keeping track of contributions of adjacent subdomains to overlapping regions. Such a correction term balances localization errors along overlapping regions, acting as a regularization constraint on local solutions. Furthermore, such a localization excludes remote observations from each analyzed location improving the conditioning of the error covariance matrices. As dynamic model we consider Shallow Water equations which can be regarded a consistent tool to get a proof of concept of the reliability assessment of DD-KF in monitoring and forecasting of weather systems and ocean currents


title: A generalized character associated to element orders

abstract: Let $G$ be a finite group. We study the generalized character defined by $\Xi(g)=|G|o(g)$, for $g\in G$, which is closely related to a function that has been very studied recently from a group theoretical point of view.


title: Risk-Aware and Explainable Framework for Ensuring Guaranteed Coverage in Evolving Hardware Trojan Detection

abstract: As the semiconductor industry has shifted to a fabless paradigm, the risk of hardware Trojans being inserted at various stages of production has also increased. Recently, there has been a growing trend toward the use of machine learning solutions to detect hardware Trojans more effectively, with a focus on the accuracy of the model as an evaluation metric. However, in a high-risk and sensitive domain, we cannot accept even a small misclassification. Additionally, it is unrealistic to expect an ideal model, especially when Trojans evolve over time. Therefore, we need metrics to assess the trustworthiness of detected Trojans and a mechanism to simulate unseen ones. In this paper, we generate evolving hardware Trojans using our proposed novel conformalized generative adversarial networks and offer an efficient approach to detecting them based on a non-invasive algorithm-agnostic statistical inference framework that leverages the Mondrian conformal predictor. The method acts like a wrapper over any of the machine learning models and produces set predictions along with uncertainty quantification for each new detected Trojan for more robust decision-making. In the case of a NULL set, a novel method to reject the decision by providing a calibrated explainability is discussed. The proposed approach has been validated on both synthetic and real chip-level benchmarks and proven to pave the way for researchers looking to find informed machine learning solutions to hardware security problems.


title: Efficient calculation of the integral equation for simulating 2D TE scattering in a homogeneous medium using the Ewald method and a Gabor frame discretization

abstract: We utilize the domain integral equation formulation to simulate two-dimensional transverse electric scattering in a homogeneous medium and a summation of modulated Gaussian functions to approximate the dual Gabor window. Then we apply Ewald Green function transformation to separate the integrals related to x and z in the integral equation, which produce Gaussian functions. These Gaussian functions in the integrands can be integrated analytically, which greatly simplifies the calculation process. Finally, we discuss the convergence and the selection of the Ewald splitting parameter E.


title: The Bivariate Normal Integral via Owen's T Function as a Modified Euler's Arctangent Series

abstract: The Owen's T function is presented in four new ways, one of them as a series similar to the Euler's arctangent series divided by $2\pi$, which is its majorant series. All possibilities enable numerically stable and fast convergent computation of the bivariate normal integral with simple recursion. When tested $\Phi_\varrho^2(x,y)$ computation on a random sample of one million parameter triplets with uniformly distributed components and using double precision arithmetic, the maximum absolute error was $3.45\cdot 10^{-16}$. In additional testing, focusing on cases with correlation coefficients close to one in absolute value, when the computation may be very sensitive to small rounding errors, the accuracy was retained. In rare potentially critical cases, a simple adjustment to the computation procedure was performed - one potentially critical computation was replaced with two equivalent non-critical ones. All new series are suitable for vector and high-precision computation, assuming they are supplemented with appropriate efficient and accurate computation of the arctangent and standard normal cumulative distribution functions. They are implemented by the R package Phi2rho, available on CRAN. Its functions allow vector arguments and are ready to work with the Rmpfr package, which enables the use of arbitrary precision instead of double precision numbers. A special test with up to 1024-bit precision computation is also presented.


title: Multi-fidelity uncertainty quantification for homogenization problems in structure-property relationships from crystal plasticity finite elements

abstract: Crystal plasticity finite element method (CPFEM) has been an integrated computational materials engineering (ICME) workhorse to study materials behaviors and structure-property relationships for the last few decades. These relations are mappings from the microstructure space to the materials properties space. Due to the stochastic and random nature of microstructures, there is always some uncertainty associated with materials properties, for example, in homogenized stress-strain curves. For critical applications with strong reliability needs, it is often desirable to quantify the microstructure-induced uncertainty in the context of structure-property relationships. However, this uncertainty quantification (UQ) problem often incurs a large computational cost because many statistically equivalent representative volume elements (SERVEs) are needed. In this paper, we apply a multi-level Monte Carlo (MLMC) method to CPFEM to study the uncertainty in stress-strain curves, given an ensemble of SERVEs at multiple mesh resolutions. By using the information at coarse meshes, we show that it is possible to approximate the response at fine meshes with a much reduced computational cost. We focus on problems where the model output is multi-dimensional, which requires us to track multiple quantities of interest (QoIs) at the same time. Our numerical results show that MLMC can accelerate UQ tasks around 2.23x, compared to the classical Monte Carlo (MC) method, which is widely known as the ensemble average in the CPFEM literature.


title: Biometric Technologies and the Law: Developing a Taxonomy for Guiding Policymakers

abstract: Despite the increasing adoption of biometric technologies, their regulation has not kept up with the same pace, particularly with regard to safeguarding individuals' privacy and personal data. Policymakers may struggle to comprehend the technology behind biometric systems and their potential impact on fundamental rights, resulting in insufficient or inadequate legal regulation. This study seeks to bridge this gap by proposing a taxonomy of biometric technologies that can aid in their effective deployment and supervision. Through a literature review, the technical characteristics of biometric systems were identified and categorised. The resulting taxonomy can enhance the understanding of biometric technologies and facilitate the development of regulation that prioritises privacy and personal data protection.


title: A class of fractional differential equations via power non-local and non-singular kernels: existence, uniqueness and numerical approximations

abstract: We prove a useful formula and new properties for the recently introduced power fractional calculus with non-local and non-singular kernels. In particular, we prove a new version of Gronwall's inequality involving the power fractional integral; and we establish existence and uniqueness results for nonlinear power fractional differential equations using fixed point techniques. Moreover, based on Lagrange polynomial interpolation, we develop a new explicit numerical method in order to approximate the solutions of a rich class of fractional differential equations. The approximation error of the proposed numerical scheme is analyzed. For illustrative purposes, we apply our method to a fractional differential equation for which the exact solution is computed, as well as to a nonlinear problem for which no exact solution is known. The numerical simulations show that the proposed method is very efficient, highly accurate and converges quickly.


title: On the $\ell_0$ Isoperimetric Coefficient of Measurable Sets

abstract: In this paper we prove that the $\ell_0$ isoperimetric coefficient for any axis-aligned cubes, $\psi_{\mathcal{C}}$, is $\Theta(n^{-1/2})$ and that the isoperimetric coefficient for any measurable body $K$, $\psi_K$, is of order $O(n^{-1/2})$. As a corollary we deduce that axis-aligned cubes essentially "maximize" the $\ell_0$ isoperimetric coefficient: There exists a positive constant $q > 0$ such that $\psi_K \leq q \cdot \psi_{\mathcal{C}}$, whenever $\mathcal{C}$ is an axis-aligned cube and $K$ is any measurable set. Lastly, we give immediate applications of our results to the mixing time of Coordinate-Hit-and-Run for sampling points uniformly from convex bodies.


title: Preserving The Safety And Confidentiality Of Data Mining Information In Health Care: A literature review

abstract: Daily, massive volume of data are produced due to the internet of things' rapid development, which has now permeated the healthcare industry. Recent advances in data mining have spawned a new field of a study dubbed privacy-preserving data mining (PPDM). PPDM technique or approach enables the extraction of actionable insight from enormous volume of data while safeguarding the privacy of individual information and benefiting the entire society Medical research has taken a new course as a result of data mining with healthcare data to detect diseases earlier and improve patient care. Data integration necessitates the sharing of sensitive patient information. However, substantial privacy issues are raised in connection with the storage and transmission of potentially sensitive information. Disclosing sensitive information infringes on patients' privacy. This paper aims to conduct a review of related work on privacy-preserving mechanisms, data protection regulations, and mitigating tactics. The review concluded that no single strategy outperforms all others. Hence, future research should focus on adequate techniques for privacy solutions in the age of massive medical data and the standardization of evaluation standards.


title: A constructive approach for investigating the stability of incommensurate fractional differential systems

abstract: This paper is devoted to studying the asymptotic behaviour of solutions to generalized non-commensurate fractional systems. To this end, we first consider fractional systems with rational orders and introduce a criterion that is necessary and sufficient to ensure the stability of such systems. Next, from the fractional-order pseudospectrum definition proposed by \v{S}anca et al., we formulate the concept of a rational approximation for the fractional spectrum of a noncommensurate fractional systems with general, not necessarily rational, orders. Our first important new contribution is to show the equivalence between the fractional spectrum of a noncommensurate linear system and its rational approximation. With this result in hand, we use ideas developed in our earlier work to demonstrate the stability of an equilibrium point to nonlinear systems in arbitrary finite-dimensional spaces. A second novel aspect of our work is the fact that the approach is constructive. Finally, we give numerical simulations to illustrate the merit of the proposed theoretical results.


title: Security Challenges in Autonomous Systems Design

abstract: Autonomous systems are emerging in many application domains. With the recent advancements in artificial intelligence and machine learning, sensor technology, perception algorithms and robotics, scenarios previously requiring strong human involvement can be handled by autonomous systems. With the independence from human control, cybersecurity of such systems becomes even more critical as no human intervention in case of undesired behavior is possible. In this context, this paper discusses emerging security challenges in autonomous systems design which arise in many domains such as autonomous incident response, risk assessment, data availability, systems interaction, trustworthiness, updatability, access control, as well as the reliability and explainability of machine learning methods. In all these areas, this paper thoroughly discusses the state of the art, identifies emerging security challenges and proposes research directions to address these challenges for developing secure autonomous systems.


title: The theoretical limits of biometry

abstract: Biometry has proved its capability in terms of recognition accuracy. Now, it is widely used for automated border control with the biometric passport, to unlock a smartphone or a computer with a fingerprint or a face recognition algorithm. While identity verification is widely democratized, pure identification with no additional clues is still a work in progress. The identification difficulty depends on the population size, as the larger the group is, the larger the confusion risk. For collision prevention, biometric traits must be sufficiently distinguishable to scale to considerable groups, and algorithms should be able to capture their differences accurately. Most biometric works are purely experimental, and it is impossible to extrapolate the results to a smaller or a larger group. In this work, we propose a theoretical analysis of the distinguishability problem, which governs the error rates of biometric systems. We demonstrate simple relationships between the population size and the number of independent bits necessary to prevent collision in the presence of noise. This work provides the lowest lower bound for memory requirements. The results are very encouraging, as the biometry of the whole Earth population can fit in a regular disk, leaving some space for noise and redundancy.


title: A new numerical technique based on Chelyshkov polynomials for solving two-dimensional stochastic It\^o-Volterra Fredholm integral equation

abstract: In this paper, a two-dimensional operational matrix method based on Chelyshkov polynomials is implemented to numerically solve the two-dimensional stochastic It\^o-Volterra Fredholm integral equations. These equations arise in several problems such as an exponential population growth model with several independent white noise sources. In this paper a new stochastic operational matrix has been derived first time ever by using Chelyshkov polynomials. After that, the operational matrices are used to transform the It\^o-Volterra Fredholm integral equation into a system of linear algebraic equations by using Newton cotes nodes as collocation point that can be easily solved. Furthermore, the convergence and error bound of the suggested method are well established. In order to illustrate the effectiveness, plausibility, reliability, and applicability of the existing technique, two typical examples have been presented.


title: Technical Report relating to CVE-2022-46480, CVE-2023-26941, CVE-2023-26942, and CVE-2023-26943

abstract: The following technical report provides background information relating to four CVEs found in the following products: Ultraloq UL3 BT (CVE-2022-46480); Yale Conexis L1 Smart Lock (CVE-2023-26941); Yale IA-210 Intruder Alarm (CVE-2023-26942); Yale Keyless Smart Lock (CVE-2023-26943). The work discussed here was carried out by Ash Allen, Dr. Alexios Mylonas, and Dr. Stilianos Vidalis as part of a wider research project into smart device security. Responsible disclosure of all four issues has been made with the appropriate vendors, and they have been acknowledged as vulnerabilities.


title: A finite element method for stochastic diffusion equations using fluctuating hydrodynamics

abstract: We present a finite element approach for diffusion problems with thermal fluctuations based on a fluctuating hydrodynamics model. The governing transport equations are stochastic partial differential equations with a fluctuating forcing term. We propose a discrete formulation of the stochastic forcing term that has the correct covariance matrix up to a standard discretization error. Furthermore, to obtain a numerical solution with spatial correlations that converge to those of the continuum equation, we derive a linear mapping to transform the finite element solution into an equivalent discrete solution that is free from the artificial correlations introduced by the spatial discretization. The method is validated by applying it to two diffusion problems: a second-order diffusion equation and a fourth-order diffusion equation. The theoretical (continuum) solution to the first case presents spatially decorrelated fluctuations, while the second case presents fluctuations correlated over a finite length. In both cases, the numerical solution presents a structure factor that approximates well the continuum one.


title: Hypergraph Topological Features for Autoencoder-Based Intrusion Detection for Cybersecurity Data

abstract: In this position paper, we argue that when hypergraphs are used to capture multi-way local relations of data, their resulting topological features describe global behaviour. Consequently, these features capture complex correlations that can then serve as high fidelity inputs to autoencoder-driven anomaly detection pipelines. We propose two such potential pipelines for cybersecurity data, one that uses an autoencoder directly to determine network intrusions, and one that de-noises input data for a persistent homology system, PHANTOM. We provide heuristic justification for the use of the methods described therein for an intrusion detection pipeline for cyber data. We conclude by showing a small example over synthetic cyber attack data.


title: Can LLMs Patch Security Issues?

abstract: Large Language Models (LLMs) have shown impressive proficiency in code generation. Nonetheless, similar to human developers, these models might generate code that contains security vulnerabilities and flaws. Writing secure code remains a substantial challenge, as vulnerabilities often arise during interactions between programs and external systems or services, such as databases and operating systems. In this paper, we propose a novel approach, Feedback-Driven Solution Synthesis (FDSS), designed to explore the use of LLMs in receiving feedback from Bandit, which is a static code analysis tool, and then the LLMs generate potential solutions to resolve security vulnerabilities. Each solution, along with the vulnerable code, is then sent back to the LLM for code refinement. Our approach shows a significant improvement over the baseline and outperforms existing approaches. Furthermore, we introduce a new dataset, PythonSecurityEval, collected from real-world scenarios on Stack Overflow to evaluate the LLMs' ability to generate secure code. Code and data are available at \url{https://github.com/Kamel773/LLM-code-refine}


title: Secure Transformer Inference Protocol

abstract: Security of model parameters and user data is critical for Transformer-based services, such as ChatGPT. While recent strides in secure two-party protocols have successfully addressed security concerns in serving Transformer models, their adoption is practically infeasible due to the prohibitive cryptographic overheads involved. Drawing insights from our hands-on experience in developing two real-world Transformer-based services, we identify the inherent efficiency bottleneck in the two-party assumption. To overcome this limitation, we propose a novel three-party threat model. Within this framework, we design a semi-symmetric permutation-based protection scheme and present STIP, the first secure Transformer inference protocol without any inference accuracy loss. Experiments on representative Transformer models in real systems show that STIP has practical security and outperforms state-of-the-art secure two-party protocols in efficiency by millions of times.


title: A Quality-of-Service Compliance System using Federated Learning and Optimistic Rollups

abstract: Edge computing brings a new paradigm in which the sharing of computing, storage, and bandwidth resources as close as possible to the mobile devices or sensors generating a large amount of data. A parallel trend is the rise of phones and tablets as primary computing devices for many people. The powerful sensors present on these devices combined with the fact that they are mobile, mean they have access to data of an unprecedentedly diverse and private nature. Models learned on such data hold the promise of greatly improving usability by powering more intelligent applications, but the sensitive nature of the data means there are risks and responsibilities to storing it in a centralized location. To address the data privacy required for some data in these devices we propose the use of Federated Learning (FL) so that specific data about services performed by clients do not leave the source machines. Instead of sharing data, users collaboratively train a model by only sending weight updates to a server. However, the naive use of FL in those scenarios exposes it to a risk of corruption, whether intentional or not, during the training phase. To improve the security of the FL structure, we propose a decentralized Blockchain-based FL in an edge computing scenario. We also apply blockchain to create a reward mechanism in FL to enable incentive strategy for trainers.


title: Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injections

abstract: Recent developments in Large Language Models (LLMs) have manifested significant advancements. To facilitate safeguards against malicious exploitation, a body of research has concentrated on aligning LLMs with human preferences and inhibiting their generation of inappropriate content. Unfortunately, such alignments are often vulnerable: fine-tuning with a minimal amount of harmful data can easily unalign the target LLM. While being effective, such fine-tuning-based unalignment approaches also have their own limitations: (1) non-stealthiness, after fine-tuning, safety audits or red-teaming can easily expose the potential weaknesses of the unaligned models, thereby precluding their release/use. (2) non-persistence, the unaligned LLMs can be easily repaired through re-alignment, i.e., fine-tuning again with aligned data points. In this work, we show that it is possible to conduct stealthy and persistent unalignment on large language models via backdoor injections. We also provide a novel understanding on the relationship between the backdoor persistence and the activation pattern and further provide guidelines for potential trigger design. Through extensive experiments, we demonstrate that our proposed stealthy and persistent unalignment can successfully pass the safety evaluation while maintaining strong persistence against re-alignment defense.


title: Constructive Representation of Functions in $N$-Dimensional Sobolev Space

abstract: A new representation is proposed for functions in a Sobolev space with dominating mixed smoothness on an $N$-dimensional hyperrectangle. In particular, it is shown that these functions can be expressed in terms of their highest-order mixed derivative, as well as their lower-order derivatives evaluated along suitable boundaries of the domain. The proposed expansion is proven to be invertible, uniquely identifying any function in the Sobolev space with its derivatives and boundary values. Since these boundary values are either finite-dimensional, or exist in the space of square-integrable functions, this offers a bijective relation between the Sobolev space and $L_{2}$. Using this bijection, it is shown how approximation of functions in Sobolev space can be performed in the less restrictive space $L_{2}$, reconstructing such an approximation of the function from an $L_{2}$-optimal projection of its boundary values and highest-order derivative. This approximation method is presented using a basis of Legendre polynomials and a basis of step functions, and results using both bases are demonstrated to exhibit better convergence behavior than a direct projection approach for two numerical examples.


title: Bergeron: Combating Adversarial Attacks through a Conscience-Based Alignment Framework

abstract: Research into AI alignment has grown considerably since the recent introduction of increasingly capable Large Language Models (LLMs). Unfortunately, modern methods of alignment still fail to fully prevent harmful responses when models are deliberately attacked. These attacks can trick seemingly aligned models into giving manufacturing instructions for dangerous materials, inciting violence, or recommending other immoral acts. To help mitigate this issue, we introduce Bergeron: a framework designed to improve the robustness of LLMs against attacks without any additional parameter fine-tuning. Bergeron is organized into two tiers; with a secondary LLM emulating the conscience of a protected, primary LLM. This framework better safeguards the primary model against incoming attacks while monitoring its output for any harmful content. Empirical analysis shows that, by using Bergeron to complement models with existing alignment training, we can improve the robustness and safety of multiple, commonly used commercial and open-source LLMs.


title: Artificial Intelligence in Sustainable Vertical Farming

abstract: As global challenges of population growth, climate change, and resource scarcity intensify, the agricultural landscape is at a critical juncture. Sustainable vertical farming emerges as a transformative solution to address these challenges by maximizing crop yields in controlled environments. This paradigm shift necessitates the integration of cutting-edge technologies, with Artificial Intelligence (AI) at the forefront. The paper provides a comprehensive exploration of the role of AI in sustainable vertical farming, investigating its potential, challenges, and opportunities. The review synthesizes the current state of AI applications, encompassing machine learning, computer vision, the Internet of Things (IoT), and robotics, in optimizing resource usage, automating tasks, and enhancing decision-making. It identifies gaps in research, emphasizing the need for optimized AI models, interdisciplinary collaboration, and the development of explainable AI in agriculture. The implications extend beyond efficiency gains, considering economic viability, reduced environmental impact, and increased food security. The paper concludes by offering insights for stakeholders and suggesting avenues for future research, aiming to guide the integration of AI technologies in sustainable vertical farming for a resilient and sustainable future in agriculture.


title: Crypto analysis of the key distribution scheme using noise-free resistances

abstract: Known key exchange schemes offering information-theoretic (unconditional) security are complex and costly to implement. Nonetheless, they remain the only known methods for achieving unconditional security in key exchange. Therefore, the explorations for simpler solutions for information-theoretic security are highly justified. Lin et al. [1] proposed an interesting hardware key distribution scheme that utilizes thermal-noise-free resistances and DC voltages. A crypto analysis of this system is presented. It is shown that, if Eve gains access to the initial shared secret at any time in the past or future, she can successfully crack all the generated keys in the past and future, even retroactively, using passively obtained and recorded voltages and currents. Therefore, the scheme is not a secure key exchanger, but it is rather a key expander with no more information entropy than the originally shared secret at the beginning. We also point out that the proposed defense methods against active attacks do not function when the original shared secret is compromised because then the communication cannot be efficiently authenticated. However, they do work when an unconditionally secure key exchanger is applied to enable the authenticated communication protocol.


title: An algorithm for forensic toolmark comparisons

abstract: Forensic toolmark analysis traditionally relies on subjective human judgment, leading to inconsistencies and lack of transparency. The multitude of variables, including angles and directions of mark generation, further complicates comparisons. To address this, we first generate a dataset of 3D toolmarks from various angles and directions using consecutively manufactured slotted screwdrivers. By using PAM clustering, we find that there is clustering by tool rather than angle or direction. Using Known Match and Known Non-Match densities, we establish thresholds for classification. Fitting Beta distributions to the densities, we allow for the derivation of likelihood ratios for new toolmark pairs. With a cross-validated sensitivity of 98% and specificity of 96%, our approach enhances the reliability of toolmark analysis. This approach is applicable to slotted screwdrivers, and for screwdrivers that are made with a similar production method. With data collection of other tools and factors, it could be applied to compare toolmarks of other types. This empirically trained, open-source solution offers forensic examiners a standardized means to objectively compare toolmarks, potentially decreasing the number of miscarriages of justice in the legal system.


title: DeFi Security: Turning The Weakest Link Into The Strongest Attraction

abstract: The primary innovation we pioneer -- focused on blockchain information security -- is called the Safe-House. The Safe-House is badly needed since there are many ongoing hacks and security concerns in the DeFi space right now. The Safe-House is a piece of engineering sophistication that utilizes existing blockchain principles to bring about greater security when customer assets are moved around. The Safe-House logic is easily implemented as smart contracts on any decentralized system. The amount of funds at risk from both internal and external parties -- and hence the maximum one time loss -- is guaranteed to stay within the specified limits based on cryptographic fundamentals. To improve the safety of the Safe-House even further, we adapt the one time password (OPT) concept to operate using blockchain technology. Well suited to blockchain cryptographic nuances, our secondary advancement can be termed the one time next time password (OTNTP) mechanism. The OTNTP is designed to complement the Safe-House making it even more safe. We provide a detailed threat assessment model -- discussing the risks faced by DeFi protocols and the specific risks that apply to blockchain fund management -- and give technical arguments regarding how these threats can be overcome in a robust manner. We discuss how the Safe-House can participate with other external yield generation protocols in a secure way. We provide reasons for why the Safe-House increases safety without sacrificing the efficiency of operation. We start with a high level intuitive description of the landscape, the corresponding problems and our solutions. We then supplement this overview with detailed discussions including the corresponding mathematical formulations and pointers for technological implementation. This approach ensures that the article is accessible to a broad audience.


title: Enhancing IoT Security via Automatic Network Traffic Analysis: The Transition from Machine Learning to Deep Learning

abstract: This work provides a comparative analysis illustrating how Deep Learning (DL) surpasses Machine Learning (ML) in addressing tasks within Internet of Things (IoT), such as attack classification and device-type identification. Our approach involves training and evaluating a DL model using a range of diverse IoT-related datasets, allowing us to gain valuable insights into how adaptable and practical these models can be when confronted with various IoT configurations. We initially convert the unstructured network traffic data from IoT networks, stored in PCAP files, into images by processing the packet data. This conversion process adapts the data to meet the criteria of DL classification methods. The experiments showcase the ability of DL to surpass the constraints tied to manually engineered features, achieving superior results in attack detection and maintaining comparable outcomes in device-type identification. Additionally, a notable feature extraction time difference becomes evident in the experiments: traditional methods require around 29 milliseconds per data packet, while DL accomplishes the same task in just 2.9 milliseconds. The significant time gap, DL's superior performance, and the recognized limitations of manually engineered features, presents a compelling call to action within the IoT community. This encourages us to shift from exploring new IoT features for each dataset to addressing the challenges of integrating DL into IoT, making it a more efficient solution for real-world IoT scenarios.


title: FBChain: A Blockchain-based Federated Learning Model with Efficiency and Secure Communication

abstract: Privacy and security in the parameter transmission process of federated learning are currently among the most prominent concerns. However, there are two thorny problems caused by unprotected communication methods: "parameter-leakage" and "inefficient-communication". This article proposes Blockchain-based Federated Learning (FBChain) model for federated learning parameter communication to overcome the above two problems. First, we utilize the immutability of blockchain to store the global model and hash value of local model parameters in case of tampering during the communication process, protect data privacy by encrypting parameters, and verify data consistency by comparing the hash values of local parameters, thus addressing the "parameter-leakage" problem. Second, the Proof of Weighted Link Speed (PoWLS) consensus algorithm comprehensively selects nodes with the higher weighted link speed to aggregate global model and package blocks, thereby solving the "inefficient-communication" problem. Experimental results demonstrate the effectiveness of our proposed FBChain model and its ability to improve model communication efficiency in federated learning.


title: Privacy-Preserving Load Forecasting via Personalized Model Obfuscation

abstract: The widespread adoption of smart meters provides access to detailed and localized load consumption data, suitable for training building-level load forecasting models. To mitigate privacy concerns stemming from model-induced data leakage, federated learning (FL) has been proposed. This paper addresses the performance challenges of short-term load forecasting models trained with FL on heterogeneous data, emphasizing privacy preservation through model obfuscation. Our proposed algorithm, Privacy Preserving Federated Learning (PPFL), incorporates personalization layers for localized training at each smart meter. Additionally, we employ a differentially private mechanism to safeguard against data leakage from shared layers. Simulations on the NREL ComStock dataset corroborate the effectiveness of our approach.


title: Photonic Neural Networks and Optics-informed Deep Learning Fundamentals

abstract: The recent explosive compute growth, mainly fueled by the boost of AI and DNNs, is currently instigating the demand for a novel computing paradigm that can overcome the insurmountable barriers imposed by conventional electronic computing architectures. PNNs implemented on silicon integration platforms stand out as a promising candidate to endow NN hardware, offering the potential for energy efficient and ultra-fast computations through the utilization of the unique primitives of photonics i.e. energy efficiency, THz bandwidth and low-latency. Thus far, several demonstrations have revealed the huge potential of PNNs in performing both linear and non-linear NN operations at unparalleled speed and energy consumption metrics. Transforming this potential into a tangible reality for DL applications requires, however, a deep understanding of the basic PNN principles, requirements and challenges across all constituent architectural, technological and training aspects. In this tutorial, we, initially, review the principles of DNNs along with their fundamental building blocks, analyzing also the key mathematical operations needed for their computation in a photonic hardware. Then, we investigate, through an intuitive mathematical analysis, the interdependence of bit precision and energy efficiency in analog photonic circuitry, discussing the opportunities and challenges of PNNs. Followingly, a performance overview of PNN architectures, weight technologies and activation functions is presented, summarizing their impact in speed, scalability and power consumption. Finally, we provide an holistic overview of the optics-informed NN training framework that incorporates the physical properties of photonic building blocks into the training process in order to improve the NN classification accuracy and effectively elevate neuromorphic photonic hardware into high-performance DL computational settings.


title: A Posteriori Evaluation of a Physics-Constrained Neural Ordinary Differential Equations Approach Coupled with CFD Solver for Modeling Stiff Chemical Kinetics

abstract: The high computational cost associated with solving for detailed chemistry poses a significant challenge for predictive computational fluid dynamics (CFD) simulations of turbulent reacting flows. These models often require solving a system of coupled stiff ordinary differential equations (ODEs). While deep learning techniques have been experimented with to develop faster surrogate models, they often fail to integrate reliably with CFD solvers. This instability arises because deep learning methods optimize for training error without ensuring compatibility with ODE solvers, leading to accumulation of errors over time. Recently, NeuralODE-based techniques have offered a promising solution by effectively modeling chemical kinetics. In this study, we extend the NeuralODE framework for stiff chemical kinetics by incorporating mass conservation constraints directly into the loss function during training. This ensures that the total mass and the elemental mass are conserved, a critical requirement for reliable downstream integration with CFD solvers. Proof-of-concept studies are performed with physics-constrained neuralODE (PC-NODE) approach for homogeneous autoignition of hydrogen-air mixture over a range of composition and thermodynamic conditions. Our results demonstrate that this enhancement not only improves the physical consistency with respect to mass conservation criteria but also ensures better robustness. Lastly, a posteriori studies are performed wherein the trained PC-NODE model is coupled with a 3D CFD solver for computing the chemical source terms. PC-NODE is shown to be more accurate relative to the purely data-driven neuralODE approach. Moreover, PC-NODE also exhibits robustness and generalizability to unseen initial conditions from within (interpolative capability) as well as outside (extrapolative capability) the training regime.

categories: physics.comp-ph cs.LG physics.chem-ph physics.flu-dyn


title: Acoustic Cybersecurity: Exploiting Voice-Activated Systems

abstract: In this study, we investigate the emerging threat of inaudible acoustic attacks targeting digital voice assistants, a critical concern given their projected prevalence to exceed the global population by 2024. Our research extends the feasibility of these attacks across various platforms like Amazon's Alexa, Android, iOS, and Cortana, revealing significant vulnerabilities in smart devices. The twelve attack vectors identified include successful manipulation of smart home devices and automotive systems, potential breaches in military communication, and challenges in critical infrastructure security. We quantitatively show that attack success rates hover around 60%, with the ability to activate devices remotely from over 100 feet away. Additionally, these attacks threaten critical infrastructure, emphasizing the need for multifaceted defensive strategies combining acoustic shielding, advanced signal processing, machine learning, and robust user authentication to mitigate these risks.


title: Presentation Attack detection using Wavelet Transform and Deep Residual Neural Net

abstract: Biometric authentication is becoming more prevalent for secured authentication systems. However, the biometric substances can be deceived by the imposters in several ways. Among other imposter attacks, print attacks, mask attacks, and replay attacks fall under the presentation attack category. The bio-metric images, especially the iris and face, are vulnerable to different presentation attacks. This research applies deep learning approaches to mitigate presentation attacks in a biometric access control system. Our contribution in this paper is two-fold: First, we applied the wavelet transform to extract the features from the biometric images. Second, we modified the deep residual neural net and applied it to the spoof datasets in an attempt to detect the presentation attacks. This research applied the proposed approach to biometric spoof datasets, namely ATVS, CASIA two class, and CASIA cropped image sets. The datasets used in this research contain images that are captured in both a controlled and uncontrolled environment along with different resolutions and sizes. We obtained the best accuracy of 93% on the ATVS Iris datasets. For CASIA two class and CASIA cropped datasets, we achieved test accuracies of 91% and 82%, respectively.


title: Presentation Attack Detection using Convolutional Neural Networks and Local Binary Patterns

abstract: The use of biometrics to authenticate users and control access to secure areas has become extremely popular in recent years, and biometric access control systems are frequently used by both governments and private corporations. However, these systems may represent risks to security when deployed without considering the possibility of biometric presentation attacks (also known as spoofing). Presentation attacks are a serious threat because they do not require significant time, expense, or skill to carry out while remaining effective against many biometric systems in use today. This research compares three different software-based methods for facial and iris presentation attack detection in images. The first method uses Inception-v3, a pre-trained deep Convolutional Neural Network (CNN) made by Google for the ImageNet challenge, which is retrained for this problem. The second uses a shallow CNN based on a modified Spoofnet architecture, which is trained normally. The third is a texture-based method using Local Binary Patterns (LBP). The datasets used are the ATVS-FIr dataset, which contains real and fake iris images, and the CASIA Face Anti-Spoofing Dataset, which contains real images as well as warped photos, cut photos, and video replay presentation attacks. We also present a third set of results, based on cropped versions of the CASIA images.


title: DeepTreeGANv2: Iterative Pooling of Point Clouds

abstract: In High Energy Physics, detailed and time-consuming simulations are used for particle interactions with detectors. To bypass these simulations with a generative model, the generation of large point clouds in a short time is required, while the complex dependencies between the particles must be correctly modelled. Particle showers are inherently tree-based processes, as each particle is produced by the decay or detector interaction of a particle of the previous generation. In this work, we present a significant extension to DeepTreeGAN, featuring a critic, that is able to aggregate such point clouds iteratively in a tree-based manner. We show that this model can reproduce complex distributions, and we evaluate its performance on the public JetNet 150 dataset.


title: Who is leading in AI? An analysis of industry AI research

abstract: AI research is increasingly industry-driven, making it crucial to understand company contributions to this field. We compare leading AI companies by research publications, citations, size of training runs, and contributions to algorithmic innovations. Our analysis reveals the substantial role played by Google, OpenAI and Meta. We find that these three companies have been responsible for some of the largest training runs, developed a large fraction of the algorithmic innovations that underpin large language models, and led in various metrics of citation impact. In contrast, leading Chinese companies such as Tencent and Baidu had a lower impact on many of these metrics compared to US counterparts. We observe many industry labs are pursuing large training runs, and that training runs from relative newcomers -- such as OpenAI and Anthropic -- have matched or surpassed those of long-standing incumbents such as Google. The data reveals a diverse ecosystem of companies steering AI progress, though US labs such as Google, OpenAI and Meta lead across critical metrics.


title: Advancing AI Audits for Enhanced AI Governance

abstract: As artificial intelligence (AI) is integrated into various services and systems in society, many companies and organizations have proposed AI principles, policies, and made the related commitments. Conversely, some have proposed the need for independent audits, arguing that the voluntary principles adopted by the developers and providers of AI services and systems insufficiently address risk. This policy recommendation summarizes the issues related to the auditing of AI services and systems and presents three recommendations for promoting AI auditing that contribute to sound AI governance. Recommendation1.Development of institutional design for AI audits. Recommendation2.Training human resources for AI audits. Recommendation3. Updating AI audits in accordance with technological progress. In this policy recommendation, AI is assumed to be that which recognizes and predicts data with the last chapter outlining how generative AI should be audited.


title: AI-driven E-Liability Knowledge Graphs: A Comprehensive Framework for Supply Chain Carbon Accounting and Emissions Liability Management

abstract: While carbon accounting plays a fundamental role in our fight against climate change, it is not without its challenges. We begin the paper with a critique of the conventional carbon accounting practices, after which we proceed to introduce the E-liability carbon accounting methodology and Emissions Liability Management (ELM) originally proposed by Kaplan and Ramanna, highlighting their strengths. Recognizing the immense value of this novel approach for real-world carbon accounting improvement, we introduce a novel data-driven integrative framework that leverages AI and computation - the E-Liability Knowledge Graph framework - to achieve real-world implementation of the E-liability carbon accounting methodology. In addition to providing a path-to-implementation, our proposed framework brings clarity to the complex environmental interactions within supply chains, thus enabling better informed and more responsible decision-making. We analyze the implementation aspects of this framework and conclude with a discourse on the role of this AI-aided knowledge graph in ensuring the transparency and decarbonization of global supply chains.


title: Retail Analytics in the New Normal: The Influence of Artificial Intelligence and the Covid-19 Pandemic

abstract: The COVID-19 pandemic has severely disrupted the retail landscape and has accelerated the adoption of innovative technologies. A striking example relates to the proliferation of online grocery orders and the technology deployed to facilitate such logistics. In fact, for many retailers, this disruption was a wake-up call after which they started recognizing the power of data analytics and artificial intelligence (AI). In this article, we discuss the opportunities that AI can offer to retailers in the new normal retail landscape. Some of the techniques described have been applied at scale to adapt previously deployed AI models, whereas in other instances, fresh solutions needed to be developed to help retailers cope with recent disruptions, such as unexpected panic buying, retraining predictive models, and leveraging online-offline synergies.


title: chatGPT for generating questions and assessments based on accreditations

abstract: This research aims to take advantage of artificial intelligence techniques in producing students assessment that is compatible with the different academic accreditations of the same program. The possibility of using generative artificial intelligence technology was studied to produce an academic accreditation compliant test the National Center for Academic Accreditation of Kingdom of Saudi Arabia and Accreditation Board for Engineering and Technology. A novel method was introduced to map the verbs used to create the questions introduced in the tests. The method allows a possibility of using the generative artificial intelligence technology to produce and check the validity of questions that measure educational outcomes. A questionnaire was distributed to ensure that the use of generative artificial intelligence to create exam questions is acceptable by the faculty members, as well as to ask about the acceptance of assistance in validating questions submitted by faculty members and amending them in accordance with academic accreditations. The questionnaire was distributed to faculty members of different majors in the Kingdom of Saudi Arabias universities. one hundred twenty responses obtained with eight five percentile approval percentage for generate complete exam questions by generative artificial intelligence . Whereas ninety eight percentage was the approval percentage for editing and improving already existed questions.


title: Tokenized Model: A Blockchain-Empowered Decentralized Model Ownership Verification Platform

abstract: With the development of practical deep learning models like generative AI, their excellent performance has brought huge economic value. For instance, ChatGPT has attracted more than 100 million users in three months. Since the model training requires a lot of data and computing power, a well-performing deep learning model is behind a huge effort and cost. Facing various model attacks, unauthorized use and abuse from the network that threaten the interests of model owners, in addition to considering legal and other administrative measures, it is equally important to protect the model's copyright from the technical means. By using the model watermarking technology, we point out the possibility of building a unified platform for model ownership verification. Given the application history of blockchain in copyright verification and the drawbacks of a centralized third-party, this paper considers combining model watermarking technology and blockchain to build a unified model copyright protection platform. By a new solution we called Tokenized Model, it protects the model's copyright by reliable ownership record and verification mechanism. It also promotes the financial value of model by constructing the model's transaction process and contribution shares of a model. In the typical case study, we also study the various performance under usual scenario to verify the effectiveness of this platform.


title: The K-theory of the conjugation action

abstract: In 1999, Brylinski and Zhang computed the complex equivariant K-theory of the conjugation self-action of a compact, connected Lie group with torsion-free fundamental group. In this note we show it is possible to do so in under a page.


title: Elijah: Eliminating Backdoors Injected in Diffusion Models via Distribution Shift

abstract: Diffusion models (DM) have become state-of-the-art generative models because of their capability to generate high-quality images from noises without adversarial training. However, they are vulnerable to backdoor attacks as reported by recent studies. When a data input (e.g., some Gaussian noise) is stamped with a trigger (e.g., a white patch), the backdoored model always generates the target image (e.g., an improper photo). However, effective defense strategies to mitigate backdoors from DMs are underexplored. To bridge this gap, we propose the first backdoor detection and removal framework for DMs. We evaluate our framework Elijah on hundreds of DMs of 3 types including DDPM, NCSN and LDM, with 13 samplers against 3 existing backdoor attacks. Extensive experiments show that our approach can have close to 100% detection accuracy and reduce the backdoor effects to close to zero without significantly sacrificing the model utility.


title: MIA-BAD: An Approach for Enhancing Membership Inference Attack and its Mitigation with Federated Learning

abstract: The membership inference attack (MIA) is a popular paradigm for compromising the privacy of a machine learning (ML) model. MIA exploits the natural inclination of ML models to overfit upon the training data. MIAs are trained to distinguish between training and testing prediction confidence to infer membership information. Federated Learning (FL) is a privacy-preserving ML paradigm that enables multiple clients to train a unified model without disclosing their private data. In this paper, we propose an enhanced Membership Inference Attack with the Batch-wise generated Attack Dataset (MIA-BAD), a modification to the MIA approach. We investigate that the MIA is more accurate when the attack dataset is generated batch-wise. This quantitatively decreases the attack dataset while qualitatively improving it. We show how training an ML model through FL, has some distinct advantages and investigate how the threat introduced with the proposed MIA-BAD approach can be mitigated with FL approaches. Finally, we demonstrate the qualitative effects of the proposed MIA-BAD methodology by conducting extensive experiments with various target datasets, variable numbers of federated clients, and training batch sizes.


title: A Case for Competent AI Systems $-$ A Concept Note

abstract: The efficiency of an AI system is contingent upon its ability to align with the specified requirements of a given task. How-ever, the inherent complexity of tasks often introduces the potential for harmful implications or adverse actions. This note explores the critical concept of capability within AI systems, representing what the system is expected to deliver. The articulation of capability involves specifying well-defined out-comes. Yet, the achievement of this capability may be hindered by deficiencies in implementation and testing, reflecting a gap in the system's competency (what it can do vs. what it does successfully). A central challenge arises in elucidating the competency of an AI system to execute tasks effectively. The exploration of system competency in AI remains in its early stages, occasionally manifesting as confidence intervals denoting the probability of success. Trust in an AI system hinges on the explicit modeling and detailed specification of its competency, connected intricately to the system's capability. This note explores this gap by proposing a framework for articulating the competency of AI systems. Motivated by practical scenarios such as the Glass Door problem, where an individual inadvertently encounters a glass obstacle due to a failure in their competency, this research underscores the imperative of delving into competency dynamics. Bridging the gap between capability and competency at a detailed level, this note contributes to advancing the discourse on bolstering the reliability of AI systems in real-world applications.


title: Anti-Sexism Alert System: Identification of Sexist Comments on Social Media Using AI Techniques

abstract: Social relationships in the digital sphere are becoming more usual and frequent, and they constitute a very important aspect for all of us. {Violent interactions in this sphere are very frequent, and have serious effects on the victims}. Within this global scenario, there is one kind of digital violence that is becoming really worrying: sexism against women. Sexist comments that are publicly posted in social media (newspaper comments, social networks, etc.), usually obtain a lot of attention and become viral, with consequent damage to the persons involved. In this paper, we introduce an anti-sexism alert system, based on natural language processing (NLP) and artificial intelligence (AI), that analyzes any public post, and decides if it could be considered a sexist comment or not. Additionally, this system also works on analyzing all the public comments linked to any multimedia content (piece of news, video, tweet, etc.) and decides, using a color-based system similar to traffic lights, if there is sexism in the global set of posts. We have created a labeled data set in Spanish, since the majority of studies focus on English, to train our system, which offers a very good performance after the validation experiments.


title: Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective

abstract: Inverse Reinforcement Learning (IRL) -- the problem of learning reward functions from demonstrations of an \emph{expert policy} -- plays a critical role in developing intelligent systems. While widely used in applications, theoretical understandings of IRL present unique challenges and remain less developed compared with standard RL. For example, it remains open how to do IRL efficiently in standard \emph{offline} settings with pre-collected data, where states are obtained from a \emph{behavior policy} (which could be the expert policy itself), and actions are sampled from the expert policy. This paper provides the first line of results for efficient IRL in vanilla offline and online settings using polynomial samples and runtime. Our algorithms and analyses seamlessly adapt the pessimism principle commonly used in offline RL, and achieve IRL guarantees in stronger metrics than considered in existing work. We provide lower bounds showing that our sample complexities are nearly optimal. As an application, we also show that the learned rewards can \emph{transfer} to another target MDP with suitable guarantees when the target MDP satisfies certain similarity assumptions with the original (source) MDP.


title: LEAP: LLM-Generation of Egocentric Action Programs

abstract: We introduce LEAP (illustrated in Figure 1), a novel method for generating video-grounded action programs through use of a Large Language Model (LLM). These action programs represent the motoric, perceptual, and structural aspects of action, and consist of sub-actions, pre- and post-conditions, and control flows. LEAP's action programs are centered on egocentric video and employ recent developments in LLMs both as a source for program knowledge and as an aggregator and assessor of multimodal video information. We apply LEAP over a majority (87\%) of the training set of the EPIC Kitchens dataset, and release the resulting action programs as a publicly available dataset here (https://drive.google.com/drive/folders/1Cpkw_TI1IIxXdzor0pOXG3rWJWuKU5Ex?usp=drive_link). We employ LEAP as a secondary source of supervision, using its action programs in a loss term applied to action recognition and anticipation networks. We demonstrate sizable improvements in performance in both tasks due to training with the LEAP dataset. Our method achieves 1st place on the EPIC Kitchens Action Recognition leaderboard as of November 17 among the networks restricted to RGB-input (see Supplementary Materials).


title: Combinatorial decompositions for deformed or decorated classes of maps

abstract: The perturbative expansion of tensorial field theories in Feynman graphs can be interpreted as weighted generating series of some piecewise linear varieties. This simple fact establishes a link between two a priori distinct fields: the combinatorics of discrete manifolds on one hand and tensorial field theories on the other hand. In this thesis, we study different aspects revolving around this connection between combinatorics and field theory. First, we consider constellations model, which generalize maps and their algebraic properties. This makes them suited to probe the b-deformation, a deformation of the algebra of symmetric functions. We will study the constraints satisfied by the generating series of cubical b-deformed constellations. Second, we analyze the double scaling limit of particular tensor models of order 3. For tensor of order greater than two, the nature of the 1/N-expansion is qualitatively different from the matrix case of order 2. In particular, only the leading order graphs are fully characterized. Despite this fact, it is possible to identify graphs of subleading orders contributing to the double scaling limit by implementing the scheme decomposition for Feynman graphs of these theories. An analysis of the singularity of the schemes then allows us to give a complete characterization of the graphs contributing to the double scaling limit. Finally, we investigate a particular link between a tensor and a vector field theory which both admit a melonic limit. Namely, we will show that we can obtain the vectorial Amit-Roginski model by considering perturbations around a classical solution of the Boulatov model, a tensorial theory. We give sufficient conditions on the classical solution so that the effective action for the perturbation around this solution takes the form of the Amit-Roginski action.


title: VA3: Virtually Assured Amplification Attack on Probabilistic Copyright Protection for Text-to-Image Generative Models

abstract: The booming use of text-to-image generative models has raised concerns about their high risk of producing copyright-infringing content. While probabilistic copyright protection methods provide a probabilistic guarantee against such infringement, in this paper, we introduce Virtually Assured Amplification Attack (VA3), a novel online attack framework that exposes the vulnerabilities of these protection mechanisms. The proposed framework significantly amplifies the probability of generating infringing content on the sustained interactions with generative models and a non-trivial lower-bound on the success probability of each engagement. Our theoretical and experimental results demonstrate the effectiveness of our approach under various scenarios. These findings highlight the potential risk of implementing probabilistic copyright protection in practical applications of text-to-image generative models. Code is available at https://github.com/South7X/VA3.


title: Uniform estimates for a fully discrete scheme integrating the linear heat equation on a bounded interval with pure Neumann boundary conditions

abstract: This manuscript deals with the analysis of numerical methods for the full discretization (in time and space) of the linear heat equation with Neumann boundary conditions, and it provides the reader with error estimates that are uniform in time. First, we consider the homogeneous equation with homogeneous Neumann boundary conditions over a finite interval. Using finite differences in space and the Euler method in time, we prove that our method is of order 1 in space, uniformly in time, under a classical CFL condition, and despite its lack of consistency at the boundaries. Second, we consider the nonhomogeneous equation with nonhomogeneous Neumann boundary conditions over a finite interval. Using a tailored similar scheme, we prove that our method is also of order 1 in space, uniformly in time, under a classical CFL condition. We indicate how this numerical method allows for a new way to compute steady states of such equations when they exist. We conclude by several numerical experiments to illustrate the sharpness and relevance of our theoretical results, as well as to examine situations that do not meet the hypotheses of our theoretical results, and to illustrate how our results extend to higher dimensions.


title: Photo-induced charge carrier dynamics in a semiconductor-based ion trap investigated via motion-sensitive qubit transitions

abstract: Ion trap systems built upon microfabricated chips have emerged as a promising platform for quantum computing to achieve reproducible and scalable structures. However, photo-induced charging of materials in such chips can generate undesired stray electric fields that disrupt the quantum state of the ion, limiting high-fidelity quantum control essential for practical quantum computing. While crude understanding of the phenomena has been gained heuristically over the past years, explanations for the microscopic mechanism of photo-generated charge carrier dynamics remains largely elusive. Here, we present a photo-induced charging model for semiconductors, whose verification is enabled by a systematic interaction between trapped ions and photo-induced stray fields from exposed silicon surfaces in our chip. We use motion-sensitive qubit transitions to directly characterize the stray field and analyze its effect on the quantum dynamics of the trapped ion. In contrast to incoherent errors arising from the thermal motion of the ion, coherent errors are induced by the stray field, whose effect is significantly imprinted during the quantum control of the ion. These errors are investigated in depth and methods to mitigate them are discussed. Finally, we extend the implications of our study to other photo-induced charging mechanisms prevalent in ion traps.


title: Open data ecosystems: what models to co-create service innovations in smart cities?

abstract: While smart cities are recently providing open data, how to organise the collective creation of data, knowledge and related products and services produced from this collective resource, still remains to be thought. This paper aims at gathering the literature review on open data ecosystems to tackle the following research question: what models can be imagined to stimulate the collective co-creation of services between smart cities' stakeholders acting as providers and users of open data? Such issue is currently at stake in many municipalities such as Lisbon which decided to position itself as a platform (O'Reilly, 2010) in the local digital ecosystem. With the implementation of its City Operation Center (COI), Lisbon's municipality provides an Information Infrastructure (Bowker et al., 2009) to many different types of actors such as telecom companies, municipalities, energy utilities or transport companies. Through this infrastructure, Lisbon encourages such actors to gather, integrate and release heterogeneous datasets and tries to orchestrate synergies among them so data-driven solution to urban problems can emerge (Carvalho and Vale, 2018). The remaining question being: what models for the municipalities such as Lisbon to lean on so as to drive this cutting-edge type of service innovation?


title: The Geopolitics behind the Cryptocurrency Mining in Kazakhstan

abstract: On 25 January 2022, following a beginning to the year which had been affected by major political unrest resulting in the deaths of more than 230 people 1 , Kazakhstan was hit by a major power blackout. For around twelve hours, the entire southern regions of the country were left without power, as were Uzbekistan and Kyrgyzstan, two neighboring countries with which the southern country shares the same electricity network. This event is the culmination of several months of over-consumption, power surges and localized outages on the Central Asian networks. In Kazakhstan, electricity consumption rose by 8% in 2021, according to Ministry of Energy 2 , the national electricity operator, compared with the 1-2% annual increase previously recorded. This explosion in consumption is said to have been driven by the widespread development of cryptocurrency mining, a digital activity that can be extremely energy-intensive when operated on a large scale.


title: Atmospheric Escape From Three Terrestrial Planets in the L 98-59 System

abstract: A critically important process affecting the climate evolution and potential habitability of an exoplanet is atmospheric escape, in which high-energy radiation from a star drives the escape of hydrogen atoms and other light elements from a planet's atmosphere. L 98-59 is a benchmark system for studying such atmospheric processes, with three transiting terrestrial-size planets receiving Venus-like instellations (4-25 S$_\oplus$) from their M3 host star. We use the VPLanet model to simulate the evolution of the L 98-59 system and the atmospheric escape of its inner three small planets, given different assumed initial water quantities. We find that, regardless of their initial water content, all three planets accumulate significant quantities of oxygen due to efficient water photolysis and hydrogen loss. All three planets also receive enough XUV flux to drive rapid water loss, which considerably affects their developing climates and atmospheres. Even in scenarios of low initial water content, our results suggest that the James Webb Space Telescope (JWST) will be sensitive to observations of retained oxygen on the L 98-59 planets in its future scheduled observations, with planets b and c being the most likely targets to possess an extended atmosphere. Our results constrain the atmospheric evolution of these small rocky planets, and they provide context for current and future observations of the L 98-59 system to generalize our understanding of multi-terrestrial planet systems.


title: MoMask: Generative Masked Modeling of 3D Human Motions

abstract: We introduce MoMask, a novel masked modeling framework for text-driven 3D human motion generation. In MoMask, a hierarchical quantization scheme is employed to represent human motion as multi-layer discrete motion tokens with high-fidelity details. Starting at the base layer, with a sequence of motion tokens obtained by vector quantization, the residual tokens of increasing orders are derived and stored at the subsequent layers of the hierarchy. This is consequently followed by two distinct bidirectional transformers. For the base-layer motion tokens, a Masked Transformer is designated to predict randomly masked motion tokens conditioned on text input at training stage. During generation (i.e. inference) stage, starting from an empty sequence, our Masked Transformer iteratively fills up the missing tokens; Subsequently, a Residual Transformer learns to progressively predict the next-layer tokens based on the results from current layer. Extensive experiments demonstrate that MoMask outperforms the state-of-art methods on the text-to-motion generation task, with an FID of 0.045 (vs e.g. 0.141 of T2M-GPT) on the HumanML3D dataset, and 0.228 (vs 0.514) on KIT-ML, respectively. MoMask can also be seamlessly applied in related tasks without further model fine-tuning, such as text-guided temporal inpainting.


title: Yang-Baxter integrable open quantum systems

abstract: This work is based on the author's PhD thesis. The main result of the thesis is the use of the boost operator to develop a systematic method to construct new integrable spin chains with nearest-neighbour interaction and characterized by an R-matrix of non-difference form. This method has the advantage of being more feasible than directly solving the Yang-Baxter equation. We applied this approach to various contexts, in particular, in the realm of open quantum systems, we achieved the first classification of integrable Lindbladians. These operators describe the dynamics of physical systems in contact with a Markovian environment. Within this classification, we discovered a novel deformation of the Hubbard model spanning three sites of the spin chain. Additionally, we applied our method to classify models with $\mathfrak{su}(2)\oplus \mathfrak{su}(2)$ symmetry and we recovered the matrix part of the S-matrix of $AdS_5 \times S^5$ derived by requiring centrally extended $\mathfrak{su}(2|2)$ symmetry. Furthermore, we focus on spin 1/2 chain on models of 8-Vertex type and we showed that the models of this class satisfy the free fermion condition. This enables us to express the transfer matrix associated to some of the models in a diagonal form, simplifying the computation of the eigenvalues and eigenvectors. The thesis is based on the works: 2003.04332, 2010.11231, 2011.08217, 2101.08279, 2207.14193, 2301.01612, 2305.01922.

categories: cond-mat.stat-mech hep-th math-ph math.MP nlin.SI


title: Unsupervised Keypoints from Pretrained Diffusion Models

abstract: Unsupervised learning of keypoints and landmarks has seen significant progress with the help of modern neural network architectures, but performance is yet to match the supervised counterpart, making their practicability questionable. We leverage the emergent knowledge within text-to-image diffusion models, towards more robust unsupervised keypoints. Our core idea is to find text embeddings that would cause the generative model to consistently attend to compact regions in images (i.e. keypoints). To do so, we simply optimize the text embedding such that the cross-attention maps within the denoising network are localized as Gaussians with small standard deviations. We validate our performance on multiple datasets: the CelebA, CUB-200-2011, Tai-Chi-HD, DeepFashion, and Human3.6m datasets. We achieve significantly improved accuracy, sometimes even outperforming supervised ones, particularly for data that is non-aligned and less curated. Our code is publicly available and can be found through our project page: https://ubc-vision.github.io/StableKeypoints/


title: Exploring Factors Affecting Pedestrian Crash Severity Using TabNet: A Deep Learning Approach

abstract: This study presents the first investigation of pedestrian crash severity using the TabNet model, a novel tabular deep learning method exceptionally suited for analyzing the tabular data inherent in transportation safety research. Through the application of TabNet to a comprehensive dataset from Utah covering the years 2010 to 2022, we uncover intricate factors contributing to pedestrian crash severity. The TabNet model, capitalizing on its compatibility with structured data, demonstrates remarkable predictive accuracy, eclipsing that of traditional models. It identifies critical variables, such as pedestrian age, involvement in left or right turns, lighting conditions, and alcohol consumption, which significantly influence crash outcomes. The utilization of SHapley Additive exPlanations (SHAP) enhances our ability to interpret the TabNet model's predictions, ensuring transparency and understandability in our deep learning approach. The insights derived from our analysis provide a valuable compass for transportation safety engineers and policymakers, enabling the identification of pivotal factors that affect pedestrian crash severity. Such knowledge is instrumental in formulating precise, data-driven interventions aimed at bolstering pedestrian safety across diverse urban and rural settings.


title: Predicting breast cancer with AI for individual risk-adjusted MRI screening and early detection

abstract: Women with an increased life-time risk of breast cancer undergo supplemental annual screening MRI. We propose to predict the risk of developing breast cancer within one year based on the current MRI, with the objective of reducing screening burden and facilitating early detection. An AI algorithm was developed on 53,858 breasts from 12,694 patients who underwent screening or diagnostic MRI and accrued over 12 years, with 2,331 confirmed cancers. A first U-Net was trained to segment lesions and identify regions of concern. A second convolutional network was trained to detect malignant cancer using features extracted by the U-Net. This network was then fine-tuned to estimate the risk of developing cancer within a year in cases that radiologists considered normal or likely benign. Risk predictions from this AI were evaluated with a retrospective analysis of 9,183 breasts from a high-risk screening cohort, which were not used for training. Statistical analysis focused on the tradeoff between number of omitted exams versus negative predictive value, and number of potential early detections versus positive predictive value. The AI algorithm identified regions of concern that coincided with future tumors in 52% of screen-detected cancers. Upon directed review, a radiologist found that 71.3% of cancers had a visible correlate on the MRI prior to diagnosis, 65% of these correlates were identified by the AI model. Reevaluating these regions in 10% of all cases with higher AI-predicted risk could have resulted in up to 33% early detections by a radiologist. Additionally, screening burden could have been reduced in 16% of lower-risk cases by recommending a later follow-up without compromising current interval cancer rate. With increasing datasets and improving image quality we expect this new AI-aided, adaptive screening to meaningfully reduce screening burden and improve early detection.


title: GLiDR: Topologically Regularized Graph Generative Network for Sparse LiDAR Point Clouds

abstract: Sparse LiDAR point clouds cause severe loss of detail of static structures and reduce the density of static points available for navigation. Reduced density can be detrimental to navigation under several scenarios. We observe that despite high sparsity, in most cases, the global topology of LiDAR outlining the static structures can be inferred. We utilize this property to obtain a backbone skeleton of a LiDAR scan in the form of a single connected component that is a proxy to its global topology. We utilize the backbone to augment new points along static structures to overcome sparsity. Newly introduced points could correspond to existing static structures or to static points that were earlier obstructed by dynamic objects. To the best of our knowledge, we are the first to use such a strategy for sparse LiDAR point clouds. Existing solutions close to our approach fail to identify and preserve the global static LiDAR topology and generate sub-optimal points. We propose GLiDR, a Graph Generative network that is topologically regularized using 0-dimensional Persistent Homology ($\mathcal{PH}$) constraints. This enables GLiDR to introduce newer static points along a topologically consistent global static LiDAR backbone. GLiDR generates precise static points using $32\times$ sparser dynamic scans and performs better than the baselines across three datasets. GLiDR generates a valuable byproduct - an accurate binary segmentation mask of static and dynamic objects that are helpful for navigation planning and safety in constrained environments. The newly introduced static points allow GLiDR to outperform LiDAR-based navigation using SLAM in several settings. Source code is available at https://kshitijbhat.github.io/glidr


title: SICKLE: A Multi-Sensor Satellite Imagery Dataset Annotated with Multiple Key Cropping Parameters

abstract: The availability of well-curated datasets has driven the success of Machine Learning (ML) models. Despite greater access to earth observation data in agriculture, there is a scarcity of curated and labelled datasets, which limits the potential of its use in training ML models for remote sensing (RS) in agriculture. To this end, we introduce a first-of-its-kind dataset called SICKLE, which constitutes a time-series of multi-resolution imagery from 3 distinct satellites: Landsat-8, Sentinel-1 and Sentinel-2. Our dataset constitutes multi-spectral, thermal and microwave sensors during January 2018 - March 2021 period. We construct each temporal sequence by considering the cropping practices followed by farmers primarily engaged in paddy cultivation in the Cauvery Delta region of Tamil Nadu, India; and annotate the corresponding imagery with key cropping parameters at multiple resolutions (i.e. 3m, 10m and 30m). Our dataset comprises 2,370 season-wise samples from 388 unique plots, having an average size of 0.38 acres, for classifying 21 crop types across 4 districts in the Delta, which amounts to approximately 209,000 satellite images. Out of the 2,370 samples, 351 paddy samples from 145 plots are annotated with multiple crop parameters; such as the variety of paddy, its growing season and productivity in terms of per-acre yields. Ours is also one among the first studies that consider the growing season activities pertinent to crop phenology (spans sowing, transplanting and harvesting dates) as parameters of interest. We benchmark SICKLE on three tasks: crop type, crop phenology (sowing, transplanting, harvesting), and yield prediction


title: Fully lifted random duality theory

abstract: We study a generic class of \emph{random optimization problems} (rops) and their typical behavior. The foundational aspects of the random duality theory (RDT), associated with rops, were discussed in \cite{StojnicRegRndDlt10}, where it was shown that one can often infer rops' behavior even without actually solving them. Moreover, \cite{StojnicRegRndDlt10} uncovered that various quantities relevant to rops (including, for example, their typical objective values) can be determined (in a large dimensional context) even completely analytically. The key observation was that the \emph{strong deterministic duality} implies the, so-called, \emph{strong random duality} and therefore the full exactness of the analytical RDT characterizations. Here, we attack precisely those scenarios where the strong deterministic duality is not necessarily present and connect them to the recent progress made in studying bilinearly indexed (bli) random processes in \cite{Stojnicnflgscompyx23,Stojnicsflgscompyx23}. In particular, utilizing a fully lifted (fl) interpolating comparison mechanism introduced in \cite{Stojnicnflgscompyx23}, we establish corresponding \emph{fully lifted} RDT (fl RDT). We then rely on a stationarized fl interpolation realization introduced in \cite{Stojnicsflgscompyx23} to obtain complete \emph{statitionarized} fl RDT (sfl RDT). A few well known problems are then discussed as illustrations of a wide range of practical applications implied by the generality of the considered rops.

categories: math.PR cs.IT math.IT math.OC math.ST stat.TH


title: Studying Hopfield models via fully lifted random duality theory

abstract: Relying on a recent progress made in studying bilinearly indexed (bli) random processes in \cite{Stojnicnflgscompyx23,Stojnicsflgscompyx23}, the main foundational principles of fully lifted random duality theory (fl RDT) were established in \cite{Stojnicflrdt23}. We here study famous Hopfield models and show that their statistical behavior can be characterized via the fl RDT. Due to a nestedly lifted nature, the resulting characterizations and, therefore, the whole analytical machinery that produces them, become fully operational only if one can successfully conduct underlying numerical evaluations. After conducting such evaluations for both positive and negative Hopfield models, we observe a remarkably fast convergence of the fl RDT mechanism. Namely, for the so-called square case, the fourth decimal precision is achieved already on the third (second non-trivial) level of lifting (3-sfl RDT) for the positive and on the fourth (third non-trivial) level of lifting (4-sfl RDT) for the corresponding negative model. In particular, we obtain the scaled ground state free energy $\approx 1.7788$ for the positive and $\approx 0.3279$ for the negative model.

categories: math.PR cond-mat.stat-mech cs.IT math-ph math.IT math.MP math.ST stat.TH


title: CRAFT: Contextual Re-Activation of Filters for face recognition Training

abstract: The first layer of a deep CNN backbone applies filters to an image to extract the basic features available to later layers. During training, some filters may go inactive, mean ing all weights in the filter approach zero. An inactive fil ter in the final model represents a missed opportunity to extract a useful feature. This phenomenon is especially prevalent in specialized CNNs such as for face recogni tion (as opposed to, e.g., ImageNet). For example, in one the most widely face recognition model (ArcFace), about half of the convolution filters in the first layer are inactive. We propose a novel approach designed and tested specif ically for face recognition networks, known as "CRAFT: Contextual Re-Activation of Filters for Face Recognition Training". CRAFT identifies inactive filters during training and reinitializes them based on the context of strong filters at that stage in training. We show that CRAFT reduces fraction of inactive filters from 44% to 32% on average and discovers filter patterns not found by standard training. Compared to standard training without reactivation, CRAFT demonstrates enhanced model accuracy on standard face-recognition benchmark datasets including AgeDB-30, CPLFW, LFW, CALFW, and CFP-FP, as well as on more challenging datasets like IJBB and IJBC.