title: Sleep Deprivation in the Forward-Forward Algorithm

abstract: This paper aims to explore the separation of the two forward passes in the Forward-Forward algorithm from a biological perspective in the context of sleep. We show the size of the gap between the sleep and awake phase influences the learning capabilities of the algorithm and highlight the importance of negative data in diminishing the devastating effects of sleep deprivation.


title: Generative Artificial Intelligence for Software Engineering -- A Research Agenda

abstract: Generative Artificial Intelligence (GenAI) tools have become increasingly prevalent in software development, offering assistance to various managerial and technical project activities. Notable examples of these tools include OpenAIs ChatGPT, GitHub Copilot, and Amazon CodeWhisperer. Although many recent publications have explored and evaluated the application of GenAI, a comprehensive understanding of the current development, applications, limitations, and open challenges remains unclear to many. Particularly, we do not have an overall picture of the current state of GenAI technology in practical software engineering usage scenarios. We conducted a literature review and focus groups for a duration of five months to develop a research agenda on GenAI for Software Engineering. We identified 78 open Research Questions (RQs) in 11 areas of Software Engineering. Our results show that it is possible to explore the adoption of GenAI in partial automation and support decision-making in all software development activities. While the current literature is skewed toward software implementation, quality assurance and software maintenance, other areas, such as requirements engineering, software design, and software engineering education, would need further research attention. Common considerations when implementing GenAI include industry-level assessment, dependability and accuracy, data accessibility, transparency, and sustainability aspects associated with the technology. GenAI is bringing significant changes to the field of software engineering. Nevertheless, the state of research on the topic still remains immature. We believe that this research agenda holds significance and practical value for informing both researchers and practitioners about current applications and guiding future research.


title: Weighted $L^p$-norm inequality of multi-parameter fractional integration

abstract: We study a family of strong fractional integral operators whose kernels have singularity on every coordinate subspace. We prove a desired two-weight, L^p-norm inequality provided that the corresponding multi-parameter theta-bump characteristic is finite.


title: Designing a Kinetic Fa\c{c}ade Using BB-BC Algorithm with a Focus on Enhancing Building Energy Efficiency

abstract: In order to increase energy efficiency in buildings, optimizing the parameters of the facade form can be challenging due to the dynamic nature of solar radiation. One effective solution is the use of kinetic facades as a second skin, which can control energy consumption. This study proposes a parametric kinetic facade to increase building energy efficiency, along with a framework to optimize its form using the Bang-Big Crunch (BB-BC) optimization algorithm. The study involved modeling a two-story office building in Shiraz city and calculating the energy consumption resulting from building operation over a three-day period without considering the second skin of the facade. In the second stage of the study, the second skin was optimized for the same three-day interval and calculated as a parametric, static facade. In the last step, the parameters of the second skin were optimized for three one-day intervals, assuming the possibility of kinematic changes each day. The total energy consumption of building operation for the three days was then calculated and analyzed.The Python programming language was used to develop the optimization algorithm, while Rhino software, and Grasshopper, Ladybug, and Honeybee plugins were used for building modeling and simulation of light, energy, and weather parameters.The results of the study demonstrate the effectiveness of the proposed kinetic facade and the proper performance of the proposed algorithm for solving similar problems. The study found that the use of the second skin with kinetic function reduced energy consumption by 28%. Additionally, the results from the second and third stages of the study showed that the use of the second facade shell with kinematic function, compared to its function in static mode, reduced energy consumption by 4%.Overall...


title: Patch-Wise Self-Supervised Visual Representation Learning: A Fine-Grained Approach

abstract: Self-supervised visual representation learning traditionally focuses on image-level instance discrimination. Our study introduces an innovative, fine-grained dimension by integrating patch-level discrimination into these methodologies. This integration allows for the simultaneous analysis of local and global visual features, thereby enriching the quality of the learned representations. Initially, the original images undergo spatial augmentation. Subsequently, we employ a distinctive photometric patch-level augmentation, where each patch is individually augmented, independent from other patches within the same view. This approach generates a diverse training dataset with distinct color variations in each segment. The augmented images are then processed through a self-distillation learning framework, utilizing the Vision Transformer (ViT) as its backbone. The proposed method minimizes the representation distances across both image and patch levels to capture details from macro to micro perspectives. To this end, we present a simple yet effective patch-matching algorithm to find the corresponding patches across the augmented views. Thanks to the efficient structure of the patch-matching algorithm, our method reduces computational complexity compared to similar approaches. Consequently, we achieve an advanced understanding of the model without adding significant computational requirements. We have extensively pretrained our method on datasets of varied scales, such as Cifar10, ImageNet-100, and ImageNet-1K. It demonstrates superior performance over state-of-the-art self-supervised representation learning methods in image classification and downstream tasks, such as copy detection and image retrieval. The implementation of our method is accessible on GitHub.


title: EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Images

abstract: Electronic Health Records (EHRs), which contain patients' medical histories in various multi-modal formats, often overlook the potential for joint reasoning across imaging and table modalities underexplored in current EHR Question Answering (QA) systems. In this paper, we introduce EHRXQA, a novel multi-modal question answering dataset combining structured EHRs and chest X-ray images. To develop our dataset, we first construct two uni-modal resources: 1) The MIMIC-CXR-VQA dataset, our newly created medical visual question answering (VQA) benchmark, specifically designed to augment the imaging modality in EHR QA, and 2) EHRSQL (MIMIC-IV), a refashioned version of a previously established table-based EHR QA dataset. By integrating these two uni-modal resources, we successfully construct a multi-modal EHR QA dataset that necessitates both uni-modal and cross-modal reasoning. To address the unique challenges of multi-modal questions within EHRs, we propose a NeuralSQL-based strategy equipped with an external VQA API. This pioneering endeavor enhances engagement with multi-modal EHR sources and we believe that our dataset can catalyze advances in real-world medical scenarios such as clinical decision-making and research. EHRXQA is available at https://github.com/baeseongsu/ehrxqa.


title: Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensing

abstract: Self-supervised learning guided by masked image modelling, such as Masked AutoEncoder (MAE), has attracted wide attention for pretraining vision transformers in remote sensing. However, MAE tends to excessively focus on pixel details, thereby limiting the model's capacity for semantic understanding, in particular for noisy SAR images. In this paper, we explore spectral and spatial remote sensing image features as improved MAE-reconstruction targets. We first conduct a study on reconstructing various image features, all performing comparably well or better than raw pixels. Based on such observations, we propose Feature Guided Masked Autoencoder (FG-MAE): reconstructing a combination of Histograms of Oriented Graidents (HOG) and Normalized Difference Indices (NDI) for multispectral images, and reconstructing HOG for SAR images. Experimental results on three downstream tasks illustrate the effectiveness of FG-MAE with a particular boost for SAR imagery. Furthermore, we demonstrate the well-inherited scalability of FG-MAE and release a first series of pretrained vision transformers for medium resolution SAR and multispectral images.


title: Causal discovery in a complex industrial system: A time series benchmark

abstract: Causal discovery outputs a causal structure, represented by a graph, from observed data. For time series data, there is a variety of methods, however, it is difficult to evaluate these on real data as realistic use cases very rarely come with a known causal graph to which output can be compared. In this paper, we present a dataset from an industrial subsystem at the European Spallation Source along with its causal graph which has been constructed from expert knowledge. This provides a testbed for causal discovery from time series observations of complex systems, and we believe this can help inform the development of causal discovery methodology.


title: Systematic Improvement of Empirical Energy Functions in the Era of Machine Learning

abstract: The impact of targeted replacement of individual terms in empirical force fields is quantitatively assessed for pure water, dichloromethane (DCM), and solvated K$^+$ and Cl$^-$ ions. For the electrostatics, point charges (PCs) and machine learning (ML)based minimally distributed charges (MDCM) fitted to the molecular electrostatic potential are evaluated together with electrostatics based on the Coulomb integral. The impact of explicitly including second-order terms is investigated by adding a fragment molecular orbital (FMO)-derived polarization energy to an existing force field, in this case CHARMM. It is demonstrated that anisotropic electrostatics reduce the RMSE for water (by 1.6 kcal/mol), DCM (by 0.8 kcal/mol) and for solvated Cl$^-$ clusters (by 0.4 kcal/mol). An additional polarization term can be neglected for DCM but notably improves errors in pure water (by 1.1 kcal/mol) and in Cl$^-$ clusters (by 0.4 kcal/mol) and is key to describing solvated K$^+$, reducing the RMSE by 2.3 kcal/mol. A 12-6 Lennard-Jones functional form is found to perform satisfactorily with PC and MDCM electrostatics, but is not appropriate for descriptions that account for the electrostatic penetration energy. The importance of many-body contributions is assessed by comparing a strictly 2-body approach with self-consistent reference data. DCM can be approximated well with a 2-body potential while water and solvated K$^+$ and Cl$^-$ ions require explicit many-body corrections. The present work systematically quantifies which terms improve the performance of an existing force field and what reference data to use for parametrizing these terms in a tractable fashion for ML fitting of pure and heterogeneous systems.


title: Med-DANet V2: A Flexible Dynamic Architecture for Efficient Medical Volumetric Segmentation

abstract: Recent works have shown that the computational efficiency of 3D medical image (e.g. CT and MRI) segmentation can be impressively improved by dynamic inference based on slice-wise complexity. As a pioneering work, a dynamic architecture network for medical volumetric segmentation (i.e. Med-DANet) has achieved a favorable accuracy and efficiency trade-off by dynamically selecting a suitable 2D candidate model from the pre-defined model bank for different slices. However, the issues of incomplete data analysis, high training costs, and the two-stage pipeline in Med-DANet require further improvement. To this end, this paper further explores a unified formulation of the dynamic inference framework from the perspective of both the data itself and the model structure. For each slice of the input volume, our proposed method dynamically selects an important foreground region for segmentation based on the policy generated by our Decision Network and Crop Position Network. Besides, we propose to insert a stage-wise quantization selector to the employed segmentation model (e.g. U-Net) for dynamic architecture adapting. Extensive experiments on BraTS 2019 and 2020 show that our method achieves comparable or better performance than previous state-of-the-art methods with much less model complexity. Compared with previous methods Med-DANet and TransBTS with dynamic and static architecture respectively, our framework improves the model efficiency by up to nearly 4.1 and 17.3 times with comparable segmentation results on BraTS 2019.


title: Fairness in online vehicle-cargo matching: An intuitionistic fuzzy set theory and tripartite evolutionary game approach

abstract: This paper explores the concept of fairness and equitable matching in an on-line vehicle-cargo matching setting, addressing the varying degrees of satisfaction experienced by shippers and carriers. Relevant indicators for shippers and carriers in the on-line matching process are categorized as attributes, expectations, and reliability, which are subsequent quantified to form satisfaction indicators. Employing the intuitionistic fuzzy set theory, we devise a transformed vehicle-cargo matching optimization model by combining the fuzzy set's membership, non-membership, and uncertainty information. Through an adaptive interactive algorithm, the matching scheme with fairness concerns is solved using CPLEX. The effectiveness of the proposed matching mechanism in securing high levels of satisfaction is established by comparison with three benchmark methods. To further investigate the impact of considering fairness in vehicle-cargo matching, a shipper-carrier-platform tripartite evolutionary game framework is developed under the waiting response time cost (WRTC) sharing mechanism. Simulation results show that with fairness concerns in vehicle-cargo matching, all stakeholders are better off: The platform achieves positive revenue growth, and shippers and carriers receive positive subsidy. This study offers both theoretical insights and practical guidance for the long-term and stable operation of the on-line freight stowage industry.


title: Estimating Systemic Risk within Financial Networks: A Two-Step Nonparametric Method

abstract: CoVaR (conditional value-at-risk) is a crucial measure for assessing financial systemic risk, which is defined as a conditional quantile of a random variable, conditioned on other random variables reaching specific quantiles. It enables the measurement of risk associated with a particular node in financial networks, taking into account the simultaneous influence of risks from multiple correlated nodes. However, estimating CoVaR presents challenges due to the unobservability of the multivariate-quantiles condition. To address the challenges, we propose a two-step nonparametric estimation approach based on Monte-Carlo simulation data. In the first step, we estimate the unobservable multivariate-quantiles using order statistics. In the second step, we employ a kernel method to estimate the conditional quantile conditional on the order statistics. We establish the consistency and asymptotic normality of the two-step estimator, along with a bandwidth selection method. The results demonstrate that, under a mild restriction on the bandwidth, the estimation error arising from the first step can be ignored. Consequently, the asymptotic results depend solely on the estimation error of the second step, as if the multivariate-quantiles in the condition were observable. Numerical experiments demonstrate the favorable performance of the two-step estimator.

categories: q-fin.RM math.ST q-fin.ST stat.AP stat.TH


title: DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy

abstract: Recent advances in large language models (LLMs) have revolutionized the landscape of reasoning tasks. To enhance the capabilities of LLMs to emulate human reasoning, prior studies have focused on modeling reasoning steps using various thought structures like chains, trees, or graphs. However, LLM-based reasoning still encounters the following challenges: (1) Limited adaptability of preset structures to diverse tasks; (2) Insufficient precision in exploiting known conditions to derive new ones; and (3) Inadequate consideration of historical reasoning experiences for subsequent reasoning steps. To this end, we propose DetermLR, a novel perspective that rethinks the reasoning process as an evolution from indeterminacy to determinacy. First, we categorize known conditions into two types: determinate and indeterminate premises This provides an oveall direction for the reasoning process and guides LLMs in converting indeterminate data into progressively determinate insights. Subsequently, we leverage quantitative measurements to prioritize more relevant premises to explore new insights. Furthermore, we automate the storage and extraction of available premises and reasoning paths with reasoning memory, preserving historical reasoning details for subsequent reasoning steps. Comprehensive experimental results demonstrate that DetermLR surpasses all baselines on various logical reasoning benchmarks: LogiQA, ProofWriter, FOLIO, PrOntoQA, and LogicalDeduction. Compared to previous multi-step reasoning methods, DetermLR achieves higher accuracy with fewer reasoning steps, highlighting its superior efficiency and effectiveness in solving logical reasoning tasks.


title: Foundation Models for Generalist Geospatial Artificial Intelligence

abstract: Significant progress in the development of highly adaptable and reusable Artificial Intelligence (AI) models is expected to have a significant impact on Earth science and remote sensing. Foundation models are pre-trained on large unlabeled datasets through self-supervision, and then fine-tuned for various downstream tasks with small labeled datasets. This paper introduces a first-of-a-kind framework for the efficient pre-training and fine-tuning of foundational models on extensive geospatial data. We have utilized this framework to create Prithvi, a transformer-based geospatial foundational model pre-trained on more than 1TB of multispectral satellite imagery from the Harmonized Landsat-Sentinel 2 (HLS) dataset. Our study demonstrates the efficacy of our framework in successfully fine-tuning Prithvi to a range of Earth observation tasks that have not been tackled by previous work on foundation models involving multi-temporal cloud gap imputation, flood mapping, wildfire scar segmentation, and multi-temporal crop segmentation. Our experiments show that the pre-trained model accelerates the fine-tuning process compared to leveraging randomly initialized weights. In addition, pre-trained Prithvi compares well against the state-of-the-art, e.g., outperforming a conditional GAN model in multi-temporal cloud imputation by up to 5pp (or 5.7%) in the structural similarity index. Finally, due to the limited availability of labeled data in the field of Earth observation, we gradually reduce the quantity of available labeled data for refining the model to evaluate data efficiency and demonstrate that data can be decreased significantly without affecting the model's accuracy. The pre-trained 100 million parameter model and corresponding fine-tuning workflows have been released publicly as open source contributions to the global Earth sciences community through Hugging Face.


title: Mode-locked GaN waveguide polariton laser

abstract: We introduce a novel mode-locking scheme based on polariton-polariton interactions and we demonstrate its operation in a $60 \ \mu m-$long GaN-based waveguide cavity. The exciton-polariton active layer acts both as a gain medium and as a self-focusing medium with a large Kerr-like nonlinearity. The monitoring of the cavity free spectral range across the lasing threshold evidences the linearization of the strongly nonlinear mode dispersion due to self-phase modulation. The Gross-Pitaevskii equation for the coupled exciton and photon fields taking into account the exact cavity geometry, predicts the appearance of optical pulses within the cavity that correspond to optical solitons. Pulses widths are about 100 fs, both in theory and experiment.


title: ASTormer: An AST Structure-aware Transformer Decoder for Text-to-SQL

abstract: Text-to-SQL aims to generate an executable SQL program given the user utterance and the corresponding database schema. To ensure the well-formedness of output SQLs, one prominent approach adopts a grammar-based recurrent decoder to produce the equivalent SQL abstract syntax tree (AST). However, previous methods mainly utilize an RNN-series decoder, which 1) is time-consuming and inefficient and 2) introduces very few structure priors. In this work, we propose an AST structure-aware Transformer decoder (ASTormer) to replace traditional RNN cells. The structural knowledge, such as node types and positions in the tree, is seamlessly incorporated into the decoder via both absolute and relative position embeddings. Besides, the proposed framework is compatible with different traversing orders even considering adaptive node selection. Extensive experiments on five text-to-SQL benchmarks demonstrate the effectiveness and efficiency of our structured decoder compared to competitive baselines.


title: Smooth linear eigenvalue statistics on random covers of compact hyperbolic surfaces -- A central limit theorem and almost sure RMT statistics

abstract: We study smooth linear spectral statistics of twisted Laplacians on random $n$-covers of a fixed compact hyperbolic surface $X$. We consider two aspects of such statistics. The first is the fluctuations of such statistics in a small energy window around a fixed energy level when averaged over the space of all degree $n$ covers of $X$. The second is the energy variance of a typical surface. In the first case, we show a central limit theorem. Specifically, we show that the distribution of such fluctuations tends to a Gaussian with variance given by the corresponding quantity for the Gaussian Orthogonal/Unitary Ensemble (GOE/GUE). In the second case, we show that the energy variance of a typical random $n$-cover is that of the GOE/GUE. In both cases, we consider a double limit where first we let $n$, the covering degree, go to $\infty$ then let $L\to \infty$ where $1/L$ is the window length.

categories: math.SP math-ph math.DS math.GT math.MP math.NT math.PR


title: Node Cardinality Estimation in the Internet of Things Using Privileged Feature Distillation

abstract: The Internet of Things (IoT) is emerging as a critical technology to connect resource-constrained devices such as sensors and actuators as well as appliances to the Internet. In this paper, we propose a novel methodology for node cardinality estimation in wireless networks such as the IoT and Radio-Frequency IDentification (RFID) systems, which uses the privileged feature distillation (PFD) technique and works using a neural network with a teacher-student model. The teacher is trained using both privileged and regular features, and the student is trained with predictions from the teacher and regular features. We propose node cardinality estimation algorithms based on the PFD technique for homogeneous as well as heterogeneous wireless networks. We show via extensive simulations that the proposed PFD based algorithms for homogeneous as well as heterogeneous networks achieve much lower mean squared errors in the computed node cardinality estimates than state-of-the-art protocols proposed in prior work, while taking the same number of time slots for executing the node cardinality estimation process as the latter protocols.


title: A power law solution for FRLW Universe with observational constraints

abstract: This paper examines a power law solution under $f(R,T)$ gravity for an isotropic and homogeneous universe by considering its functional form as $f(R,T) = R + \xi RT$, where $\xi$ is a positive constant. In $f(R,T)$ gravity, we have built the field equation for homogeneous and isotropic spacetime. The developed model's solution is $a = \alpha t^{\beta}$. We have used the redshift in the range $0 \leq z \leq 1.965$ and obtained the model parameters $\alpha$, $\beta$, $H_0$ by using the Markov Chain Monte Carlo (MCMC) method. The constrained values of the model parameter are as follows: $H_0 = 67.098^{+2.148}_{-1.792}$ km s$^{-1}$ Mpc$^{-1}$, $H_0 = 67.588^{+2.229}_{-2.170}$ km s$^{-1}$ Mpc$^{-1}$, $H_0 = 66.270^{+2.215}_{-2.181}$ km s$^{-1}$ Mpc$^{-1}$, $H_0 = 65.960^{+2.380}_{-1.834}$ km s$^{-1}$ Mpc$^{-1}$, $H_0 = 66.274^{+2.015}_{-1.864}$ km s$^{-1}$ Mpc$^{-1}$ which have been achieved by bounding the model with the Hubble parameter ($H(z)$) dataset, Baryon Acoustic Oscillations (BAO) dataset, Pantheon dataset, joint $H(z)$ + Pantheon dataset and collective $H(z)$ + BAO + Pantheon dataset, respectively. These computed $H_o$ observational values agree well with the outcomes from the Plank collaboration group. Through an analysis of the energy conditions' behaviour on our obtained solution, the model has been examined and analysed. Using the Om diagnostic as the state finder diagnostic tool and the jerk parameter, we have also investigated the model's validity. Our results show that, within a certain range of restrictions, the proposed model agrees with the observed signatures.


title: An efficient stochastic particle method for high-dimensional nonlinear PDEs

abstract: Numerical resolution of high-dimensional nonlinear PDEs remains a huge challenge due to the curse of dimensionality. Starting from the weak formulation of the Lawson-Euler scheme, this paper proposes a stochastic particle method (SPM) by tracking the deterministic motion, random jump, resampling and reweighting of particles. Real-valued weighted particles are adopted by SPM to approximate the high-dimensional solution, which automatically adjusts the point distribution to intimate the relevant feature of the solution. A piecewise constant reconstruction with virtual uniform grid is employed to evaluate the nonlinear terms, which fully exploits the intrinsic adaptive characteristic of SPM. Combining both can SPM achieve the goal of adaptive sampling in time. Numerical experiments on the 6-D Allen-Cahn equation and the 7-D Hamiltonian-Jacobi-Bellman equation demonstrate the potential of SPM in solving high-dimensional nonlinear PDEs efficiently while maintaining an acceptable accuracy.


title: Defect-influenced particle advection in highly confined liquid crystal flows

abstract: We study the morphology of the Saturn ring defect and director structure around a colloidal particle with normal anchoring conditions and within the flow of the nematic host phase through a rectangular duct of comparable size to the particle. The changes in the defect structures and director profile influence the advection behaviour of the particle, which we compare to that in a simple Newtonian host phase. These effects lead to a non-monotonous dependence of the differential velocity of particle and fluid, also known as retardation ratio, on the Ericksen number.


title: FinBTech: Blockchain-Based Video and Voice Authentication System for Enhanced Security in Financial Transactions Utilizing FaceNet512 and Gaussian Mixture Models

abstract: In the digital age, it is crucial to make sure that financial transactions are as secure and reliable as possible. This abstract offers a ground-breaking method that combines smart contracts, blockchain technology, FaceNet512 for improved face recognition, and Gaussian Mixture Models (GMM) for speech authentication to create a system for video and audio verification that is unmatched. Smart contracts and the immutable ledger of the blockchain are combined to offer a safe and open environment for financial transactions. FaceNet512 and GMM offer multi-factor biometric authentication simultaneously, enhancing security to new heights. By combining cutting-edge technology, this system offers a strong defense against identity theft and illegal access, establishing a new benchmark for safe financial transactions.


title: Probing the non-thermal physics of stellar bow shocks using radio observations

abstract: We aim to interpret the radio emission of stellar bow shocks and assess under which conditions it could be either thermal or non-thermal, and how to use the observational data to infer their physical properties. We used an extended non-thermal emission model for stellar bow shocks for which we incorporated a consistent calculation of the thermal emission from the forward shock. We fitted this model to the available radio data, including largely unexplored data at low frequencies. In addition, we used a simplified one-zone model to estimate the gamma-ray emission from particles escaping the bow shocks. We can only explain the radio data from (BD+43{\deg}3654 and BD+60{\deg}2522) assuming a hard electron energy distribution below $\sim$1 GeV, a high efficiency of conversion of (shocked) wind kinetic power into relativistic electrons, and a relatively high magnetic-to-thermal pressure ratio. In the other systems, the interpretation of the observed flux density is ambiguous, although a non-thermal scenario is also favoured. We also show how complementary observations at other frequencies can allow us to place stronger constraints in the model. We also estimated the gamma-ray fluxes from the HII regions around the bow shocks of BD+43{\deg}3654 and BD+60{\deg}2522, and obtained luminosities at GeV energies of $\sim 10^{33}$~erg\,s$^{-1}$ and $10^{32}$~erg\,s$^{-1}$, respectively. Stellar bow shocks can potentially be very efficient particle accelerators. This work provides multi-wavelength predictions of their emission and demonstrates the key role of low-frequency radio observations in unveiling particle acceleration processes. The prospects of detections with next-generation observatories such as SKA and ngVLA are promising. Finally, BD+43{\deg}3654 may be detected in GeV in the near future, while bow shocks in general may turn out to be non-negligible sources of low-energy cosmic rays.


title: Two-stage space construction for real-time modeling of distributed parameter systems under sparse sensing

abstract: Numerous industrial processes can be defined using distributed parameter systems (DPSs). This study introduces a two-stage spatial construction approach for real-time modeling of DPSs in cases of limited sensors. Initially, a discrete space-completion approach is created to recuperate the spatiotemporal patterns of non-monitored locations under sparse sensing. The high-dimensional space construction method is employed to derive continuous spatial basis functions (SBFs). The identification and adjustment of the nonlinear temporal model are carried out via the long short-term memory (LSTM) neural network. Eventually, the amalgamation of the derived SBFs and temporal model results in a spatially continuous model. The use of a cubic B-spline surface is validated as an effective solution for optimizing space construction in the sense of least squares approximation. Experimental tests conducted on a pouch-type Li-ion battery demonstrate the efficacy of the proposed modeling technique under sparse sensing. This work highlights the promise of sparse sensors in real-time full-space modeling for large-scale battery energy storage systems.


title: NNLO Matrix-Element Corrections in VINCIA

abstract: We report on a new formalism for parton showers whose fixed-order expansion can be corrected through next-to-next-to-leading order (NNLO) in QCD. It is the first such formalism we are aware of that has no dependence on any auxiliary scales or external resummations and which is fully differential in all of the relevant phase spaces. Since the shower acts as the phase-space generator, the dominant singularity structures are encoded by construction and the method can generate unweighted events with very high efficiency without any significant initialisation time. We argue that the the method should be capable of achieving (at least) NNLO+NNDL accuracy for the shower evolution variable and use hadronic Z decays as a specific example.


title: Maximum Independent Set: Self-Training through Dynamic Programming

abstract: This work presents a graph neural network (GNN) framework for solving the maximum independent set (MIS) problem, inspired by dynamic programming (DP). Specifically, given a graph, we propose a DP-like recursive algorithm based on GNNs that firstly constructs two smaller sub-graphs, predicts the one with the larger MIS, and then uses it in the next recursive call. To train our algorithm, we require annotated comparisons of different graphs concerning their MIS size. Annotating the comparisons with the output of our algorithm leads to a self-training process that results in more accurate self-annotation of the comparisons and vice versa. We provide numerical evidence showing the superiority of our method vs prior methods in multiple synthetic and real-world datasets.


title: Internalizations of decorated bicategories via $\pi_2$-indexings

abstract: We treat the problem of lifting bicategories into double categories through categories of vertical morphisms. We consider structures on decorated 2-categories allowing us to formally implement arguments of sliding certain squares along vertical subdivisions in double categories. We call these structures $\pi_2$-indexings. We present a construction associating, to every $\pi_2$-indexing on a decorated 2-category, a length 1 double internalization.


title: Resolved Photons in Sherpa

abstract: We present the first complete simulation framework, in the Sherpa event generator, for resolved photon interactions at next-to leading order accuracy. It includes photon spectra obtained through the equivalent-photon approximation, parton distribution functions to parametrize the hadronic structure of quasi-real photons, the matching of the parton shower to next-to leading order QCD calculations for resolved photon cross sections, and the modelling of multiple-parton interactions. We validate our framework against a wide range of photo-production data from LEP and HERA experiments, observing good overall agreement. We identify important future steps relevant for high-quality simulations at the planned Electron-Ion Collider.


title: Interplay between Chiral Charge Density Wave and Superconductivity in Kagome Superconductors: A Self-consistent Theoretical Analysis

abstract: Inspired by the recent discovery of a successive evolutions of electronically ordered states, we present a self-consistent theoretical analysis that treats the interactions responsible for the chiral charge order and superconductivity on an equal footing. It is revealed that the self-consistent theory captures the essential features of the successive temperature evolutions of the electronic states from the high-temperature ``triple-$Q$" $2\times 2$ charge-density-wave state to the nematic charge-density-wave phase, and finally to the low-temperature superconducting state coexisting with the nematic charge density wave. We provide a comprehensive explanation for the temperature evolutions of the charge ordered states and discuss the consequences of the intertwining of the superconductivity with the nematic charge density wave. Our findings not only account for the successive temperature evolutions of the ordered electronic states discovered in experiments but also provide a natural explanation for the two-fold rotational symmetry observed in both the charge-density-wave and superconducting states. Moreover, the intertwining of the superconductivity with the nematic charge density wave order may also be an advisable candidate to reconcile the divergent or seemingly contradictory experimental outcomes regarding the superconducting properties.


title: Efficient Object Detection in Optical Remote Sensing Imagery via Attention-based Feature Distillation

abstract: Efficient object detection methods have recently received great attention in remote sensing. Although deep convolutional networks often have excellent detection accuracy, their deployment on resource-limited edge devices is difficult. Knowledge distillation (KD) is a strategy for addressing this issue since it makes models lightweight while maintaining accuracy. However, existing KD methods for object detection have encountered two constraints. First, they discard potentially important background information and only distill nearby foreground regions. Second, they only rely on the global context, which limits the student detector's ability to acquire local information from the teacher detector. To address the aforementioned challenges, we propose Attention-based Feature Distillation (AFD), a new KD approach that distills both local and global information from the teacher detector. To enhance local distillation, we introduce a multi-instance attention mechanism that effectively distinguishes between background and foreground elements. This approach prompts the student detector to focus on the pertinent channels and pixels, as identified by the teacher detector. Local distillation lacks global information, thus attention global distillation is proposed to reconstruct the relationship between various pixels and pass it from teacher to student detector. The performance of AFD is evaluated on two public aerial image benchmarks, and the evaluation results demonstrate that AFD in object detection can attain the performance of other state-of-the-art models while being efficient.


title: Energy-Based Models for Anomaly Detection: A Manifold Diffusion Recovery Approach

abstract: We present a new method of training energy-based models (EBMs) for anomaly detection that leverages low-dimensional structures within data. The proposed algorithm, Manifold Projection-Diffusion Recovery (MPDR), first perturbs a data point along a low-dimensional manifold that approximates the training dataset. Then, EBM is trained to maximize the probability of recovering the original data. The training involves the generation of negative samples via MCMC, as in conventional EBM training, but from a different distribution concentrated near the manifold. The resulting near-manifold negative samples are highly informative, reflecting relevant modes of variation in data. An energy function of MPDR effectively learns accurate boundaries of the training data distribution and excels at detecting out-of-distribution samples. Experimental results show that MPDR exhibits strong performance across various anomaly detection tasks involving diverse data types, such as images, vectors, and acoustic signals.


title: Diffusion processes as Wasserstein gradient flows via stochastic control of the volatility matrix

abstract: We consider a class of time-homogeneous diffusion processes on $\mathbb{R}^{n}$ with common invariant measure but varying volatility matrices. In Euclidean space, we show via stochastic control of the diffusion coefficient that the corresponding flow of time-marginal distributions admits an entropic gradient flow formulation in the quadratic Wasserstein space if the volatility matrix of the diffusion is the identity. After equipping $\mathbb{R}^{n}$ with a Riemannian metric, we prove that the diffusion process can be viewed as a gradient flow in the inherited Wasserstein space if the volatility matrix is the inverse of the underlying metric tensor. In the Euclidean case, our probabilistic result corresponds to the gradient flow characterization of the Fokker$-$Planck equation, first discovered in a seminal paper by Jordan, Kinderlehrer, and Otto. In the Riemannian setting, the corresponding result on the level of partial differential equations was established by Lisini, building on the metric theory developed by Ambrosio, Gigli, and Savar\'{e}.


title: N-Critics: Self-Refinement of Large Language Models with Ensemble of Critics

abstract: We propose a self-correction mechanism for Large Language Models (LLMs) to mitigate issues such as toxicity and fact hallucination. This method involves refining model outputs through an ensemble of critics and the model's own feedback. Drawing inspiration from human behavior, we explore whether LLMs can emulate the self-correction process observed in humans who often engage in self-reflection and seek input from others to refine their understanding of complex topics. Our approach is model-agnostic and can be applied across various domains to enhance trustworthiness by addressing fairness, bias, and robustness concerns. We consistently observe performance improvements in LLMs for reducing toxicity and correcting factual errors.


title: Towards High-Energy Few-Cycle Optical Vortices with Minimized Topological Charge Dispersion

abstract: A simple approach to generate high-energy few-cycle optical vortices with minimized topological charge dispersion is introduced. By means of numerical simulations it is shown that, by leveraging the intrinsic properties of optical parametric chirped pulse amplification (OPCPA), clean transfer of topological charge from a high energy narrowband pump pulse to a broadband idler is feasible under certain particular conditions, enabling the generation of high-energy few-cycle vortex pulses with extremely low topological charge dispersion.


title: DySurv: Dynamic Deep Learning Model for Survival Prediction in the ICU

abstract: Survival analysis focuses on estimating time-to-event distributions which can help in dynamic risk prediction in healthcare. Extending beyond the classical Cox model, deep learning techniques have been developed which moved away from the constraining assumptions of proportional hazards. Traditional statistical models often only include static information where, in this work, we propose a novel conditional variational autoencoder-based method called DySurv, which uses a combination of static and time-series measurements from patient electronic health records to estimate the risk of death dynamically. DySurv has been tested on several time-to-event benchmarks where it outperforms existing methods, including deep learning methods, and we evaluate it on real-world intensive care unit data from MIMIC-IV and eICU. The predictive capacity of DySurv is consistent and the survival estimates remain disentangled across different datasets supporting the idea that dynamic deep learning models based on conditional variational inference in multi-task cases can be robust models for survival analysis.


title: Lusztig sheaves and tensor products of integrable highest weight modules

abstract: By introducing $N$-framed quivers, we define the localization of Lusztig's sheaves for $N$-framed quivers and functors $E^{(n)}_{i}, F^{(n)}_{i}, K^{\pm}_i$ for localizations. This gives a categorical realization of tensor products of integrable highest weight modules of the quantized enveloping algebra. The simple perverse sheaves in the localization provide the canonical basis of tensor products. It is bar-invariant and has positivity. Moreover, we give a categorical interpretation of the Yang-Baxter equation.


title: Single pulse polarization study of pulsars B0950+08 and B1642-03: micropulse properties and mixing of orthogonal modes

abstract: We present the results of a high-time resolution polarization study of single pulses from pulsars B0950+08 and B1642-03. Single pulses from pulsar B0950+08 sometimes show isolated micropulses without any significant associated subpulse emission. Assuming that the properties of such micropulses represent the intrinsic nature of micropulse emission, we characterize the width and polarization properties of these `intrinsic' microstructures. Most of the `intrinsic' micropulses (~90%) follow common characteristic polarization properties, while the average width of these micropulses is consistent with the general micropulse population from this pulsar. Single pulses from these pulsars show a diverse range of polarization properties, including depolarization and mixing of two orthogonal modes resulting in polarization position angle jumps. We present a superposition model of the two orthogonal modes which can explain depolarization, the observed position angle jumps, and associated changes in other polarization parameters.


title: Thermalization and hydrodynamics in an interacting integrable system: the case of hard rods

abstract: We consider the relaxation of an initial non-equilibrium state in a one-dimensional fluid of hard rods. Since it is an interacting integrable system, we expect it to reach the Generalized Gibbs Ensemble (GGE) at long times for generic initial conditions. Here we show that there exist initial conditions for which the system does not reach GGE even at very long times and in the thermodynamic limit. In particular, we consider an initial condition of uniformly distributed hard-rods in a box with the left half having particles with a singular velocity distribution (all moving with unit velocity) and the right half particles in thermal equilibrium. We find that the density profile for the singular component does not spread to the full extent of the box and keeps moving with a fixed effective speed at long times. We show that such density profiles can be well described by the solution of the Euler equations almost everywhere except at the location of the shocks, where we observe slight discrepancies due to dissipation arising from the initial fluctuations of the thermal background. To demonstrate this effect of dissipation analytically, we consider a second initial condition with a single particle at the origin with unit velocity in a thermal background. We find that the probability distribution of the position of the unit velocity quasi-particle has diffusive spreading which can be understood from the solution of the Navier-Stokes equation of the hard rods. Finally, we consider an initial condition with a spread in velocity distribution for which we show convergence to GGE. Our conclusions are based on molecular dynamics simulations supported by analytical arguments.


title: When Reviewers Lock Horn: Finding Disagreement in Scientific Peer Reviews

abstract: To this date, the efficacy of the scientific publishing enterprise fundamentally rests on the strength of the peer review process. The journal editor or the conference chair primarily relies on the expert reviewers' assessment, identify points of agreement and disagreement and try to reach a consensus to make a fair and informed decision on whether to accept or reject a paper. However, with the escalating number of submissions requiring review, especially in top-tier Artificial Intelligence (AI) conferences, the editor/chair, among many other works, invests a significant, sometimes stressful effort to mitigate reviewer disagreements. Here in this work, we introduce a novel task of automatically identifying contradictions among reviewers on a given article. To this end, we introduce ContraSciView, a comprehensive review-pair contradiction dataset on around 8.5k papers (with around 28k review pairs containing nearly 50k review pair comments) from the open review-based ICLR and NeurIPS conferences. We further propose a baseline model that detects contradictory statements from the review pairs. To the best of our knowledge, we make the first attempt to identify disagreements among peer reviewers automatically. We make our dataset and code public for further investigations.


title: Ringdown gravitational waves from close scattering of two black holes

abstract: We have numerically investigated close scattering processes of two black holes (BHs). Our careful analysis shows for the first time a non-merging ringdown gravitational wave coming from dynamical tidal deformations of individual BHs during their close encounter. The ringdown wave frequencies turn out to agree well with the quasi-normal ones of a single BH in perturbation theory, despite its distinctive physical context from the merging case. Our study shows a new type of gravitational waveform and opens up a new exploration of strong gravitational interactions using BH encounters.


title: Unsupervised Behavior Extraction via Random Intent Priors

abstract: Reward-free data is abundant and contains rich prior knowledge of human behaviors, but it is not well exploited by offline reinforcement learning (RL) algorithms. In this paper, we propose UBER, an unsupervised approach to extract useful behaviors from offline reward-free datasets via diversified rewards. UBER assigns different pseudo-rewards sampled from a given prior distribution to different agents to extract a diverse set of behaviors, and reuse them as candidate policies to facilitate the learning of new tasks. Perhaps surprisingly, we show that rewards generated from random neural networks are sufficient to extract diverse and useful behaviors, some even close to expert ones. We provide both empirical and theoretical evidence to justify the use of random priors for the reward function. Experiments on multiple benchmarks showcase UBER's ability to learn effective and diverse behavior sets that enhance sample efficiency for online RL, outperforming existing baselines. By reducing reliance on human supervision, UBER broadens the applicability of RL to real-world scenarios with abundant reward-free data.


title: Clairvoyance: A Pipeline Toolkit for Medical Time Series

abstract: Time-series learning is the bread and butter of data-driven *clinical decision support*, and the recent explosion in ML research has demonstrated great potential in various healthcare settings. At the same time, medical time-series problems in the wild are challenging due to their highly *composite* nature: They entail design choices and interactions among components that preprocess data, impute missing values, select features, issue predictions, estimate uncertainty, and interpret models. Despite exponential growth in electronic patient data, there is a remarkable gap between the potential and realized utilization of ML for clinical research and decision support. In particular, orchestrating a real-world project lifecycle poses challenges in engineering (i.e. hard to build), evaluation (i.e. hard to assess), and efficiency (i.e. hard to optimize). Designed to address these issues simultaneously, Clairvoyance proposes a unified, end-to-end, autoML-friendly pipeline that serves as a (i) software toolkit, (ii) empirical standard, and (iii) interface for optimization. Our ultimate goal lies in facilitating transparent and reproducible experimentation with complex inference workflows, providing integrated pathways for (1) personalized prediction, (2) treatment-effect estimation, and (3) information acquisition. Through illustrative examples on real-world data in outpatient, general wards, and intensive-care settings, we illustrate the applicability of the pipeline paradigm on core tasks in the healthcare journey. To the best of our knowledge, Clairvoyance is the first to demonstrate viability of a comprehensive and automatable pipeline for clinical time-series ML.


title: Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision

abstract: Foundation models, large-scale, pre-trained deep-learning models adapted to a wide range of downstream tasks have gained significant interest lately in various deep-learning problems undergoing a paradigm shift with the rise of these models. Trained on large-scale dataset to bridge the gap between different modalities, foundation models facilitate contextual reasoning, generalization, and prompt capabilities at test time. The predictions of these models can be adjusted for new tasks by augmenting the model input with task-specific hints called prompts without requiring extensive labeled data and retraining. Capitalizing on the advances in computer vision, medical imaging has also marked a growing interest in these models. To assist researchers in navigating this direction, this survey intends to provide a comprehensive overview of foundation models in the domain of medical imaging. Specifically, we initiate our exploration by providing an exposition of the fundamental concepts forming the basis of foundation models. Subsequently, we offer a methodical taxonomy of foundation models within the medical domain, proposing a classification system primarily structured around training strategies, while also incorporating additional facets such as application domains, imaging modalities, specific organs of interest, and the algorithms integral to these models. Furthermore, we emphasize the practical use case of some selected approaches and then discuss the opportunities, applications, and future directions of these large-scale pre-trained models, for analyzing medical images. In the same vein, we address the prevailing challenges and research pathways associated with foundational models in medical imaging. These encompass the areas of interpretability, data management, computational requirements, and the nuanced issue of contextual comprehension.


title: Modeling of an efficient singlet-triplet spin qubit to photon interface assisted by a photonic crystal cavity

abstract: Efficient interconnection between distant semiconductor spin qubits with the help of photonic qubits would offer exciting new prospects for future quantum communication applications. In this paper, we optimize the extraction efficiency of a novel interface between a singlet-triplet spin qubit and a photonic qubit. The interface is based on a 220 nm thick GaAs/AlGaAs heterostructure membrane and consists of a gate-defined double quantum dot (GDQD) supporting a singlet-triplet qubit, an optically active quantum dot (OAQD) consisting of a gate-defined exciton trap, a photonic crystal cavity providing in-plane optical confinement and efficient out-coupling to an ideal free space Gaussian beam while accommodating the gate wiring of the GDQD and OAQD, and a bottom gold reflector to recycle photons and increase the optical extraction efficiency. All essential components can be lithographically defined and deterministically fabricated on the GaAs/AlGaAs heterostructure membrane, which greatly increases the scalability of on-chip integration. According to our simulations, the interface provides an overall coupling efficiency of 28.7% into a free space Gaussian beam, assuming an SiO2 interlayer filling the space between the reflector and the membrane. The performance can be further increased by undercutting this SiO2 interlayer below the photonic crystal. In this case, the overall efficiency is calculated to be 48.5%.


title: Terahertz Logic Gate Operations in a Toroidal Metasurface

abstract: The applications of terahertz metamaterials are being actively explored in recent times for applications in high-speed communication devices, miniature photonic circuits, and bio-chemical devices because of their wide advantages. The toroidal resonance, a new type of metasurface resonance, has been examined with great interest to utilize its properties in terahertz metasurface applications. The study reports a proof of concept design of a toroidal metasurface that experimentally demonstrates logic gate operations in the terahertz frequency regime by passive tuning of the split ring resonators compromising the meta-atom design. The amplitude modulation is utilized as a method of determining the Boolean logic output of the system. The proposed metasurface could be further optimized for high amplitude modulations, as well as for active logic gate operations using tunable materials including graphene and ITO. The proposed metasurface consists of three split-ring resonators, and the near-field coupling between the adjacent resonators dictates the logic gate operations. The toroidal excitation in the metasurface is determined by a multipole analysis of the scattered powers of terahertz radiation. The proposed metasurfaces experimentally define AND Boolean logic gate at 0.89 terahertz, and OR Boolean logic gate at 0.97 terahertz. Numerical simulations support the experimentally obtained results. Additionally, we numerically report the excitation of NAND gate at 0.87 THz. Such toroidal logic-gate metasurfaces in the terahertz region could find applications in digitized terahertz circuits for photonic device applications.


title: Efficient augmented block designs for unreplicated test treatments

abstract: Augmented block designs for unreplicated test treatments are investigated under the A- and MV-criteria with respect to control versus control, test versus test and control versus test comparisons. We derive design-independent lower bounds on these criteria over a wide class of competing designs. These bounds are useful benchmarks and the resulting expressions for efficiencies enable objective assessment of any given design under the A- and MV-criteria. It is seen that the use of BIB or PBIB designs or duals thereof often leads to very high efficiencies, which compare extremely well with algorithmic constructions. Our findings also cover the case of partial replication of some test treatments. Illustrative examples, including large-scale ones, are presented.


title: Implications of the helicity conservation for on-shell and off-shell chirality production

abstract: We analyze the helicity conservation following from the axial Ward identity for the massless fermion. We discuss the pair production of a fermion, $\ell$, and an anti-fermion, $\bar{\ell}$, from the annihilation process of photons, $\gamma$'s and conclude that the chirality production is prohibited in any annihilation process of on-shell $\gamma$'s. We demonstrate this selection rule in the tree-level process of $\gamma+\gamma\to \ell+\bar{\ell}$. We next consider the off-shell effect on the chirality production in the presence of the background magnetic field. We point out that the particle conversion process under the magnetic field induces the polarization of an incident $\gamma$, which can also be interpreted intuitively from the helicity conservation.


title: Comparison of Methods to Determine the Fluence of Monoenergetic Neutrons in the Energy Range from 30 keV to 14.8 MeV

abstract: The primary reference instruments for neutron fluence measurements used at the Physikalisch-Technische Bundesanstalt (PTB) are based on the primary standard for neutron measurements which is the differential neutron-proton scattering cross section. Such instruments require considerable effort for their operation and analysis. Therefore, routine measurements are carried out using a transfer instrument to facilitate the efficient provision of services to customers. A series of measurements was conducted to compare the transfer device to the primary reference instruments and ensure the traceability of neutron fluence measurements. This resulted in an improved characterization of the instrument and new analysis procedures.


title: The homology of moduli spaces of 4-manifolds may be infinitely generated

abstract: For a simply-connected closed manifold $X$ of $\dim X \neq 4$, the mapping class group $\pi_0(\mathrm{Diff}(X))$ is known to be finitely generated. We prove that analogous finite generation fails in dimension 4. Namely, we show that there exist simply-connected closed smooth 4-manifolds whose mapping class groups are not finitely generated. More generally, for each $k>0$, we prove that there are simply-connected closed smooth 4-manifolds $X$ for which $H_k(B\mathrm{Diff}(X);\mathbb{Z})$ are not finitely generated. The infinitely generated subgroup of $H_k(B\mathrm{Diff}(X);\mathbb{Z})$ which we detect are topologically trivial, and unstable under the connected sum of $S^2 \times S^2$. The proof uses characteristic classes obtained from Seiberg-Witten theory.


title: Probing LLMs for Joint Encoding of Linguistic Categories

abstract: Large Language Models (LLMs) exhibit impressive performance on a range of NLP tasks, due to the general-purpose linguistic knowledge acquired during pretraining. Existing model interpretability research (Tenney et al., 2019) suggests that a linguistic hierarchy emerges in the LLM layers, with lower layers better suited to solving syntactic tasks and higher layers employed for semantic processing. Yet, little is known about how encodings of different linguistic phenomena interact within the models and to what extent processing of linguistically-related categories relies on the same, shared model representations. In this paper, we propose a framework for testing the joint encoding of linguistic categories in LLMs. Focusing on syntax, we find evidence of joint encoding both at the same (related part-of-speech (POS) classes) and different (POS classes and related syntactic dependency relations) levels of linguistic hierarchy. Our cross-lingual experiments show that the same patterns hold across languages in multilingual LLMs.


title: KernelGPA: A Globally Optimal Solution to Deformable SLAM in Closed-form

abstract: We study the generalized Procrustes analysis (GPA), as a minimal formulation to the simultaneous localization and mapping (SLAM) problem. We propose KernelGPA, a novel global registration technique to solve SLAM in the deformable environment. We propose the concept of deformable transformation which encodes the entangled pose and deformation. We define deformable transformations using a kernel method, and show that both the deformable transformations and the environment map can be solved globally in closed-form, up to global scale ambiguities. We solve the scale ambiguities by an optimization formulation that maximizes rigidity. We demonstrate KernelGPA using the Gaussian kernel, and validate the superiority of KernelGPA with various datasets. Code and data are available at \url{https://bitbucket.org/FangBai/deformableprocrustes}.


title: Triplet Attention Transformer for Spatiotemporal Predictive Learning

abstract: Spatiotemporal predictive learning offers a self-supervised learning paradigm that enables models to learn both spatial and temporal patterns by predicting future sequences based on historical sequences. Mainstream methods are dominated by recurrent units, yet they are limited by their lack of parallelization and often underperform in real-world scenarios. To improve prediction quality while maintaining computational efficiency, we propose an innovative triplet attention transformer designed to capture both inter-frame dynamics and intra-frame static features. Specifically, the model incorporates the Triplet Attention Module (TAM), which replaces traditional recurrent units by exploring self-attention mechanisms in temporal, spatial, and channel dimensions. In this configuration: (i) temporal tokens contain abstract representations of inter-frame, facilitating the capture of inherent temporal dependencies; (ii) spatial and channel attention combine to refine the intra-frame representation by performing fine-grained interactions across spatial and channel dimensions. Alternating temporal, spatial, and channel-level attention allows our approach to learn more complex short- and long-range spatiotemporal dependencies. Extensive experiments demonstrate performance surpassing existing recurrent-based and recurrent-free methods, achieving state-of-the-art under multi-scenario examination including moving object trajectory prediction, traffic flow prediction, driving scene prediction, and human motion capture.


title: Some new concentration inequalities for the It\^o stochastic integral

abstract: In this paper, based on the techniques of Malliavin calculus, we provide some new concentration inequalities for the running supremum of the It\^o stochastic integral with unbounded integrands. Several applications and examples are provided as well.


title: Empowering Collaborative Filtering with Principled Adversarial Contrastive Loss

abstract: Contrastive Learning (CL) has achieved impressive performance in self-supervised learning tasks, showing superior generalization ability. Inspired by the success, adopting CL into collaborative filtering (CF) is prevailing in semi-supervised top-K recommendations. The basic idea is to routinely conduct heuristic-based data augmentation and apply contrastive losses (e.g., InfoNCE) on the augmented views. Yet, some CF-tailored challenges make this adoption suboptimal, such as the issue of out-of-distribution, the risk of false negatives, and the nature of top-K evaluation. They necessitate the CL-based CF scheme to focus more on mining hard negatives and distinguishing false negatives from the vast unlabeled user-item interactions, for informative contrast signals. Worse still, there is limited understanding of contrastive loss in CF methods, especially w.r.t. its generalization ability. To bridge the gap, we delve into the reasons underpinning the success of contrastive loss in CF, and propose a principled Adversarial InfoNCE loss (AdvInfoNCE), which is a variant of InfoNCE, specially tailored for CF methods. AdvInfoNCE adaptively explores and assigns hardness to each negative instance in an adversarial fashion and further utilizes a fine-grained hardness-aware ranking criterion to empower the recommender's generalization ability. Training CF models with AdvInfoNCE, we validate the effectiveness of AdvInfoNCE on both synthetic and real-world benchmark datasets, thus showing its generalization ability to mitigate out-of-distribution problems. Given the theoretical guarantees and empirical superiority of AdvInfoNCE over most contrastive loss functions, we advocate its adoption as a standard loss in recommender systems, particularly for the out-of-distribution tasks. Codes are available at https://github.com/LehengTHU/AdvInfoNCE.


title: Efficient Algorithms for Generalized Linear Bandits with Heavy-tailed Rewards

abstract: This paper investigates the problem of generalized linear bandits with heavy-tailed rewards, whose $(1+\epsilon)$-th moment is bounded for some $\epsilon\in (0,1]$. Although there exist methods for generalized linear bandits, most of them focus on bounded or sub-Gaussian rewards and are not well-suited for many real-world scenarios, such as financial markets and web-advertising. To address this issue, we propose two novel algorithms based on truncation and mean of medians. These algorithms achieve an almost optimal regret bound of $\widetilde{O}(dT^{\frac{1}{1+\epsilon}})$, where $d$ is the dimension of contextual information and $T$ is the time horizon. Our truncation-based algorithm supports online learning, distinguishing it from existing truncation-based approaches. Additionally, our mean-of-medians-based algorithm requires only $O(\log T)$ rewards and one estimator per epoch, making it more practical. Moreover, our algorithms improve the regret bounds by a logarithmic factor compared to existing algorithms when $\epsilon=1$. Numerical experimental results confirm the merits of our algorithms.


title: Towards Combinatorial Generalization for Catalysts: A Kohn-Sham Charge-Density Approach

abstract: The Kohn-Sham equations underlie many important applications such as the discovery of new catalysts. Recent machine learning work on catalyst modeling has focused on prediction of the energy, but has so far not yet demonstrated significant out-of-distribution generalization. Here we investigate another approach based on the pointwise learning of the Kohn-Sham charge-density. On a new dataset of bulk catalysts with charge densities, we show density models can generalize to new structures with combinations of elements not seen at train time, a form of combinatorial generalization. We show that over 80% of binary and ternary test cases achieve faster convergence than standard baselines in Density Functional Theory, amounting to an average reduction of 13% in the number of iterations required to reach convergence, which may be of independent interest. Our results suggest that density learning is a viable alternative, trading greater inference costs for a step towards combinatorial generalization, a key property for applications.


title: Detangling the role of climate in vegetation productivity with an explainable convolutional neural network

abstract: Forests of the Earth are a vital carbon sink while providing an essential habitat for biodiversity. Vegetation productivity (VP) is a critical indicator of carbon uptake in the atmosphere. The leaf area index is a crucial vegetation index used in VP estimation. This work proposes to predict the leaf area index (LAI) using climate variables to better understand future productivity dynamics; our approach leverages the capacities of the V-Net architecture for spatiotemporal LAI prediction. Preliminary results are well-aligned with established quality standards of LAI products estimated from Earth observation data. We hope that this work serves as a robust foundation for subsequent research endeavours, particularly for the incorporation of prediction attribution methodologies, which hold promise for elucidating the underlying climate change drivers of global vegetation productivity.


title: Statistics of inhomogeneous turbulence in large scale quasi-geostrophic dynamics

abstract: A remarkable feature of two-dimensional turbulence is the transfer of energy from small to large scales. This process can result in the self-organization of the flow into large, coherent structures due to energy condensation at the largest scales. We investigate the formation of this condensate in a quasi-geostropic flow in the limit of small Rossby deformation radius, namely the large scale quasi-geostrophic model. In this model potential energy is transferred up-scale while kinetic energy is transferred down-scale in a direct cascade. We focus on a jet mean flow and carry out a thorough investigation of the second order statistics for this flow, combining a quasi-linear analytical approach with direct numerical simulations. We show that the quasi-linear approach applies in regions where jets are strong and is able to capture all second order correlators in that region, including those related to the kinetic energy. This is a consequence of the blocking of the direct cascade by the mean flow in jet regions, suppressing fluctuation-fluctuation interactions. The suppression of the direct cascade is demonstrated using a local coarse-graining approach allowing to measure space dependent inter-scale kinetic energy fluxes, which we show are concentrated in between jets in our simulations. We comment on the possibility of a similar direct cascade arrest in other two-dimensional flows, arguing that it is a special feature of flows in which the fluid element interactions are local in space


title: Assessing global ion thermal confinement in critical-gradient-optimized stellarators

abstract: We investigate the confinement properties of two recently devised quasi-helically symmetric stellarator configurations, HSK and QSTK. Both have been optimized for large critical gradients of the ion temperature gradient mode, which is an important driver of turbulent transport in magnetic confinement fusion devices. To predict the resulting core plasma profiles, we utilize an advanced theoretical framework based on the gyrokinetic codes GENE and GENE-3D, coupled to the transport code TANGO. Compared to the HSX stellarator, both HSK and QSTK achieve significantly higher core-to-edge temperature ratios, partly thanks to their smaller aspect ratios, with the other part due to more detailed shaping of the magnetic geometry achieved during optimization. The computed confinement time, however, is less sensitive to core temperature than edge temperature, simply due to the disproportionate influence the edge has on stored plasma energy. We therefore emphasize the possible benefits of further optimizing turbulence in the outer core region, and the need to include accurate modelling of confinement in the edge region in order to assess overall plasma performance of turbulence optimized stellarators.


title: ALERTA-Net: A Temporal Distance-Aware Recurrent Networks for Stock Movement and Volatility Prediction

abstract: For both investors and policymakers, forecasting the stock market is essential as it serves as an indicator of economic well-being. To this end, we harness the power of social media data, a rich source of public sentiment, to enhance the accuracy of stock market predictions. Diverging from conventional methods, we pioneer an approach that integrates sentiment analysis, macroeconomic indicators, search engine data, and historical prices within a multi-attention deep learning model, masterfully decoding the complex patterns inherent in the data. We showcase the state-of-the-art performance of our proposed model using a dataset, specifically curated by us, for predicting stock market movements and volatility.


title: Jamming modulates coalescence dynamics of shear-thickening colloidal droplets

abstract: Recent investigations into coalescence dynamics of complex fluid droplets revealed the existence of sub-Newtonian behaviour for polymeric fluids (elastic and shear thinning). We hypothesize that such delayed coalescence or sub-Newtonian coalescence dynamics may be extended to the general class of shear thickening fluids. To investigate this droplets of aqueous corn-starch suspensions were chosen and its coalescence in sessile pendant configuration was probed by high-speed real time imaging. Temporal evolution of the neck (growth) during coalescence was quantified as a function of suspended particle weight fraction \phi_w. The necking behaviour was found to evolve as the power-law relation $R=at^b$ where R is neck radius with exponent $\b\le0.5$ implying it is a subset of the generic sub-Newtonian coalescence. Second significant delay in the coalescence dynamics is observed for particle fractions beyond the jamming fraction {\ \phi}_w>\ \phi_J\geq0.35}. Our proposed theoretical model captures this delay implicitly through altered suspension viscosity stemming from increased particle content.


title: Simultaneous embedding of multiple attractor manifolds in a recurrent neural network using constrained gradient optimization

abstract: The storage of continuous variables in working memory is hypothesized to be sustained in the brain by the dynamics of recurrent neural networks (RNNs) whose steady states form continuous manifolds. In some cases, it is thought that the synaptic connectivity supports multiple attractor manifolds, each mapped to a different context or task. For example, in hippocampal area CA3, positions in distinct environments are represented by distinct sets of population activity patterns, each forming a continuum. It has been argued that the embedding of multiple continuous attractors in a single RNN inevitably causes detrimental interference: quenched noise in the synaptic connectivity disrupts the continuity of each attractor, replacing it by a discrete set of steady states that can be conceptualized as lying on local minima of an abstract energy landscape. Consequently, population activity patterns exhibit systematic drifts towards one of these discrete minima, thereby degrading the stored memory over time. Here we show that it is possible to dramatically attenuate these detrimental interference effects by adjusting the synaptic weights. Synaptic weight adjustments are derived from a loss function that quantifies the roughness of the energy landscape along each of the embedded attractor manifolds. By minimizing this loss function, the stability of states can be dramatically improved, without compromising the capacity.


title: Audio-Visual Instance Segmentation

abstract: In this paper, we propose a new multi-modal task, namely audio-visual instance segmentation (AVIS), in which the goal is to identify, segment, and track individual sounding object instances in audible videos, simultaneously. To our knowledge, it is the first time that instance segmentation has been extended into the audio-visual domain. To better facilitate this research, we construct the first audio-visual instance segmentation benchmark (AVISeg). Specifically, AVISeg consists of 1,258 videos with an average duration of 62.6 seconds from YouTube and public audio-visual datasets, where 117 videos have been annotated by using an interactive semi-automatic labeling tool based on the Segment Anything Model (SAM). In addition, we present a simple baseline model for the AVIS task. Our new model introduces an audio branch and a cross-modal fusion module to Mask2Former to locate all sounding objects. Finally, we evaluate the proposed method using two backbones on AVISeg. We believe that AVIS will inspire the community towards a more comprehensive multi-modal understanding.


title: Marches al\'eatoires et \'el\'ements contractants sur des espaces CAT(0)

abstract: This thesis is dedicated to random walks on spaces with non-positive curvature. In particular, we study the case of group actions on CAT(0) spaces that admit contracting elements, that is, whose properties mimic those of loxodromic isometries in Gromov-hyperbolic spaces. In this context, we prove several limit laws, among which the almost sure convergence to the boundary without moment assumption, positivity of the drift and a central limit theorem. In a second part, we study boundary maps and stationary measures on affine buildings of type $\tilde{A}_2$, and we show that there always exists a hyperbolic isometry for a non-elementary action by isometries on such a space. Our approach involves the use of hyperbolic models for CAT(0) spaces, which were constructed by H.~Petyt, D.~Spriano and A.~Zalloum, and measured boundary theory, whose principles come from H.~Furstenberg.


title: A Hubble Constant Estimate from Galaxy Cluster and type Ia SNe Observations

abstract: In this work, we constrain the Hubble constant parameter, $H_0$, using a combination of the Pantheon sample and galaxy clusters (GC) measurements from minimal cosmological assumptions. Assuming the validity of the cosmic distance duality relation, an estimator is created for $H_0$ that only depends on simple geometrical distances, which is evaluated from Pantheon and a GC angular diameter distance sample afterward. The statistical and systematic errors in GC measurements are summed in quadrature in our analysis. We find $H_0 = 67.22 \pm 6.07$ ${\rm km \ s^{-1} Mpc^{-1}}$ in $1\sigma$ confidence level (C.L.). This measurement presents an error of around 9\%, showing that future and better GC measurements can shed light on the current Hubble tension


title: On the ascent of atomicity to one-dimensional monoid algebras

abstract: A commutative cancellative monoid is atomic if every non-invertible element factors into irreducibles (also called atoms), while an integral domain is atomic if its multiplicative monoid is atomic. Back in the eighties, Gilmer posed the question of whether the fact that a torsion-free monoid~$M$ and an integral domain $R$ are both atomic implies that the monoid algebra $R[M]$ of $M$ over $R$ is also atomic. In general this is not true, and the first negative answer to this question was given by Roitman in 1993: he constructed of an atomic integral domain whose polynomial extension is not atomic. More recently, Coykendall and the first author constructed finite-rank torsion-free atomic monoids whose algebras over certain finite fields are not atomic. Still, the ascent of atomicity from finite-rank torsion-free monoids to their monoid algebras over fields of characteristic zero is an open problem. The main purpose of this paper is to provide a negative answer to this problem. We actually construct a rank-one torsion-free atomic monoid whose monoid algebras over any field are not atomic. To do so, we introduce and study a methodological construction inside the class of rank-one torsion-free monoid that we call lifting: it consists in embedding a given monoid into another monoid that is often more tractable from the arithmetic viewpoint.


title: Episodic Multi-Task Learning with Heterogeneous Neural Processes

abstract: This paper focuses on the data-insufficiency problem in multi-task learning within an episodic training setup. Specifically, we explore the potential of heterogeneous information across tasks and meta-knowledge among episodes to effectively tackle each task with limited data. Existing meta-learning methods often fail to take advantage of crucial heterogeneous information in a single episode, while multi-task learning models neglect reusing experience from earlier episodes. To address the problem of insufficient data, we develop Heterogeneous Neural Processes (HNPs) for the episodic multi-task setup. Within the framework of hierarchical Bayes, HNPs effectively capitalize on prior experiences as meta-knowledge and capture task-relatedness among heterogeneous tasks, mitigating data-insufficiency. Meanwhile, transformer-structured inference modules are designed to enable efficient inferences toward meta-knowledge and task-relatedness. In this way, HNPs can learn more powerful functional priors for adapting to novel heterogeneous tasks in each meta-test episode. Experimental results show the superior performance of the proposed HNPs over typical baselines, and ablation studies verify the effectiveness of the designed inference modules.


title: An Investigation of Darwiche and Pearl's Postulates for Iterated Belief Update

abstract: Belief revision and update, two significant types of belief change, both focus on how an agent modify her beliefs in presence of new information. The most striking difference between them is that the former studies the change of beliefs in a static world while the latter concentrates on a dynamically-changing world. The famous AGM and KM postulates were proposed to capture rational belief revision and update, respectively. However, both of them are too permissive to exclude some unreasonable changes in the iteration. In response to this weakness, the DP postulates and its extensions for iterated belief revision were presented. Furthermore, Rodrigues integrated these postulates in belief update. Unfortunately, his approach does not meet the basic requirement of iterated belief update. This paper is intended to solve this problem of Rodrigues's approach. Firstly, we present a modification of the original KM postulates based on belief states. Subsequently, we migrate several well-known postulates for iterated belief revision to iterated belief update. Moreover, we provide the exact semantic characterizations based on partial preorders for each of the proposed postulates. Finally, we analyze the compatibility between the above iterated postulates and the KM postulates for belief update.


title: Robust Offline Reinforcement learning with Heavy-Tailed Rewards

abstract: This paper endeavors to augment the robustness of offline reinforcement learning (RL) in scenarios laden with heavy-tailed rewards, a prevalent circumstance in real-world applications. We propose two algorithmic frameworks, ROAM and ROOM, for robust off-policy evaluation and offline policy optimization (OPO), respectively. Central to our frameworks is the strategic incorporation of the median-of-means method with offline RL, enabling straightforward uncertainty estimation for the value function estimator. This not only adheres to the principle of pessimism in OPO but also adeptly manages heavy-tailed rewards. Theoretical results and extensive experiments demonstrate that our two frameworks outperform existing methods on the logged dataset exhibits heavy-tailed reward distributions. The implementation of the proposal is available at https://github.com/Mamba413/ROOM.


title: Laplacian Canonization: A Minimalist Approach to Sign and Basis Invariant Spectral Embedding

abstract: Spectral embedding is a powerful graph embedding technique that has received a lot of attention recently due to its effectiveness on Graph Transformers. However, from a theoretical perspective, the universal expressive power of spectral embedding comes at the price of losing two important invariance properties of graphs, sign and basis invariance, which also limits its effectiveness on graph data. To remedy this issue, many previous methods developed costly approaches to learn new invariants and suffer from high computation complexity. In this work, we explore a minimal approach that resolves the ambiguity issues by directly finding canonical directions for the eigenvectors, named Laplacian Canonization (LC). As a pure pre-processing method, LC is light-weighted and can be applied to any existing GNNs. We provide a thorough investigation, from theory to algorithm, on this approach, and discover an efficient algorithm named Maximal Axis Projection (MAP) that works for both sign and basis invariance and successfully canonizes more than 90% of all eigenvectors. Experiments on real-world benchmark datasets like ZINC, MOLTOX21, and MOLPCBA show that MAP consistently outperforms existing methods while bringing minimal computation overhead. Code is available at https://github.com/PKU-ML/LaplacianCanonization.


title: On the Accuracy of Hotelling-Type Asymmetric Tensor Deflation: A Random Tensor Analysis

abstract: This work introduces an asymptotic study of Hotelling-type tensor deflation in the presence of noise, in the regime of large tensor dimensions. Specifically, we consider a low-rank asymmetric tensor model of the form $\sum_{i=1}^r \beta_i{\mathcal{A}}_i + {\mathcal{W}}$ where $\beta_i\geq 0$ and the ${\mathcal{A}}_i$'s are unit-norm rank-one tensors such that $\left| \langle {\mathcal{A}}_i, {\mathcal{A}}_j \rangle \right| \in [0, 1]$ for $i\neq j$ and ${\mathcal{W}}$ is an additive noise term. Assuming that the dominant components are successively estimated from the noisy observation and subsequently subtracted, we leverage recent advances in random tensor theory in the regime of asymptotically large tensor dimensions to analytically characterize the estimated singular values and the alignment of estimated and true singular vectors at each step of the deflation procedure. Furthermore, this result can be used to construct estimators of the signal-to-noise ratios $\beta_i$ and the alignments between the estimated and true rank-1 signal components.


title: Carbon-Awareness in CI/CD

abstract: While the environmental impact of digitalization is becoming more and more evident, the climate crisis has become a major issue for society. For instance, data centers alone account for 2.7% of Europe's energy consumption today. A considerable part of this load is accounted for by cloud-based services for automated software development, such as continuous integration and delivery (CI/CD) workflows. In this paper, we discuss opportunities and challenges for greening CI/CD services by better aligning their execution with the availability of low-carbon energy. We propose a system architecture for carbon-aware CI/CD services, which uses historical runtime information and, optionally, user-provided information. We examined the potential effectiveness of different scheduling strategies using real carbon intensity data and 7,392 workflow executions of Github Actions, a popular CI/CD service. Our results show, that user-provided information on workflow deadlines can effectively improve carbon-aware scheduling.


title: Oriented trees and paths in digraphs

abstract: Which conditions ensure that a digraph contains all oriented paths of some given length, or even a all oriented trees of some given size, as a subgraph? One possible condition could be that the host digraph is a tournament of a certain order. In arbitrary digraphs and oriented graphs, conditions on the chromatic number, on the edge density, on the minimum outdegree and on the minimum semidegree have been proposed. In this survey, we review the known results, and highlight some open questions in the area.


title: Proceedings of the First Workshop on Trends in Configurable Systems Analysis

abstract: The analysis of configurable systems, i.e., systems those behaviors depend on parameters or support various features, is challenging due to the exponential blowup arising in the number of configuration options. This volume contains the post-proceedings of TiCSA 2023, the first workshop on Trends in Configurable Systems Analysis, where current challenges and solutions in configurable systems analysis were presented and discussed.


title: Conant's Metric Spectra Problem

abstract: In this paper, we try to minimize the scope of possible unique metric spectra up to equivalence. While it is well known that every spectra $S\subseteq \mathbb{R}^+$ is equivalent to a spectra $T\subseteq \mathbb{N}$, it has remained open if $T$ could also maintain a desirable combinatorial form. Conant questioned if $T= \{t_1,...,t_n \}_{<}$ could be taken such that $2^i-1 \leq t_i \leq 2^n -1$. In this paper, we come to two partial answers. The first is that the largest element $t_n$ can be chosen such that $t_n\leq 2^n $. Approximating a full solution, we also observe $T$ with the combinatorial form $2^i \leq t_i \leq 2^{n+1}$. Our methods are rather unique in the field as we utilize linear optimization and polygonal geometry to achieve our results. Our work aims to approach a full characterization of metric spectra, and simplify future computational endeavors in the field.


title: B-splines and kernels of trigonometric interpolation splines

abstract: This article focuses on trigonometric Riemann B-splines and Riemann kernels of trigonometric interpolation splines of arbitrary order; it is shown that trigonometric interpolation splines are a convolution of trigonometric B-splines with corresponding kernels. Theoretical statements are followed by examples, and the results are applicable in many practical areas.


title: Single-photon scattering on a two-qubit system. Spatio-temporal structure of the scattered field

abstract: In this paper, we study the spatiotemporal distribution of the photon electric field produced by the scattering of a single photon narrow pulse from a system of two identical qubits coupled to continuum modes in a one-dimensional (1D) open waveguide. We derive the time-dependent dynamical equations for qubits' and photon amplitudes which allow the calculation of the photon backward and forward scattering fields in the whole space: before qubits, between qubits, and behind the qubits. The scattered field consists of several contributions that describe a free field of incoming photon, a spontaneous exponential decay of excited qubits, a slowly decaying part that dies out as the inverse powers of $t$, and a lossless part that represents a steady state solution as $t\rightarrow\infty$. For our system, we find the transmittance and reflectance fields as both time and distance from the qubits tend to infinity. We show that as the time after the event of scattering tends to infinity, the steady state photon the field is being formed in the whole one-dimensional space. If the distance $d$ between qubits is equal to the integer of the wavelength $\lambda$, the field energy exhibits temporal beatings between the qubit frequency $\Omega$ and the photon frequency $\omega_S$ with the period $T=2\pi/(\omega_S-\Omega)$.


title: WCLD: Curated Large Dataset of Criminal Cases from Wisconsin Circuit Courts

abstract: Machine learning based decision-support tools in criminal justice systems are subjects of intense discussions and academic research. There are important open questions about the utility and fairness of such tools. Academic researchers often rely on a few small datasets that are not sufficient to empirically study various real-world aspects of these questions. In this paper, we contribute WCLD, a curated large dataset of 1.5 million criminal cases from circuit courts in the U.S. state of Wisconsin. We used reliable public data from 1970 to 2020 to curate attributes like prior criminal counts and recidivism outcomes. The dataset contains large number of samples from five racial groups, in addition to information like sex and age (at judgment and first offense). Other attributes in this dataset include neighborhood characteristics obtained from census data, detailed types of offense, charge severity, case decisions, sentence lengths, year of filing etc. We also provide pseudo-identifiers for judge, county and zipcode. The dataset will not only enable researchers to more rigorously study algorithmic fairness in the context of criminal justice, but also relate algorithmic challenges with various systemic issues. We also discuss in detail the process of constructing the dataset and provide a datasheet. The WCLD dataset is available at \url{https://clezdata.github.io/wcld/}.


title: The Evolution of the Interplay Between Input Distributions and Linear Regions in Networks

abstract: It is commonly recognized that the expressiveness of deep neural networks is contingent upon a range of factors, encompassing their depth, width, and other relevant considerations. Currently, the practical performance of the majority of deep neural networks remains uncertain. For ReLU (Rectified Linear Unit) networks with piecewise linear activations, the number of linear convex regions serves as a natural metric to gauge the network's expressivity. In this paper, we count the number of linear convex regions in deep neural networks based on ReLU. In particular, we prove that for any one-dimensional input, there exists a minimum threshold for the number of neurons required to express it. We also empirically observe that for the same network, intricate inputs hinder its capacity to express linear regions. Furthermore, we unveil the iterative refinement process of decision boundaries in ReLU networks during training. We aspire for our research to serve as an inspiration for network optimization endeavors and aids in the exploration and analysis of the behaviors exhibited by deep networks.


title: Stochastic partial differential equations associated with Feller processes

abstract: For the stochastic partial differential equation $\frac{\partial u}{\partial t}=\mathcal L u +u\dot W$ where $\dot W$ is Gaussian noise colored in time and $\mathcal L$ is the infinitesimal generator of a Feller process $X$, we obtain Feynman-Kac type of representations for the Stratonovich and Skorohod solutions as well as for their moments. The regularity of the law and the H\"older continuity of the solutions are also studied.


title: Latent class analysis by regularized spectral clustering

abstract: The latent class model is a powerful tool for identifying latent classes within populations that share common characteristics for categorical data in social, psychological, and behavioral sciences. In this article, we propose two new algorithms to estimate a latent class model for categorical data. Our algorithms are developed by using a newly defined regularized Laplacian matrix calculated from the response matrix. We provide theoretical convergence rates of our algorithms by considering a sparsity parameter and show that our algorithms stably yield consistent latent class analysis under mild conditions. Additionally, we propose a metric to capture the strength of latent class analysis and several procedures designed based on this metric to infer how many latent classes one should use for real-world categorical data. The efficiency and accuracy of our algorithms are verified by extensive simulated experiments, and we further apply our algorithms to real-world categorical data with promising results.


title: Debunking Free Fusion Myth: Online Multi-view Anomaly Detection with Disentangled Product-of-Experts Modeling

abstract: Multi-view or even multi-modal data is appealing yet challenging for real-world applications. Detecting anomalies in multi-view data is a prominent recent research topic. However, most of the existing methods 1) are only suitable for two views or type-specific anomalies, 2) suffer from the issue of fusion disentanglement, and 3) do not support online detection after model deployment. To address these challenges, our main ideas in this paper are three-fold: multi-view learning, disentangled representation learning, and generative model. To this end, we propose dPoE, a novel multi-view variational autoencoder model that involves (1) a Product-of-Experts (PoE) layer in tackling multi-view data, (2) a Total Correction (TC) discriminator in disentangling view-common and view-specific representations, and (3) a joint loss function in wrapping up all components. In addition, we devise theoretical information bounds to control both view-common and view-specific representations. Extensive experiments on six real-world datasets markedly demonstrate that the proposed dPoE outperforms baselines.


title: Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies

abstract: Thematic analysis and other variants of inductive coding are widely used qualitative analytic methods within empirical legal studies (ELS). We propose a novel framework facilitating effective collaboration of a legal expert with a large language model (LLM) for generating initial codes (phase 2 of thematic analysis), searching for themes (phase 3), and classifying the data in terms of the themes (to kick-start phase 4). We employed the framework for an analysis of a dataset (n=785) of facts descriptions from criminal court opinions regarding thefts. The goal of the analysis was to discover classes of typical thefts. Our results show that the LLM, namely OpenAI's GPT-4, generated reasonable initial codes, and it was capable of improving the quality of the codes based on expert feedback. They also suggest that the model performed well in zero-shot classification of facts descriptions in terms of the themes. Finally, the themes autonomously discovered by the LLM appear to map fairly well to the themes arrived at by legal experts. These findings can be leveraged by legal researchers to guide their decisions in integrating LLMs into their thematic analyses, as well as other inductive coding projects.