title: SG-VAD: Stochastic Gates Based Speech Activity Detection

abstract: We propose a novel voice activity detection (VAD) model in a low-resource environment. Our key idea is to model VAD as a denoising task, and construct a network that is designed to identify nuisance features for a speech classification task. We train the model to simultaneously identify irrelevant features while predicting the type of speech event. Our model contains only 7.8K parameters, outperforms the previously proposed methods on the AVA-Speech evaluation set, and provides comparative results on the HAVIC dataset. We present its architecture, experimental results, and ablation study on the model's components. We publish the code and the models here https://www.github.com/jsvir/vad.


title: LegoNet: A Fast and Exact Unlearning Architecture

abstract: Machine unlearning aims to erase the impact of specific training samples upon deleted requests from a trained model. Re-training the model on the retained data after deletion is an effective but not efficient way due to the huge number of model parameters and re-training samples. To speed up, a natural way is to reduce such parameters and samples. However, such a strategy typically leads to a loss in model performance, which poses the challenge that increasing the unlearning efficiency while maintaining acceptable performance. In this paper, we present a novel network, namely \textit{LegoNet}, which adopts the framework of ``fixed encoder + multiple adapters''. We fix the encoder~(\ie the backbone for representation learning) of LegoNet to reduce the parameters that need to be re-trained during unlearning. Since the encoder occupies a major part of the model parameters, the unlearning efficiency is significantly improved. However, fixing the encoder empirically leads to a significant performance drop. To compensate for the performance loss, we adopt the ensemble of multiple adapters, which are independent sub-models adopted to infer the prediction by the encoding~(\ie the output of the encoder). Furthermore, we design an activation mechanism for the adapters to further trade off unlearning efficiency against model performance. This mechanism guarantees that each sample can only impact very few adapters, so during unlearning, parameters and samples that need to be re-trained are both reduced. The empirical experiments verify that LegoNet accomplishes fast and exact unlearning while maintaining acceptable performance, synthetically outperforming unlearning baselines.


title: Addressing Bias in Face Detectors using Decentralised Data collection with incentives

abstract: Recent developments in machine learning have shown that successful models do not rely only on huge amounts of data but the right kind of data. We show in this paper how this data-centric approach can be facilitated in a decentralized manner to enable efficient data collection for algorithms. Face detectors are a class of models that suffer heavily from bias issues as they have to work on a large variety of different data. We also propose a face detection and anonymization approach using a hybrid MultiTask Cascaded CNN with FaceNet Embeddings to benchmark multiple datasets to describe and evaluate the bias in the models towards different ethnicities, gender, and age groups along with ways to enrich fairness in a decentralized system of data labeling, correction, and verification by users to create a robust pipeline for model retraining.


title: Ultra-fast, programmable, and electronics-free soft robots enabled by snapping metacaps

abstract: Soft robots have a myriad of potentials because of their intrinsically compliant bodies, enabling safe interactions with humans and adaptability to unpredictable environments. However, most of them have limited actuation speeds, require complex control systems, and lack sensing capabilities. To address these challenges, here we geometrically design a class of metacaps whose rich nonlinear mechanical behaviors can be harnessed to create soft robots with unprecedented functionalities. Specifically, we demonstrate a sensor-less metacap gripper that can grasp objects in 3.75 ms upon physical contact and a pneumatically actuated gripper with tunable actuation behaviors that have little dependence on the rate of input. Both grippers can be readily integrated into a robotic platform for practical applications. Furthermore, we demonstrate that the metacap enables propelling of a swimming robot, exhibiting amplified swimming speed as well as untethered, electronics-free swimming with tunable speeds. Our metacaps provide new strategies to design the next-generation soft robots that require high transient output energy and are capable of autonomous and electronics-free maneuvering.


title: The Skorokhod Topologies: What They Are and Why We Should Care

abstract: This paper presents a gentle and informal introduction to the Skorokhod topologies. Focus is on motivating examples and concepts.


title: Extending Cobot's Motion Intention Visualization by Haptic Feedback

abstract: Nowadays, robots are found in a growing number of areas where they collaborate closely with humans. Enabled by lightweight materials and safety sensors, these cobots are gaining increasing popularity in domestic care, supporting people with physical impairments in their everyday lives. However, when cobots perform actions autonomously, it remains challenging for human collaborators to understand and predict their behavior, which is crucial for achieving trust and user acceptance. One significant aspect of predicting cobot behavior is understanding their motion intention and comprehending how they "think" about their actions. Moreover, other information sources often occupy human visual and audio modalities, rendering them frequently unsuitable for transmitting such information. We work on a solution that communicates cobot intention via haptic feedback to tackle this challenge. In our concept, we map planned motions of the cobot to different haptic patterns to extend the visual intention feedback.


title: Laugh Betrays You? Learning Robust Speaker Representation From Speech Containing Non-Verbal Fragments

abstract: The success of automatic speaker verification shows that discriminative speaker representations can be extracted from neutral speech. However, as a kind of non-verbal voice, laughter should also carry speaker information intuitively. Thus, this paper focuses on exploring speaker verification about utterances containing non-verbal laughter segments. We collect a set of clips with laughter components by conducting a laughter detection script on VoxCeleb and part of the CN-Celeb dataset. To further filter untrusted clips, probability scores are calculated by our binary laughter detection classifier, which is pre-trained by pure laughter and neutral speech. After that, based on the clips whose scores are over the threshold, we construct trials under two different evaluation scenarios: Laughter-Laughter (LL) and Speech-Laughter (SL). Then a novel method called Laughter-Splicing based Network (LSN) is proposed, which can significantly boost performance in both scenarios and maintain the performance on the neutral speech, such as the VoxCeleb1 test set. Specifically, our system achieves relative 20% and 22% improvement on Laughter-Laughter and Speech-Laughter trials, respectively. The meta-data and sample clips have been released at https://github.com/nevermoreLin/Laugh_LSN.


title: Assessing Phrase Break of ESL speech with Pre-trained Language Models

abstract: This work introduces an approach to assessing phrase break in ESL learners' speech with pre-trained language models (PLMs). Different with traditional methods, this proposal converts speech to token sequences, and then leverages the power of PLMs. There are two sub-tasks: overall assessment of phrase break for a speech clip; fine-grained assessment of every possible phrase break position. Speech input is first force-aligned with texts, then pre-processed to a token sequence, including words and associated phrase break information. The token sequence is then fed into the pre-training and fine-tuning pipeline. In pre-training, a replaced break token detection module is trained with token data where each token has a certain percentage chance to be randomly replaced. In fine-tuning, overall and fine-grained scoring are optimized with text classification and sequence labeling pipeline, respectively. With the introduction of PLMs, the dependence on labeled training data has been greatly reduced, and performance has improved.


title: Reduction of electrostatic turbulence in a quasi-helically symmetric stellarator via critical gradient optimization

abstract: We present a stellarator configuration optimized for a large threshold (``critical gradient'') for the onset of the ion temperature gradient (ITG) driven mode, which achieves the largest critical gradient we have seen in any stellarator. Above this threshold, gyrokinetic simulations show that the configuration has low turbulence levels over an experimentally relevant range of the drive strength. The applied optimization seeks to maximize the drift curvature, leading to enhanced local-shear stabilization of toroidal ITG modes, and the associated turbulence. These benefits are combined with excellent quasisymmetry, yielding low neoclassical transport and vanishingly small alpha particle losses. Analysis of the resulting configuration suggests a trade-off between magnetohydrodynamic (MHD) and ITG stability.


title: UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance

abstract: Diffusion generative models have recently greatly improved the power of text-conditioned image generation. Existing image generation models mainly include text conditional diffusion model and cross-modal guided diffusion model, which are good at small scene image generation and complex scene image generation respectively. In this work, we propose a simple yet effective approach, namely UPainting, to unify simple and complex scene image generation, as shown in Figure 1. Based on architecture improvements and diverse guidance schedules, UPainting effectively integrates cross-modal guidance from a pretrained image-text matching model into a text conditional diffusion model that utilizes a pretrained Transformer language model as the text encoder. Our key findings is that combining the power of large-scale Transformer language model in understanding language and image-text matching model in capturing cross-modal semantics and style, is effective to improve sample fidelity and image-text alignment of image generation. In this way, UPainting has a more general image generation capability, which can generate images of both simple and complex scenes more effectively. To comprehensively compare text-to-image models, we further create a more general benchmark, UniBench, with well-written Chinese and English prompts in both simple and complex scenes. We compare UPainting with recent models and find that UPainting greatly outperforms other models in terms of caption similarity and image fidelity in both simple and complex scenes. UPainting project page \url{https://upainting.github.io/}.


title: Parameter-efficient transfer learning of pre-trained Transformer models for speaker verification using adapters

abstract: Recently, the pre-trained Transformer models have received a rising interest in the field of speech processing thanks to their great success in various downstream tasks. However, most fine-tuning approaches update all the parameters of the pre-trained model, which becomes prohibitive as the model size grows and sometimes results in overfitting on small datasets. In this paper, we conduct a comprehensive analysis of applying parameter-efficient transfer learning (PETL) methods to reduce the required learnable parameters for adapting to speaker verification tasks. Specifically, during the fine-tuning process, the pre-trained models are frozen, and only lightweight modules inserted in each Transformer block are trainable (a method known as adapters). Moreover, to boost the performance in a cross-language low-resource scenario, the Transformer model is further tuned on a large intermediate dataset before directly fine-tuning it on a small dataset. With updating fewer than 4% of parameters, (our proposed) PETL-based methods achieve comparable performances with full fine-tuning methods (Vox1-O: 0.55%, Vox1-E: 0.82%, Vox1-H:1.73%).


title: Improving Multi-class Classifier Using Likelihood Ratio Estimation with Regularization

abstract: The universal-set naive Bayes classifier (UNB)~\cite{Komiya:13}, defined using likelihood ratios (LRs), was proposed to address imbalanced classification problems. However, the LR estimator used in the UNB overestimates LRs for low-frequency data, degrading the classification performance. Our previous study~\cite{Kikuchi:19} proposed an effective LR estimator even for low-frequency data. This estimator uses regularization to suppress the overestimation, but we did not consider imbalanced data. In this paper, we integrated the estimator with the UNB. Our experiments with imbalanced data showed that our proposed classifier effectively adjusts the classification scores according to the class balance using regularization parameters and improves the classification performance.


title: A Survey on Causal Representation Learning and Future Work for Medical Image Analysis

abstract: Statistical machine learning algorithms have achieved state-of-the-art results on benchmark datasets, outperforming humans in many tasks. However, the out-of-distribution data and confounder, which have an unpredictable causal relationship, significantly degrade the performance of the existing models. Causal Representation Learning (CRL) has recently been a promising direction to address the causal relationship problem in vision understanding. This survey presents recent advances in CRL in vision. Firstly, we introduce the basic concept of causal inference. Secondly, we analyze the CRL theoretical work, especially in invariant risk minimization, and the practical work in feature understanding and transfer learning. Finally, we propose a future research direction in medical image analysis and CRL general theory.


title: Precise Estimate of Charged Wino Decay Rate

abstract: The Wino is an $\mathrm{SU}(2)_{L}$ triplet Majorana fermion and a well-motivated dark matter candidate. The mass difference between the charged and the neutral Winos is small thanks to the $\mathrm{SU}(2)_{L}$ symmetry. The small mass difference makes the charged Wino meta-stable, which provides disappearing charged track signatures at collider experiments. The constraint on the Wino dark matter at the LHC strongly depends on the Wino lifetime. We compute the next-to-leading order (NLO) correction of the charged Wino decay and make the most precise estimate of the decay rate. We find that the NLO decay rate is determined by the mass difference and scarcely depend on the Wino mass itself in the heavy Wino limit. As a result, we find the NLO correction gives a minor impact on the lifetime of 2-4% increase.


title: Lattice effects on the physical properties of half doped perovskite ruthenates

abstract: We investigate the unusual phase transitions in SrRuO$_{3}$ and Sr$_{0.5}$Ca$_{0.5}$Ru$_{1-x}$Cr$_{x}$O$_{3}$ (x=0,0.05 and 0.1) employing x-ray diffraction, resistivity, magnetic studies and x-ray photoemission spectroscopy. Our results show the compounds undergo crossover from $itinerant$ ferromagnetism to $localised$ ferromagnetism. The combined studies suggests Ru and Cr to be in 4+ valence state. A Griffith phase and an enhancement in Curie temperature (Tc) from 38 K to 107 K is observed with Cr doping. A shift in the chemical potential towards the valence band is observed with Cr doping. In the metallic samples, interestingly, a direct link between the resistivity and orthorhombic strain is observed. Detailed studies in this direction will be helpful to understand the nature of interactions and hence manoeuvre its properties. In the non metallic samples, the resistivity is mainly governed by disorder and electron-electron correlation effects. The value of the resistivity for the 5% Cr doped sample suggests semi metallic behaviour. Understanding its nature in detail using electron spectroscopic techniques could unravel the possibility of its utility in high mobility transistors at room temperature and its combined property with ferromagnetism will be helpful in making spintronic devices.


title: Cavity Jahn-Teller Polaritons in Molecules

abstract: We investigate Jahn-Teller (JT) polaritons, which emerge from the interaction of the two normal-incidence electromagnetic modes with perpendicular polarizations in a Fabry-Perot cavity resonator with JT active systems. These JT polaritons are characterized by a mixed $(+/-)$--circular electromagnetic polarization that originates from the molecular JT vibronic coupling of the material subsystem. Consequently, the exchange of photonic and vibronic angular momenta can be very efficient; exciting the cavity-JT system with short, polarized light pulses results in a dynamical and oscillatory response of the polarization in the cavity medium. Due to the photonic-vibronic coupling, we show how the cavity polarization direction becomes frequency dependent and does not necessarily coincide with the polarization direction of the external fields used to drive the system.


title: Deep Learning-Based Anomaly Detection in Synthetic Aperture Radar Imaging

abstract: In this paper, we proposed to investigate unsupervised anomaly detection in Synthetic Aperture Radar (SAR) images. Our approach considers anomalies as abnormal patterns that deviate from their surroundings but without any prior knowledge of their characteristics. In the literature, most model-based algorithms face three main issues. First, the speckle noise corrupts the image and potentially leads to numerous false detections. Second, statistical approaches may exhibit deficiencies in modeling spatial correlation in SAR images. Finally, neural networks based on supervised learning approaches are not recommended due to the lack of annotated SAR data, notably for the class of abnormal patterns. Our proposed method aims to address these issues through a self-supervised algorithm. The speckle is first removed through the deep learning SAR2SAR algorithm. Then, an adversarial autoencoder is trained to reconstruct an anomaly-free SAR image. Finally, a change detection processing step is applied between the input and the output to detect anomalies. Experiments are performed to show the advantages of our method compared to the conventional Reed-Xiaoli algorithm, highlighting the importance of an efficient despeckling pre-processing step.


title: Majda and ZND models for detonation: nonlinear stability vs. formation of singularities

abstract: For the ZND model, we show also singularity formation on the downstream side for arbitrary exponentially-growing weighted norms. For the Majda model on the other hand, we establish for appropriate such weighted norms a set of energy estimates implying not only non-formation of singularities near waves of arbitrary amplitude but also full asymptotic orbital stability for small-amplitude ones.


title: Review on Classification Techniques used in Biophysiological Stress Monitoring

abstract: Cardiovascular activities are directly related to the response of a body in a stressed condition. Stress, based on its intensity, can be divided into two types i.e. Acute stress (short-term stress) and Chronic stress (long-term stress). Repeated acute stress and continuous chronic stress may play a vital role in inflammation in the circulatory system and thus leads to a heart attack or to a stroke. In this study, we have reviewed commonly used machine learning classification techniques applied to different stress-indicating parameters used in stress monitoring devices. These parameters include Photoplethysmograph (PPG), Electrocardiographs (ECG), Electromyograph (EMG), Galvanic Skin Response (GSR), Heart Rate Variation (HRV), skin temperature, respiratory rate, Electroencephalograph (EEG) and salivary cortisol, used in stress monitoring devices. This study also provides a discussion on choosing a classifier, which depends upon a number of factors other than accuracy, like the number of subjects involved in an experiment, type of signals processing and computational limitations.


title: Centralization Problem for Opinion Convergence in Decentralized Networks

abstract: This paper aims to provide a new perspective on the interplay between decentralization -- a prevalent character of multi-agent systems -- and centralization, i.e., the task of imposing central control to meet system-level goals. In particular, in the context of networked opinion dynamic model, the paper proposes and discusses a framework for centralization. More precisely, a decentralized network consists of autonomous agents and their social structure that is unknown and dynamic. Centralization is a process of appointing agents in the network to act as access units who provide information and exert influence over their local surroundings. We discuss centralization for the DeGroot model of opinion dynamics, aiming to enforce opinion convergence using the minimum number of access units. We show that the key to the centralization process lies in selecting access units so that they form a dominating set. We then propose algorithms under a new local algorithmic framework, namely prowling, to accomplish this task. To validate our algorithm, we perform systematic experiments over both real-world and synthetic networks and verify that our algorithm outperforms benchmarks.


title: Eigenvalue tests for the number of latent factors in short panels

abstract: This paper studies new tests for the number of latent factors in a large cross-sectional factor model with small time dimension. These tests are based on the eigenvalues of variance-covariance matrices of (possibly weighted) asset returns, and rely on either the assumption of spherical errors, or instrumental variables for factor betas. We establish the asymptotic distributional results using expansion theorems based on perturbation theory for symmetric matrices. Our framework accommodates semi-strong factors in the systematic components. We propose a novel statistical test for weak factors against strong or semi-strong factors. We provide an empirical application to US equity data. Evidence for a different number of latent factors according to market downturns and market upturns, is statistically ambiguous in the considered subperiods. In particular, our results contradicts the common wisdom of a single factor model in bear markets.

categories: econ.EM q-fin.PR q-fin.ST stat.AP stat.ME


title: Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Models

abstract: Given the strong results of self-supervised models on various tasks, there have been surprisingly few studies exploring self-supervised representations for acoustic word embeddings (AWE), fixed-dimensional vectors representing variable-length spoken word segments. In this work, we study several pre-trained models and pooling methods for constructing AWEs with self-supervised representations. Owing to the contextualized nature of self-supervised representations, we hypothesize that simple pooling methods, such as averaging, might already be useful for constructing AWEs. When evaluating on a standard word discrimination task, we find that HuBERT representations with mean-pooling rival the state of the art on English AWEs. More surprisingly, despite being trained only on English, HuBERT representations evaluated on Xitsonga, Mandarin, and French consistently outperform the multilingual model XLSR-53 (as well as Wav2Vec 2.0 trained on English).


title: Sequence entropy for amenable group actions

abstract: We study the sequence entropy for amenable group actions and investigate systematically spectrum and several mixing concepts via sequence entropy both in measure-theoretic dynamical systems and topological dynamical systems. Moreover, we use sequence entropy pairs to characterize weakly mixing and null systems.


title: Towards zero-shot Text-based voice editing using acoustic context conditioning, utterance embeddings, and reference encoders

abstract: Text-based voice editing (TBVE) uses synthetic output from text-to-speech (TTS) systems to replace words in an original recording. Recent work has used neural models to produce edited speech that is similar to the original speech in terms of clarity, speaker identity, and prosody. However, one limitation of prior work is the usage of finetuning to optimise performance: this requires further model training on data from the target speaker, which is a costly process that may incorporate potentially sensitive data into server-side models. In contrast, this work focuses on the zero-shot approach which avoids finetuning altogether, and instead uses pretrained speaker verification embeddings together with a jointly trained reference encoder to encode utterance-level information that helps capture aspects such as speaker identity and prosody. Subjective listening tests find that both utterance embeddings and a reference encoder improve the continuity of speaker identity and prosody between the edited synthetic speech and unedited original recording in the zero-shot setting.


title: Rawgment: Noise-Accounted RAW Augmentation Enables Recognition in a Wide Variety of Environments

abstract: Image recognition models that work in challenging environments (e.g., extremely dark, blurry, or high dynamic range conditions) must be useful. However, creating training datasets for such environments is expensive and hard due to the difficulties of data collection and annotation. It is desirable if we could get a robust model without the need for hard-to-obtain datasets. One simple approach is to apply data augmentation such as color jitter and blur to standard RGB (sRGB) images in simple scenes. Unfortunately, this approach struggles to yield realistic images in terms of pixel intensity and noise distribution due to not considering the non-linearity of Image Signal Processors (ISPs) and noise characteristics of image sensors. Instead, we propose a noise-accounted RAW image augmentation method. In essence, color jitter and blur augmentation are applied to a RAW image before applying non-linear ISP, resulting in realistic intensity. Furthermore, we introduce a noise amount alignment method that calibrates the domain gap in the noise property caused by the augmentation. We show that our proposed noise-accounted RAW augmentation method doubles the image recognition accuracy in challenging environments only with simple training data.


title: A full quantum mechanical approach assessing the chemical and electromagnetic effect in TERS

abstract: Tip-enhanced Raman spectroscopy (TERS) is a valuable method for surface analysis with nanometer to angstrom-scale resolution, however, the accurate simulation of particular TERS signals remains a computational challenge. We present a unique approach to this challenge by combining the two main contributors to plasmon-enhanced Raman spectroscopy and to the high resolution in TERS in particular, the electromagnetic and the chemical effect, into one quantum mechanical simulation. The electromagnetic effect describes the sample's interaction with the strong, highly localized and inhomogeneous electric fields associated with the plasmonic tip, and is typically the thematic focus for most mechanistic studies. On the other hand, the chemical effect covers the different responses to the extremely close-range and highly position-sensitive chemical interaction between the apex tip atom(s) and the sample, and, as we could show in previous works, plays an often underestimated role. Starting from a (time-dependent) density functional theory description of the chemical model system, comprised of a tin(II) phthalocyanine (SnPc) sample molecule and a single silver atom as tip, we introduce the electromagnetic effect through a series of point charges that recreate the electric field in the vicinity of the plasmonic Ag nanoparticle. By scanning the tip over the molecule along a 3D grid, we can investigate the system's Raman response on each position for non-resonant and resonant illumination. Simulating both effects on their own already hints at the achievable signal enhancement and resolution, but the combination of both creates even stronger evidence that TERS is capable of resolving sub-molecular features.


title: Next-to-leading order perturbative QCD predictions for exclusive $J/\psi$ photoproduction in oxygen-oxygen and lead-lead collisions at the LHC

abstract: We present predictions for the cross sections of coherent $J/\psi$ photoproduction in lead-lead and oxygen-oxygen ultraperipheral collisions (UPCs) as a function of the $J/\psi$ rapidity at the LHC in the framework of collinear factorization at next-to-leading order (NLO) in perturbative QCD. Taking generalized parton distribution functions in their forward limit and using the EPPS21, nNNPDF3.0, and nCTEQ15WZSIH nuclear parton distribution functions, we update our recent results for Pb-Pb collisions, make detailed predictions for O-O collisions for several beam energy configurations, and examine the ratio of O-O and Pb-Pb UPC cross sections. We show that the latter observable allows one to significantly reduce the scale uncertainty of NLO predictions for this process.


title: Measuring the Confidence of Traffic Forecasting Models: Techniques, Experimental Comparison and Guidelines towards Their Actionability

abstract: The estimation of the amount of uncertainty featured by predictive machine learning models has acquired a great momentum in recent years. Uncertainty estimation provides the user with augmented information about the model's confidence in its predicted outcome. Despite the inherent utility of this information for the trustworthiness of the user, there is a thin consensus around the different types of uncertainty that one can gauge in machine learning models and the suitability of different techniques that can be used to quantify the uncertainty of a specific model. This subject is mostly non existent within the traffic modeling domain, even though the measurement of the confidence associated to traffic forecasts can favor significantly their actionability in practical traffic management systems. This work aims to cover this lack of research by reviewing different techniques and metrics of uncertainty available in the literature, and by critically discussing how confidence levels computed for traffic forecasting models can be helpful for researchers and practitioners working in this research area. To shed light with empirical evidence, this critical discussion is further informed by experimental results produced by different uncertainty estimation techniques over real traffic data collected in Madrid (Spain), rendering a general overview of the benefits and caveats of every technique, how they can be compared to each other, and how the measured uncertainty decreases depending on the amount, quality and diversity of data used to produce the forecasts.


title: Link Climate: An Interoperable Knowledge Graph Platform for Climate Data

abstract: Climate science has become more ambitious in recent years as global awareness about the environment has grown. To better understand climate, historical climate (e.g. archived meteorological variables such as temperature, wind, water, etc.) and climate-related data (e.g. geographical features and human activities) are widely used by today's climate research to derive models for an explainable climate change and its effects. However, such data sources are often dispersed across a multitude of disconnected data silos on the Web. Moreover, there is a lack of advanced climate data platforms to enable multi-source heterogeneous climate data analysis, therefore, researchers must face a stern challenge in collecting and analyzing multi-source data. In this paper, we address this problem by proposing a climate knowledge graph for the integration of multiple climate data and other data sources into one service, leveraging Web technologies (e.g. HTTP) for multi-source climate data analysis. The proposed knowledge graph is primarily composed of data from the National Oceanic and Atmospheric Administration's daily climate summaries, OpenStreetMap, and Wikidata, and it supports joint data queries on these widely used databases. This paper shows, with a use case in Ireland and the United Kingdom, how climate researchers could benefit from this platform as it allows them to easily integrate datasets from different domains and geographical locations.


title: Fuzzy Logic Model for Predicting the Heat Index

abstract: A fuzzy inference system was developed for predicting the heat index from temperature and relative humidity data. The effectiveness of fuzzy logic in using imprecise mapping of input to output to encode interconnectedness of system variables was exploited to uncover a linguistic model of how the temperature and humidity conditions impact the heat index in a growth room. The developed model achieved an R2 of 0.974 and a RMSE of 0.084 when evaluated on a test set, and the results were statistically significant (F1,5915 = 222900.858, p < 0.001). By providing the advantage of linguistic summarization of data trends as well as high prediction accuracy, the fuzzy logic model proved to be an effective machine learning method for heat control problems.


title: Non-perturbative computation of lattice correlation functions by differential equations

abstract: We show that methods developed in the context of perturbative calculations can be transferred to non-perturbative calculations. We demonstrate that correlation functions on the lattice can be computed with the method of differential equations, supplemented with techniques from twisted cohomology. We derive differential equations for the variation with the coupling or -- more generally -- with the parameters of the action. Already simple examples show that the differential equation with respect to the coupling has an essential singularity at zero coupling and a regular singularity at infinite coupling. The properties of the differential equation at zero coupling can be used to prove that the perturbative series is only an asymptotic series.


title: Automated analysis of diabetic retinopathy using vessel segmentation maps as inductive bias

abstract: Recent studies suggest that early stages of diabetic retinopathy (DR) can be diagnosed by monitoring vascular changes in the deep vascular complex. In this work, we investigate a novel method for automated DR grading based on optical coherence tomography angiography (OCTA) images. Our work combines OCTA scans with their vessel segmentations, which then serve as inputs to task specific networks for lesion segmentation, image quality assessment and DR grading. For this, we generate synthetic OCTA images to train a segmentation network that can be directly applied on real OCTA data. We test our approach on MICCAI 2022's DR analysis challenge (DRAC). In our experiments, the proposed method performs equally well as the baseline model.


title: Determination of zeta-potential of nanofluids based on electrolyte solutions from the measurements by the methods of electrical spectroscopy and laser correlation spectroscopy

abstract: The work discusses the problem of measurement of the zeta-potential for electrolyte-based suspensions of nanoparticle. A theory is presented for the effect of the diffuse electric double layer, including the interphase (stagnant) layer, on the effective conductivity of such suspensions. The theory is based on the method of compact groups of inhomogeneities applied to a model system of hard-core-penetrable-shell particles embedded together with the base liquid in a uniform host of the conductivity. The cores represent the particles. The shells are electrically inhomogeneous in the radial direction, their conductivity profile being a continuous function. This model is possible to analyze rigorously in the static limit. The desired conductivity is found from the integral relation which allows us to express the electrical conductivity in the terms of the zeta-potential, thickness of the stagnant layer, matrix molarity, etc. The theory reveals the existence of different scenarios of behavior of the conductivity depending on the geometrical and electrical parameters of the stagnant layer. It is shown that the functional dependence between the thickness and zeta-potential can be obtained from the rate of change of the conductivity with concentration for diluted suspensions; this rate can be measured experimentally. It is pointed out that the hydrodynamic radius of a nanoparticle can be obtained from the Smoluchowski-Einstein diffusion coefficient, which can be measured by the method of laser correlation spectroscopy. In order to find the zeta-potential, it is necessary to simultaneously do a series of independent measurements: find the slope of the conductivity and estimate the thickness of the stagnant layer.


title: Pressure-dependent semiconductor-metal transition and elastic, electronic, optical, and thermophysical properties of SnS binary chalcogenide

abstract: Density functional theory based study of the pressure dependent physical properties of binary SnS compound has been carried out. The computed elastic constants reveal that SnS is mechanically stable and brittle under ambient conditions. With increasing pressure, the compound becomes ductile. The Poisson's ratio also indicates brittle-ductile transition with increasing pressure. The hardness of SnS increases significantly with pressure. The compound possesses elastic anisotropy. The ground state electronic band structure is semiconducting with a small band gap which becomes metallic under pressure. The band becomes more and more dispersive with the increase in pressure while the electronic correlations decrease as pressure is raised. Both the Debye temperature and the phonon thermal conductivity of SnS increase sharply with pressure. The Melting temperature of the compound is low. Mixed bonding characteristics are found with ionic and covalent contributions. SnS is a good absorber of ultraviolet light. The reflectivity of the material increases with the increase in pressure. The reflectivity is nonselective over a wide spectral range. The low energy refractive index is high. All these optical characteristics are useful for prospective optoelectronic device applications. The optical anisotropy is low.


title: MagicMix: Semantic Mixing with Diffusion Models

abstract: Have you ever imagined what a corgi-alike coffee machine or a tiger-alike rabbit would look like? In this work, we attempt to answer these questions by exploring a new task called semantic mixing, aiming at blending two different semantics to create a new concept (e.g., corgi + coffee machine -- > corgi-alike coffee machine). Unlike style transfer, where an image is stylized according to the reference style without changing the image content, semantic blending mixes two different concepts in a semantic manner to synthesize a novel concept while preserving the spatial layout and geometry. To this end, we present MagicMix, a simple yet effective solution based on pre-trained text-conditioned diffusion models. Motivated by the progressive generation property of diffusion models where layout/shape emerges at early denoising steps while semantically meaningful details appear at later steps during the denoising process, our method first obtains a coarse layout (either by corrupting an image or denoising from a pure Gaussian noise given a text prompt), followed by injection of conditional prompt for semantic mixing. Our method does not require any spatial mask or re-training, yet is able to synthesize novel objects with high fidelity. To improve the mixing quality, we further devise two simple strategies to provide better control and flexibility over the synthesized content. With our method, we present our results over diverse downstream applications, including semantic style transfer, novel object synthesis, breed mixing, and concept removal, demonstrating the flexibility of our method. More results can be found on the project page https://magicmix.github.io


title: Semi-UFormer: Semi-supervised Uncertainty-aware Transformer for Image Dehazing

abstract: Image dehazing is fundamental yet not well-solved in computer vision. Most cutting-edge models are trained in synthetic data, leading to the poor performance on real-world hazy scenarios. Besides, they commonly give deterministic dehazed images while neglecting to mine their uncertainty. To bridge the domain gap and enhance the dehazing performance, we propose a novel semi-supervised uncertainty-aware transformer network, called Semi-UFormer. Semi-UFormer can well leverage both the real-world hazy images and their uncertainty guidance information. Specifically, Semi-UFormer builds itself on the knowledge distillation framework. Such teacher-student networks effectively absorb real-world haze information for quality dehazing. Furthermore, an uncertainty estimation block is introduced into the model to estimate the pixel uncertainty representations, which is then used as a guidance signal to help the student network produce haze-free images more accurately. Extensive experiments demonstrate that Semi-UFormer generalizes well from synthetic to real-world images.


title: Goal Exploration Augmentation via Pre-trained Skills for Sparse-Reward Long-Horizon Goal-Conditioned Reinforcement Learning

abstract: Reinforcement learning (RL) often struggles to accomplish a sparse-reward long-horizon task in a complex environment. Goal-conditioned reinforcement learning (GCRL) has been employed to tackle this difficult problem via a curriculum of easy-to-reach sub-goals. In GCRL, exploring novel sub-goals is essential for the agent to ultimately find the pathway to the desired goal. How to explore novel sub-goals efficiently is one of the most challenging issues in GCRL. Several goal exploration methods have been proposed to address this issue but still struggle to find the desired goals efficiently. In this paper, we propose a novel learning objective by optimizing the entropy of both achieved and new goals to be explored for more efficient goal exploration in sub-goal selection based GCRL. To optimize this objective, we first explore and exploit the frequently occurring goal-transition patterns mined in the environments similar to the current task to compose skills via skill learning. Then, the pretrained skills are applied in goal exploration. Evaluation on a variety of spare-reward long-horizon benchmark tasks suggests that incorporating our method into several state-of-the-art GCRL baselines significantly boosts their exploration efficiency while improving or maintaining their performance. The source code is available at: https://github.com/GEAPS/GEAPS.


title: An Artificial Intelligence driven Learning Analytics Method to Examine the Collaborative Problem solving Process from a Complex Adaptive Systems Perspective

abstract: Collaborative problem solving (CPS) enables student groups to complete learning tasks, construct knowledge, and solve problems. Previous research has argued the importance to examine the complexity of CPS, including its multimodality, dynamics, and synergy from the complex adaptive systems perspective. However, there is limited empirical research examining the adaptive and temporal characteristics of CPS which might lead to an oversimplified representation of the real complexity of the CPS process. To further understand the nature of CPS in online interaction settings, this research collected multimodal process and performance data (i.e., verbal audios, computer screen recordings, concept map data) and proposed a three-layered analytical framework that integrated AI algorithms with learning analytics to analyze the regularity of groups collaboration patterns. The results detected three types of collaborative patterns in groups, namely the behaviour-oriented collaborative pattern (Type 1) associated with medium-level performance, the communication - behaviour - synergistic collaborative pattern (Type 2) associated with high-level performance, and the communication-oriented collaborative pattern (Type 3) associated with low-level performance. The research further highlighted the multimodal, dynamic, and synergistic characteristics of groups collaborative patterns to explain the emergence of an adaptive, self-organizing system during the CPS process.


title: Deep network series for large-scale high-dynamic range imaging

abstract: We propose a new approach for large-scale high-dynamic range computational imaging. Deep Neural Networks (DNNs) trained end-to-end can solve linear inverse imaging problems almost instantaneously. While unfolded architectures provide robustness to measurement setting variations, embedding large-scale measurement operators in DNN architectures is impractical. Alternative Plug-and-Play (PnP) approaches, where the denoising DNNs are blind to the measurement setting, have proven effective to address scalability and high-dynamic range challenges, but rely on highly iterative algorithms. We propose a residual DNN series approach, also interpretable as a learned version of matching pursuit, where the reconstructed image is a sum of residual images progressively increasing the dynamic range, and estimated iteratively by DNNs taking the back-projected data residual of the previous iteration as input. We demonstrate on radio-astronomical imaging simulations that a series of only few terms provides a reconstruction quality competitive with PnP, at a fraction of the cost.


title: Measuring the orbit shrinkage rate of hot Jupiters due to tides

abstract: A tidal interaction between a star and a close-in exoplanet leads to shrinkage of the planetary orbit and eventual tidal disruption of the planet. Measuring the shrinkage of the orbits will allow for the tidal quality parameter of the star ($Q'_\star$) to be measured, which is an important parameter to obtain information about stellar interiors. We analyse data from TESS for two targets known to host close-in hot Jupiters, WASP-18 and WASP-19, to measure the current limits on orbital period variation and provide new constrains on $Q'_\star$. We modelled the transit shape using all the available TESS observations and fitted the individual transit times of each transit. We used previously published transit times together with our results to fit two models, a constant period model, and a quadratic orbital decay model, MCMC algorithms. We find period change rates of $(-0.11\pm0.21)\times10^{-10}$ for WASP-18b and $(-0.35\pm0.22)\times10^{-10}$ for WASP-19b and we do not find significant evidence of orbital decay in these targets. We obtain new lower limits for $Q'_\star$ of $(1.42\pm0.34)\times10^7$ in WASP-18 and $(1.26\pm0.10)\times10^6$ in WASP-19, corresponding to upper limits of the orbital decay rate of $-0.45\times10^{-10}$ and $-0.71\times10^{-10}$, respectively, with a 95% confidence level. We compare our results with other relevant targets for tidal decay studies. We find that the orbital decay rate in both WASP-18b and WASP-19b appears to be smaller than the measured orbital decay of WASP-12b. We show that the minimum value of $Q'_\star$ in WASP-18 is two orders of magnitude higher than that of WASP-12, while WASP-19 has a minimum value one order of magnitude higher, which is consistent with other similar targets. Further observations are required to constrain the orbital decay of WASP-18 and WASP-19.


title: Neural Network based Formation of Cognitive Maps of Semantic Spaces and the Emergence of Abstract Concepts

abstract: The hippocampal-entorhinal complex plays a major role in the organization of memory and thought. The formation of and navigation in cognitive maps of arbitrary mental spaces via place and grid cells can serve as a representation of memories and experiences and their relations to each other. The multi-scale successor representation is proposed to be the mathematical principle underlying place and grid cell computations. Here, we present a neural network, which learns a cognitive map of a semantic space based on 32 different animal species encoded as feature vectors. The neural network successfully learns the similarities between different animal species, and constructs a cognitive map of 'animal space' based on the principle of successor representations with an accuracy of around 30% which is near to the theoretical maximum regarding the fact that all animal species have more than one possible successor, i.e. nearest neighbor in feature space. Furthermore, a hierarchical structure, i.e. different scales of cognitive maps, can be modeled based on multi-scale successor representations. We find that, in fine-grained cognitive maps, the animal vectors are evenly distributed in feature space. In contrast, in coarse-grained maps, animal vectors are highly clustered according to their biological class, i.e. amphibians, mammals and insects. This could be a possible mechanism explaining the emergence of new abstract semantic concepts. Finally, even completely new or incomplete input can be represented by interpolation of the representations from the cognitive map with remarkable high accuracy of up to 95%. We conclude that the successor representation can serve as a weighted pointer to past memories and experiences, and may therefore be a crucial building block for future machine learning to include prior knowledge, and to derive context knowledge from novel input.


title: Defense Against Smart Invaders with Swarms of Sweeping Agents

abstract: The goal of this research is to devise guaranteed defense policies that allow to protect a given region from the entrance of smart mobile invaders by detecting them using a team of defending agents equipped with identical line sensors. By designing cooperative defense strategies that ensure all invaders are detected, conditions on the defenders' speed are derived. Successful accomplishment of the defense task implies invaders with a known limit on their speed cannot slip past the defenders and enter the guarded region undetected. The desired outcome of the defense protocols is to defend the area and additionally to expand it as much as possible. Expansion becomes possible if the defenders' speed exceeds a critical speed that is necessary to only defend the initial region. We present results on the total search time, critical speeds and maximal expansion possible for two types of novel pincer-movement defense processes, circular and spiral, for any even number of defenders. The proposed spiral process allows to detect invaders at nearly the lowest theoretically optimal speed, and if this speed is exceeded, it also allows to expand the protected region almost to the maximal area.


title: DORE: Document Ordered Relation Extraction based on Generative Framework

abstract: In recent years, there is a surge of generation-based information extraction work, which allows a more direct use of pre-trained language models and efficiently captures output dependencies. However, previous generative methods using lexical representation do not naturally fit document-level relation extraction (DocRE) where there are multiple entities and relational facts. In this paper, we investigate the root cause of the underwhelming performance of the existing generative DocRE models and discover that the culprit is the inadequacy of the training paradigm, instead of the capacities of the models. We propose to generate a symbolic and ordered sequence from the relation matrix which is deterministic and easier for model to learn. Moreover, we design a parallel row generation method to process overlong target sequences. Besides, we introduce several negative sampling strategies to improve the performance with balanced signals. Experimental results on four datasets show that our proposed method can improve the performance of the generative DocRE models. We have released our code at https://github.com/ayyyq/DORE.


title: Polarization and reliability of news sources in Wikipedia

abstract: Wikipedia is the largest online encyclopedia: its open contribution policy allows everyone to edit and share their knowledge. A challenge of radical openness is that it facilitates introducing biased contents or perspectives in Wikipedia. Wikipedia relies on numerous external sources such as journal articles, books, news media, and more. News media sources, in particular, take up nearly third of all citations from Wikipedia. However, despite their importance for providing up-to-date and factual contents, there is still a limited understanding on which news media sources are cited from Wikipedia. Relying on a large-scale open dataset of nearly 30M citations from English Wikipedia, we find a moderate yet systematic liberal polarization in the selection of news media sources. We also show that this effect is not mitigated by controlling for news media factual reliability. Our results contribute to Wikipedia's knowledge integrity agenda in suggesting that a systematic effort would help to better map potential biases in Wikipedia and find means to strengthen its neutral point of view policy.


title: Time-dependent Monte Carlo continuum radiative transfer

abstract: Aims. We present an implementation of an algorithm for 3D time-dependent Monte Carlo radiative transfer. It allows one to simulate temperature distributions as well as images and spectral energy distributions of the scattered light and thermal reemission radiation for variable illuminating and heating sources embedded in dust distributions, such as circumstellar disks and dust shells on time scales up to weeks. Methods. We extended the publicly available 3D Monte Carlo radiative transfer code POLARIS with efficient methods for the simulation of temperature distributions, scattering, and thermal reemission of dust distributions illuminated by temporally variable radiation sources. The influence of the chosen temporal step width and the number of photon packages per time step as key parameters for a given configuration is shown by simulating the temperature distribution in a spherical envelope around an embedded central star. The effect of the optical depth on the temperature simulation is discussed for the spherical envelope as well as for a model of a circumstellar disk with an embedded star. Finally, we present simulations of an outburst of a star surrounded by a circumstellar disk. Results. The presented algorithm for time-dependent 3D continuum Monte Carlo radiative transfer is a valuable basis for preparatory studies as well as for the analysis of continuum observations of the dusty environment around variable sources, such as accreting young stellar objects. In particular, the combined study of light echos in the optical and near-infrared wavelength range and the corresponding time-dependent thermal reemission observables of variable, for example outbursting sources, becomes possible on all involved spatial scales.

categories: astro-ph.IM astro-ph.EP astro-ph.GA astro-ph.SR


title: Islands of shape coexistence from single-particle spectra in covariant density functional theory

abstract: Using covariant density functional theory with the DDME2 functional and labeling single-particle energy orbitals by Nilsson quantum numbers, a search for particle-hole (p-h) excitations connected to the appearance of shape coexistence is performed for Z=38 to 84. Islands of shape coexistence are found near the magic numbers Z=82 and Z=50, restricted in regions around the relevant neutron midshells N=104 and N=66 respectively, in accordance to the well accepted p-h interpretation of shape coexistence in these regions, which we call neutron-induced shape coexistence, since the neutrons act as elevators creating holes in the proton orbitals. Similar but smaller islands of shape coexistence are found near N=90 and N=60, restricted in regions around the relevant proton midshells Z=66 and Z=39 respectively, related to p-h excitations across the 3-dimensional isotropic harmonic oscillator (3D-HO) magic numbers N=112 and N=70, which correspond to the beginning of the participation of the opposite parity orbitals 1i13/2 and 1h11/2 respectively to the onset of deformation. We call this case proton-induced shape coexistence, since the protons act as elevators creating holes in the neutron orbitals, thus offering a possible microscopic mechanism for the appearance of shape coexistence in these regions. In the region around N=40, Z=40, an island is located on which both neutron p-h excitations and proton p-h excitations are present.


title: Using Supervised Deep-Learning to Model Edge-FBG Shape Sensors

abstract: Continuum robots in robot-assisted minimally invasive surgeries provide adequate access to target anatomies that are not directly reachable through small incisions. Achieving precise and reliable motion control of such snake-like manipulators necessitates an accurate navigation system that requires no line-of-sight and is immune to electromagnetic noises. Fiber Bragg Grating (FBG) shape sensors, particularly edge-FBGs, are promising tools for this task. However, in edge-FBG sensors, the intensity ratio between Bragg wavelengths carries the strain information that can be affected by undesired bending-related phenomena, making standard characterization techniques less suitable for these sensors. We showed in our previous work that a deep learning model has the potential to extract the strain information from the full edge-FBG spectrum and accurately predict the sensor's shape. In this paper, we conduct a more thorough investigation to find a suitable architectural design with lower prediction errors. We use the Hyperband algorithm to search for optimal hyperparameters in two steps. First, we limit the search space to layer settings, where the best-performing configuration gets selected. Then, we modify the search space for tuning the training and loss calculation hyperparameters. We also analyze various data transformations on the input and output variables, as data rescaling can directly influence the model's performance. Moreover, we performed discriminative training using Siamese network architecture that employs two CNNs with identical parameters to learn similarity metrics between the spectra of similar target values. The best-performing network architecture among all evaluated configurations can predict the sensor's shape with a median tip error of 3.11 mm.


title: Entropy functions for accelerating black holes

abstract: We introduce an entropy function for supersymmetric accelerating black holes in $AdS_4$, that uplift on general Sasaki-Einstein manifolds $X_7$ to solutions of M-theory. This allows one to compute the black hole entropy without knowing the explicit solutions. A dual holographic microstate counting would follow from computing certain supersymmetric partition functions of Chern-Simons-matter theories compactified on a spindle. We make a general prediction for a class of such partition functions in terms of ``blocks'', with each block being constructed from the partition function on a three-sphere.


title: Ethereum Proof-of-Stake under Scrutiny

abstract: Ethereum has undergone a recent change called \textit{the Merge}, which made Ethereum a Proof-of-Stake blockchain, shifting closer to BFT consensus. Ethereum, which wished to keep the best of the two protocol designs (BFT and Nakomoto-style), now has a convoluted consensus protocol as its core. The result is a blockchain being possibly produced in a tree-like form while participants try to finalize blocks. We categorize different attacks jeopardizing the liveness of the protocol. The Ethereum community has responded by creating patches against some of them. We discovered a new attack on the patched protocol. To support our analysis, we propose a new high-level formalization of the properties of liveness and availability of the Ethereum blockchain, and we provide a pseudo-code. We believe this formalization to be helpful for other analyses as well. Our results yield that the Ethereum Proof-of-Stake has safety but only probabilistic liveness. The probability of the liveness is influenced by the parameter describing the time frame allowed for validators to change their mind about the current main chain.


title: A Rosenbrock framework for tangential interpolation of port-Hamiltonian descriptor systems

abstract: We present a new structure-preserving model order reduction (MOR) framework for large-scale port-Hamiltonian descriptor systems (pH-DAEs). Our method exploits the structural properties of the Rosenbrock system matrix for this system class and utilizes condensed forms which often arise in applications and reveal the solution behaviour of a system. Provided that the original system has such a form, our method produces reduced-order models (ROMs) of minimal dimension, which tangentially interpolate the original model's transfer function and are guaranteed to be again in pH-DAE form. This allows the ROM to be safely coupled with other dynamical systems when modelling large system networks, which is useful, for instance, in electric circuit simulation.


title: Form Factors in Asymptotically Safe Quantum Gravity

abstract: Asymptotic Safety implies that observables including scattering amplitudes remain finite at the highest energy scales. Traditionally, this feature is connected to an interacting fixed point of the Wilsonian renormalization group that provides the high-energy completion of the theory. In this article, we discuss a different facet of Asymptotic Safety, reviewing its imprint on the quantum effective action. We start with a concise introduction to the form factor framework for gravity and gravity-matter systems, before giving an encyclopaedic overview of gravity-mediated scattering amplitudes derived from the quantum effective action. We illustrate the use of the framework based on form factors appearing in the context of quadratic gravity and Asymptotic Safety, before making the connections to positivity bounds derived for low-energy effective field theories and the computation of form factors from first principles. We conclude that the form factor framework offers a unique, unifying perspective on quantum gravity. In particular, it plays a pivotal role in determining the phenomenological consequences of Asymptotic Safety at the level of observables.


title: Additive and multiplicative Gower's Ramsey theorem

abstract: W. T. Gower generalized Hindman's Finite sum theorem over $X_{k}=\left\{ \left(n_{1},n_{2},\ldots,n_{k}\right):n_{1}\neq0\right\} $ by showing that for any finite coloring of $X_{k}$ there exists a sequence such that the Gower subspace generated by that sequence is monochromatic. For $k=1,$ this immediately gives the finite sum theorem. In this article we will show that for any finite coloring of $X_{k}$ there exist two sequences $\left\{ \mathbf{n_{i}}:i\in I\right\} $ and $\left\{ \mathbf{m_{i}}:i\in I\right\} $ such that the Gower subspace generated by $\left\{ \mathbf{n_{i}}:i\in I\right\} $ and set of all finite products of $\left\{ \mathbf{m_{i}}:i\in I\right\} $ are in a single color. This immediately generalize a result of V. Bergelson and N. Hindman which says that for any finite coloring of $\mathbb{N}$, there exist two sequences $\left(x_{n}\right)_{n}$ and $\left(y_{n}\right)_{n}$ such that the finite sum and product generated by $\left(x_{n}\right)_{n}$ and $\left(y_{n}\right)_{n}$ are in a same color.


title: Improving Chest X-Ray Classification by RNN-based Patient Monitoring

abstract: Chest X-Ray imaging is one of the most common radiological tools for detection of various pathologies related to the chest area and lung function. In a clinical setting, automated assessment of chest radiographs has the potential of assisting physicians in their decision making process and optimize clinical workflows, for example by prioritizing emergency patients. Most work analyzing the potential of machine learning models to classify chest X-ray images focuses on vision methods processing and predicting pathologies for one image at a time. However, many patients undergo such a procedure multiple times during course of a treatment or during a single hospital stay. The patient history, that is previous images and especially the corresponding diagnosis contain useful information that can aid a classification system in its prediction. In this study, we analyze how information about diagnosis can improve CNN-based image classification models by constructing a novel dataset from the well studied CheXpert dataset of chest X-rays. We show that a model trained on additional patient history information outperforms a model trained without the information by a significant margin. We provide code to replicate the dataset creation and model training.


title: Convergence of Hamiltonian Particle methods for Vlasov--Poisson equations with a nonhomogeneous magnetic field

abstract: In high-temperature plasma physics, a strong magnetic field is usually used to confine charged particles. Therefore, for studying the classical mathematical models of the physical problems it is needed to consider the effect of external magnetic fields. One of the important model equations in plasma is the Vlasov-Poisson equation with an external magnetic field. In this paper, we study the error analysis of Hamiltonian particle methods for this kind of system. The convergence of particle method for Vlasov equation and that of Hamiltonian method for particle equation are provided independently. By combining them, it can be concluded that the numerical solutions converge to the exact particle trajectories.


title: An Efficient Alternating Riemannian/Projected Gradient Descent Ascent Algorithm for Fair Principal Component Analysis

abstract: Fair principal component analysis (FPCA), a ubiquitous dimensionality reduction technique in signal processing and machine learning, aims to find a low-dimensional representation for a high-dimensional dataset in view of fairness. The FPCA problem involves optimizing a non-convex and non-smooth function over the Stiefel manifold. The state-of-the-art methods for solving the problem are subgradient methods and semidefinite relaxation-based methods. However, these two types of methods have their obvious limitations and thus are only suitable for efficiently solving the FPCA problem in special scenarios. This paper aims at developing efficient algorithms for solving the FPCA problem in general, especially large-scale, settings. In this paper, we first transform FPCA into a smooth non-convex linear minimax optimization problem over the Stiefel manifold. To solve the above general problem, we propose an efficient alternating Riemannian/projected gradient descent ascent (ARPGDA) algorithm, which performs a Riemannian gradient descent step and an ordinary projected gradient ascent step at each iteration. We prove that ARPGDA can find an $\varepsilon$-stationary point of the above problem within $\mathcal{O}(\varepsilon^{-3})$ iterations. Simulation results show that, compared with the state-of-the-art methods, our proposed ARPGDA algorithm can achieve a better performance in terms of solution quality and speed for solving the FPCA problems.


title: Scalar fields and Lifshitz black holes from Derrick's theorem evasion

abstract: We study classical scalar fields in asymptotically Lifshitz spacetimes. By evading Derrick's theorem requiring the scalar potential to explicitly depend on the background coordinates, we induce a diffeomorphism invariance breaking and settle a first-order formalism to find spatially localized solutions in the probe field limit. By inserting a backreaction equipped with a cosmological constant and a Maxwell term coupled to a dielectric function in the model, we find a Lifshitz black hole solution.


title: Adaptive Mask-based Pyramid Network for Realistic Bokeh Rendering

abstract: Bokeh effect highlights an object (or any part of the image) while blurring the rest of the image, and creates a visually pleasant artistic effect. Due to the sensor-based limitations on mobile devices, machine learning (ML) based bokeh rendering has gained attention as a reliable alternative. In this paper, we focus on several improvements in ML-based bokeh rendering; i) on-device performance with high-resolution images, ii) ability to guide bokeh generation with user-editable masks and iii) ability to produce varying blur strength. To this end, we propose Adaptive Mask-based Pyramid Network (AMPN), which is formed of a Mask-Guided Bokeh Generator (MGBG) block and a Laplacian Pyramid Refinement (LPR) block. MGBG consists of two lightweight networks stacked to each other to generate the bokeh effect, and LPR refines and upsamples the output of MGBG to produce the high-resolution bokeh image. We achieve i) via our lightweight, mobile-friendly design choices, ii) via the stacked-network design of MGBG and the weakly-supervised mask prediction scheme and iii) via manually or automatically editing the intensity values of the mask that guide the bokeh generation. In addition to these features, our results show that AMPN produces competitive or better results compared to existing methods on the EBB! dataset, while being faster and smaller than the alternatives.


title: Debiasing Masks: A New Framework for Shortcut Mitigation in NLU

abstract: Debiasing language models from unwanted behaviors in Natural Language Understanding tasks is a topic with rapidly increasing interest in the NLP community. Spurious statistical correlations in the data allow models to perform shortcuts and avoid uncovering more advanced and desirable linguistic features. A multitude of effective debiasing approaches has been proposed, but flexibility remains a major issue. For the most part, models must be retrained to find a new set of weights with debiased behavior. We propose a new debiasing method in which we identify debiased pruning masks that can be applied to a finetuned model. This enables the selective and conditional application of debiasing behaviors. We assume that bias is caused by a certain subset of weights in the network; our method is, in essence, a mask search to identify and remove biased weights. Our masks show equivalent or superior performance to the standard counterparts, while offering important benefits. Pruning masks can be stored with high efficiency in memory, and it becomes possible to switch among several debiasing behaviors (or revert back to the original biased model) at inference time. Finally, it opens the doors to further research on how biases are acquired by studying the generated masks. For example, we observed that the early layers and attention heads were pruned more aggressively, possibly hinting towards the location in which biases may be encoded.


title: RESUS: Warm-Up Cold Users via Meta-Learning Residual User Preferences in CTR Prediction

abstract: Click-Through Rate (CTR) prediction on cold users is a challenging task in recommender systems. Recent researches have resorted to meta-learning to tackle the cold-user challenge, which either perform few-shot user representation learning or adopt optimization-based meta-learning. However, existing methods suffer from information loss or inefficient optimization process, and they fail to explicitly model global user preference knowledge which is crucial to complement the sparse and insufficient preference information of cold users. In this paper, we propose a novel and efficient approach named RESUS, which decouples the learning of global preference knowledge contributed by collective users from the learning of residual preferences for individual users. Specifically, we employ a shared predictor to infer basis user preferences, which acquires global preference knowledge from the interactions of different users. Meanwhile, we develop two efficient algorithms based on the nearest neighbor and ridge regression predictors, which infer residual user preferences via learning quickly from a few user-specific interactions. Extensive experiments on three public datasets demonstrate that our RESUS approach is efficient and effective in improving CTR prediction accuracy on cold users, compared with various state-of-the-art methods.


title: Object Segmentation of Cluttered Airborne LiDAR Point Clouds

abstract: Airborne topographic LiDAR is an active remote sensing technology that emits near-infrared light to map objects on the Earth's surface. Derived products of LiDAR are suitable to service a wide range of applications because of their rich three-dimensional spatial information and their capacity to obtain multiple returns. However, processing point cloud data still requires a significant effort in manual editing. Certain human-made objects are difficult to detect because of their variety of shapes, irregularly-distributed point clouds, and low number of class samples. In this work, we propose an efficient end-to-end deep learning framework to automatize the detection and segmentation of objects defined by an arbitrary number of LiDAR points surrounded by clutter. Our method is based on a light version of PointNet that achieves good performance on both object recognition and segmentation tasks. The results are tested against manually delineated power transmission towers and show promising accuracy.


title: Optimal Transportation for Electrical Impedance Tomography

abstract: This work establishes a framework for solving inverse boundary problems with the geodesic based quadratic Wasserstein distance ($W_{2}$). A general form of the Fr\'echet gradient is systematically derived by optimal transportation (OT) theory. In addition, a fast algorithm based on the new formulation of OT on $\mathbb{S}^{1}$ is developed to solve the corresponding optimal transport problem. The computational complexity of the algorithm is reduced to $O(N)$ from $O(N^{3})$ of the traditional method. Combining with the adjoint-state method, this framework provides a new computational approach for solving the challenging electrical impedance tomography (EIT) problem. Numerical examples are presented to illustrate the effectiveness of our method.


title: ROMA: Run-Time Object Detection To Maximize Real-Time Accuracy

abstract: This paper analyzes the effects of dynamically varying video contents and detection latency on the real-time detection accuracy of a detector and proposes a new run-time accuracy variation model, ROMA, based on the findings from the analysis. ROMA is designed to select an optimal detector out of a set of detectors in real time without label information to maximize real-time object detection accuracy. ROMA utilizing four YOLOv4 detectors on an NVIDIA Jetson Nano shows real-time accuracy improvements by 4 to 37% for a scenario of dynamically varying video contents and detection latency consisting of MOT17Det and MOT20Det datasets, compared to individual YOLOv4 detectors and two state-of-the-art runtime techniques.


title: Sudakov effects and the dipole amplitude

abstract: In this study we incorporate the Sudakov form factor into the dipole factorization formula, where the hard scale of the former is provided by the photon virtuality $Q^2$. We obtain a general formula which we then apply to the well-known GBW and BGK saturation models. Parameters of the above Sudakov-improved models are successfully fitted to the $F_2$ data from HERA. We observe, in particular, that inclusion of the Sudakov factor on top of the GBW model improves description of data at large $Q^2$.


title: Near-field Localization with Dynamic Metasurface Antennas

abstract: Sixth generation (6G) cellular communications are expected to support enhanced wireless localization capabilities. The widespread deployment of large arrays and high-frequency bandwidths give rise to new considerations for localization applications. First, emerging antenna architectures, such as dynamic metasurface antennas (DMAs), are expected to be frequently utilized thanks to the achievable high angular resolution and low hardware complexity. Further, wireless localization is likely to take place in the radiating near-field (Fresnel) region, which provides new degrees of freedom, because of the adoption of arrays with large apertures. While current studies mostly focus on the use of costly fully-digital antenna arrays, in this paper we investigate how DMAs can be applied for near-field localization of a single user. We use a direct positioning estimation method based on curvature-of-arrival of the impinging wavefront to obtain the user location, and characterize the effects of DMA tuning on the estimation accuracy. Next, we propose an algorithm for configuring the DMA to optimize near-field localization, by first tuning the adjustable DMA coefficients to minimize the estimation error using postulated knowledge of the actual user position. Finally, we propose a sub-optimal iterative algorithm that does not rely on such knowledge. Simulation results show that the DMA-based near-field localization accuracy could approach that of fully-digital arrays at lower cost.


title: KD-EKF: A Consistent Cooperative Localization Estimator Based on Kalman Decomposition

abstract: In this paper, we revisit the inconsistency problem of EKF-based cooperative localization (CL) from the perspective of system decomposition. By transforming the linearized system used by the standard EKF into its Kalman observable canonical form, the observable and unobservable components of the system are separated. Consequently, the factors causing the dimension reduction of the unobservable subspace are explicitly isolated in the state propagation and measurement Jacobians of the Kalman observable canonical form. Motivated by these insights, we propose a new CL algorithm called KD-EKF which aims to enhance consistency. The key idea behind the KD-EKF algorithm involves perform state estimation in the transformed coordinates so as to eliminate the influencing factors of observability in the Kalman observable canonical form. As a result, the KD-EKF algorithm ensures correct observability properties and consistency. We extensively verify the effectiveness of the KD-EKF algorithm through both Monte Carlo simulations and real-world experiments. The results demonstrate that the KD-EKF outperforms state-of-the-art algorithms in terms of accuracy and consistency.


title: Non relativistic string compactifications on Calabi-Yau manifolds

abstract: We study Calabi-Yau compactifications of non relativistic string theory and show that it can be derived from the corresponding relativistic Calabi-Yau compactifications by taking the non relativistic limit of the resulting 4D theory without changing the internal part.


title: Dynamically generated quadrupole polarization using Floquet adiabatic evolution

abstract: We investigate the nonequilibrium dynamics of the $S=1$ quantum spin chain subjected to a time-dependent external drive, where the driving frequency is adiabatically decreased as a function of time (``Floquet adiabatic evolution''). We show that when driving the rhombic anisotropy term (known as the ``two-axis countertwisting'' in the context of squeezed spin states) of a N\'eel antiferromagnet, we are able to induce an overall enhancement in the quadrupole polarization, while at the same time suppressing the staggered magnetization order. The system evolves into a new state with a net quadrupole moment and antiferroquadrupolar correlations. This state remains stable at long times once the driving frequency is kept constant. On the other hand, we find that we cannot achieve a quadrupole polarization for the symmetry-protected Haldane phase, which remains robust against such driving.


title: Let's Go to the Whiteboard (Again):Perceptions from Software Architects on Whiteboard Architecture Meetings

abstract: The whiteboard plays a crucial role in the day-to-day lives of software architects, as they frequently will organize meetings at the whiteboard to discuss a new architecture, some proposed changes to the architecture, a mismatch between the architecture and the code, and more. While much has been studied about software architects, the architectures they produce, and how they produce them, a detailed understanding of these whiteboards meetings is still lacking. In this paper, we contribute a mixed-methods study involving semi-structured interviews and a subsequent survey to understand the perceptions of software architects on whiteboard architecture meetings. We focus on five aspects: (1) why do they hold these meetings, what is the impact of the experience levels of the participants in these meetings, how do the architects document the meetings, what kinds of changes are made after the meetings have concluded and their results are moved to implementation, and what role do digital whiteboards plays? In studying these aspects, we identify 12 observations related to both technical aspects and social aspects of the meetings. These insights have implications for further research, offer concrete advice to practitioners, provide guidance for future tool design, and suggest ways of educating future software architects.


title: Sterile neutrino dark matter in the super-weak model

abstract: The super-weak model is a U(1) extension of the Standard Model. In addition to a mediator $Z'$, a singlet scalar field $\chi$ is added to deal with the meta-stability of the SM vacuum, and right-handed neutrinos are introduced to account for the non-vanishing neutrino masses. We demonstrate that the lightest right-handed neutrino is a possible dark matter candidate with masses of $\mathcal{O}(10)~\mathrm{MeV}$ via the resonant freeze-out scenario. The dark matter production is explored and the available parameter space is discussed from the point of view of various experiments and beyond SM aspects.


title: Oblique Quasi-Kink Modes in Solar Coronal Slabs Embedded in an Asymmetric Magnetic Environment: Resonant Damping, Phase and Group Diagrams

abstract: There has been considerable interest in magnetoacoustic waves in static, straight, field-aligned, one-dimensional equilibria where the exteriors of a magnetic slab are different between the two sides. We focus on trapped, transverse fundamental, oblique quasi-kink modes in pressureless setups where the density varies continuously from a uniform interior (with density $\rho_{\rm i}$) to a uniform exterior on either side (with density $\rho_{\rm L}$ or $\rho_{\rm R}$), assuming $\rho_{\rm L}\le\rho_{\rm R}\le\rho_{\rm i}$. The continuous structuring and oblique propagation make our study new relative to pertinent studies, and lead to wave damping via the Alfv$\acute{\rm e}$n resonance. We compute resonantly damped quasi-kink modes as resistive eigenmodes, and isolate the effects of system asymmetry by varying $\rho_{\rm i}/\rho_{\rm R}$ from the ``Fully Symmetric'' ($\rho_{\rm i}/\rho_{\rm R}=\rho_{\rm i}/\rho_{\rm L}$) to the ``Fully Asymmetric'' limit ($\rho_{\rm i}/\rho_{\rm R}=1$). We find that the damping rates possess a nonmonotonic $\rho_{\rm i}/\rho_{\rm R}$-dependence as a result of the difference between the two Alfv$\acute{\rm e}$n continua, and resonant absorption occurs only in one continuum when $\rho_{\rm i}/\rho_{\rm R}$ is below some threshold. We also find that the system asymmetry results in two qualitatively different regimes for the phase and group diagrams. The phase and group trajectories lie essentially on the same side (different sides) relative to the equilibrium magnetic field when the configuration is not far from a ``Fully Asymmetric'' (``Fully Symmetric'') one. Our numerical results are understood by making analytical progress in the thin-boundary limit, and discussed for imaging observations of axial standing modes and impulsively excited wavetrains.


title: Jahn-Teller-driven Phase Segregation in Mn$_{x}$Co$_{3-x}$O$_{4}$ Spinel Thin Films

abstract: Transition metal spinel oxides comprised of Earth-abundant Mn and Co have long been explored for their use in catalytic reactions and energy storage. However, understanding of functional properties can be challenging due to differences in sample preparation and the ultimate structural properties of the materials. Epitaxial thin film synthesis provides a novel means of producing precisely-controlled materials to explore the variations reported in the literature. In this work, Mn$_{x}$Co$_{3-x}$O$_{4}$ samples from x = 0 to x = 1.28 were synthesized through molecular beam epitaxy and characterized to develop a material properties map as a function of stoichiometry. Films were characterized via in situ X-ray photoelectron spectroscopy, X-ray diffraction, scanning transmission electron microscopy, and polarized K-edge X-ray absorption spectroscopy. Mn cations within this range were found to be octahedrally coordinated, in line with an inverse spinel structure. Samples largely show mixed Mn$^{3+}$ and Mn$^{4+}$ character with evidence of phase segregation tendencies with increasing Mn content and increasing Mn$^{3+}$ formal charge. Phase segregation may occur due to structural incompatibility between cubic and tetragonal crystal structures associated with Mn$^{4+}$ and Jahn-Teller active Mn$^{3+}$ octahedra, respectively. Our results help to explain the reported differences across samples in these promising materials for renewable energy technologies.


title: A CNN-LSTM Combination Network for Cataract Detection using Eye Fundus Images

abstract: According to multiple authoritative authorities, including the World Health Organization, vision-related impairments and disorders are becoming a significant issue. According to a recent report, one of the leading causes of irreversible blindness in persons over the age of 50 is delayed cataract treatment. A cataract is a cloudy spot in the eye's lens that causes visual loss. Cataracts often develop slowly and consequently result in difficulty in driving, reading, and even recognizing faces. This necessitates the development of rapid and dependable diagnosis and treatment solutions for ocular illnesses. Previously, such visual illness diagnosis were done manually, which was time-consuming and prone to human mistake. However, as technology advances, automated, computer-based methods that decrease both time and human labor while producing trustworthy results are now accessible. In this study, we developed a CNN-LSTM-based model architecture with the goal of creating a low-cost diagnostic system that can classify normal and cataractous cases of ocular disease from fundus images. The proposed model was trained on the publicly available ODIR dataset, which included fundus images of patients' left and right eyes. The suggested architecture outperformed previous systems with a state-of-the-art 97.53% accuracy.


title: Constraining the Merger History of Primordial-Black-Hole Binaries from GWTC-3

abstract: Primordial black holes (PBHs) can be not only cold dark matter candidates but also progenitors of binary black holes observed by LIGO-Virgo-KAGRA (LVK) Collaboration. The PBH mass can be shifted to the heavy distribution if multi-merger processes occur. In this work, we constrain the merger history of PBH binaries using the gravitational wave events from the third Gravitational-Wave Transient Catalog (GWTC-3). Considering four commonly used PBH mass functions, namely the log-normal, power-law, broken power-law, and critical collapse forms, we find that the multi-merger processes make a subdominant contribution to the total merger rate. Therefore, the effect of merger history can be safely ignored when estimating the merger rate of PBH binaries. We also find that GWTC-3 is best fitted by the log-normal form among the four PBH mass functions and confirm that the stellar-mass PBHs cannot dominate cold dark matter.


title: Sequential hypothesis testing for Axion Haloscopes

abstract: The goal of this paper is to introduce a novel likelihood-based inferential framework for axion haloscopes which is valid under the commonly applied "rescanning" protocol. The proposed method enjoys short data acquisition times and a simple tuning of the detector configuration. Local statistical significance and power are computed analytically, avoiding the need of burdensome simulations. Adequate corrections for the look-elsewhere effect are also discussed. The performance of our inferential strategy is compared with that of a simple method which exploits the geometric probability of rescan. Finally, we exemplify the method with an application to a HAYSTAC type axion haloscope.

categories: physics.data-an astro-ph.IM hep-ex hep-ph


title: Upgrading the Detection of Electrocatalyst Degradation During the Oxygen Evolution Reaction

abstract: Electrocatalysts for the oxygen evolution reaction (OER) are an important component for the transition from fossil to sustainable energy. Commercialization of cost-effective earth-abundant electrocatalysts is in large parts hindered by their degradation. In this short review, I identify common processes leading to a decrease in electrocatalyst activity, followed by an introduction of staple methods to determine degradation electrochemically and by additional physical characterization, which has the potential to remove ambiguities of purely electrochemical studies. I conclude by a summary of the key challenges for an accurate determination of degradation processes and highlight interesting directions to advance the understanding of degradation processes on electrocatalysts.


title: cRedAnno+: Annotation Exploitation in Self-Explanatory Lung Nodule Diagnosis

abstract: Recently, attempts have been made to reduce annotation requirements in feature-based self-explanatory models for lung nodule diagnosis. As a representative, cRedAnno achieves competitive performance with considerably reduced annotation needs by introducing self-supervised contrastive learning to do unsupervised feature extraction. However, it exhibits unstable performance under scarce annotation conditions. To improve the accuracy and robustness of cRedAnno, we propose an annotation exploitation mechanism by conducting semi-supervised active learning with sparse seeding and training quenching in the learned semantically meaningful reasoning space to jointly utilise the extracted features, annotations, and unlabelled data. The proposed approach achieves comparable or even higher malignancy prediction accuracy with 10x fewer annotations, meanwhile showing better robustness and nodule attribute prediction accuracy under the condition of 1% annotations. Our complete code is open-source available: https://github.com/diku-dk/credanno.


title: Predicting Protein-Ligand Binding Affinity with Equivariant Line Graph Network

abstract: Binding affinity prediction of three-dimensional (3D) protein ligand complexes is critical for drug repositioning and virtual drug screening. Existing approaches transform a 3D protein-ligand complex to a two-dimensional (2D) graph, and then use graph neural networks (GNNs) to predict its binding affinity. However, the node and edge features of the 2D graph are extracted based on invariant local coordinate systems of the 3D complex. As a result, the method can not fully learn the global information of the complex, such as, the physical symmetry and the topological information of bonds. To address these issues, we propose a novel Equivariant Line Graph Network (ELGN) for affinity prediction of 3D protein ligand complexes. The proposed ELGN firstly adds a super node to the 3D complex, and then builds a line graph based on the 3D complex. After that, ELGN uses a new E(3)-equivariant network layer to pass the messages between nodes and edges based on the global coordinate system of the 3D complex. Experimental results on two real datasets demonstrate the effectiveness of ELGN over several state-of-the-art baselines.


title: An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization

abstract: Molecule optimization is an important problem in chemical discovery and has been approached using many techniques, including generative modeling, reinforcement learning, genetic algorithms, and much more. Recent work has also applied zeroth-order (ZO) optimization, a subset of gradient-free optimization that solves problems similarly to gradient-based methods, for optimizing latent vector representations from an autoencoder. In this paper, we study the effectiveness of various ZO optimization methods for optimizing molecular objectives, which are characterized by variable smoothness, infrequent optima, and other challenges. We provide insights on the robustness of various ZO optimizers in this setting, show the advantages of ZO sign-based gradient descent (ZO-signGD), discuss how ZO optimization can be used practically in realistic discovery tasks, and demonstrate the potential effectiveness of ZO optimization methods on widely used benchmark tasks from the Guacamol suite. Code is available at: https://github.com/IBM/QMO-bench.


title: An OSSS-type inequality for uniformly drawn subsets of fixed size

abstract: The OSSS inequality [O'Donnell, Saks, Schramm and Servedio, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05), Pittsburgh (2005)] gives an upper bound for the variance of a function f of independent 0-1 valued random variables, in terms of the influences of these random variables and the computational complexity of a (randomised) algorithm for determining the value of f. Duminil-Copin, Raoufi and Tassion [Annals of Mathematics 189, 75-99 (2019)] obtained a generalization to monotonic measures and used it to prove new results for Potts models and random-cluster models. Their generalization of the OSSS inequality raises the question if there are still other measures for which a version of that inequality holds. We derive a version of the OSSS inequality for a family of measures that are far from monotonic, namely the k-out-of-n measures (these measures correspond with drawing k elements from a set of size n uniformly). We illustrate the inequality by studying the event that there is an occupied horizontal crossing of an R times R box on the triangular lattice in the site percolation model where exactly half of the vertices in the box are occupied.


title: A Generic Shared Attention Mechanism for Various Backbone Neural Networks

abstract: The self-attention mechanism has emerged as a critical component for improving the performance of various backbone neural networks. However, current mainstream approaches individually incorporate newly designed self-attention modules (SAMs) into each layer of the network for granted without fully exploiting their parameters' potential. This leads to suboptimal performance and increased parameter consumption as the network depth increases. To improve this paradigm, in this paper, we first present a counterintuitive but inherent phenomenon: SAMs tend to produce strongly correlated attention maps across different layers, with an average Pearson correlation coefficient of up to 0.85. Inspired by this inherent observation, we propose Dense-and-Implicit Attention (DIA), which directly shares SAMs across layers and employs a long short-term memory module to calibrate and bridge the highly correlated attention maps of different layers, thus improving the parameter utilization efficiency of SAMs. This design of DIA is also consistent with the neural network's dynamical system perspective. Through extensive experiments, we demonstrate that our simple yet effective DIA can consistently enhance various network backbones, including ResNet, Transformer, and UNet, across tasks such as image classification, object detection, and image generation using diffusion models.


title: Modeling the effects of clumpy winds in the high-energy light curves of {\gamma}-ray binaries

abstract: High-mass gamma-ray binaries are powerful nonthermal galactic sources, some of them hosting a pulsar whose relativistic wind interacts with a likely inhomogeneous stellar wind. So far, modeling these sources including stellar wind inhomogeneities has been done using either simple analytical approaches or heavy numerical simulations, none of which allow for an exploration of the parameter space that is both reasonably realistic and general. Applying different semi-analytical tools together, we study the dynamics and high-energy radiation of a pulsar wind colliding with a stellar wind with different degrees of inhomogeneity to assess the related observable effects. We computed the arrival of clumps to the pulsar wind-stellar wind interaction structure using a Monte Carlo method and a phenomenological clumpy-wind model. The dynamics of the clumps that reach deep into the pulsar wind zone was computed using a semi-analytical approach. This approach allows for the characterization of the evolution of the shocked pulsar wind region in times much shorter than the orbital period. With this three-dimensional information about the emitter, we applied analytical adiabatic and radiative models to compute the variable high-energy emission produced on binary scales. An inhomogeneous stellar wind induces stochastic hour-timescale variations in the geometry of the two-wind interaction structure on binary scales. Depending on the degree of stellar wind inhomogeneity, 10-100% level hour-scale variability in the X-rays and gamma rays is predicted, with the largest variations occurring roughly once per orbit. Our results, based on a comprehensive approach, show that present X-ray and future very-high-energy instrumentation can allow us to trace the impact of a clumpy stellar wind on the shocked pulsar wind emission in a gamma-ray binary.