title: Bandwidth-efficient distributed neural network architectures with application to body sensor networks

abstract: In this paper, we describe a conceptual design methodology to design distributed neural network architectures that can perform efficient inference within sensor networks with communication bandwidth constraints. The different sensor channels are distributed across multiple sensor devices, which have to exchange data over bandwidth-limited communication channels to solve, e.g., a classification task. Our design methodology starts from a user-defined centralized neural network and transforms it into a distributed architecture in which the channels are distributed over different nodes. The distributed network consists of two parallel branches of which the outputs are fused at the fusion center. The first branch collects classification results from local, node-specific classifiers while the second branch compresses each node's signal and then reconstructs the multi-channel time series for classification at the fusion center. We further improve bandwidth gains by dynamically activating the compression path when the local classifications do not suffice. We validate this method on a motor execution task in an emulated EEG sensor network and analyze the resulting bandwidth-accuracy trade-offs. Our experiments show that the proposed framework enables up to a factor 20 in bandwidth reduction with minimal loss (up to 2%) in classification accuracy compared to the centralized baseline on the demonstrated motor execution task. The proposed method offers a way to smoothly transform a centralized architecture to a distributed, bandwidth-efficient network amenable for low-power sensor networks. While the application focus of this paper is on wearable brain-computer interfaces, the proposed methodology can be applied in other sensor network-like applications as well.


title: Blind Super-Resolution for Remote Sensing Images via Conditional Stochastic Normalizing Flows

abstract: Remote sensing images (RSIs) in real scenes may be disturbed by multiple factors such as optical blur, undersampling, and additional noise, resulting in complex and diverse degradation models. At present, the mainstream SR algorithms only consider a single and fixed degradation (such as bicubic interpolation) and cannot flexibly handle complex degradations in real scenes. Therefore, designing a super-resolution (SR) model that can cope with various degradations is gradually attracting the attention of researchers. Some studies first estimate the degradation kernels and then perform degradation-adaptive SR but face the problems of estimation error amplification and insufficient high-frequency details in the results. Although blind SR algorithms based on generative adversarial networks (GAN) have greatly improved visual quality, they still suffer from pseudo-texture, mode collapse, and poor training stability. In this article, we propose a novel blind SR framework based on the stochastic normalizing flow (BlindSRSNF) to address the above problems. BlindSRSNF learns the conditional probability distribution over the high-resolution image space given a low-resolution (LR) image by explicitly optimizing the variational bound on the likelihood. BlindSRSNF is easy to train and can generate photo-realistic SR results that outperform GAN-based models. Besides, we introduce a degradation representation strategy based on contrastive learning to avoid the error amplification problem caused by the explicit degradation estimation. Comprehensive experiments show that the proposed algorithm can obtain SR results with excellent visual perception quality on both simulated LR and real-world RSIs.


title: Optimization Studies for the Long-Baseline Neutrino Facility at Fermilab

abstract: The Deep Underground Neutrino Experiment and Long-Baseline Neutrino Facility (DUNE-LBNF) are under development at Fermilab since early 2010s [1]. At present, the work is being performed towards a comprehensive review conducted by US Department of Energy (DOE)-the Critical Decision 2 (CD-2)-that is planned to take place in the middle of 2022. The primary scientific objectives of DUNE are to carry out a comprehensive investigation of neutrino oscillations to test CP violation in the lepton sector, determine the ordering of the neutrino masses, and to test the three-neutrino paradigm (electron, muon and tau neutrino). The LBNF will provide a 120-GeV proton beam on a neutrino production target utilizing a new 800-MeV superconducting Linac which is expected to be completed in 2027 [2]. The neutrino beamline, which utilizes a target and horn systems, decay pipe, hadron absorber and other systems, is a core component of the LBNF. At present-as a result of numerous iterations-there exists an optimized design with a 1.5-m graphite target and focusing system consisting of three horns. Further optimization energy deposition and radiological calculations are performed towards CD-2 and beyond. This paper describes results of the most recent MARS15 [3] optimization studies.


title: A note on coCartesian fibrations

abstract: We prove properness of (co)Cartesian fibrations as well as a straightening and unstraightening equivalence, which is compatible with cartesian products, when the base is the nerve of a small category.


title: Zero-Rate Thresholds and New Capacity Bounds for List-Decoding and List-Recovery

abstract: In this work we consider the list-decodability and list-recoverability of arbitrary $q$-ary codes, for all integer values of $q\geq 2$. A code is called $(p,L)_q$-list-decodable if every radius $pn$ Hamming ball contains less than $L$ codewords; $(p,\ell,L)_q$-list-recoverability is a generalization where we place radius $pn$ Hamming balls on every point of a combinatorial rectangle with side length $\ell$ and again stipulate that there be less than $L$ codewords. Our main contribution is to precisely calculate the maximum value of $p$ for which there exist infinite families of positive rate $(p,\ell,L)_q$-list-recoverable codes, the quantity we call the zero-rate threshold. Denoting this value by $p_*$, we in fact show that codes correcting a $p_*+\varepsilon$ fraction of errors must have size $O_{\varepsilon}(1)$, i.e., independent of $n$. Such a result is typically referred to as a ``Plotkin bound.'' To complement this, a standard random code with expurgation construction shows that there exist positive rate codes correcting a $p_*-\varepsilon$ fraction of errors. We also follow a classical proof template (typically attributed to Elias and Bassalygo) to derive from the zero-rate threshold other tradeoffs between rate and decoding radius for list-decoding and list-recovery. Technically, proving the Plotkin bound boils down to demonstrating the Schur convexity of a certain function defined on the $q$-simplex as well as the convexity of a univariate function derived from it. We remark that an earlier argument claimed similar results for $q$-ary list-decoding; however, we point out that this earlier proof is flawed.


title: Simpson's Paradox in Recommender Fairness: Reconciling differences between per-user and aggregated evaluations

abstract: There has been a flurry of research in recent years on notions of fairness in ranking and recommender systems, particularly on how to evaluate if a recommender allocates exposure equally across groups of relevant items (also known as provider fairness). While this research has laid an important foundation, it gave rise to different approaches depending on whether relevant items are compared per-user/per-query or aggregated across users. Despite both being established and intuitive, we discover that these two notions can lead to opposite conclusions, a form of Simpson's Paradox. We reconcile these notions and show that the tension is due to differences in distributions of users where items are relevant, and break down the important factors of the user's recommendations. Based on this new understanding, practitioners might be interested in either notions, but might face challenges with the per-user metric due to partial observability of the relevance and user satisfaction, typical in real-world recommenders. We describe a technique based on distribution matching to estimate it in such a scenario. We demonstrate on simulated and real-world recommender data the effectiveness and usefulness of such an approach.


title: National-scale bi-directional EV fleet control for ancillary service provision

abstract: Deploying real-time control on large-scale fleets of electric vehicles (EVs) is becoming pivotal as the share of EVs over internal combustion engine vehicles increases. In this paper, we present a Vehicle-to-Grid (V2G) algorithm to simultaneously schedule thousands of EVs charging and discharging operations, that can be used to provide ancillary services. To achieve scalability, the monolithic problem is decomposed using the alternating direction method of multipliers (ADMM). Furthermore, we propose a method to handle bilinear constraints of the original problem inside the ADMM iterations, which changes the problem class from Mixed-Integer Quadratic Program (MIQP) to Quadratic Program (QP), allowing for a substantial computational speed up. We test the algorithm using real data from the largest carsharing company in Switzerland and show how our formulation can be used to retrieve flexibility boundaries for the EV fleet. Our work thus enables fleet operators to make informed bids on ancillary services provision, thereby facilitating the integration of electric vehicles.


title: Transversal transport of magnons in a modified Lieb lattice

abstract: We studied a two-band magnon insulating model whose geometry is that of a modified Lieb lattice in which one of the sites was removed. Anisotropic ferromagnetic exchange interactions exist between the three nearest neighbors, and the anisotropy opens a gap in the magnon energy band structure. A non-vanishing Berry curvature is induced by a Dzyaloshinskii-Moriya interaction (DMI). The topology of the bands is trivial (in the sense of a null Chern number), but the finite Berry curvature induces Hall-like transport effects whose coefficients were calculated. Their dependence on temperature was studied and shows a resemblance with other magnon insulating systems found in the literature. The dependence on exchange couplings, DMI parameter, and external magnetic field was also investigated.


title: A Study and Analysis of Manuscript Publications in the Open Access Journals

abstract: The purpose of this study is to analyze the research article publishing with special reference to preparing to publish and peer reviewing. Peer reviewing is the process required for standardizing any publications. Manuscript writing is an art. Though it appears to be simple there is a lot of effort required. Peer reviewing is the process that eliminates articles that do not meet the standard of the journals and the scope of the journals. The study investigated authors views on manuscript submissions to the publishing process. There are 375 samples selected for this study who have experienced publishing journals listed in refereed journals. For the selection of the sample 50 ScimagoJR Library and Information Science open access journals between 2019 to 2021 are verified by the authors.


title: Counting Statistics of Single Electron Transport in Bilayer Graphene Quantum Dots

abstract: We measure telegraph noise of current fluctuations in an electrostatically defined quantum dot in bilayer graphene by real-time detection of single electron tunneling with a capacitively coupled neighboring quantum dot. Suppression of the second and third cumulant (related to shot noise) in a tunable graphene quantum dot is demonstrated experimentally. With this method we demonstrate the ability to measure very low current and noise levels. Furthermore, we use this method to investigate the first spin excited state, an essential prerequisite to measure spin relaxation.


title: Lightweight Alpha Matting Network Using Distillation-Based Channel Pruning

abstract: Recently, alpha matting has received a lot of attention because of its usefulness in mobile applications such as selfies. Therefore, there has been a demand for a lightweight alpha matting model due to the limited computational resources of commercial portable devices. To this end, we suggest a distillation-based channel pruning method for the alpha matting networks. In the pruning step, we remove channels of a student network having fewer impacts on mimicking the knowledge of a teacher network. Then, the pruned lightweight student network is trained by the same distillation loss. A lightweight alpha matting model from the proposed method outperforms existing lightweight methods. To show superiority of our algorithm, we provide various quantitative and qualitative experiments with in-depth analyses. Furthermore, we demonstrate the versatility of the proposed distillation-based channel pruning method by applying it to semantic segmentation.


title: Improved automated lesion segmentation in whole-body FDG/PET-CT via Test-Time Augmentation

abstract: Numerous oncology indications have extensively quantified metabolically active tumors using positron emission tomography (PET) and computed tomography (CT). F-fluorodeoxyglucose-positron emission tomography (FDG-PET) is frequently utilized in clinical practice and clinical drug research to detect and measure metabolically active malignancies. The assessment of tumor burden using manual or computer-assisted tumor segmentation in FDG-PET images is widespread. Deep learning algorithms have also produced effective solutions in this area. However, there may be a need to improve the performance of a pre-trained deep learning network without the opportunity to modify this network. We investigate the potential benefits of test-time augmentation for segmenting tumors from PET-CT pairings. We applied a new framework of multilevel and multimodal tumor segmentation techniques that can simultaneously consider PET and CT data. In this study, we improve the network using a learnable composition of test time augmentations. We trained U-Net and Swin U-Netr on the training database to determine how different test time augmentation improved segmentation performance. We also developed an algorithm that finds an optimal test time augmentation contribution coefficient set. Using the newly trained U-Net and Swin U-Netr results, we defined an optimal set of coefficients for test-time augmentation and utilized them in combination with a pre-trained fixed nnU-Net. The ultimate idea is to improve performance at the time of testing when the model is fixed. Averaging the predictions with varying ratios on the augmented data can improve prediction accuracy. Our code will be available at \url{https://github.com/sepidehamiri/pet\_seg\_unet}


title: Controllable Style Transfer via Test-time Training of Implicit Neural Representation

abstract: We propose a controllable style transfer framework based on Implicit Neural Representation that pixel-wisely controls the stylized output via test-time training. Unlike traditional image optimization methods that often suffer from unstable convergence and learning-based methods that require intensive training and have limited generalization ability, we present a model optimization framework that optimizes the neural networks during test-time with explicit loss functions for style transfer. After being test-time trained once, thanks to the flexibility of the INR-based model, our framework can precisely control the stylized images in a pixel-wise manner and freely adjust image resolution without further optimization or training. We demonstrate several applications.


title: Extracting Cultural Commonsense Knowledge at Scale

abstract: Structured knowledge is important for many AI applications. Commonsense knowledge, which is crucial for robust human-centric AI, is covered by a small number of structured knowledge projects. However, they lack knowledge about human traits and behaviors conditioned on socio-cultural contexts, which is crucial for situative AI. This paper presents CANDLE, an end-to-end methodology for extracting high-quality cultural commonsense knowledge (CCSK) at scale. CANDLE extracts CCSK assertions from a huge web corpus and organizes them into coherent clusters, for 3 domains of subjects (geography, religion, occupation) and several cultural facets (food, drinks, clothing, traditions, rituals, behaviors). CANDLE includes judicious techniques for classification-based filtering and scoring of interestingness. Experimental evaluations show the superiority of the CANDLE CCSK collection over prior works, and an extrinsic use case demonstrates the benefits of CCSK for the GPT-3 language model. Code and data can be accessed at https://candle.mpi-inf.mpg.de/.


title: Intel Labs at Ego4D Challenge 2022: A Better Baseline for Audio-Visual Diarization

abstract: This report describes our approach for the Audio-Visual Diarization (AVD) task of the Ego4D Challenge 2022. Specifically, we present multiple technical improvements over the official baselines. First, we improve the detection performance of the camera wearer's voice activity by modifying the training scheme of its model. Second, we discover that an off-the-shelf voice activity detection model can effectively remove false positives when it is applied solely to the camera wearer's voice activities. Lastly, we show that better active speaker detection leads to a better AVD outcome. Our final method obtains 65.9% DER on the test set of Ego4D, which significantly outperforms all the baselines. Our submission achieved 1st place in the Ego4D Challenge 2022.


title: Activity-aware Human Mobility Prediction with Hierarchical Graph Attention Recurrent Network

abstract: Human mobility prediction is a fundamental task essential for various applications, including urban planning, location-based services and intelligent transportation systems. Existing methods often ignore activity information crucial for reasoning human preferences and routines, or adopt a simplified representation of the dependencies between time, activities and locations. To address these issues, we present Hierarchical Graph Attention Recurrent Network (HGARN) for human mobility prediction. Specifically, we construct a hierarchical graph based on all users' history mobility records and employ a Hierarchical Graph Attention Module to capture complex time-activity-location dependencies. This way, HGARN can learn representations with rich human travel semantics to model user preferences at the global level. We also propose a model-agnostic history-enhanced confidence (MAHEC) label to focus our model on each user's individual-level preferences. Finally, we introduce a Temporal Module, which employs recurrent structures to jointly predict users' next activities (as an auxiliary task) and their associated locations. By leveraging the predicted future user activity features through a hierarchical and residual design, the accuracy of the location predictions can be further enhanced. For model evaluation, we test the performances of our HGARN against existing SOTAs in both the recurring and explorative settings. The recurring setting focuses on assessing models' capabilities to capture users' individual-level preferences, while the results in the explorative setting tend to reflect the power of different models to learn users' global-level preferences. Overall, our model outperforms other baselines significantly in all settings based on two real-world human mobility data benchmarks. Source codes of HGARN are available at https://github.com/YihongT/HGARN.


title: The CMS Outer Tracker for the High-Luminosity LHC

abstract: The era of High-Luminosity Large Hadron Collider will pose unprecedented challenges for detector design and operation. The planned luminosity of the upgraded machine is 5-7.5 x 10$^{34}$ cm$^{-2}$ s$^{-1}$, reaching an integrated luminosity of 3000-4000 fb$^{-1}$ by the end of 2039. The CMS Tracker detector will have to be replaced in order to fully exploit the delivered luminosity and cope with the demanding operating conditions. The new detector will provide robust tracking as well as input for the first level trigger. This report is focusing on the replacement of the CMS Outer Tracker system, describing new layout and technological choices together with some highlights of research and development activities.


title: Resolving the $R_{\rm pA}$ and $v_2$ puzzle of $D^0$ mesons in $p-$Pb collisions at the LHC

abstract: It has been a challenge to understand the experimental data on both the nuclear modification factor and elliptic flow of $D^0$ mesons in $p-$Pb collisions at LHC energies. In this work, we study these collisions with an improved multi-phase transport model. By including the Cronin effect (i.e., transverse momentum broadening) and independent fragmentation for charm quarks, we provide the first simultaneous description of the $D^0$ meson $R_{\rm pA}$ and $v_2$ data at $p_{\rm T} \leq 8$ GeV$/c$. The model also reasonably describes the $D^0$ meson $p_{\rm T}$ spectra and the low-$p_{\rm T}$ charged hadron spectra, $R_{\rm pA}$ and $v_2$. Our results show that both parton interactions and the Cronin effect are important for the $D^0$ meson $R_{\rm pA}$, while parton interactions are mostly responsible for the $D^0$ meson $v_2$. It is thus essential to include the Cronin effect for the simultaneous description of the $D^0$ meson $R_{\rm pA}$ and $v_2$. This work implies that the Cronin effect could also be important for heavy hadrons in large systems.


title: FeatureBox: Feature Engineering on GPUs for Massive-Scale Ads Systems

abstract: Deep learning has been widely deployed for online ads systems to predict Click-Through Rate (CTR). Machine learning researchers and practitioners frequently retrain CTR models to test their new extracted features. However, the CTR model training often relies on a large number of raw input data logs. Hence, the feature extraction can take a significant proportion of the training time for an industrial-level CTR model. In this paper, we propose FeatureBox, a novel end-to-end training framework that pipelines the feature extraction and the training on GPU servers to save the intermediate I/O of the feature extraction. We rewrite computation-intensive feature extraction operators as GPU operators and leave the memory-intensive operator on CPUs. We introduce a layer-wise operator scheduling algorithm to schedule these heterogeneous operators. We present a light-weight GPU memory management algorithm that supports dynamic GPU memory allocation with minimal overhead. We experimentally evaluate FeatureBox and compare it with the previous in-production feature extraction framework on two real-world ads applications. The results confirm the effectiveness of our proposed method.


title: Flattened Graph Convolutional Networks For Recommendation

abstract: Graph Convolutional Networks (GCNs) and their variants have achieved significant performances on various recommendation tasks. However, many existing GCN models tend to perform recursive aggregations among all related nodes, which can arise severe computational burden to hinder their application to large-scale recommendation tasks. To this end, this paper proposes the flattened GCN~(FlatGCN) model, which is able to achieve superior performance with remarkably less complexity compared with existing models. Our main contribution is three-fold. First, we propose a simplified but powerful GCN architecture which aggregates the neighborhood information using one flattened GCN layer, instead of recursively. The aggregation step in FlatGCN is parameter-free such that it can be pre-computed with parallel computation to save memory and computational cost. Second, we propose an informative neighbor-infomax sampling method to select the most valuable neighbors by measuring the correlation among neighboring nodes based on a principled metric. Third, we propose a layer ensemble technique which improves the expressiveness of the learned representations by assembling the layer-wise neighborhood representations at the final layer. Extensive experiments on three datasets verify that our proposed model outperforms existing GCN models considerably and yields up to a few orders of magnitude speedup in training efficiency.


title: Towards Trustworthy AI-Empowered Real-Time Bidding for Online Advertisement Auctioning

abstract: Artificial intelligence-empowred Real-Time Bidding (AIRTB) is regarded as one of the most enabling technologies for online advertising. It has attracted significant research attention from diverse fields such as pattern recognition, game theory and mechanism design. Despite of its remarkable development and deployment, the AIRTB system can sometimes harm the interest of its participants (e.g., depleting the advertisers' budget with various kinds of fraud). As such, building trustworthy AIRTB auctioning systems has emerged as an important direction of research in this field in recent years. Due to the highly interdisciplinary nature of this field and a lack of a comprehensive survey, it is a challenge for researchers to enter this field and contribute towards building trustworthy AIRTB technologies. This paper bridges this important gap in trustworthy AIRTB literature. We start by analysing the key concerns of various AIRTB stakeholders and identify three main dimensions of trust building in AIRTB, namely security, robustness and fairness. For each of these dimensions, we propose a unique taxonomy of the state of the art, trace the root causes of possible breakdown of trust, and discuss the necessity of the given dimension. This is followed by a comprehensive review of existing strategies for fulfilling the requirements of each trust dimension. In addition, we discuss the promising future directions of research essential towards building trustworthy AIRTB systems to benefit the field of online advertising.


title: Learning to Jointly Transcribe and Subtitle for End-to-End Spontaneous Speech Recognition

abstract: TV subtitles are a rich source of transcriptions of many types of speech, ranging from read speech in news reports to conversational and spontaneous speech in talk shows and soaps. However, subtitles are not verbatim (i.e. exact) transcriptions of speech, so they cannot be used directly to improve an Automatic Speech Recognition (ASR) model. We propose a multitask dual-decoder Transformer model that jointly performs ASR and automatic subtitling. The ASR decoder (possibly pre-trained) predicts the verbatim output and the subtitle decoder generates a subtitle, while sharing the encoder. The two decoders can be independent or connected. The model is trained to perform both tasks jointly, and is able to effectively use subtitle data. We show improvements on regular ASR and on spontaneous and conversational ASR by incorporating the additional subtitle decoder. The method does not require preprocessing (aligning, filtering, pseudo-labeling, ...) of the subtitles.


title: First moments of a polyhedron clipped by a paraboloid

abstract: We provide closed-form expressions for the first moments (i.e., the volume and volume-weighted centroid) of a polyhedron clipped by a paraboloid, that is, of a polyhedron intersected with the subset of the three-dimensional real space located on one side of a paraboloid. These closed-form expressions are derived following successive applications of the divergence theorem and the judicious parametrization of the intersection of the polyhedron's faces with the paraboloid. We provide means for identifying ambiguous discrete intersection topologies, and propose a corrective procedure for preventing their occurence. Finally, we put our proposed closed-form expressions and numerical approach to the test with millions of random and manually engineered polyhedron/paraboloid intersection configurations. The results of these tests show that we are able to provide robust machine-accurate estimates of the first moments at a computational cost that is within one order of magnitude of that of state-of-the-art half-space clipping algorithms.


title: Diversified Recommendations for Agents with Adaptive Preferences

abstract: When an Agent visits a platform recommending a menu of content to select from, their choice of item depends not only on fixed preferences, but also on their prior engagements with the platform. The Recommender's primary objective is typically to encourage content consumption which optimizes some reward, such as ad revenue, but they often also aim to ensure that a wide variety of content is consumed by the Agent over time. We formalize this problem as an adversarial bandit task. At each step, the Recommender presents a menu of $k$ (out of $n$) items to the Agent, who selects one item in the menu according to their unknown preference model, which maps their history of past items to relative selection probabilities. The Recommender then observes the Agent's chosen item and receives bandit feedback of the item's reward. In addition to optimizing reward from selected items, the Recommender must also ensure that the total distribution of chosen items has sufficiently high entropy. We define a class of preference models which are locally learnable, i.e. behavior over the entire domain can be estimated by only observing behavior in a small region; this includes models representable by bounded-degree polynomials as well as functions with a sparse Fourier basis. For this class, we give an algorithm for the Recommender which obtains $\tilde{O}(T^{3/4})$ regret against all item distributions satisfying two conditions: they are sufficiently diversified, and they are instantaneously realizable at any history by some distribution over menus. We show that these conditions are closely connected: all sufficiently high-entropy distributions are instantaneously realizable at any item history. We also give a set of negative results justifying our assumptions, in the form of a runtime lower bound for non-local learning and linear regret lower bounds for alternate benchmarks.


title: Learning To Rank Diversely At Airbnb

abstract: Airbnb is a two-sided marketplace, bringing together hosts who own listings for rent, with prospective guests from around the globe. Applying neural network-based learning to rank techniques has led to significant improvements in matching guests with hosts. These improvements in ranking were driven by a core strategy: order the listings by their estimated booking probabilities, then iterate on techniques to make these booking probability estimates more and more accurate. Embedded implicitly in this strategy was an assumption that the booking probability of a listing could be determined independently of other listings in search results. In this paper we discuss how this assumption, pervasive throughout the commonly-used learning to rank frameworks, is false. We provide a theoretical foundation correcting this assumption, followed by efficient neural network architectures based on the theory. Explicitly accounting for possible similarities between listings, and reducing them to diversify the search results generated strong positive impact. We discuss these metric wins as part of the online A/B tests of the theory. Our method provides a practical way to diversify search results for large-scale production ranking systems.


title: PrivMVMF: Privacy-Preserving Multi-View Matrix Factorization for Recommender Systems

abstract: With an increasing focus on data privacy, there have been pilot studies on recommender systems in a federated learning (FL) framework, where multiple parties collaboratively train a model without sharing their data. Most of these studies assume that the conventional FL framework can fully protect user privacy. However, there are serious privacy risks in matrix factorization in federated recommender systems based on our study. This paper first provides a rigorous theoretical analysis of the server reconstruction attack in four scenarios in federated recommender systems, followed by comprehensive experiments. The empirical results demonstrate that the FL server could infer users' information with accuracy >80% based on the uploaded gradients from FL nodes. The robustness analysis suggests that our reconstruction attack analysis outperforms the random guess by >30% under Laplace noises with b no larger than 0.5 for all scenarios. Then, the paper proposes a new privacy-preserving framework based on homomorphic encryption, Privacy-Preserving Multi-View Matrix Factorization (PrivMVMF), to enhance user data privacy protection in federated recommender systems. The proposed PrivMVMF is successfully implemented and tested thoroughly with the MovieLens dataset.


title: Optimizing optical potentials with physics-inspired learning algorithms

abstract: We present our new experimental and theoretical framework which combines a broadband superluminescent diode (SLED/SLD) with fast learning algorithms to provide speed and accuracy improvements for the optimization of 1D optical dipole potentials, here generated with a Digital Micromirror Device (DMD). To characterize the setup and potential speckle patterns arising from coherence, we compare the superluminescent diode to a single-mode laser by investigating interference properties. We employ Machine Learning (ML) tools to train a physics-inspired model acting as a digital twin of the optical system predicting the behavior of the optical apparatus including all its imperfections. Implementing an iterative algorithm based on Iterative Learning Control (ILC) we optimize optical potentials an order of magnitude faster than heuristic optimization methods. We compare iterative model-based offline optimization and experimental feedback-based online optimization. Our methods provide a new route to fast optimization of optical potentials which is relevant for the dynamical manipulation of ultracold gases.


title: LEATHER: A Framework for Learning to Generate Human-like Text in Dialogue

abstract: Algorithms for text-generation in dialogue can be misguided. For example, in task-oriented settings, reinforcement learning that optimizes only task-success can lead to abysmal lexical diversity. We hypothesize this is due to poor theoretical understanding of the objectives in text-generation and their relation to the learning process (i.e., model training). To this end, we propose a new theoretical framework for learning to generate text in dialogue. Compared to existing theories of learning, our framework allows for analysis of the multi-faceted goals inherent to text-generation. We use our framework to develop theoretical guarantees for learners that adapt to unseen data. As an example, we apply our theory to study data-shift within a cooperative learning algorithm proposed for the GuessWhat?! visual dialogue game. From this insight, we propose a new algorithm, and empirically, we demonstrate our proposal improves both task-success and human-likeness of the generated text. Finally, we show statistics from our theory are empirically predictive of multiple qualities of the generated dialogue, suggesting our theory is useful for model-selection when human evaluations are not available.


title: Study of vibrational kinetics of CO2 and CO in CO2-O2 plasmas under non-equilibrium conditions

abstract: This work explores the effect of O2 addition on CO2 dissociation and on the vibrational kinetics of CO2 and CO under various non-equilibrium plasma conditions. A self-consistent model, previously validated for pure CO2 discharges, is further extended by adding the vibrational kinetics of CO, including electron impact excitation and de-excitation (e-V), vibration-to-translation relaxation (V-T) and vibration-to-vibration energy exchange (V-V) processes. The vibrational kinetics considered include levels up to v = 10 for CO and up to v1=2 and v2=v3=5, respectively for the symmetric stretch, bending and asymmetric stretch modes of CO2, and accounts for e-V, V-T in collisions between CO, CO2 and O2 molecules and O atoms and V-V processes involving all possible transfers involving CO2 and CO molecules. The kinetic scheme is validated by comparing the model predictions with recent experimental data measured in a DC glow discharge, operating at pressures in the range 0.4 - 5 Torr (53.33 - 666.66 Pa). The experimental results show a lower vibrational temperature of the different modes of CO2 and a decreased dissociation fraction of CO2 when O2 is added to the plasma but an increase of the vibrational temperature of CO. On the one hand, the simulations suggest that the former effect is the result of the stronger V-T energy-transfer collisions with O atoms which leads to an increase of the relaxation of the CO2 vibrational modes; On the other hand, the back reactions with O2 contribute to the lower CO2 dissociation fraction with increased O2 content in the mixture.


title: Robust Supermassive Black Hole Spin Mass-Energy Characteristics: A New Method and Results

abstract: The rotational properties of astrophysical black holes are fundamental quantities that characterization the black holes. A new method to empirically determine the spin mass-energy characteristics of astrophysical black holes is presented and applied here. Results are obtained for a sample of 100 supermassive black holes with collimated dual outflows and redshifts between about zero and two. An analysis indicates that about two-thirds of the black holes are maximally spinning, while one-third have a broad distribution of spin values; it is shown that the same distributions describe the quantity $\rm{(M_{rot}/M_{irr})}$. The new method is applied to obtain the black hole spin mass-energy, $\rm{M_{spin}}$, available for extraction relative to: the maximum possible value, the irreducible black hole mass, and the total black hole mass, $\rm{M_{dyn}}$. The total energy removed from the black hole system and deposited into the circumgalactic medium via dual outflows over the entire outflow lifetime of the source, $\rm{E_T}$, is studied relative to $\rm{M_{dyn}}$ and relative to the spin energy available per black hole, $\rm{E_{spin}/(M_{\odot}c^2)}$. The mean value of $\rm{Log(E_T/M_{dyn})}$ is about $(-2.47\pm 0.27)$. Several explanations of this and related results are discussed. For example, the energy input to the ambient gas from the outflow could turn off the accretion, or the impact of the black hole mass loss on the system could destabilize and terminate the outflow. The small values and restricted range of values of $\rm{Log(E_T/M_{dyn})}$ and $\rm{Log(E_T/E_{spin})}$ could suggest that these are fundamental properties of the primary process responsible for producing the dual collimated outflows.


title: Federated Best Arm Identification with Heterogeneous Clients

abstract: We study best arm identification in a federated multi-armed bandit setting with a central server and multiple clients, when each client has access to a {\em subset} of arms and each arm yields independent Gaussian observations. The goal is to identify the best arm of each client subject to an upper bound on the error probability; here, the best arm is one that has the largest {\em average} value of the means averaged across all clients having access to the arm. Our interest is in the asymptotics as the error probability vanishes. We provide an asymptotic lower bound on the growth rate of the expected stopping time of any algorithm. Furthermore, we show that for any algorithm whose upper bound on the expected stopping time matches with the lower bound up to a multiplicative constant ({\em almost-optimal} algorithm), the ratio of any two consecutive communication time instants must be {\em bounded}, a result that is of independent interest. We thereby infer that an algorithm can communicate no more sparsely than at exponential time instants in order to be almost-optimal. For the class of almost-optimal algorithms, we present the first-of-its-kind asymptotic lower bound on the expected number of {\em communication rounds} until stoppage. We propose a novel algorithm that communicates at exponential time instants, and demonstrate that it is asymptotically almost-optimal.


title: Wide Binaries as a Modified Gravity test: prospects for detecting triple-system contamination

abstract: Recent studies have shown that velocity differences of very wide binary stars, measured to high precision with GAIA, can provide an interesting test for modified-gravity theories which emulate dark matter; in essence, MOND-like theories (with external field effect included) predict that wide binaries (wider than ~ 7 kAU) should orbit ~ 15% faster than Newtonian for similar orbits; such a shift is readily detectable in principle in the sample of 9,000 candidate systems selected from GAIA EDR3 by Pittordis and Sutherland (2022; PS22). However, the main obstacle at present is the observed "fat tail" of candidate wide-binary systems with velocity differences at ~1.5 - 6x circular velocity; this tail cannot be bound pure binary systems, but a possible explanation of the tail is triple or quadruple systems with unresolved or undetected additional star(s). While this tail can be modelled and statistically subtracted, obtaining an accurate model for the triple population is crucial for a robust test for modified gravity. In this paper we explore prospects for observationally constraining the triple population: we simulate a population of hierarchical triple systems "observed" as in PS22 at random epochs and viewing angles; for each simulated system we evaluate various possible methods for detecting the third star, including GAIA astrometry, radial velocity drift, and several imaging methods from direct Rubin images, speckle imaging and coronagraphic imaging. Results are generally encouraging, in that typically 90% of the triple systems in the key regions of parameter space are detectable; there is a moderate "dead zone" of cool brown-dwarf companions at ~ 25-100 AU separation which are not detectable with any of our baseline methods. A large but feasible observing campaign can greatly clarify the effect of the triple/quadruple population and make the gravity test decisive.


title: A computational study on the energy efficiency of species production by single-pulse streamers in air

abstract: We study the energy efficiency of species production by streamer discharges with a single voltage pulse in atmospheric dry air, using a 2D axisymmetric fluid model. Sixty different positive streamers are simulated by varying the electrode length, the pulse duration and the applied voltage. Between these cases, the streamer radius and velocity vary by about an order of magnitude, but the variation in the maximal electric field is significantly smaller, about 30%. We find that G-values for the production of N(4S), O(3P), NO and N2O, which have relatively high activation energies, vary by about 30% to 55%. This variation is mainly caused by two factors: differences in the fraction of energy deposited in the streamer head region, and differences in the maximal electric field at the streamer head. When accounting for both factors, our computed G-values are in good agreement with an analytic estimate proposed by Naidis (2012 Plasma Sources Sci. Technol. 21 042001). We also simulate negative streamers and find that their production of N(4S), O(3P) and NO is less energy efficient. The results suggest that energy efficiency can be increased by reducing Joule heating in the streamer channel and by increasing the maximal electric field at the streamer head, for example by using short voltage pulses with a high applied voltage.


title: Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue

abstract: Lifelong learning (LL) is vital for advanced task-oriented dialogue (ToD) systems. To address the catastrophic forgetting issue of LL, generative replay methods are widely employed to consolidate past knowledge with generated pseudo samples. However, most existing generative replay methods use only a single task-specific token to control their models. This scheme is usually not strong enough to constrain the generative model due to insufficient information involved. In this paper, we propose a novel method, prompt conditioned VAE for lifelong learning (PCLL), to enhance generative replay by incorporating tasks' statistics. PCLL captures task-specific distributions with a conditional variational autoencoder, conditioned on natural language prompts to guide the pseudo-sample generation. Moreover, it leverages a distillation process to further consolidate past knowledge by alleviating the noise in pseudo samples. Experiments on natural language understanding tasks of ToD systems demonstrate that PCLL significantly outperforms competitive baselines in building LL models.


title: Towards Immersive Collaborative Sensemaking

abstract: When collaborating face-to-face, people commonly use the surfaces and spaces around them to perform sensemaking tasks, such as spatially organising documents, notes or images. However, when people collaborate remotely using desktop interfaces they no longer feel like they are sharing the same space. This limitation may be overcome through collaboration in immersive environments, which simulate the physical in-person experience. In this paper, we report on a between-groups study comparing collaborations on image organisation tasks, in an immersive Virtual Reality (VR) environment to more conventional desktop conferencing. Collecting data from 40 subjects in groups of four, we measured task performance, user behaviours, collaboration engagement and awareness. Overall, the VR and desktop interface resulted in similar speed, accuracy and social presence rating, but we observed more conversations and interaction with objects, and more equal contributions to the interaction from participants within groups in VR. We also identified differences in coordination and collaborative awareness behaviours between VR and desktop platforms. We report on a set of systematic measures for assessing VR collaborative experience and a new analysis tool that we have developed to capture user behaviours in collaborative setting. Finally, we provide design considerations and directions for future work.


title: Classical Density Functional Theory: Representability and Universal Bounds

abstract: We provide upper and lower bounds on the lowest free energy of a classical system at given one-particle density $\rho(x)$. We study both the canonical and grand-canonical cases, assuming the particles interact with a pair potential which decays fast enough at infinity.


title: Young Star-Forming Complexes in the Ring of the S0 galaxy NGC 4324

abstract: We present the results of our study of starforming regions in the lenticular galaxy NGC 4324. During a complex analysis of multiwavelength observational data -- the narrow-band emission-line images obtained with the 2.5-m telescope at the Caucasus Mountain Observatory of the Sternberg Astronomical Institute of the Moscow State University and the archival images in the broad bands of the SDSS, GALEX and WISE surveys -- we have detected young starforming complexes (clumps) located in the inner ring of the lenticular galaxy NGC 4324, and we have established a regular pattern of their distribution along the ring, which, nevertheless, changes with time (with age of starforming regions). We suggest several possible evolutionary paths of the lenticular galaxy NGC 4324, of which the accretion of gas-rich satellites or giant clouds (the so-called minor merging) is the most probable one.


title: Competing Laughlin state and Wigner crystal in bilayer graphene

abstract: We study the fractional quantum Hall effect in the central Landau level of bilayer graphene. By tuning the external applied magnetic field and the electric bias between the two layers one can access a regime where there is a degeneracy between Landau levels with orbital characters corresponding to N=0 and N=1 Galilean Landau levels. While the Laughlin state is generically the ground state for filling $\nu=1/3$ we find that it can be destroyed and replaced by a Wigner crystal at the same filling factor by tuning the bias and applied field. This competition does not take place at $\nu=2/3$ where the incompressible ground state remains stable. The possibility of electrically inducing the Wigner crystal state opens a new range of studies of this state of matter.


title: Observational Constraints on the $f(\phi,T)$ gravity theory

abstract: We investigate inflation in modified gravity framework by introducing a direct coupling term between a scalar field $\phi$ and the trace of the energy momentum tensor $T$ as $f(\phi,T) = 2 \phi( \kappa^{1/2} \alpha T + \kappa^{5/2} \beta T^2) $ to the Einstein-Hilbert action. We consider a class of inflaton potentials (i) $V_0 \phi^p e^{-\lambda\phi}$, (ii) $V_0\frac{ \lambda \phi^p}{1+\lambda\phi^p}$ and investigate the sensitivity of the modified gravity parameters $\alpha$ and $\beta$ on the inflaton dynamics. We derive the potential slow-roll parameters, scalar spectral index $n_s$, and tensor-to-scalar ratio $r$ in the above $f(\phi,T)$ gravity theory and analyze the following three choices of modified gravity parameters~(i) Case I:~ $\alpha \neq 0, ~\beta=0$ i.e. neglecting higher order terms, (ii) Case II:~ $\alpha=0$, $\beta \neq 0$~ and do the analysis for $T^2$ term, (iii) Case III:~ $\alpha \neq 0$ and $\beta \neq 0$ i.e. keeping all terms. For a range of potential parameters, we obtain constraints on $\alpha$ and $\beta$ in each of the above three cases using the WMAP and the PLANCK data.


title: i13DR: A Real-Time Demand Response Infrastructure for Integrating Renewable Energy Resources

abstract: With the ongoing integration of Renewable Energy Sources (RES), the complexity of power grids is increasing. Due to the fluctuating nature of RES, ensuring the reliability of power grids can be challenging. One possible approach for addressing these challenges is Demand Response (DR) which is described as matching the demand for electrical energy according to the changes and the availability of supply. However, implementing a DR system to monitor and control a broad set of electrical appliances in real-time introduces several new complications, including ensuring the reliability and financial feasibility of the system. In this work, we address these issues by designing and implementing a distributed real-time DR infrastructure for laptops, which estimates and controls the power consumption of a network of connected laptops in response to the fast, irregular changes of RES. Furthermore, since our approach is entirely software-based, we dramatically reduce the initial costs of the demand side participants. The result of our field experiments confirms that our system successfully schedules and executes rapid and effective DR events. However, the accuracy of the estimated power consumption of all participating laptops is relatively low, directly caused by our software-based approach.


title: Ionization dynamics and damage conditions for transparent materials irradiated with Mid-Infrared femtosecond pulses

abstract: The employment of ultrashort laser sources at the mid-IR spectral region for transparent materials is designed to open new routes for laser patterning and a wealth of exciting applications in optics and photonics. To elucidate the material response to irradiation with mid-IR laser sources, a consistent analysis of the interaction of long wavelength femtosecond pulses with dielectric materials is presented. The influence of the pulse duration is, in particularly, emphasized towards specifying the laser parameters for which photoionization and impact ionization are important. Simulations results using pulses at 2.2 mum, 3.2 mum and 5 mum are conducted to illustrate the optimum conditions for the onset of damage on the solid that is related to the occurrence of the optical breakdown. Results predict that the damage threshold scales as ~$\tau_p^\alpha$ (0.31<$\alpha$<0.37) at all laser wavelengths. Given the significant effect of the induced excitation level on the excitation of Surface Plasmons (SP) which account for the formation of laser induced periodic structures (LIPSS) oriented perpendicular to the laser polarization, a correlation of the produced electron densities with SP and the threshold of SP excitation (~$\tau_p^\beta$, 0.33<$\beta$<0.39) are also discussed in this yet unexplored spectral region. Results are expected to allow a novel surface patterning approach with strong mid-IR pulses for advanced applications.


title: A centra-limit theorem for conservative fragmentation chain

abstract: We are interested in a fragmentation process. We observe fragments frozen when their sizes are less than {\epsilon} ({\epsilon} > 0). It is known ([BM05]) that the empirical measure of these fragments converges in law, under some renormalization. In [HK11], the authors show a bound for the rate of convergence. Here, we show a central-limit theorem, under some assumptions. This gives us an exact rate of convergence.


title: Robust Preference Learning for Storytelling via Contrastive Reinforcement Learning

abstract: Controlled automated story generation seeks to generate natural language stories satisfying constraints from natural language critiques or preferences. Existing methods to control for story preference utilize prompt engineering which is labor intensive and often inconsistent. They may also use logit-manipulation methods which require annotated datasets to exist for the desired attributes. To address these issues, we first train a contrastive bi-encoder model to align stories with corresponding human critiques, named CARP, building a general purpose preference model. This is subsequently used as a reward function to fine-tune a generative language model via reinforcement learning. However, simply fine-tuning a generative language model with a contrastive reward model does not always reliably result in a story generation system capable of generating stories that meet user preferences. To increase story generation robustness we further fine-tune the contrastive reward model using a prompt-learning technique. A human participant study is then conducted comparing generations from our full system, ablations, and two baselines. We show that the full fine-tuning pipeline results in a story generator preferred over a LLM 20x as large as well as logit-based methods. This motivates the use of contrastive learning for general purpose human preference modeling.


title: Discrete & Bayesian Transaction Fee Mechanisms

abstract: Cryptocurrencies employ auction-esque transaction fee mechanisms (TFMs) to allocate transactions to blocks, and to determine how much fees miners can collect from transactions. Several impossibility results show that TFMs that satisfy a standard set of "good" properties obtain low revenue, and in certain cases, no revenue at all. In this work, we circumvent previous impossibilities by showing that when desired TFM properties are reasonably relaxed, simple mechanisms can obtain strictly positive revenue. By discretizing fees, we design a TFM that satisfies the extended TFM desiderata: it is dominant strategy incentive-compatible (DSIC), myopic miner incentive-compatible (MMIC), side-contract-proof (SCP) and obtains asymptotically optimal revenue (i.e., linear in the number of allocated bids), and optimal revenue when considering separable TFMs. If instead of discretizing fees we relax the DSIC and SCP properties, we show that Bitcoin's TFM, after applying the revelation principle, is Bayesian incentive-compatible (BIC), MMIC, off-chain-agreement (OCA) proof, and approximately revenue-optimal. We reach our results by characterizing the class of multi-item OCA-proof mechanisms, which may be of independent interest.


title: Existence and uniqueness of a weak solution to fractional single-phase-lag heat equation

abstract: In this article, we study the existence and uniqueness of a weak solution to the fractional single-phase lag heat equation. This model contains the terms $\cal{D}_t^\alpha(u_t)$ and $\cal{D}_t^\alpha u $ (with $\alpha \in(0,1)$), where $\cal{D}_t^\alpha$ denotes the Caputo fractional derivative in time of constant order $\alpha\in(0,1)$. We consider homogeneous Dirichlet boundary data for the temperature. We rigorously show the existence of a unique weak solution under low regularity assumptions on the data. Our main strategy is to use the variational formulation and a semidiscretisation in time based on Rothe's method. We obtain a priori estimates on the discrete solutions and show convergence of the Rothe functions to a weak solution. The variational approach is employed to show the uniqueness of this weak solution to the problem. We also consider the one-dimensional problem and derive a representation formula for the solution. We establish bounds on this explicit solution and its time derivative by extending properties of the multinomial Mittag-Leffler function.


title: EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning

abstract: Pre-trained vision-language models (VLMs) have achieved impressive results in a range of vision-language tasks. However, popular VLMs usually consist of hundreds of millions of parameters which brings challenges for fine-tuning and deployment in real-world applications due to space, memory, and latency constraints. In this work, we introduce a distilling then pruning framework to compress large vision-language models into smaller, faster, and more accurate ones. We first shrink the size of a pre-trained large VLM and apply knowledge distillation in the vision-language pre-training stage to obtain a task-agnostic compact VLM. Then we propose a modal-adaptive pruning algorithm to automatically infer the importance of vision and language modalities for different downstream tasks and adaptively remove redundant structures and neurons in different encoders with controllable target sparsity. We apply our framework to train EfficientVLM, a fast and accurate vision-language model consisting of 6 vision layers, 3 text layers, and 3 cross-modal fusion layers, accounting for only 93 million parameters in total, which is 44.3% of the teacher model. EfficientVLM retains 98.4% performance of the teacher model and accelerates its inference speed by 2.2x. EfficientVLM achieves a large absolute improvement over previous SoTA efficient VLMs of similar sizes by a large margin on various vision-language tasks, including VQAv2 (+4.9%), NLVR2 (+5.6%), ITR (R@1 on TR +17.2%, on IR + 15.6% ) and COCO caption generation (CIDEr +6.5), demonstrating a large potential on training lightweight VLMs.


title: Nobody Wants to Work Anymore: An Analysis of r/antiwork and the Interplay between Social and Mainstream Media during the Great Resignation

abstract: r/antiwork is a Reddit community that focuses on the discussion of worker exploitation, labour rights and related left-wing political ideas (e.g. universal basic income). In late 2021, r/antiwork became the fastest growing community on Reddit, coinciding with what the mainstream media began referring to as the Great Resignation. This same media coverage was attributed with popularising the subreddit and, therefore, accelerating its growth. In this article, we explore how the r/antiwork community was affected by the exponential increase in subscribers and the media coverage that chronicled its rise. We investigate how subreddit activity changed over time, the behaviour of heavy and light users, and how the topical nature of the discourse evolved with the influx of new subscribers. We report that, despite the continuing rise of subscribers well into 2022, activity on the subreddit collapsed after January 25th 2022, when a moderator's Fox news interview was widely criticised. While many users never commented again, longer running trends of users' posting and commenting behaviour did not change. Finally, while many users expressed their discontent at the changing nature of the subreddit as it became more popular, we found no evidence of major shifts in the topical content of discussion over the period studied, with the exception of the introduction of topics related to seasonal events (e.g. holidays, such as Thanksgiving) and ongoing developments in the news (e.g. working from home and the curtailing of reproductive rights in the United States).


title: Air showers and hadronic interactions with CORSIKA 8

abstract: The CORSIKA 8 project is a collaborative effort aiming to develop a versatile C++ framework for the simulation of extensive air showers, intended to eventually succeed the long-standing FORTRAN version. I present an overview of its current capabilities, focusing on aspects concerning the hadronic and muonic shower components. In particular, I demonstrate the "cascade lineage" feature and its application to quantify the importance of certain phase-space regions in hadronic interactions for muon production. Additionally, I show first results using Pythia 8.3, which as of late is usable as interaction model in cosmic-ray applications and is currently being integrated into CORSIKA 8.


title: A Formal-Methods Approach to Provide Evidence in Automated-Driving Safety Cases

abstract: The safety of automated driving systems must be justified by convincing arguments and supported by compelling evidence to persuade certification agencies, regulatory entities, and the general public to allow the systems on public roads. This persuasion is typically facilitated by compiling the arguments and the compelling evidence into a safety case. Reviews and testing, two common approaches to ensure correctness of automotive systems cannot explore the typically infinite set of possible behaviours. In contrast, formal methods are exhaustive methods that can provide mathematical proofs of correctness of models, and they can be used to prove that formalizations of functional safety requirements are fulfilled by formal models of system components. This paper shows how formal methods can provide evidence for the correct break-down of the functional safety requirements onto the components that are part of feedback loops, and how this evidence fits into the argument of the safety case. If a proof is obtained, the formal models are used as requirements on the components. This structure of the safety argumentation can be used to alleviate the need for reviews and tests to ensure that the break-down is correct, thereby saving effort both in data collection and verification time.


title: Chaos bound and its violation in the torus-like black hole

abstract: In this paper, we have studied the variation of the chaos bound in two regions of the torus-like black hole, i.e., the region close to the black hole horizon and the region at a certain distance from the black hole horizon. The angular momentum of the particle affects the effective potential and influences the magnitude of the chaotic behavior of the particle. Therefore, the angular momentum of particle is important in the study. The angular momentum of a particle not only affects the particle equilibrium orbital position, but also affects the Lyapunov exponent. As the angular momentum of the particle increases, the particle equilibrium position gradually moves away from the black hole horizon. In the near black hole horizon region, the chaos bound is not violated, however, at the far black hole horizon region, the chaos bound is violated. In addition unlike the charged AdS black hole which has a spherical topology of the horizon, the torus-like black hole has a toroidal topology of the horizon.


title: Multiple Choice Hard Thresholding Pursuit (MCHTP) for Simultaneous Sparse Recovery and Sparsity Order Estimation

abstract: We address the problem of sparse recovery using greedy compressed sensing recovery algorithms, without explicit knowledge of the sparsity. Estimating the sparsity order is a crucial problem in many practical scenarios, e.g., wireless communications, where exact value of the sparsity order of the unknown channel may be unavailable a priori. In this paper we have proposed a new greedy algorithm, referred to as the Multiple Choice Hard Thresholding Pursuit (MCHTP), which modifies the popular hard thresholding pursuit (HTP) suitably to iteratively recover the unknown sparse vector along with the sparsity order of the unknown vector. We provide provable performance guarantees which ensures that MCHTP can estimate the sparsity order exactly, along with recovering the unknown sparse vector exactly with noiseless measurements. The simulation results corroborate the theoretical findings, demonstrating that even without exact sparsity knowledge, with only the knowledge of a loose upper bound of the sparsity, MCHTP exhibits outstanding recovery performance, which is almost identical to that of the conventional HTP with exact sparsity knowledge. Furthermore, simulation results demonstrate much lower computational complexity of MCHTP compared to other state-of-the-art techniques like MSP.


title: A combinatorial algebraic approach for the modified second-generation time-delay interferometry

abstract: We generalize the combinatorial algebraic approach first proposed by Dhurandhar et al. to construct various classes of modified second-generation time-delay interferometry (TDI) solutions. The main idea behind the algorithm is to enumerate, in a given order, a specific type of commutator between two monomials defined by the products of particular time-displacement operators. On the one hand, the above commutators can be systematically rewritten as the elements of a left ideal, defined by the l.h.s. of the relevant equation for the TDI solution. On the other hand, these commutators are shown to vanish if we only keep up the first-order contributions regarding the rate of change of armlengths. In other words, each commutator furnishes a valid TDI solution pertaining to the given type of modified second-generation combinations. In this work, the original algorithm, which only involves time-delay operators, is extended by introducing the time-advance ones and then utilized to seek solutions of the Beacon, Relay, Monitor, Sagnac, and fully symmetric Sagnac types. We discuss the relation between the present scheme's solutions and those obtained by the geometric TDI approach, a well-known method of exhaustion of virtual optical paths. In particular, we report the results on novel Sagnac-inspired solutions that cannot be straightforwardly obtained using the geometric TDI algorithm. The average response functions, floor noise power spectral densities, and sensitivity functions are evaluated for the obtained solutions.


abstract: Object-oriented reinforcement learning (OORL) is a promising way to improve the sample efficiency and generalization ability over standard RL. Recent works that try to solve OORL tasks without additional feature engineering mainly focus on learning the object representations and then solving tasks via reasoning based on these object representations. However, none of these works tries to explicitly model the inherent similarity between different object instances of the same category. Objects of the same category should share similar functionalities; therefore, the category is the most critical property of an object. Following this insight, we propose a novel framework named Object-Category Aware Reinforcement Learning (OCARL), which utilizes the category information of objects to facilitate both perception and reasoning. OCARL consists of three parts: (1) Category-Aware Unsupervised Object Discovery (UOD), which discovers the objects as well as their corresponding categories; (2) Object-Category Aware Perception, which encodes the category information and is also robust to the incompleteness of (1) at the same time; (3) Object-Centric Modular Reasoning, which adopts multiple independent and object-category-specific networks when reasoning based on objects. Our experiments show that OCARL can improve both the sample efficiency and generalization in the OORL domain.


title: An Efficient FPGA Accelerator for Point Cloud

abstract: Deep learning-based point cloud processing plays an important role in various vision tasks, such as autonomous driving, virtual reality (VR), and augmented reality (AR). The submanifold sparse convolutional network (SSCN) has been widely used for the point cloud due to its unique advantages in terms of visual results. However, existing convolutional neural network accelerators suffer from non-trivial performance degradation when employed to accelerate SSCN because of the extreme and unstructured sparsity, and the complex computational dependency between the sparsity of the central activation and the neighborhood ones. In this paper, we propose a high performance FPGA-based accelerator for SSCN. Firstly, we develop a zero removing strategy to remove the coarse-grained redundant regions, thus significantly improving computational efficiency. Secondly, we propose a concise encoding scheme to obtain the matching information for efficient point-wise multiplications. Thirdly, we develop a sparse data matching unit and a computing core based on the proposed encoding scheme, which can convert the irregular sparse operations into regular multiply-accumulate operations. Finally, an efficient hardware architecture for the submanifold sparse convolutional layer is developed and implemented on the Xilinx ZCU102 field-programmable gate array board, where the 3D submanifold sparse U-Net is taken as the benchmark. The experimental results demonstrate that our design drastically improves computational efficiency, and can dramatically improve the power efficiency by 51 times compared to GPU.


title: Colored Tverberg theorem with new constraints on the faces

abstract: In this paper, we prove a version of the Colored Tverberg Theorem with new constraints on the faces, in which we limit the number of faces with each one of the colors.


title: Meta-Query-Net: Resolving Purity-Informativeness Dilemma in Open-set Active Learning

abstract: Unlabeled data examples awaiting annotations contain open-set noise inevitably. A few active learning studies have attempted to deal with this open-set noise for sample selection by filtering out the noisy examples. However, because focusing on the purity of examples in a query set leads to overlooking the informativeness of the examples, the best balancing of purity and informativeness remains an important question. In this paper, to solve this purity-informativeness dilemma in open-set active learning, we propose a novel Meta-Query-Net,(MQ-Net) that adaptively finds the best balancing between the two factors. Specifically, by leveraging the multi-round property of active learning, we train MQ-Net using a query set without an additional validation set. Furthermore, a clear dominance relationship between unlabeled examples is effectively captured by MQ-Net through a novel skyline regularization. Extensive experiments on multiple open-set active learning scenarios demonstrate that the proposed MQ-Net achieves 20.14% improvement in terms of accuracy, compared with the state-of-the-art methods.


title: Comparison of different automatic solutions for resection cavity segmentation in postoperative MRI volumes including longitudinal acquisitions

abstract: In this work, we compare five deep learning solutions to automatically segment the resection cavity in postoperative MRI. The proposed methods are based on the same 3D U-Net architecture. We use a dataset of postoperative MRI volumes, each including four MRI sequences and the ground truth of the corresponding resection cavity. Four solutions are trained with a different MRI sequence. Besides, a method designed with all the available sequences is also presented. Our experiments show that the method trained only with the T1 weighted contrast-enhanced MRI sequence achieves the best results, with a median DICE index of 0.81.


title: The Impact of Personality on Requirements Engineering Activities: A Mixed-Methods Study

abstract: Context: Requirements engineering (RE) is an important part of Software Engineering (SE), consisting of various human-centric activities that require the frequent collaboration of a variety of roles. Prior research has shown that personality is one such human aspect that has a huge impact on the success of a software project. However, a limited number of empirical studies exist focusing on the impact of personality on RE activities. Objective: The objective of this study is to explore and identify the impact of personality on RE activities, provide a better understanding of these impacts, and provide guidance on how to better handle these impacts in RE. Method: We used a mixed-methods approach, including a personality test-based survey (50 participants) and an in-depth interview study (15 participants) with software practitioners from around the world involved in RE activities. Results: Through personality test analysis, we found a majority of the practitioners have a high score on agreeableness and conscientiousness traits and an average score on extraversion and neuroticism traits. Through analysis of the interviews, we found a range of impacts related to the personality traits of software practitioners, their team members, and external stakeholders. These impacts can be positive or negative, depending on the RE activities, the overall software development process, and the people involved in these activities. Moreover, we found a set of strategies that can be applied to mitigate the negative impact of personality on RE activities. Conclusion: Our identified impacts of personality on RE activities and mitigation strategies serve to provide guidance to software practitioners on handling such possible personality impacts on RE activities and for researchers to investigate these impacts in greater depth in future.


title: Optimal AdaBoost Converges

abstract: The following work is a preprint collection of formal proofs regarding the convergence properties of the AdaBoost machine learning algorithm's classifier and margins. Various math and computer science papers have been written regarding conjectures and special cases of these convergence properties. Furthermore, the margins of AdaBoost feature prominently in the research surrounding the algorithm. At the zenith of this paper we present how AdaBoost's classifier and margins converge on a value that agrees with decades of research. After this, we show how various quantities associated with the combined classifier converge.


title: Free Fine-tuning: A Plug-and-Play Watermarking Scheme for Deep Neural Networks

abstract: Watermarking has been widely adopted for protecting the intellectual property (IP) of Deep Neural Networks (DNN) to defend the unauthorized distribution. Unfortunately, the popular data-poisoning DNN watermarking scheme relies on target model fine-tuning to embed watermarks, which limits its practical applications in tackling real-world tasks. Specifically, the learning of watermarks via tedious model fine-tuning on a poisoned dataset (carefully-crafted sample-label pairs) is not efficient in tackling the tasks on challenging datasets and production-level DNN model protection. To address the aforementioned limitations, in this paper, we propose a plug-and-play watermarking scheme for DNN models by injecting an independent proprietary model into the target model to serve the watermark embedding and ownership verification. In contrast to the prior studies, our proposed method by incorporating a proprietary model is free of target model fine-tuning without involving any parameters update of the target model, thus the fidelity is well preserved. Our research findings reveal that model fine-tuning with poisoned data is not prepared for the IP protection of DNN models deployed in real-world tasks and poses a new research direction toward a more thorough understanding and investigation of adopting the proprietary model for DNN watermarking. The source code and models are available at https://github.com/AntigoneRandy/PTYNet.


title: A Consistent and Differentiable Lp Canonical Calibration Error Estimator

abstract: Calibrated probabilistic classifiers are models whose predicted probabilities can directly be interpreted as uncertainty estimates. It has been shown recently that deep neural networks are poorly calibrated and tend to output overconfident predictions. As a remedy, we propose a low-bias, trainable calibration error estimator based on Dirichlet kernel density estimates, which asymptotically converges to the true $L_p$ calibration error. This novel estimator enables us to tackle the strongest notion of multiclass calibration, called canonical (or distribution) calibration, while other common calibration methods are tractable only for top-label and marginal calibration. The computational complexity of our estimator is $\mathcal{O}(n^2)$, the convergence rate is $\mathcal{O}(n^{-1/2})$, and it is unbiased up to $\mathcal{O}(n^{-2})$, achieved by a geometric series debiasing scheme. In practice, this means that the estimator can be applied to small subsets of data, enabling efficient estimation and mini-batch updates. The proposed method has a natural choice of kernel, and can be used to generate consistent estimates of other quantities based on conditional expectation, such as the sharpness of a probabilistic classifier. Empirical results validate the correctness of our estimator, and demonstrate its utility in canonical calibration error estimation and calibration error regularized risk minimization.


title: SAILOR: Scaling Anchors via Insights into Latent Object Representation

abstract: LiDAR 3D object detection models are inevitably biased towards their training dataset. The detector clearly exhibits this bias when employed on a target dataset, particularly towards object sizes. However, object sizes vary heavily between domains due to, for instance, different labeling policies or geographical locations. State-of-the-art unsupervised domain adaptation approaches outsource methods to overcome the object size bias. Mainstream size adaptation approaches exploit target domain statistics, contradicting the original unsupervised assumption. Our novel unsupervised anchor calibration method addresses this limitation. Given a model trained on the source data, we estimate the optimal target anchors in a completely unsupervised manner. The main idea stems from an intuitive observation: by varying the anchor sizes for the target domain, we inevitably introduce noise or even remove valuable object cues. The latent object representation, perturbed by the anchor size, is closest to the learned source features only under the optimal target anchors. We leverage this observation for anchor size optimization. Our experimental results show that, without any retraining, we achieve competitive results even compared to state-of-the-art weakly-supervised size adaptation approaches. In addition, our anchor calibration can be combined with such existing methods, making them completely unsupervised.


title: Surface abnormality detection in medical and inspection systems using energy variations in co-occurrence matrixes

abstract: Detection of surface defects is one of the most important issues in the field of image processing and machine vision. In this article, a method for detecting surface defects based on energy changes in co-occurrence matrices is presented. The presented method consists of two stages of training and testing. In the training phase, the co-occurrence matrix operator is first applied on healthy images and then the amount of output energy is calculated. In the following, according to the changes in the amount of energy, a suitable feature vector is defined, and with the help of it, a suitable threshold for the health of the images is obtained. Then, in the test phase, with the help of the calculated quorum, the defective parts are distinguished from the healthy ones. In the results section, the mentioned method has been applied on stone and ceramic images and its detection accuracy has been calculated and compared with some previous methods. Among the advantages of the presented method, we can mention high accuracy, low calculations and compatibility with all types of levels due to the use of the training stage. The proposed approach can be used in medical applications to detect abnormalities such as diseases. So, the performance is evaluated on 2d-hela dataset to classify cell phenotypes. The proposed approach provides about 89.56 percent accuracy on 2d-hela.


title: A construction of Sbrana-Cartan hypersurfaces in the discrete class

abstract: The classical classifications of the locally isometrically deformable Euclidean hypersurfaces obtained by U. Sbrana in 1909 and E. Cartan in 1916 includes four classes, among them the one formed by submanifolds that allow just a single deformation. The question of whether these Sbrana-Cartan hypersurfaces do, in fact, exist was not addressed by either of them. Positive answers to this question were given by Dajczer-Florit-Tojeiro in 1998 for the ones called of hyperbolic type and by Dajczer-Florit in 2004 when of elliptic type which is the other possibility. In both cases the examples constructed are rather special. The main result of this paper yields an abundance of examples of hypersurfaces of either type and seems to point in the direction of a classification although that goal remains elusive.


title: A Sequence-Aware Recommendation Method Based on Complex Networks

abstract: Online stores and service providers rely heavily on recommendation softwares to guide users through the vast amount of available products. Consequently, the field of recommender systems has attracted increased attention from the industry and academia alike, but despite this joint effort, the field still faces several challenges. For instance, most existing work models the recommendation problem as a matrix completion problem to predict the user preference for an item. This abstraction prevents the system from utilizing the rich information from the ordered sequence of user actions logged in online sessions. To address this limitation, researchers have recently developed a promising new breed of algorithms called sequence-aware recommender systems to predict the user's next action by utilizing the time series composed of the sequence of actions in an ongoing user session. This paper proposes a novel sequence-aware recommendation approach based on a complex network generated by the hidden metric space model, which combines node similarity and popularity to generate links. We build a network model from data and then use it to predict the user's subsequent actions. The network model provides an additional source of information that improves the accuracy of the recommendations. The proposed method is implemented and tested experimentally on a large dataset. The results prove that the proposed approach performs better than state-of-the-art recommendation methods.


title: Intra-session Context-aware Feed Recommendation in Live Systems

abstract: Feed recommendation allows users to constantly browse items until feel uninterested and leave the session, which differs from traditional recommendation scenarios. Within a session, user's decision to continue browsing or not substantially affects occurrences of later clicks. However, such type of exposure bias is generally ignored or not explicitly modeled in most feed recommendation studies. In this paper, we model this effect as part of intra-session context, and propose a novel intra-session Context-aware Feed Recommendation (INSCAFER) framework to maximize the total views and total clicks simultaneously. User click and browsing decisions are jointly learned by a multi-task setting, and the intra-session context is encoded by the session-wise exposed item sequence. We deploy our model online with all key business benchmarks improved. Our method sheds some lights on feed recommendation studies which aim to optimize session-level click and view metrics.


title: A Recommendation Approach based on Similarity-Popularity Models of Complex Networks

abstract: Recommender systems have become an essential tool for providers and users of online services and goods, especially with the increased use of the Internet to access information and purchase products and services. This work proposes a novel recommendation method based on complex networks generated by a similarity-popularity model to predict ones. We first construct a model of a network having users and items as nodes from observed ratings and then use it to predict unseen ratings. The prospect of producing accurate rating predictions using a similarity-popularity model with hidden metric spaces and dot-product similarity is explored. The proposed approach is implemented and experimentally compared against baseline and state-of-the-art recommendation methods on 21 datasets from various domains. The experimental results demonstrate that the proposed method produces accurate predictions and outperforms existing methods. We also show that the proposed approach produces superior results in low dimensions, proving its effectiveness for data visualization and exploration.


title: Discussion about Attacks and Defenses for Fair and Robust Recommendation System Design

abstract: Information has exploded on the Internet and mobile with the advent of the big data era. In particular, recommendation systems are widely used to help consumers who struggle to select the best products among such a large amount of information. However, recommendation systems are vulnerable to malicious user biases, such as fake reviews to promote or demote specific products, as well as attacks that steal personal information. Such biases and attacks compromise the fairness of the recommendation model and infringe the privacy of users and systems by distorting data.Recently, deep-learning collaborative filtering recommendation systems have shown to be more vulnerable to this bias. In this position paper, we examine the effects of bias that cause various ethical and social issues, and discuss the need for designing the robust recommendation system for fairness and stability.


title: ISTA-Inspired Network for Image Super-Resolution

abstract: Deep learning for image super-resolution (SR) has been investigated by numerous researchers in recent years. Most of the works concentrate on effective block designs and improve the network representation but lack interpretation. There are also iterative optimization-inspired networks for image SR, which take the solution step as a whole without giving an explicit optimization step. This paper proposes an unfolding iterative shrinkage thresholding algorithm (ISTA) inspired network for interpretable image SR. Specifically, we analyze the problem of image SR and propose a solution based on the ISTA method. Inspired by the mathematical analysis, the ISTA block is developed to conduct the optimization in an end-to-end manner. To make the exploration more effective, a multi-scale exploitation block and multi-scale attention mechanism are devised to build the ISTA block. Experimental results show the proposed ISTA-inspired restoration network (ISTAR) achieves competitive or better performances than other optimization-inspired works with fewer parameters and lower computation complexity.


title: It takes two to know one: Computing accurate one-point PDF covariances from effective two-point PDF models

abstract: One-point probability distribution functions (PDFs) of the cosmic matter density are powerful cosmological probes that extract non-Gaussian properties of the matter distribution and complement two-point statistics. Computing the covariance of one-point PDFs is key for building a robust galaxy survey analysis for upcoming surveys like Euclid and the Rubin Observatory LSST and requires good models for the two-point PDFs characterising spatial correlations. In this work, we obtain accurate PDF covariances using effective shifted lognormal two-point PDF models for the mildly non-Gaussian weak lensing convergence and validate our predictions against large sets of Gaussian and non-Gaussian maps. We show how the dominant effects in the covariance matrix capturing super-sample covariance arise from a large-separation expansion of the two-point PDF and discuss differences between the covariances obtained from small patches and full sky maps. Finally, we describe how our formalism can be extended to characterise the PDF covariance for 3D-dimensional spectroscopic fields using the 3D matter PDF as an example. We describe how covariances from simulated boxes with fixed overall density can be supplemented with the missing super-sample covariance effect by relying on theoretical predictions validated against separate-universe style simulations.


title: Probing itinerant carrier dynamics at the diamond surface using single nitrogen vacancy centers

abstract: Color centers in diamond are widely explored for applications in quantum sensing, computing, and networking. Their optical, spin, and charge properties have been extensively studied, while their interactions with itinerant carriers are relatively unexplored. Here we show that NV centers situated within 10 nm of the diamond surface can be converted to the neutral charge state via hole capture. By measuring the hole capture rate, we extract the capture cross section, which is suppressed by proximity to the diamond surface. The distance dependence is consistent with a carrier diffusion model, indicating that the itinerant carrier lifetime can be long, even at the diamond surface. Measuring dynamics of near-surface NV centers offers a new tool for characterizing the diamond surface and investigating charge transport in diamond devices.


title: Receiver Bandwidth Extension Beyond Nyquist Using Channel Bonding

abstract: Current and upcoming communication and sensing technologies require ever larger bandwidths. Channel bonding can be utilized to extend a receiver's instantaneous bandwidth beyond a single converter's Nyquist limit. Two potential joint front-end and converter design approaches are theoretically introduced, realized and evaluated in this paper. The Xilinx RFSoC platform with its 5 GSa/s analog to digital converters (ADCs) is used to implement both a hybrid coupler based in-phase/quadrature (I/Q) sampling and a time-interleaved sampling approach along with channel bonding. Both realizations are demonstrated to be able to reconstruct instantaneous bandwidths of 5 GHz with up to 49 dB image rejection ratio (IRR) typically within 4 to 8 dB the front-ends' theoretical limits.


title: Shadfa 0.1: The Iranian Movie Knowledge Graph and Graph-Embedding-Based Recommender System

abstract: Movies are a great source of entertainment. However, the problem arises when one is trying to find the desired content within this vast amount of data which is significantly increasing every year. Recommender systems can provide appropriate algorithms to solve this problem. The content_based technique has found popularity due to the lack of available user data in most cases. Content_based recommender systems are based on the similarity of items' demographic information; Term Frequency _ Inverse Document Frequency (TF_IDF) and Knowledge Graph Embedding (KGE) are two approaches used to vectorize data to calculate these similarities. In this paper, we propose a weighted content_based movie RS by combining TF_IDF which is an appropriate approach for embedding textual data such as plot/description, and KGE which is used to embed named entities such as the director's name. The weights between features are determined using a Genetic algorithm. Additionally, the Iranian movies dataset is created by scraping data from movie_related websites. This dataset and the structure of the FarsBase KG are used to create the MovieFarsBase KG which is a component in the implementation process of the proposed content_based RS. Using precision, recall, and F1 score metrics, this study shows that the proposed approach outperforms the conventional approach that uses TF_IDF for embedding all attributes.


title: A polynomial-time approximation scheme for the maximal overlap of two independent Erd\H{o}s-R\'enyi graphs

abstract: For two independent Erd\H{o}s-R\'enyi graphs $\mathbf G(n,p)$, we study the maximal overlap (i.e., the number of common edges) of these two graphs over all possible vertex correspondence. We present a polynomial-time algorithm which finds a vertex correspondence whose overlap approximates the maximal overlap up to a multiplicative factor that is arbitrarily close to 1. As a by-product, we prove that the maximal overlap is asymptotically $\frac{n}{2\alpha-1}$ for $p=n^{-\alpha}$ with some constant $\alpha\in (1/2,1)$.


title: TechRank

abstract: We introduce TechRank, a recursive algorithm based on a bi-partite graph with weighted nodes. We develop TechRank to link companies and technologies based on the method of reflection. We allow the algorithm to incorporate exogenous variables that reflect an investor's preferences. We calibrate the algorithm in the cybersecurity sector. First, our results help estimate each entity's influence and explain companies' and technologies' ranking. Second, they provide investors with a quantitative optimal ranking of technologies and thus, help them design their optimal portfolio. We propose this method as an alternative to traditional portfolio management and, in the case of private equity investments, as a new way to price assets for which cash flows are not observable.


title: Quenched invariance principle for biased random walks in random conductances in the sub-ballistic regime

abstract: We consider a biased random walk in positive random conductances on $\mathbb{Z}^d$ for $d\geq 5$. In the sub-ballistic regime, we prove the quenched convergence of the properly rescaled random walk towards a Fractional Kinetics.


title: Low-power In-pixel Computing with Current-modulated Switched Capacitors

abstract: We present a scalable in-pixel processing architecture that can reduce the data throughput by 10X and consume less than 30 mW per megapixel at the imager frontend. Unlike the state-of-the-art (SOA) analog process-in-pixel (PIP) that modulates the exposure time of photosensors when performing matrix-vector multiplications, we use switched capacitors and pulse width modulation (PWM). This non-destructive approach decouples the sensor exposure and computing, providing processing parallelism and high data fidelity. Our design minimizes the computational complexity and chip density by leveraging the patch-based feature extraction that can perform as well as the CNN. We further reduce data using partial observation of the attended objects, which performs closely to the full frame observations. We have been studying the reduction of output features as a function of accuracy, chip density and power consumption from a transformer-based backend model for object classification and detection.


title: Stabilized exponential time differencing schemes for the convective Allen-Cahn equation

abstract: The convective Allen-Cahn equation has been widely used to simulate multi-phase flows in many phase-field models. As a generalized form of the classic Allen-Cahn equation, the convective Allen-Cahn equation still preserves the maximum bound principle (MBP) in the sense that the time-dependent solution of the equation with appropriate initial and boundary conditions preserves for all time a uniform pointwise bound in absolute value. In this paper, we develop efficient first- and second-order exponential time differencing (ETD) schemes combined with the linear stabilizing technique to preserve the MBP unconditionally in the discrete setting. The space discretization is done using the upwind difference scheme for the convective term and the central difference scheme for the diffusion term, and both the mobility and nonlinear terms are approximated through the linear convex interpolation. The unconditional preservation of the MBP of the proposed schemes is proven, and their convergence analysis is presented. Various numerical experiments in two and three dimensions are also carried out to verify the theoretical results.


title: Parameter-Free Average Attention Improves Convolutional Neural Network Performance (Almost) Free of Charge

abstract: Visual perception is driven by the focus on relevant aspects in the surrounding world. To transfer this observation to the digital information processing of computers, attention mechanisms have been introduced to highlight salient image regions. Here, we introduce a parameter-free attention mechanism called PfAAM, that is a simple yet effective module. It can be plugged into various convolutional neural network architectures with a little computational overhead and without affecting model size. PfAAM was tested on multiple architectures for classification and segmentic segmentation leading to improved model performance for all tested cases. This demonstrates its wide applicability as a general easy-to-use module for computer vision tasks. The implementation of PfAAM can be found on https://github.com/nkoerb/pfaam.


title: Asymmetric Student-Teacher Networks for Industrial Anomaly Detection

abstract: Industrial defect detection is commonly addressed with anomaly detection (AD) methods where no or only incomplete data of potentially occurring defects is available. This work discovers previously unknown problems of student-teacher approaches for AD and proposes a solution, where two neural networks are trained to produce the same output for the defect-free training examples. The core assumption of student-teacher networks is that the distance between the outputs of both networks is larger for anomalies since they are absent in training. However, previous methods suffer from the similarity of student and teacher architecture, such that the distance is undesirably small for anomalies. For this reason, we propose asymmetric student-teacher networks (AST). We train a normalizing flow for density estimation as a teacher and a conventional feed-forward network as a student to trigger large distances for anomalies: The bijectivity of the normalizing flow enforces a divergence of teacher outputs for anomalies compared to normal data. Outside the training distribution the student cannot imitate this divergence due to its fundamentally different architecture. Our AST network compensates for wrongly estimated likelihoods by a normalizing flow, which was alternatively used for anomaly detection in previous work. We show that our method produces state-of-the-art results on the two currently most relevant defect detection datasets MVTec AD and MVTec 3D-AD regarding image-level anomaly detection on RGB and 3D data.