title: Evidence for chiral superconductivity on a silicon surface

abstract: Sn adatoms on a Si(111) substrate with 1/3 monolayer coverage form a two-dimensional triangular adatom lattice with one unpaired electron per site and an antiferromagnetic Mott insulating state. The Sn layers can be modulation hole-doped and metallized using heavily-doped $p$-type Si(111) substrates, and become superconducting at low temperatures. While the pairing symmetry of the superconducting state is currently unknown, the combination of repulsive interactions and frustration inherent to the triangular adatom lattice opens up the possibility for a chiral order parameter. Here, we study the superconducting state of Sn/Si(111) using scanning tunneling microscopy/spectroscopy and quasi-particle interference imaging. We find evidence for a doping-dependent $T_c$ with a fully gapped order parameter, the presence of time-reversal symmetry breaking, and a strong enhancement of the zero-bias conductance near the edges of the superconducting domains. While each individual piece of evidence could have a more mundane interpretation, our combined results suggest the tantalizing possibility that Sn/Si(111) is an unconventional chiral d-wave superconductor.


title: Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learning

abstract: We introduce hybrid execution in multi-agent reinforcement learning (MARL), a new paradigm in which agents aim to successfully complete cooperative tasks with arbitrary communication levels at execution time by taking advantage of information-sharing among the agents. Under hybrid execution, the communication level can range from a setting in which no communication is allowed between agents (fully decentralized), to a setting featuring full communication (fully centralized), but the agents do not know beforehand which communication level they will encounter at execution time. To formalize our setting, we define a new class of multi-agent partially observable Markov decision processes (POMDPs) that we name hybrid-POMDPs, which explicitly model a communication process between the agents. We contribute MARO, an approach that makes use of an auto-regressive predictive model, trained in a centralized manner, to estimate missing agents' observations at execution time. We evaluate MARO on standard scenarios and extensions of previous benchmarks tailored to emphasize the negative impact of partial observability in MARL. Experimental results show that our method consistently outperforms relevant baselines, allowing agents to act with faulty communication while successfully exploiting shared information.


title: Uniqueness in weighted Lebesgue spaces for an elliptic equation with drift on manifolds

abstract: We investigate the uniqueness, in suitable weighted Lebesgue spaces, of solutions to a class of elliptic equations with a drift posed on a complete, noncompact, Riemannian manifold $M$ of infinite volume and dimension $N\ge2$. Furthermore, in the special case of a model manifold with polynomial volume growth, we show that the conditions on the drift term are sharp.


title: Pre-emergence Signatures Of Horizontal Divergent Flows In Solar Active Regions

abstract: Solar active regions (ARs) play a fundamental role in driving many of the geo-effective eruptions which propagate into the Solar System. However, we are still unable to consistently predict where and when ARs will occur across the solar disk by identifying pre-emergence signatures in observables such as the Doppler velocity (without using Helioseismic methods). Here we aim to determine the earliest time at which pre-emergence signatures, specifically the Horizontal Divergent Flow (HDF), can be confidently detected using data from the Solar Dynamics Observatory's Helioseismic and Magnetic Imager (SDO/HMI). Initially, we follow previous studies using the thresholding method, which searches for significant increases in the number of pixels that display a specific line-of-sight velocity. We expand this method to more velocity windows and conduct a basic parameter study investigating the effect of cadence on the inferred results. Our findings agree with previous studies with $37.5$% of ARs displaying a HDF, with average lead times between the HDF and flux emergence of $58$ minutes. We present a new potential signature of flux emergence which manifests as cadence-independent transient disruptions to the amplitudes of multiple velocity windows and recover potential pre-emergence signatures for 10 of the 16 ARs studied, with lead times of 60-156 minutes. Several effects can influence both the estimated times of both HDF and flux emergence suggesting that one may need to combine Doppler and magnetic field data to get a reliable indicator of continued flux emergence.


title: Task Compass: Scaling Multi-task Pre-training with Task Prefix

abstract: Leveraging task-aware annotated data as supervised signals to assist with self-supervised learning on large-scale unlabeled data has become a new trend in pre-training language models. Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks. To tackle the challenge, we propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks. We conduct extensive experiments on 40 datasets, which show that our model can not only serve as the strong foundation backbone for a wide range of tasks but also be feasible as a probing tool for analyzing task relationships. The task relationships reflected by the prefixes align transfer learning performance between tasks. They also suggest directions for data augmentation with complementary tasks, which help our model achieve human-parity results on commonsense reasoning leaderboards. Code is available at https://github.com/cooelf/CompassMTL


title: On the Nonlinear Shaping Gain with Probabilistic Shaping and Carrier Phase Recovery

abstract: The performance of different probabilistic amplitude shaping (PAS) techniques in the nonlinear regime is investigated, highlighting its dependence on the PAS block length and the interaction with carrier phase recovery (CPR). Different PAS implementations are considered, based on different distribution matching (DM) techniques-namely, sphere shaping, shell mapping with different number of shells, and constant composition DM-and amplitude-to-symbol maps. When CPR is not included, PAS with optimal block length provides a nonlinear shaping gain with respect to a linearly optimized PAS (with infinite block length); among the considered DM techniques, the largest gain is obtained with sphere shaping. On the other hand, the nonlinear shaping gain becomes smaller, or completely vanishes, when CPR is included, meaning that in this case all the considered implementations achieve a similar performance for a sufficiently long block length. Similar results are obtained in different link configurations (1x180km, 15x80km, and 27x80km single-mode-fiber links), and also including laser phase noise, except when in-line dispersion compensation is used. Furthermore, we define a new metric, the nonlinear phase noise (NPN) metric, which is based on the frequency resolved logarithmic perturbation models and explains the interaction of CPR and PAS. We show that the NPN metric is highly correlated with the performance of the system. Our results suggest that, in general, the optimization of PAS in the nonlinear regime should always account for the presence of a CPR algorithm. In this case, the reduction of the rate loss (obtained by using sphere shaping and increasing the DM block length) turns out to be more important than the mitigation of the nonlinear phase noise (obtained by using constant-energy DMs and reducing the block length), the latter being already granted by the CPR algorithm.


title: The moment problem for random objects in a category

abstract: The moment problem in probability theory asks for criteria for when there exists a unique measure with a given tuple of moments. We study a variant of this problem for random objects in a category, where a moment is given by the average number of epimorphisms to a fixed object. When the moments do not grow too fast, we give a necessary and sufficient condition for existence of a distribution with those moments, show that a unique such measure exists, give formulas for the measure in terms of the moments, and prove that measures with those limiting moments approach that particular measure. Our result applies to categories satisfying some finiteness conditions and a condition that gives an analog of the second isomorphism theorem, including the categories of finite groups, finite modules, finite rings, as well as many variations of these categories. This work is motivated by the non-abelian Cohen-Lenstra-Martinet program in number theory, which aims to calculate the distribution of random profinite groups arising as Galois groups of maximal unramified extensions of random number fields.


title: Language Models are Realistic Tabular Data Generators

abstract: Tabular data is among the oldest and most ubiquitous forms of data. However, the generation of synthetic samples with the original data's characteristics remains a significant challenge for tabular data. While many generative models from the computer vision domain, such as variational autoencoders or generative adversarial networks, have been adapted for tabular data generation, less research has been directed towards recent transformer-based large language models (LLMs), which are also generative in nature. To this end, we propose GReaT (Generation of Realistic Tabular data), which exploits an auto-regressive generative LLM to sample synthetic and yet highly realistic tabular data. Furthermore, GReaT can model tabular data distributions by conditioning on any subset of features; the remaining features are sampled without additional overhead. We demonstrate the effectiveness of the proposed approach in a series of experiments that quantify the validity and quality of the produced data samples from multiple angles. We find that GReaT maintains state-of-the-art performance across numerous real-world and synthetic data sets with heterogeneous feature types coming in various sizes.


title: TwiRGCN: Temporally Weighted Graph Convolution for Question Answering over Temporal Knowledge Graphs

abstract: Recent years have witnessed much interest in temporal reasoning over knowledge graphs (KG) for complex question answering (QA), but there remains a substantial gap in human capabilities. We explore how to generalize relational graph convolutional networks (RGCN) for temporal KGQA. Specifically, we propose a novel, intuitive and interpretable scheme to modulate the messages passed through a KG edge during convolution, based on the relevance of its associated time period to the question. We also introduce a gating device to predict if the answer to a complex temporal question is likely to be a KG entity or time and use this prediction to guide our scoring mechanism. We evaluate the resulting system, which we call TwiRGCN, on TimeQuestions, a recently released, challenging dataset for multi-hop complex temporal QA. We show that TwiRGCN significantly outperforms state-of-the-art systems on this dataset across diverse question types. Notably, TwiRGCN improves accuracy by 9--10 percentage points for the most difficult ordinal and implicit question types.


title: DialoGen: Generalized Long-Range Context Representation for Dialogue Systems

abstract: Long-range context modeling is crucial to both dialogue understanding and generation. The most popular method for dialogue context representation is to concatenate the last-$k$ utterances in chronological order. However, this method may not be ideal for conversations containing long-range dependencies, i.e., when there is a need to look beyond last-$k$ utterances to generate a meaningful response. In this work, we propose DialoGen, a novel encoder-decoder based framework for dialogue generation with a generalized context representation that can look beyond the last-$k$ utterances. The main idea of our approach is to identify and utilize the most relevant historical utterances instead of last-$k$, which also enables the compact representation of dialogue history with fewer tokens. We study the effectiveness of our proposed method on both dialogue generation (open-domain) and understanding (DST). Even with a compact context representation, DialoGen performs comparably to the state-of-the-art models on the open-domain DailyDialog dataset. We observe a similar behavior on the DST task of the MultiWOZ dataset when the proposed context representation is applied to existing DST models. We also discuss the generalizability and interpretability of DialoGen and show that the relevance score of previous utterances agrees well with human cognition.


title: Use of Superfluid Helium to Observe Directionality of Galactic Dark Matter

abstract: The quasiparticle propagation away from the track of a highly ionizing particle in superfluid helium at low temperatures has previously been shown to exhibit anisotropy. We discuss the mechanism responsible for this behavior and show that it occurs for nuclear scattering by dark matter for recoil energies down to a few keV, and perhaps lower. This makes it possible to extend WIMP searches with interaction cross sections that reach into the neutrino floor in a meaningful energy range.


title: Visual Prompting for Adversarial Robustness

abstract: In this work, we leverage visual prompting (VP) to improve adversarial robustness of a fixed, pre-trained model at testing time. Compared to conventional adversarial defenses, VP allows us to design universal (i.e., data-agnostic) input prompting templates, which have plug-and-play capabilities at testing time to achieve desired model performance without introducing much computation overhead. Although VP has been successfully applied to improving model generalization, it remains elusive whether and how it can be used to defend against adversarial attacks. We investigate this problem and show that the vanilla VP approach is not effective in adversarial defense since a universal input prompt lacks the capacity for robust learning against sample-specific adversarial perturbations. To circumvent it, we propose a new VP method, termed Class-wise Adversarial Visual Prompting (C-AVP), to generate class-wise visual prompts so as to not only leverage the strengths of ensemble prompts but also optimize their interrelations to improve model robustness. Our experiments show that C-AVP outperforms the conventional VP method, with 2.1X standard accuracy gain and 2X robust accuracy gain. Compared to classical test-time defenses, C-AVP also yields a 42X inference time speedup.


title: Smart Cup: An impedance sensing based fluid intake monitoring system for beverages classification and freshness detection

abstract: This paper presents a novel beverage intake monitoring system that can accurately recognize beverage kinds and freshness. By mounting carbon electrodes on the commercial cup, the system measures the electrochemical impedance spectrum of the fluid in the cup. We studied the frequency sensitivity of the electrochemical impedance spectrum regarding distinct beverages and the importance of features like amplitude, phase, and real and imaginary components for beverage classification. The results show that features from a low-frequency domain (100 Hz to 1000 Hz) provide more meaningful information for beverage classification than the higher frequency domain. Twenty beverages, including carbonated drinks and juices, were classified with nearly perfect accuracy using a supervised machine learning approach. The same performance was also observed in the freshness recognition, where four different kinds of milk and fruit juice were studied.


title: Self-supervised Learning for Label-Efficient Sleep Stage Classification: A Comprehensive Evaluation

abstract: The past few years have witnessed a remarkable advance in deep learning for EEG-based sleep stage classification (SSC). However, the success of these models is attributed to possessing a massive amount of labeled data for training, limiting their applicability in real-world scenarios. In such scenarios, sleep labs can generate a massive amount of data, but labeling these data can be expensive and time-consuming. Recently, the self-supervised learning (SSL) paradigm has shined as one of the most successful techniques to overcome the scarcity of labeled data. In this paper, we evaluate the efficacy of SSL to boost the performance of existing SSC models in the few-labels regime. We conduct a thorough study on three SSC datasets, and we find that fine-tuning the pretrained SSC models with only 5% of labeled data can achieve competitive performance to the supervised training with full labels. Moreover, self-supervised pretraining helps SSC models to be more robust to data imbalance and domain shift problems. The code is publicly available at https://github.com/emadeldeen24/eval_ssl_ssc.


title: An Energy-Efficient Spiking Neural Network for Finger Velocity Decoding for Implantable Brain-Machine Interface

abstract: Brain-machine interfaces (BMIs) are promising for motor rehabilitation and mobility augmentation. High-accuracy and low-power algorithms are required to achieve implantable BMI systems. In this paper, we propose a novel spiking neural network (SNN) decoder for implantable BMI regression tasks. The SNN is trained with enhanced spatio-temporal backpropagation to fully leverage its ability in handling temporal problems. The proposed SNN decoder achieves the same level of correlation coefficient as the state-of-the-art ANN decoder in offline finger velocity decoding tasks, while it requires only 6.8% of the computation operations and 9.4% of the memory access.


title: fAux: Testing Individual Fairness via Gradient Alignment

abstract: Machine learning models are vulnerable to biases that result in unfair treatment of individuals from different populations. Recent work that aims to test a model's fairness at the individual level either relies on domain knowledge to choose metrics, or on input transformations that risk generating out-of-domain samples. We describe a new approach for testing individual fairness that does not have either requirement. We propose a novel criterion for evaluating individual fairness and develop a practical testing method based on this criterion which we call fAux (pronounced fox). This is based on comparing the derivatives of the predictions of the model to be tested with those of an auxiliary model, which predicts the protected variable from the observed data. We show that the proposed method effectively identifies discrimination on both synthetic and real-world datasets, and has quantitative and qualitative advantages over contemporary methods.


title: A Cooperative Perception System Robust to Localization Errors

abstract: Cooperative perception is challenging for safety-critical autonomous driving applications.The errors in the shared position and pose cause an inaccurate relative transform estimation and disrupt the robust mapping of the Ego vehicle. We propose a distributed object-level cooperative perception system called OptiMatch, in which the detected 3D bounding boxes and local state information are shared between the connected vehicles. To correct the noisy relative transform, the local measurements of both connected vehicles (bounding boxes) are utilized, and an optimal transport theory-based algorithm is developed to filter out those objects jointly detected by the vehicles along with their correspondence, constructing an associated co-visible set. A correction transform is estimated from the matched object pairs and further applied to the noisy relative transform, followed by global fusion and dynamic mapping. Experiment results show that robust performance is achieved for different levels of location and heading errors, and the proposed framework outperforms the state-of-the-art benchmark fusion schemes, including early, late, and intermediate fusion, on average precision by a large margin when location and/or heading errors occur.


title: The evolution of AI approaches for motor imagery EEG-based BCIs

abstract: The Motor Imagery (MI) electroencephalography (EEG) based Brain Computer Interfaces (BCIs) allow the direct communication between humans and machines by exploiting the neural pathways connected to motor imagination. Therefore, these systems open the possibility of developing applications that could span from the medical field to the entertainment industry. In this context, Artificial Intelligence (AI) approaches become of fundamental importance especially when wanting to provide a correct and coherent feedback to BCI users. Moreover, publicly available datasets in the field of MI EEG-based BCIs have been widely exploited to test new techniques from the AI domain. In this work, AI approaches applied to datasets collected in different years and with different devices but with coherent experimental paradigms are investigated with the aim of providing a concise yet sufficiently comprehensive survey on the evolution and influence of AI techniques on MI EEG-based BCI data.


title: ECG for high-throughput screening of multiple diseases: Proof-of-concept using multi-diagnosis deep learning from population-based datasets

abstract: Electrocardiogram (ECG) abnormalities are linked to cardiovascular diseases, but may also occur in other non-cardiovascular conditions such as mental, neurological, metabolic and infectious conditions. However, most of the recent success of deep learning (DL) based diagnostic predictions in selected patient cohorts have been limited to a small set of cardiac diseases. In this study, we use a population-based dataset of >250,000 patients with >1000 medical conditions and >2 million ECGs to identify a wide range of diseases that could be accurately diagnosed from the patient's first in-hospital ECG. Our DL models uncovered 128 diseases and 68 disease categories with strong discriminative performance.


title: A review on Epileptic Seizure Detection using Machine Learning

abstract: Epilepsy is a neurological brain disorder which life threatening and gives rise to recurrent seizures that are unprovoked. It occurs due to the abnormal chemical changes in our brain. Over the course of many years, studies have been conducted to support automatic diagnosis of epileptic seizures for the ease of clinicians. For that, several studies entail the use of machine learning methods for the early prediction of epileptic seizures. Mainly, feature extraction methods have been used to extract the right features from the EEG data generated by the EEG machine and then various machine learning classifiers are used for the classification process. This study provides a systematic literature review of feature selection process as well as the classification performance. This study was limited to the finding of most used feature extraction methods and the classifiers used for accurate classification of normal to epileptic seizures. The existing literature was examined from well-known repositories such as MPDI, IEEEXplore, Wiley, Elsevier, ACM, Springerlink and others. Furthermore, a taxonomy was created that recapitulates the state-of-the-art used solutions for this problem. We also studied the nature of different benchmark and unbiased datasets and gave a rigorous analysis of the working of classifiers. Finally, we concluded the research by presenting the gaps, challenges and opportunities which can further help researchers in prediction of epileptic seizure


title: Two-stream Network for ECG Signal Classification

abstract: Electrocardiogram (ECG), a technique for medical monitoring of cardiac activity, is an important method for identifying cardiovascular disease. However, analyzing the increasing quantity of ECG data consumes a lot of medical resources. This paper explores an effective algorithm for automatic classifications of multi-classes of heartbeat types based on ECG. Most neural network based methods target the individual heartbeats, ignoring the secrets embedded in the temporal sequence. And the ECG signal has temporal variation and unique individual characteristics, which means that the same type of ECG signal varies among patients under different physical conditions. A two-stream architecture is used in this paper and presents an enhanced version of ECG recognition based on this. The architecture achieves classification of holistic ECG signal and individual heartbeat and incorporates identified and temporal stream networks. Identified networks are used to extract features of individual heartbeats, while temporal networks aim to extract temporal correlations between heartbeats. Results on the MIT-BIH Arrhythmia Database demonstrate that the proposed algorithm performs an accuracy of 99.38\%. In addition, the proposed algorithm reaches an 88.07\% positive accuracy on massive data in real life, showing that the proposed algorithm can efficiently categorize different classes of heartbeat with high diagnostic performance.


title: Indoor Localization with Robust Global Channel Charting: A Time-Distance-Based Approach

abstract: Fingerprinting-based positioning significantly improves the indoor localization performance in non-line-of-sight-dominated areas. However, its deployment and maintenance is cost-intensive as it needs ground-truth reference systems for both the initial training and the adaption to environmental changes. In contrast, channel charting (CC) works without explicit reference information and only requires the spatial correlations of channel state information (CSI). While CC has shown promising results in modelling the geometry of the radio environment, a deeper insight into CC for localization using multi-anchor large-bandwidth measurements is still pending. We contribute a novel distance metric for time-synchronized single-input/single-output CSIs that approaches a linear correlation to the Euclidean distance. This allows to learn the environment's global geometry without annotations. To efficiently optimize the global channel chart we approximate the metric with a Siamese neural network. This enables full CC-assisted fingerprinting and positioning only using a linear transformation from the chart to the real-world coordinates. We compare our approach to the state-of-the-art of CC on two different real-world data sets recorded with a 5G and UWB radio setup. Our approach outperforms others with localization accuracies of 0.69m for the UWB and 1.4m for the 5G setup. We show that CC-assisted fingerprinting enables highly accurate localization and reduces (or eliminates) the need for annotated training data.


title: Transfer Learning on Electromyography (EMG) Tasks: Approaches and Beyond

abstract: Machine learning on electromyography (EMG) has recently achieved remarkable success on a variety of tasks, while such success relies heavily on the assumption that the training and future data must be of the same data distribution. However, this assumption may not hold in many real-world applications. Model calibration is required via data re-collection and label annotation, which is generally very expensive and time-consuming. To address this problem, transfer learning (TL), which aims to improve target learners' performance by transferring the knowledge from related source domains, is emerging as a new paradigm to reduce the amount of calibration effort. In this survey, we assess the eligibility of more than fifty published peer-reviewed representative transfer learning approaches for EMG applications. Unlike previous surveys on purely transfer learning or EMG-based machine learning, this survey aims to provide an insight into the biological foundations of existing transfer learning methods on EMG-related analysis. In specific, we first introduce the physiological structure of the muscles and the EMG generating mechanism, and the recording of EMG to provide biological insights behind existing transfer learning approaches. Further, we categorize existing research endeavors into data based, model based, training scheme based, and adversarial based. This survey systematically summarizes and categorizes existing transfer learning approaches for EMG related machine learning applications. In addition, we discuss possible drawbacks of existing works and point out the future direction of better EMG transfer learning algorithms to enhance practicality for real-world applications.


title: Transmission of high-definition video signals underwater using surface electromagnetic waves

abstract: A portable radio communication system operating in the 30 MHz band and capable of transmitting high-definition live underwater video images is presented. The system operation is based on launching electromagnetic surface waves propagating along water-air interface using specially designed surface wave antennas. Since the propagation length of the surface electromagnetic waves far exceeds the skin depth of bulk radio waves at the same frequency, this technique is useful for video communication underwater over distances of several meters. Also, this system appears to be efficient at communicating through the water-air interface.


title: Multimodality Multi-Lead ECG Arrhythmia Classification using Self-Supervised Learning

abstract: Electrocardiogram (ECG) signal is one of the most effective sources of information mainly employed for the diagnosis and prediction of cardiovascular diseases (CVDs) connected with the abnormalities in heart rhythm. Clearly, single modality ECG (i.e. time series) cannot convey its complete characteristics, thus, exploiting both time and time-frequency modalities in the form of time-series data and spectrogram is needed. Leveraging the cutting-edge self-supervised learning (SSL) technique on unlabeled data, we propose SSL-based multimodality ECG classification. Our proposed network follows SSL learning paradigm and consists of two modules corresponding to pre-stream task, and down-stream task, respectively. In the SSL-pre-stream task, we utilize self-knowledge distillation (KD) techniques with no labeled data, on various transformations and in both time and frequency domains. In the down-stream task, which is trained on labeled data, we propose a gate fusion mechanism to fuse information from multimodality.To evaluate the effectiveness of our approach, ten-fold cross validation on the 12-lead PhysioNet 2020 dataset has been conducted.


title: Cross Task Neural Architecture Search for EEG Signal Classifications

abstract: Electroencephalograms (EEGs) are brain dynamics measured outside the brain, which have been widely utilized in non-invasive brain-computer interface applications. Recently, various neural network approaches have been proposed to improve the accuracy of EEG signal recognition. However, these approaches severely rely on manually designed network structures for different tasks which generally are not sharing the same empirical design cross-task-wise. In this paper, we propose a cross-task neural architecture search (CTNAS-EEG) framework for EEG signal recognition, which can automatically design the network structure across tasks and improve the recognition accuracy of EEG signals. Specifically, a compatible search space for cross-task searching and an efficient constrained searching method is proposed to overcome challenges brought by EEG signals. By unifying structure search on different EEG tasks, this work is the first to explore and analyze the searched structure difference cross-task-wise. Moreover, by introducing architecture search, this work is the first to analyze model performance by customizing model structure for each human subject. Detailed experimental results suggest that the proposed CTNAS-EEG could reach state-of-the-art performance on different EEG tasks, such as Motor Imagery (MI) and Emotion recognition. Extensive experiments and detailed analysis are provided as a good reference for follow-up researchers.


title: SeKron: A Decomposition Method Supporting Many Factorization Structures

abstract: While convolutional neural networks (CNNs) have become the de facto standard for most image processing and computer vision applications, their deployment on edge devices remains challenging. Tensor decomposition methods provide a means of compressing CNNs to meet the wide range of device constraints by imposing certain factorization structures on their convolution tensors. However, being limited to the small set of factorization structures presented by state-of-the-art decomposition approaches can lead to sub-optimal performance. We propose SeKron, a novel tensor decomposition method that offers a wide variety of factorization structures, using sequences of Kronecker products. By recursively finding approximating Kronecker factors, we arrive at optimal decompositions for each of the factorization structures. We show that SeKron is a flexible decomposition that generalizes widely used methods, such as Tensor-Train (TT), Tensor-Ring (TR), Canonical Polyadic (CP) and Tucker decompositions. Crucially, we derive an efficient convolution projection algorithm shared by all SeKron structures, leading to seamless compression of CNN models. We validate SeKron for model compression on both high-level and low-level computer vision tasks and find that it outperforms state-of-the-art decomposition methods.


title: Generalised Mutual Information for Discriminative Clustering

abstract: In the last decade, recent successes in deep clustering majorly involved the mutual information (MI) as an unsupervised objective for training neural networks with increasing regularisations. While the quality of the regularisations have been largely discussed for improvements, little attention has been dedicated to the relevance of MI as a clustering objective. In this paper, we first highlight how the maximisation of MI does not lead to satisfying clusters. We identified the Kullback-Leibler divergence as the main reason of this behaviour. Hence, we generalise the mutual information by changing its core distance, introducing the generalised mutual information (GEMINI): a set of metrics for unsupervised neural network training. Unlike MI, some GEMINIs do not require regularisations when training. Some of these metrics are geometry-aware thanks to distances or kernels in the data space. Finally, we highlight that GEMINIs can automatically select a relevant number of clusters, a property that has been little studied in deep clustering context where the number of clusters is a priori unknown.

categories: stat.ML cs.AI cs.IT cs.LG math.IT stat.ME


title: FontTransformer: Few-shot High-resolution Chinese Glyph Image Synthesis via Stacked Transformers

abstract: Automatic generation of high-quality Chinese fonts from a few online training samples is a challenging task, especially when the amount of samples is very small. Existing few-shot font generation methods can only synthesize low-resolution glyph images that often possess incorrect topological structures or/and incomplete strokes. To address the problem, this paper proposes FontTransformer, a novel few-shot learning model, for high-resolution Chinese glyph image synthesis by using stacked Transformers. The key idea is to apply the parallel Transformer to avoid the accumulation of prediction errors and utilize the serial Transformer to enhance the quality of synthesized strokes. Meanwhile, we also design a novel encoding scheme to feed more glyph information and prior knowledge to our model, which further enables the generation of high-resolution and visually-pleasing glyph images. Both qualitative and quantitative experimental results demonstrate the superiority of our method compared to other existing approaches in the few-shot Chinese font synthesis task.


title: Maximum entropy exploration in contextual bandits with neural networks and energy based models

abstract: Contextual bandits can solve a huge range of real-world problems. However, current popular algorithms to solve them either rely on linear models, or unreliable uncertainty estimation in non-linear models, which are required to deal with the exploration-exploitation trade-off. Inspired by theories of human cognition, we introduce novel techniques that use maximum entropy exploration, relying on neural networks to find optimal policies in settings with both continuous and discrete action spaces. We present two classes of models, one with neural networks as reward estimators, and the other with energy based models, which model the probability of obtaining an optimal reward given an action. We evaluate the performance of these models in static and dynamic contextual bandit simulation environments. We show that both techniques outperform well-known standard algorithms, where energy based models have the best overall performance. This provides practitioners with new techniques that perform well in static and dynamic settings, and are particularly well suited to non-linear scenarios with continuous action spaces.


title: Dynamic neuronal networks efficiently achieve classification in robotic interactions with real-world objects

abstract: Biological cortical networks are potentially fully recurrent networks without any distinct output layer, where recognition may instead rely on the distribution of activity across its neurons. Because such biological networks can have rich dynamics, they are well-designed to cope with dynamical interactions of the types that occur in nature, while traditional machine learning networks may struggle to make sense of such data. Here we connected a simple model neuronal network (based on the 'linear summation neuron model' featuring biologically realistic dynamics (LSM), consisting of 10 of excitatory and 10 inhibitory neurons, randomly connected) to a robot finger with multiple types of force sensors when interacting with materials of different levels of compliance. Scope: to explore the performance of the network on classification accuracy. Therefore, we compared the performance of the network output with principal component analysis of statistical features of the sensory data as well as its mechanical properties. Remarkably, even though the LSM was a very small and untrained network, and merely designed to provide rich internal network dynamics while the neuron model itself was highly simplified, we found that the LSM outperformed these other statistical approaches in terms of accuracy.


title: Development of a Cobalt Electrochemical Sensor for Measuring Phosphate in Municipal Wastewaters

abstract: The introduction of the Water Framework directive sets stringent limits on phosphorous discharge from wastewater treatment plants to maintain the complex interdependent relationship between water tributaries and the ecosystem. This paper studies a cobalt based electrochemical sensor for phosphate detection in wastewater. An evaluation of the sensors operational envelope, impact of pH, detection limits, linearity of response, accuracy and reproducibility in a single ion solution was conducted. An indirect method was employed to assess the effect of all of these parameters; the parameter was kept constant, while the phosphate concertation was varied. Tests on real wastewater samples verified the effect of the interfering factors, as phosphate measurements from three different sampling points (influent, activated sludge mixed liquors and effluent) did not correlate favourably with measurements acquired from a specialised laboratory. The success of this sensor is probably dependent on the simultaneous measurement of, or the calibration for, interfering parameters. However, the former approach would most likely require additional probes to measure these interfering parameters and the latter would probably require a complex calibrating matrix to account for all the interfering parameters. Nonetheless, variations of such sensors reviewed in this paper and their encouraging results offer an optimistic field of improvement on the design of the sensor studied in this paper for it to be employed on real wastewater systems.


title: Steering of Quantum Walks through Coherent Control of High-dimensional Bi-photon Quantum Frequency Combs with Tunable State Entropies

abstract: Quantum walks are central to a wide range of applications such as quantum search, quantum information processing, and entanglement transport. Gaining control over the duration and the direction of quantum walks (QWs) is crucial to implementing dedicated processing. However, in current systems, it is cumbersome to achieve in a scalable format. High-dimensional quantum states, encoded in the photons' frequency degree of freedom in on-chip devices are great assets for the scalable generation and reliable manipulation of large-scale complex quantum systems. These states, viz. quantum frequency combs (QFCs) accommodating huge information in a single spatial mode, are intrinsically noise tolerant, and suitable for transmission through optical fibers, thereby promising to revolutionize quantum technologies. Existing literature aimed to generate maximally entangled QFCs excited from continuous-wave lasers either from nonlinear microcavities or from waveguides with the help of filter arrays. QWs with flexible depth/duration have been lately demonstrated from such QFCs. Here, instead of maximally-entangled QFCs, we generate high-dimensional quantum photonic states with tunable entropies from periodically poled lithium niobate waveguides by exploiting a novel pulsed excitation and filtering scheme. We confirm the generation of QFCs with normalized entropies from $\sim 0.35$ to $1$ by performing quantum tomography with high fidelities. These states can be an excellent testbed for several quantum computation and communication protocols in nonideal scenarios and enable artificial neural networks to classify unknown quantum states. Further, we experimentally demonstrate the steering and coherent control of the directionality of QWs initiated from such QFCs with tunable entropies. Our findings offer a new control mechanism for QWs as well as novel modification means for joint probability distributions.


title: Thermal disruption of a Luttinger liquid

abstract: The Tomonaga-Luttinger liquid (TLL) theory describes the low-energy excitations of strongly correlated one-dimensional (1D) fermions. In the past years, a number of studies have provided a detailed understanding of this universality class. More recently, theoretical investigations that go beyond the standard low-temperature, linear-response, TLL regime have been developed. While these provide a basis for understanding the dynamics of the spin-incoherent Luttinger liquid, there are few experimental investigations in this regime. Here we report the observation of a thermally-induced, spin-incoherent Luttinger liquid in a $^6$Li atomic Fermi gas confined to 1D. We use Bragg spectroscopy to measure the suppression of spin-charge separation and the decay of correlations as the temperature is increased. Our results probe the crossover between the coherent and incoherent regimes of the Luttinger liquid, and elucidate the roles of the charge and the spin degrees of freedom in this regime.


title: DinoDroid: Testing Android Apps Using Deep Q-Networks

abstract: The large demand of mobile devices creates significant concerns about the quality of mobile applications (apps). Developers need to guarantee the quality of mobile apps before it is released to the market. There have been many approaches using different strategies to test the GUI of mobile apps. However, they still need improvement due to their limited effectiveness. In this paper, we propose DinoDroid, an approach based on deep Q-networks to automate testing of Android apps. DinoDroid learns a behavior model from a set of existing apps and the learned model can be used to explore and generate tests for new apps. DinoDroid is able to capture the fine-grained details of GUI events (e.g., the content of GUI widgets) and use them as features that are fed into deep neural network, which acts as the agent to guide app exploration. DinoDroid automatically adapts the learned model during the exploration without the need of any modeling strategies or pre-defined rules. We conduct experiments on 64 open-source Android apps. The results showed that DinoDroid outperforms existing Android testing tools in terms of code coverage and bug detection.


title: Time-dependent exchange creates the time-frustrated state of matter

abstract: Magnetic systems governed by exchange interactions between magnetic moments harbor frustration that leads to ground state degeneracy and results in the new topological state often referred to as a frustrated state of matter (FSM). The frustration in the commonly discussed magnetic systems has a spatial origin. Here we demonstrate that an array of nanomagnets coupled by the real retarded exchange interactions develops a new state of matter, time frustrated matter (TFM). In a spin system with the time-dependent retarded exchange interaction, a single spin-flip influences other spins not instantly but after some delay. This implies that the sign of the exchange interaction changes, leading to either ferro- or antiferromagnetic interaction, depends on time. As a result, the system's temporal evolution is essentially non-Markovian. The emerging competition between different magnetic orders leads to a new kind of time-core frustration. To establish this paradigmatic shift, we focus on the exemplary system, a granular multiferroic, where the exchange transferring medium has a pronounced frequency dispersion and hence develops the TFM.


title: On-Demand Entanglement of Molecules in a Reconfigurable Optical Tweezer Array

abstract: Entanglement is crucial to many quantum applications including quantum information processing, simulation of quantum many-body systems, and quantum-enhanced sensing. Molecules, because of their rich internal structure and interactions, have been proposed as a promising platform for quantum science. Deterministic entanglement of individually controlled molecules has nevertheless been a long-standing experimental challenge. Here we demonstrate, for the first time, on-demand entanglement of individually prepared molecules. Using the electric dipolar interaction between pairs of molecules prepared using a reconfigurable optical tweezer array, we realize an entangling two-qubit gate, and use it to deterministically create Bell pairs. Our results demonstrate the key building blocks needed for quantum information processing, simulation of quantum spin models, and quantum-enhanced sensing. They also open up new possibilities such as using trapped molecules for quantum-enhanced fundamental physics tests and exploring collisions and chemical reactions with entangled matter.


title: Deep learning extraction of band structure parameters from density of states: a case study on trilayer graphene

abstract: The development of two-dimensional materials has resulted in a diverse range of novel, high-quality compounds with increasing complexity. A key requirement for a comprehensive quantitative theory is the accurate determination of these materials' band structure parameters. However, this task is challenging due to the intricate band structures and the indirect nature of experimental probes. In this work, we introduce a general framework to derive band structure parameters from experimental data using deep neural networks. We applied our method to the penetration field capacitance measurement of trilayer graphene, an effective probe of its density of states. First, we demonstrate that a trained deep network gives accurate predictions for the penetration field capacitance as a function of tight-binding parameters. Next, we use the fast and accurate predictions from the trained network to automatically determine tight-binding parameters directly from experimental data, with extracted parameters being in a good agreement with values in the literature. We conclude by discussing potential applications of our method to other materials and experimental techniques beyond penetration field capacitance.


title: Semantic Cross Attention for Few-shot Learning

abstract: Few-shot learning (FSL) has attracted considerable attention recently. Among existing approaches, the metric-based method aims to train an embedding network that can make similar samples close while dissimilar samples as far as possible and achieves promising results. FSL is characterized by using only a few images to train a model that can generalize to novel classes in image classification problems, but this setting makes it difficult to learn the visual features that can identify the images' appearance variations. The model training is likely to move in the wrong direction, as the images in an identical semantic class may have dissimilar appearances, whereas the images in different semantic classes may share a similar appearance. We argue that FSL can benefit from additional semantic features to learn discriminative feature representations. Thus, this study proposes a multi-task learning approach to view semantic features of label text as an auxiliary task to help boost the performance of the FSL task. Our proposed model uses word-embedding representations as semantic features to help train the embedding network and a semantic cross-attention module to bridge the semantic features into the typical visual modal. The proposed approach is simple, but produces excellent results. We apply our proposed approach to two previous metric-based FSL methods, all of which can substantially improve performance. The source code for our model is accessible from github.


title: Changing the Representation: Examining Language Representation for Neural Sign Language Production

abstract: Neural Sign Language Production (SLP) aims to automatically translate from spoken language sentences to sign language videos. Historically the SLP task has been broken into two steps; Firstly, translating from a spoken language sentence to a gloss sequence and secondly, producing a sign language video given a sequence of glosses. In this paper we apply Natural Language Processing techniques to the first step of the SLP pipeline. We use language models such as BERT and Word2Vec to create better sentence level embeddings, and apply several tokenization techniques, demonstrating how these improve performance on the low resource translation task of Text to Gloss. We introduce Text to HamNoSys (T2H) translation, and show the advantages of using a phonetic representation for sign language translation rather than a sign level gloss representation. Furthermore, we use HamNoSys to extract the hand shape of a sign and use this as additional supervision during training, further increasing the performance on T2H. Assembling best practise, we achieve a BLEU-4 score of 26.99 on the MineDGS dataset and 25.09 on PHOENIX14T, two new state-of-the-art baselines.


title: The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers

abstract: This paper studies the curious phenomenon for machine learning models with Transformer architectures that their activation maps are sparse. By activation map we refer to the intermediate output of the multi-layer perceptrons (MLPs) after a ReLU activation function, and by sparse we mean that on average very few entries (e.g., 3.0% for T5-Base and 6.3% for ViT-B16) are nonzero for each input to MLP. Moreover, larger Transformers with more layers and wider MLP hidden dimensions are sparser as measured by the percentage of nonzero entries. Through extensive experiments we demonstrate that the emergence of sparsity is a prevalent phenomenon that occurs for both natural language processing and vision tasks, on both training and evaluation data, for Transformers of various configurations, at layers of all depth levels, as well as for other architectures including MLP-mixers and 2-layer MLPs. We show that sparsity also emerges using training datasets with random labels, or with random inputs, or with infinite amount of data, demonstrating that sparsity is not a result of a specific family of datasets. We discuss how sparsity immediately implies a way to significantly reduce the FLOP count and improve efficiency for Transformers. Moreover, we demonstrate perhaps surprisingly that enforcing an even sparser activation via Top-k thresholding with a small value of k brings a collection of desired but missing properties for Transformers, namely less sensitivity to noisy training data, more robustness to input corruptions, and better calibration for their prediction confidence.


title: Sliding, vibrating and swinging droplets on an oscillating fibre

abstract: We study experimentally the dynamics of a water droplet on a tilted and vertically oscillating rigid fibre. As we vary the frequency and amplitude of the oscillations the droplet transitions between different modes: harmonic pumping, subharmonic pumping, a combination of rocking and pumping modes, and a combination of pumping and swinging modes. We characterise these responses and report how they affect the droplet's sliding speed along the fibre. The droplet swinging mode is explained by a minimal model making an analogy between the droplet and a forced elastic pendulum.


title: Approximate formulas for the energy release rate of a crack perpendicular to a material interface

abstract: Rock formations are very often characterized by the presence of fractures that have grown subcritically over geological time scales and under evolving stress fields. In mechanically layered systems, such fractures can either become layer-bound or penetrate into adjacent strata. The growth of fractures in brittle materials is generally dependent on the energy release rate, however no closed form analytical solutions exist for these when a crack tip is in the proximity of a material interface. In this study, we present new empirical formulas for calculating the energy release rate at the tip of a crack perpendicular to a material interface in a symmetric 3-layer system. In these formulas, the normalized energy release rate is expressed as the product of a base term that integrates the normalized stiffness modulus over the crack length, and a correction factor that accounts for the presence of a material interface. The latter is assumed to be dependent on two quantities: the ratio of the crack length to the inner layer thickness, and the contrast in material stiffness between the inner and outer layers. The correction factors are obtained by fitting the parameters of carefully chosen expressions to a set of finite element solutions in order to yield predictions that are accurate to within one percent of the numerical results.


title: Non-Axiomatic Term Logic: A Computational Theory of Cognitive Symbolic Reasoning

abstract: This paper presents Non-Axiomatic Term Logic (NATL) as a theoretical computational framework of humanlike symbolic reasoning in artificial intelligence. NATL unites a discrete syntactic system inspired from Aristotle's term logic and a continuous semantic system based on the modern idea of distributed representations, or embeddings. This paper positions the proposed approach in the phylogeny and the literature of logic, and explains the framework. As it is yet no more than a theory and it requires much further elaboration to implement it, no quantitative evaluation is presented. Instead, qualitative analyses of arguments using NATL, some applications to possible cognitive science/robotics-related research, and remaining issues towards a machinery implementation are discussed.


title: A local-global principle for polyquadratic twists of abelian surfaces

abstract: We say that two abelian varieties $A$ and $A'$ defined over a field $F$ are polyquadratic twists if they are isogenous over a Galois extension of $F$ whose Galois group has exponent dividing $2$. Let $A$ and $A'$ be abelian varieties defined over a number field $K$ of dimension $g\geq 1$. In this article we prove that, if $g\leq 2$, then $A$ and $A'$ are polyquadratic twists if and only if for almost all primes $\p$ of $K$ their reductions modulo $\p$ are polyquadratic twists. We exhibit a counterexample to this local-global principle for $g=3$. This work builds on a geometric analogue by Khare and Larsen, and on a similar criterion for quadratic twists established by Fit\'e, relying itself on the works by Rajan and Ramakrishnan.


title: Lipschitz continuity results for a class of obstacle problems

abstract: We prove Lipschitz continuity results for solutions to a class of obstacle problems under standard growth conditions of $p$-type, $p \geq 2$. The main novelty is the use of a linearization technique going back to [28] in order to interpret our constrained minimizer as a solution to a nonlinear elliptic equation, with a bounded right-hand side. This leads us to start a Moser iteration scheme which provides the $L^\infty$ bound for the gradient. The application of a recent higher differentiability result [24] allows us to simplify the procedure of the identification of the Radon measure in the linearization technique employed in [32]. To our knowledge, this is the first result for non-autonomous functionals with standard growth conditions in the direction of the Lipschitz regularity.


title: Supersymmetric solitons in gauged $\mathcal{N}=8$ supergravity

abstract: We consider soliton solutions in AdS$_{4}$ with a flat slicing and Wilson loops around one cycle. We study the phase structure and find the ground state and identify supersymmetric solutions as a function of the Wilson loops. We work in the context of a scalar field truncation of gauged $\mathcal{N}=8$ supergravity, where all the dilatons are equal and all the axions vanish in the STU model. In this theory, we construct new soliton solutions parameterized by two Wilson lines. We find that there is a degeneracy of supersymmetric solutions. We also show that, for alternate boundary conditions, there exists a non-supersymmetric soliton solution with energy lower than the supersymmetric one.


title: Cosmic evolution and thermal stability of Barrow holographic dark energy in nonflat Friedmann-Robertson-Walker Universe

abstract: We study the cosmological evolution of a nonflat Friedmann-Robertson-Walker Universe filled by pressureless dark matter and Barrow Holographic Dark Energy (BHDE). The latter is a dark energy model based on the holographic principle with Barrow entropy instead of the standard Bekenstein-Hawking one. By assuming the apparent horizon of the Universe as IR cutoff, we explore both the cases where a mutual interaction between the dark components of the cosmos is absent/present. We analyze the behavior of various model parameters, such as the BHDE density parameter, the equation of state parameter, the deceleration parameter, the jerk parameter and the square of sound speed. We also comment on the observational consistency of our predictions, showing that the interacting model turns out to be favored by recent experimental constraints from Planck+WP+BAO, SNIa+CMB+LSS and Union2 SNIa joint analysis over the non-interacting one. The thermal stability of our framework is finally discussed by demanding the positivity of the heat capacities and compressibilities of the cosmic fluid.


title: Smooth solutions of a class of iterative functional equations

abstract: Imposing some conditions on derivatives of the known functions, using the Fiber Contraction Theorem we prove the existence of $C^1$ solutions of a class of iterative functional equations which involves iterates of the unknown functions and a nonlinear term.


title: Realization of an atomic quantum Hall system in four dimensions

abstract: Modern condensed matter physics relies on the concept of topology to classify matter, from quantum Hall systems to topological insulators. Engineered systems, benefiting from synthetic dimensions, can potentially give access to novel topological states predicted in dimensions $D > 3$. We report the realization of an atomic quantum Hall system evolving in four dimensions (4D), with two spatial dimensions and two synthetic ones encoded in the large spin of dysprosium atoms. The non-trivial topology is evidenced by measuring a quantized electromagnetic non-linear response and observing anisotropic hyperedge modes. We also excite non-planar cyclotron motion, contrasting with its circular equivalents in $D\leq3$. Our work opens to the investigation of strongly-correlated topological liquids in 4D generalizing fractional quantum Hall states.


title: AISFormer: Amodal Instance Segmentation with Transformer

abstract: Amodal Instance Segmentation (AIS) aims to segment the region of both visible and possible occluded parts of an object instance. While Mask R-CNN-based AIS approaches have shown promising results, they are unable to model high-level features coherence due to the limited receptive field. The most recent transformer-based models show impressive performance on vision tasks, even better than Convolution Neural Networks (CNN). In this work, we present AISFormer, an AIS framework, with a Transformer-based mask head. AISFormer explicitly models the complex coherence between occluder, visible, amodal, and invisible masks within an object's regions of interest by treating them as learnable queries. Specifically, AISFormer contains four modules: (i) feature encoding: extract ROI and learn both short-range and long-range visual features. (ii) mask transformer decoding: generate the occluder, visible, and amodal mask query embeddings by a transformer decoder (iii) invisible mask embedding: model the coherence between the amodal and visible masks, and (iv) mask predicting: estimate output masks including occluder, visible, amodal and invisible. We conduct extensive experiments and ablation studies on three challenging benchmarks i.e. KINS, D2SA, and COCOA-cls to evaluate the effectiveness of AISFormer. The code is available at: https://github.com/UARK-AICV/AISFormer


title: SQuId: Measuring Speech Naturalness in Many Languages

abstract: Much of text-to-speech research relies on human evaluation, which incurs heavy costs and slows down the development process. The problem is particularly acute in heavily multilingual applications, where recruiting and polling judges can take weeks. We introduce SQuId (Speech Quality Identification), a multilingual naturalness prediction model trained on over a million ratings and tested in 65 locales-the largest effort of this type to date. The main insight is that training one model on many locales consistently outperforms mono-locale baselines. We present our task, the model, and show that it outperforms a competitive baseline based on w2v-BERT and VoiceMOS by 50.0%. We then demonstrate the effectiveness of cross-locale transfer during fine-tuning and highlight its effect on zero-shot locales, i.e., locales for which there is no fine-tuning data. Through a series of analyses, we highlight the role of non-linguistic effects such as sound artifacts in cross-locale transfer. Finally, we present the effect of our design decision, e.g., model size, pre-training diversity, and language rebalancing with several ablation experiments.


title: Fundamental effective temperature measurements for eclipsing binary stars -- II. The detached F-type eclipsing binary CPD-54 810

abstract: CPD-54 810 is a double-lined detached eclipsing binary containing two mid-F type dwarfs on an eccentric 26-day orbit. We perform a combined analysis of the extensive photometry obtained by the TESS space mission along with previously published observations to obtain a full orbital and physical solution for the system. We measure the following model-independent masses and radii: M1 = 1.3094+/-0.0051 Msun, M2 = 1.0896+/-0.0034 Msun, R1 = 1.9288+/-0.0030 Rsun, and R2 = 1.1815+/-0.0037 Rsun. We employ a Bayesian approach to obtain the bolometric flux for both stars from observed magnitudes, colours, and flux ratios. These bolometric fluxes combined with the stars' angular diameters (from R1, R2 and the parallax from Gaia EDR3) lead directly to the stars' effective temperatures: Teff,1 = 6462+/-43 K, and Teff,2 = 6331+/-43 K, with an additional systematic error of 0.8% (13 K) from the uncertainty in the zero-point of the flux scale. Our results are robust against the choice of model spectra and other details of the analysis. CPD-54 810 is an ideal benchmark system that can be used to test stellar parameters measured by large spectroscopic surveys or derived from asteroseismology, and calibrate stellar models by providing robust constraints on the measured parameters. The methods presented here can be applied to many other detached eclipsing binary systems to build a catalogue of well-measured benchmark stars.


title: Optical nanoresonators

abstract: The review presents an analysis and generalization of classical and most modern approaches to the description and development of operation of open optical nanoresonators, that is, resonators all sizes of which are smaller than the resonant wavelength of radiation in a vacuum. Particular attention is paid to the physics of such phenomena as bound states in a continuum, anapole states, supercavity modes, and perfect nonradiating modes with extremely high quality factors and localizations of electromagnetic fields. An analysis of the optical properties of natural oscillations in nanoresonators made of metamaterials is also presented in the review. The effects considered in this review, besides being of fundamental import can also find applications in the development of optical nanoantennas, nanolasers, biosensors, photovoltaic devices, and nonlinear nanophotonics.


title: Betting the system: Using lineups to predict football scores

abstract: This paper aims to reduce randomness in football by analysing the role of lineups in final scores using machine learning prediction models we have developed. Football clubs invest millions of dollars on lineups and knowing how individual statistics translate to better outcomes can optimise investments. Moreover, sports betting is growing exponentially and being able to predict the future is profitable and desirable. We use machine learning models and historical player data from English Premier League (2020-2022) to predict scores and to understand how individual performance can improve the outcome of a match. We compared different prediction techniques to maximise the possibility of finding useful models. We created heuristic and machine learning models predicting football scores to compare different techniques. We used different sets of features and shown goalkeepers stats are more important than attackers stats to predict goals scored. We applied a broad evaluation process to assess the efficacy of the models in real world applications. We managed to predict correctly all relegated teams after forecast 100 consecutive matches. We show that Support Vector Regression outperformed other techniques predicting final scores and that lineups do not improve predictions. Finally, our model was profitable (42% return) when emulating a betting system using real world odds data.


title: Momentum Aggregation for Private Non-convex ERM

abstract: We introduce new algorithms and convergence guarantees for privacy-preserving non-convex Empirical Risk Minimization (ERM) on smooth $d$-dimensional objectives. We develop an improved sensitivity analysis of stochastic gradient descent on smooth objectives that exploits the recurrence of examples in different epochs. By combining this new approach with recent analysis of momentum with private aggregation techniques, we provide an $(\epsilon,\delta)$-differential private algorithm that finds a gradient of norm $\tilde O\left(\frac{d^{1/3}}{(\epsilon N)^{2/3}}\right)$ in $O\left(\frac{N^{7/3}\epsilon^{4/3}}{d^{2/3}}\right)$ gradient evaluations, improving the previous best gradient bound of $\tilde O\left(\frac{d^{1/4}}{\sqrt{\epsilon N}}\right)$.


title: Homogenization Theory of Elliptic System with Lower Order Terms for Dimension Two

abstract: In this paper, we consider the homogenization problem for generalized elliptic systems $$ \mathcal{L}_{\varepsilon}=-\operatorname{div}(A(x/\varepsilon)\nabla+V(x/\varepsilon))+B(x/\varepsilon)\nabla+c(x/\varepsilon)+\lambda I $$ with dimension two. Precisely, we will establish the $ W^{1,p} $ estimates, H\"{o}lder estimates, Lipschitz estimates and $ L^p $ convergence results for $ \mathcal{L}_{\varepsilon} $ with dimension two. The operator $ \mathcal{L}_{\varepsilon} $ has been studied by Qiang Xu with dimension $ d\geq 3 $ in \cite{Xu1,Xu2} and the case $ d=2 $ is remained unsolved. As a byproduct, we will construct the Green functions for $ \mathcal{L}_{\varepsilon} $ with $ d=2 $ and their convergence rates.


title: CoRRECT: A Deep Unfolding Framework for Motion-Corrected Quantitative R2* Mapping

abstract: Quantitative MRI (qMRI) refers to a class of MRI methods for quantifying the spatial distribution of biological tissue parameters. Traditional qMRI methods usually deal separately with artifacts arising from accelerated data acquisition, involuntary physical motion, and magnetic-field inhomogeneities, leading to suboptimal end-to-end performance. This paper presents CoRRECT, a unified deep unfolding (DU) framework for qMRI consisting of a model-based end-to-end neural network, a method for motion-artifact reduction, and a self-supervised learning scheme. The network is trained to produce R2* maps whose k-space data matches the real data by also accounting for motion and field inhomogeneities. When deployed, CoRRECT only uses the k-space data without any pre-computed parameters for motion or inhomogeneity correction. Our results on experimentally collected multi-Gradient-Recalled Echo (mGRE) MRI data show that CoRRECT recovers motion and inhomogeneity artifact-free R2* maps in highly accelerated acquisition settings. This work opens the door to DU methods that can integrate physical measurement models, biophysical signal models, and learned prior models for high-quality qMRI.


title: RedHOT: A Corpus of Annotated Medical Questions, Experiences, and Claims on Social Media

abstract: We present Reddit Health Online Talk (RedHOT), a corpus of 22,000 richly annotated social media posts from Reddit spanning 24 health conditions. Annotations include demarcations of spans corresponding to medical claims, personal experiences, and questions. We collect additional granular annotations on identified claims. Specifically, we mark snippets that describe patient Populations, Interventions, and Outcomes (PIO elements) within these. Using this corpus, we introduce the task of retrieving trustworthy evidence relevant to a given claim made on social media. We propose a new method to automatically derive (noisy) supervision for this task which we use to train a dense retrieval model; this outperforms baseline models. Manual evaluation of retrieval results performed by medical doctors indicate that while our system performance is promising, there is considerable room for improvement. Collected annotations (and scripts to assemble the dataset), are available at https://github.com/sominw/redhot.


title: ViewBirdiformer: Learning to recover ground-plane crowd trajectories and ego-motion from a single ego-centric view

abstract: We introduce a novel learning-based method for view birdification, the task of recovering ground-plane trajectories of pedestrians of a crowd and their observer in the same crowd just from the observed ego-centric video. View birdification becomes essential for mobile robot navigation and localization in dense crowds where the static background is hard to see and reliably track. It is challenging mainly for two reasons; i) absolute trajectories of pedestrians are entangled with the movement of the observer which needs to be decoupled from their observed relative movements in the ego-centric video, and ii) a crowd motion model describing the pedestrian movement interactions is specific to the scene yet unknown a priori. For this, we introduce a Transformer-based network referred to as ViewBirdiformer which implicitly models the crowd motion through self-attention and decomposes relative 2D movement observations onto the ground-plane trajectories of the crowd and the camera through cross-attention between views. Most important, ViewBirdiformer achieves view birdification in a single forward pass which opens the door to accurate real-time, always-on situational awareness. Extensive experimental results demonstrate that ViewBirdiformer achieves accuracy similar to or better than state-of-the-art with three orders of magnitude reduction in execution time.


title: Pattern Characterization Using Topological Data Analysis: Application to Piezo Vibration Striking Treatment

abstract: Quantifying patterns in visual or tactile textures provides important information about the process or phenomena that generated these patterns. In manufacturing, these patterns can be intentionally introduced as a design feature, or they can be a byproduct of a specific process. Since surface texture has significant impact on the mechanical properties and the longevity of the workpiece, it is important to develop tools for quantifying surface patterns and, when applicable, comparing them to their nominal counterparts. While existing tools may be able to indicate the existence of a pattern, they typically do not provide more information about the pattern structure, or how much it deviates from a nominal pattern. Further, prior works do not provide automatic or algorithmic approaches for quantifying other pattern characteristics such as depths' consistency, and variations in the pattern motifs at different level sets. This paper leverages persistent homology from Topological Data Analysis (TDA) to derive noise-robust scores for quantifying motifs' depth and roundness in a pattern. Specifically, sublevel persistence is used to derive scores that quantify the consistency of indentation depths at any level set in Piezo Vibration Striking Treatment (PVST) surfaces. Moreover, we combine sublevel persistence with the distance transform to quantify the consistency of the indentation radii, and to compare them with the nominal ones. Although the tool in our PVST experiments had a semi-spherical profile, we present a generalization of our approach to tools/motifs of arbitrary shapes thus making our method applicable to other pattern-generating manufacturing processes.


title: A Self-attention Guided Multi-scale Gradient GAN for Diversified X-ray Image Synthesis

abstract: Imbalanced image datasets are commonly available in the domain of biomedical image analysis. Biomedical images contain diversified features that are significant in predicting targeted diseases. Generative Adversarial Networks (GANs) are utilized to address the data limitation problem via the generation of synthetic images. Training challenges such as mode collapse, non-convergence, and instability degrade a GAN's performance in synthesizing diversified and high-quality images. In this work, MSG-SAGAN, an attention-guided multi-scale gradient GAN architecture is proposed to model the relationship between long-range dependencies of biomedical image features and improves the training performance using a flow of multi-scale gradients at multiple resolutions in the layers of generator and discriminator models. The intent is to reduce the impact of mode collapse and stabilize the training of GAN using an attention mechanism with multi-scale gradient learning for diversified X-ray image synthesis. Multi-scale Structural Similarity Index Measure (MS-SSIM) and Frechet Inception Distance (FID) are used to identify the occurrence of mode collapse and evaluate the diversity of synthetic images generated. The proposed architecture is compared with the multi-scale gradient GAN (MSG-GAN) to assess the diversity of generated synthetic images. Results indicate that the MSG-SAGAN outperforms MSG-GAN in synthesizing diversified images as evidenced by the MS-SSIM and FID scores.


title: A deep learning network with differentiable dynamic programming for retina OCT surface segmentation

abstract: Multiple-surface segmentation in Optical Coherence Tomography (OCT) images is a challenge problem, further complicated by the frequent presence of weak image boundaries. Recently, many deep learning (DL) based methods have been developed for this task and yield remarkable performance. Unfortunately, due to the scarcity of training data in medical imaging, it is challenging for DL networks to learn the global structure of the target surfaces, including surface smoothness. To bridge this gap, this study proposes to seamlessly unify a U-Net for feature learning with a constrained differentiable dynamic programming module to achieve an end-to-end learning for retina OCT surface segmentation to explicitly enforce surface smoothness. It effectively utilizes the feedback from the downstream model optimization module to guide feature learning, yielding a better enforcement of global structures of the target surfaces. Experiments on Duke AMD (age-related macular degeneration) and JHU MS (multiple sclerosis) OCT datasets for retinal layer segmentation demonstrated very promising segmentation accuracy.


title: Synthetic Text Detection: Systemic Literature Review

abstract: Within the text analysis and processing fields, generated text attacks have been made easier to create than ever before. To combat these attacks open sourcing models and datasets have become a major trend to create automated detection algorithms in defense of authenticity. For this purpose, synthetic text detection has become an increasingly viable topic of research. This review is written for the purpose of creating a snapshot of the state of current literature and easing the barrier to entry for future authors. Towards that goal, we identified few research trends and challenges in this field.


title: Global well-posedness of large scale moist atmosphere system with only horizontal viscosity in the dynamic equation

abstract: In order to find a better physical model to describe the large-scale cloud-water transformation and rainfall, we consider a moist atmosphere model consisting of the primitive equations with only horizontal viscosity in the dynamic equation and a set of humidity equations describing water vapor, rain water and cloud condensates. To overcome difficulties caused by the absence of vertical viscosity in the dynamic equation, we get the local existence of $v$ in $H^{1}$ space by combining the viscous elimination method and the $z-$weak solution method and using the generalized Bihari-Lasalle inequality. And then, we get the global existence of $v$ under higher regularity assumption of initial data. In turn, the existence of quasi-strong and strong solutions to the whole system is obtained. By introducing two new unknown quantities appropriately and utilizing the monotone operator theory to overcome difficulties caused by the Heaviside function in the source terms, we get the uniqueness of solutions.


title: Lorentz invariance violation induced threshold anomaly versus very-high energy cosmic photon emission from GRB 221009A

abstract: It has been reported that the Large High Altitude Air Shower Observatory (LHAASO) observed very high energy photons from GRB 221009A, with the highest energy reaching 18~TeV. We find that observation of such high energy photons is quite nontrivial since extragalactic background light could absorb these photons severely and the flux is too weak to be observed. Therefore we discuss a potential mechanism for us to observe these photons, and suggest that Lorentz invariance violation induced threshold anomaly of the process \(\gamma\gamma\to e^-e^+\) provides a candidate to explain this phenomenon.


title: Self-Attention Message Passing for Contrastive Few-Shot Learning

abstract: Humans have a unique ability to learn new representations from just a handful of examples with little to no supervision. Deep learning models, however, require an abundance of data and supervision to perform at a satisfactory level. Unsupervised few-shot learning (U-FSL) is the pursuit of bridging this gap between machines and humans. Inspired by the capacity of graph neural networks (GNNs) in discovering complex inter-sample relationships, we propose a novel self-attention based message passing contrastive learning approach (coined as SAMP-CLR) for U-FSL pre-training. We also propose an optimal transport (OT) based fine-tuning strategy (we call OpT-Tune) to efficiently induce task awareness into our novel end-to-end unsupervised few-shot classification framework (SAMPTransfer). Our extensive experimental results corroborate the efficacy of SAMPTransfer in a variety of downstream few-shot classification scenarios, setting a new state-of-the-art for U-FSL on both miniImagenet and tieredImagenet benchmarks, offering up to 7%+ and 5%+ improvements, respectively. Our further investigations also confirm that SAMPTransfer remains on-par with some supervised baselines on miniImagenet and outperforms all existing U-FSL baselines in a challenging cross-domain scenario. Our code can be found in our GitHub repository at https://github.com/ojss/SAMPTransfer/.


title: Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priors

abstract: Current deep learning models trained to generate radiology reports from chest radiographs are capable of producing clinically accurate, clear, and actionable text that can advance patient care. However, such systems all succumb to the same problem: making hallucinated references to non-existent prior reports. Such hallucinations occur because these models are trained on datasets of real-world patient reports that inherently refer to priors. To this end, we propose two methods to remove references to priors in radiology reports: (1) a GPT-3-based few-shot approach to rewrite medical reports without references to priors; and (2) a BioBERT-based token classification approach to directly remove words referring to priors. We use the aforementioned approaches to modify MIMIC-CXR, a publicly available dataset of chest X-rays and their associated free-text radiology reports; we then retrain CXR-RePaiR, a radiology report generation system, on the adapted MIMIC-CXR dataset. We find that our re-trained model--which we call CXR-ReDonE--outperforms previous report generation methods on clinical metrics, achieving an average BERTScore of 0.2351 (2.57% absolute improvement). We expect our approach to be broadly valuable in enabling current radiology report generation systems to be more directly integrated into clinical pipelines.


title: TaskMix: Data Augmentation for Meta-Learning of Spoken Intent Understanding

abstract: Meta-Learning has emerged as a research direction to better transfer knowledge from related tasks to unseen but related tasks. However, Meta-Learning requires many training tasks to learn representations that transfer well to unseen tasks; otherwise, it leads to overfitting, and the performance degenerates to worse than Multi-task Learning. We show that a state-of-the-art data augmentation method worsens this problem of overfitting when the task diversity is low. We propose a simple method, TaskMix, which synthesizes new tasks by linearly interpolating existing tasks. We compare TaskMix against many baselines on an in-house multilingual intent classification dataset of N-Best ASR hypotheses derived from real-life human-machine telephony utterances and two datasets derived from MTOP. We show that TaskMix outperforms baselines, alleviates overfitting when task diversity is low, and does not degrade performance even when it is high.


title: Derivation of generalized Einstein's equations of gravitation in some non-inertial reference frames based on the theory of vacuum mechanics

abstract: When solving the Einstein's equations for an isolated system of masses, V. Fock introduces harmonic reference frame and obtains an unambiguous solution. Further, he concludes that there exists a harmonic reference frame which is determined uniquely apart from a Lorentz transformation if suitable supplementary conditions are imposed. It is known that wave equations keep the same form under Lorentz transformations. Thus, we speculate that Fock's special harmonic reference frames may have provided us a clue to derive the Einstein's equations in some special class of non-inertial reference frames. Following this clue, generalized Einstein's equations in some special non-inertial reference frames are derived based on the theory of vacuum mechanics. If the field is weak and the reference frame is quasi-inertial, these generalized Einstein's equations reduce to Einstein's equations. Thus, this theory may also explain all the experiments which support the theory of general relativity. There exist some differences between this theory and the theory of general relativity.


title: A chemomechanical model of sperm locomotion reveals two modes of swimming

abstract: The propulsion of mammalian spermatozoa relies on the spontaneous periodic oscillation of their flagella. These oscillations are driven internally by the coordinated action of ATP-powered dynein motors that exert sliding forces between microtubule doublets, resulting in bending waves that propagate along the flagellum and enable locomotion. We present an integrated chemomechanical model of a freely swimming spermatozoon that uses a sliding-control model of the axoneme capturing the two-way feedback between motor kinetics and elastic deformations while accounting for detailed fluid mechanics around the moving cell. We develop a robust computational framework that solves a boundary integral equation for the passive sperm head alongside the slender-body equation for the deforming flagellum described as a geometrically nonlinear internally actuated Euler-Bernoulli beam, and captures full hydrodynamic interactions. Nonlinear simulations are shown to produce spontaneous oscillations with realistic beating patterns and trajectories, which we analyze as a function of sperm number and motor activity. Our results indicate that the swimming velocity does not vary monotonically with dynein activity, but instead displays two maxima corresponding to distinct modes of swimming, each characterized by qualitatively different waveforms and trajectories. Our model also provides an estimate for the efficiency of swimming, which peaks at low sperm number.


title: Rational nonlinear electrodynamics of AdS black holes and extended phase space thermodynamics

abstract: The critical behavior of magnetically charged by rational nonlinear electrodynamics (RNED) AdS black holes in an extended phase space is investigated. The cosmological constant is considered as a thermodynamic pressure and the black hole mass is identified with the chemical enthalpy. The analogy with the Van der Waals liquid-gas system is found and the critical exponents coincide with those of the Van der Waals system. The thermodynamics of RNED-AdS black holes and phase transitions are studied. A new thermodynamic quantities conjugated to the nonlinear parameter of RNED and a magnetic charge are defined. It is demonstrated the consistency of the first law of black hole thermodynamics and the Smarr formula.


title: Variational Open-Domain Question Answering

abstract: Retrieval-augmented models have proven to be effective in natural language processing tasks, yet there remains a lack of research on their optimization using variational inference. We introduce the Variational Open-Domain (VOD) framework for end-to-end training and evaluation of retrieval-augmented models, focusing on open-domain question answering and language modelling. The VOD objective, a self-normalized estimate of the R\'enyi variational bound, approximates the task marginal likelihood and is evaluated under samples drawn from an auxiliary sampling distribution (cached retriever and/or approximate posterior). It remains tractable, even for retriever distributions defined on large corpora. We demonstrate VOD's versatility by training reader-retriever BERT-sized models on multiple-choice medical exam questions. On the MedMCQA dataset, we outperform the domain-tuned Med-PaLM by +5.3% despite using 2.500$\times$ fewer parameters. Our retrieval-augmented BioLinkBERT model scored 62.9% on the MedMCQA and 55.0% on the MedQA-USMLE. Last, we show the effectiveness of our learned retriever component in the context of medical semantic search.


title: Predicting the clinical citation count of biomedical papers using multilayer perceptron neural network

abstract: The number of clinical citations received from clinical guidelines or clinical trials has been considered as one of the most appropriate indicators for quantifying the clinical impact of biomedical papers. Therefore, the early prediction of the clinical citation count of biomedical papers is critical to scientific activities in biomedicine, such as research evaluation, resource allocation, and clinical translation. In this study, we designed a four-layer multilayer perceptron neural network (MPNN) model to predict the clinical citation count of biomedical papers in the future by using 9,822,620 biomedical papers published from 1985 to 2005. We extracted ninety-one paper features from three dimensions as the input of the model, including twenty-one features in the paper dimension, thirty-five in the reference dimension, and thirty-five in the citing paper dimension. In each dimension, the features can be classified into three categories, i.e., the citation-related features, the clinical translation-related features, and the topic-related features. Besides, in the paper dimension, we also considered the features that have previously been demonstrated to be related to the citation counts of research papers. The results showed that the proposed MPNN model outperformed the other five baseline models, and the features in the reference dimension were the most important.


title: A counterexample to $L^{\infty}$-gradient type estimates for Ornstein-Uhlenbeck operators

abstract: Let $(\lambda_k)$ be a strictly increasing sequence of positive numbers such that $\sum_{k=1}^{\infty} \frac{1}{\lambda_k} < \infty.$ Let $f $ be a bounded smooth function and denote by $u= u^f$ the bounded classical solution to $u(x) - \frac{1}{2}\sum_{k=1}^m D^2_{kk} u(x) + \sum_{k =1}^m \lambda_k x_k D_k u(x) = f(x), $ $ x \in \R^m$. It is known that the following dimension-free estimate holds: $$ \displaystyle \int_{\R^m} \Big (\sum_{k=1}^m \lambda_k \, (D_k u (y))^2 \Big)^{p/2} \mu_m (dy) \le (c_p)^p \, \int_{\R^m} |f( y)|^p \mu_m (dy),\;\;\; 1 < p < \infty; $$ here $\mu_m$ is the "diagonal" Gaussian measure determined by $\lambda_1, \ldots, \lambda_m$ and $c_p > 0$ is independent of $f$ and $m$. This is a consequence of generalized Meyer's inequalities [Chojnowska-Michalik, Goldys, J. Funct. Anal. 182 (2001)]. We show that, if $\lambda_k \sim k^2$, then such estimate does not hold when $p= \infty$. Indeed we prove $$ \sup_{\substack{f \in C^{ 2}_b(\R^m),\;\; \|f\|_{\infty} \leq 1}} \Big \{ \sum_{k=1}^m \lambda_k \, (D_k u^f (0))^2 \Big \} \to \infty \;\; \text {as} \; m \to \infty. $$ This is in contrast to the case of $\lambda_k = \lambda >0$, $k \ge 1$, where a dimension-free bound holds for $p =\infty$.


title: Electronic Origin of High-Tc Maximization and Persistence in Trilayer Cuprate Superconductors

abstract: In high temperature cuprate superconductors, it was found that the superconducting transition temperature Tc depends on the number of CuO2 planes (n) in the structural unit and the maximum Tc is realized in the trilayer system (n=3). It was also found that the trilayer superconductors exhibit an unusual phase diagram that Tc keeps nearly constant in the overdoped region which is in strong contrast to the Tc decrease usually found in other cuprate superconductors. The electronic origin of the Tc maximization in the trilayer superconductors and its high Tc persistence in the overdoped region remains unclear. By taking high resolution laser-based angle resolved photoemission (ARPES) measurements, here we report our revelation of the microscopic origin of the unusual superconducting properties in the trilayer superconductors. For the first time we have observed the trilayer splitting in Bi2Sr2Ca2Cu3O10+d (Bi2223) superconductor. The observed Fermi surface, band structures, superconducting gap and the selective Bogoliubov band hybridizations can be well described by a three-layer interaction model. Quantitative information of the microscopic processes involving intra- and interlayer hoppings and pairings are extracted. The electronic origin of the maximum Tc in Bi2223 and the persistence of the high Tc in the overdoped region is revealed. These results provide key insights in understanding high Tc superconductivity and pave a way to further enhance Tc in the cuprate superconductors.


title: Context Generation Improves Open Domain Question Answering

abstract: Closed-book question answering (QA) requires a model to directly answer an open-domain question without access to any external knowledge. Prior work on closed-book QA either directly finetunes or prompts a pretrained language model (LM) to leverage the stored knowledge. However, they do not fully exploit the parameterized knowledge. To address this issue, we propose a two-stage, closed-book QA framework which employs a coarse-to-fine approach to extract relevant knowledge and answer a question. Our approach first generates a related context for a given question by prompting a pretrained LM. We then prompt the same LM for answer prediction using the generated context and the question. Additionally, to eliminate failure caused by context uncertainty, we marginalize over generated contexts. Experimental results on three QA benchmarks show that our method significantly outperforms previous closed-book QA methods (e.g. exact matching 68.6% vs. 55.3%), and is on par with open-book methods that exploit external knowledge sources (e.g. 68.6% vs. 68.0%). Our method is able to better exploit the stored knowledge in pretrained LMs without adding extra learnable parameters or needing finetuning, and paves the way for hybrid models that integrate pretrained LMs with external knowledge.


title: CTL++: Evaluating Generalization on Never-Seen Compositional Patterns of Known Functions, and Compatibility of Neural Representations

abstract: Well-designed diagnostic tasks have played a key role in studying the failure of neural nets (NNs) to generalize systematically. Famous examples include SCAN and Compositional Table Lookup (CTL). Here we introduce CTL++, a new diagnostic dataset based on compositions of unary symbolic functions. While the original CTL is used to test length generalization or productivity, CTL++ is designed to test systematicity of NNs, that is, their capability to generalize to unseen compositions of known functions. CTL++ splits functions into groups and tests performance on group elements composed in a way not seen during training. We show that recent CTL-solving Transformer variants fail on CTL++. The simplicity of the task design allows for fine-grained control of task difficulty, as well as many insightful analyses. For example, we measure how much overlap between groups is needed by tested NNs for learning to compose. We also visualize how learned symbol representations in outputs of functions from different groups are compatible in case of success but not in case of failure. These results provide insights into failure cases reported on more complex compositions in the natural language domain. Our code is public.


title: A Keyword Based Approach to Understanding the Overpenalization of Marginalized Groups by English Marginal Abuse Models on Twitter

abstract: Harmful content detection models tend to have higher false positive rates for content from marginalized groups. In the context of marginal abuse modeling on Twitter, such disproportionate penalization poses the risk of reduced visibility, where marginalized communities lose the opportunity to voice their opinion on the platform. Current approaches to algorithmic harm mitigation, and bias detection for NLP models are often very ad hoc and subject to human bias. We make two main contributions in this paper. First, we design a novel methodology, which provides a principled approach to detecting and measuring the severity of potential harms associated with a text-based model. Second, we apply our methodology to audit Twitter's English marginal abuse model, which is used for removing amplification eligibility of marginally abusive content. Without utilizing demographic labels or dialect classifiers, we are still able to detect and measure the severity of issues related to the over-penalization of the speech of marginalized communities, such as the use of reclaimed speech, counterspeech, and identity related terms. In order to mitigate the associated harms, we experiment with adding additional true negative examples and find that doing so provides improvements to our fairness metrics without large degradations in model performance.


title: Automatic Discovery of Composite SPMD Partitioning Strategies in PartIR

abstract: Large neural network models are commonly trained through a combination of advanced parallelism strategies in a single program, multiple data (SPMD) paradigm. For example, training large transformer models requires combining data, model, and pipeline partitioning; and optimizer sharding techniques. However, identifying efficient combinations for many model architectures and accelerator systems requires significant manual analysis. In this work, we present an automatic partitioner that identifies these combinations through a goal-oriented search. Our key findings are that a Monte Carlo Tree Search-based partitioner leveraging partition-specific compiler analysis directly into the search and guided goals matches expert-level strategies for various models.


title: Russian Web Tables: A Public Corpus of Web Tables for Russian Language Based on Wikipedia

abstract: Corpora that contain tabular data such as WebTables are a vital resource for the academic community. Essentially, they are the backbone of any modern research in information management. They are used for various tasks of data extraction, knowledge base construction, question answering, column semantic type detection and many other. Such corpora are useful not only as a source of data, but also as a base for building test datasets. So far, there were no such corpora for the Russian language and this seriously hindered research in the aforementioned areas. In this paper, we present the first corpus of Web tables created specifically out of Russian language material. It was built via a special toolkit we have developed to crawl the Russian Wikipedia. Both the corpus and the toolkit are open-source and publicly available. Finally, we present a short study that describes Russian Wikipedia tables and their statistics.