title: An algebraic model for rational $T^2$-equivariant elliptic cohomology

abstract: We construct a rational $T^2$-equivariant elliptic cohomology theory for the 2-torus $T^2$, starting from an elliptic curve C over the complex numbers and a coordinate data around the identity. The theory is defined by constructing an object $EC_{T^2}$ in the algebraic model category $dA(T^2)$, which by Greenlees and Shipley is Quillen-equivalent to rational $T^2$-spectra. This result is a generalization to the 2-torus of the construction [Gre05] for the circle. The object $EC_{T^2}$ is directly built using geometric inputs coming from the Cousin complex of the structure sheaf of the surface CxC.


title: Self-training with Two-phase Self-augmentation for Few-shot Dialogue Generation

abstract: In task-oriented dialogue systems, response generation from meaning representations (MRs) often suffers from limited training examples, due to the high cost of annotating MR-to-Text pairs. Previous works on self-training leverage fine-tuned conversational models to automatically generate pseudo-labeled MR-to-Text pairs for further fine-tuning. However, some self-augmented data may be noisy or uninformative for the model to learn from. In this work, we propose a two-phase self-augmentation procedure to generate high-quality pseudo-labeled MR-to-Text pairs: the first phase selects the most informative MRs based on model's prediction uncertainty; with the selected MRs, the second phase generates accurate responses by aggregating multiple perturbed latent representations from each MR. Empirical experiments on two benchmark datasets, FewShotWOZ and FewShotSGD, show that our method generally outperforms existing self-training methods on both automatic and human evaluations.


title: Characterising the Apertif primary beam response

abstract: Context. Phased Array Feeds (PAFs) are multi element receivers in the focal plane of a telescope that make it possible to form simultaneously multiple beams on the sky by combining the complex gains of the individual antenna elements. Recently the Westerbork Synthesis Radio Telescope (WSRT) was upgraded with PAF receivers and carried out several observing programs including two imaging surveys and a time domain survey. The Apertif imaging surveys use a configuration, where 40 partially overlapping compound beams (CBs) are simultaneously formed on the sky and arranged in an approximately rectangular shape. Aims. This manuscript aims to characterise the response of the 40 Apertif CBs to create frequency-resolved, I, XX and YY polarization empirical beam shapes. The measured CB maps can be used for image deconvolution, primary beam correction and mosaicing of Apertif imaging data. Methods. We use drift scan measurements to measure the response of each of the 40 CBs of Apertif. We derive beam maps for all individual beams in I, XX and YY polarisation in 10 or 18 frequency bins over the same bandwidth as the Apertif imaging surveys. We sample the main lobe of the beams and the side lobes up to a radius of 0.6 degrees from the beam centres. In addition, we derive beam maps for each individual WSRT dish as well. Results. We present the frequency and time dependence of the beam shapes and sizes. We compare the compound beam shapes derived with the drift scan method to beam shapes derived with an independent method using a Gaussian Process Regression comparison between the Apertif continuum images and the NRAO VLA Sky Survey (NVSS) catalogue. We find a good agreement between the beam shapes derived with the two independent methods.


title: Collision Detection Accelerated: An Optimization Perspective

abstract: Collision detection between two convex shapes is an essential feature of any physics engine or robot motion planner. It has often been tackled as a computational geometry problem, with the Gilbert, Johnson and Keerthi (GJK) algorithm being the most common approach today. In this work we leverage the fact that collision detection is fundamentally a convex optimization problem. In particular, we establish that the GJK algorithm is a specific sub-case of the well-established Frank-Wolfe (FW) algorithm in convex optimization. We introduce a new collision detection algorithm by adapting recent works linking Nesterov acceleration and Frank-Wolfe methods. We benchmark the proposed accelerated collision detection method on two datasets composed of strictly convex and non-strictly convex shapes. Our results show that our approach significantly reduces the number of iterations to solve collision detection problems compared to the state-of-the-art GJK algorithm, leading to up to two times faster computation times.


title: The Arabic Ontology -- An Arabic Wordnet with Ontologically Clean Content

abstract: We present a formal Arabic wordnet built on the basis of a carefully designed ontology hereby referred to as the Arabic Ontology. The ontology provides a formal representation of the concepts that the Arabic terms convey, and its content was built with ontological analysis in mind, and benchmarked to scientific advances and rigorous knowledge sources as much as this is possible, rather than to only speakers' beliefs as lexicons typically are. A comprehensive evaluation was conducted thereby demonstrating that the current version of the top-levels of the ontology can top the majority of the Arabic meanings. The ontology consists currently of about 1,300 well-investigated concepts in addition to 11,000 concepts that are partially validated. The ontology is accessible and searchable through a lexicographic search engine (https://ontology.birzeit.edu) that also includes about 150 Arabic-multilingual lexicons, and which are being mapped and enriched using the ontology. The ontology is fully mapped with Princeton WordNet, Wikidata, and other resources.


title: Automated Crossword Solving

abstract: We present the Berkeley Crossword Solver, a state-of-the-art approach for automatically solving crossword puzzles. Our system works by generating answer candidates for each crossword clue using neural question answering models and then combines loopy belief propagation with local search to find full puzzle solutions. Compared to existing approaches, our system improves exact puzzle accuracy from 71% to 82% on crosswords from The New York Times and obtains 99.9% letter accuracy on themeless puzzles. Additionally, in 2021, a hybrid of our system and the existing Dr.Fill system outperformed all human competitors for the first time at the American Crossword Puzzle Tournament. To facilitate research on question answering and crossword solving, we analyze our system's remaining errors and release a dataset of over six million question-answer pairs.


title: Personalized Prompt for Sequential Recommendation

abstract: Pre-training models have shown their power in sequential recommendation. Recently, prompt has been widely explored and verified for tuning in NLP pre-training, which could help to more effectively and efficiently extract useful knowledge from pre-training models for downstream tasks, especially in cold-start scenarios. However, it is challenging to bring prompt-tuning from NLP to recommendation, since the tokens in recommendation (i.e., items) do not have explicit explainable semantics, and the sequence modeling should be personalized. In this work, we first introduces prompt to recommendation and propose a novel Personalized prompt-based recommendation (PPR) framework for cold-start recommendation. Specifically, we build the personalized soft prefix prompt via a prompt generator based on user profiles and enable a sufficient training of prompts via a prompt-oriented contrastive learning with both prompt- and behavior-based augmentations. We conduct extensive evaluations on various tasks. In both few-shot and zero-shot recommendation, PPR models achieve significant improvements over baselines on various metrics in three large-scale open datasets. We also conduct ablation tests and sparsity analysis for a better understanding of PPR. Moreover, We further verify PPR's universality on different pre-training models, and conduct explorations on PPR's other promising downstream tasks including cross-domain recommendation and user profile prediction.


title: The AI Mechanic: Acoustic Vehicle Characterization Neural Networks

abstract: In a world increasingly dependent on road-based transportation, it is essential to understand vehicles. We introduce the AI mechanic, an acoustic vehicle characterization deep learning system, as an integrated approach using sound captured from mobile devices to enhance transparency and understanding of vehicles and their condition for non-expert users. We develop and implement novel cascading architectures for vehicle understanding, which we define as sequential, conditional, multi-level networks that process raw audio to extract highly-granular insights. To showcase the viability of cascading architectures, we build a multi-task convolutional neural network that predicts and cascades vehicle attributes to enhance fault detection. We train and test these models on a synthesized dataset reflecting more than 40 hours of augmented audio and achieve >92% validation set accuracy on attributes (fuel type, engine configuration, cylinder count and aspiration type). Our cascading architecture additionally achieved 93.6% validation and 86.8% test set accuracy on misfire fault prediction, demonstrating margins of 16.4% / 7.8% and 4.2% / 1.5% improvement over na\"ive and parallel baselines. We explore experimental studies focused on acoustic features, data augmentation, feature fusion, and data reliability. Finally, we conclude with a discussion of broader implications, future directions, and application areas for this work.


title: Isomorphisms and properties of TAR reconfiguration graphs for zero forcing and other $X$-set parameters

abstract: An $X$-TAR (token addition/removal) reconfiguration graph has as its vertices sets that satisfy some property $X$, with an edge between two sets if one is obtained from the other by adding or removing one element. This paper considers the $X$-TAR graph for $X-$ sets of vertices of a base graph $G$ where the $X$-sets of $G$ must satisfy certain conditions. Dominating sets, power dominating sets, zero forcing sets, and positive semidefinite zero forcing sets are all examples of $X$-sets. For graphs $G$ and $G'$ with no isolated vertices, it is shown that $G$ and $G'$ have isomorphic $X$-TAR reconfiguration graphs if and only if there is a relabeling of the vertices of $G'$ such that $G$ and $G'$ have exactly the same $X$-sets. The concept of an $X$-irrelevant vertex is introduced to facilitate analysis of $X$-TAR graph isomorphisms. Furthermore, results related to the connectedness of the zero forcing TAR graph are given. We present families of graphs that exceed known lower bounds for connectedness parameters.


title: Semi-WTC: A Practical Semi-supervised Framework for Attack Categorization through Weight-Task Consistency

abstract: Supervised learning has been widely used for attack categorization, requiring high-quality data and labels. However, the data is often imbalanced and it is difficult to obtain sufficient annotations. Moreover, supervised models are subject to real-world deployment issues, such as defending against unseen artificial attacks. To tackle the challenges, we propose a semi-supervised fine-grained attack categorization framework consisting of an encoder and a two-branch structure and this framework can be generalized to different supervised models. The multilayer perceptron with residual connection is used as the encoder to extract features and reduce the complexity. The Recurrent Prototype Module (RPM) is proposed to train the encoder effectively in a semi-supervised manner. To alleviate the data imbalance problem, we introduce the Weight-Task Consistency (WTC) into the iterative process of RPM by assigning larger weights to classes with fewer samples in the loss function. In addition, to cope with new attacks in real-world deployment, we propose an Active Adaption Resampling (AAR) method, which can better discover the distribution of unseen sample data and adapt the parameters of encoder. Experimental results show that our model outperforms the state-of-the-art semi-supervised attack detection methods with a 3% improvement in classification accuracy and a 90% reduction in training time.


title: A Unified Collaborative Representation Learning for Neural-Network based Recommender Systems

abstract: Most NN-RSs focus on accuracy by building representations from the direct user-item interactions (e.g., user-item rating matrix), while ignoring the underlying relatedness between users and items (e.g., users who rate the same ratings for the same items should be embedded into similar representations), which is an ideological disadvantage. On the other hand, ME models directly employ inner products as a default loss function metric that cannot project users and items into a proper latent space, which is a methodological disadvantage. In this paper, we propose a supervised collaborative representation learning model - Magnetic Metric Learning (MML) - to map users and items into a unified latent vector space, enhancing the representation learning for NN-RSs. Firstly, MML utilizes dual triplets to model not only the observed relationships between users and items, but also the underlying relationships between users as well as items to overcome the ideological disadvantage. Specifically, a modified metric-based dual loss function is proposed in MML to gather similar entities and disperse the dissimilar ones. With MML, we can easily compare all the relationships (user to user, item to item, user to item) according to the weighted metric, which overcomes the methodological disadvantage. We conduct extensive experiments on four real-world datasets with large item space. The results demonstrate that MML can learn a proper unified latent space for representations from the user-item matrix with high accuracy and effectiveness, and lead to a performance gain over the state-of-the-art RS models by an average of 17%.


title: A graph-transformer for whole slide image classification

abstract: Deep learning is a powerful tool for whole slide image (WSI) analysis. Typically, when performing supervised deep learning, a WSI is divided into small patches, trained and the outcomes are aggregated to estimate disease grade. However, patch-based methods introduce label noise during training by assuming that each patch is independent with the same label as the WSI and neglect overall WSI-level information that is significant in disease grading. Here we present a Graph-Transformer (GT) that fuses a graph-based representation of an WSI and a vision transformer for processing pathology images, called GTP, to predict disease grade. We selected $4,818$ WSIs from the Clinical Proteomic Tumor Analysis Consortium (CPTAC), the National Lung Screening Trial (NLST), and The Cancer Genome Atlas (TCGA), and used GTP to distinguish adenocarcinoma (LUAD) and squamous cell carcinoma (LSCC) from adjacent non-cancerous tissue (normal). First, using NLST data, we developed a contrastive learning framework to generate a feature extractor. This allowed us to compute feature vectors of individual WSI patches, which were used to represent the nodes of the graph followed by construction of the GTP framework. Our model trained on the CPTAC data achieved consistently high performance on three-label classification (normal versus LUAD versus LSCC: mean accuracy$= 91.2$ $\pm$ $2.5\%$) based on five-fold cross-validation, and mean accuracy $= 82.3$ $\pm$ $1.0\%$ on external test data (TCGA). We also introduced a graph-based saliency mapping technique, called GraphCAM, that can identify regions that are highly associated with the class label. Our findings demonstrate GTP as an interpretable and effective deep learning framework for WSI-level classification.


title: A Note on Categories about Rough Sets

abstract: Using the concepts of category and functor, we provide some insights and prove an intrinsic property of the category ${\bf AprS}$ of approximation spaces and relation-preserving functions, the category ${\bf RCls}$ of rough closure spaces and continuous functions, and the category ${\bf RInt}$ of rough interior spaces and continuous functions. Furthermore, we define the category ${\bf IS}$ of information systems and O-A-D homomorphisms, and establish the relationship between the category ${\bf IS}$ and the category ${\bf AprS}$ by considering a subcategory ${\bf NeIS}$ of ${\bf IS}$ whose objects are information systems and whose arrows are non-expensive O-A-D homomorphisms with surjective attribute mappings.


title: Detect Professional Malicious User with Metric Learning in Recommender Systems

abstract: In e-commerce, online retailers are usually suffering from professional malicious users (PMUs), who utilize negative reviews and low ratings to their consumed products on purpose to threaten the retailers for illegal profits. Specifically, there are three challenges for PMU detection: 1) professional malicious users do not conduct any abnormal or illegal interactions (they never concurrently leave too many negative reviews and low ratings at the same time), and they conduct masking strategies to disguise themselves. Therefore, conventional outlier detection methods are confused by their masking strategies. 2) the PMU detection model should take both ratings and reviews into consideration, which makes PMU detection a multi-modal problem. 3) there are no datasets with labels for professional malicious users in public, which makes PMU detection an unsupervised learning problem. To this end, we propose an unsupervised multi-modal learning model: MMD, which employs Metric learning for professional Malicious users Detection with both ratings and reviews. MMD first utilizes a modified RNN to project the informational review into a sentiment score, which jointly considers the ratings and reviews. Then professional malicious user profiling (MUP) is proposed to catch the sentiment gap between sentiment scores and ratings. MUP filters the users and builds a candidate PMU set. We apply a metric learning-based clustering to learn a proper metric matrix for PMU detection. Finally, we can utilize this metric and labeled users to detect PMUs. Specifically, we apply the attention mechanism in metric learning to improve the model's performance. The extensive experiments in four datasets demonstrate that our proposed method can solve this unsupervised detection problem. Moreover, the performance of the state-of-the-art recommender models is enhanced by taking MMD as a preprocessing stage.


title: Disentangling Active and Passive Cosponsorship in the U.S. Congress

abstract: In the U.S. Congress, legislators can use active and passive cosponsorship to support bills. We show that these two types of cosponsorship are driven by two different motivations: the backing of political colleagues and the backing of the bill's content. To this end, we develop an Encoder+RGCN based model that learns legislator representations from bill texts and speech transcripts. These representations predict active and passive cosponsorship with an F1-score of 0.88. Applying our representations to predict voting decisions, we show that they are interpretable and generalize to unseen tasks.

categories: cs.LG cs.CL cs.CY physics.data-an stat.ML


title: Fermi-LAT Detection of a GeV Afterglow from a Compact Stellar Merger

abstract: It is usually thought that long-duration gamma-ray bursts (GRBs) are associated with massive star core collapse, whereas short-duration GRBs are associated with mergers of compact stellar binaries. The discovery of a kilonova associated with a nearby (350 Mpc) long-duration GRB-GRB 211211A, however, indicates that the progenitor of this long-duration GRB is a compact object merger. Here we report the Fermi-LAT detection of gamma-ray ($>100 {\rm \ MeV}$) afterglow emission from GRB 211211A, which lasts $\sim$20,000 s after the burst, the longest event for conventional short-duration GRBs ever detected. We suggest that this gamma-ray emission results from afterglow synchrotron emission. The soft spectrum of GeV emission may arise from a limited maximum synchrotron energy of only a few hundreds of MeV at $\sim$20,000 s. The unusually long duration of the GeV emission could be due to the proximity of this GRB and the long deceleration time of the GRB jet that is expanding in a low-density circumburst medium, consistent with the compact stellar merger scenario.


title: Beyond Greedy Search: Tracking by Multi-Agent Reinforcement Learning-based Beam Search

abstract: To track the target in a video, current visual trackers usually adopt greedy search for target object localization in each frame, that is, the candidate region with the maximum response score will be selected as the tracking result of each frame. However, we found that this may be not an optimal choice, especially when encountering challenging tracking scenarios such as heavy occlusion and fast motion. To address this issue, we propose to maintain multiple tracking trajectories and apply beam search strategy for visual tracking, so that the trajectory with fewer accumulated errors can be identified. Accordingly, this paper introduces a novel multi-agent reinforcement learning based beam search tracking strategy, termed BeamTracking. It is mainly inspired by the image captioning task, which takes an image as input and generates diverse descriptions using beam search algorithm. Accordingly, we formulate the tracking as a sample selection problem fulfilled by multiple parallel decision-making processes, each of which aims at picking out one sample as their tracking result in each frame. Each maintained trajectory is associated with an agent to perform the decision-making and determine what actions should be taken to update related information. When all the frames are processed, we select the trajectory with the maximum accumulated score as the tracking result. Extensive experiments on seven popular tracking benchmark datasets validated the effectiveness of the proposed algorithm.


title: Reconstructing complex field through opaque scattering layer with structured light illumination

abstract: The wavefront is scrambled when coherent light propagates through a random scattering medium and which makes direct use of the conventional optical methods ineffective. In this paper, we propose and demonstrate a structured light illumination for imaging through an opaque scattering layer. Proposed technique is reference free and capable to recover the complex field from intensities of the speckle patterns. This is realized by making use of the phase-shifting in the structured light illumination and applying spatial averaging of the speckle pattern in the intensity correlation measurement. An experimental design is presented and simulated results based on the experimental design are shown to demonstrate imaging of different complex-valued objects through scattering layer.


title: Semi-Supervised Learning for Image Classification using Compact Networks in the BioMedical Context

abstract: The development of mobile and on the edge applications that embed deep convolutional neural models has the potential to revolutionise biomedicine. However, most deep learning models require computational resources that are not available in smartphones or edge devices; an issue that can be faced by means of compact models. The problem with such models is that they are, at least usually, less accurate than bigger models. In this work, we study how this limitation can be addressed with the application of semi-supervised learning techniques. We conduct several statistical analyses to compare performance of deep compact architectures when trained using semi-supervised learning methods for tackling image classification tasks in the biomedical context. In particular, we explore three families of compact networks, and two families of semi-supervised learning techniques for 10 biomedical tasks. By combining semi-supervised learning methods with compact networks, it is possible to obtain a similar performance to standard size networks. In general, the best results are obtained when combining data distillation with MixNet, and plain distillation with ResNet-18. Also, in general, NAS networks obtain better results than manually designed networks and quantized networks. The work presented in this paper shows the benefits of apply semi-supervised methods to compact networks; this allow us to create compact models that are not only as accurate as standard size models, but also faster and lighter. Finally, we have developed a library that simplifies the construction of compact models using semi-supervised learning methods.


title: Dynamic Pricing Provides Robust Equilibria in Stochastic Ridesharing Networks

abstract: Ridesharing markets are complex: drivers are strategic, rider demand and driver availability are stochastic, and complex city-scale phenomena like weather induce large scale correlation across space and time. At the same time, past work has focused on a subset of these challenges. We propose a model of ridesharing networks with strategic drivers, spatiotemporal dynamics, and stochasticity. Supporting both computational tractability and better modeling flexibility than classical fluid limits, we use a two-level stochastic model that allows correlated shocks caused by weather or large public events. Using this model, we propose a novel pricing mechanism: stochastic spatiotemporal pricing (SSP). We show that the SSP mechanism is asymptotically incentive-compatible and that all (approximate) equilibria of the resulting game are asymptotically welfare-maximizing when the market is large enough. The SSP mechanism iteratively recomputes prices based on realized demand and supply, and in this sense prices dynamically. We show that this is critical: while a static variant of the SSP mechanism (whose prices vary with the market-level stochastic scenario but not individual rider and driver decisions) has a sequence of asymptotically welfare-optimal approximate equilibria, we demonstrate that it also has other equilibria producing extremely low social welfare. Thus, we argue that dynamic pricing is important for ensuring robustness in stochastic ride-sharing networks.


title: Metrics of calibration for probabilistic predictions

abstract: Predictions are often probabilities; e.g., a prediction could be for precipitation tomorrow, but with only a 30% chance. Given such probabilistic predictions together with the actual outcomes, "reliability diagrams" help detect and diagnose statistically significant discrepancies -- so-called "miscalibration" -- between the predictions and the outcomes. The canonical reliability diagrams histogram the observed and expected values of the predictions; replacing the hard histogram binning with soft kernel density estimation is another common practice. But, which widths of bins or kernels are best? Plots of the cumulative differences between the observed and expected values largely avoid this question, by displaying miscalibration directly as the slopes of secant lines for the graphs. Slope is easy to perceive with quantitative precision, even when the constant offsets of the secant lines are irrelevant; there is no need to bin or perform kernel density estimation. The existing standard metrics of miscalibration each summarize a reliability diagram as a single scalar statistic. The cumulative plots naturally lead to scalar metrics for the deviation of the graph of cumulative differences away from zero; good calibration corresponds to a horizontal, flat graph which deviates little from zero. The cumulative approach is currently unconventional, yet offers many favorable statistical properties, guaranteed via mathematical theory backed by rigorous proofs and illustrative numerical examples. In particular, metrics based on binning or kernel density estimation unavoidably must trade-off statistical confidence for the ability to resolve variations as a function of the predicted probability or vice versa. Widening the bins or kernels averages away random noise while giving up some resolving power. Narrowing the bins or kernels enhances resolving power while not averaging away as much noise.


title: Amalur: Data Integration Meets Machine Learning

abstract: The data needed for machine learning (ML) model training, can reside in different separate sites often termed data silos. For data-intensive ML applications, data silos pose a major challenge: the integration and transformation of data demand a lot of manual work and computational resources. With data privacy and security constraints, data often cannot leave the local sites, and a model has to be trained in a decentralized manner. In this work, we present a vision on how to bridge the traditional data integration (DI) techniques with the requirements of modern machine learning. We explore the possibilities of utilizing metadata obtained from data integration processes for improving the effectiveness and efficiency of ML models. We analyze two common use cases over data silos, feature augmentation and federated learning. Bringing data integration and machine learning together, we highlight the new research opportunities from the aspects of systems, representations, factorized learning and federated learning.


title: Comparing single-node and multi-node performance of an important fusion HPC code benchmark

abstract: Fusion simulations have traditionally required the use of leadership scale High Performance Computing (HPC) resources in order to produce advances in physics. The impressive improvements in compute and memory capacity of many-GPU compute nodes are now allowing for some problems that once required a multi-node setup to be also solvable on a single node. When possible, the increased interconnect bandwidth can result in order of magnitude higher science throughput, especially for communication-heavy applications. In this paper we analyze the performance of the fusion simulation tool CGYRO, an Eulerian gyrokinetic turbulence solver designed and optimized for collisional, electromagnetic, multiscale simulation, which is widely used in the fusion research community. Due to the nature of the problem, the application has to work on a large multi-dimensional computational mesh as a whole, requiring frequent exchange of large amounts of data between the compute processes. In particular, we show that the average-scale nl03 benchmark CGYRO simulation can be run at an acceptable speed on a single Google Cloud instance with 16 A100 GPUs, outperforming 8 NERSC Perlmutter Phase1 nodes, 16 ORNL Summit nodes and 256 NERSC Cori nodes. Moving from a multi-node to a single-node GPU setup we get comparable simulation times using less than half the number of GPUs. Larger benchmark problems, however, still require a multi-node HPC setup due to GPU memory capacity needs, since at the time of writing no vendor offers nodes with a sufficient GPU memory setup. The upcoming external NVSWITCH does however promise to deliver an almost equivalent solution for up to 256 NVIDIA GPUs.


title: Dexterous Robotic Manipulation using Deep Reinforcement Learning and Knowledge Transfer for Complex Sparse Reward-based Tasks

abstract: This paper describes a deep reinforcement learning (DRL) approach that won Phase 1 of the Real Robot Challenge (RRC) 2021, and then extends this method to a more difficult manipulation task. The RRC consisted of using a TriFinger robot to manipulate a cube along a specified positional trajectory, but with no requirement for the cube to have any specific orientation. We used a relatively simple reward function, a combination of goal-based sparse reward and distance reward, in conjunction with Hindsight Experience Replay (HER) to guide the learning of the DRL agent (Deep Deterministic Policy Gradient (DDPG)). Our approach allowed our agents to acquire dexterous robotic manipulation strategies in simulation. These strategies were then applied to the real robot and outperformed all other competition submissions, including those using more traditional robotic control techniques, in the final evaluation stage of the RRC. Here we extend this method, by modifying the task of Phase 1 of the RRC to require the robot to maintain the cube in a particular orientation, while the cube is moved along the required positional trajectory. The requirement to also orient the cube makes the agent unable to learn the task through blind exploration due to increased problem complexity. To circumvent this issue, we make novel use of a Knowledge Transfer (KT) technique that allows the strategies learned by the agent in the original task (which was agnostic to cube orientation) to be transferred to this task (where orientation matters). KT allowed the agent to learn and perform the extended task in the simulator, which improved the average positional deviation from 0.134 m to 0.02 m, and average orientation deviation from 142{\deg} to 76{\deg} during evaluation. This KT concept shows good generalisation properties and could be applied to any actor-critic learning algorithm.


title: Reverse engineered Diophantine equations

abstract: We answer a question of Samir Siksek, asked at the open problems session of the conference ``Rational Points 2022'', which, in a broader sense, can be viewed as a reverse engineering of Diophantine equations. For any finite set $S$ of perfect integer powers, using Mih\u{a}ilescu's theorem, we construct a polynomial $f_S\in \Z[x]$ such that the set $f_S(\Z)$ contains a perfect integer power if and only if it belongs to $S$. We first discuss the easier case where we restrict to all powers with the same exponent. In this case, the constructed polynomials are inspired by Runge's method and Fermat's Last Theorem. Therefore we can construct a polynomial-exponential Diophantine equation whose solutions are described in advance.


title: ArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD

abstract: Using pre-trained transformer models such as BERT has proven to be effective in many NLP tasks. This paper presents our work to fine-tune BERT models for Arabic Word Sense Disambiguation (WSD). We treated the WSD task as a sentence-pair binary classification task. First, we constructed a dataset of labeled Arabic context-gloss pairs (~167k pairs) we extracted from the Arabic Ontology and the large lexicographic database available at Birzeit University. Each pair was labeled as True or False and target words in each context were identified and annotated. Second, we used this dataset for fine-tuning three pre-trained Arabic BERT models. Third, we experimented the use of different supervised signals used to emphasize target words in context. Our experiments achieved promising results (accuracy of 84%) although we used a large set of senses in the experiment.


title: A new statistic on Dyck paths for counting 3-dimensional Catalan words

abstract: A 3-dimensional Catalan word is a word on three letters so that the subword on any two letters is a Dyck path. For a given Dyck path $D$, a recently defined statistic counts the number of Catalan words with the property that any subword on two letters is exactly $D$. In this paper, we enumerate Dyck paths with this statistic equal to certain values, including all primes. The formulas obtained are in terms of Motzkin numbers and Motzkin ballot numbers.


title: A short overview on low mass scalars at future lepton colliders

abstract: In this manuscript, I give a short summary on scenarios with new physics scalars that could be investigated at future $e^+e^-$ colliders. I concentrate on cases where at least one of the additional scalar has a mass below 125 GeV, and discuss both models where this could be realized, as well as studies which focus on such scenarios. This work is based on several overview talks I recently gave at the CEPC workshop, FCC week and ECFA future collider workshop, as well as a Snowmass White Paper.


title: Pressure-induced Shape-shifting of Helical Bacteria

abstract: Many bacterial species are helical in form, including the widespread pathogen H. pylori. Motivated by recent experiments on H. pylori showing that cell wall synthesis is not uniform, we investigate the possible formation of helical cell shape induced by elastic heterogeneity. We show, experimentally and theoretically, that helical morphogenesis can be produced by pressurizing an elastic cylindrical vessel with helical reinforced lines. The properties of the pressurized helix are highly dependent on the initial helical angle of the reinforced region. We find that steep angles result in crooked helices with, surprisingly, reduced end-to-end distance upon pressurization. This work helps to explain the possible mechanisms for the generation of helical cell morphologies and may inspire the design of novel pressure-controlled helical actuators.


title: GRB Afterglow of the Sub-relativistic Materials with Energy Injection

abstract: Sub-relativistic materials launched during the merger of binary compact objects and the core-collapse of massive stars acquire velocity structures when expanding in a stratified environment. The remnant (either a spinning magnetized neutron star (NS) or a central black hole) from the compact-object or core-collapse could additionally inject energy into the afterglow via spin-down luminosity or/and by accreting fall-back material, producing a refreshed shock, modifying the dynamics, and leading to rich radiation signatures at distinct timescales and energy bands with contrasting intensities. We derive the synchrotron light curves evolving in a stratified environment when a power-law velocity distribution parametrizes the energy of the shock, and the remnant continuously injects energy into the blastwave. As the most relevant case, we describe the latest multi-wavelength afterglow observations ($\gtrsim 900$ days) of the GW170817/GRB 170817A event via a synchrotron afterglow model with energy injection of a sub-relativistic material. The features of the remnant and the synchrotron emission of the sub-relativistic material are consistent with a spinning magnetized NS and the faster "blue" kilonova afterglow, respectively. Using the multi-band observations of some short-bursts with evidence of kilonova, we provide constraints on the expected afterglow emission.


title: VNT-Net: Rotational Invariant Vector Neuron Transformers

abstract: Learning 3D point sets with rotational invariance is an important and challenging problem in machine learning. Through rotational invariant architectures, 3D point cloud neural networks are relieved from requiring a canonical global pose and from exhaustive data augmentation with all possible rotations. In this work, we introduce a rotational invariant neural network by combining recently introduced vector neurons with self-attention layers to build a point cloud vector neuron transformer network (VNT-Net). Vector neurons are known for their simplicity and versatility in representing SO(3) actions and are thereby incorporated in common neural operations. Similarly, Transformer architectures have gained popularity and recently were shown successful for images by applying directly on sequences of image patches and achieving superior performance and convergence. In order to benefit from both worlds, we combine the two structures by mainly showing how to adapt the multi-headed attention layers to comply with vector neurons operations. Through this adaptation attention layers become SO(3) and the overall network becomes rotational invariant. Experiments demonstrate that our network efficiently handles 3D point cloud objects in arbitrary poses. We also show that our network achieves higher accuracy when compared to related state-of-the-art methods and requires less training due to a smaller number of hyperparameters in common classification and segmentation tasks.


title: High-dimensional Data Bootstrap

abstract: This article reviews recent progress in high-dimensional bootstrap. We first review high-dimensional central limit theorems for distributions of sample mean vectors over the rectangles, bootstrap consistency results in high dimensions, and key techniques used to establish those results. We then review selected applications of high-dimensional bootstrap: construction of simultaneous confidence sets for high-dimensional vector parameters, multiple hypothesis testing via stepdown, post-selection inference, intersection bounds for partially identified parameters, and inference on best policies in policy evaluation. Finally, we also comment on a couple of future research directions.


title: Curras + Baladi: Towards a Levantine Corpus

abstract: The processing of the Arabic language is a complex field of research. This is due to many factors, including the complex and rich morphology of Arabic, its high degree of ambiguity, and the presence of several regional varieties that need to be processed while taking into account their unique characteristics. When its dialects are taken into account, this language pushes the limits of NLP to find solutions to problems posed by its inherent nature. It is a diglossic language; the standard language is used in formal settings and in education and is quite different from the vernacular languages spoken in the different regions and influenced by older languages that were historically spoken in those regions. This should encourage NLP specialists to create dialect-specific corpora such as the Palestinian morphologically annotated Curras corpus of Birzeit University. In this work, we present the Lebanese Corpus Baladi that consists of around 9.6K morphologically annotated tokens. Since Lebanese and Palestinian dialects are part of the same Levantine dialectal continuum, and thus highly mutually intelligible, our proposed corpus was constructed to be used to (1) enrich Curras and transform it into a more general Levantine corpus and (2) improve Curras by solving detected errors.


title: Individual and Collective Performance Deteriorate in a New Team: A Case Study of CS:GO Tournaments

abstract: How does the team formation relates to team performance in professional video game playing? This study examined one aspect of group dynamics - team switching - and aims to answer how changing a team affects individual and collective performance in eSports tournaments. In this study we test the hypothesis that switching teams can be detrimental to individual and team performance both in short term and in a long run. We collected data from professional tournaments of a popular first-person shooter game {\itshape Counter-Strike: Global Offensive (CS:GO)} and perform two natural experiments. We found that the player's performance was inversely correlated with the number of teams a player had joined. After a player switched to a new team, both the individual and the collective performance dropped initially, and then slowly recovered. The findings in this study can provide insights for understanding group dynamics in eSports team play and eventually emphasize the importance of team cohesion in facilitating team collaboration, coordination, and knowledge sharing in teamwork in general.


title: Ultrafast structural color change in indium tin oxide/titanium dioxide 1D photonic crystal

abstract: Photonic crystals can integrate plasmonic materials such as indium tin oxide (ITO) in their structure. Exploiting ITO plasmonic properties it is possible to tune the photonic band gap of the photonic crystal upon the application of an external stimuli. In this work, we have fabricated a one-dimensional multilayer photonic crystal via radiofrequency sputtering and we have triggered its optical response with ultrafast pump-probe spectroscopy. Upon photoexcitation we observe a change in the refractive index of indium tin oxide. Such effect has been used to create a photonic crystal that change its photonic bandgap in an ultrafast time scale. All optical modulation in the visible region, that can be tuned by designing the photonic crystal, has been demonstrated.


title: Local H\"older regularity for nonlocal parabolic $p$-Laplace equations

abstract: We prove local H\"older regularity for a nonlocal parabolic equations of the form \begin{align*} \partial_t u + \text{P.V.}\int_{\mathbb{R}^N} \frac{|u(x,t)-u(y,t)|^{p-2}(u(x,t)-u(y,t))}{|x-y|^{N+sp}}\,dy=0, \end{align*} for $p\in (1,\infty)$ and $s \in (0,1)$.


title: Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging

abstract: Details of the designs and mechanisms in support of human-AI collaboration must be considered in the real-world fielding of AI technologies. A critical aspect of interaction design for AI-assisted human decision making are policies about the display and sequencing of AI inferences within larger decision-making workflows. We have a poor understanding of the influences of making AI inferences available before versus after human review of a diagnostic task at hand. We explore the effects of providing AI assistance at the start of a diagnostic session in radiology versus after the radiologist has made a provisional decision. We conducted a user study where 19 veterinary radiologists identified radiographic findings present in patients' X-ray images, with the aid of an AI tool. We employed two workflow configurations to analyze (i) anchoring effects, (ii) human-AI team diagnostic performance and agreement, (iii) time spent and confidence in decision making, and (iv) perceived usefulness of the AI. We found that participants who are asked to register provisional responses in advance of reviewing AI inferences are less likely to agree with the AI regardless of whether the advice is accurate and, in instances of disagreement with the AI, are less likely to seek the second opinion of a colleague. These participants also reported the AI advice to be less useful. Surprisingly, requiring provisional decisions on cases in advance of the display of AI inferences did not lengthen the time participants spent on the task. The study provides generalizable and actionable insights for the deployment of clinical AI tools in human-in-the-loop systems and introduces a methodology for studying alternative designs for human-AI collaboration. We make our experimental platform available as open source to facilitate future research on the influence of alternate designs on human-AI workflows.


title: Beam splitter as quantum coherence-maker

abstract: The aim of this work is to answer the question of how much quantum coherence a beam splitter is able to produce. To this end we consider as the variables under study both the amount of coherence of the input states as well as the beam splitter characteristics. We conclude that there is an optimal combination of these factors making the gain of coherence maximum. In addition, the two mode squeezed vacuum arises as the studied state most capable of gaining coherence when passing through a beam splitter. These results are qualitatively equivalent for the the l1-norm of coherence and for the relative entropy of coherence.


title: Quantum-enhanced differential atom interferometers and clocks with spin-squeezing swapping

abstract: Thanks to common-mode noise rejection, differential configurations are crucial for realistic applications of phase and frequency estimation with atom interferometers. Currently, differential protocols with uncorrelated particles and mode-separable settings reach a sensitivity bounded by the standard quantum limit (SQL). Here we show that differential interferometry can be understood as a distributed multiparameter estimation problem and can benefit from both mode and particle entanglement. Our protocol uses a single spin-squeezed state that is mode-swapped among common interferometric modes. The mode swapping is optimized to estimate the differential phase shift with sub-SQL sensitivity. Numerical calculations are supported by analytical approximations that guide the optimization of the protocol. The scheme is also tested with simulation of noise in atomic clocks and interferometers.


title: Snake net and balloon force with a neural network for detecting multiple phases

abstract: Unsupervised machine learning applied to the study of phase transitions is an ongoing and interesting research direction. The active contour model, also called the snake model, was initially proposed for target contour extraction in two-dimensional images. In order to obtain a physical phase diagram, the snake model with an artificial neural network is applied in an unsupervised learning way by the authors of [Phys.Rev.Lett. 120, 176401(2018)]. It guesses the phase boundary as an initial snake and then drives the snake to convergence with forces estimated by the artificial neural network. In this paper, we extend this unsupervised learning method with one contour to a snake net with multiple contours for the purpose of obtaining several phase boundaries in a phase diagram. For the classical Blume-Capel model, the phase diagram containing three and four phases is obtained. Moreover, to overcome the limitations of the initial position and speed up the movement of the snake, the balloon force decaying with the iteration steps is introduced and applied to the snake net structure. Our method is helpful in determining the phase diagram with multiple phases, using just snapshots of configurations from cold atoms or other experiments without knowledge of the phases.

categories: cond-mat.stat-mech cond-mat.dis-nn cond-mat.quant-gas cs.LG


title: On the affine Springer fibers inside the invariant center of the small quantum group

abstract: Let $\mathfrak{u}_\zeta^\vee$ denote the small quantum group associated with a simple Lie algebra $\mathfrak{g}^\vee$ and a root of unity $\zeta$. Based on the geometric realization of the center of $\mathfrak{u}_\zeta^\vee$ in [8], we use a combinatorial method to derive a formula for the dimension of a subalgebra in the $G^\vee$-invariant part of the center $Z(\mathfrak{u}_\zeta^\vee)^{G^\vee}$ of $\mathfrak{u}_\zeta^\vee$, that conjecturally coincides with the whole $G^\vee$-invariant center. In case $G=SL_n$ we study a refinement of the obtained dimension formula provided by two geometrically defined gradings.


title: Homophily and Incentive Effects in Use of Algorithms

abstract: As algorithmic tools increasingly aid experts in making consequential decisions, the need to understand the precise factors that mediate their influence has grown commensurately. In this paper, we present a crowdsourcing vignette study designed to assess the impacts of two plausible factors on AI-informed decision-making. First, we examine homophily -- do people defer more to models that tend to agree with them? -- by manipulating the agreement during training between participants and the algorithmic tool. Second, we considered incentives -- how do people incorporate a (known) cost structure in the hybrid decision-making setting? -- by varying rewards associated with true positives vs. true negatives. Surprisingly, we found limited influence of either homophily and no evidence of incentive effects, despite participants performing similarly to previous studies. Higher levels of agreement between the participant and the AI tool yielded more confident predictions, but only when outcome feedback was absent. These results highlight the complexity of characterizing human-algorithm interactions, and suggest that findings from social psychology may require re-examination when humans interact with algorithms.


title: Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis

abstract: Graph neural networks (GNNs) are among the most powerful tools in deep learning. They routinely solve complex problems on unstructured networks, such as node classification, graph classification, or link prediction, with high accuracy. However, both inference and training of GNNs are complex, and they uniquely combine the features of irregular graph processing with dense and regular computations. This complexity makes it very challenging to execute GNNs efficiently on modern massively parallel architectures. To alleviate this, we first design a taxonomy of parallelism in GNNs, considering data and model parallelism, and different forms of pipelining. Then, we use this taxonomy to investigate the amount of parallelism in numerous GNN models, GNN-driven machine learning tasks, software frameworks, or hardware accelerators. We use the work-depth model, and we also assess communication volume and synchronization. We specifically focus on the sparsity/density of the associated tensors, in order to understand how to effectively apply techniques such as vectorization. We also formally analyze GNN pipelining, and we generalize the established Message-Passing class of GNN models to cover arbitrary pipeline depths, facilitating future optimizations. Finally, we investigate different forms of asynchronicity, navigating the path for future asynchronous parallel GNN pipelines. The outcomes of our analysis are synthesized in a set of insights that help to maximize GNN performance, and a comprehensive list of challenges and opportunities for further research into efficient GNN computations. Our work will help to advance the design of future GNNs.


title: Extract Dynamic Information To Improve Time Series Modeling: a Case Study with Scientific Workflow

abstract: In modeling time series data, we often need to augment the existing data records to increase the modeling accuracy. In this work, we describe a number of techniques to extract dynamic information about the current state of a large scientific workflow, which could be generalized to other types of applications. The specific task to be modeled is the time needed for transferring a file from an experimental facility to a data center. The key idea of our approach is to find recent past data transfer events that match the current event in some ways. Tests showed that we could identify recent events matching some recorded properties and reduce the prediction error by about 12% compared to the similar models with only static features. We additionally explored an application specific technique to extract information about the data production process, and was able to reduce the average prediction error by 44%.


title: Quantum chaos in supersymmetric quantum mechanics: an exact diagonalization study

abstract: We use exact diagonalization to study energy level statistics and out-of-time-order correlators (OTOCs) for the simplest supersymmetric extension $\hat{H}_S = \hat{H}_B \otimes I + \hat{x}_1 \otimes \sigma_1 + \hat{x}_2 \otimes \sigma_3$ of the bosonic Hamiltonian $\hat{H}_B = \hat{p}_1^2 + \hat{p}_2^2 + \hat{x}_1^2 \, \hat{x}_2^2$. For a long time, this bosonic Hamiltonian was considered one of the simplest systems which exhibit dynamical chaos both classically and quantum-mechanically. Its structure closely resembles that of spatially compactified pure Yang-Mills theory. Correspondingly, the structure of our supersymmetric Hamiltonian is similar to that of spatially compactified supersymmetric Yang-Mills theory, also known as the BFSS model. We present numerical evidence that a continuous energy spectrum of the supersymmetric model leads to monotonous growth of OTOCs down to the lowest temperatures, which is also expected for the BFSS model from holographic duality. This growth is saturated by low-energy eigenstates with effectively one-dimensional wave functions and a completely non-chaotic energy level distribution. We observe a sharp boundary separating these low-energy states from the bulk of chaotic high-energy states. Our data suggests, although with a limited confidence, that at low temperatures the OTOC growth might be exponential over a finite range of time, with the corresponding Lyapunov exponent scaling linearly with temperature. In contrast, the gapped low-energy spectrum of the bosonic Hamiltonian leads to oscillating OTOCs at low temperatures without any signatures of exponential growth. We also find that the OTOCs for the bosonic Hamiltonian are never sufficiently close to the classical Lyapunov distance. On the other hand, the OTOCs for the supersymmetric system agree with the classical limit reasonably well over a finite range of temperatures and evolution times.


title: Distributed Multi-Agent Deep Reinforcement Learning for Robust Coordination against Noise

abstract: In multi-agent systems, noise reduction techniques are important for improving the overall system reliability as agents are required to rely on limited environmental information to develop cooperative and coordinated behaviors with the surrounding agents. However, previous studies have often applied centralized noise reduction methods to build robust and versatile coordination in noisy multi-agent environments, while distributed and decentralized autonomous agents are more plausible for real-world application. In this paper, we introduce a \emph{distributed attentional actor architecture model for a multi-agent system} (DA3-X), using which we demonstrate that agents with DA3-X can selectively learn the noisy environment and behave cooperatively. We experimentally evaluate the effectiveness of DA3-X by comparing learning methods with and without DA3-X and show that agents with DA3-X can achieve better performance than baseline agents. Furthermore, we visualize heatmaps of \emph{attentional weights} from the DA3-X to analyze how the decision-making process and coordinated behavior are influenced by noise.


title: k-strip: A novel segmentation algorithm in k-space for the application of skull stripping

abstract: Objectives: Present a novel deep learning-based skull stripping algorithm for magnetic resonance imaging (MRI) that works directly in the information rich k-space. Materials and Methods: Using two datasets from different institutions with a total of 36,900 MRI slices, we trained a deep learning-based model to work directly with the complex raw k-space data. Skull stripping performed by HD-BET (Brain Extraction Tool) in the image domain were used as the ground truth. Results: Both datasets were very similar to the ground truth (DICE scores of 92\%-98\% and Hausdorff distances of under 5.5 mm). Results on slices above the eye-region reach DICE scores of up to 99\%, while the accuracy drops in regions around the eyes and below, with partially blurred output. The output of k-strip often smoothed edges at the demarcation to the skull. Binary masks are created with an appropriate threshold. Conclusion: With this proof-of-concept study, we were able to show the feasibility of working in the k-space frequency domain, preserving phase information, with consistent results. Future research should be dedicated to discovering additional ways the k-space can be used for innovative image analysis and further workflows.


title: PLAID: An Efficient Engine for Late Interaction Retrieval

abstract: Pre-trained language models are increasingly important components across multiple information retrieval (IR) paradigms. Late interaction, introduced with the ColBERT model and recently refined in ColBERTv2, is a popular paradigm that holds state-of-the-art status across many benchmarks. To dramatically speed up the search latency of late interaction, we introduce the Performance-optimized Late Interaction Driver (PLAID). Without impacting quality, PLAID swiftly eliminates low-scoring passages using a novel centroid interaction mechanism that treats every passage as a lightweight bag of centroids. PLAID uses centroid interaction as well as centroid pruning, a mechanism for sparsifying the bag of centroids, within a highly-optimized engine to reduce late interaction search latency by up to 7$\times$ on a GPU and 45$\times$ on a CPU against vanilla ColBERTv2, while continuing to deliver state-of-the-art retrieval quality. This allows the PLAID engine with ColBERTv2 to achieve latency of tens of milliseconds on a GPU and tens or just few hundreds of milliseconds on a CPU at large scale, even at the largest scales we evaluate with 140M passages.


title: Evaluation of the Effects of Compressive Spectrum Sensing Parameters on Primary User Behavior Estimation

abstract: As the Internet of Things (IoT) technology is being deployed, the demand for radio spectrum is increasing. Cognitive radio (CR) is one of the most promising solutions to allow opportunistic spectrum access for IoT secondary users through utilizing spectrum holes resulting from the underutilization of frequency spectrum. A CR needs to frequently sense the spectrum to avoid interference with primary users (PUs). Compressive spectrum sensing techniques have been gaining increasing interest in wideband spectrum sensing, as they reduce the need for high-rate analog-to-digital converters, reducing the complexity and energy requirements of the CR. In order to enhance spectrum sensing performance, researchers proposed to incorporate PU spectrum usage information into the process of spectrum sensing. Spectrum usage information can be obtained through pilot signals, geo-locational databases or through evaluation of previous spectrum sensing results. In this paper, we are studying the effects of compressive sensing parameters namely compression ratio, sensing period, and sensing duration on the estimation of primary user behavior statistics. We achieved an accurate estimation of the primary user's behavior while saving 40% of the sampling rate by using compressive spectrum sensing compared to traditional spectrum sensing with Nyquist rate sampling.


title: Bi-LSTM Scoring Based Similarity Measurement with Agglomerative Hierarchical Clustering (AHC) for Speaker Diarization

abstract: Majority of speech signals across different scenarios are never available with well-defined audio segments containing only a single speaker. A typical conversation between two speakers consists of segments where their voices overlap, interrupt each other or halt their speech in between multiple sentences. Recent advancements in diarization technology leverage neural network-based approaches to improvise multiple subsystems of speaker diarization system comprising of extracting segment-wise embedding features and detecting changes in the speaker during conversation. However, to identify speaker through clustering, models depend on methodologies like PLDA to generate similarity measure between two extracted segments from a given conversational audio. Since these algorithms ignore the temporal structure of conversations, they tend to achieve a higher Diarization Error Rate (DER), thus leading to misdetections both in terms of speaker and change identification. Therefore, to compare similarity of two speech segments both independently and sequentially, we propose a Bi-directional Long Short-term Memory network for estimating the elements present in the similarity matrix. Once the similarity matrix is generated, Agglomerative Hierarchical Clustering (AHC) is applied to further identify speaker segments based on thresholding. To evaluate the performance, Diarization Error Rate (DER%) metric is used. The proposed model achieves a low DER of 34.80% on a test set of audio samples derived from ICSI Meeting Corpus as compared to traditional PLDA based similarity measurement mechanism which achieved a DER of 39.90%.


title: Voxel-informed Language Grounding

abstract: Natural language applied to natural 2D images describes a fundamentally 3D world. We present the Voxel-informed Language Grounder (VLG), a language grounding model that leverages 3D geometric information in the form of voxel maps derived from the visual input using a volumetric reconstruction model. We show that VLG significantly improves grounding accuracy on SNARE, an object reference game task. At the time of writing, VLG holds the top place on the SNARE leaderboard, achieving SOTA results with a 2.0% absolute improvement.


title: Information transmission with continuous variable quantum erasure channels

abstract: Quantum capacity, as the key figure of merit for a given quantum channel, upper bounds the channel's ability in transmitting quantum information. Identifying different type of channels, evaluating the corresponding quantum capacity and finding the capacity-approaching coding scheme are the major tasks in quantum communication theory. Quantum channel in discrete variables has been discussed enormously involving various error models, while error model in the continuous variable channel has been less studied due to the infinite dimensional problem. In this paper, we investigate a general continuous variable quantum erasure channel. By defining an effective subspace of the continuous variable system, we find a continuous variable random coding model. We then derive the quantum capacity of the continuous variable erasure channel in the framework of decoupling theory. The discussion in this paper fills the gap of quantum erasure channel in continuous variable settings and sheds light on the understanding of other type of continuous variable quantum channels.


title: Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning

abstract: Large language models (LLMs) have been shown to be capable of impressive few-shot generalisation to new tasks. However, they still tend to perform poorly on multi-step logical reasoning problems. Here we carry out a comprehensive evaluation of LLMs on 50 tasks that probe different aspects of logical reasoning. We show that language models tend to perform fairly well at single step inference or entailment tasks, but struggle to chain together multiple reasoning steps to solve more complex problems. In light of this, we propose a Selection-Inference (SI) framework that exploits pre-trained LLMs as general processing modules, and alternates between selection and inference to generate a series of interpretable, casual reasoning steps leading to the final answer. We show that a 7B parameter LLM used within the SI framework in a 5-shot generalisation setting, with no fine-tuning, yields a performance improvement of over 100% compared to an equivalent vanilla baseline on a suite of 10 logical reasoning tasks. The same model in the same setting even outperforms a significantly larger 280B parameter baseline on the same suite of tasks. Moreover, answers produced by the SI framework are accompanied by a causal natural-language-based reasoning trace, which has important implications for the safety and trustworthiness of the system.


title: 2n-Stream Radiative Transfer

abstract: We use 2n streams, where n is an integer, of axially symmetric radiation to solve the equation of transfer for a layered medium. This is a generalization of Schuster's classic 2 stream model. As is well known, using only the first 2n Legendre polynomials to describe the angular dependence of radiation reduces the equation of transfer to a first order differential equation in a space of 2n dimensions. It is convenient to characterize the radiation as 2n stream intensities propagating at zenith angles having cosines called the 2n Gauss-Legendre cosines defined to be solutions of equating the Legendre polynomial of degree 2n to zero. We show how to efficiently and accurately solve the equation of transfer with vector and matrix methods analogous to those used to solve Schroedinger's equation of quantum mechanics. To model strong forward scattering, like that of visible light by Earth's clouds, we have introduced a new family of phase functions. These give the maximum possible forward scattering p(p+1) for a phase function constructed from the first 2p Legendre polynomials, where p is an integer. We show illustrative examples of radiative-transfer phenomena calculated with this new method.


title: Threshold solutions for the intercritical inhomogeneous NLS

abstract: We consider the focusing inhomogeneous nonlinear Schr\"odinger equation in $H^1(\mathbb{R}^3)$, \begin{equation} i\partial_t u + \Delta u + |x|^{-b}|u|^{2}u=0,{equation} where $0 < b <\tfrac{1}{2}$. Previous works have established a blowup/scattering dichotomy below a mass-energy threshold determined by the ground state solution $Q$. In this work, we study solutions exactly at this mass-energy threshold. In addition to the ground state solution, we prove the existence of solutions $Q^\pm$, which approach the standing wave in the positive time direction, but either blow up or scatter in the negative time direction. Using these particular solutions, we classify all possible behaviors for threshold solutions. In particular, the solution either behaves as in the sub-threshold case, or it agrees with $e^{it}Q$, $Q^+$, or $Q^-$ up to the symmetries of the equation.


title: Highly tree-connected complementary modulo factors with bounded degrees

abstract: Let $G$ be a bipartite graph with bipartition $(X,Y)$, let $k$ be a positive integer, and let $f:V(G)\rightarrow Z_k$ be a mapping with $\sum_{v\in X}f(v) \stackrel{k}{\equiv}\sum_{v\in Y}f(v)$. In this paper, we show that if $G$ is $(2m+2m_0+4k-4)$-edge-connected and $m+m_0>0$, then $G$ has an $m$-tree-connected factor $H$ such that its complement is $m_0$-tree-connected and for each vertex $v$, $d_H(v)\stackrel{k}{\equiv} f(v)$, and $$\lfloor\frac{d_G(v)}{2}\rfloor-(k-1)-m_0\le d_{H}(v)\le \lceil\frac{d_G(v)}{2}\rceil+k-1+m.$$ Next, we generalize this result to general graphs and derive a sufficient degree condition for a highly edge-connected general graph $G$ to have a connected factor $H$ such that for each vertex $v$, $d_H(v)\in \{f(v),f(v)+k\}$. Finally, we show that every $(4k-2)$-tree-connected graph admits a bipartite connected factor whose degrees are divisible by $k$.


title: Energy and entropy in the Geometrical Trinity of gravity

abstract: All energy is gravitational energy. That is the consequence of the equivalence principle, according to which gravity is the universal interaction. The physical charges of this interaction have remained undisclosed, but the Adventof the Geometrical Trinity opened a new approach to this foundational problem. Here it is shown to provide a background-independent unification of the previous, non-covariant approaches of Bergmann-Thomson, Cooperstock, Einstein, von Freud, Landau-Lifshitz, Papapetrou and Weinberg. First, the Noether currents are derived for a generic Palatini theory of gravity coupled with generic matter fields, and then the canonical i.e. the unique charges are robustly derived and analysed, particularly in the metric teleparallel and the symmetric teleparallel versions of General Relativity. These results, and their application to black holes and gravitational waves, are new.


title: Flexible Modeling and Multitask Learning using Differentiable Tree Ensembles

abstract: Decision tree ensembles are widely used and competitive learning models. Despite their success, popular toolkits for learning tree ensembles have limited modeling capabilities. For instance, these toolkits support a limited number of loss functions and are restricted to single task learning. We propose a flexible framework for learning tree ensembles, which goes beyond existing toolkits to support arbitrary loss functions, missing responses, and multi-task learning. Our framework builds on differentiable (a.k.a. soft) tree ensembles, which can be trained using first-order methods. However, unlike classical trees, differentiable trees are difficult to scale. We therefore propose a novel tensor-based formulation of differentiable trees that allows for efficient vectorization on GPUs. We perform experiments on a collection of 28 real open-source and proprietary datasets, which demonstrate that our framework can lead to 100x more compact and 23% more expressive tree ensembles than those by popular toolkits.


title: Basic Metric Geometry of the Bottleneck Distance

abstract: Given a metric pair $(X,A)$, i.e. a metric space $X$ and a distinguished closed set $A \subset X$, one may construct in a functorial way a pointed pseudometric space $\mathcal{D}_\infty(X,A)$ of persistence diagrams equipped with the bottleneck distance. We investigate the basic metric properties of the spaces $\mathcal{D}_\infty(X,A)$ and obtain characterizations of their metrizability, completeness, separability, and geodesicity.


title: $\mathscr{L}$-invariants of Artin motives

abstract: We compute Benois $\mathscr{L}$-invariants of weight $1$ cuspforms and of their adjoint representations and show how this extends Gross' $p$-adic regulator to Artin motives which are not critical in the sense of Deligne. Benois' construction depends on the choice of a regular submodule which is well understood when the representation is $p$-regular, as it then amounts to the choice of a ``motivic'' $p$-refinement. The situation is dramatically different in the $p$-irregular case, where the regular submodules are parametrized by a flag variety and thus depend on continuous parameters. We are nevertheless able to show in some examples, how Hida theory and the geometry of the eigencurve can be used to detect a finite number of choices of arithmetic and ``mixed-motivic'' significance.


title: Smooth hypersurfaces in abelian varieties over arithmetic rings

abstract: Let $A$ be an abelian scheme of dimension at least four over a $\mathbb{Z}$-finitely generated integral domain $R$ of characteristic zero, and let $L$ be an ample line bundle on $A$. We prove that the set of smooth hypersurfaces $D$ in $A$ representing $L$ is finite by showing that the moduli stack of such hypersurfaces has only finitely many $R$-points. We accomplish this by using level structures to interpolate finiteness results between this moduli stack and the stack of canonically polarized varieties.


title: HyperAid: Denoising in hyperbolic spaces for tree-fitting and hierarchical clustering

abstract: The problem of fitting distances by tree-metrics has received significant attention in the theoretical computer science and machine learning communities alike, due to many applications in natural language processing, phylogeny, cancer genomics and a myriad of problem areas that involve hierarchical clustering. Despite the existence of several provably exact algorithms for tree-metric fitting of data that inherently obeys tree-metric constraints, much less is known about how to best fit tree-metrics for data whose structure moderately (or substantially) differs from a tree. For such noisy data, most available algorithms perform poorly and often produce negative edge weights in representative trees. Furthermore, it is currently not known how to choose the most suitable approximation objective for noisy fitting. Our contributions are as follows. First, we propose a new approach to tree-metric denoising (HyperAid) in hyperbolic spaces which transforms the original data into data that is ``more'' tree-like, when evaluated in terms of Gromov's $\delta$ hyperbolicity. Second, we perform an ablation study involving two choices for the approximation objective, $\ell_p$ norms and the Dasgupta loss. Third, we integrate HyperAid with schemes for enforcing nonnegative edge-weights. As a result, the HyperAid platform outperforms all other existing methods in the literature, including Neighbor Joining (NJ), TreeRep and T-REX, both on synthetic and real-world data. Synthetic data is represented by edge-augmented trees and shortest-distance metrics while the real-world datasets include Zoo, Iris, Glass, Segmentation and SpamBase; on these datasets, the average improvement with respect to NJ is $125.94\%$.


title: Light In The Black: An Evaluation of Data Augmentation Techniques for COVID-19 CT's Semantic Segmentation

abstract: With the COVID-19 global pandemic, computer-assisted diagnoses of medical images have gained much attention, and robust methods of Semantic Segmentation of Computed Tomography (CT) became highly desirable. Semantic Segmentation of CT is one of many research fields of automatic detection of COVID-19 and has been widely explored since the COVID-19 outbreak. In this work, we propose an extensive analysis of how different data augmentation techniques improve the training of encoder-decoder neural networks on this problem. Twenty different data augmentation techniques were evaluated on five different datasets. Each dataset was validated through a five-fold cross-validation strategy, thus resulting in over 3,000 experiments. Our findings show that spatial level transformations are the most promising to improve the learning of neural networks on this problem.


title: Robust and Efficient Medical Imaging with Self-Supervision

abstract: Recent progress in Medical Artificial Intelligence (AI) has delivered systems that can reach clinical expert level performance. However, such systems tend to demonstrate sub-optimal "out-of-distribution" performance when evaluated in clinical settings different from the training environment. A common mitigation strategy is to develop separate systems for each clinical setting using site-specific data [1]. However, this quickly becomes impractical as medical data is time-consuming to acquire and expensive to annotate [2]. Thus, the problem of "data-efficient generalization" presents an ongoing difficulty for Medical AI development. Although progress in representation learning shows promise, their benefits have not been rigorously studied, specifically for out-of-distribution settings. To meet these challenges, we present REMEDIS, a unified representation learning strategy to improve robustness and data-efficiency of medical imaging AI. REMEDIS uses a generic combination of large-scale supervised transfer learning with self-supervised learning and requires little task-specific customization. We study a diverse range of medical imaging tasks and simulate three realistic application scenarios using retrospective data. REMEDIS exhibits significantly improved in-distribution performance with up to 11.5% relative improvement in diagnostic accuracy over a strong supervised baseline. More importantly, our strategy leads to strong data-efficient generalization of medical imaging AI, matching strong supervised baselines using between 1% to 33% of retraining data across tasks. These results suggest that REMEDIS can significantly accelerate the life-cycle of medical imaging AI development thereby presenting an important step forward for medical imaging AI to deliver broad impact.


title: Chemotactically induced search and defense strategies in a tritrophic system

abstract: In this paper we study the question of the survival of a predator which in a static scenario vanishes. we analyze the role of migration on the coexistence of three species interacting through a intraguild relationship.


title: Enhancing the performance of coupled quantum Otto thermal machines without entanglement and quantum correlations

abstract: We start with a revision study of two coupled spin-$1/2$ under the influence of Kaplan-Shekhtman-Entin-Wohlman-Aharony (KSEA) interaction and a magnetic field. We first show the role of idle levels, i.e., levels that do not couple to the external magnetic field, when the system is working as a heat engine as well as when it is a refrigerator. Then we extend the results reported in [PRE. 92, (2015) 022142] by showing that it is not necessary to change both the magnetic field as well as the coupling parameters to break the extensive property of the work extracted globally from two coupled spin-$1/2$ as has been demonstrated there. Then we study the role of increasing the number of coupled spins on efficiency, extractable work, and coefficient of performance (COP). First, we consider two- and three-coupled spin-$1/2$ Heisenberg $\mathrm{XXX}$-chain. We prove that the latter can outperform the former in terms of efficiency, extractable work, and COP. Then we consider the Ising model, where the number of interacting spins ranges from two to six. We show that only when the number of interacting spins is odd the system can work as a heat engine in the strong coupling regime. The enhancements in efficiency and COP are explored in detail. Finally, this model confirms the idea that entanglement and quantum correlations are not the reasons behind the enhancements observed in efficiency, extracatable work, and COP, but only due to the structure of the energy levels of the Hamiltonian of the working substance. In addition to this, the extensive property of global work as well, is not affected by entanglement and quantum correlations.


title: RankGen: Improving Text Generation with Large Ranking Models

abstract: Given an input sequence (or prefix), modern language models often assign high probabilities to output sequences that are repetitive, incoherent, or irrelevant to the prefix; as such, model-generated text also contains such artifacts. To address these issues we present RankGen, a 1.2B parameter encoder model for English that scores model generations given a prefix. RankGen can be flexibly incorporated as a scoring function in beam search and used to decode from any pretrained language model. We train RankGen using large-scale contrastive learning to map a prefix close to the ground-truth sequence that follows it and far away from two types of negatives: (1) random sequences from the same document as the prefix, and (2) sequences generated from a large language model conditioned on the prefix. Experiments across four different language models (345M-11B parameters) and two domains show that RankGen significantly outperforms decoding algorithms like nucleus, top-k, and typical sampling, as well as contrastive decoding and search, on both automatic metrics (85.0 vs 77.3 MAUVE over nucleus) as well as human evaluations with English writers (74.5% human preference over nucleus sampling). Analysis reveals that RankGen outputs are more relevant to the prefix and improve continuity and coherence compared to baselines. We release our model checkpoints, code, and human preference data with explanations to facilitate future research.


title: The Franz-Parisi Criterion and Computational Trade-offs in High Dimensional Statistics

abstract: Many high-dimensional statistical inference problems are believed to possess inherent computational hardness. Various frameworks have been proposed to give rigorous evidence for such hardness, including lower bounds against restricted models of computation (such as low-degree functions), as well as methods rooted in statistical physics that are based on free energy landscapes. This paper aims to make a rigorous connection between the seemingly different low-degree and free-energy based approaches. We define a free-energy based criterion for hardness and formally connect it to the well-established notion of low-degree hardness for a broad class of statistical problems, namely all Gaussian additive models and certain models with a sparse planted signal. By leveraging these rigorous connections we are able to: establish that for Gaussian additive models the "algebraic" notion of low-degree hardness implies failure of "geometric" local MCMC algorithms, and provide new low-degree lower bounds for sparse linear regression which seem difficult to prove directly. These results provide both conceptual insights into the connections between different notions of hardness, as well as concrete technical tools such as new methods for proving low-degree lower bounds.

categories: math.ST cond-mat.stat-mech cs.CC cs.DS stat.ML stat.TH


title: The TESS Grand Unified Hot Jupiter Survey. I. Ten TESS Planets

abstract: We report the discovery of ten short-period giant planets (TOI-2193A b, TOI-2207 b, TOI-2236 b, TOI-2421 b, TOI-2567 b, TOI-2570 b, TOI-3331 b, TOI-3540A b, TOI-3693 b, TOI-4137 b). All of the planets were identified as planet candidates based on periodic flux dips observed by NASA's Transiting Exoplanet Survey Satellite (TESS). The signals were confirmed to be from transiting planets using ground-based time-series photometry, high angular resolution imaging, and high-resolution spectroscopy coordinated with the TESS Follow-up Observing Program. The ten newly discovered planets orbit relatively bright F and G stars ($G < 12.5$,~$T_\mathrm{eff}$ between 4800 and 6200 K). The planets' orbital periods range from 2 to 10~days, and their masses range from 0.2 to 2.2 Jupiter masses. TOI-2421 b is notable for being a Saturn-mass planet and TOI-2567 b for being a ``sub-Saturn'', with masses of $0.322\pm 0.073$ and $0.195\pm 0.030$ Jupiter masses, respectively. In most cases, we have little information about the orbital eccentricities. Two exceptions are TOI-2207 b, which has an 8-day period and a detectably eccentric orbit ($e = 0.17\pm0.05$), and TOI-3693 b, a 9-day planet for which we can set an upper limit of $e < 0.052$. The ten planets described here are the first new planets resulting from an effort to use TESS data to unify and expand on the work of previous ground-based transit surveys in order to create a large and statistically useful sample of hot Jupiters.


title: Reinforcement Learning with Brain-Inspired Modulation can Improve Adaptation to Environmental Changes

abstract: Developments in reinforcement learning (RL) have allowed algorithms to achieve impressive performance in highly complex, but largely static problems. In contrast, biological learning seems to value efficiency of adaptation to a constantly-changing world. Here we build on a recently-proposed neuronal learning rule that assumes each neuron can optimize its energy balance by predicting its own future activity. That assumption leads to a neuronal learning rule that uses presynaptic input to modulate prediction error. We argue that an analogous RL rule would use action probability to modulate reward prediction error. This modulation makes the agent more sensitive to negative experiences, and more careful in forming preferences. We embed the proposed rule in both tabular and deep-Q-network RL algorithms, and find that it outperforms conventional algorithms in simple, but highly-dynamic tasks. We suggest that the new rule encapsulates a core principle of biological intelligence; an important component for allowing algorithms to adapt to change in a human-like way.


title: Dissemination Control in Dynamic Data Clustering For Dense IIoT Against False Data Injection Attack

abstract: The IoT has made possible the development of increasingly driven services, like industrial IIoT services, that often deal with massive amounts of data. Meantime, as IIoT networks grow, the threats are even greater, and false data injection attacks (FDI) stand out as being one of the most aggressive. The majority of current solutions to handle this attack do not take into account the data validation, especially on the data clustering service. Aiming to advance on the issue, this work introduces CONFINIT, an intrusion detection system for mitigating FDI attacks on the data dissemination service performing in dense IIoT networks. CONFINIT combines watchdog surveillance and collaborative consensus strategies for assertively excluding various FDI attacks. The simulations showed that CONFINIT compared to DDFC increased by up to 35% - 40% the number of clusters without attackers in a gas pressure IIoT environment. CONFINIT achieved attack detection rates of 99%, accuracy of 90 and F1 score of 0.81 in multiple IIoT scenarios, with only up to 3.2% and 3.6% of false negatives and positives rates, respectively. Moreover, under two variants of FDI attacks, called Churn and Sensitive attacks, CONFINIT achieved detection rates of 100%, accuracy of 99 and F1 of 0.93 with less than 2% of false positives and negatives rates.


title: Towards Unified Keyframe Propagation Models

abstract: Many video editing tasks such as rotoscoping or object removal require the propagation of context across frames. While transformers and other attention-based approaches that aggregate features globally have demonstrated great success at propagating object masks from keyframes to the whole video, they struggle to propagate high-frequency details such as textures faithfully. We hypothesize that this is due to an inherent bias of global attention towards low-frequency features. To overcome this limitation, we present a two-stream approach, where high-frequency features interact locally and low-frequency features interact globally. The global interaction stream remains robust in difficult situations such as large camera motions, where explicit alignment fails. The local interaction stream propagates high-frequency details through deformable feature aggregation and, informed by the global interaction stream, learns to detect and correct errors of the deformation field. We evaluate our two-stream approach for inpainting tasks, where experiments show that it improves both the propagation of features within a single frame as required for image inpainting, as well as their propagation from keyframes to target frames. Applied to video inpainting, our approach leads to 44% and 26% improvements in FID and LPIPS scores. Code at https://github.com/runwayml/guided-inpainting


title: Enhancing Slot Tagging with Intent Features for Task Oriented Natural Language Understanding using BERT

abstract: Recent joint intent detection and slot tagging models have seen improved performance when compared to individual models. In many real-world datasets, the slot labels and values have a strong correlation with their intent labels. In such cases, the intent label information may act as a useful feature to the slot tagging model. In this paper, we examine the effect of leveraging intent label features through 3 techniques in the slot tagging task of joint intent and slot detection models. We evaluate our techniques on benchmark spoken language datasets SNIPS and ATIS, as well as over a large private Bixby dataset and observe an improved slot-tagging performance over state-of-the-art models.


title: On the number and size of holes in the growing ball of first-passage percolation

abstract: First-passage percolation is a random growth model defined on $\mathbb{Z}^d$ using i.i.d. nonnegative weights $(\tau_e)$ on the edges. Letting $T(x,y)$ be the distance between vertices $x$ and $y$ induced by the weights, we study the random ball of radius $t$ centered at the origin, $B(t) = \{x \in \mathbb{Z}^d : T(0,x) \leq t\}$. It is known that for all such $\tau_e$, the number of vertices (volume) of $B(t)$ is at least order $t^d$, and under mild conditions on $\tau_e$, this volume grows like a deterministic constant times $t^d$. Defining a hole in $B(t)$ to be a bounded component of the complement $B(t)^c$, we prove that if $\tau_e$ is not deterministic, then a.s., for all large $t$, $B(t)$ has at least $ct^{d-1}$ many holes, and the maximal volume of any hole is at least $c\log t$. Conditionally on the (unproved) uniform curvature assumption, we prove that a.s., for all large $t$, the number of holes is at most $(\log t)^C t^{d-1}$, and for $d=2$, no hole in $B(t)$ has volume larger than $(\log t)^C$. Without curvature, we show that no hole has volume larger than $Ct \log t$.


title: Saturation and recurrence of quantum complexity in random local quantum dynamics

abstract: Quantum complexity is a measure of the minimal number of elementary operations required to approximately prepare a given state or unitary channel. Recently, this concept has found applications beyond quantum computing -- in studying the dynamics of quantum many-body systems and the long-time properties of AdS black holes. In this context Brown and Susskind \cite{BrownSusskind17} conjectured that the complexity of a chaotic quantum system grows linearly in time up to times exponential in the system size, saturating at a maximal value, and remaining maximally complex until undergoing recurrences at doubly-exponential times. In this work we prove the saturation and recurrence of complexity in two models of chaotic time evolutions based on (i) random local quantum circuits and (ii) stochastic local Hamiltonian evolution. Our results advance an understanding of the long-time behaviour of chaotic quantum systems and could shed light on the physics of black hole interiors. From a technical perspective our results are based on establishing new quantitative connections between the Haar measure and high-degree approximate designs, as well as the fact that random quantum circuits of sufficiently high depth converge to approximate designs.

categories: quant-ph cond-mat.str-el hep-th math-ph math.MP


title: Foundation Posteriors for Approximate Probabilistic Inference

abstract: Probabilistic programs provide an expressive representation language for generative models. Given a probabilistic program, we are interested in the task of posterior inference: estimating a latent variable given a set of observed variables. Existing techniques for inference in probabilistic programs often require choosing many hyper-parameters, are computationally expensive, and/or only work for restricted classes of programs. Here we formulate inference as masked language modeling: given a program, we generate a supervised dataset of variables and assignments, and randomly mask a subset of the assignments. We then train a neural network to unmask the random values, defining an approximate posterior distribution. By optimizing a single neural network across a range of programs we amortize the cost of training, yielding a "foundation" posterior able to do zero-shot inference for new programs. The foundation posterior can also be fine-tuned for a particular program and dataset by optimizing a variational inference objective. We show the efficacy of the approach, zero-shot and fine-tuned, on a benchmark of STAN programs.


title: Balanced and Robust Randomized Treatment Assignments: The Finite Selection Model for the Health Insurance Experiment and Beyond

abstract: The Finite Selection Model (FSM) was developed by Carl Morris in the 1970s for the design of the RAND Health Insurance Experiment (HIE) (Morris 1979, Newhouse et al. 1993), one of the largest and most comprehensive social science experiments conducted in the U.S. The idea behind the FSM is that each treatment group takes its turns selecting units in a fair and random order to optimize a common assignment criterion. At each of its turns, a treatment group selects the available unit that maximally improves the combined quality of its resulting group of units in terms of the criterion. In the HIE and beyond, we revisit, formalize, and extend the FSM as a general tool for experimental design. Leveraging the idea of D-optimality, we propose and analyze a new selection criterion in the FSM. The FSM using the D-optimal selection function has no tuning parameters, is affine invariant, and when appropriate, retrieves several classical designs such as randomized block and matched-pair designs. For multi-arm experiments, we propose algorithms to generate a fair and random selection order of treatments. We demonstrate FSM's performance in a case study based on the HIE and in ten randomized studies from the health and social sciences. On average, the FSM achieves 68% better covariate balance than complete randomization and 56% better covariate balance than rerandomization in a typical study. We recommend the FSM be considered in experimental design for its conceptual simplicity, efficiency, and robustness.


title: A Penrose-type inequality with angular momenta for black holes with 3-sphere horizon topology

abstract: We establish a Penrose-type inequality with angular momenta for four dimensional, biaxially symmetric, maximal, asymptotically flat initial data sets $(M,g,k)$ for the Einstein equations with fixed angular momenta and horizon inner boundary associated to a 3-sphere outermost minimal surface. Moreover, equality holds if and only if the initial data set is isometric to a canonical time slice of a stationary Myers-Perry black hole.


title: AIGenC: An AI generalisation model via creativity

abstract: Inspired by cognitive theories of creativity, this paper introduces a computational model (AIGenC) that lays down the necessary components to enable artificial agents to learn, use and generate transferable representations. Unlike machine representation learning, which relies exclusively on raw sensory data, biological representations incorporate relational and associative information that embeds rich and structured concept spaces. The AIGenC model poses a hierarchical graph architecture with various levels and types of representations procured by different components. The first component, Concept Processing, extracts objects and affordances from sensory input and encodes them into a concept space. The resulting representations are stored in a dual memory system and enriched with goal-directed and temporal information acquired through reinforcement learning, creating a higher-level of abstraction. Two additional components work in parallel to detect and recover relevant concepts and create new ones, respectively, in a process akin to cognitive Reflective Reasoning and Blending. The Reflective Reasoning unit detects and recovers from memory concepts relevant to the task by means of a matching process that calculates a similarity value between the current state and memory graph structures. Once the matching interaction ends, rewards and temporal information are added to the graph, building further abstractions. If the reflective reasoning processing fails to offer a suitable solution, a blending operation comes into place, creating new concepts by combining past information. We discuss the model's capability to yield better out-of-distribution generalisation in artificial agents, thus advancing toward Artificial General Intelligence.


title: Diverse Weight Averaging for Out-of-Distribution Generalization

abstract: Standard neural networks struggle to generalize under distribution shifts in computer vision. Fortunately, combining multiple networks can consistently improve out-of-distribution generalization. In particular, weight averaging (WA) strategies were shown to perform best on the competitive DomainBed benchmark; they directly average the weights of multiple networks despite their nonlinearities. In this paper, we propose Diverse Weight Averaging (DiWA), a new WA strategy whose main motivation is to increase the functional diversity across averaged models. To this end, DiWA averages weights obtained from several independent training runs: indeed, models obtained from different runs are more diverse than those collected along a single run thanks to differences in hyperparameters and training procedures. We motivate the need for diversity by a new bias-variance-covariance-locality decomposition of the expected error, exploiting similarities between WA and standard functional ensembling. Moreover, this decomposition highlights that WA succeeds when the variance term dominates, which we show occurs when the marginal distribution changes at test time. Experimentally, DiWA consistently improves the state of the art on DomainBed without inference overhead.