title: Localising In-Domain Adaptation of Transformer-Based Biomedical Language Models

abstract: In the era of digital healthcare, the huge volumes of textual information generated every day in hospitals constitute an essential but underused asset that could be exploited with task-specific, fine-tuned biomedical language representation models, improving patient care and management. For such specialized domains, previous research has shown that fine-tuning models stemming from broad-coverage checkpoints can largely benefit additional training rounds over large-scale in-domain resources. However, these resources are often unreachable for less-resourced languages like Italian, preventing local medical institutions to employ in-domain adaptation. In order to reduce this gap, our work investigates two accessible approaches to derive biomedical language models in languages other than English, taking Italian as a concrete use-case: one based on neural machine translation of English resources, favoring quantity over quality; the other based on a high-grade, narrow-scoped corpus natively written in Italian, thus preferring quality over quantity. Our study shows that data quantity is a harder constraint than data quality for biomedical adaptation, but the concatenation of high-quality data can improve model performance even when dealing with relatively size-limited corpora. The models published from our investigations have the potential to unlock important research opportunities for Italian hospitals and academia. Finally, the set of lessons learned from the study constitutes valuable insights towards a solution to build biomedical language models that are generalizable to other less-resourced languages and different domain settings.


title: Fine-Grained Distillation for Long Document Retrieval

abstract: Long document retrieval aims to fetch query-relevant documents from a large-scale collection, where knowledge distillation has become de facto to improve a retriever by mimicking a heterogeneous yet powerful cross-encoder. However, in contrast to passages or sentences, retrieval on long documents suffers from the scope hypothesis that a long document may cover multiple topics. This maximizes their structure heterogeneity and poses a granular-mismatch issue, leading to an inferior distillation efficacy. In this work, we propose a new learning framework, fine-grained distillation (FGD), for long-document retrievers. While preserving the conventional dense retrieval paradigm, it first produces global-consistent representations crossing different fine granularity and then applies multi-granular aligned distillation merely during training. In experiments, we evaluate our framework on two long-document retrieval benchmarks, which show state-of-the-art performance.


title: A Passivity Preserving H-infinity Synthesis Technique for Robot Control

abstract: Most impedance control schemes in robotics implement a desired passive impedance, allowing for stable interaction between the controlled robot and the environment. However, there is little guidance on the selection of the desired impedance. In general, finding the best stiffness and damping parameters is a challenging task. This paper contributes to this problem by connecting impedance control to robust control, with the goal of shaping the robot performances via feedback. We provide a method based on linear matrix inequalities with sparsity constraints to derive impedance controllers that satisfy a H-infinity performance criterion. Our controller guarantees passivity of the controlled robot and local performances near key poses.


title: Evaluating Multimodal Interaction of Robots Assisting Older Adults

abstract: We outline our work on evaluating robots that assist older adults by engaging with them through multiple modalities that include physical interaction. Our thesis is that to increase the effectiveness of assistive robots: 1) robots need to understand and effect multimodal actions, 2) robots should not only react to the human, they need to take the initiative and lead the task when it is necessary. We start by briefly introducing our proposed framework for multimodal interaction and then describe two different experiments with the actual robots. In the first experiment, a Baxter robot helps a human find and locate an object using the Multimodal Interaction Manager (MIM) framework. In the second experiment, a NAO robot is used in the same task, however, the roles of the robot and the human are reversed. We discuss the evaluation methods that were used in these experiments, including different metrics employed to characterize the performance of the robot in each case. We conclude by providing our perspective on the challenges and opportunities for the evaluation of assistive robots for older adults in realistic settings.


title: Deep Riemannian Networks for EEG Decoding

abstract: State-of-the-art performance in electroencephalography (EEG) decoding tasks is currently often achieved with either Deep-Learning (DL) or Riemannian-Geometry-based decoders (RBDs). Recently, there is growing interest in Deep Riemannian Networks (DRNs) possibly combining the advantages of both previous classes of methods. However, there are still a range of topics where additional insight is needed to pave the way for a more widespread application of DRNs in EEG. These include architecture design questions such as network size and end-to-end ability.How these factors affect model performance has not been explored. Additionally, it is not clear how the data within these networks is transformed, and whether this would correlate with traditional EEG decoding. Our study aims to lay the groundwork in the area of these topics through the analysis of DRNs for EEG with a wide range of hyperparameters. Networks were tested on two public EEG datasets and compared with state-of-the-art ConvNets. Here we propose end-to-end EEG SPDNet (EE(G)-SPDNet), and we show that this wide, end-to-end DRN can outperform the ConvNets, and in doing so use physiologically plausible frequency regions. We also show that the end-to-end approach learns more complex filters than traditional band-pass filters targeting the classical alpha, beta, and gamma frequency bands of the EEG, and that performance can benefit from channel specific filtering approaches. Additionally, architectural analysis revealed areas for further improvement due to the possible loss of Riemannian specific information throughout the network. Our study thus shows how to design and train DRNs to infer task-related information from the raw EEG without the need of handcrafted filterbanks and highlights the potential of end-to-end DRNs such as EE(G)-SPDNet for high-performance EEG decoding.


title: ModularFed: Leveraging Modularity in Federated Learning Frameworks

abstract: Numerous research recently proposed integrating Federated Learning (FL) to address the privacy concerns of using machine learning in privacy-sensitive firms. However, the standards of the available frameworks can no longer sustain the rapid advancement and hinder the integration of FL solutions, which can be prominent in advancing the field. In this paper, we propose ModularFed, a research-focused framework that addresses the complexity of FL implementations and the lack of adaptability and extendability in the available frameworks. We provide a comprehensive architecture that assists FL approaches through well-defined protocols to cover three dominant FL paradigms: adaptable workflow, datasets distribution, and third-party application support. Within this architecture, protocols are blueprints that strictly define the framework's components' design, contribute to its flexibility, and strengthen its infrastructure. Further, our protocols aim to enable modularity in FL, supporting third-party plug-and-play architecture and dynamic simulators coupled with major built-in data distributors in the field. Additionally, the framework support wrapping multiple approaches in a single environment to enable consistent replication of FL issues such as clients' deficiency, data distribution, and network latency, which entails a fair comparison of techniques outlying FL technologies. In our evaluation, we examine the applicability of our framework addressing three major FL domains, including statistical distribution and modular-based approaches for resource monitoring and client selection.


title: HouseCat6D -- A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarios

abstract: Estimating 6D object poses is a major challenge in 3D computer vision. Building on successful instance-level approaches, research is shifting towards category-level pose estimation for practical applications. Current category-level datasets, however, fall short in annotation quality and pose variety. Addressing this, we introduce HouseCat6D, a new category-level 6D pose dataset. It features 1) multi-modality with Polarimetric RGB and Depth (RGBD+P), 2) encompasses 194 diverse objects across 10 household categories, including two photometrically challenging ones, and 3) provides high-quality pose annotations with an error range of only 1.35 mm to 1.74 mm. The dataset also includes 4) 41 large-scale scenes with comprehensive viewpoint and occlusion coverage, 5) a checkerboard-free environment, and 6) dense 6D parallel-jaw robotic grasp annotations. Additionally, we present benchmark results for leading category-level pose estimation networks.


title: Independent component analysis in the light of Information Geometry

abstract: I recall my first encounter with Professor Shun-ichi Amari who, once upon a time in Las Vegas, gave me a precious hint about connecting Independent Component Analysis (ICA) to Information Geometry. The paper sketches, rather informally, some of the insights gained in following this lead.


title: Walking Noise: On Layer-Specific Robustness of Neural Architectures against Noisy Computations and Associated Characteristic Learning Dynamics

abstract: Deep neural networks are extremely successful in various applications, however they exhibit high computational demands and energy consumption. This is exacerbated by stuttering technology scaling, prompting the need for novel approaches to handle increasingly complex neural architectures. At the same time, alternative computing technologies such as analog computing, which promise groundbreaking improvements in energy efficiency, are inevitably fraught with noise and inaccurate calculations. Such noisy computations are more energy efficient, and, given a fixed power budget, also more time efficient. However, like any kind of unsafe optimization, they require countermeasures to ensure functionally correct results. This work considers noisy computations in an abstract form, and gears to understand the implications of such noise on the accuracy of neural network classifiers as an exemplary workload. We propose a methodology called Walking Noise which injects layer-specific noise to measure the robustness and to provide insights on the learning dynamics. In more detail, we investigate the implications of additive, multiplicative and mixed noise for different classification tasks and model architectures. While noisy training significantly increases robustness for all noise types, we observe in particular that it results in increased weight magnitudes and thus inherently improves the signal-to-noise ratio for additive noise injection. Contrarily, training with multiplicative noise can lead to a form of self-binarization of the model parameters, leading to extreme robustness. We conclude with a discussion of the use of this methodology in practice, among others, discussing its use for tailored multi-execution in noisy environments.


title: QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity

abstract: The mechanism of existing style transfer algorithms is by minimizing a hybrid loss function to push the generated image toward high similarities in both content and style. However, this type of approach cannot guarantee visual fidelity, i.e., the generated artworks should be indistinguishable from real ones. In this paper, we devise a new style transfer framework called QuantArt for high visual-fidelity stylization. QuantArt pushes the latent representation of the generated artwork toward the centroids of the real artwork distribution with vector quantization. By fusing the quantized and continuous latent representations, QuantArt allows flexible control over the generated artworks in terms of content preservation, style similarity, and visual fidelity. Experiments on various style transfer settings show that our QuantArt framework achieves significantly higher visual fidelity compared with the existing style transfer methods.


title: AlphaSparse: Generating High Performance SpMV Codes Directly from Sparse Matrices

abstract: Sparse Matrix-Vector multiplication (SpMV) is an essential computational kernel in many application scenarios. Tens of sparse matrix formats and implementations have been proposed to compress the memory storage and speed up SpMV performance. We develop AlphaSparse, a superset of all existing works that goes beyond the scope of human-designed format(s) and implementation(s). AlphaSparse automatically \emph{creates novel machine-designed formats and SpMV kernel implementations} entirely from the knowledge of input sparsity patterns and hardware architectures. Based on our proposed Operator Graph that expresses the path of SpMV format and kernel design, AlphaSparse consists of three main components: Designer, Format \& Kernel Generator, and Search Engine. It takes an arbitrary sparse matrix as input while outputs the performant machine-designed format and SpMV implementation. By extensively evaluating 843 matrices from SuiteSparse Matrix Collection, AlphaSparse achieves significant performance improvement by 3.2$\times$ on average compared to five state-of-the-art artificial formats and 1.5$\times$ on average (up to 2.7$\times$) over the up-to-date implementation of traditional auto-tuning philosophy.


title: Scheduling with Predictions

abstract: There is significant interest in deploying machine learning algorithms for diagnostic radiology, as modern learning techniques have made it possible to detect abnormalities in medical images within minutes. While machine-assisted diagnoses cannot yet reliably replace human reviews of images by a radiologist, they could inform prioritization rules for determining the order by which to review patient cases so that patients with time-sensitive conditions could benefit from early intervention. We study this scenario by formulating it as a learning-augmented online scheduling problem. We are given information about each arriving patient's urgency level in advance, but these predictions are inevitably error-prone. In this formulation, we face the challenges of decision making under imperfect information, and of responding dynamically to prediction error as we observe better data in real-time. We propose a simple online policy and show that this policy is in fact the best possible in certain stylized settings. We also demonstrate that our policy achieves the two desiderata of online algorithms with predictions: consistency (performance improvement with prediction accuracy) and robustness (protection against the worst case). We complement our theoretical findings with empirical evaluations of the policy under settings that more accurately reflect clinical scenarios in the real world.


title: Comment on "Extension of Rietveld Refinement for Benchtop Powder XRD Analysis of Ultrasmall Supported Nanoparticles" by J. Lipp et al. (Chem. Mater. 2022, 34, 18, 8091-8111)

abstract: The comment gives principal arguments proving that the claims given in the title of the commented paper and some its conclusions have to be wrong for principal reasons. An example illustrating the arguments is provided via atomistic simulation.


title: The Expertise Level

abstract: Computers are quickly gaining on us. Artificial systems are now exceeding the performance of human experts in several domains. However, we do not yet have a deep definition of expertise. This paper examines the nature of expertise and presents an abstract knowledge-level and skill-level description of expertise. A new level lying above the Knowledge Level, called the Expertise Level, is introduced to describe the skills of an expert without having to worry about details of the knowledge required. The Model of Expertise is introduced combining the knowledge-level and expertise-level descriptions. Application of the model to the fields of cognitive architectures and human cognitive augmentation is demonstrated and several famous intelligent systems are analyzed with the model.


title: Fermat's Last Theorem for Special Case

abstract: In this paper we present an elementary proof for a special case of Fermat's last theorem for specific category of a, b and c. In fact, we assume that $n$ is prime and $4\rvert (n+1),$ then for $a,b$ and $c$ that $ n\nmid abc$ the equation $a^{2n} = b^{2n} + c^{2n}$ does not have any solution in natural numbers.


title: Wireless Power Transfer Analysis using Ansys

abstract: In this report, methods of wireless power transfer of inside a cage is compared and simulated. Software Ansys Electronics Maxwell was used to simulate magnetic coupling of transmitter(s) and receiver with 10mA winding current in the mouse cage.


title: Is Semantic Communications Secure? A Tale of Multi-Domain Adversarial Attacks

abstract: Semantic communications seeks to transfer information from a source while conveying a desired meaning to its destination. We model the transmitter-receiver functionalities as an autoencoder followed by a task classifier that evaluates the meaning of the information conveyed to the receiver. The autoencoder consists of an encoder at the transmitter to jointly model source coding, channel coding, and modulation, and a decoder at the receiver to jointly model demodulation, channel decoding and source decoding. By augmenting the reconstruction loss with a semantic loss, the two deep neural networks (DNNs) of this encoder-decoder pair are interactively trained with the DNN of the semantic task classifier. This approach effectively captures the latent feature space and reliably transfers compressed feature vectors with a small number of channel uses while keeping the semantic loss low. We identify the multi-domain security vulnerabilities of using the DNNs for semantic communications. Based on adversarial machine learning, we introduce test-time (targeted and non-targeted) adversarial attacks on the DNNs by manipulating their inputs at different stages of semantic communications. As a computer vision attack, small perturbations are injected to the images at the input of the transmitter's encoder. As a wireless attack, small perturbations signals are transmitted to interfere with the input of the receiver's decoder. By launching these stealth attacks individually or more effectively in a combined form as a multi-domain attack, we show that it is possible to change the semantics of the transferred information even when the reconstruction loss remains low. These multi-domain adversarial attacks pose as a serious threat to the semantics of information transfer (with larger impact than conventional jamming) and raise the need of defense methods for the safe adoption of semantic communications.


title: Policy Gradient in Robust MDPs with Global Convergence Guarantee

abstract: Robust Markov decision processes (RMDPs) provide a promising framework for computing reliable policies in the face of model errors. Many successful reinforcement learning algorithms build on variations of policy-gradient methods, but adapting these methods to RMDPs has been challenging. As a result, the applicability of RMDPs to large, practical domains remains limited. This paper proposes a new Double-Loop Robust Policy Gradient (DRPG), the first generic policy gradient method for RMDPs. In contrast with prior robust policy gradient algorithms, DRPG monotonically reduces approximation errors to guarantee convergence to a globally optimal policy in tabular RMDPs. We introduce a novel parametric transition kernel and solve the inner loop robust policy via a gradient-based method. Finally, our numerical results demonstrate the utility of our new algorithm and confirm its global convergence properties.


title: Perplexed by Quality: A Perplexity-based Method for Adult and Harmful Content Detection in Multilingual Heterogeneous Web Data

abstract: As demand for large corpora increases with the size of current state-of-the-art language models, using web data as the main part of the pre-training corpus for these models has become a ubiquitous practice. This, in turn, has introduced an important challenge for NLP practitioners, as they are now confronted with the task of developing highly optimized models and pipelines for pre-processing large quantities of textual data, which implies, effectively classifying and filtering multilingual, heterogeneous and noisy data, at web scale. One of the main components of this pre-processing step for the pre-training corpora of large language models, is the removal of adult and harmful content. In this paper we explore different methods for detecting adult and harmful of content in multilingual heterogeneous web data. We first show how traditional methods in harmful content detection, that seemingly perform quite well in small and specialized datasets quickly break down when confronted with heterogeneous noisy web data. We then resort to using a perplexity based approach but with a twist: Instead of using a so-called "clean" corpus to train a small language model and then use perplexity so select the documents with low perplexity, i.e., the documents that resemble this so-called "clean" corpus the most. We train solely with adult and harmful textual data, and then select the documents having a perplexity value above a given threshold. This approach will virtually cluster our documents into two distinct groups, which will greatly facilitate the choice of the threshold for the perplexity and will also allow us to obtain higher precision than with the traditional classification methods for detecting adult and harmful content.


title: First CE Matters: On the Importance of Long Term Properties on Memory Failure Prediction

abstract: Dynamic random access memory failures are a threat to the reliability of data centres as they lead to data loss and system crashes. Timely predictions of memory failures allow for taking preventive measures such as server migration and memory replacement. Thereby, memory failure prediction prevents failures from externalizing, and it is a vital task to improve system reliability. In this paper, we revisited the problem of memory failure prediction. We analyzed the correctable errors (CEs) from hardware logs as indicators for a degraded memory state. As memories do not always work with full occupancy, access to faulty memory parts is time distributed. Following this intuition, we observed that important properties for memory failure prediction are distributed through long time intervals. In contrast, related studies, to fit practical constraints, frequently only analyze the CEs from the last fixed-size time interval while ignoring the predating information. Motivated by the observed discrepancy, we study the impact of including the overall (long-range) CE evolution and propose novel features that are calculated incrementally to preserve long-range properties. By coupling the extracted features with machine learning methods, we learn a predictive model to anticipate upcoming failures three hours in advance while improving the average relative precision and recall for 21% and 19% accordingly. We evaluated our methodology on real-world memory failures from the server fleet of a large cloud provider, justifying its validity and practicality.


title: Link auxiliary field method in the Extended Hubbard model

abstract: The aim of this work is to generalize the method of Hubbard fields in fermion Quantum Monte Carlo simulation to the case of link fields. The introduced Hubbard link fields play a role of the interaction fields responsible for the attraction and repulsion of electronic excitations at the nodes. Such improvements leads to a number of advantages concerning thermalization time, autocorrelations, metastable states and probability distributions of the quantities. It is demonstrated that computations using five fields are more stable and reliable in case of energy observables.


title: Software Ecosystems: A Tertiary Study and a Thematic Model

abstract: A software ecosystem (SECO) is an interaction, communication, cooperation, and synergy among a set of players. Depending on the actors type of interaction with others, each one can play a different role. These interactions provide a set of positive relationships (symbiosis) between actors who work together around a common technology platform or a service. SECO has been explored in several studies, some related to their general characteristics and others focusing on a specific topic (e.g., requirements, governance, open-source, mobile). There are many literature reviews of different natures (e.g., systematic literature reviews and systematic mapping studies). This study presents the status of the SECO field motivated by analyzing several secondary studies published over the years. To do so, we conducted a tertiary study. From an initial set of 518 studies on the subject, we selected 22 studies. We identified the theoretical foundations used by researchers and their influences and relationships with other ecosystems. We performed a thematic synthesis and identified one high-order theme, 5 themes, 10 subthemes, and 206 categories. As a result, we proposed a thematic model for SECO containing five themes, namely: social, technical, business, management, and an evaluation theme named Software Ecosystems Assessment Models (SEAM). Our main conclusion is that relationships between SECO themes should not be seen in isolation, and it must be interpreted in a holistic approach, given the number of implications to other themes mainly related to the distinction of governance and management activities in the SECO interactions. Finally, this work provides an overview of the field and points out areas for future research, such as the need of SECO community to further investigate the results from other ecosystems, mainly from the Digital Ecosystem and Digital Business Ecosystem communities.


title: Deep Multi-Emitter Spectrum Occupancy Mapping that is Robust to the Number of Sensors, Noise and Threshold

abstract: One of the primary goals in spectrum occupancy mapping is to create a system that is robust to assumptions about the number of sensors, occupancy threshold (in dBm), sensor noise, number of emitters and the propagation environment. We show that such a system may be designed with neural networks using a process of aggregation to allow a variable number of sensors during training and testing. This process transforms the variable number of measurements into approximate log-likelihood ratios (LLRs), which are fed as a fixed-resolution image into a neural network. The use of LLR's provides robustness to the effects of noise and occupancy threshold. In other words, a system may be trained for a nominal number of sensors, threshold and noise levels, and still operate well at various other levels without retraining. Our system operates without knowledge of the number of emitters and does not explicitly attempt to estimate their number or power. Receiver operating curves with realistic propagation environments using topographic maps with commercial network design tools show how performance of the neural network varies with the environment. The use of very low-resolution sensors in this system can still yield good performance.


title: Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization

abstract: Foundation models are redefining how AI systems are built. Practitioners now follow a standard procedure to build their machine learning solutions: from a pre-trained foundation model, they fine-tune the weights on the target task of interest. So, the Internet is swarmed by a handful of foundation models fine-tuned on many diverse tasks: these individual fine-tunings exist in isolation without benefiting from each other. In our opinion, this is a missed opportunity, as these specialized models contain rich and diverse features. In this paper, we thus propose model ratatouille, a new strategy to recycle the multiple fine-tunings of the same foundation model on diverse auxiliary tasks. Specifically, we repurpose these auxiliary weights as initializations for multiple parallel fine-tunings on the target task; then, we average all fine-tuned weights to obtain the final model. This recycling strategy aims at maximizing the diversity in weights by leveraging the diversity in auxiliary tasks. Empirically, it improves the state of the art on the reference DomainBed benchmark for out-of-distribution generalization. Looking forward, this work contributes to the emerging paradigm of updatable machine learning where, akin to open-source software development, the community collaborates to reliably update machine learning models. Our code is released: https://github.com/facebookresearch/ModelRatatouille.


title: Neural Network Learner for Minesweeper

abstract: Minesweeper is an interesting single player game based on logic, memory and guessing. Solving Minesweeper has been shown to be an NP-hard task. Deterministic solvers are the best known approach for solving Minesweeper. This project proposes a neural network based learner for solving Minesweeper. To choose the best learner, different architectures and configurations of neural networks were trained on hundreds of thousands of games. Surprisingly, the proposed neural network based learner has shown to be a very good approximation function for solving Minesweeper. The neural network learner competes well with the CSP solvers, especially in Beginner and Intermediate modes of the game. It was also observed that despite having high success rates, the best neural learner was considerably slower than the best deterministic solver. This report also discusses the overheads and limitations faced while creating highly successful neural networks for Minesweeper.


title: Factorization at Next-to-Leading Power and Endpoint Divergences in $gg\to h$ Production

abstract: We derive a factorization theorem for the Higgs-boson production amplitude in gluon-gluon fusion induced by a light-quark loop, working at next-to-leading power in soft-collinear effective theory. The factorization is structurally similar to that obtained for the $h\to\gamma\gamma$ decay amplitude induced by a light-quark loop, but additional complications arise because of external color charges. We show how the refactorization-based subtraction scheme developed in previous work leads to a factorization theorem free of endpoint divergences. We use renormalization-group techniques to predict the logarithmically enhanced terms in the three-loop $gg\to h$ form factor of order $\alpha_s^3\ln^k(-M_h^2/m_b^2)$ with $k=6,5,4,3$. We also resum the first three towers of leading logarithms, $\alpha_s^n\ln^{2n-k}(-M_h^2/m_b^2)$ with $k=0,1,2$, to all orders of perturbation theory.


title: Parameter-efficient Zero-shot Transfer for Cross-Language Dense Retrieval with Adapters

abstract: A popular approach to creating a zero-shot cross-language retrieval model is to substitute a monolingual pretrained language model in the retrieval model with a multilingual pretrained language model such as Multilingual BERT. This multilingual model is fined-tuned to the retrieval task with monolingual data such as English MS MARCO using the same training recipe as the monolingual retrieval model used. However, such transferred models suffer from mismatches in the languages of the input text during training and inference. In this work, we propose transferring monolingual retrieval models using adapters, a parameter-efficient component for a transformer network. By adding adapters pretrained on language tasks for a specific language with task-specific adapters, prior work has shown that the adapter-enhanced models perform better than fine-tuning the entire model when transferring across languages in various NLP tasks. By constructing dense retrieval models with adapters, we show that models trained with monolingual data are more effective than fine-tuning the entire model when transferring to a Cross Language Information Retrieval (CLIR) setting. However, we found that the prior suggestion of replacing the language adapters to match the target language at inference time is suboptimal for dense retrieval models. We provide an in-depth analysis of this discrepancy between other cross-language NLP tasks and CLIR.


title: Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization

abstract: In long document controllable summarization, where labeled data is scarce, pretrained models struggle to adapt to the task and effectively respond to user queries. In this paper, we introduce Socratic pretraining, a question-driven, unsupervised pretraining objective specifically designed to improve controllability in summarization tasks. By training a model to generate and answer relevant questions in a given context, Socratic pretraining enables the model to more effectively adhere to user-provided queries and identify relevant content to be summarized. We demonstrate the effectiveness of this approach through extensive experimentation on two summarization domains, short stories and dialogue, and multiple control strategies: keywords, questions, and factoid QA pairs. Our pretraining method relies only on unlabeled documents and a question generation system and outperforms pre-finetuning approaches that use additional supervised data. Furthermore, our results show that Socratic pretraining cuts task-specific labeled data requirements in half, is more faithful to user-provided queries, and achieves state-of-the-art performance on QMSum and SQuALITY.


title: Is GPT-3 a Good Data Annotator?

abstract: Data annotation is the process of labeling data that could be used to train machine learning models. Having high-quality annotation is crucial, as it allows the model to learn the relationship between the input data and the desired output. GPT-3, a large-scale language model developed by OpenAI, has demonstrated impressive zero- and few-shot performance on a wide range of NLP tasks. It is therefore natural to wonder whether it can be used to effectively annotate data for NLP tasks. In this paper, we evaluate the performance of GPT-3 as a data annotator by comparing it with traditional data annotation methods and analyzing its output on a range of tasks. Through this analysis, we aim to provide insight into the potential of GPT-3 as a general-purpose data annotator in NLP.


title: Ramsey Theory and Geometry of Closed Loops

abstract: We apply the Ramsey theory to the analysis of geometrical properties of closed contours. Consider a set of six points placed on a closed contour. The straight lines connecting these points are y_ik (x)={\alpha}_ik x+\b{eta}_ik (i,k=1...6), {\alpha}_ik is not equal to 0. We color the edges connecting the points for which {\alpha}_ik>0 holds with red, and the edges for which {\alpha}_ik<0 with green with red. At least one monochromic triangle should necessarily appear within the curve (according to the Ramsey number R(3,3)=6). This result is immediately applicable for the analysis of dynamical billiards. The second theorem emerges from the combination of the Jordan curve and Ramsey theorem. The closed curve is considered. We connect the points located within the same region with green links and the points placed within the different regions with red links. In this case, transitivity/intransitivity of the relations between the points should be considered. Ramsey constructions arising from the differential geometry of closed contours are discussed.


title: Towards Sequence Utility Maximization under Utility Occupancy Measure

abstract: The discovery of utility-driven patterns is a useful and difficult research topic. It can extract significant and interesting information from specific and varied databases, increasing the value of the services provided. In practice, the measure of utility is often used to demonstrate the importance, profit, or risk of an object or a pattern. In the database, although utility is a flexible criterion for each pattern, it is a more absolute criterion due to the neglect of utility sharing. This leads to the derived patterns only exploring partial and local knowledge from a database. Utility occupancy is a recently proposed model that considers the problem of mining with high utility but low occupancy. However, existing studies are concentrated on itemsets that do not reveal the temporal relationship of object occurrences. Therefore, this paper towards sequence utility maximization. We first define utility occupancy on sequence data and raise the problem of High Utility-Occupancy Sequential Pattern Mining (HUOSPM). Three dimensions, including frequency, utility, and occupancy, are comprehensively evaluated in HUOSPM. An algorithm called Sequence Utility Maximization with Utility occupancy measure (SUMU) is proposed. Furthermore, two data structures for storing related information about a pattern, Utility-Occupancy-List-Chain (UOL-Chain) and Utility-Occupancy-Table (UO-Table) with six associated upper bounds, are designed to improve efficiency. Empirical experiments are carried out to evaluate the novel algorithm's efficiency and effectiveness. The influence of different upper bounds and pruning strategies is analyzed and discussed. The comprehensive results suggest that the work of our algorithm is intelligent and effective.


title: Pragmatic isomorphism proofs between Coq representations: application to lambda-term families

abstract: There are several ways to formally represent families of data, such as lambda terms, in a type theory such as the dependent type theory of Coq. Mathematical representations are very compact ones and usually rely on the use of dependent types, but they tend to be difficult to handle in practice. On the contrary, implementations based on a larger (and simpler) data structure combined with a restriction property are much easier to deal with. In this work, we study several families related to lambda terms, among which Motzkin trees, seen as lambda term skeletons, closable Motzkin trees, corresponding to closed lambda terms, and a parameterized family of open lambda terms. For each of these families, we define two different representations, show that they are isomorphic and provide tools to switch from one representation to another. All these datatypes and their associated transformations are implemented in the Coq proof assistant. Furthermore we implement random generators for each representation, using the QuickChick plugin.


title: Wind Power Scenario Generation Using Graph Convolutional Generative Adversarial Network

abstract: Generating wind power scenarios is very important for studying the impacts of multiple wind farms that are interconnected to the grid. We develop a graph convolutional generative adversarial network (GCGAN) approach by leveraging GAN's capability in generating large number of realistic scenarios without using statistical modeling. Unlike existing GAN-based wind power data generation approaches, we design GAN's hidden layers to match the underlying spatial and temporal characteristics. We advocate the use of graph filters to embed the spatial correlation among multiple wind farms, and a one-dimensional (1D) convolutional layer to represent the temporal feature filters. The proposed graph and feature filter design significantly reduce the GAN model complexity, leading to improvements in training efficiency and computation complexity. Numerical results using real wind power data from Australia demonstrate that the scenarios generated by the proposed GCGAN exhibit more realistic spatial and temporal statistics than other GAN-based outputs.


title: MULTI3NLU++: A Multilingual, Multi-Intent, Multi-Domain Dataset for Natural Language Understanding in Task-Oriented Dialogue

abstract: Task-oriented dialogue (TOD) systems have been widely deployed in many industries as they deliver more efficient customer support. These systems are typically constructed for a single domain or language and do not generalise well beyond this. To support work on Natural Language Understanding (NLU) in TOD across multiple languages and domains simultaneously, we constructed MULTI3NLU++, a multilingual, multi-intent, multi-domain dataset. MULTI3NLU++ extends the English only NLU++ dataset to include manual translations into a range of high, medium, and low resource languages (Spanish, Marathi, Turkish and Amharic), in two domains (BANKING and HOTELS). Because of its multi-intent property, MULTI3NLU++ represents complex and natural user goals, and therefore allows us to measure the realistic performance of TOD systems in a varied set of the world's languages. We use MULTI3NLU++ to benchmark state-of-the-art multilingual models for the NLU tasks of intent detection and slot labelling for TOD systems in the multilingual setting. The results demonstrate the challenging nature of the dataset, particularly in the low-resource language setting, offering ample room for future experimentation in multi-domain multilingual TOD setups.


title: Analysis of the Y(4500) as the vector molecular state

abstract: The study of hadronic parameters of mesons is of great importance for understanding the nature of strong interactions and QCD vacuum. One of the approaches to investigate these properties is the QCD Sum Rule Method (QCDSR), which has been extensively used for studying masses, decay constants, form factors, decay widths, and so on properties of both traditional and exotic hadrons. Recently BESIII collaboration discovered a new particle with a statistical significance greater than 8{\sigma} in the hidden-charm channel of the e^+ e^- annihilation, denoted as Y(4500). In this study, we investigate the Y(4500) state in the framework of the QCDSR, assuming this particle is a molecular state with quantum numbers J^PC=1^(--). We examined the two-point correlation function and we take into account non-perturbative contributions in OPE side evaluating up to operator dimension 10. We calculate the mass and decay constant of the Y(4500) state and obtain a mass value consistent with the results in the literature. We hope the studies presented in this paper provide new insights to establish the hidden-charm molecular spectrum.


title: Obtaining gluon propagator in a new generalized gauge

abstract: Gauge theories play a fundamental role in particle physics, nuclear physics, and cosmology. The basic idea of these theories is that the Lagrangian density should be invariant under some transformations. Lagrangian invariance implies a certain freedom in defining gauge fields. In this study, the standard path integral quantization formalism is used. Gauge degrees of freedom manifest themselves in the difficulties in obtaining gauge field propagators. For a consistent quantization, it is necessary to eliminate non-physical gauge degrees of freedom. The standard procedure is to break the gauge symmetry by applying a gauge condition. In this work, we introduced a new generalized gauge condition $W_{\mu} A^{a\mu}=0$, where $W_{\mu}=\lambda \partial_{\mu}+\beta n_{\mu}$, $n_{\mu}$ is an arbitrary constant four-vector, $\lambda$, and $\beta$ are real constant parameters. Using standard path integral quantization formalism, we obtained gluon propagators expressions in this new gauge. In the $\lambda=1,\beta\rightarrow0$, and $\beta=1, \lambda\rightarrow0$ limit cases, the obtained expression provides us the gluon propagators expressions available in the literature in covariant and non-covariant gauges, respectively. This study can give us a different perspective on quantization in Yang-Mills (YM) theories and can show the way to some ambiguities in quantum field theories (QFT).


title: Base Station Selection and Task Offloading of the Mobile Edge Computing System

abstract: Based on the two decision variables, service location and base station selection, construct a computational model of the switching delay, communication delay, and queuing delay patterns of a mobile edge computing system in each time horizon; minimize the non-switching latency to obtain the service deployment and base station selection decision at the initial time horizon; compute the switching latency and non-switching latency of the current time horizon based on the decision of the previous time horizon, and determine the current time horizon based on the principle of larger non-switching latency tolerance. If the service is not reallocated, the decision for the current time slot remains the same as the decision for the previous time slot; if the service is reallocated, the non-reach time of the current time slot is minimized and the service allocation and base station selection decision for the current time slot is obtained; the service allocation and base station selection decisions for all time slots are iteratively computed. This paper provides low-latency processing and quality of service for users to satisfy the randomness of user movement.


title: Pareto Pairwise Ranking for Fairness Enhancement of Recommender Systems

abstract: Learning to rank is an effective recommendation approach since its introduction around 2010. Famous algorithms such as Bayesian Personalized Ranking and Collaborative Less is More Filtering have left deep impact in both academia and industry. However, most learning to rank approaches focus on improving technical accuracy metrics such as AUC, MRR and NDCG. Other evaluation metrics of recommender systems like fairness have been largely overlooked until in recent years. In this paper, we propose a new learning to rank algorithm named Pareto Pairwise Ranking. We are inspired by the idea of Bayesian Personalized Ranking and power law distribution. We show that our algorithm is competitive with other algorithms when evaluated on technical accuracy metrics. What is more important, in our experiment section we demonstrate that Pareto Pairwise Ranking is the most fair algorithm in comparison with 9 other contemporary algorithms.


title: PoissonMat: Remodeling Matrix Factorization using Poisson Distribution and Solving the Cold Start Problem without Input Data

abstract: Matrix Factorization is one of the most successful recommender system techniques over the past decade. However, the classic probabilistic theory framework for matrix factorization is modeled using normal distributions. To find better probabilistic models, algorithms such as RankMat, ZeroMat and DotMat have been invented in recent years. In this paper, we model the user rating behavior in recommender system as a Poisson process, and design an algorithm that relies on no input data to solve the recommendation problem and the cold start issue at the same time. We prove the superiority of our algorithm in comparison with matrix factorization, random placement, Zipf placement, ZeroMat, DotMat, etc.


title: Go-tuning: Improving Zero-shot Learning Abilities of Smaller Language Models

abstract: With increasing scale, large language models demonstrate both quantitative improvement and new qualitative capabilities, especially as zero-shot learners, like GPT-3. However, these results rely heavily on delicate prompt design and large computation. In this work, we explore whether the strong zero-shot ability could be achieved at a smaller model scale without any external supervised data. To achieve this goal, we revisit masked language modeling and present a geometry-guided self-supervised learning method (Go-tuningfor short) by taking a small number of task-aware self-supervised data to update language models further. Experiments show that Go-tuning can enable T5-small (80M) competitive zero-shot results compared with large language models, such as T5-XL (3B). We also apply Go-tuning on multi-task settings and develop a multi-task model, mgo-T5 (250M). It can reach the average performance of OPT (175B) on 9 datasets.


title: Radiation of a Massless Closed String from Interacting D$p$-branes with Background Fields in the Superstring Theory

abstract: In the context of the superstring theory, we calculate the radiation amplitude of a general massless closed string from the interaction of two parallel D$p$-branes. The background field is the constant antisymmetric tensor $B_{\mu\nu}$, and the branes have been equipped with the different $U(1)$ gauge fields. For the large distance of the branes, the Kalb-Ramond emission will be investigated.


title: Structurally damped $\sigma-$evolution equations with power-law memory

abstract: We consider an integro-differential counterpart of the $\sigma-$evolution equation of the type \[ \partial_t^2 u(t,x)+\mu (-\Delta)^{\frac{\sigma}{2}} \partial_t u(t,x)+(-\Delta)^\sigma u(t,x)=f(t,x), \] with $\sigma>0$ and $\mu>0$, that encodes memory of \textit{power-law} type. To do so, we replace the time derivatives $\partial_t$ and $\partial_t^2$ by the so-called Caputo-Djrbashian derivatives $\partial_t^\gamma$ of order $\gamma=\alpha$ and $\gamma=2\alpha$, respectively, and the inhomogeneous term $f(t,x)$ by the Riemann-Liouville integral $I^{\beta-2\alpha}_{0^+}f(t,x)$, whereby $0<\alpha\leq 1$ and $2\alpha\leq \beta<2\alpha+1$. For the solution representation of the underlying Cauchy problems on the space-time $[0,T]\times \mathbb{R}^n$ we then consider a wide class of pseudo-differential operators $\displaystyle (-\Delta)^{\frac{\eta}{2}}E_{\alpha,\beta}\left(~-\lambda(-\Delta)^{\frac{\sigma}{2}} t^\alpha~\right)$, endowed by the fractional Laplacian $-(-\Delta)^{\frac{\sigma}{2}}$ and the two-parameter Mittag-Leffler functions $E_{\alpha,\beta}$. On our approach we are also able to provide dispersive and Strichartz estimates for the solutions with the aid of decay properties of $E_{\alpha,\beta}(-z)$ ($z\in \mathbb{C}$) and the boundedness properties of the Hankel transform.


title: Quantum and semi--quantum key distribution in networks

abstract: In this paper, we utilize the potential offered by multidimensional separable states (MSS) for secure and simultaneous distributions of keys in a layered network. We present protocols for both quantum and semi-quantum key distribution and discuss their robustness against various eavesdropping strategies. We provide a procedure to identify the requisite resource states to generalise these protocols for arbitrary layered networks. Finally, we study the interrelation between the local dimensionalities of states and achievable key rates in a given layer. These proposals are realisable with current technology, thanks to employment of MSS and many advances in generation, manipulation and measurement of higher-dimensional orbital angular momentum states of light.


title: SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization

abstract: Data scarcity has been a long standing issue in the field of open-domain social dialogue. To quench this thirst, we present SODA: the first publicly available, million-scale high-quality social dialogue dataset. By contextualizing social commonsense knowledge from a knowledge graph, we are able to distill an exceptionally broad spectrum of social interactions from a large language model. Human evaluation shows that conversations in SODA are more consistent, specific, and (surprisingly) natural than those in prior human-authored datasets. Using SODA, we train COSMO: a generalizable conversation model that is significantly more natural and consistent on unseen datasets than best-performing conversation models (e.g., GODEL, BlenderBot-1, Koala, Vicuna). Experiments reveal COSMO is sometimes even preferred to the original human-written gold responses. Additionally, our results shed light on the distinction between knowledge-enriched conversations and natural social chitchats. We plan to make our data, model, and code public.


title: Controllable Text Generation with Language Constraints

abstract: We consider the task of text generation in language models with constraints specified in natural language. To this end, we first create a challenging benchmark Cognac that provides as input to the model a topic with example text, along with a constraint on text to be avoided. Unlike prior work, our benchmark contains knowledge-intensive constraints sourced from databases like Wordnet and Wikidata, which allows for straightforward evaluation while striking a balance between broad attribute-level and narrow lexical-level controls. We find that even state-of-the-art language models like GPT-3 fail often on this task, and propose a solution to leverage a language model's own internal knowledge to guide generation. Our method, called CognacGen, first queries the language model to generate guidance terms for a specified topic or constraint, and uses the guidance to modify the model's token generation probabilities. We propose three forms of guidance (binary verifier, top-k tokens, textual example), and employ prefix-tuning approaches to distill the guidance to tackle diverse natural language constraints. Through extensive empirical evaluations, we demonstrate that CognacGen can successfully generalize to unseen instructions and outperform competitive baselines in generating constraint conforming text.


title: Generic Temporal Reasoning with Differential Analysis and Explanation

abstract: Temporal reasoning is the task of predicting temporal relations of event pairs. While temporal reasoning models can perform reasonably well on in-domain benchmarks, we have little idea of these systems' generalizability due to existing datasets' limitations. In this work, we introduce a novel task named TODAY that bridges this gap with temporal differential analysis, which as the name suggests, evaluates whether systems can correctly understand the effect of incremental changes. Specifically, TODAY introduces slight contextual changes for given event pairs, and systems are asked to tell how this subtle contextual change would affect relevant temporal relation distributions. To facilitate learning, TODAY also annotates human explanations. We show that existing models, including GPT-3.5, drop to random guessing on TODAY, suggesting that they heavily rely on spurious information rather than proper reasoning for temporal predictions. On the other hand, we show that TODAY's supervision style and explanation annotations can be used in joint learning, encouraging models to use more appropriate signals during training and thus outperform across several benchmarks. TODAY can also be used to train models to solicit incidental supervision from noisy sources such as GPT-3.5, thus moving us more toward the goal of generic temporal reasoning systems.


title: Super-resolution enhancement in bi-photon spatial mode demultiplexin

abstract: Imaging systems measuring intensity in the far field succumb to Rayleigh's curse, a resolution limitation dictated by the finite aperture of the optical system. Many proof-of-principle and some two-dimensional imaging experiments have shown that, by using spatial mode demultiplexing (SPADE), the field information collected is maximal, and thus, the resolution increases beyond the Rayleigh criterion. Hitherto, the SPADE approaches are based on resolving the lateral splitting of a Gaussian wavefunction. Here, we consider the case in which the light field originates from a bi-photon source, i.e. spontaneous parametric down-conversion, and a horizontal separation is introduced in one of the two photons. We show that a separation induced in the signal photon arm can be super-resolved using coincidence measurements after projecting both photons on Hermite-Gauss modes. Remarkably the Fisher information associated with the measurement is enhanced compared to the ordinary SPADE techniques by $\sqrt{K}$, where $K$ is the Schmidt number of the two-photon state that quantifies the amount of spatial entanglement between the two photons.


title: BMX: Boosting Natural Language Generation Metrics with Explainability

abstract: State-of-the-art natural language generation evaluation metrics are based on black-box language models. Hence, recent works consider their explainability with the goals of better understandability for humans and better metric analysis, including failure cases. In contrast, our proposed method BMX: Boosting Natural Language Generation Metrics with explainability explicitly leverages explanations to boost the metrics' performance. In particular, we perceive feature importance explanations as word-level scores, which we convert, via power means, into a segment-level score. We then combine this segment-level score with the original metric to obtain a better metric. Our tests show improvements for multiple metrics across MT and summarization datasets. While improvements in machine translation are small, they are strong for summarization. Notably, BMX with the LIME explainer and preselected parameters achieves an average improvement of 0.087 points in Spearman correlation on the system-level evaluation of SummEval.


title: Spin Squeezing with Itinerant Dipoles: A Case for Shallow Lattices

abstract: Entangled spin squeezed states generated via dipolar interactions in lattice models provide unique opportunities for quantum enhanced sensing and are now within reach of current experiments. A critical question in this context is which parameter regimes offer the best prospects under realistic conditions. Light scattering in deep lattices can induce significant decoherence and strong Stark shifts, while shallow lattices face motional decoherence as a fundamental obstacle. Here we analyze the interplay between motion and spin squeezing in itinerant fermionic dipoles in one dimensional chains using exact matrix product state simulations. We demonstrate that shallow lattices can achieve more than 5dB of squeezing, outperforming deep lattices by up to more than 3dB, even in the presence of low filling, loss and decoherence. We relate this finding to SU(2)-symmetric superexchange interactions, which keep spins aligned and protect collective correlations. We show that the optimal regime is achieved for small repulsive off-site interactions, with a trade-off between maximal squeezing and optimal squeezing time.


title: Little Red Riding Hood Goes Around the Globe:Crosslingual Story Planning and Generation with Large Language Models

abstract: Previous work has demonstrated the effectiveness of planning for story generation exclusively in a monolingual setting focusing primarily on English. We consider whether planning brings advantages to automatic story generation across languages. We propose a new task of cross-lingual story generation with planning and present a new dataset for this task. We conduct a comprehensive study of different plans and generate stories in several languages, by leveraging the creative and reasoning capabilities of large pre-trained language models. Our results demonstrate that plans which structure stories into three acts lead to more coherent and interesting narratives, while allowing to explicitly control their content and structure.


title: Anomalous softness in amorphous matter in the reversible plastic regime

abstract: We study an integer automaton elasto-plastic model of an amorphous solid subject to cyclic shear of amplitude $\Gamma$. We focus on the reversible plastic regime at intermediate $\Gamma_0<\Gamma<\Gamma_y$, where, after a transient, the system settles into a periodic limit cycle with hysteretic, dissipative plastic events which repeat after an integer number of cycles. We study the plastic strain rate, $\frac{d\epsilon}{d\gamma}$, (where $\gamma$ is the applied strain and $\epsilon$ is the plastic strain) during the terminal limit cycles and show that it consists of a creeping regime at low $\gamma$ with very low $\frac{d\epsilon}{d\gamma}$ followed by a sharp transition at a characteristic strain, $\gamma_*$, and stress, $\sigma_*$, to a flowing regime with higher $\frac{d\epsilon}{d\gamma}$. We show that while increasing $\Gamma$ above $\Gamma_0$ results in lower terminal ground state energy, $U_{\text{min}}$, and a correspondingly narrower distribution of stresses, it, surprisingly, results in lower $\gamma_*$, and $\sigma_*$. The stress distribution, $P(\sigma)$, also becomes skewed for $\Gamma>\Gamma_0$. That is, the systems in the RPR are anomalously soft and mechanically polarized. We relate this to an emergent characteristic feature in the stress distribution, $P(\sigma)$, at a value, $\sigma_0$, which is independent of $\Gamma$ and show that $\sigma_0$ implies a relation between the $\Gamma$ dependence of $\sigma_*$, $\gamma_*$, and the amplitude of plastic strain, $\epsilon_p$. We show that the onset of hysteresis is characterized by a power-law scaling, indicative of a second order transition with $\epsilon_p\propto (\Gamma-\Gamma_0)^{1.2\pm0.1}$. We argue that $\sigma_0$ and, correspondingly, the onset of the RPR at $\Gamma=\Gamma_0$, is simply set by the so-called Eshelby-stress. Furthermore, we show that cycling at $\Gamma_0$ results in a maximally hardened state.


title: On nonlinear Kantorovich problems for cost functions of a special form

abstract: We study Kantorovich type optimal transportation problems with nonlinear cost functions, including dependence on conditional measures of transport plans. A range of nonlinear Kantorovich problems for cost functions of a special form is considered and results on existence (or non-existence) of optimal solutions are proved. We also establish the connection between the nonlinear Kantorovich problem with the cost function of some special form and the Monge problem with convex dominance.


title: ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models

abstract: State-of-the-art poetry generation systems are often complex. They either consist of task-specific model pipelines, incorporate prior knowledge in the form of manually created constraints, or both. In contrast, end-to-end models would not suffer from the overhead of having to model prior knowledge and could learn the nuances of poetry from data alone, reducing the degree of human supervision required. In this work, we investigate end-to-end poetry generation conditioned on styles such as rhyme, meter, and alliteration. We identify and address lack of training data and mismatching tokenization algorithms as possible limitations of past attempts. In particular, we successfully pre-train ByGPT5, a new token-free decoder-only language model, and fine-tune it on a large custom corpus of English and German quatrains annotated with our styles. We show that ByGPT5 outperforms other models such as mT5, ByT5, GPT-2 and ChatGPT, while also being more parameter efficient and performing favorably compared to humans. In addition, we analyze its runtime performance and demonstrate that it is not prone to memorization. We make our code, models, and datasets publicly available.


title: On the major role played by the curvature of intracranial aneurysms walls in determining their mechanical response, local hemodynamics, and rupture likelihood

abstract: The properties of intracranial aneurysms (IAs) walls are known to be driven by the underlying hemodynamics adjacent to the IA sac. Different pathways exist explaining the connections between hemodynamics and local tissue properties. The emergence of such theories is essential if one wishes to compute the mechanical response of a patient-specific IA wall and predict its rupture. Apart from the hemodynamics and tissue properties, one could assume that the mechanical response also depends on the local morphology, more specifically, the wall curvature, with larger values at highly-curved wall portions. Nonetheless, this contradicts observations of IA rupture sites more often found at the dome, where the curvature is lower. This seeming contradiction indicates a complex interaction between local hemodynamics, wall morphology, and mechanical response, which warrants further investigation. This was the main goal of this work. We accomplished this by analysing the stress and stretch fields in different regions of the wall for a sample of IAs, which have been classified based on particular local hemodynamics and local curvature. Pulsatile numerical simulations were performed using the one-way fluid-solid interaction strategy implemented in OpenFOAM (solids4foam toolbox). We found that the variable best correlated with regions of high stress and stretch was the wall curvature. Additionally, our data suggest a connection between the local curvature and local hemodynamics, indicating that the curvature is a property that could be used to assess both mechanical response and hemodynamic conditions, and, moreover, to suggest new metrics based on the curvature to predict the likelihood of rupture.


title: Pierre Auger Observatory and Super Heavy Dark Matter

abstract: We briefly discuss the connection of the Pierre Auger Observatory data with a large class of dark matter models based on the early universe generation of super heavy particles, their role in the solution of the dark matter problem, highlighting the remarkable constraining capabilities of the Auger observations.


title: Generalized Simultaneous Perturbation-based Gradient Search with Reduced Estimator Bias

abstract: We present in this paper a family of generalized simultaneous perturbation-based gradient search (GSPGS) estimators that use noisy function measurements. The number of function measurements required by each estimator is guided by the desired level of accuracy. We first present in detail unbalanced generalized simultaneous perturbation stochastic approximation (GSPSA) estimators and later present the balanced versions (B-GSPSA) of these. We extend this idea further and present the generalized smoothed functional (GSF) and generalized random directions stochastic approximation (GRDSA) estimators, respectively, as well as their balanced variants. We show that estimators within any specified class requiring more number of function measurements result in lower estimator bias. We present a detailed analysis of both the asymptotic and non-asymptotic convergence of the resulting stochastic approximation schemes. We further present a series of experimental results with the various GSPGS estimators on the Rastrigin and quadratic function objectives. Our experiments are seen to validate our theoretical findings.


title: Machine Learning and Polymer Self-Consistent Field Theory in Two Spatial Dimensions

abstract: A computational framework that leverages data from self-consistent field theory simulations with deep learning to accelerate the exploration of parameter space for block copolymers is presented. This is a substantial two-dimensional extension of the framework introduced in [1]. Several innovations and improvements are proposed. (1) A Sobolev space-trained, convolutional neural network (CNN) is employed to handle the exponential dimension increase of the discretized, local average monomer density fields and to strongly enforce both spatial translation and rotation invariance of the predicted, field-theoretic intensive Hamiltonian. (2) A generative adversarial network (GAN) is introduced to efficiently and accurately predict saddle point, local average monomer density fields without resorting to gradient descent methods that employ the training set. This GAN approach yields important savings of both memory and computational cost. (3) The proposed machine learning framework is successfully applied to 2D cell size optimization as a clear illustration of its broad potential to accelerate the exploration of parameter space for discovering polymer nanostructures. Extensions to three-dimensional phase discovery appear to be feasible.

categories: cond-mat.mtrl-sci cs.CE cs.LG cs.NA math.NA physics.comp-ph


title: Alexandrov's embedding theorem

abstract: Written for the book "Mathematicians from Saint Petersburg and their theorems".


title: A note on Flavell's theorem associated with Frobenius groups

abstract: Let G be a Frobenius group with the Frobenius kernel K. Suppose that G contains a nontrival subgroup D \subseteq K such that the normalizer N_G(D) \not\subseteq K. When D is no 2-group, Flavell proved, without using character theory, that K is a subgroup of G. Based on this result, we further prove that K is a subgroup when D is a 2-group.


title: Execution-Based Evaluation for Open-Domain Code Generation

abstract: To extend the scope of coding queries to more realistic settings, we propose ODEX, the first Open-Domain EXecution-based natural language (NL) to Python code generation dataset. ODEX has 945 NL-Code pairs spanning 79 diverse libraries, along with 1,707 human-written test cases for execution. Our NL-Code pairs are harvested from StackOverflow forums to encourage natural and practical coding queries. Moreover, ODEX supports four natural languages as intents, in English, Spanish, Japanese, and Russian. ODEX unveils intriguing behavioral differences among top-performing code language models (LM). While CODEX achieves better overall results, CODEGEN improves effectively via scaling -- CODEGEN 6.1B performs comparably with CODEX 12B. Both models show substantial gaps between open and closed domains, but CODEGEN gaps tend to decrease with model size while CODEX gaps increase. We release ODEX to facilitate research into open-domain problems for the code generation community.


title: Performance of Uncoded Implementation of Grover's Algorithm on Today's Quantum Processors

abstract: This work tests the performance of Grover's search circuits on some IBM superconducting quantum devices in case of the size of search space $N=2^4$ and $N=2^5$. Ideally, we expect to get an outcome probability distribution that is clearly peaked at the goal (marked) state. However, the quantum circuit executed on real devices is vulnerable to noise which leads to fluctuations in the results. The contributions of the paper are therefore the following: a) it presents two new Grover's search circuits for $N=16$ which were not yet reported in the state of the art; b) it shows performance difference between simulation results and results obtained on real devices; c) it shows the need of adding error-correction on the circuit for $N\ge 2^5$.


title: Quasar UV/X-ray relation luminosity distances are shorter than reverberation-measured radius-luminosity relation luminosity distances

abstract: We use measurements of 59/58 quasars (QSOs), over a redshift range $0.0041\leq z \leq 1.686$, to do a comparative study of the radius--luminosity ($R-L$) and X-ray$-$UV luminosity ($L_X-L_{UV}$) relations and the implication of these relations for cosmological parameter estimation. By simultaneously determining $R-L$ or $L_X-L_{UV}$ relation parameters and cosmological parameters in six different cosmological models, we find that both $R-L$ and $L_X-L_{UV}$ relations are standardizable but provide only weak cosmological parameter constraints, with $L_X-L_{UV}$ relation data favoring larger current non-relativistic matter density parameter $\Omega_{m0}$ values than $R-L$ relation data and most other available data. We derive $L_X-L_{UV}$ and $R-L$ luminosity distances for each of the sources in the six cosmological models and find that $L_X-L_{UV}$ relation luminosity distances are shorter than $R-L$ relation luminosity distances as well as standard flat $\Lambda$CDM model luminosity distances. This explains why $L_X-L_{UV}$ relation QSO data favor larger $\Omega_{m0}$ values than do $R-L$ relation QSO data or most other cosmological measurements. While our sample size is small and only spans a small $z$ range, these results indicate that more work is needed to determine whether the $L_X-L_{UV}$ relation can be used as a cosmological probe.


title: DIS dijet production at next-to-eikonal accuracy in the CGC

abstract: We compute dijet production in Deep Inelastic Scattering at low $x$ in the dipole formalism at next-to-eikonal accuracy. We calculate the contributions induced by single photon exchange of either longitudinal or transverse polarization. We include all types of corrections to the eikonal approximation in the gluon background field: (i) finite longitudinal width of the target, (ii) interaction of the quark-antiquark pair with the subleading (transverse) component of the background field and (iii) dynamics of the target which is encoded in the $z^-$-coordinate dependence of the background field. The final expressions for the dijet cross section are written as sum of a "generalized eikonal" contribution (where longitudinal momentum $p^+$ exchange between the target and the incoming quark-antiquark pair is allowed since the average $z^-$ dependence of the background field is kept) and explicit next-to-eikonal corrections that involve decorated dipole and quadrupole operators.


title: The effects of rotation on massive star spectroscopic observables

abstract: Rotation is ubiquitous among massive stars. With rotation comes a deformation to the surface geometry, which in turn leads to alterations in the distribution of parameters across the surface including surface gravity, temperature and ionization balance of surface elements. Often, these 3D effects are neglected when analyzing spectra of rapidly rotating massive stars. We aim to determine whether neglecting the 3D deformations resulting from rapid rotation has an impact on the final spectroscopic observables, and if so to what degree. Using the SPAMMS code, we generate a grid of synthetic spectra that account for the 3D geometry of rapidly rotating stars and compare them to synthetic spectra generated assuming spherical geometry. Using equivalent width and full width half maximum measurements as proxies, we determine how the measured temperature, helium abundance and projected rotation rates of individual lines in different ionization states vary with rotation rate and inclination. We find that the 3D geometry can have a significant impact on the measured parameters. We show that the temperature is highly dependent on both the rotation rate and the inclination, and that the same system viewed at different inclinations can have measured temperatures that differ by as much as 10\%. We also find that the helium abundance can be underestimated by as much as 60\%, and that lines in different ionization states can have measurable differences in rotation rates. We demonstrate that these differences in rotation rates can be seen in observed data and show that this could allow for an inclination independent measurement of the rotational velocity. Our results indicate that neglecting the 3D effects of rotation can cause significant biases in the measured spectroscopic parameters, and that in many cases, the measured values are more than 3$\sigma$ away from the true values.


title: Effective-one-body formalism for leading-order radiative effects in the post-linear framework

abstract: In recent years, significant progress has been made in the computation of conservative and dissipative scattering observables using the post-Minkowskian approach to gravitational dynamics. However, for accurate modeling of unbound orbits, an appropriate effective-one-body (EOB) resummation of the post-Minkowski results that also accounts for dissipative dynamics is desirable. As a step in this direction, we consider the electromagnetic analog of this problem here. We show that a six-parameter equation of motion encapsulates the effective-one-body dynamics for the electromagnetic scattering problem appropriate to third-order in the coupling constant. Three of these six parameters describe the conservative part of the dynamics, while the rest correspond to the radiation-reaction effects. Here we show that only two radiation-reaction-related parameters are important at the desired order, making the effective number of parameters in our formalism to be five. We compute the explicit forms of these five parameters by matching EOB scattering observables to that of the original two-body ones computed by [Saketh et al., Phys.Rev.Res. 4 (2022) 1]. Interestingly, our formalism leads to a conjecture for the sub-leading angular momentum loss, for which no precise computations exist. In addition, we demonstrate that the bound-orbit observables computed using our method are in perfect agreement with those calculated using unbound-to-bound analytical continuation techniques. Lastly, we qualitatively discuss the extension of our formalism to gravity.


title: Finite temperature energy-momentum tensor in compactified cosmic string spacetime

abstract: In this paper we analyze the expectation value of the field squared and the energy-momentum tensor associated with a massive charged scalar quantum field with a nonzero chemical potential propagating in a high-dimensional compactified cosmic string spacetime in thermal equilibrium at finite temperature $T$. Moreover, we assume that the charged quantum field interacts with a very thin magnetic flux running along the core of the idealized cosmic string, and with a magnetic flux enclosed by the compact dimension. These observables are expressed as the vacuum expectation values and the finite temperature contributions coming from the particles and antiparticles excitations. Due to the compactification, the thermal corrections can be decomposed in a part induced by the cosmic string spacetime without compactification, plus a contribution induced by the compactification. This decompositions explicitly follows from the Abel-Plana formula used to proceed the summation over the discrete quantum number associated with the quasiperiodic condition imposed on the quantum field along the compact dimension. The expectations values of the field squared and the energy-momentum tensor are even periodic functions of the magnetic flux with period being the quantum flux, and also even functions of the chemical potential. Our main objectives in this paper concern in the investigation of the thermal corrections only. In this way we explicitly calculate the behavior of these observables in the limits of low and high temperature. We show that the temperature enhance the induced densities. In addition some graphs are also included in order to exhibit these behaviors.


title: Derived Grassmannians and derived Schur functors

abstract: This paper develops two theories, the geometric theory of derived Grassmannians (and flag schemes) and the algebraic theory of derived Schur (and Weyl) functors, and establishes their connection, a derived generalization of the Borel-Weil-Bott theorem. More specifically: (1) The theory of derived Grassmannians and flag schemes is the natural extension of the theory of derived projectivizations [arXiv:2202.11636] and generalizes Grothendieck's theory of Grassmannians and flag schemes of sheaves to the case of complexes. We establish their fundamental properties and study various natural morphisms among them. (2) The theory of derived Schur and Weyl functors extends the classical theory of Schur and Weyl module functors studied in $\mathrm{GL}_n(\mathbb{Z})$-representation theory to the case of complexes. We show that these functors have excellent functorial properties and satisfy derived generalizations of classical formulae such as Cauchy's decomposition formula, direct-sum decomposition formula and Littlewood-Richardson rule. We also generalize various results from the case of derived symmetric powers to derived Schur functors, such as Illusie-Lurie's d\'ecalage isomorphism. (3) These two theories are connected by a derived version of the Borel-Weil-Bott theorem, which generalizes the classical Borel-Weil-Bott theorem and calculates the derived pushforwards of tautological perfect complexes on derived flag schemes in terms of derived Schur functors when the complexes have perfect-amplitude $\le 1$ and positive ranks.


title: Double Higgs production at NNLO interfaced to parton showers in GENEVA

abstract: In this work, we study the production of Higgs boson pairs at next-to-next-to-leading order in QCD matched to parton showers, using the \textsc{Geneva} framework and working in the heavy-top-limit approximation. This includes the resummation of large logarithms of the zero-jettiness $\mathcal{T}_0$ up to the next-to-next-to-next-to-leading-log accuracy. This process features an extremely large momentum transfer, which makes its study particularly relevant for matching schemes such as that employed in \textsc{Geneva}, where the resummation of a variable different from that used in the ordering of the parton shower is used. To further study this effect, we extend the original shower interface designed for \textsc{Pythia8} to include other parton showers, such as \textsc{Dire} and \textsc{Sherpa}.


title: Short \& intermediate distance HVP contributions to muon g-2: SM (lattice) prediction versus $e^+e^-$ annihilation data

abstract: We present new lattice results of the ETM Collaboration, obtained from extensive simulations of lattice QCD with dynamical up, down, strange and charm quarks at physical mass values, different volumes and lattice spacings, concerning the SM prediction for the so-called intermediate window (W) and short-distance (SD) contributions to the leading order hadronic vacuum polarization (LO-HVP) term of the muon anomalous magnetic moment, $a_\mu$. Results for $a_{\mu}^{\rm LO-HVP,W}$ and $a_{\mu}^{\rm LO-HVP,SD}$, besides representing a step forward to a complete lattice computation of $a_{\mu}^{\rm LO-HVP}$ and a useful benchmark among lattice groups, are compared here with their dispersive counterparts based on experimental data for $e^+e^-$ into hadrons. The comparison confirms the tension in $a_{\mu}^{\rm LO-HVP,W}$, already noted in 2020 by the BMW Collaboration, while showing no tension in $a_{\mu}^{\rm LO-HVP,SD}$.


title: Twistor quadrics and black holes

abstract: A simple procedure is given to construct curved, non-self-dual (complexified) Kaehler metrics on space-time in terms of deformations of holomorphic quadric surfaces in flat twistor space. Imposing Lorentzian reality conditions, the Schwarzschild, Kerr, and Plebanski-Demianski space-times (among others) are derived as examples of the construction.


title: Black hole lensing in Kerr-de Sitter spacetimes

abstract: We have derived analytical solutions using Jacobi elliptic functions for bound and nearly bound photon orbits in Kerr-de Sitter (KdS) and Kerr-de Sitter Revisited (RKdS) spacetimes. Leveraging our obtained solutions, we have conducted an analytic ray-tracing in both spacetimes. We have obtained direct images, lensing rings and photon rings for equatorial disks considering inclined locally static observers. Images corresponding to n=(2,3) exhibit a significantly closer resemblance to the critical curve as compared to the n=1 image. This highlights the remarkable potential of these higher-order images as robust testing grounds for General Relativity (GR). Furthermore in both spacetimes, we have obtained analytical solutions for the critical parameters governing the structure of the photon ring and analyzed these parameters in details.


title: Performance versus Resilience in Modern Quark-Gluon Tagging

abstract: Discriminating quark-like from gluon-like jets is, in many ways, a key challenge for many LHC analyses. First, we use a known difference in Pythia and Herwig simulations to show how decorrelated taggers would break down when the most distinctive feature is aligned with theory uncertainties. We propose conditional training on interpolated samples, combined with a controlled Bayesian network, as a more resilient framework. The interpolation parameter can be used to optimize the training evaluated on a calibration dataset, and to test the stability of this optimization. The interpolated training might also be useful to track generalization errors when training networks on simulation.


title: The ordered exponential representation of GKM using the $W_{1+\infty}$ operator

abstract: The generalized Kontsevich model (GKM) is a one-matrix model with arbitrary potential. Its partition function belongs to the KP hierarchy. When the potential is monomial, it is an $r$-reduced tau-function that governs the $r$-spin intersection numbers. In this paper, we present an ordered exponential representation of monomial GKM in terms of the $W_{1+\infty}$ operators that preserves the KP integrability. In fact, this representation is naturally the solution of a $W_{1+\infty}$ constraint that uniquely determines the tau-function. Furthermore, we show that, for the cases of Kontsevich-Witten and generalized BGW tau-functions, their $W_{1+\infty}$ representations can be reduced to their cut-and-join representations under the reduction of the even time independence and Virasoro constraints.


title: The density of integral quadratic forms having a $k$-dimensional totally isotropic subspace

abstract: We investigate the probability that a random quadratic form in ${\mathbb{Z}}[x_1,...,x_n]$ has a totally isotropic subspace of a given dimension. We show that this global probability is a product of local probabilities. Our main result computes these local probabilities for quadratic forms over the $p$-adics. The formulae we obtain are rational functions in $p$ invariant upon substituting $p \mapsto 1/p$.


title: Precise Zero-Shot Dense Retrieval without Relevance Labels

abstract: While dense retrieval has been shown effective and efficient across tasks and languages, it remains difficult to create effective fully zero-shot dense retrieval systems when no relevance label is available. In this paper, we recognize the difficulty of zero-shot learning and encoding relevance. Instead, we propose to pivot through Hypothetical Document Embeddings~(HyDE). Given a query, HyDE first zero-shot instructs an instruction-following language model (e.g. InstructGPT) to generate a hypothetical document. The document captures relevance patterns but is unreal and may contain false details. Then, an unsupervised contrastively learned encoder~(e.g. Contriever) encodes the document into an embedding vector. This vector identifies a neighborhood in the corpus embedding space, where similar real documents are retrieved based on vector similarity. This second step ground the generated document to the actual corpus, with the encoder's dense bottleneck filtering out the incorrect details. Our experiments show that HyDE significantly outperforms the state-of-the-art unsupervised dense retriever Contriever and shows strong performance comparable to fine-tuned retrievers, across various tasks (e.g. web search, QA, fact verification) and languages~(e.g. sw, ko, ja).


title: Rare $b$ decays meet high-mass Drell-Yan

abstract: Rare $b$ hadron decays are considered excellent probes of new semileptonic four-fermion interactions of microscopic origin. However, the same interactions also correct the high-mass Drell-Yan tails. In this work, we revisit the first statement in the context of this complementarity and chart the space of short-distance new physics that could show up in rare $b$ decays. We analyze the latest $b \to q \ell^+ \ell^-$ measurements, where $q = d$ or $s$ and $\ell = e$ or $\mu$, including the most recent LHCb $R_{K^{(*)}}$ update, together with the latest charged and neutral current high-mass Drell-Yan data, $p p \to \ell \nu$ and $p p \to \ell^+ \ell^-$. We implement a sophisticated interpretation pipeline within the flavio framework, allowing us to investigate the multidimensional SMEFT parameter space thoroughly and efficiently. To showcase the new functionalities of flavio, we construct several explicit models featuring either a $Z'$ or a leptoquark, which can explain the tension in $b \to s \mu^+ \mu^-$ angular distributions and branching fractions while predicting lepton flavor universality (LFU) ratios to be SM-like, $R_{K^{(*)}} \approx R^{{\rm SM}}_{K^{(*)}}$, as indicated by the recent data. Those models are then confronted against the global likelihood, including the high-mass Drell-Yan, either finding tensions or compatibility.


title: SimpleStyle: An Adaptable Style Transfer Approach

abstract: Attribute-controlled text rewriting, also known as text style-transfer, has a crucial role in regulating attributes and biases of textual training data and a machine generated text. In this work we present SimpleStyle, a minimalist yet effective approach for style-transfer composed of two simple ingredients: controlled denoising and output filtering. Despite the simplicity of our approach, which can be succinctly described with a few lines of code, it is competitive with previous state-of-the-art methods both in automatic and in human evaluation. To demonstrate the adaptability and practical value of our system beyond academic data, we apply SimpleStyle to transfer a wide range of text attributes appearing in real-world textual data from social networks. Additionally, we introduce a novel "soft noising" technique that further improves the performance of our system. We also show that teaching a student model to generate the output of SimpleStyle can result in a system that performs style transfer of equivalent quality with only a single greedy-decoded sample. Finally, we suggest our method as a remedy for the fundamental incompatible baseline issue that holds progress in the field. We offer our protocol as a simple yet strong baseline for works that wish to make incremental advancements in the field of attribute controlled text rewriting.


title: On the boundary behavior of weak $(p,q)$-quasiconformal mappings

abstract: Let $\Omega$ and $\widetilde{\Omega}$ be domains in the Euclidean space $\mathbb R^n$. We study the boundary behavior of weak $(p,q)$-quasiconformal mappings, $\varphi:\Omega\to \widetilde{\Omega}$, $n-1<q\leq p<n$. The suggested method is based on the capacitary distortion properties of the weak $(p,q)$-quasiconformal mappings.


title: Continuous Trajectory Optimization via B-splines for Multi-jointed Robotic Systems

abstract: Continuous formulations of trajectory planning problems have two main benefits. First, constraints are guaranteed to be satisfied at all times. Secondly, dynamic obstacles can be naturally considered with time. This paper introduces a novel B-spline based trajectory optimization method for multi-jointed robots that provides a continuous trajectory with guaranteed continuous constraints satisfaction. At the core of this method, B-spline basic operations, like addition, multiplication, and derivative, are rigorously defined and applied for problem formulation. B-spline unique characteristics, such as the convex hull and smooth curves properties, are utilized to reformulate the original continuous optimization problem into a finite-dimensional problem. Collision avoidance with static obstacles is achieved using the signed distance field, while that with dynamic obstacles is accomplished via constructing time-varying separating hyperplanes. Simulation results on various robots validate the effectiveness of the algorithm. In addition, this paper provides experimental validations with a 6-link FANUC robot avoiding static and moving obstacles.


title: Evolution of the long-period pulsar PSR J0901-4046

abstract: The fallback disc model predicted that anomalous X-ray pulsars (AXPs) and soft-gamma repeaters (SGRs) will evolve to isolated long period pulsars before the discovery of the first two long-period pulsars (LPPs) this year. Unlike normal radio pulsars, LPPs show transient pulsed-radio epochs with unusual and variable pulse shapes, similar to the radio behaviour of the few radio emitting AXP/SGRs. We show that the present properties of the recently discovered second LPP, PSR J0901-4046 ($P \simeq 76$ s), are obtained as a result of evolution in interaction with a fallback disc, as we had already shown for the first discovered LPP, GLEAM-X J162759.5-523504.3 ($P \simeq 1091$ s). While there is only an upper limit to the period derivative, $\dot{P}$, of GLEAM-X J162759.5-523504.3, the $\dot{P}$ of the PSR J0901-4046 has already been measured, providing better constraints for the evolutionary models. The model can produce the source properties with a dipole moment $\mu \simeq 10^{30}$ G cm$^3$. The results are not sensitive to the initial pulsar period. Our results indicate that PSR J0901-4046 went through an AXP/SGR epoch at an age of a few $10^4$ yr, and is $\sim (6 - 8) \times 10^5$ yr old at present.


title: A Measure-Theoretic Characterization of Tight Language Models

abstract: Language modeling, a central task in natural language processing, involves estimating a probability distribution over strings. In most cases, the estimated distribution sums to 1 over all finite strings. However, in some pathological cases, probability mass can ``leak'' onto the set of infinite sequences. In order to characterize the notion of leakage more precisely, this paper offers a measure-theoretic treatment of language modeling. We prove that many popular language model families are in fact tight, meaning that they will not leak in this sense. We also generalize characterizations of tightness proposed in previous works.


title: Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training

abstract: Prior work shows that it is possible to expand pretrained Masked Language Models (MLMs) to new languages by learning a new set of embeddings, while keeping the transformer body frozen. Despite learning a small subset of parameters, this approach is not compute-efficient, as training the new embeddings requires a full forward and backward pass over the entire model. We propose mini-model adaptation, a compute-efficient alternative that builds a shallow mini-model from a fraction of a large model's parameters. New language-specific embeddings can then be efficiently trained over the mini-model and plugged into the aligned large model for rapid cross-lingual transfer. We explore two approaches to learn mini-models: MiniJoint, which jointly pretrains the primary model and the mini-model using a single transformer with a secondary MLM head at a middle layer; and MiniPost, where we start from a regular pretrained model, build a mini-model by extracting and freezing a few layers, and learn a small number of parameters on top. Experiments on XNLI, MLQA and PAWS-X show that mini-model adaptation matches the performance of the standard approach using 2.3x less compute on average.


title: Can Current Task-oriented Dialogue Models Automate Real-world Scenarios in the Wild?

abstract: Task-oriented dialogue (TOD) systems are mainly based on the slot-filling-based TOD (SF-TOD) framework, in which dialogues are broken down into smaller, controllable units (i.e., slots) to fulfill a specific task. A series of approaches based on this framework achieved remarkable success on various TOD benchmarks. However, we argue that the current TOD benchmarks are limited to surrogate real-world scenarios and that the current TOD models are still a long way to cover the scenarios. In this position paper, we first identify current status and limitations of SF-TOD systems. After that, we explore the WebTOD framework, the alternative direction for building a scalable TOD system when a web/mobile interface is available. In WebTOD, the dialogue system learns how to understand the web/mobile interface that the human agent interacts with, powered by a large-scale language model.


title: DePlot: One-shot visual language reasoning by plot-to-table translation

abstract: Visual language such as charts and plots is ubiquitous in the human world. Comprehending plots and charts requires strong reasoning skills. Prior state-of-the-art (SOTA) models require at least tens of thousands of training examples and their reasoning capabilities are still much limited, especially on complex human-written queries. This paper presents the first one-shot solution to visual language reasoning. We decompose the challenge of visual language reasoning into two steps: (1) plot-to-text translation, and (2) reasoning over the translated text. The key in this method is a modality conversion module, named as DePlot, which translates the image of a plot or chart to a linearized table. The output of DePlot can then be directly used to prompt a pretrained large language model (LLM), exploiting the few-shot reasoning capabilities of LLMs. To obtain DePlot, we standardize the plot-to-table task by establishing unified task formats and metrics, and train DePlot end-to-end on this task. DePlot can then be used off-the-shelf together with LLMs in a plug-and-play fashion. Compared with a SOTA model finetuned on more than >28k data points, DePlot+LLM with just one-shot prompting achieves a 24.0% improvement over finetuned SOTA on human-written queries from the task of chart QA.


title: Distant 2-Colored Components on Embeddings Part II: The Short-Inseparable Case

abstract: This is the second in a sequence of three papers in which we prove the following generalization of Thomassen's 5-choosability theorem: Let $G$ be a graph embedded on a surface of genus $g$. Then $G$ can be $L$-colored, where $L$ is a list-assignment for $G$ in which every vertex has a 5-list except for a collection of pairwise far-apart components, each precolored with an ordinary 2-coloring, as long as the face-width of $G$ is at least $2^{\Omega(g)}$ and the precolored components are of distance at least $2^{\Omega(g)}$ apart. This provides an affirmative answer to a generalized version of a conjecture of Thomassen and also generalizes a result from 2017 of Dvo\v{r}\'ak, Lidick\'y, Mohar, and Postle about distant precolored vertices. In this paper we prove that the above result holds for a restricted class of embeddings, i.e. those embeddings which satisfy certain triangulation conditions and do not have separating cycles of length at most four.