title: The Universe from a Single Particle III

abstract: In parts I [6] and II [7] we studied how metrics $g_{ij}$ on $\mathfrak{su}(n)$ may spontaneously break symmetry and $\textit{crystallize}$ into a form which is kaq, $\textit{knows about qubits}$. We did this for $n=2^N$ and then away from powers of 2. Here we address the Fermionic version and find kam metrics, these $\textit{know about Majoranas}$. That is, there is a basis of principal axes $\{H_k\}$ of which is of homogeneous Majorana degree. In part I, we searched unsuccessfully for functional minima representing crystallized metrics exhibiting the Brown-Susskind penalty schedule, motivated by their study of black hole scrambling time. Here, by segueing to the Fermionic setting we find, to good approximation, kam metrics adhering to this schedule on both $\mathfrak{su}(4)$ and $\mathfrak{su}(8)$. Thus, with this preliminary finding, our toy model exhibits two of the three features required for the spontaneous emergence of spatial structure: 1. localized degrees of freedom and, 2. a preference for low body-number (or low Majorana number) interactions. The final feature, 3. constraints on who may interact with whom, i.e. a neighborhood structure, must await an effective analytic technique, being entirely beyond what we can approach with classical numerics.


title: KAT: A Knowledge Augmented Transformer for Vision-and-Language

abstract: The primary focus of recent work with largescale transformers has been on optimizing the amount of information packed into the model's parameters. In this work, we ask a different question: Can multimodal transformers leverage explicit knowledge in their reasoning? Existing, primarily unimodal, methods have explored approaches under the paradigm of knowledge retrieval followed by answer prediction, but leave open questions about the quality and relevance of the retrieved knowledge used, and how the reasoning processes over implicit and explicit knowledge should be integrated. To address these challenges, we propose a novel model - Knowledge Augmented Transformer (KAT) - which achieves a strong state-of-the-art result (+6 points absolute) on the open-domain multimodal task of OK-VQA. Our approach integrates implicit and explicit knowledge in an end to end encoder-decoder architecture, while still jointly reasoning over both knowledge sources during answer generation. An additional benefit of explicit knowledge integration is seen in improved interpretability of model predictions in our analysis.


title: Knowledge-Augmented Language Models for Cause-Effect Relation Classification

abstract: Previous studies have shown the efficacy of knowledge augmentation methods in pretrained language models. However, these methods behave differently across domains and downstream tasks. In this work, we investigate the augmentation of pretrained language models with commonsense knowledge in the cause-effect relation classification and commonsense causal reasoning tasks. After automatically verbalizing ATOMIC2020, a wide coverage commonsense reasoning knowledge graph, and GLUCOSE, a dataset of implicit commonsense causal knowledge, we continually pretrain BERT and RoBERTa with the verbalized data. Then we evaluate the resulting models on cause-effect pair classification and answering commonsense causal reasoning questions. Our results show that continually pretrained language models augmented with commonsense knowledge outperform our baselines on two commonsense causal reasoning benchmarks, COPA and BCOPA-CE, and the Temporal and Causal Reasoning (TCR) dataset, without additional improvement in model architecture or using quality-enhanced data for fine-tuning.


title: Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Context

abstract: Physical measurements constitute a large portion of numbers in academic papers, engineering reports, and web tables. Current benchmarks fall short of properly evaluating numeracy of pretrained language models on measurements, hindering research on developing new methods and applying them to numerical tasks. To that end, we introduce a novel task, Masked Measurement Prediction (MMP), where a model learns to reconstruct a number together with its associated unit given masked text. MMP is useful for both training new numerically informed models as well as evaluating numeracy of existing systems. In order to address this task, we introduce a new Generative Masked Measurement (GeMM) model that jointly learns to predict numbers along with their units. We perform fine-grained analyses comparing our model with various ablations and baselines. We use linear probing of traditional pretrained transformer models (RoBERTa) to show that they significantly underperform jointly trained number-unit models, highlighting the difficulty of this new task and the benefits of our proposed pretraining approach. We hope this framework accelerates the progress towards building more robust numerical reasoning systems in the future.


title: System dependence of away-side broadening and $\alpha$-clustering light nuclei structure effect in dihadron azimuthal correlations

abstract: A collision system scan involving $\alpha$-clustered $^{12}$C and $^{16}$O is studied by using a multiphase transport model for central collisions at $\sqrt{s_{NN}} = 6.37$ TeV. Background subtracted away-side dihadron azimuthal correlation is performed via the zero yield at minimum (ZYAM) method from raw signals, and the quantitative parameters, such as RMS width and Kurtosis, seem nicely follow the $A^{-1/3}$ law of the system size if the nucleus has the normal Woods-Saxon nucleon distribution. However, for $\alpha$-clustering light nuclei, specifically for $^{12}$C and $^{16}$O, the RMS width and Kurtosis of away-side azimuthal correlation are deviated from the baseline of $A^{-1/3}$ law. In addition, the momentum dependence of away-side broadening parameters is also presented. The results show that there is a distinction in away-side broadening parameters of dihadron correlation function between the Woods-Saxon distribution and the $\alpha$-clustered structures, which sheds light on that the collision system scan for dihadron azimuthal correlation as a potential probe to distinguish $\alpha$-clustered nuclei.


title: A Statistics and Deep Learning Hybrid Method for Multivariate Time Series Forecasting and Mortality Modeling

abstract: Hybrid methods have been shown to outperform pure statistical and pure deep learning methods at forecasting tasks and quantifying the associated uncertainty with those forecasts (prediction intervals). One example is Exponential Smoothing Recurrent Neural Network (ES-RNN), a hybrid between a statistical forecasting model and a recurrent neural network variant. ES-RNN achieves a 9.4\% improvement in absolute error in the Makridakis-4 Forecasting Competition. This improvement and similar outperformance from other hybrid models have primarily been demonstrated only on univariate datasets. Difficulties with applying hybrid forecast methods to multivariate data include ($i$) the high computational cost involved in hyperparameter tuning for models that are not parsimonious, ($ii$) challenges associated with auto-correlation inherent in the data, as well as ($iii$) complex dependency (cross-correlation) between the covariates that may be hard to capture. This paper presents Multivariate Exponential Smoothing Long Short Term Memory (MES-LSTM), a generalized multivariate extension to ES-RNN, that overcomes these challenges. MES-LSTM utilizes a vectorized implementation. We test MES-LSTM on several aggregated coronavirus disease of 2019 (COVID-19) morbidity datasets and find our hybrid approach shows consistent, significant improvement over pure statistical and deep learning methods at forecast accuracy and prediction interval construction.


title: Call for Customized Conversation: Customized Conversation Grounding Persona and Knowledge

abstract: Humans usually have conversations by making use of prior knowledge about a topic and background information of the people whom they are talking to. However, existing conversational agents and datasets do not consider such comprehensive information, and thus they have a limitation in generating the utterances where the knowledge and persona are fused properly. To address this issue, we introduce a call For Customized conversation (FoCus) dataset where the customized answers are built with the user's persona and Wikipedia knowledge. To evaluate the abilities to make informative and customized utterances of pre-trained language models, we utilize BART and GPT-2 as well as transformer-based models. We assess their generation abilities with automatic scores and conduct human evaluations for qualitative results. We examine whether the model reflects adequate persona and knowledge with our proposed two sub-tasks, persona grounding (PG) and knowledge grounding (KG). Moreover, we show that the utterances of our data are constructed with the proper knowledge and persona through grounding quality assessment.


title: Narrow bands in magnetic field and strong-coupling Hofstadter spectra

abstract: We develop a new, efficient, and general method to determine the Hofstadter spectrum of isolated narrow bands. The method works for topological as well as for trivial narrow bands by projecting the zero $\mathbf{B}$-field hybrid Wannier states -- which are localized in one direction and Bloch extended in another direction -- onto a representation of the magnetic translation group in the Landau gauge. We then apply this method to find, for the first time, the Hofstadter spectrum for the exact single particle charged excitations in the strong coupling limit of the magic angle twisted bilayer graphene at the charge neutrality point and at $|\nu|=2$ down to low magnetic fields when the flux through the moir\'e unit cell is only $\sim 1/25$ of the electronic flux quantum i.e. $\sim 1$T at the first magic angle. The resulting spectra provide a means to investigate Landau quantization of the quasiparticles even if their dispersion is interaction induced.


title: Thermal Conductivity due to Spins in the Two-Dimensional Spin System Ba$_2$Cu$_3$O$_4$Cl$_2$

abstract: We have measured the temperature dependences of the thermal conductivity of Ba$_2$Cu$_{3-x}M_x$O$_4$Cl$_2$ ($M$ = Pd, Ni, Co; $x$ = 0, 0.03) single crystals including two-dimensional (2D) Cu$_3$O$_4$ planes consisting of a strong 2D spin network of CuA$^{2+}$ spins and a weak 2D spin network of CuB$^{2+}$ spins. It has been found that the thermal conductivity due to spins, $\kappa_{\mathrm{spin}}$, exists in the thermal conductivity parallel to the Cu$_3$O$_4$ plane owing to the strong 2D spin network of CuA$^{2+}$ spins and exhibits a broad peak around room temperature. The maximum value of $\kappa_{\mathrm{spin}}$ is ~7 W/Km and comparable with that in Nd$_2$CuO$_4$ with almost the same 2D spin network of Cu$^{2+}$ spins. The $\kappa_{\mathrm{spin}}$ has been found to be suppressed by 1% impurities on account the decrease in the mean free path of magnetic excitations, suggesting that $\kappa_{\mathrm{spin}}$ is expected to be enhanced in 2D quantum spin systems such as Ba$_2$Cu$_3$O$_4$Cl$_2$ by reducing the amount of impurities in a single crystal. Moreover, it has concluded that the frustration between CuA$^{2+}$ and CuB$^{2+}$ spins little affects the existence of $\kappa_{\mathrm{spin}}$.


title: Bayesianism, Conditional Probability and Laplace Law of Succession in Quantum Mechanics

abstract: We present a comparative study between classical probability and quantum probability from the Bayesian viewpoint, where probability is construed as our rational degree of belief on whether a given statement is true. From this viewpoint, including conditional probability, three issues are discussed: i) Given a measure of the rational degree of belief, does it satisfy the axioms of the probability? ii) Given the probability satisfying these axioms, is it seen as the measure of the rational degree of belief? iii) Can the measure of the rational degree of belief be evaluated in terms of the relative frequency of events occurring? Here we show that as with the classical probability, all these issues can be resolved affirmatively in the quantum probability, provided that the relation to the relative frequency is slightly modified in case of a small number of observations. This implies that the relation between the Bayesian probability and the relative frequency in quantum mechanics is the same as that in the classical probability theory, including conditional probability.


title: Local invariants identify topology in metals and gapless systems

abstract: Although topological band theory has been used to discover and classify a wide array of novel topological phases in insulating and semi-metal systems, it is not well-suited to identifying topological phenomena in metallic or gapless systems. Here, we develop a theory of topological metals based on the system's spectral localizer and associated Clifford pseudospectrum, which can both determine whether a system exhibits boundary-localized states despite the presence of degenerate bulk bands and provide a measure of these states' topological protection even in the absence of a bulk band gap. We demonstrate the generality of this method across symmetry classes in two lattice systems, a Chern metal and a higher-order topological metal, and prove the topology of these systems is robust to relatively strong perturbations. The ability to define invariants for metallic and gapless systems allows for the possibility of finding topological phenomena in a broad range of natural, photonic, and other artificial materials that could not be previously explored.

categories: cond-mat.mes-hall cond-mat.mtrl-sci math-ph math.MP physics.optics


title: Melting of generalized Wigner crystals in transition metal dichalcogenide heterobilayer Moir\'e systems

abstract: Moir\'e superlattice systems such as transition metal dichalcogenide heterobilayers have garnered significant recent interest due to their promising utility as tunable solid state simulators. Recent experiments on a WSe$_2$/WS$_2$ heterobilayer detected incompressible charge ordered states that one can view as generalized Wigner crystals. The tunability of the hetero-TMD Moir\'e system presents an opportunity to study the rich set of possible phases upon melting these charge-ordered states. Here we use Monte Carlo simulations to study these intermediate phases in between incompressible charge-ordered states in the strong coupling limit. We find two distinct stripe solid states to be each preceded by distinct types of nematic states. In particular, we discover microscopic mechanisms that stabilize each of the nematic states, whose order parameter transforms as the two-dimensional $E$ representation of the Moir\'e lattice point group. Our results provide a testable experimental prediction of where both types of nematic occur, and elucidate the microscopic mechanism driving their formation.


abstract: We introduce a new framework, Bayesian Distributionally Robust Optimization (Bayesian-DRO), for data-driven stochastic optimization where the underlying distribution is unknown. Bayesian-DRO contrasts with most of the existing DRO approaches in the use of Bayesian estimation of the unknown distribution. To make computation of Bayesian updating tractable, Bayesian-DRO first assumes the underlying distribution takes a parametric form with unknown parameter and then computes the posterior distribution of the parameter. To address the model uncertainty brought by the assumed parametric distribution, Bayesian-DRO constructs an ambiguity set of distributions with the assumed parametric distribution as the reference distribution and then optimizes with respect to the worst case in the ambiguity set. We show the consistency of the Bayesian posterior distribution and subsequently the convergence of objective functions and optimal solutions of Bayesian-DRO. Our consistency result of the Bayesian posterior requires simpler assumptions than the classical literature on Bayesian consistency. We also consider several approaches for selecting the ambiguity set size in Bayesian-DRO and compare them numerically. Our numerical experiments demonstrate the out-of-sample performance of Bayesian-DRO in comparison with Kullback-Leibler-based (KL-) and Wasserstein-based empirical DRO as well as risk-neutral Bayesian Risk Optimization. Our numerical results shed light on how to choose the modeling framework (Bayesian-DRO, KL-DRO, Wasserstein-DRO) for specific problems, but the choice for general problems still remains an important and open question.


title: Analysis and Evaluation of Kinect-based Action Recognition Algorithms

abstract: Human action recognition still exists many challenging problems such as different viewpoints, occlusion, lighting conditions, human body size and the speed of action execution, although it has been widely used in different areas. To tackle these challenges, the Kinect depth sensor has been developed to record real time depth sequences, which are insensitive to the color of human clothes and illumination conditions. Many methods on recognizing human action have been reported in the literature such as HON4D, HOPC, RBD and HDG, which use the 4D surface normals, pointclouds, skeleton-based model and depth gradients respectively to capture discriminative information from depth videos or skeleton data. In this research project, the performance of four aforementioned algorithms will be analyzed and evaluated using five benchmark datasets, which cover challenging issues such as noise, change of viewpoints, background clutters and occlusions. We also implemented and improved the HDG algorithm, and applied it in cross-view action recognition using the UWA3D Multiview Activity dataset. Moreover, we used different combinations of individual feature vectors in HDG for performance evaluation. The experimental results show that our improvement of HDG outperforms other three state-of-the-art algorithms for cross-view action recognition.


title: On the Use of Quality Diversity Algorithms for The Traveling Thief Problem

abstract: In real-world optimisation, it is common to face several sub-problems interacting and forming the main problem. There is an inter-dependency between the sub-problems, making it impossible to solve such a problem by focusing on only one component. The traveling thief problem~(TTP) belongs to this category and is formed by the integration of the traveling salesperson problem~(TSP) and the knapsack problem~(KP). In this paper, we investigate the inter-dependency of the TSP and the KP by means of quality diversity~(QD) approaches. QD algorithms provide a powerful tool not only to obtain high-quality solutions but also to illustrate the distribution of high-performing solutions in the behavioural space. We introduce a MAP-Elite based evolutionary algorithm using well-known TSP and KP search operators, taking the TSP and KP score as the behavioural descriptor. Afterwards, we conduct comprehensive experimental studies that show the usefulness of using the QD approach applied to the TTP. First, we provide insights regarding high-quality TTP solutions in the TSP/KP behavioural space. Afterwards, we show that better solutions for the TTP can be obtained by using our QD approach and it can improve the best-known solution for a number of TTP instances used for benchmarking in the literature.


title: Explainable Natural Language Processing with Matrix Product States

abstract: Despite empirical successes of recurrent neural networks (RNNs) in natural language processing (NLP), theoretical understanding of RNNs is still limited due to intrinsically complex non-linear computations. We systematically analyze RNNs' behaviors in a ubiquitous NLP task, the sentiment analysis of movie reviews, via the mapping between a class of RNNs called recurrent arithmetic circuits (RACs) and a matrix product state (MPS). Using the von-Neumann entanglement entropy (EE) as a proxy for information propagation, we show that single-layer RACs possess a maximum information propagation capacity, reflected by the saturation of the EE. Enlarging the bond dimension beyond the EE saturation threshold does not increase model prediction accuracies, so a minimal model that best estimates the data statistics can be inferred. Although the saturated EE is smaller than the maximum EE allowed by the area law, our minimal model still achieves ~99% training accuracies in realistic sentiment analysis data sets. Thus, low EE is not a warrant against the adoption of single-layer RACs for NLP. Contrary to a common belief that long-range information propagation is the main source of RNNs' successes, we show that single-layer RACs harness high expressiveness from the subtle interplay between the information propagation and the word vector embeddings. Our work sheds light on the phenomenology of learning in RACs, and more generally on the explainability of RNNs for NLP, using tools from many-body quantum physics.

categories: cond-mat.dis-nn cond-mat.stat-mech cs.CL cs.LG quant-ph


title: Seismic Inversion by Multi-dimensional Newtonian Machine

abstract: Newtonian machine learning (NML) is a wave-equation inversion method that inverts single-dimensional latent space (LS) features of the seismic data for retrieving the subsurface background velocity model. The single-dimensional LS features mainly contain the kinematic information of the seismic data, which are automatically extracted from the seismic signal by using an autoencoder network. Because its LS feature dimension is too small to preserve the dynamic information, such as the waveform variations, of the seismic data. Therefore the NML inversion is not able to recover the high-wavenumber velocity details. To mitigate this problem, we propose to invert multi-dimensional LS features, which can fully represent the entire characters of the seismic data. We denote this method as multi-dimensional Newtonian machine learning (MNML). In MNML, we define a new multi-variable connective function that works together with the multi-variable implicit function theorem to connect the velocity perturbations to the multi-dimensional LS feature perturbations. Numerical tests show that (1) the multi-dimensional LS features can preserve more data information than the single-dimensional LS features; (2) a higher resolution velocity model can be recovered by inverting the multi-dimensional LS features, and the inversion quality is comparable to that of FWI; (3) the MNML method requires a much smaller storage space than conventional FWI because only the low-dimensional representations of the high-dimensional seismic data are needed to be stored. The disadvantage of MNML is that it can more easily get stuck in local minima compared to the NML method. So we suggest a multiscale inversion approach that inverts for higher dimensional LS features as the iteration count increase.


title: Spin Hall Conductivity and Anomalous Hall Conductivity in Full Heusler compounds

abstract: The spin Hall conductivity (SHC) and anomalous Hall conductivity (AHC) in more than 120 full Heusler compounds are calculated using density functional theory in a high-throughtput way. The electronic structures are mapped to the Wannier basis and the linear response theory is used to get the conductivity. Our results show that the mechanism under the SHC or AHC cannot be simply related to the valence electron numbers or atomic weights, is related to the very details of the electronic structure, which can only be obtained by calculations. A high throughput calculation is efficient to screen out the desired materials. According to our present results, Cu2CoSn, as well as Co2MnAl and Co2MnGa are candidates in spintronic materials regarding to their high SHC and AHC values, which can benefit the spin-torque-driven nanodevices.


title: Superselection, Boundary Algebras and Duality in Gauge Theories

abstract: We consider the generators of gauge transformations with test functions which do not vanish on the boundary of a spacelike region of interest. These are known to generate the edge degrees of freedom in a gauge theory. In this paper, we augment these by introducing the dual or magnetic analogue of such operators. We then study the algebra of these operators, focusing on implications for the superselection sectors of the gauge theory. A manifestly duality-invariant action is also considered, from which alternate descriptions which are $SL(2, \mathbb{Z})$ transforms of each other can be obtained. We also comment on a number of issues related to local charges, definition of confinement and the appearance of interesting mathematical structures such as the Drinfel'd double and the Manin triple.


title: CDRec: Cayley-Dickson Recommender

abstract: In this paper, we propose a recommendation framework named Cayley-Dickson Recommender. We introduce Cayley-Dickson construction which uses a recursive process to define hypercomplex algebras and their mathematical operations. We also design a graph convolution operator to learn representations in the hypercomplex space. To the best of our knowledge, it is the first time that Cayley-Dickson construction and graph convolution techniques have been used in hypercomplex recommendation. Compared with the state-of-the-art recommendation methods, our method achieves superior performance on real-world datasets.


title: Learning To Retrieve Prompts for In-Context Learning

abstract: In-context learning is a recent paradigm in natural language understanding, where a large pre-trained language model (LM) observes a test instance and a few training examples as its input, and directly decodes the output without any update to its parameters. However, performance has been shown to strongly depend on the selected training examples (termed prompt). In this work, we propose an efficient method for retrieving prompts for in-context learning using annotated data and a LM. Given an input-output pair, we estimate the probability of the output given the input and a candidate training example as the prompt, and label training examples as positive or negative based on this probability. We then train an efficient dense retriever from this data, which is used to retrieve training examples as prompts at test time. We evaluate our approach on three sequence-to-sequence tasks where language utterances are mapped to meaning representations, and find that it substantially outperforms prior work and multiple baselines across the board.


title: FRUIT: Faithfully Reflecting Updated Information in Text

abstract: Textual knowledge bases such as Wikipedia require considerable effort to keep up to date and consistent. While automated writing assistants could potentially ease this burden, the problem of suggesting edits grounded in external knowledge has been under-explored. In this paper, we introduce the novel generation task of *faithfully reflecting updated information in text* (FRUIT) where the goal is to update an existing article given new evidence. We release the FRUIT-WIKI dataset, a collection of over 170K distantly supervised data produced from pairs of Wikipedia snapshots, along with our data generation pipeline and a gold evaluation set of 914 instances whose edits are guaranteed to be supported by the evidence. We provide benchmark results for popular generation systems as well as EDIT5 -- a T5-based approach tailored to editing we introduce that establishes the state of the art. Our analysis shows that developing models that can update articles faithfully requires new capabilities for neural generation models, and opens doors to many new applications.


title: Road-aware Monocular Structure from Motion and Homography Estimation

abstract: Structure from motion (SFM) and ground plane homography estimation are critical to autonomous driving and other robotics applications. Recently, much progress has been made in using deep neural networks for SFM and homography estimation respectively. However, directly applying existing methods for ground plane homography estimation may fail because the road is often a small part of the scene. Besides, the performances of deep SFM approaches are still inferior to traditional methods. In this paper, we propose a method that learns to solve both problems in an end-to-end manner, improving performance on both. The proposed networks consist of a Depth-CNN, a Pose-CNN and a Ground-CNN. The Depth-CNN and Pose-CNN estimate dense depth map and ego-motion respectively, solving SFM, while the Pose-CNN and Ground-CNN followed by a homography layer solve the ground plane estimation problem. By enforcing coherency between SFM and homography estimation results, the whole network can be trained end to end using photometric loss and homography loss without any groundtruth except the road segmentation provided by an off-the-shelf segmenter. Comprehensive experiments are conducted on KITTI benchmark to demonstrate promising results compared with various state-of-the-art approaches.


title: A model sufficiency test using permutation entropy

abstract: Using the ordinal pattern concept in permutation entropy, we propose a model sufficiency test to study a given model's point prediction accuracy. Compared to some classical model sufficiency tests, such as the Broock et al.'s (1996) test, our proposal does not require a sufficient model to eliminate all structures exhibited in the estimated residuals. When the innovations in the investigated data's underlying dynamics show a certain structure, such as higher-moment serial dependence, the Broock et al.'s (1996) test can lead to erroneous conclusions about the sufficiency of point predictors. Due to the structured innovations, inconsistency between the model sufficiency tests and prediction accuracy criteria can occur. Our proposal fills in this incoherence between model and prediction evaluation approaches and remains valid when the underlying process has non-white additive innovation.


title: Analyzing the Limits of Self-Supervision in Handling Bias in Language

abstract: Prompting inputs with natural language task descriptions has emerged as a popular mechanism to elicit reasonably accurate outputs from large-scale generative language models with little to no in-context supervision. This also helps gain insight into how well language models capture the semantics of a wide range of downstream tasks purely from self-supervised pre-training on massive corpora of unlabeled text. Such models have naturally also been exposed to a lot of undesirable content like racist and sexist language and there is limited work on awareness of models along these dimensions. In this paper, we define and comprehensively evaluate how well such language models capture the semantics of four tasks for bias: diagnosis, identification, extraction and rephrasing. We define three broad classes of task descriptions for these tasks: statement, question, and completion, with numerous lexical variants within each class. We study the efficacy of prompting for each task using these classes and the null task description across several decoding methods and few-shot examples. Our analyses indicate that language models are capable of performing these tasks to widely varying degrees across different bias dimensions, such as gender and political affiliation. We believe our work is an important step towards unbiased language models by quantifying the limits of current self-supervision objectives at accomplishing such sociologically challenging tasks.


title: Evaluating Hybrid Graph Pattern Queries Using Runtime Index Graphs

abstract: Graph pattern matching is a fundamental operation for the analysis and exploration ofdata graphs. In thispaper, we presenta novel approachfor efficiently finding homomorphic matches for hybrid graph patterns, where each pattern edge may be mapped either to an edge or to a path in the input data, thus allowing for higher expressiveness and flexibility in query formulation. A key component of our approach is a lightweight index structure that leverages graph simulation to compactly encode the query answer search space. The index can be built on-the-fly during query execution and does not have to be persisted to disk. Using the index, we design a multi-way join algorithm to enumerate query solutions without generating any potentially exploding intermediate results. We demonstrate through extensive experiments that our approach can efficiently evaluate a wide range / broad spectrum of graph pattern queries and greatly outperforms existing approaches and recent graph query engines/systems.


title: Real-space density kernel method for Kohn-Sham density functional theory calculations at high temperature

abstract: Kohn-Sham density functional theory calculations using conventional diagonalization based methods become increasingly expensive as temperature increases due to the need to compute increasing numbers of partially occupied states. We present a density matrix based method for Kohn-Sham calculations at high temperature that eliminates the need for diagonalization entirely, thus reducing the cost of such calculations significantly. Specifically, we develop real-space expressions for the electron density, electronic free energy, Hellmann-Feynman forces, and Hellmann-Feynman stress tensor in terms of an orthonormal auxiliary orbital basis and its density kernel transform, the density kernel being the matrix representation of the density operator in the auxiliary basis. Using Chebyshev filtering to generate the auxiliary basis, we next develop an approach akin to Clenshaw-Curtis spectral quadrature to calculate the individual columns of the density kernel based on the Fermi operator expansion in Chebyshev polynomials; and employ a similar approach to evaluate band structure and entropic energy components. We implement the proposed formulation in the SPARC electronic structure code, using which we show systematic convergence of the aforementioned quantities to exact diagonalization results, and obtain significant speedups relative to conventional diagonalization based methods. Finally, we employ the new method to compute the self-diffusion coefficient and viscosity of aluminum at 116,045 K from Kohn-Sham quantum molecular dynamics, where we find agreement with previous more approximate orbital-free density functional methods.


title: Discrete velocity Boltzmann equation in the plane: stationary solutions

abstract: The paper proves existence of mild solutions to normal discrete velocity Boltzmann equations sin the plane with no pair of colinear interacting velocities, and given ingoing boundary values. A key property is L1 compactness of the integrated collision frequency for a sequence of approximations. This is proved using the Kolmogorov-Riesz theorem, which here replaces the L1 compactness of velocity averages, not available when the velocities are discrete.


title: On Gibbs Sampling for Structured Bayesian Models Discussion of paper by Zanella and Roberts

abstract: This article is a discussion of Zanella and Roberts' paper: Multilevel linear models, gibbs samplers and multigrid decompositions. We consider several extensions in which the multigrid decomposition would bring us interesting insights, including vector hierarchical models, linear mixed effects models and partial centering parametrizations.


title: Allpix$^2$ -- Silicon Detector Monte Carlo Simulations for Particle Physics and Beyond

abstract: Allpix$^2$ is a versatile, open-source simulation framework for silicon pixel detectors. Its goal is to ease the implementation of detailed simulations for both single sensors and more complex setups with multiple detectors. While originally created for silicon detectors in high-energy physics, it is capable of simulating a wide range of detector types for various application scenarios, through its interface to Geant4 to describe the interaction of particles with matter, and the different algorithms for charge transport and digitization. The simulation chain is arranged with the help of intuitive configuration files and an extensible system of modules, which implement the individual simulation steps. Detailed electric field maps imported from TCAD simulations can be used to precisely model the drift behavior of the charge carriers, bringing a new level of realism to the Monte Carlo simulation of particle detectors. Recently, Allpix$^2$ has seen major improvements to its core framework to take full advantage of multi- and many-core processor architectures for simulating events fully parallel. Furthermore, new physics models such as charge carrier recombination in silicon have been introduced, further extending the application range. This contribution provides an overview of the framework and its components, highlighting the versatility and recent developments.


title: TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning

abstract: Zero-shot learning (ZSL) tackles the novel class recognition problem by transferring semantic knowledge from seen classes to unseen ones. Existing attention-based models have struggled to learn inferior region features in a single image by solely using unidirectional attention, which ignore the transferability and discriminative attribute localization of visual features. In this paper, we propose a cross attribute-guided Transformer network, termed TransZero++, to refine visual features and learn accurate attribute localization for semantic-augmented visual embedding representations in ZSL. TransZero++ consists of an attribute$\rightarrow$visual Transformer sub-net (AVT) and a visual$\rightarrow$attribute Transformer sub-net (VAT). Specifically, AVT first takes a feature augmentation encoder to alleviate the cross-dataset problem, and improves the transferability of visual features by reducing the entangled relative geometry relationships among region features. Then, an attribute$\rightarrow$visual decoder is employed to localize the image regions most relevant to each attribute in a given image for attribute-based visual feature representations. Analogously, VAT uses the similar feature augmentation encoder to refine the visual features, which are further applied in visual$\rightarrow$attribute decoder to learn visual-based attribute features. By further introducing semantical collaborative losses, the two attribute-guided transformers teach each other to learn semantic-augmented visual embeddings via semantical collaborative learning. Extensive experiments show that TransZero++ achieves the new state-of-the-art results on three challenging ZSL benchmarks. The codes are available at: \url{https://github.com/shiming-chen/TransZero_pp}.


title: A comparative study of paired versus unpaired deep learning methods for physically enhancing digital rock image resolution

abstract: X-ray micro-computed tomography (micro-CT) has been widely leveraged to characterise pore-scale geometry in subsurface porous rock. Recent developments in super resolution (SR) methods using deep learning allow the digital enhancement of low resolution (LR) images over large spatial scales, creating SR images comparable to the high resolution (HR) ground truth. This circumvents traditional resolution and field-of-view trade-offs. An outstanding issue is the use of paired (registered) LR and HR data, which is often required in the training step of such methods but is difficult to obtain. In this work, we rigorously compare two different state-of-the-art SR deep learning techniques, using both paired and unpaired data, with like-for-like ground truth data. The first approach requires paired images to train a convolutional neural network (CNN) while the second approach uses unpaired images to train a generative adversarial network (GAN). The two approaches are compared using a micro-CT carbonate rock sample with complicated micro-porous textures. We implemented various image based and numerical verifications and experimental validation to quantitatively evaluate the physical accuracy and sensitivities of the two methods. Our quantitative results show that unpaired GAN approach can reconstruct super-resolution images as precise as paired CNN method, with comparable training times and dataset requirement. This unlocks new applications for micro-CT image enhancement using unpaired deep learning methods; image registration is no longer needed during the data processing stage. Decoupled images from data storage platforms can be exploited more efficiently to train networks for SR digital rock applications. This opens up a new pathway for various applications of multi-scale flow simulation in heterogeneous porous media.


title: Learning Interpretable Models Through Multi-Objective Neural Architecture Search

abstract: Monumental advances in deep learning have led to unprecedented achievements across various domains. While the performance of deep neural networks is indubitable, the architectural design and interpretability of such models are nontrivial. Research has been introduced to automate the design of neural network architectures through neural architecture search (NAS). Recent progress has made these methods more pragmatic by exploiting distributed computation and novel optimization algorithms. However, there is little work in optimizing architectures for interpretability. To this end, we propose a multi-objective distributed NAS framework that optimizes for both task performance and "introspectability," a surrogate metric for aspects of interpretability. We leverage the non-dominated sorting genetic algorithm (NSGA-II) and explainable AI (XAI) techniques to reward architectures that can be better comprehended by domain experts. The framework is evaluated on several image classification datasets. We demonstrate that jointly optimizing for task error and introspectability leads to more disentangled and debuggable architectures that perform within tolerable error.


title: Anomalous Josephson Effect in Topological Insulator-Based Josephson Trijunction

abstract: We studied anomalous Josephson effect (AJE) in Josephson trijunctions fabricated on Bi$_2$Se$_3$, and found that the AJE in T-shaped trijunctions significantly alters the Majorana phase diagram of the trijunctions, when an in-plane magnetic field is applied parallel to two of the three single junctions. Such a phenomenon in topological insulator-based Josephson trijunction provides unambiguous evidence for the existence of AJE in the system, and might provide an additional knob for controlling the Majorana bound states in the Fu-Kane scheme of topological quantum computation.


title: QAHOI: Query-Based Anchors for Human-Object Interaction Detection

abstract: Human-object interaction (HOI) detection as a downstream of object detection tasks requires localizing pairs of humans and objects and extracting the semantic relationships between humans and objects from an image. Recently, one-stage approaches have become a new trend for this task due to their high efficiency. However, these approaches focus on detecting possible interaction points or filtering human-object pairs, ignoring the variability in the location and size of different objects at spatial scales. To address this problem, we propose a transformer-based method, QAHOI (Query-Based Anchors for Human-Object Interaction detection), which leverages a multi-scale architecture to extract features from different spatial scales and uses query-based anchors to predict all the elements of an HOI instance. We further investigate that a powerful backbone significantly increases accuracy for QAHOI, and QAHOI with a transformer-based backbone outperforms recent state-of-the-art methods by large margins on the HICO-DET benchmark. The source code is available at $\href{https://github.com/cjw2021/QAHOI}{\text{this https URL}}$.


title: Multiple-Periods Locally-Facet-Based MIP Formulations for the Unit Commitment Problem

abstract: The thermal unit commitment (UC) problem has historically been formulated as a mixed integer quadratic programming (MIQP), which is difficult to solve efficiently, especially for large-scale systems. The tighter characteristic reduces the search space, therefore, as a natural consequence, significantly reduces the computational burden. In literatures, many tightened formulations for a single unit with parts of constraints were reported without presenting explicitly how they were derived. In this paper, a systematic approach is developed to formulate tight formulations. The idea is to use more binary variables to represent the state of the unit so as to obtain the tightest upper bound of power generation limits and ramping constraints for a single unit. In this way, we propose a multi-period formulation based on sliding windows which may have different sizes for each unit in the system. Furthermore, a multi-period model taking historical status into consideration is obtained. Besides, sufficient and necessary conditions for the facets of single-unit constraints polytope are provided and redundant inequalities are eliminated. The proposed models and three other state-of-the-art models are tested on 73 instances with a scheduling time of 24 hours. The number of generators in the test systems ranges from 10 to 1080. The simulation results show that our proposed multi-period formulations are tighter than the other three state-of-the-art models when the window size of the multi-period formulation is greater than 2.


title: The Geometry of the affine closure of $T^*(\mathrm{SL}_n/U)$

abstract: We show that the affine closure $\overline{T^*(\mathrm{SL}_n/U)}$ has symplectic singularities, in the sense of Beauville. In the special case $n=3$, we show that the affine closure $\overline{T^*(\mathrm{SL}_3/U)}$ is isomorphic to the closure $\overline{\mathcal{O}}_\textrm{min}$ of the minimal nilpotent orbit $\mathcal{O}_{\textrm{min}}$ in $\mathfrak{so}_8$. Moreover, the quasi-classical Gelfand-Graev action of the Weyl group $W$ on $\overline{T^*(\mathrm{SL}_3/U)}$ can be identified with the restriction to $\overline{\mathcal{O}}_\textrm{min}$ of the triality $S_3$-action on $\mathfrak{so}_8$.


title: A Numerical Method for Sharp-Interface Simulations of Multicomponent Alloy Solidification

abstract: We present a computational method for the simulation of the solidification of multicomponent alloys in the sharp-interface limit. Contrary to the case of binary alloys where a fixed point iteration is adequate, we hereby propose a Newton-type approach to solve the non-linear system of coupled PDEs arising from the time discretization of the governing equations, allowing for the first time sharp-interface simulations of the multialloy solidification. A combination of spatially adaptive quadtree grids, Level-Set Method, and sharp-interface numerical methods for imposing boundary conditions is used to accurately and efficiently resolve the complex behavior of the solidification front. The convergence behavior of the Newton-type iteration is theoretically analyzed in a one-dimensional setting and further investigated numerically in multiple spatial dimensions. We validate the overall computational method on the case of axisymmetric radial solidification admitting an analytical solution and show that the overall method's accuracy is close to second order. Finally, we perform numerical experiments for the directional solidification of a Co-Al-W ternary alloy with a phase diagram obtained from the PANDAT database and analyze the solutal segregation dependence on the processing conditions and alloy properties.


title: Dynamic Density Functional Theory of Multicomponent Cellular Membranes

abstract: We present a continuum model trained on molecular dynamics (MD) simulations for cellular membranes composed of an arbitrary number of lipid types. The model is constructed within the formalism of dynamic density functional theory and can be extended to include features such as the presence of proteins and membrane deformations. This framework represents a paradigm shift by enabling simulations that can access length scales on the order of microns and time scales on the order of seconds, all while maintaining near fidelity to the underlying MD models. These length and time scales are significant for accessing biological processes associated with signaling pathways within cells. Membrane interactions with RAS, a protein implicated in roughly 30% of human cancers, are considered as an application. Simulation results are presented and verified with MD simulations, and implications of this new capability are discussed.


title: Extreme Zero-Shot Learning for Extreme Text Classification

abstract: The eXtreme Multi-label text Classification (XMC) problem concerns finding most relevant labels for an input text instance from a large label set. However, the XMC setup faces two challenges: (1) it is not generalizable to predict unseen labels in dynamic environments, and (2) it requires a large amount of supervised (instance, label) pairs, which can be difficult to obtain for emerging domains. Recently, the generalized zero-shot XMC (GZ-XMC) setup has been studied and ZestXML is proposed accordingly to handle the unseen labels, which still requires a large number of annotated (instance, label) pairs. In this paper, we consider a more practical scenario called Extreme Zero-Shot XMC (EZ-XMC), in which no supervision is needed and merely raw text of instances and labels are accessible. Few-Shot XMC (FS-XMC), an extension to EZ-XMC with limited supervision is also investigated. To learn the semantic embeddings of instances and labels with raw text, we propose to pre-train Transformer-based encoders with self-supervised contrastive losses. Specifically, we develop a pre-training method MACLR, which thoroughly leverages the raw text with techniques including Multi-scale Adaptive Clustering, Label Regularization, and self-training with pseudo positive pairs. Experimental results on four public EZ-XMC datasets demonstrate that MACLR achieves superior performance compared to all other leading baseline methods, in particular with approximately 5-10% improvement in precision and recall on average. Moreover, we also show that our pre-trained encoder can be further improved on FS-XMC when there are a limited number of ground-truth positive pairs in training. By fine-tuning the encoder on such a few-shot subset, MACLR still outperforms other extreme classifiers significantly.


title: Reconsidering the Past: Optimizing Hidden States in Language Models

abstract: We present Hidden-State Optimization (HSO), a gradient-based method for improving the performance of transformer language models at inference time. Similar to dynamic evaluation (Krause et al., 2018), HSO computes the gradient of the log-probability the language model assigns to an evaluation text, but uses it to update the cached hidden states rather than the model parameters. We test HSO with pretrained Transformer-XL and GPT-2 language models, finding improvement on the WikiText103 and PG-19 datasets in terms of perplexity, especially when evaluating a model outside of its training distribution. We also demonstrate downstream applicability by showing gains in the recently developed prompt-based few-shot evaluation setting, again with no extra parameters or training data.


title: Learning to Prompt for Continual Learning

abstract: The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. Typical methods rely on a rehearsal buffer or known task identity at test time to retrieve learned knowledge and address forgetting, while this work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. In our proposed framework, prompts are small learnable parameters, which are maintained in a memory space. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity. We conduct comprehensive experiments under popular image classification benchmarks with different challenging continual learning settings, where L2P consistently outperforms prior state-of-the-art methods. Surprisingly, L2P achieves competitive results against rehearsal-based methods even without a rehearsal buffer and is directly applicable to challenging task-agnostic continual learning. Source code is available at https://github.com/google-research/l2p.


title: Feature Distillation Interaction Weighting Network for Lightweight Image Super-Resolution

abstract: Convolutional neural networks based single-image super-resolution (SISR) has made great progress in recent years. However, it is difficult to apply these methods to real-world scenarios due to the computational and memory cost. Meanwhile, how to take full advantage of the intermediate features under the constraints of limited parameters and calculations is also a huge challenge. To alleviate these issues, we propose a lightweight yet efficient Feature Distillation Interaction Weighted Network (FDIWN). Specifically, FDIWN utilizes a series of specially designed Feature Shuffle Weighted Groups (FSWG) as the backbone, and several novel mutual Wide-residual Distillation Interaction Blocks (WDIB) form an FSWG. In addition, Wide Identical Residual Weighting (WIRW) units and Wide Convolutional Residual Weighting (WCRW) units are introduced into WDIB for better feature distillation. Moreover, a Wide-Residual Distillation Connection (WRDC) framework and a Self-Calibration Fusion (SCF) unit are proposed to interact features with different scales more flexibly and efficiently.Extensive experiments show that our FDIWN is superior to other models to strike a good balance between model performance and efficiency. The code is available at https://github.com/IVIPLab/FDIWN.


title: DREAM: Improving Situational QA by First Elaborating the Situation

abstract: When people answer questions about a specific situation, e.g., "I cheated on my mid-term exam last week. Was that wrong?", cognitive science suggests that they form a mental picture of that situation before answering. While we do not know how language models (LMs) answer such questions, we conjecture that they may answer more accurately if they are also provided with additional details about the question situation, elaborating the "scene". To test this conjecture, we train a new model, DREAM, to answer questions that elaborate the scenes that situated questions are about, and then provide those elaborations as additional context to a question-answering (QA) model. We find that DREAM is able to create better scene elaborations (more accurate, useful, and consistent) than a representative state-of-the-art, zero-shot model (Macaw). We also find that using the scene elaborations as additional context improves the answer accuracy of a downstream QA system, including beyond that obtainable by simply further finetuning the QA system on DREAM's training data. These results suggest that adding focused elaborations about a situation can improve a system's reasoning about it, and may serve as an effective way of injecting new scenario based knowledge into QA models. Finally, our approach is dataset-neutral; we observe improved QA performance across different models, with even bigger gains on models with fewer parameters. We make our dataset and model publicly available at https://github.com/allenai/dream.


title: Taming Repetition in Dialogue Generation

abstract: The wave of pre-training language models has been continuously improving the quality of the machine-generated conversations, however, some of the generated responses still suffer from excessive repetition, sometimes repeating words from utterance, sometimes repeating words within self-generated responses, or both. Inappropriate repetition of words can significantly degrade the quality of the generated texts. Penalized sampling is one popular solution, reducing the sampling probability of existing words during inference, however, it is highly vulnerable to the inappropriate setting of the static weight. Setting it too high can yield strange and unrealistic sentences while setting it too low makes the task of suppressing repetition trivial. To remedy the shortcomings of the above methods, we design a context-aware classifier to explicitly decide when to allow repetition and when to employ penalized sampling. Such a classifier can be easily integrated with existing decoding methods, reducing repetitions where appropriate while preserving the diversity of the text. Experimental results demonstrate that our method can generate higher quality and more authentic dialogues.


title: Revisiting Fuzzy Signatures: Towards a More Risk-Free Cryptographic Authentication System based on Biometrics

abstract: Biometric authentication is one of the promising alternatives to standard password-based authentication offering better usability and security. In this work, we revisit the biometric authentication based on "fuzzy signatures" introduced by Takahashi et al. (ACNS'15, IJIS'19). These are special types of digital signatures where the secret signing key can be a "fuzzy" data such as user's biometrics. Compared to other cryptographically secure biometric authentications as those relying on fuzzy extractors, the fuzzy signature-based scheme provides a more attractive security guarantee. However, despite their potential values, fuzzy signatures have not attracted much attention owing to their theory-oriented presentations in all prior works. For instance, the discussion on the practical feasibility of the assumptions (such as the entropy of user biometrics), which the security of fuzzy signatures hinges on, is completely missing. In this work, we revisit fuzzy signatures and show that we can indeed efficiently and securely implement them in practice. At a high level, our contribution is threefold: (i) we provide a much simpler, more efficient, and direct construction of fuzzy signature compared to prior works; (ii) we establish novel statistical techniques to experimentally evaluate the conditions on biometrics that are required to securely instantiate fuzzy signatures; and (iii) we provide experimental results using a real-world finger-vein dataset to show that finger-veins from a single hand are sufficient to construct efficient and secure fuzzy signatures. Our performance analysis shows that in a practical scenario with 112-bits of security, the size of the signature is 1256 bytes, and the running time for signing/verification is only a few milliseconds.


title: q-ary Propelinear Perfect Codes from the Regular Subgroups of the GA(r,q) and Their Ranks

abstract: We propose a new method of constructing q-ary propelinear perfect codes. The approach utilizes permutations of the fixed length q-ary vectors that arise from the automorphisms of the regular subgroups of the affine group. For any prime q it is shown that the new class contains an infinite series of q-ary propelinear perfect codes of varying ranks of growing length.


title: Equilibrium of Free Surfaces and Nanoparticles in Self-Consistent Field Theory of Block Copolymers

abstract: This work presents a general and unified theory describing block copolymer self-assembly in the presence of free surfaces and nanoparticles in the context of Self-Consistent Filed Theory. Specifically, the derived theory applies to free and tethered polymer chains, nanoparticles of any shape, arbitrary non-uniform surface energies and grafting densities, and takes into account a possible formation of triple-junction points (e.g., polymer-air-substrate). One of the main ingredients of the proposed theory is a simple procedure for a consistent imposition of boundary conditions on surfaces with non-zero surface energies and/or non-zero grafting densities that results in singularity-free pressure-like fields, which is crucial for the calculation of forces. The generality of the theory is demonstrated using several representative examples such as the meniscus formation in the graphoepitaxy and the co-assembly of "polarized" nano-rods and diblock copolymer melts.


title: Double layered transformation cavities: Crossover between local and global transformations

abstract: We study the resonant modes in double-layered transformation cavities consisting of inner and outer layer boundaries where the mode intensity is mainly located inside the inner layer. This is an intermediate design between a transformation cavity and a cavity in a wholly transformed space of transformation optics. We demonstrate the crossover between these two extreme cases as the outer layer of the cavity varies and also explore the properties of the resonant modes in the cavity. While the near-field patterns of the resonant modes do not change as the outer layer becomes larger, the Q-factors approach those of a cavity in the wholly transformed space of transformation optics, and the far-field patterns are modified.


title: Construction of Differentially Private Summaries over Fully Homomorphic Encryption

abstract: Cloud computing has garnered attention as a platform of query processing systems. However, data privacy leakage is a critical problem. Chowdhury et al. proposed Crypt(epsilon), which executes differential privacy (DP) over encrypted data on two non-colluding semi-honest servers. Further, the DP index proposed by these authors summarizes a dataset to prevent information leakage while improving the performance. However, two problems persist: 1) the original data are decrypted to apply sorting via a garbled circuit, and 2) the added noise becomes large because the sorted data are partitioned with equal width, regardless of the data distribution. To solve these problems, we propose a new method called DP-summary that summarizes a dataset into differentially private data over a homomorphic encryption without decryption, thereby enhancing data security. Furthermore, our scheme adopts Li et al.'s data-aware and workload-aware (DAWA) algorithm for the encrypted data, thereby minimizing the noise caused by DP and reducing the errors of query responses. An experimental evaluation using torus fully homomorphic encryption (TFHE), a bit-wise fully homomorphic encryption library, confirms the applicability of the proposed method, which summarized eight 16-bit data in 12.5 h. We also confirmed that there was no accuracy degradation even after adopting TFHE along with the DAWA algorithm.


title: MAVE: A Product Dataset for Multi-source Attribute Value Extraction

abstract: Attribute value extraction refers to the task of identifying values of an attribute of interest from product information. Product attribute values are essential in many e-commerce scenarios, such as customer service robots, product ranking, retrieval and recommendations. While in the real world, the attribute values of a product are usually incomplete and vary over time, which greatly hinders the practical applications. In this paper, we introduce MAVE, a new dataset to better facilitate research on product attribute value extraction. MAVE is composed of a curated set of 2.2 million products from Amazon pages, with 3 million attribute-value annotations across 1257 unique categories. MAVE has four main and unique advantages: First, MAVE is the largest product attribute value extraction dataset by the number of attribute-value examples. Second, MAVE includes multi-source representations from the product, which captures the full product information with high attribute coverage. Third, MAVE represents a more diverse set of attributes and values relative to what previous datasets cover. Lastly, MAVE provides a very challenging zero-shot test set, as we empirically illustrate in the experiments. We further propose a novel approach that effectively extracts the attribute value from the multi-source product information. We conduct extensive experiments with several baselines and show that MAVE is an effective dataset for attribute value extraction task. It is also a very challenging task on zero-shot attribute extraction. Data is available at {\it \url{https://github.com/google-research-datasets/MAVE}}.


title: Strong intervalley correlation induced a magnetic order transition in monolayer $\text{MoS}_{2}$

abstract: In this work, we study a model for monolayer molybdenum disulfide with including the intravalley and intervalley electron-electron interaction. We solve the model at a self-consistent mean-field level and get three solutions $L_{0}$, $L_{+}$ and $L_{-}$. As for $L_{0}$, the spin polarizations are opposite at $\textbf{K}$ and $\textbf{K}^{\prime}$ valley and the total magnetization is zero. $L_{\pm}$ describe two degenerate spin-polarized states, and the directions of polarization are opposite for the states of $L_{+}$ and $L_{-}$. Based on these results, the ground state can be deduced to be spin polarized in domains in which their particular states can be randomly described by $L_{+}$ or $L_{-}$. Therefore, a zero net magnetization is induced for zero external magnetic field $\mathbf{B}$, but a global ferromagnetic ground state for a nonzero $\mathbf{B}$. We estimate the size of domains as several nanometers. As the increase of the chemical potential, the ground state changes between $L_{0}$ and $L_{\pm}$, indicating first order phase transitions at the borders, which is coincident with the observation of photoluminescence experiments in the absence of the external magnetic field [J. G. Roch $\it{et}$ $\it{al.}$, Phys. Rev. Lett. ${\bf 124}$, 187602 (2020)].


title: Joint Power-control and Antenna Selection in User-Centric Cell-Free Systems with Mixed Resolution ADC

abstract: In this paper, we propose a scheme for the joint optimization of the user transmit power and the antenna selection at the access points (AP)s of a user-centric cell-free massive multiple-input-multiple-output (UC CF-mMIMO) system. We derive an approximate expression for the achievable uplink rate of the users in a UC CF-mMIMO system in the presence of a mixed analog-to-digital converter (ADC) resolution profile at the APs. Using the derived approximation, we propose to maximize the uplink sum rate of UC CF-mMIMO systems subject to energy constraints at the APs. An alternating-optimization solution is proposed using binary particle swarm optimization (BPSO) and successive convex approximation (SCA). We also study the impact of various system parameters on the performance of the system.


title: Rotational Symmetry and Gauge Invariant Degeneracies on 2D Noncommutative Plane

abstract: We obtain the gauge invariant energy eigenvalues and degeneracies together with rotationally symmetric wavefunctions of a particle moving on 2D noncommutative plane subjected to homogeneous magnetic field $B$ and harmonic potential. This has been done by using the phase space coordinates transformation based on 2-parameter family of unitarily equivalent irreducible representations of the nilpotent Lie group $G_{NC}$. We find that the energy levels and states of the system are unique and hence, same goes to the degeneracies as well since they are heavily reliant on the applied $B$ and the noncommutativity $\theta $ of coordinates. Without $B$, we essentially have a noncommutative planar harmonic oscillator under generalized Bopp shift or Seiberg-Witten map. The degenerate energy levels can always be found if $\theta$ is proportional to the ratio between $\hbar$ and $m\omega$. For the scale $B\theta = \hbar$, the spectrum of energy is isomorphic to Landau problem in symmetric gauge and hence, each energy level is infinitely degenerate regardless of any values of $\theta$. Finally, if $0 < B\theta < \hbar$, $\theta$ has to also be proportional to the ratio between $\hbar$ and $m\omega$ for the degeneracy to occur. These proportionality parameters are evaluated and if they are not satisfied then we will have non-degenerate energy levels. Finally, the probability densities and effects of $B$ and $\theta$ on the system are properly shown for all cases.


title: Plastic vortex creep and dimensional crossovers in the highly anisotropic superconductor HgBa$_2$CuO$_{4+x}$

abstract: In type-II superconductors exposed to magnetic fields between upper and lower critical values, $H_{c1}$ and $H_{c2}$, penetrating magnetic flux forms a lattice of vortices whose motion can induce dissipation. Consequently, the magnetization $M$ of superconductors is typically progressively weakened with increasing magnetic field $B \propto n_v$ (for vortex density $n_v$). However, some materials exhibit a non-monotonic $M(B)$, presenting a maximum in $M$ at what is known as the second magnetization peak. This phenomenon appears in most classes of superconductors, including low $T_c$ materials, iron-based, and cuprates, complicating pinpointing its origin and garnering intense interest. Here, we report on vortex dynamics in optimally doped and overdoped HgBa$_2$CuO$_{4+x}$ crystals, with a focus on a regime in which plastic deformations of the vortex lattice govern magnetic properties. Specifically, we find that both crystals exhibit conspicuous second magnetization peaks and, from measurements of the field- and temperature- dependent vortex creep rates, identify and characterize phase boundaries between elastic and plastic vortex dynamics, as well as multiple previously unreported transitions within the plastic flow regime. We find that the second magnetization peak coincides with the elastic-to-plastic crossover for a very small range of high fields, and a sharp crossover within the plastic flow regime for a wider range of lower fields. We find evidence that this transition in the plastic flow regime is due to a dimensional crossover, specifically a transition from 3D to 2D plastic dynamics.


title: Simulation of Cosmic Rays in the Earth's Atmosphere and Interpretation of Observed Counts in an X-ray Detector at Balloon Altitude Near Tropical Region

abstract: The study of secondary particles produced by the cosmic-ray interaction in the Earth's atmosphere is very crucial as these particles mainly constitute the background counts produced in the high-energy detectors at balloon and satellite altitudes. In the present work, we calculate the abundance of cosmic-ray generated secondary particles at various heights of the atmosphere by means of a Monte Carlo simulation and use this result to understand the background counts in our X-ray observations using balloon-borne instruments operating near the tropical latitude (geomagnetic latitude: $\sim 14.50^{\circ}$ N). For this purpose, we consider a 3D description of the atmospheric and geomagnetic field configurations surrounding the Earth, as well as the electromagnetic and nuclear interaction processes using Geant4 simulation toolkit. Subsequently, we use a realistic mass model description of the detector under consideration, to simulate the counts produced in the detector due to secondary cosmic-ray particles.


title: Skyrmion Alignment and Pinning Effects in a Disordered Multi-Phase Skyrmion Material Co8Zn8Mn4

abstract: Underlying disorder in skyrmion materials may both inhibit and facilitate skyrmion reorientations and changes in topology. The identification of these disorder-induced topologically active regimes is critical to realizing robust skyrmion spintronic implementations, yet few studies exist for disordered bulk samples. Here, we employ small-angle neutron scattering (SANS) and micromagnetic simulations to examine the influence of skyrmion order on skyrmion lattice formation, transition, and reorientation dynamics across the phase space of a disordered polycrystalline Co$_{8}$Zn$_{8}$Mn$_{4}$ bulk sample. Our measurements reveal a new disordered-to-ordered skyrmion square lattice transition pathway characterized by the novel promotion of four-fold order in SANS and accompanied by a change in topology of the system, reinforced through micromagnetic simulations. Pinning responses are observed to dominate skyrmion dynamics in the metastable triangular lattice phase, enhancing skyrmion stabilization through a remarkable and previously undetected skyrmion memory effect which reproduces previous ordering processes and persists in zero field. These results uncover the cooperative interplay of anisotropy and disorder in skyrmion formation and restructuring dynamics, establishing new tunable pathways for skyrmion manipulation.


title: Amortized Noisy Channel Neural Machine Translation

abstract: Noisy channel models have been especially effective in neural machine translation (NMT). However, recent approaches like "beam search and rerank" (BSR) incur significant computation overhead during inference, making real-world application infeasible. We aim to study if it is possible to build an amortized noisy channel NMT model such that when we do greedy decoding during inference, the translation accuracy matches that of BSR in terms of reward (based on the source-to-target log probability and the target-to-source log probability) and quality (based on BLEU and BLEURT). We attempt three approaches to train the new model: knowledge distillation, one-step-deviation imitation learning, and Q learning. The first approach obtains the noisy channel signal from a pseudo-corpus, and the latter two approaches aim to optimize toward a noisy-channel MT reward directly. For all three approaches, the generated translations fail to achieve rewards comparable to BSR, but the translation quality approximated by BLEU and BLEURT is similar to the quality of BSR-produced translations. Additionally, all three approaches speed up inference by 1-2 orders of magnitude.


title: Model-free Bootstrap Prediction Regions for Multivariate Time Series

abstract: In Das and Politis(2020), a model-free bootstrap(MFB) paradigm was proposed for generating prediction intervals of univariate, (locally) stationary time series. Theoretical guarantees for this algorithm was resolved in Wang and Politis(2019) under stationarity and weak dependence condition. Following this line of work, here we extend MFB for predictive inference under a multivariate time series setup. We describe two algorithms, the first one works for a particular class of time series under any fixed dimension d; the second one works for a more generalized class of time series under low-dimensional setting. We justify our procedure through theoretical validity and simulation performance.


title: Thermodynamics of Kerr-AdS black holes in the restricted phase space

abstract: The thermodynamics for Kerr-AdS black hole in four dimensions is revisited using the recently proposed restricted phase space formalism, which includes the central charge $C$ of the dual CFT and the chemical potential $\mu$, but excludes the pressure and the conjugate volume, as thermodynamic variables. The Euler relation holds automatically, and the first order homogeneity of the mass and the zeroth order homogeneity of the intensive variables are made explicit. Thermodynamic processes involving each pair of conjugate variables are studied in some detail, with emphasis on the scaling properties of the equations of states. It turns out that the thermodynamic behavior of the Kerr-AdS black hole is very similar to that of the RN-AdS black hole studied earlier. In particular, it is found that, there is a first order supercritical phase equilibrium in the $T-S$ processes at fixed nonvanishing angular momentum, while at vanishing angular momentum or at fixed angular velocities, there is always a non-equilibrium transition from a small unstable black hole state to a large stable black hole state. Moreover, there is a Hawking-Page phase transition in the $\mu-C$ processes. Due to the complicatedness of the Kerr metric, the exact critical point and the Hawking-Page temperature are worked out explicitly only in the slow rotating limit, however the characteristic thermodynamic properties do not rely on the slow rotating approximation.


title: Intelligent Bearing Fault Diagnosis Method Combining Mixed Input and Hybrid CNN-MLP model

abstract: Rolling bearings are one of the most widely used bearings in industrial machines. Deterioration in the condition of rolling bearings can result in the total failure of rotating machinery. AI-based methods are widely applied in the diagnosis of rolling bearings. Hybrid NN-based methods have been shown to achieve the best diagnosis results. Typically, raw data is generated from accelerometers mounted on the machine housing. However, the diagnostic utility of each signal is highly dependent on the location of the corresponding accelerometer. This paper proposes a novel hybrid CNN-MLP model-based diagnostic method which combines mixed input to perform rolling bearing diagnostics. The method successfully detects and localizes bearing defects using acceleration data from a shaft-mounted wireless acceleration sensor. The experimental results show that the hybrid model is superior to the CNN and MLP models operating separately, and can deliver a high detection accuracy of 99,6% for the bearing faults compared to 98% for CNN and 81% for MLP models.


title: Reframing Human-AI Collaboration for Generating Free-Text Explanations

abstract: Large language models are increasingly capable of generating fluent-appearing text with relatively little task-specific supervision. But can these models accurately explain classification decisions? We consider the task of generating free-text explanations using human-written examples in a few-shot manner. We find that (1) authoring higher quality prompts results in higher quality generations; and (2) surprisingly, in a head-to-head comparison, crowdworkers often prefer explanations generated by GPT-3 to crowdsourced explanations in existing datasets. Our human studies also show, however, that while models often produce factual, grammatical, and sufficient explanations, they have room to improve along axes such as providing novel information and supporting the label. We create a pipeline that combines GPT-3 with a supervised filter that incorporates binary acceptability judgments from humans in the loop. Despite the intrinsic subjectivity of acceptability judgments, we demonstrate that acceptability is partially correlated with various fine-grained attributes of explanations. Our approach is able to consistently filter GPT-3-generated explanations deemed acceptable by humans.


title: Volterra type operators on minimal Mobius invariant space

abstract: In this note, we mainly study operator-theoretic properties on Besov space $B_{1}$ on the unit disc. This space is the minimal Mobius invariant space. Firstly, we consider the boundedness of Volterra type operators. Secondly, we prove that Volterra type operators belong to the Deddens algebra of composition operator. Thirdly, we obtain estimates for the essential norm of Volterra type operators. Finally, we give a complete characterization of spectrum of Volterra type operators.


title: Machine Learning-Accelerated Computational Solid Mechanics: Application to Linear Elasticity

abstract: This work presents a novel physics-informed deep learning based super-resolution framework to reconstruct high-resolution deformation fields from low-resolution counterparts, obtained from coarse mesh simulations or experiments. We leverage the governing equations and boundary conditions of the physical system to train the model without using any high-resolution labeled data. The proposed approach is applied to obtain the super-resolved deformation fields from the low-resolution stress and displacement fields obtained by running simulations on a coarse mesh for a body undergoing linear elastic deformation. We demonstrate that the super-resolved fields match the accuracy of an advanced numerical solver running at 400 times the coarse mesh resolution, while simultaneously satisfying the governing laws. A brief evaluation study comparing the performance of two deep learning based super-resolution architectures is also presented.


title: Monte Carlo Simulation of CRAND Protons Trapped at Low Earth Orbits

abstract: The Cosmic Ray Albedo Neutron Decay (CRAND) is believed to be the principal mechanism for the formation of inner proton radiation belt -- at least for relatively higher energy particles. We implement this mechanism in a Monte Carlo simulation procedure to calculate the trapped proton radiation at the low Earth orbits, through event-by-event interaction of the cosmic ray particles in the Earth's atmosphere and their transportation in the magnetosphere. We consider the generation of protons from subsequent decay of the secondary neutrons from the cosmic ray interaction in the atmosphere and their transport (and/or trapping) in the geomagnetic field. We address the computational challenges for this type of calculations and develop an optimized algorithm to minimize the computation time. We consider a full 3D description of the Earth's atmospheric and magnetic-field configurations using the latest available models. We present the spatial and phase-space distribution of the trapped protons considering the adiabatic invariants and other parameters at the low Earth orbits. We compare the simulation results with the trapped proton flux measurements made by PAMELA experiment at low Earth orbit and explain certain features observed by the measurement.


title: Asymptotically Optimal Golay-ZCZ Sequence Sets with Flexible Length

abstract: Zero correlation zone (ZCZ) sequences and Golay complementary sequences are two kinds of sequences with different preferable correlation properties. Golay-ZCZ sequences are special kinds of complementary sequences which also possess a large ZCZ and are good candidates for pilots in OFDM systems. Known Golay-ZCZ sequences reported in the literature have a limitation in the length which is the form of a power of 2. In this paper, we propose two constructions of Golay-ZCZ sequence sets with new parameters which generalize the constructions of Gong et al. (IEEE Transaction on Communications 61(9), 2013) and Chen et al (IEEE Transaction on Communications 61(9), 2018). Notably, one of the constructions results in optimal binary Golay-ZCZ sequences, while the other results in asymptotically optimal polyphase Golay-ZCZ sequences as the number of sequences increases.


title: Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation

abstract: Contrastive learning (CL) recently has spurred a fruitful line of research in the field of recommendation, since its ability to extract self-supervised signals from the raw data is well-aligned with recommender systems' needs for tackling the data sparsity issue. A typical pipeline of CL-based recommendation models is first augmenting the user-item bipartite graph with structure perturbations, and then maximizing the node representation consistency between different graph augmentations. Although this paradigm turns out to be effective, what underlies the performance gains is still a mystery. In this paper, we first experimentally disclose that, in CL-based recommendation models, CL operates by learning more evenly distributed user/item representations that can implicitly mitigate the popularity bias. Meanwhile, we reveal that the graph augmentations, which were considered necessary, just play a trivial role. Based on this finding, we propose a simple CL method which discards the graph augmentations and instead adds uniform noises to the embedding space for creating contrastive views. A comprehensive experimental study on three benchmark datasets demonstrates that, though it appears strikingly simple, the proposed method can smoothly adjust the uniformity of learned representations and has distinct advantages over its graph augmentation-based counterparts in terms of recommendation accuracy and training efficiency. The code is released at https://github.com/Coder-Yu/QRec.


title: Completeness of Discrete Translates in $H^1(\mathbb{R})$

abstract: We provide a characterization of discrete sets $\Lambda \subset \mathbb{R}$ that admit a function whose $\Lambda$-translates are complete in the Hardy space $H^1(\mathbb{R})$. In particular, we show that such a set cannot be uniformly discrete. We then give a uniformly discrete $\Lambda \subset \mathbb{R}$ which admits a pair of functions such that their $\Lambda$-translates are complete in $H^1(\mathbb{R})$.


title: On the commuting probability of p-elements in a finite group

abstract: Let $G$ be a finite group, let $p$ be a prime and let ${\rm Pr}_p(G)$ be the probability that two random $p$-elements of $G$ commute. In this paper we prove that ${\rm Pr}_p(G) > (p^2+p-1)/p^3$ if and only if $G$ has a normal and abelian Sylow $p$-subgroup, which generalizes previous results on the widely studied commuting probability of a finite group. This bound is best possible in the sense that for each prime $p$ there are groups with ${\rm Pr}_p(G) = (p^2+p-1)/p^3$ and we classify all such groups. Our proof is based on bounding the proportion of $p$-elements in $G$ that commute with a fixed $p$-element in $G \setminus \textbf{O}_p(G)$, which in turn relies on recent work of the first two authors on fixed point ratios for finite primitive permutation groups.


title: Isometric MT: Neural Machine Translation for Automatic Dubbing

abstract: Automatic dubbing (AD) is among the machine translation (MT) use cases where translations should match a given length to allow for synchronicity between source and target speech. For neural MT, generating translations of length close to the source length (e.g. within +-10% in character count), while preserving quality is a challenging task. Controlling MT output length comes at a cost to translation quality, which is usually mitigated with a two step approach of generating N-best hypotheses and then re-ranking based on length and quality. This work introduces a self-learning approach that allows a transformer model to directly learn to generate outputs that closely match the source length, in short Isometric MT. In particular, our approach does not require to generate multiple hypotheses nor any auxiliary ranking function. We report results on four language pairs (English - French, Italian, German, Spanish) with a publicly available benchmark. Automatic and manual evaluations show that our method for Isometric MT outperforms more complex approaches proposed in the literature.


title: Normal modes for two-dimensional gravitating kinks

abstract: We study small perturbations around an arbitrary static kink solution of a two-dimensional (2D) gravity-scalar system, where the gravity part is described by a subclass of 2D dilaton gravity theory, and the scalar matter field has generalized dynamics. We expand the action around an arbitrary static solution and keep terms up to the second order of the perturbations. After variation the linear-order action leads to background field equations, as expected. The quadratic action of the normal modes are obtained after fixing the gauge and using the constraint equation. The linear perturbation equations obtained from the quadratic action are consistent with those obtained by linearizing the field equations under the dilaton gauge. All the calculations are assisted by a Mathematica code, which is also provided as a supplementary material.


title: Mimic Embedding via Adaptive Aggregation: Learning Generalizable Person Re-identification

abstract: Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time, which is a realistic but challenging problem. In contrast to methods assuming an identical model for different domains, Mixture of Experts (MoE) exploits multiple domain-specific networks for leveraging complementary information between domains, obtaining impressive results. However, prior MoE-based DG ReID methods suffer from a large model size with the increase of the number of source domains, and most of them overlook the exploitation of domain-invariant characteristics. To handle the two issues above, this paper presents a new approach called Mimic Embedding via adapTive Aggregation (META) for DG person ReID. To avoid the large model size, experts in META do not adopt a branch network for each source domain but share all the parameters except for the batch normalization layers. Besides multiple experts, META leverages Instance Normalization (IN) and introduces it into a global branch to pursue invariant features across domains. Meanwhile, META considers the relevance of an unseen target sample and source domains via normalization statistics and develops an aggregation module to adaptively integrate multiple experts for mimicking unseen target domain. Benefiting from a proposed consistency loss and an episodic training algorithm, META is expected to mimic embedding for a truly unseen target domain. Extensive experiments verify that META surpasses state-of-the-art DG person ReID methods by a large margin. Our code is available at https://github.com/xbq1994/META.


title: The maximum measure of 3-wise t-intersecting families

abstract: Let $\mathcal G$ be a family of subsets of an $n$-element set. The family $\mathcal G$ is called $3$-wise $t$-intersecting if the intersection of any three subsets in $\mathcal G$ is of size at least $t$. For a real number $p\in(0,1)$ we define the measure of the family by the sum of $p^{|G|}(1-p)^{n-|G|}$ over all $G\in\mathcal G$. We prove that if $t\geq 15$ and $p\leq 2/(\sqrt{4t+9}-1)$ then $p^t$ is the maximum measure of $3$-wise $t$-intersecting families, and the bound for $p$ is sharp. We also present the corresponding stability result for shifted families.


title: Ruta: Dis-aggregated routing system over multi-cloud

abstract: Over the years, the SDN evolution create multiple overlay technologies which is inefficient and hard to deploy end-to-end traffic engineering services, Ruta is designed as an unified encapsulation with Segment Routing, Crypto and NAT-Traversal capabilities over UDP. Ruta could be deployed as a cloud native SDN platform globally over multi-cloud and integrated with each applications on transport layer, which provide nearly zero loss and almost less than 200ms latency to access anywhere in the world over internet.


title: BLEND: A Fast, Memory-Efficient, and Accurate Mechanism to Find Fuzzy Seed Matches in Genome Analysis

abstract: Generating the hash values of short subsequences, called seeds, enables quickly identifying similarities between genomic sequences by matching seeds with a single lookup of their hash values. However, these hash values can be used only for finding exact-matching seeds as the conventional hashing methods assign distinct hash values for different seeds, including highly similar seeds. Finding only exact-matching seeds causes either 1) increasing the use of the costly sequence alignment or 2) limited sensitivity. We introduce BLEND, the first efficient and accurate mechanism that can identify both exact-matching and highly similar seeds with a single lookup of their hash values, called fuzzy seed matches. BLEND 1) utilizes a technique called SimHash, that can generate the same hash value for similar sets, and 2) provides the proper mechanisms for using seeds as sets with the SimHash technique to find fuzzy seed matches efficiently. We show the benefits of BLEND when used in read overlapping and read mapping. For read overlapping, BLEND is faster by 2.4x - 83.9x (on average 19.3x), has a lower memory footprint by 0.9x - 14.1x (on average 3.8x), and finds higher quality overlaps leading to accurate de novo assemblies than the state-of-the-art tool, minimap2. For read mapping, BLEND is faster by 0.8x - 4.1x (on average 1.7x) than minimap2. Source code is available at https://github.com/CMU-SAFARI/BLEND.


title: Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks

abstract: Retrieval-augmented generation models have shown state-of-the-art performance across many knowledge-intensive NLP tasks such as open question answering and fact verification. These models are trained to generate the final output given the retrieved passages, which can be irrelevant to the original query, leading to learning spurious cues or answer memorization. This work introduces a method to incorporate the evidentiality of passages -- whether a passage contains correct evidence to support the output -- into training the generator. We introduce a multi-task learning framework to jointly generate the final output and predict the evidentiality of each passage, leveraging a new task-agnostic method to obtain silver evidentiality labels for supervision. Our experiments on five datasets across three knowledge-intensive tasks show that our new evidentiality-guided generator significantly outperforms its direct counterpart with the same-size model and advances the state of the art on FaVIQ-Ambig. We attribute these improvements to both the auxiliary multi-task learning and silver evidentiality mining techniques.


title: A Toda bracket convergence theorem for multiplicative spectral sequences

abstract: Moss' theorem, which relates Massey products in the $E_r$-page of the classical Adams spectral sequence to Toda brackets of homotopy groups, is one of the main tools for calculating Adams differentials. Working in an arbitrary symmetric monoidal stable topological model category, we prove a general version of Moss' theorem which applies to spectral sequences that arise from filtrations compatible with the monoidal structure. The theorem has broad applications, e.g. to the computation of the motivic slice and motivic Adams spectral sequences.


title: Extracting Photometric Redshift from Galaxy Flux and Image Data using Neural Networks in the CSST Survey

abstract: The accuracy of galaxy photometric redshift (photo-$z$) can significantly affect the analysis of weak gravitational lensing measurements, especially for future high-precision surveys. In this work, we try to extract photo-$z$ information from both galaxy flux and image data expected to be obtained by China Space Station Telescope (CSST) using neural networks. We generate mock galaxy images based on the observational images from the Advanced Camera for Surveys of Hubble Space Telescope (HST-ACS) and COSMOS catalogs, considering the CSST instrumental effects. Galaxy flux data are then measured directly from these images by aperture photometry. The Multi-Layer Perceptron (MLP) and Convolutional Neural Network (CNN) are constructed to predict photo-$z$ from fluxes and images, respectively. We also propose to use an efficient hybrid network, which combines MLP and CNN, by employing transfer learning techniques to investigate the improvement of the result with both flux and image data included. We find that the photo-$z$ accuracy and outlier fraction can achieve $\sigma_{\rm NMAD} = 0.023$ and $\eta = 1.43\%$ for the MLP using flux data only, and $\sigma_{\rm NMAD} = 0.025$ and $\eta = 1.21\%$ for the CNN using image data only. The result can be further improved in high efficiency as $\sigma_{\rm NMAD} = 0.020$ and $\eta = 0.90\%$ for the hybrid transfer network. These approaches result in similar galaxy median and mean redshifts ~0.8 and 0.9, respectively, for the redshift range from 0 to 4. This indicates that our networks can effectively and properly extract photo-$z$ information from the CSST galaxy flux and image data.


title: Towards Robust Neural Image Compression: Adversarial Attack and Model Finetuning

abstract: Deep neural network-based image compression has been extensively studied. However, the model robustness which is crucial to practical application is largely overlooked. We propose to examine the robustness of prevailing learned image compression models by injecting negligible adversarial perturbation into the original source image. Severe distortion in decoded reconstruction reveals the general vulnerability in existing methods regardless of their settings (e.g., network architecture, loss function, quality scale). A variety of defense strategies including geometric self-ensemble based pre-processing, and adversarial training, are investigated against the adversarial attack to improve the model's robustness. Later the defense efficiency is further exemplified in real-life image recompression case studies. Overall, our methodology is simple, effective, and generalizable, making it attractive for developing robust learned image compression solutions. All materials are made publicly accessible at https://njuvision.github.io/RobustNIC for reproducible research.


title: Lacuna Reconstruction: Self-supervised Pre-training for Low-Resource Historical Document Transcription

abstract: We present a self-supervised pre-training approach for learning rich visual language representations for both handwritten and printed historical document transcription. After supervised fine-tuning of our pre-trained encoder representations for low-resource document transcription on two languages, (1) a heterogeneous set of handwritten Islamicate manuscript images and (2) early modern English printed documents, we show a meaningful improvement in recognition accuracy over the same supervised model trained from scratch with as few as 30 line image transcriptions for training. Our masked language model-style pre-training strategy, where the model is trained to be able to identify the true masked visual representation from distractors sampled from within the same line, encourages learning robust contextualized language representations invariant to scribal writing style and printing noise present across documents.


title: Helmholtz equation and non-singular boundary elements applied to multi-disciplinary physical problems

abstract: The famous scientist Hermann von Helmholtz was born 200 years ago. Many complex physical wave phenomena in engineering can effectively be described using one or a set of equations named after him: the Helmholtz equation. Although this has been known for a long time from a theoretical point of view, the actual numerical implementation has often been hindered by divergence free and/or curl free constraints. There is further a need for a numerical method that is accurate, reliable and takes into account radiation conditions at infinity. The classical boundary element method (BEM) satisfies the last condition, yet one has to deal with singularities in the implementation. We review here how a recently developed singularity-free three-dimensional (3D) boundary element framework with superior accuracy can be used to tackle such problems only using one or a few Helmholtz equations with higher order (quadratic) elements which can tackle complex curved shapes. Examples are given for acoustics (a Helmholtz resonator among others) and electromagnetic scattering.

categories: math.NA cs.NA physics.class-ph physics.comp-ph physics.flu-dyn