title: Automated Audio Captioning via Fusion of Low- and High- Dimensional Features

abstract: Automated audio captioning (AAC) aims to describe the content of an audio clip using simple sentences. Existing AAC methods are developed based on an encoder-decoder architecture that success is attributed to the use of a pre-trained CNN10 called PANNs as the encoder to learn rich audio representations. AAC is a highly challenging task due to its high-dimensional talent space involves audio of various scenarios. Existing methods only use the high-dimensional representation of the PANNs as the input of the decoder. However, the low-dimension representation may retain as much audio information as the high-dimensional representation may be neglected. In addition, although the high-dimensional approach may predict the audio captions by learning from existing audio captions, which lacks robustness and efficiency. To deal with these challenges, a fusion model which integrates low- and high-dimensional features AAC framework is proposed. In this paper, a new encoder-decoder framework is proposed called the Low- and High-Dimensional Feature Fusion (LHDFF) model for AAC. Moreover, in LHDFF, a new PANNs encoder is proposed called Residual PANNs (RPANNs) by fusing the low-dimensional feature from the intermediate convolution layer output and the high-dimensional feature from the final layer output of PANNs. To fully explore the information of the low- and high-dimensional fusion feature and high-dimensional feature respectively, we proposed dual transformer decoder structures to generate the captions in parallel. Especially, a probabilistic fusion approach is proposed that can ensure the overall performance of the system is improved by concentrating on the respective advantages of the two transformer decoders. Experimental results show that LHDFF achieves the best performance on the Clotho and AudioCaps datasets compared with other existing models


title: Fighting FIRe with FIRE: Assessing the Validity of Text-to-Video Retrieval Benchmarks

abstract: Searching troves of videos with textual descriptions is a core multimodal retrieval task. Owing to the lack of a purpose-built dataset for text-to-video retrieval, video captioning datasets have been re-purposed to evaluate models by (1) treating captions as positive matches to their respective videos and (2) assuming all other videos to be negatives. However, this methodology leads to a fundamental flaw during evaluation: since captions are marked as relevant only to their original video, many alternate videos also match the caption, which introduces false-negative caption-video pairs. We show that when these false negatives are corrected, a recent state-of-the-art model gains 25\% recall points -- a difference that threatens the validity of the benchmark itself. To diagnose and mitigate this issue, we annotate and release 683K additional caption-video pairs. Using these, we recompute effectiveness scores for three models on two standard benchmarks (MSR-VTT and MSVD). We find that (1) the recomputed metrics are up to 25\% recall points higher for the best models, (2) these benchmarks are nearing saturation for Recall@10, (3) caption length (generality) is related to the number of positives, and (4) annotation costs can be mitigated through sampling. We recommend retiring these benchmarks in their current form, and we make recommendations for future text-to-video retrieval benchmarks.


title: Contrastive Video-Language Learning with Fine-grained Frame Sampling

abstract: Despite recent progress in video and language representation learning, the weak or sparse correspondence between the two modalities remains a bottleneck in the area. Most video-language models are trained via pair-level loss to predict whether a pair of video and text is aligned. However, even in paired video-text segments, only a subset of the frames are semantically relevant to the corresponding text, with the remainder representing noise; where the ratio of noisy frames is higher for longer videos. We propose FineCo (Fine-grained Contrastive Loss for Frame Sampling), an approach to better learn video and language representations with a fine-grained contrastive objective operating on video frames. It helps distil a video by selecting the frames that are semantically equivalent to the text, improving cross-modal correspondence. Building on the well established VideoCLIP model as a starting point, FineCo achieves state-of-the-art performance on YouCookII, a text-video retrieval benchmark with long videos. FineCo also achieves competitive results on text-video retrieval (MSR-VTT), and video question answering datasets (MSR-VTT QA and MSR-VTT MC) with shorter videos.


title: Constraints on the contributions to the observed binary black hole population from individual evolutionary pathways in isolated binary evolution

abstract: Gravitational waves from merging binary black holes can be used to shed light on poorly understood aspects of massive binary stellar evolution, such as the evolution of massive stars (including their mass-loss rates), the common envelope phase, and the rate at which massive stars form throughout the cosmic history of the Universe. In this paper we explore the \emph{correlated} impact of these phases on predictions for the merger rate and chirp mass distribution of merging binary black holes, aiming to identify possible degeneracies between model parameters. In many of our models, a large fraction (more than 70% of detectable binary black holes) arise from the chemically homogeneous evolution scenario; these models tend to over-predict the binary black hole merger rate and produce systems which are on average too massive. Our preferred models favour enhanced mass-loss rates for helium rich Wolf--Rayet stars, in tension with recent theoretical and observational developments. We identify correlations between the impact of the mass-loss rates of Wolf--Rayet stars and the metallicity evolution of the Universe on the rates and properties of merging binary black holes. Based on the observed mass distribution, we argue that the $\sim 10\%$ of binary black holes with chirp masses greater than $40$ M$_\odot$ (the maximum predicted by our models) are unlikely to have formed through isolated binary evolution, implying a significant contribution (> 10%) from other formation channels such as dense star clusters or active galactic nuclei. Our models will enable inference on the uncertain parameters governing binary evolution in the near future.


title: Carrier thermalization and zero-point bandgap renormalization in halide perovskites from the Urbach tails of the emission spectrum

abstract: We develop techniques to study the temperature dependent localization, thermalization, and the effects of phonon scattering on the excitons in halide perovskites from the analysis of the emission spectra. The excitonic Urbach edge, when inferred from the low energy tails of the temperature dependent luminescence spectra, is shown to be sensitive to the electron distribution and thermalization. A method to observe the Urbach focus is devised for halide perovskites where the temperature dependence of the excitonic gap is anomalous. The value of the zero-point bandgap renormalization is inferred to be about 33 meV. This small value of the bandgap renormalization rules out the formation of small polarons and points to weak electron-phonon coupling. The experiments are performed on the nanosheets of the archetypal halide perovskite, CsPbBr$_3$.


title: Conventional high-temperature superconductivity in metallic, covalently bonded, binary-guest C-B clathrates

abstract: Inspired by the synthesis of XB3C3 (X= Sr, La) compounds in the bipartite sodalite clathrate structure, density functional theory (DFT) calculations are performed on members of this family containing up to two different metal atoms. A DFT-chemical pressure analysis on systems with X= Mg, Ca, Sr, Ba reveals that the size of the metal cation, which can be tuned to stabilize the B-C framework, is key for their ambient-pressure dynamic stability. High-throughput density functional theory calculations on 105 Pm-3 symmetry XYB6C6 binary-guest compounds (where X, Y are electropositive metal atoms) find 22 that are dynamically stable at 1 atmosphere, expanding the number of potentially synthesizable phases by 19 (18 metals and 1 insulator). The density of states at the Fermi level and superconducting critical temperature, Tc, can be tuned by changing the average oxidation state of the metal atoms, with Tc being highest for an average valence of +1.5. KPbB6C6, with an ambient-pressure Eliashberg Tc of 88 K, is predicted to possess the highest-Tc among the studied Pm-3n XB3C3 or Pm-3 XY B6C6 phases, and calculations suggest it may be synthesized using high-pressure high-temperature techniques then quenched to ambient conditions.


title: Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling

abstract: Ensembling BERT models often significantly improves accuracy, but at the cost of significantly more computation and memory footprint. In this work, we propose Multi-CLS BERT, a novel ensembling method for CLS-based prediction tasks that is almost as efficient as a single BERT model. Multi-CLS BERT uses multiple CLS tokens with a parameterization and objective that encourages their diversity. Thus instead of fine-tuning each BERT model in an ensemble (and running them all at test time), we need only fine-tune our single Multi-CLS BERT model (and run the one model at test time, ensembling just the multiple final CLS embeddings). To test its effectiveness, we build Multi-CLS BERT on top of a state-of-the-art pretraining method for BERT (Aroca-Ouellette and Rudzicz, 2020). In experiments on GLUE and SuperGLUE we show that our Multi-CLS BERT reliably improves both overall accuracy and confidence estimation. When only 100 training samples are available in GLUE, the Multi-CLS BERT_Base model can even outperform the corresponding BERT_Large model. We analyze the behavior of our Multi-CLS BERT, showing that it has many of the same characteristics and behavior as a typical BERT 5-way ensemble, but with nearly 4-times less computation and memory.


title: Modelling the Relationship Between Post Encroachment Time and Signal Timings Using UAV Video data

abstract: Intersection safety often relies on the correct modelling of signal phasing and timing parameters. A slight increase in yellow time or red time can have significant impact on the rear end crashes or conflicts. This paper aims to identify the relationship between surrogate safety measures and signal phasing. Unmanned Aerial Vehicle (UAV) video data has been used to study an intersection. Post Encroachment Time (PET) between vehicles was calculated from the video data as well as speed, heading and relevant signal timing parameters such as all red time, red clearance time, yellow time, etc. Random Parameter Ordered Logit Model was used to model the relationship between PET and these signal timing parameters. Overall, the results showed that yellow time and red clearance time is positively related to PETs. The model was also able to idendity certain signal phases that could be a potential safety hazard and would need to be retimed by considering the PETs. The odds ratios from the models also indicates that increasing the yellow and red clearance times by one second can improve the PET levels by 16% and 3% respectively.


title: Formation of amorphous carbon multi-walled nanotubes from random initial configurations

abstract: Amorphous carbon nanotubes (a-CNT) with up to four walls and sizes ranging from 200 to 3200 atoms have been simulated, starting from initial random configurations and using the Gaussian Approximation Potential [Phys. Rev. B 95, 094203 (2017)]. The important variables (like density, height, and diameter) required to successfully simulate a-CNTs, were predicted with a machine learning random forest technique. The models were validated $\textit{ex post facto}$ using density functional codes. The a-CNT models ranged from 0.55 nm - 2 nm wide with an average inter-wall spacing of 0.31 nm. The topological defects in a-CNTs were discussed and new defect configurations were observed. The electronic density of states and localization in these phases were discussed and delocalized electrons in the $\pi$ subspace were identified as an important factor for inter-layer cohesion. Spatial projection of the electronic conductivity favors axial transport along connecting hexagons, while non-hexagonal parts of the network either hinder or bifurcate the electronic transport. A vibrational density of states was calculated and is potentially an experimentally testable fingerprint of the material and the appearance of a low-frequency radial breathing mode was discussed. The thermal conductivity at 300 K was calculated using the Green-Kubo formula.


title: Data-Driven Feedback Linearization using the Koopman Generator

abstract: This paper contributes a theoretical framework for data-driven feedback linearization of nonlinear control-affine systems. We unify the traditional geometric perspective on feedback linearization with an operator-theoretic perspective involving the Koopman operator. We first show that if the distribution of the control vector field and its repeated Lie brackets with the drift vector field is involutive, then there exists an output and a feedback control law for which the Koopman generator is finite-dimensional and locally nilpotent. We use this connection to propose a data-driven algorithm Koopman Generator-based Feedback Linearization (KGFL) for feedback linearization. Particularly, we use experimental data to identify the state transformation and control feedback from a dictionary of functions for which feedback linearization is achieved in a least-squares sense. We also propose a single-step data-driven formula which can be used to compute the linearizing transformations. When the system is feedback linearizable and the chosen dictionary is complete, our data-driven algorithm provides the same solution as model-based feedback linearization. Finally, we provide numerical examples for the data-driven algorithm and compare it with model-based feedback linearization. We also numerically study the effect of the richness of the dictionary and the size of the data set on the effectiveness of feedback linearization.


title: Improving Retrieval Augmented Neural Machine Translation by Controlling Source and Fuzzy-Match Interactions

abstract: We explore zero-shot adaptation, where a general-domain model has access to customer or domain specific parallel data at inference time, but not during training. We build on the idea of Retrieval Augmented Translation (RAT) where top-k in-domain fuzzy matches are found for the source sentence, and target-language translations of those fuzzy-matched sentences are provided to the translation model at inference time. We propose a novel architecture to control interactions between a source sentence and the top-k fuzzy target-language matches, and compare it to architectures from prior work. We conduct experiments in two language pairs (En-De and En-Fr) by training models on WMT data and testing them with five and seven multi-domain datasets, respectively. Our approach consistently outperforms the alternative architectures, improving BLEU across language pair, domain, and number k of fuzzy matches.


title: Speeding up entanglement generation by proximity to higher-order exceptional points

abstract: Entanglement is a key resource for quantum information technologies ranging from quantum sensing to quantum computing. Conventionally, the entanglement between two coupled qubits is established at the time scale of the inverse of the coupling strength. In this work, we study two weakly coupled non-Hermitian qubits and observe entanglement generation at a significantly shorter time scale by proximity to a higher-order exceptional point. We establish a non-Hermitian perturbation theory based on constructing a biorthogonal complete basis and further identify the optimal condition to obtain the maximally entangled state. Our study of speeding up entanglement generation in non-Hermitian quantum systems opens new avenues for harnessing coherent nonunitary dissipation for quantum technologies.


title: Single Event Tolerance of X-ray SOI Pixel Sensors

abstract: We evaluate the single event tolerance of the X-ray silicon-on-insulator (SOI) pixel sensor named XRPIX, developed for the future X-ray astronomical satellite FORCE. In this work, we measure the cross-section of single event upset (SEU) of the shift register on XRPIX by irradiating heavy ion beams with linear energy transfer (LET) ranging from 0.022 MeV/(mg/cm2) to 68 MeV/(mg/cm2). From the SEU cross-section curve, the saturation cross-section and threshold LET are successfully obtained to be $3.4^{+2.9}_{-0.9}\times 10^{-10}~{\rm cm^2/bit}$ and $7.3^{+1.9}_{-3.5}~{\rm MeV/(mg/cm^2)}$, respectively. Using these values, the SEU rate in orbit is estimated to be $\lesssim$ 0.1 event/year primarily due to the secondary particles induced by cosmic-ray protons. This SEU rate of the shift register on XRPIX is negligible in the FORCE orbit.


title: Neurosymbolic Programming for Science

abstract: Neurosymbolic Programming (NP) techniques have the potential to accelerate scientific discovery. These models combine neural and symbolic components to learn complex patterns and representations from data, using high-level concepts or known constraints. NP techniques can interface with symbolic domain knowledge from scientists, such as prior knowledge and experimental context, to produce interpretable outputs. We identify opportunities and challenges between current NP models and scientific workflows, with real-world examples from behavior analysis in science: to enable the use of NP broadly for workflows across the natural and social sciences.


title: Spreading Processes with Mutations over Multi-layer Networks

abstract: A key scientific challenge during the outbreak of novel infectious diseases is to predict how the course of the epidemic changes under different countermeasures that limit interaction in the population. Most epidemiological models do not consider the role of mutations and heterogeneity in the type of contact events. However, pathogens have the capacity to mutate in response to changing environments, especially caused by the increase in population immunity to existing strains and the emergence of new pathogen strains poses a continued threat to public health. Further, in light of differing transmission risks in different congregate settings (e.g., schools and offices), different mitigation strategies may need to be adopted to control the spread of infection. We analyze a multi-layer multi-strain model by simultaneously accounting for i) pathways for mutations in the pathogen leading to the emergence of new pathogen strains, and ii) differing transmission risks in different congregate settings, modeled as network-layers. Assuming complete cross-immunity among strains, namely, recovery from any infection prevents infection with any other (an assumption that will need to be relaxed to deal with COVID-19 or influenza), we derive the key epidemiological parameters for the proposed multi-layer multi-strain framework. We demonstrate that reductions to existing network-based models that discount heterogeneity in either the strain or the network layers can lead to incorrect predictions for the course of the outbreak. In addition, our results highlight that the impact of imposing/lifting mitigation measures concerning different contact network layers (e.g., school closures or work-from-home policies) should be evaluated in connection with their effect on the likelihood of the emergence of new pathogen strains.


title: Risk Automatic Prediction for Social Economy Companies using Camels

abstract: Governments have to supervise and inspect social economy enterprises (SEEs). However, inspecting all SEEs is not possible due to the large number of SEEs and the low number of inspectors in general. We proposed a prediction model based on a machine learning approach. The method was trained with the random forest algorithm with historical data provided by each SEE. Three consecutive periods of data were concatenated. The proposed method uses these periods as input data and predicts the risk of each SEE in the fourth period. The model achieved 76\% overall accuracy. In addition, it obtained good accuracy in predicting the high risk of a SEE. We found that the legal nature and the variation of the past-due portfolio are good predictors of the future risk of a SEE. Thus, the risk of a SEE in a future period can be predicted by a supervised machine learning method. Predicting the high risk of a SEE improves the daily work of each inspector by focusing only on high-risk SEEs.


title: Strong coupling of a Gd$^{3+}$ multilevel spin system to an on-chip superconducting resonator

abstract: We report the realization of a strong coupling between a Gd$^{3+}$ spin ensemble hosted in a scheelite (CaWO$_4$) single crystal and the resonant mode of a coplanar stripline superconducting cavity leading to a large separation of spin-photon states of 146 MHz. The interaction is well described by the Dicke model and the crystal-field Hamiltonian of the multilevel spin system. We observe a change of the crystal-field parameters due to the presence of photons in the cavity that generates a significant perturbation of the crystal ground state. Using finite-element calculations, we numerically estimate the cavity sensing volume as well as the average spin-photon coupling strength of $g_0\approx$ 620 Hz. Lastly, the dynamics of the spin-cavity states are explored via pulsed measurements by recording the cavity ring-down signal as a function of pulse length and amplitude. The results indicate a potential method to initialize this multilevel system in its ground state via an active cooling process.


title: Relative slow entropy

abstract: In 1997, Katok--Thouvenot and Ferenczi independently introduced a notion of ``slow entropy'' as a way to quantitatively compare measure-preserving systems with zero entropy. We develop a relative version of this theory for a measure-preserving system conditioned on a given factor. Our new definition inherits many desirable properties that make it a natural generalization of both the Katok--Thouvenot/Ferenczi theory and the classical conditional Kolmogorov--Sinai entropy. As an application, we prove a relative version of a result of Ferenczi that classifies isometric systems in terms of their slow entropy. We also introduce a new definition for the notion of a rigid extension and investigate its relationship to relative slow entropy.


title: Smart Hybrid Beamforming and Pilot Assignment for 6G Cell-Free Massive MIMO

abstract: This paper investigates Cell-Free massive MIMO networks, where each access point (AP) is equipped with a hybrid transceiver, reducing the complexity and cost compared to a fully digital transceiver. Asymptotic approximations for the spectral efficiency are derived for uplink and downlink. Capitalizing on these expressions, a max-min problem is formulated to optimize the (i) analog beamformer at the APs and (ii) pilot assignment. Simulations show that the optimization of these variables substantially increases the network performance.


title: Experimental characterization of photoemission from plasmonic nanogroove arrays

abstract: Metal photocathodes are an important source of high-brightness electron beams, ubiquitous in the operation of both large-scale accelerators and table-top microscopes. When the surface of a metal is nano-engineered with patterns on the order of the optical wavelength, it can lead to the excitation and confinement of surface plasmon polariton waves which drive nonlinear photoemission. In this work, we aim to evaluate gold plasmonic nanogrooves as a concept for producing bright electron beams for accelerators via nonlinear photoemission. We do this by first comparing their optical properties to numerical calculations from first principles to confirm our ability to fabricate these nanoscale structures. Their nonlinear photoemission yield is found by measuring emitted photocurrent as the intensity of their driving laser is varied. Finally, the mean transverse energy of this electron source is found using the solenoid scan technique. Our data demonstrate the ability of these cathodes to provide a tenfold enhancement in the efficiency of photoemission over flat metals driven with a linear process. We find that these cathodes are robust and capable of reaching sustained average currents over 100 nA at optical intensities larger than 2 GW/cm$^2$ with no degradation of performance. The emittance of the generated beam is found to be highly asymmetric, a fact we can explain with calculations involving the also asymmetric roughness of the patterned surface. These results demonstrate the use of nano-engineered surfaces as enhanced photocathodes, providing a robust, air-stable source of high average current electron beams with great potential for industrial and scientific applications.


title: Dark Energy Survey Year 3 Results: Measurement of the Baryon Acoustic Oscillations with Three-dimensional Clustering

abstract: The three-dimensional correlation function offers an effective way to summarize the correlation of the large-scale structure even for imaging galaxy surveys. We have applied the projected three-dimensional correlation function, $\xi_{\rm p}$ to measure the Baryonic Acoustic Oscillations (BAO) scale on the first-three years Dark Energy Survey data. The sample consists of about 7 million galaxies in the redshift range $ 0.6 < z_{\rm p } < 1.1 $ over a footprint of $4108 \, \mathrm{deg}^2 $. Our theory modeling includes the impact of realistic true redshift distributions beyond Gaussian photo-$z$ approximation. To increase the signal-to-noise of the measurements, a Gaussian stacking window function is adopted in place of the commonly used top-hat. Using the full sample, $ D_{\rm M}(z_{\rm eff} ) / r_{\rm s} $, the ratio between the comoving angular diameter distance and the sound horizon, is constrained to be $ 19.00 \pm 0.67 $ (top-hat) and $ 19.15 \pm 0.58 $ (Gaussian) at $z_{\rm eff} = 0.835$. The constraint is weaker than the angular correlation $w$ constraint ($18.84 \pm 0.50$) because the BAO signals are heterogeneous across redshift. When a homogeneous BAO-signal sub-sample in the range $ 0.7 < z_{\rm p } < 1.0 $ ($z_{\rm eff} = 0.845$) is considered, $\xi_{\rm p} $ yields $ 19.80 \pm 0.67 $ (top-hat) and $ 19.84 \pm 0.53 $ (Gaussian). The latter is mildly stronger than the $w$ constraint ($19.86 \pm 0.55 $). We find that the $\xi_{\rm p} $ results are more sensitive to photo-$z$ errors than $w$ because $\xi_{\rm p}$ keeps the three-dimensional clustering information causing it to be more prone to photo-$z$ noise. The Gaussian window gives more robust results than the top-hat as the former is designed to suppress the low signal modes. $\xi_{\rm p}$ and the angular statistics such as $w$ have their own pros and cons, and they serve an important crosscheck with each other.


title: Evolving Quantum Circuits

abstract: We develop genetic algorithms for searching quantum circuits, in particular stabilizer quantum error correction codes. Quantum codes equivalent to notable examples such as the 5-qubit perfect code, Shor's code, and the 7-qubit color code are evolved out of initially random quantum circuits. We anticipate evolution as a promising tool in the NISQ era, with applications such as the search for novel topological ordered states, quantum compiling, and hardware optimization.


title: Improving Robustness of Retrieval Augmented Translation via Shuffling of Suggestions

abstract: Several recent studies have reported dramatic performance improvements in neural machine translation (NMT) by augmenting translation at inference time with fuzzy-matches retrieved from a translation memory (TM). However, these studies all operate under the assumption that the TMs available at test time are highly relevant to the testset. We demonstrate that for existing retrieval augmented translation methods, using a TM with a domain mismatch to the test set can result in substantially worse performance compared to not using a TM at all. We propose a simple method to expose fuzzy-match NMT systems during training and show that it results in a system that is much more tolerant (regaining up to 5.8 BLEU) to inference with TMs with domain mismatch. Also, the model is still competitive to the baseline when fed with suggestions from relevant TMs.


title: AVE-CLIP: AudioCLIP-based Multi-window Temporal Transformer for Audio Visual Event Localization

abstract: An audio-visual event (AVE) is denoted by the correspondence of the visual and auditory signals in a video segment. Precise localization of the AVEs is very challenging since it demands effective multi-modal feature correspondence to ground the short and long range temporal interactions. Existing approaches struggle in capturing the different scales of multi-modal interaction due to ineffective multi-modal training strategies. To overcome this limitation, we introduce AVE-CLIP, a novel framework that integrates the AudioCLIP pre-trained on large-scale audio-visual data with a multi-window temporal transformer to effectively operate on different temporal scales of video frames. Our contributions are three-fold: (1) We introduce a multi-stage training framework to incorporate AudioCLIP pre-trained with audio-image pairs into the AVE localization task on video frames through contrastive fine-tuning, effective mean video feature extraction, and multi-scale training phases. (2) We propose a multi-domain attention mechanism that operates on both temporal and feature domains over varying timescales to fuse the local and global feature variations. (3) We introduce a temporal refining scheme with event-guided attention followed by a simple-yet-effective post processing step to handle significant variations of the background over diverse events. Our method achieves state-of-the-art performance on the publicly available AVE dataset with 5.9% mean accuracy improvement which proves its superiority over existing approaches.


title: InQMAD: Incremental Quantum Measurement Anomaly Detection

abstract: Streaming anomaly detection refers to the problem of detecting anomalous data samples in streams of data. This problem poses challenges that classical and deep anomaly detection methods are not designed to cope with, such as conceptual drift and continuous learning. State-of-the-art flow anomaly detection methods rely on fixed memory using hash functions or nearest neighbors that may not be able to constrain high-frequency values as in a moving average or remove seamless outliers and cannot be trained in an end-to-end deep learning architecture. We present a new incremental anomaly detection method that performs continuous density estimation based on random Fourier features and the mechanism of quantum measurements and density matrices that can be viewed as an exponential moving average density. It can process potentially endless data and its update complexity is constant $O(1)$. A systematic evaluation against 12 state-of-the-art streaming anomaly detection algorithms using 12 streaming datasets is presented.


title: Relational Attention: Generalizing Transformers for Graph-Structured Tasks

abstract: Transformers flexibly operate over sets of real-valued vectors representing task-specific entities and their attributes, where each vector might encode one word-piece token and its position in a sequence, or some piece of information that carries no position at all. But as set processors, transformers are at a disadvantage in reasoning over more general graph-structured data where nodes represent entities and edges represent relations between entities. To address this shortcoming, we generalize transformer attention to consider and update edge vectors in each transformer layer. We evaluate this relational transformer on a diverse array of graph-structured tasks, including the large and challenging CLRS Algorithmic Reasoning Benchmark. There, it dramatically outperforms state-of-the-art graph neural networks expressly designed to reason over graph-structured data. Our analysis demonstrates that these gains are attributable to relational attention's inherent ability to leverage the greater expressivity of graphs over sets.


title: Improving Dense Contrastive Learning with Dense Negative Pairs

abstract: Many contrastive representation learning methods learn a single global representation of an entire image. However, dense contrastive representation learning methods such as DenseCL (Wang et al., 2021) can learn better representations for tasks requiring stronger spatial localization of features, such as multi-label classification, detection, and segmentation. In this work, we study how to improve the quality of the representations learned by DenseCL by modifying the training scheme and objective function, and propose DenseCL++. We also conduct several ablation studies to better understand the effects of: (i) various techniques to form dense negative pairs among augmentations of different images, (ii) cross-view dense negative and positive pairs, and (iii) an auxiliary reconstruction task. Our results show 3.5% and 4% mAP improvement over SimCLR (Chen et al., 2020a) andDenseCL in COCO multi-label classification. In COCO and VOC segmentation tasks, we achieve 1.8% and 0.7% mIoU improvements over SimCLR, respectively.


title: VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangement

abstract: We present Variable Experience Rollout (VER), a technique for efficiently scaling batched on-policy reinforcement learning in heterogenous environments (where different environments take vastly different times to generate rollouts) to many GPUs residing on, potentially, many machines. VER combines the strengths of and blurs the line between synchronous and asynchronous on-policy RL methods (SyncOnRL and AsyncOnRL, respectively). VER learns from on-policy experience (like SyncOnRL) and has no synchronization points (like AsyncOnRL). VER leads to significant and consistent speed-ups across a broad range of embodied navigation and mobile manipulation tasks in photorealistic 3D simulation environments. Specifically, for PointGoal navigation and ObjectGoal navigation in Habitat 1.0, VER is 60-100% faster (1.6-2x speedup) than DD-PPO, the current state of art distributed SyncOnRL, with similar sample efficiency. For mobile manipulation tasks (open fridge/cabinet, pick/place objects) in Habitat 2.0 VER is 150% faster (2.5x speedup) on 1 GPU and 170% faster (2.7x speedup) on 8 GPUs than DD-PPO. Compared to SampleFactory (the current state-of-the-art AsyncOnRL), VER matches its speed on 1 GPU, and is 70% faster (1.7x speedup) on 8 GPUs with better sample efficiency. We leverage these speed-ups to train chained skills for GeometricGoal rearrangement tasks in the Home Assistant Benchmark (HAB). We find a surprising emergence of navigation in skills that do not ostensible require any navigation. Specifically, the Pick skill involves a robot picking an object from a table. During training the robot was always spawned close to the table and never needed to navigate. However, we find that if base movement is part of the action space, the robot learns to navigate then pick an object in new environments with 50% success, demonstrating surprisingly high out-of-distribution generalization.


title: Multiple Topological Magnetism in van der Waals Heterostructure of MnTe2/ZrS2

abstract: Topological magnetism in low-dimensional systems is of fundamental and practical importance in condensed-matter physics and material science. Here, using first-principles and Monte-Carlo simulations, we present that multiple topological magnetism (i.e., skyrmion and bimeron) can survive in van der Waals Heterostructure of MnTe2ZrS2. Arising from interlayer coupling, MnTe2ZrS2 can harbor a large Dzyaloshinskii-Moriya interaction. This, combined with ferromagnetic exchange interaction, yields an intriguing skyrmion phase consisting of sub-10 nm magnetic skyrmions under a tiny magnetic field of 75 mT. Meanwhile, upon harnessing a small electric field, magnetic bimeron can be observed in MnTe2ZrS2 as well, suggesting the existence of multiple topological magnetism. Through interlayer sliding, both topological spin textures can be switched on-off, suggesting their stacking-dependent character. In addition, the impacts of d and Keff on these spin textures are revealed, and a dimensionless parameter is utilized to describe their joint effect. These explored phenomena and insights not only are useful for fundamental research in topological magnetism, but also enable novel applications in nanodevices.


title: A Linearly Convergent Algorithm for Rotationally Invariant $\ell_1$-Norm Principal Component Analysis

abstract: To do dimensionality reduction on the datasets with outliers, the $\ell_1$-norm principal component analysis (L1-PCA) as a typical robust alternative of the conventional PCA has enjoyed great popularity over the past years. In this work, we consider a rotationally invariant L1-PCA, which is hardly studied in the literature. To tackle it, we propose a proximal alternating linearized minimization method with a nonlinear extrapolation for solving its two-block reformulation. Moreover, we show that the proposed method converges at least linearly to a limiting critical point of the reformulated problem. Such a point is proved to be a critical point of the original problem under a condition imposed on the step size. Finally, we conduct numerical experiments on both synthetic and real datasets to support our theoretical developments and demonstrate the efficacy of our approach.


title: Is Relativistic Hydrodynamics always Symmetric-Hyperbolic in the Linear Regime?

abstract: Close to equilibrium, the kinetic coefficients of a thermodynamic system must satisfy a set of symmetry conditions, which follow from the Onsager-Casimir principle. Here, we show that, if a system of hydrodynamic equations is analysed from the perspective of the Onsager-Casimir principle, then it is possible to impose very strong symmetry conditions also on the principal part of such equations (the part with highest derivatives). In particular, we find that, in the absence of macroscopic magnetic fields and spins, relativistic hydrodynamics should always be symmetric-hyperbolic, when linearised about equilibrium. We use these results to prove that Carter's multifluid theory and the Israel-Stewart theory in the pressure frame are both symmetric-hyperbolic in the linear regime. Connections with the GENERIC formalism are also explored.


title: In-Hand Gravitational Pivoting Using Tactile Sensing

abstract: We study gravitational pivoting, a constrained version of in-hand manipulation, where we aim to control the rotation of an object around the grip point of a parallel gripper. To achieve this, instead of controlling the gripper to avoid slip, we embrace slip to allow the object to rotate in-hand. We collect two real-world datasets, a static tracking dataset and a controller-in-the loop dataset, both annotated with object angle and angular velocity labels. Both datasets contain force-based tactile information on ten different household objects. We train an LSTM model to predict the angular position and velocity of the held object from purely tactile data. We integrate this model with a controller that opens and closes the gripper allowing the object to rotate to desired relative angles. We conduct real-world experiments where the robot is tasked to achieve a relative target angle. We show that our approach outperforms a sliding-window based MLP in a zero-shot generalization setting with unseen objects. Furthermore, we show a 16.6% improvement in performance when the LSTM model is fine-tuned on a small set of data collected with both the LSTM model and the controller in-the-loop. Code and videos are available at https://rhys-newbury.github.io/projects/pivoting/


title: Linear and Nonlinear Disturbance Evolution on the Frustum of Hypersonic Ogive-Cylinders

abstract: Aspects of transition mechanisms on a $14^{\circ}$ sharp-nosed ogive-cylinder at Mach~$6$ are elucidated by considering linear and nonlinear disturbance evolution of freestream stochastic and wave packet forcing at different locations downstream of the ogive-cylinder junction. The spectral response of the stochastic forcing displays favorable agreement with experimental observations from AFRL on which the configuration is based. Intermittent wave packets are generated from small-amplitude continuous freestream pressure forcing. Linear wave packet evolution reveals that Mack modes are the dominant primary instabilities, followed by relatively weaker first-mode waves. The non-linear wavepacket displays a three-legged wall pressure perturbation signature, which is traced to spanwise curvature effects, and fundamental resonance is the dominant secondary instability mechanism. At higher amplitudes of excitation, weak signatures of sub-harmonic and oblique resonance phenomena are identified. Downstream placement of the small-amplitude disturbance diminishes receptivity of first-mode waves; however, with high-amplitude forcing, the location of actuation does not significantly vary the disturbance evolution signature or the secondary instability mechanisms.


title: Realizing tight-binding Hamiltonians using site-controlled coupled cavity arrays

abstract: Analog quantum simulators rely on programmable quantum devices to emulate Hamiltonians describing various physical phenomenon. Photonic coupled cavity arrays are a promising platform for realizing such devices. Using a silicon photonic coupled cavity array made up of 8 high quality-factor resonators and equipped with specially designed thermo-optic island heaters for independent control of cavities, we demonstrate a programmable device implementing tight-binding Hamiltonians with access to the full eigen-energy spectrum. We report a ~50% reduction in the thermal crosstalk between neighboring sites of the cavity array compared to traditional heaters, and then present a control scheme to program the cavity array to a given tight-binding Hamiltonian.


title: Density Shift of Optical Lattice Clock via Multi-Bands Sampling Exact Diagonalization Method

abstract: Density shift plays one of the major roles in the uncertainty of optical lattice clock, thus has attracted lots of theoretical and experimental studies. However, most of the theoretical research considered the single-band and collective approximation, so the density shift of the system at higher temperatures can not be analyzed accurately. Here, we design a numerical algorithm that combines Monte Carlo sampling and exact diagonalization and name it as Multi-Band Sampling Exact Diagonalization (MBSED). The MBSED method considers the collision of atoms between multi-bands, so it can provide the density shift of an optical lattice clock with higher precision. Such an algorithm will benefit the numerical simulation of an optical lattice clock, and may also be used in other platforms of quantum metrology. In addition, through our numerical simulation, we also found that the density shift of Rabi spectrum is slightly non-linear with atom number.


title: Switching between topological edge states in plasmonic systems using phase-change materials

abstract: We introduce non-Hermitian plasmonic waveguide-cavity structures based on the Aubry-Andre-Harper model to realize switching between right and left topological edge states (TESs) using the phase-change material Ge$_2$Sb$_2$Te$_5$ (GST). We show that switching between the crystalline and amorphous phases of GST leads to a shift of the dispersion relation of the optimized structure so that a right TES for the crystalline phase, and a left TES for the amorphous phase occur at the same frequency. Thus, we realize switching between right and left TESs at that frequency by switching between the crystalline and amorphous phases of GST. Our results could be potentially important for developing compact reconfigurable topological photonic devices.


title: Self-supervised Model Based on Masked Autoencoders Advance CT Scans Classification

abstract: The coronavirus pandemic has been going on since the year 2019, and the trend is still not abating. Therefore, it is particularly important to classify medical CT scans to assist in medical diagnosis. At present, Supervised Deep Learning algorithms have made a great success in the classification task of medical CT scans, but medical image datasets often require professional image annotation, and many research datasets are not publicly available. To solve this problem, this paper is inspired by the self-supervised learning algorithm MAE and uses the MAE model pre-trained on ImageNet to perform transfer learning on CT Scans dataset. This method improves the generalization performance of the model and avoids the risk of overfitting on small datasets. Through extensive experiments on the COVID-CT dataset and the SARS-CoV-2 dataset, we compare the SSL-based method in this paper with other state-of-the-art supervised learning-based pretraining methods. Experimental results show that our method improves the generalization performance of the model more effectively and avoids the risk of overfitting on small datasets. The model achieved almost the same accuracy as supervised learning on both test datasets. Finally, ablation experiments aim to fully demonstrate the effectiveness of our method and how it works.


title: On Uniform Confidence Intervals for the Tail Index and the Extreme Quantile

abstract: This paper presents two results concerning uniform confidence intervals for the tail index and the extreme quantile. First, we show that it is impossible to construct a length-optimal confidence interval satisfying the correct uniform coverage over a local non-parametric family of tail distributions. Second, in light of the impossibility result, we construct honest confidence intervals that are uniformly valid by incorporating the worst-case bias in the local non-parametric family. The proposed method is applied to simulated data and a real data set of National Vital Statistics from National Center for Health Statistics.


title: Reflection of Thought: Inversely Eliciting Numerical Reasoning in Language Models via Solving Linear Systems

abstract: Numerical reasoning over natural language has been a long-standing goal for the research community. However, cutting-edge language models have proven difficult to reliably generalize to a broad range of numbers, although they have shown proficiency in reasoning over common and simple numbers. In this paper, we propose a novel method to elicit and exploit the numerical reasoning knowledge hidden in pre-trained language models using simple anchor numbers. Concretely, we first leverage simple numbers as anchors to probe the implicitly inferred arithmetic expressions from language models, and then explicitly apply the expressions on complex numbers to get corresponding answers. To inversely elicit arithmetic expressions, we transform and formulate the task as an analytically solvable linear system. Experimental results on several numerical reasoning benchmarks demonstrate that our approach significantly improves numerical reasoning capabilities of existing LMs. More importantly, our approach is training-free and simply works in the inference phase, making it highly portable and achieving consistent performance benefits across a variety of language models (GPT-3, T5, BART, etc) in all zero-shot, few-shot, and fine-tuning scenarios.


title: ConchShell: A Generative Adversarial Networks that Turns Pictures into Piano Music

abstract: We present ConchShell, a multi-modal generative adversarial framework that takes pictures as input to the network and generates piano music samples that match the picture context. Inspired by I3D, we introduce a novel image feature representation method: time-convolutional neural network (TCNN), which is used to forge features for images in the temporal dimension. Although our image data consists of only six categories, our proposed framework will be innovative and commercially meaningful. The project will provide technical ideas for work such as 3D game voice overs, short-video soundtracks, and real-time generation of metaverse background music.We have also released a new dataset, the Beach-Ocean-Piano Dataset (BOPD) 1, which contains more than 3,000 images and more than 1,500 piano pieces. This dataset will support multimodal image-to-music research.


title: Effects of charge doping on Mott insulator with strong spin-orbit coupling, Ba$_2$NaOsO$_6$

abstract: The effects of doping on the electronic evolution of the Mott insulating state have been extensively studied in efforts to understand mechanisms of emergent quantum phases of materials. The study of these effects becomes ever more intriguing in the presence of entanglement between spin and orbital degrees of freedom. Here, we present a comprehensive investigation of charge doping in the double perovskite Ba$_2$NaOsO$_6$, a a complex Mott insulator where such entanglement plays an important role. We establish that the insulating magnetic ground state evolves from canted antiferromagnet (cAF)to N\'eel order for dopant levels exceeding ~ 10 %. Furthermore, we determine that a broken local point symmetry (BLPS) phase, precursor to the magnetically ordered state, occupies an extended portion of the (H-T) phase diagram with increased doping. This finding reveals that the breaking of the local cubic symmetry is driven by a multipolar order, most-likely of the antiferro-quadrupolar type.


title: Joint Human Orientation-Activity Recognition Using WiFi Signals for Human-Machine Interaction

abstract: WiFi sensing is an important part of the new WiFi 802.11bf standard, which can detect motion and measure distances. In recent years, some machine learning methods have been proposed for human activity recognition from WiFi signals. However, to the best of our knowledge, none of these methods have explored orientation prediction of the user using WiFi signals. Orientation prediction is particularly critical for human-machine interaction in an environment with multiple smart devices. In this paper, we propose a data collection setup and machine learning models for joint human orientation and activity recognition using WiFi signals from a single access point (AP) or multiple APs. The results show feasibility of joint orientation-activity recognition in an indoor environment with a high accuracy.


title: Predicting successful clinical candidates for fiducial-free lung tumor tracking with a deep learning binary classification model

abstract: The CyberKnife system is a robotic radiosurgery platform that allows the delivery of lung SBRT treatments using fiducial-free soft-tissue tracking. However, not all lung cancer patients are eligible for lung tumor tracking. Tumor size, density and location impact the ability to successfully detect and track a lung lesion in 2D orthogonal X-ray images. The standard workflow to identify successful candidates for lung tumor tracking is called Lung Optimized Treatment (LOT) simulation, and involves multiple steps from CT acquisition to the execution of the simulation plan on CyberKnife. The aim of the study is to develop a deep learning classification model to predict which patients can be successfully treated with lung tumor tracking, thus circumventing the LOT simulation process. Target tracking is achieved by matching orthogonal x-ray images with a library of digital radiographs reconstructed from the simulation CT scan (DRRs). We developed a deep learning model to create a binary classification of lung lesions as being trackable or untrackable based on tumor template DRR extracted from the CyberKnife system, and tested five different network architectures. The study included a total of 271 images (230 trackable, 41 untrackable) from 129 patients with one or multiple lung lesions. 80% of the images were used for training, 10% for validation, and the remaining 10% for testing. For all 5 convolutional neural networks, the binary classification accuracy reached 100% after training, both in the validation and the test set, without any false classifications. A deep learning model can distinguish features of trackable and untrackable lesions in DRR images, and can predict successful candidates for fiducial-free lung tumor tracking.


title: How to construct the symmetric cycle of length 5 using Haj\'os construction with an adapted Rank Genetic Algorithm

abstract: In 2020 Bang-Jensen et. al. generalized the Haj\'os join of two graphs to the class of digraphs and generalized several results for vertex colorings in digraphs. Although, as a consequence of these results, a digraph can be obtained by Haj\'os constructions (directed Haj\'os join and identifying non-adjacent vertices), determining the Haj\'os constructions to obtain the digraph is a complex problem. In particular, Bang-Jensen et al. posed the problem of determining the Haj\'os operations to construct the symmetric 5-cycle from the complete symmetric digraph of order 3 using only Haj\'os constructions. We successfully adapted a rank-based genetic algorithm to solve this problem by the introduction of innovative recombination and mutation operators from graph theory. The Haj\'os Join became the recombination operator and the identification of independent vertices became the mutation operator. In this way, we were able to obtain a sequence of only 16 Haj\'os operations to construct the symmetric cycle of order 5.


title: Specifics of the Elemental Excitations in "True One-Dimensional" MoI$_3$ van der Waals Nanowires

abstract: We report on the temperature evolution of the polarization-dependent Raman spectrum of exfoliated MoI$_3$, a van der Waals material with a "true one-dimensional" crystal structure that can be exfoliated to individual atomic chains. The temperature evolution of several Raman features reveals anomalous behavior suggesting a phase transition of a magnetic origin. Theoretical considerations indicate that MoI$_3$ is an easy-plane antiferromagnet with alternating spins along the dimerized chains and with inter-chain helical spin ordering. The calculated frequencies of the phonons and magnons are consistent with the interpretation of the experimental Raman data. The obtained results shed light on the specifics of the phononic and magnonic states in MoI$_3$ and provide a strong motivation for future study of this unique material with potential for spintronic device applications.


title: Convergence of Type IIA manifolds and application to the Type IIA flow

abstract: The Type IIA flow is a flow of 3-forms on a 6-dimensional symplectic manifold. It was introduced by Fei, Phong, Picard, and Zhang in \cite{FPPZb}. There is little known about the singularities of this flow in general. We formulate and prove convergence theorems for this flow. We also describe the singularity models for this flow.


title: Convergence of Bi-Virus Epidemic Models with Non-Linear Rates on Networks -- A Monotone Dynamical Systems Approach

abstract: We study convergence properties of competing epidemic models of the Susceptible-Infected-Susceptible (SIS) type. The SIS epidemic model has seen widespread popularity in modelling the spreading dynamics of contagions such as viruses, infectious diseases, or even rumors/opinions over contact networks (graphs).We analyze the case of two such viruses spreading on overlaid graphs, with non-linear rates of infection spread and recovery. We call this the non-linear bi-virus model and, building upon recent results, obtain precise conditions for global convergence of the solutions to a trichotomy of possible outcomes: a virus-free state, a single-virus state, and to a coexistence state. Our techniques are based on the theory of monotone dynamical systems (MDS), in contrast to Lyapunov based techniques that have only seen partial success in determining convergence properties in the setting of competing epidemics. We demonstrate how the existing works have been unsuccessful in characterizing a large subset of the model parameter space for bi-virus epidemics, including all scenarios leading to coexistence of the epidemics. To the best of our knowledge, our results are the first in providing complete convergence analysis for the bi-virus system with nonlinear infection and recovery rates on general graphs.


title: Covert Communication Gains from Adversary's Uncertainty of Phase Angles

abstract: This work investigates the phase gain of intelligent reflecting surface (IRS) covert communication over complex-valued additive white Gaussian noise (AWGN) channels. The transmitter Alice intends to transmit covert messages to the legitimate receiver Bob via reflecting the broadcast signals from a radio frequency (RF) source, while rendering the adversary Willie's detector arbitrarily close to ineffective. Our analyses show that, compared to the covert capacity for classical AWGN channels, we can achieve a covertness gain of value 2 by leveraging Willie's uncertainty of phase angles. This covertness gain is achieved when the number of possible phase angle pairs $N=2$. More interestingly, our results show that the covertness gain will not further increase with $N$ as long as $N \ge 2$, even if it approaches infinity.


title: NGC 6153: Reality is complicated

abstract: We study the kinematics of emission lines that arise from many physical processes in NGC 6153 based upon deep, spatially-resolved, high resolution spectra acquired with the UVES spectrograph at the ESO VLT. Our most basic finding is that the plasma in NGC 6153 is complex, especially its temperature structure. The kinematics of most emission lines defines a classic expansion law, with the outer part expanding fastest (normal nebular plasma). However, the permitted lines of \ion{O}{1}, \ion{C}{2}, \ion{N}{2}, \ion{O}{2}, and \ion{Ne}{2} present a constant expansion velocity that defines a second kinematic component (additional plasma component). The physical conditions imply two plasma components, with the additional plasma component having lower temperature and higher density. The [\ion{O}{2}] density and the [\ion{N}{2}] temperature are anomalous, but may be understood considering the contribution of recombination to these forbidden lines. The two plasma components have very different temperatures. The normal nebular plasma appears to be have temperature fluctuations in part of its volume (main shell), but only small fluctuations elsewhere. The additional plasma component contains about half of the mass of the N$^{2+}$ and O$^{2+}$ ions, but only $3-5$\% of the mass of H$^+$ ions, so the two plasma components have very different chemical abundances. We estimate abundances of $12+\log(\mathrm O^{2+}/\mathrm H^+)\sim 9.2$\,dex and $\mathrm{He}/\mathrm H\sim 0.13$. Although they are all complications, multiple plasma components, temperature fluctuations, and the contributions of multiple physical processes to a given emission line are all part of the reality in NGC 6153, and should generally be taken into account.


title: Cost-effectiveness analysis for therapy sequence in advanced cancer: A microsimulation approach with application to metastatic prostate cancer

abstract: Purpose. Patients with advanced cancer may undergo multiple lines of treatment, switching therapies as their disease progresses. Motivated by a study of metastatic prostate cancer, we develop a microsimulation framework to study therapy sequence. Methods. We propose a discrete-time state transition model to study two lines of anti-cancer therapy. Based on digitized published progression-free survival (PFS) and overall survival (OS) curves, we infer event types (progression or death), and estimate transition probabilities using cumulative incidence functions with competing risks. Our model incorporates within-patient dependence over time, such that response to first-line therapy informs subsequent event probabilities. Parameters governing the degree of within-patient dependence can be used to calibrate the model-based results to those of a target trial. We demonstrate these methods in a study of two therapy sequences for metastatic prostate cancer, where Docetaxel (DCT) and Abiraterone Acetate (AA) are both appropriate for use in either first or second line treatment. We assess costs, Quality-Adjusted Life Years (QALYs) and Incremental Cost Effectiveness Ratio (ICER) for two treatment strategies: DCT then AA vs AA then DCT. Results. Using digitized survival curves from relevant clinical trials, we identified 8.6-13.9% of PFS times that should be categorized as deaths, allowing for estimation of cumulative incidence functions. Models assuming within-patient independence overestimated OS time, corrected with our calibration approach. Correction resulted in meaningful changes in the difference in QALYs between treatment strategies (0.07 vs 0.15) and the ICER (-\$76,836/QALY vs -\$21,030/QALY). Conclusions. Microsimulation models can be successfully used to study cost-effectiveness of therapy sequences, taking care to account correctly for within-patient dependence.


title: Approximation of nearly-periodic symplectic maps via structure-preserving neural networks

abstract: A continuous-time dynamical system with parameter $\varepsilon$ is nearly-periodic if all its trajectories are periodic with nowhere-vanishing angular frequency as $\varepsilon$ approaches 0. Nearly-periodic maps are discrete-time analogues of nearly-periodic systems, defined as parameter-dependent diffeomorphisms that limit to rotations along a circle action, and they admit formal $U(1)$ symmetries to all orders when the limiting rotation is non-resonant. For Hamiltonian nearly-periodic maps on exact presymplectic manifolds, the formal $U(1)$ symmetry gives rise to a discrete-time adiabatic invariant. In this paper, we construct a novel structure-preserving neural network to approximate nearly-periodic symplectic maps. This neural network architecture, which we call symplectic gyroceptron, ensures that the resulting surrogate map is nearly-periodic and symplectic, and that it gives rise to a discrete-time adiabatic invariant and a long-time stability. This new structure-preserving neural network provides a promising architecture for surrogate modeling of non-dissipative dynamical systems that automatically steps over short timescales without introducing spurious instabilities.


title: Criterions of Phase Transitions in Dispersed Multiphase Systems Based on an Extended Lattice Model

abstract: Agglomeration, adsorption, and extraction in dispersed multiphase systems are ubiquitously encountered in biological systems, energy industry, and medical science. In this work, a novel lattice model is extended to the three-component complex systems and criterions based on the determinant \Delta_i=F_i-K_c,i are accordingly proposed to predict the aforementioned behaviors based on a Helmholtz free energy formulation. Here, three characteristic factors F_i's are introduced to describe the internal energy effect, i.e.,F_1=A^11+A^22-2A^12 (agglomeration), F_2=A^22+A^13-A^12-A^23 (adsorption) and F_3=A^22+2A^13-A^33-2A^12 (extraction), where A_ij denotes the conservative potential coefficient between liquid particles in phase i and j, while the entropy factors K_c,i's in the determinants depend on the local structure of the liquid. To verify the theoretical criterions, many-body dissipative particle dynamics (mDPD) are employed to simulate the phase transition phenomena in various many-body dissipative systems, including agglomeration in a binary system, adsorption on solid surfaces from the liquid phase, and extraction-adsorption around an immiscible liquid-liquid interface. The simulation results show the notable dependence of the dispersed phase distribution transition on the quasi-order parameters \Delta_i's, which indicates the great potential of our model to analyzing various dispersed multiphase systems. The criterions are expected to be extended to study the structure effect induced by the temperature and dispersed particle shape on the phase transition phenomena in those complex systems.


title: Measurement-based quantum computation in finite one-dimensional systems: string order implies computational power

abstract: We present a new framework for assessing the power of measurement-based quantum computation (MBQC) on short-range entangled symmetric resource states, in spatial dimension one. It requires fewer assumptions than previously known. The formalism can handle finitely extended systems (as opposed to the thermodynamic limit), and does not require translation-invariance. Further, we strengthen the connection between MBQC computational power and string order. Namely, we establish that whenever a suitable set of string order parameters is non-zero, a corresponding set of unitary gates can be realized with fidelity arbitrarily close to unity.


title: New ASKAP Radio Supernova Remnants and Candidates in the Large Magellanic Cloud

abstract: We present a new Australian Square Kilometre Array Pathfinder (ASKAP) sample of 14 radio Supernova Remnant (SNR) candidates in the Large Magellanic Cloud (LMC). This new sample is a significant increase to the known number of older, larger and low surface brightness LMC SNRs. We employ a multi-frequency search for each object and found possible traces of optical and occasionally X-ray emission in several of these 14 SNR candidates. One of these 14 SNR candidates (MCSNR J0522-6543) has multi-frequency properties that strongly indicate a bona fide SNR. We also investigate a sample of 20 previously suggested LMC SNR candidates and confirm the SNR nature of MCSNR J0506-6815. We detect lower surface brightness SNR candidates which were likely formed by a combination of shock waves and strong stellar winds from massive progenitors (and possibly surrounding OB stars). Some of our new SNR candidates are also found in a lower density environments in which SNe type Ia explode inside a previously excavated interstellar medium (ISM).


title: A Copula-Based family of Bivariate Composite Models for Claim Severity Modelling

abstract: In this paper, we consider bivariate composite models for modeling jointly different types of claims and their associated costs in a flexible manner. For expository purposes, the Gumbel copula is paired with the composite Weibull-Inverse Weibull, Paralogistic-Inverse Weibull, and Inverse Burr-Inverse Weibull marginal models. The resulting bivariate copula-based composite models are fitted on motor insurance bodily injury and property damage data from a European motor insurance company and their parameters are estimated via the inference functions for margins method.


title: The DKU-Tencent System for the VoxCeleb Speaker Recognition Challenge 2022

abstract: This paper is the system description of the DKU-Tencent System for the VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC22). In this challenge, we focus on track1 and track3. For track1, multiple backbone networks are adopted to extract frame-level features. Since track1 focus on the cross-age scenarios, we adopt the cross-age trials and perform QMF to calibrate score. The magnitude-based quality measures achieve a large improvement. For track3, the semi-supervised domain adaptation task, the pseudo label method is adopted to make domain adaptation. Considering the noise labels in clustering, the ArcFace is replaced by Sub-center ArcFace. The final submission achieves 0.107 mDCF in task1 and 7.135% EER in task3.


title: Crack Modeling via Minimum-Weight Surfaces in 3d Voronoi Diagrams

abstract: Shortest paths play an important role in mathematical modeling and image processing. Usually, shortest path problems are formulated on planar graphs that consist of vertices and weighted arcs. In this context, one is interested in finding a path of minimum weight from a start vertex to an end vertex. The concept of minimum-weight surfaces extends shortest paths to 3d. The minimum-weight surface problem is formulated on a cellular complex with weighted facets. A cycle on the arcs of the complex serves as input and one is interested in finding a surface of minimum weight bounded by that cycle. In practice, minimum-weight surfaces can be used to segment 3d images. Vice versa, it is possible to use them as a modeling tool for geometric structures such as cracks. In this work, we present an approach for using minimum-weight surfaces in bounded Voronoi diagrams to generate synthetic 3d images of cracks.


title: Climate of high obliquity exo-terrestrial planets with a three-dimensional cloud system resolving climate model

abstract: Planetary climates are strongly affected by planetary orbital parameters such as obliquity, eccentricity, and precession. In exoplanetary systems, exo-terrestrial planets should have various obliquities. High-obliquity planets would have extreme seasonal cycles due to the seasonal change of the distribution of the insolation. Here, we introduce the Non-hydrostatic ICosahedral Atmospheric Model(NICAM), a global cloud-resolving model, to investigate the climate of high-obliquity planets. This model can explicitly simulate a three-dimensional cloud distribution and vertical transports of water vapor. We simulated exo-terrestrial climates with high resolution using the supercomputer FUGAKU. We assumed aqua-planet configurations with 1 bar of air as a background atmosphere, with four different obliquities ($0^{\circ}$, $23.5^{\circ}$, $45^{\circ}$, and $60^{\circ}$). We ran two sets of simulations: 1) low-resolution (~ 220 km-mesh as the standard resolution of a general circulation model for exoplanetary science) with parametrization for cloud formation, and 2) high-resolution (~ 14 km-mesh) with an explicit cloud microphysics scheme. Results suggest that high-resolution simulations with an explicit treatment of cloud microphysics reveal warmer climates due to less low cloud fraction and a large amount of water vapor in the atmosphere. It implies that treatments of cloud-related processes lead to a difference between different resolutions in climatic regimes in cases with high obliquities.


title: Digital Low-Level RF control system for Accumulator Ring at Advanced Light Source Upgrade Project

abstract: Currently ALS is undergoing an upgrade to ALSU to produce 100 times brighter soft X-ray light. The LLRF system for Accumulator Ring (AR) is composed of two identical LLRF stations, for driving RF amplifiers. The closed loop RF amplitude and phase stability is measured as $< 0.1\%$ and $< 0.1^\circ$ respectively, using the non-IQ digital down conversion together with analog up/down conversion, under a system-on-chip architecture. Realtime interlock system is implemented with $< 2 \mu$s latency, for machine protection against arc flash and unexpected RF power. Control interfaces are developed to enable PLC-FPGA-EPICS communication to support operation, timing, cavity tuning, and interlock systems. The LLRF system handles alignment of buckets to swap beams between AR and Storage Ring by synchronous phase loop ramping between the two cavities. The system also includes an optimization routine to characterize the loop dynamics and determine optimal operating point using a built-in network analyzer feature. A cavity emulator of 31 kHz bandwidth is integrated with the LLRF system to validate the performance of the overall system being developed.


title: Checks and Strategies for Enabling Code-Switched Machine Translation

abstract: Code-switching is a common phenomenon among multilingual speakers, where alternation between two or more languages occurs within the context of a single conversation. While multilingual humans can seamlessly switch back and forth between languages, multilingual neural machine translation (NMT) models are not robust to such sudden changes in input. This work explores multilingual NMT models' ability to handle code-switched text. First, we propose checks to measure switching capability. Second, we investigate simple and effective data augmentation methods that can enhance an NMT model's ability to support code-switching. Finally, by using a glass-box analysis of attention modules, we demonstrate the effectiveness of these methods in improving robustness.


title: Repainting and Imitating Learning for Lane Detection

abstract: Current lane detection methods are struggling with the invisibility lane issue caused by heavy shadows, severe road mark degradation, and serious vehicle occlusion. As a result, discriminative lane features can be barely learned by the network despite elaborate designs due to the inherent invisibility of lanes in the wild. In this paper, we target at finding an enhanced feature space where the lane features are distinctive while maintaining a similar distribution of lanes in the wild. To achieve this, we propose a novel Repainting and Imitating Learning (RIL) framework containing a pair of teacher and student without any extra data or extra laborious labeling. Specifically, in the repainting step, an enhanced ideal virtual lane dataset is built in which only the lane regions are repainted while non-lane regions are kept unchanged, maintaining the similar distribution of lanes in the wild. The teacher model learns enhanced discriminative representation based on the virtual data and serves as the guidance for a student model to imitate. In the imitating learning step, through the scale-fusing distillation module, the student network is encouraged to generate features that mimic the teacher model both on the same scale and cross scales. Furthermore, the coupled adversarial module builds the bridge to connect not only teacher and student models but also virtual and real data, adjusting the imitating learning process dynamically. Note that our method introduces no extra time cost during inference and can be plug-and-play in various cutting-edge lane detection networks. Experimental results prove the effectiveness of the RIL framework both on CULane and TuSimple for four modern lane detection methods. The code and model will be available soon.


title: IsoVec: Controlling the Relative Isomorphism of Word Embedding Spaces

abstract: The ability to extract high-quality translation dictionaries from monolingual word embedding spaces depends critically on the geometric similarity of the spaces -- their degree of "isomorphism." We address the root-cause of faulty cross-lingual mapping: that word embedding training resulted in the underlying spaces being non-isomorphic. We incorporate global measures of isomorphism directly into the Skip-gram loss function, successfully increasing the relative isomorphism of trained word embedding spaces and improving their ability to be mapped to a shared cross-lingual space. The result is improved bilingual lexicon induction in general data conditions, under domain mismatch, and with training algorithm dissimilarities. We release IsoVec at https://github.com/kellymarchisio/isovec.


title: New Gd-based magnetic compound GdPt$_2$B with a chiral crystal structure

abstract: Herein, we report the discovery of a novel Gd-based magnetic compound GdPt$_2$B with a chiral crystal structure. X-ray diffraction and chemical composition analyses reveal a CePt$_2$B-type crystal structure (space group: $P6_422$) for GdPt$_2$B. Moreover, we successfully grew single crystals of GdPt$_2$B using the Czochralski method. Magnetization measurements and the Curie$-$Weiss analysis demonstrate that the ferromagnetic interaction is dominant in GdPt$_2$B. A clear transition is observed in the temperature dependence of electrical resistivity, magnetic susceptibility, and specific heat at $T_{\rm O}$ = 87 K. The magnetic phase diagram of GdPt$_2$B, which consists of a field-polarized ferromagnetic region and a magnetically ordered region, resembles those of known chiral helimagnets. Furthermore, magnetic susceptibility measurements reveal a possible spin reorientation within the magnetically ordered phase in magnetic fields perpendicular to the screw axis. The results demonstrate that GdPt$_2$B is a suitable platform for investigating the competing effects of ferromagnetic and antisymmetric exchange interactions in rare-earth-based chiral compounds.


title: Optimization of RF phase and beam loading distribution among RF stations in SuperKEKB

abstract: SuperKEKB is the e-/e+ collider which targets the world highest luminosity. In recent operation, SuperKEKB achieved a new world record of $4.71\times 10^{34} cm^{-2}s^{-1}$ for luminosity with the beam current of 1.4 A. In the future, the beam current will be increased further to aim at the design value of 3.6 A and much higher luminosity. The RF system consists of 38 cavities (30 klystron stations), which share the huge beam loading brought by high current beam with each other cavities. For beam stability and power efficiency, it is important to distribute the beam loading properly among RF cavities. It is equivalent to adjust the acceleration phase of each cavity. However, it is difficult to evaluate the acceleration phase using only the pickup signal. Therefore, we established a method to evaluate the beam loading balance among RF stations from the RF power measurement for each cavity, and to adjust the acceleration phase. This report introduces the method for evaluating and optimizing the beam loading (acceleration phase) among stations in SuperKEKB, which has a large number of RF stations, and its operation.


title: Anticipating the DART impact: Orbit estimation of Dimorphos using a simplified model

abstract: We used the times of occultations and eclipses between the components of the 65803 Didymos binary system observed in its lightcurves from 2003-2021 to estimate the orbital parameters of Dimorphos relative to Didymos. We employed a weighted least-squares approach and a modified Keplerian orbit model in order to accommodate the effects from non-gravitational forces such as Binary YORP that could cause a linear change in mean motion over time. We estimate that the period of the mutual orbit at the epoch 2022 September 26.0 TDB, the day of the DART impact, is $11.9214869 \pm 0.000028$~h ($1\sigma$) and that the mean motion of the orbit is changing at a rate of $(5.0\pm 1.0)\times 10^{-18}$~rad s$^{-2}$ $(1\sigma$). The formal $3\sigma$ uncertainty in orbital phase of Dimorphos during the planned Double Asteroid Redirection Test (DART) mission is $5.4^\circ$. Observations from July to September 2022, a few months to days prior to the DART impact, should provide modest improvements to the orbital phase uncertainty and reduce it to about $4.2^\circ$. These results, generated using a relatively simple model, are consistent with those generated using the more sophisticated model of \citet{scheirich22}, which demonstrates the reliability of our method and adds confidence to these mission-critical results.


title: Pre-Training Representations of Binary Code Using Contrastive Learning

abstract: Compiled software is delivered as executable binary code. Developers write source code to express the software semantics, but the compiler converts it to a binary format that the CPU can directly execute. Therefore, binary code analysis is critical to applications in reverse engineering and computer security tasks where source code is not available. However, unlike source code and natural language that contain rich semantic information, binary code is typically difficult for human engineers to understand and analyze. While existing work uses AI models to assist source code analysis, few studies have considered binary code. In this paper, we propose a COntrastive learning Model for Binary cOde Analysis, or COMBO, that incorporates source code and comment information into binary code during representation learning. Specifically, we present three components in COMBO: (1) a primary contrastive learning method for cold-start pre-training, (2) a simplex interpolation method to incorporate source code, comments, and binary code, and (3) an intermediate representation learning algorithm to provide binary code embeddings. Finally, we evaluate the effectiveness of the pre-trained representations produced by COMBO using three indicative downstream tasks relating to binary code: algorithmic functionality classification, binary code similarity, and vulnerability detection. Our experimental results show that COMBO facilitates representation learning of binary code visualized by distribution analysis, and improves the performance on all three downstream tasks by 5.45% on average compared to state-of-the-art large-scale language representation models. To the best of our knowledge, COMBO is the first language representation model that incorporates source code, binary code, and comments into contrastive code representation learning and unifies multiple tasks for binary code analysis.


title: Leveraging Artificial Intelligence on Binary Code Comprehension

abstract: Understanding binary code is an essential but complex software engineering task for reverse engineering, malware analysis, and compiler optimization. Unlike source code, binary code has limited semantic information, which makes it challenging for human comprehension. At the same time, compiling source to binary code, or transpiling among different programming languages (PLs) can provide a way to introduce external knowledge into binary comprehension. We propose to develop Artificial Intelligence (AI) models that aid human comprehension of binary code. Specifically, we propose to incorporate domain knowledge from large corpora of source code (e.g., variable names, comments) to build AI models that capture a generalizable representation of binary code. Lastly, we will investigate metrics to assess the performance of models that apply to binary code by using human studies of comprehension.


title: 3D Matting: A Benchmark Study on Soft Segmentation Method for Pulmonary Nodules Applied in Computed Tomography

abstract: Usually, lesions are not isolated but are associated with the surrounding tissues. For example, the growth of a tumour can depend on or infiltrate into the surrounding tissues. Due to the pathological nature of the lesions, it is challenging to distinguish their boundaries in medical imaging. However, these uncertain regions may contain diagnostic information. Therefore, the simple binarization of lesions by traditional binary segmentation can result in the loss of diagnostic information. In this work, we introduce the image matting into the 3D scenes and use the alpha matte, i.e., a soft mask, to describe lesions in a 3D medical image. The traditional soft mask acted as a training trick to compensate for the easily mislabelled or under-labelled ambiguous regions. In contrast, 3D matting uses soft segmentation to characterize the uncertain regions more finely, which means that it retains more structural information for subsequent diagnosis and treatment. The current study of image matting methods in 3D is limited. To address this issue, we conduct a comprehensive study of 3D matting, including both traditional and deep-learning-based methods. We adapt four state-of-the-art 2D image matting algorithms to 3D scenes and further customize the methods for CT images to calibrate the alpha matte with the radiodensity. Moreover, we propose the first end-to-end deep 3D matting network and implement a solid 3D medical image matting benchmark. Its efficient counterparts are also proposed to achieve a good performance-computation balance. Furthermore, there is no high-quality annotated dataset related to 3D matting, slowing down the development of data-driven deep-learning-based methods. To address this issue, we construct the first 3D medical matting dataset. The validity of the dataset was verified through clinicians' assessments and downstream experiments.


title: Distributed-Memory Randomized Algorithms for Sparse Tensor CP Decomposition

abstract: Candecomp / PARAFAC (CP) decomposition, a generalization of the matrix singular value decomposition to higher-dimensional tensors, is a popular tool for analyzing multidimensional sparse data. On tensors with billions of nonzero entries, computing a CP decomposition is a computationally intensive task. We propose the first distributed-memory implementations of two randomized CP decomposition algorithms, CP-ARLS-LEV and STS-CP, that offer nearly an order-of-magnitude speedup at high decomposition ranks over well-tuned non-randomized decomposition packages. Both algorithms rely on leverage score sampling and enjoy strong theoretical guarantees, each with varying time and accuracy tradeoffs. We tailor the communication schedule for our random sampling algorithms, eliminating expensive reduction collectives and forcing communication costs to scale with the random sample count. Finally, we optimize the local storage format for our methods, switching between analogues of compressed sparse column and compressed sparse row formats. Experiments show that our methods are fast and scalable, producing 11x speedup over SPLATT by decomposing the billion-scale Reddit tensor on 512 CPU cores in under two minutes.


title: Photoproduction of $e^{+}e^{-}$ in peripheral isobar collisions

abstract: We investigate the photoproduction of di-electrons in peripheral collisions of $_{44}^{96}\textrm{Ru}+_{44}^{96}\textrm{Ru}$ and $_{40}^{96}\textrm{Zr}+_{40}^{96}\textrm{Zr}$ at 200 GeV. With the charge and mass density distributions given by the calculation of the density functional theory, we calculate the spectra of transverse momentum, invariant mass and azimuthal angle for di-electrons at 40-80\% centrality. The ratios of these spectra in Ru+Ru collisions over to Zr+Zr collisions are shown to be smaller than $(44/40)^{4}$ (the ratio of $Z^{4}$ for Ru and Zr) at low transverse momentum. The deviation arises from the different mass and charge density distributions in Ru and Zr. So the photoproduction of di-leptons in isobar collisions may provide a new way to probe the nuclear structure.


title: A family of non-Volterra quadratic operators corresponding to permutations

abstract: In the present paper we consider a family of non-Volterra quadratic stochastic operators depending on a parameter $\alpha$ and study their trajectory behaviors. We find all fixed points for a non-Volterra quadratic stochastic operator on a finite-dimensional simplex. We construct some Lyapunov functions. A complete description of the set of limit points is given, and we show that such operators have the ergodic property.


title: Functional Constrained Optimization for Risk Aversion and Sparsity Control

abstract: Risk and sparsity requirements often need to be enforced simultaneously in many applications, e.g., in portfolio optimization, assortment planning, and treatment planning. Properly balancing these potentially conflicting requirements entails the formulation of functional constrained optimization with either convex or nonconvex objectives. In this paper, we focus on projection-free methods that can generate a sparse trajectory for solving these challenging functional constrained optimization problems. Specifically, for the convex setting, we propose a Level Conditional Gradient (LCG) method, which leverages a level-set framework to update the approximation of the optimal value and an inner conditional gradient oracle (CGO) for solving mini-max subproblems. We show that the method achieves $\mathcal{O}\big(\frac{1}{\epsilon^2}\log\frac{1}{\epsilon}\big)$ iteration complexity for solving both smooth and nonsmooth cases without dependency on a possibly large size of optimal dual Lagrange multiplier. For the nonconvex setting, we introduce the Level Inexact Proximal Point (IPP-LCG) method and the Direct Nonconvex Conditional Gradient (DNCG) method. The first approach taps into the advantage of LCG by transforming the problem into a series of convex subproblems and exhibits an $\mathcal{O}\big(\frac{1}{\epsilon^3}\log\frac{1}{\epsilon}\big)$ iteration complexity for finding an ($\epsilon,\epsilon$)-KKT point. The DNCG is the first single-loop projection-free method, with iteration complexity bounded by $\mathcal{O}\big(1/\epsilon^4\big)$ for computing a so-called $\epsilon$-Wolfe point. We demonstrate the effectiveness of LCG, IPP-LCG and DNCG by devising formulations and conducting numerical experiments on two risk averse sparse optimization applications: a portfolio selection problem with and without cardinality requirement, and a radiation therapy planning problem in healthcare.


title: BanglaParaphrase: A High-Quality Bangla Paraphrase Dataset

abstract: In this work, we present BanglaParaphrase, a high-quality synthetic Bangla Paraphrase dataset curated by a novel filtering pipeline. We aim to take a step towards alleviating the low resource status of the Bangla language in the NLP domain through the introduction of BanglaParaphrase, which ensures quality by preserving both semantics and diversity, making it particularly useful to enhance other Bangla datasets. We show a detailed comparative analysis between our dataset and models trained on it with other existing works to establish the viability of our synthetic paraphrase data generation pipeline. We are making the dataset and models publicly available at https://github.com/csebuetnlp/banglaparaphrase to further the state of Bangla NLP.


title: Polar Metals Taxonomy for Materials Classification and Discovery

abstract: Over the past decade, materials that combine broken inversion symmetry with metallic conductivity have gone from a thought experiment to one of the fastest growing research topics. In 2013, the observation of the first uncontested polar transition in a metal, LiOsO$_3$, inspired a surge of theoretical and experimental work on the subject, uncovering a host of materials which combine properties previously thought to be contraindicated [Nat. Mater. \textbf{12}, 1024 (2013)]. As is often the case in a nascent field, the sudden rise in interest has been accompanied by diverse (and sometimes conflicting) terminology. Although ``ferroelectric-like" metals are well-defined in theory, i.e., materials that undergo a symmetry-lowering transition to a polar phase while exhibiting metallic electron transport, real materials find a myriad of ways to push the boundaries of this definition. Here, we review and explore the burgeoning polar metal frontier from the perspectives of theory, simulation, and experiment while introducing a unified taxonomy. The framework allows one to describe, identify, and classify polar metals; we also use it to discuss some of the fundamental tensions between theory and models of reality inherent in the terms ``ferroelectric" and ``metals.'' In addition, we highlight shortcomings of electrostatic doping simulations in modeling different subclasses of polar metals, noting how the assumptions of this approach depart from experiment. We include a survey of known materials that combine polar symmetry with metallic conductivity, classified according to the mechanisms used to harmonize those two orders and their resulting properties. We conclude by describing opportunities for the discovery of novel polar metals by utilizing our taxonomy.


title: Deep learning model compression using network sensitivity and gradients

abstract: Deep learning model compression is an improving and important field for the edge deployment of deep learning models. Given the increasing size of the models and their corresponding power consumption, it is vital to decrease the model size and compute requirement without a significant drop in the model's performance. In this paper, we present model compression algorithms for both non-retraining and retraining conditions. In the first case where retraining of the model is not feasible due to lack of access to the original data or absence of necessary compute resources while only having access to off-the-shelf models, we propose the Bin & Quant algorithm for compression of the deep learning models using the sensitivity of the network parameters. This results in 13x compression of the speech command and control model and 7x compression of the DeepSpeech2 models. In the second case when the models can be retrained and utmost compression is required for the negligible loss in accuracy, we propose our novel gradient-weighted k-means clustering algorithm (GWK). This method uses the gradients in identifying the important weight values in a given cluster and nudges the centroid towards those values, thereby giving importance to sensitive weights. Our method effectively combines product quantization with the EWGS[1] algorithm for sub-1-bit representation of the quantized models. We test our GWK algorithm on the CIFAR10 dataset across a range of models such as ResNet20, ResNet56, MobileNetv2 and show 35x compression on quantized models for less than 2% absolute loss in accuracy compared to the floating-point models.


title: HUE: Pretrained Model and Dataset for Understanding Hanja Documents of Ancient Korea

abstract: Historical records in Korea before the 20th century were primarily written in Hanja, an extinct language based on Chinese characters and not understood by modern Korean or Chinese speakers. Historians with expertise in this time period have been analyzing the documents, but that process is very difficult and time-consuming, and language models would significantly speed up the process. Toward building and evaluating language models for Hanja, we release the Hanja Understanding Evaluation dataset consisting of chronological attribution, topic classification, named entity recognition, and summary retrieval tasks. We also present BERT-based models continued training on the two major corpora from the 14th to the 19th centuries: the Annals of the Joseon Dynasty and Diaries of the Royal Secretariats. We compare the models with several baselines on all tasks and show there are significant improvements gained by training on the two corpora. Additionally, we run zero-shot experiments on the Daily Records of the Royal Court and Important Officials (DRRI). The DRRI dataset has not been studied much by the historians, and not at all by the NLP community.


title: Global Estimates of Spatially Distributed Surface Energy Fluxes using Thermodynamic Principles

abstract: Limited surface observations of turbulent heat fluxes result in incomplete knowledge about the surface energy balance that drives the climate system. Here, we developed a novel, purely physics-based analytical method grounded on the thermodynamic principle of maximum power. The approach derives the turbulent heat flux only from the four inputs of incoming and outgoing radiations at the land surface. The proposed approach does not use any parameterization, unlike the existing surface energy balance models, and hence does not suffer from uncertainty due to the same. We validated our methodology with 102 eddy covariance observation stations around the globe with different land use land covers. Using the satellite observations from CERES at a spatial resolution of 1 degree, we have obtained spatially distributed global analytical estimates of Sensible (H), latent heat (LE), and land surface heat storage (delQs) fluxes for the first time. For a global observed land Net radiation of 84 W/m2 from the satellite, we found H, LE and delQs to be 42 W/m2, 40 W/m2 and 2 W/m2, respectively. The theoretical and precise estimates of all surface energy balance components will improve our understanding of surface warming for different land use land covers across the globe.


title: Stable phase retrieval in function spaces

abstract: Let $(\Omega,\Sigma,\mu)$ be a measure space, and $1\leq p\leq \infty$. A subspace $E\subseteq L_p(\mu)$ is said to do stable phase retrieval (SPR) if there exists a constant $C\geq 1$ such that for any $f,g\in E$ we have $$ \inf_{|\lambda|=1} \|f-\lambda g\|\leq C\||f|-|g|\|. $$ In this case, if $|f|$ is known, then $f$ is uniquely determined up to an unavoidable global phase factor $\lambda$; moreover, the phase recovery map is $C$-Lipschitz. Phase retrieval appears in several applied circumstances, ranging from crystallography to quantum mechanics. In this article, we construct various subspaces doing stable phase retrieval, and make connections with $\Lambda(p)$-set theory. Moreover, we set the foundations for an analysis of stable phase retrieval in general function spaces. This, in particular, allows us to show that H\"older stable phase retrieval implies stable phase retrieval, improving the stability bounds in a recent article of M. Christ and the third and fourth authors. We also characterize those compact Hausdorff spaces $K$ such that $C(K)$ contains an infinite dimensional SPR subspace.


title: Bayesian analysis of mixtures of lognormal distribution with an unknown number of components from grouped data

abstract: This study proposes a reversible jump Markov chain Monte Carlo method for estimating parameters of lognormal distribution mixtures for income. Using simulated data examples, we examined the proposed algorithm's performance and the accuracy of posterior distributions of the Gini coefficients. Results suggest that the parameters were estimated accurately. Therefore, the posterior distributions are close to the true distributions even when the different data generating process is accounted for. Moreover, promising results for Gini coefficients encouraged us to apply our method to real data from Japan. The empirical examples indicate two subgroups in Japan (2020) and the Gini coefficients' integrity.


title: Algebraic Schouten Solitons of Three-Dimensional Lorentzian Lie Groups

abstract: In [10], Wears defined and studied algebraic T-solitons. In this paper, we give the definition of algebraic Schouten solitons as a special T-soliton and classify algebraic Schouten solitons associated to Levi-Civita connections, canonical connections and Kobayashi-Nomizu connections on three-dimensional Lorentzian Lie groups with some product structure.