title: AP$n$P: A Less-constrained P$n$P Solver for Pose Estimation with Unknown Anisotropic Scaling or Focal Lengths

abstract: Perspective-$n$-Point (P$n$P) stands as a fundamental algorithm for pose estimation in various applications. In this paper, we present a new approach to the P$n$P problem with relaxed constraints, eliminating the need for precise 3D coordinates or complete calibration data. We refer to it as AP$n$P due to its ability to handle unknown anisotropic scaling factors of 3D coordinates or alternatively two distinct focal lengths in addition to the conventional rigid transformation. Through algebraic manipulations and a novel parametrization, both cases are brought into similar forms that distinguish themselves primarily by the order of a rotation and an anisotropic scaling operation. AP$n$P then boils down to one unique polynomial problem, which is solved by the Gr\"obner basis approach. Experimental results on both simulated and real datasets demonstrate the effectiveness of AP$n$P as a more flexible and practical solution to camera pose estimation. Code: https://github.com/goldoak/APnP.


title: Farzi Data: Autoregressive Data Distillation

abstract: We study data distillation for auto-regressive machine learning tasks, where the input and output have a strict left-to-right causal structure. More specifically, we propose Farzi, which summarizes an event sequence dataset into a small number of synthetic sequences -- Farzi Data -- which are optimized to maintain (if not improve) model performance compared to training on the full dataset. Under the hood, Farzi conducts memory-efficient data distillation by (i) deriving efficient reverse-mode differentiation of the Adam optimizer by leveraging Hessian-Vector Products; and (ii) factorizing the high-dimensional discrete event-space into a latent-space which provably promotes implicit regularization. Empirically, for sequential recommendation and language modeling tasks, we are able to achieve 98-120% of downstream full-data performance when training state-of-the-art models on Farzi Data of size as little as 0.1% of the original dataset. Notably, being able to train better models with significantly less data sheds light on the design of future large auto-regressive models, and opens up new opportunities to further scale up model and data sizes.


title: Bowl Soliton Asymptotics and Applications

abstract: In this paper, we obtain the asymptotic expansion for the analogue of the bowl-soliton for a large `nondegenerate' class of fully nonlinear curvature flows. We use this to show the uniqueness of these bowl-type solitons in their asymptotic class. We also give examples to illustrate the situation for `degenerate' speeds and how they different they can be. Finally, we show how to construct `wing-like' solitons for these flows, which are complete, connected translators that are not graphical, entire or convex. We also obtain asymptotic expansions for them to show the variety of solutions that one can obtain depending on the choice of speed function.


title: Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets

abstract: Design space exploration (DSE) for Text-to-Image (TTI) models entails navigating a vast, opaque space of possible image outputs, through a commensurately vast input space of hyperparameters and prompt text. Minor adjustments to prompt input can surface unexpectedly disparate images. How can interfaces support end-users in reliably steering prompt-space explorations towards interesting results? Our design probe, DreamSheets, supports exploration strategies with LLM-based functions for assisted prompt construction and simultaneous display of generated results, hosted in a spreadsheet interface. The flexible layout and novel generative functions enable experimentation with user-defined workflows. Two studies, a preliminary lab study and a longitudinal study with five expert artists, revealed a set of strategies participants use to tackle the challenges of TTI design space exploration, and the interface features required to support them - like using text-generation to define local "axes" of exploration. We distill these insights into a UI mockup to guide future interfaces.


title: On Statistical Learning of Branch and Bound for Vehicle Routing Optimization

abstract: Recently, machine learning of the branch and bound algorithm has shown promise in approximating competent solutions to NP-hard problems. In this paper, we utilize and comprehensively compare the outcomes of three neural networks--graph convolutional neural network (GCNN), GraphSAGE, and graph attention network (GAT)--to solve the capacitated vehicle routing problem. We train these neural networks to emulate the decision-making process of the computationally expensive Strong Branching strategy. The neural networks are trained on six instances with distinct topologies from the CVRPLIB and evaluated on eight additional instances. Moreover, we reduced the minimum number of vehicles required to solve a CVRP instance to a bin-packing problem, which was addressed in a similar manner. Through rigorous experimentation, we found that this approach can match or improve upon the performance of the branch and bound algorithm with the Strong Branching strategy while requiring significantly less computational time. The source code that corresponds to our research findings and methodology is readily accessible and available for reference at the following web address: https://isotlaboratory.github.io/ml4vrp


title: Network Disruption via Continuous Batch Removal: The Case of Sicilian Mafia

abstract: Network disruption is pivotal in understanding the robustness and vulnerability of complex networks, which is instrumental in devising strategies for infrastructure protection, epidemic control, cybersecurity, and combating crime. In this paper, with a particular focus on disrupting criminal networks, we proposed to impose a within-the-largest-connected-component constraint in a continuous batch removal disruption process. Through a series of experiments on a recently released Sicilian Mafia network, we revealed that the constraint would enhance degree-based methods while weakening betweenness-based approaches. Moreover, based on the findings from the experiments using various disruption strategies, we propose a structurally-filtered greedy disruption strategy that integrates the effectiveness of greedy-like methods with the efficiency of structural-metric-based approaches. The proposed strategy significantly outperforms the longstanding state-of-the-art method of betweenness centrality while maintaining the same time complexity.


title: Personalization of CTC-based End-to-End Speech Recognition Using Pronunciation-Driven Subword Tokenization

abstract: Recent advances in deep learning and automatic speech recognition have improved the accuracy of end-to-end speech recognition systems, but recognition of personal content such as contact names remains a challenge. In this work, we describe our personalization solution for an end-to-end speech recognition system based on connectionist temporal classification. Building on previous work, we present a novel method for generating additional subword tokenizations for personal entities from their pronunciations. We show that using this technique in combination with two established techniques, contextual biasing and wordpiece prior normalization, we are able to achieve personal named entity accuracy on par with a competitive hybrid system.


title: Defect-induced helicity-dependent terahertz emission in Dirac semimetal PtTe2 thin films

abstract: Nonlinear transport enabled by symmetry breaking in quantum materials has aroused considerable interest in condensed matter physics and interdisciplinary electronics. However, the nonlinear optical response in centrosymmetric Dirac semimetals via the defect engineering has remained highly challenging. Here, we observe the helicity-dependent terahertz (THz) emission in Dirac semimetal PtTe2 thin films via circular photogalvanic effect (CPGE) under normal incidence. This is activated by artificially controllable out-of-plane Te-vacancy defect gradient, which is unambiguously evidenced by the electron ptychography. The defect gradient lowers the symmetry, which not only induces the band spin splitting, but also generates the giant Berry curvature dipole (BCD) responsible for the CPGE. Such BCD-induced helicity-dependent THz emission can be manipulated by the Te-vacancy defect concentration. Furthermore, temperature evolution of the THz emission features the minimum of the THz amplitude due to the carrier compensation. Our work provides a universal strategy for symmetry breaking in centrosymmetric Dirac materials for efficient nonlinear transport and facilitates the promising device applications in integrated optoelectronics and spintronics.


title: Chemotactic crawling of multivalent vesicles along ligand-density gradients

abstract: Living cells are capable of interacting with their environments in a variety of ways, including cell signalling, adhesion, and directed motion. These behaviours are often mediated by receptor molecules embedded in the cell membrane, which bind specific ligands. Adhesion mediated by a large number of weakly binding moieties - multivalent binding - is prevalent in a range of active cellular processes, such as cell crawling and pathogen-host invasion. In these circumstances, motion is often caused by gradients in ligand density, which constitutes a simple example of chemotaxis. To unravel the biophysics of chemotactic multivalent adhesion, we have designed an experimental system in which artificial cell models based on lipid vesicles adhere to a substrate through multivalent interactions, and perform chemotactic motion towards higher ligand concentrations. Adhesion occurs via vesicle-anchored receptors and substrate-anchored ligands, both consisting of synthetic DNA linkers that allow precise control over binding strength. Experimental data, rationalised through numerical and theoretical models, reveal that motion directionality is correlated to both binding strength and vesicle size. Besides providing insights into the biophysics of chemotactic multivalent adhesion, our results highlight design rules applicable to the development of biomimetic motile systems for synthetic biology and therapeutic applications.


title: Applications of Machine Learning in Biopharmaceutical Process Development and Manufacturing: Current Trends, Challenges, and Opportunities

abstract: While machine learning (ML) has made significant contributions to the biopharmaceutical field, its applications are still in the early stages in terms of providing direct support for quality-by-design based development and manufacturing of biopharmaceuticals, hindering the enormous potential for bioprocesses automation from their development to manufacturing. However, the adoption of ML-based models instead of conventional multivariate data analysis methods is significantly increasing due to the accumulation of large-scale production data. This trend is primarily driven by the real-time monitoring of process variables and quality attributes of biopharmaceutical products through the implementation of advanced process analytical technologies. Given the complexity and multidimensionality of a bioproduct design, bioprocess development, and product manufacturing data, ML-based approaches are increasingly being employed to achieve accurate, flexible, and high-performing predictive models to address the problems of analytics, monitoring, and control within the biopharma field. This paper aims to provide a comprehensive review of the current applications of ML solutions in a bioproduct design, monitoring, control, and optimisation of upstream, downstream, and product formulation processes. Finally, this paper thoroughly discusses the main challenges related to the bioprocesses themselves, process data, and the use of machine learning models in biopharmaceutical process development and manufacturing. Moreover, it offers further insights into the adoption of innovative machine learning methods and novel trends in the development of new digital biopharma solutions.


title: On an uncertainty principle for small index subgroups of finite fields

abstract: In this paper we continue the study of the nonvanishing minors property (NVM) initiated by Garcia, Karaali and Katz, for the compressed Fourier matrix attached to a subgroup $H$ of the multiplicative group of a finite field $\mathbb{F}_q$ and a character $\chi$ defined over $H$. Here we provide a characterization of this aforementioned property for \textit{symmetries} arising from an index-3 subgroup $H$ and a nontrivial character $\chi$.


title: A SEM-NCA approach towards the impact of participative budgeting on budgetary slack and managerial performance: The mediating role of leadership style and leader-member exchange

abstract: This study re-examines the impact of participative budgeting on managerial performance and budgetary slack, addressing gaps in current research. A revised conceptual model is developed, considering the conditioning roles of leadership style and leader-member exchange. The sample includes 408 employees with managerial experience in manufacturing companies. Hypotheses are tested using a combination of PLS-SEM and Necessary Condition Analysis (NCA). The results demonstrate that participative budgeting negatively affects budgetary slack and directly influences managerial performance, leadership style, and leader-member exchange. Moreover, leadership style and leader-member exchange moderate these relationships. The integration of NCA in management accounting research provides valuable insights for decision-makers, allowing for more effective measures. This study contributes by encouraging a complementary PLS-SEM and NCA approach to examine conditional effects. It also enhances understanding of budgetary control by highlighting the importance of leadership in influencing participative budgeting outcomes.


title: A Survey of Graph and Attention Based Hyperspectral Image Classification Methods for Remote Sensing Data

abstract: The use of Deep Learning techniques for classification in Hyperspectral Imaging (HSI) is rapidly growing and achieving improved performances. Due to the nature of the data captured by sensors that produce HSI images, a common issue is the dimensionality of the bands that may or may not contribute to the label class distinction. Due to the widespread nature of class labels, Principal Component Analysis is a common method used for reducing the dimensionality. However,there may exist methods that incorporate all bands of the Hyperspectral image with the help of the Attention mechanism. Furthermore, to yield better spectral spatial feature extraction, recent methods have also explored the usage of Graph Convolution Networks and their unique ability to use node features in prediction, which is akin to the pixel spectral makeup. In this survey we present a comprehensive summary of Graph based and Attention based methods to perform Hyperspectral Image Classification for remote sensing and aerial HSI images. We also summarize relevant datasets on which these techniques have been evaluated and benchmark the processing techniques.


title: Second-order bounds for the M/M/$s$ queue with random arrival rate

abstract: Consider an M/M/$s$ queue with the additional feature that the arrival rate is a random variable of which only the mean, variance, and range are known. Using semi-infinite linear programming and duality theory for moment problems, we establish for this setting tight bounds for the expected waiting time. These bounds correspond to an arrival rate that takes only two values. The proofs crucially depend on the fact that the expected waiting time, as function of the arrival rate, has a convex derivative. We apply the novel tight bounds to a rational queueing model, where arriving individuals decide to join or balk based on expected utility and only have partial knowledge about the market size.


title: Variations of Interatomic Force Constants in the Topological Phonon Phase Transition of AlGaN

abstract: The topological effects of phonons have been extensively studied in various materials, particularly in the wide-bandgap semiconductor GaN, which has the potential to improve heat dissipation in power electronics due to its intrinsic, topologically-protected, non-dissipative phonon surface states. Nevertheless, the phase transition of the Weyl phonons in nitrides and their composite alloys has yet to be elucidated. To unveil the microscale origin, topological phonon properties in AlGaN alloys are investigated using the virtual crystal approximation (VCA) and special quasi-random structure (SQS) approaches in this work. It is found that phase transitions in Weyl phonons are evidently present in AlGaN alloys and nitride single crystals. Under strain states, both GaN and AlN show a more prominent phase transition of Weyl phonons when subjected to biaxial compressive and uniaxial tensile strains. And it has been observed that the zz components in the self-term and the transverse 1NN force constants (FCs) are the most influential during the phase transition. The nonlinear Weyl phonon transition in AlGaN alloys, as modeled by the VCA, is reflected in the normalized self-term and first-nearest-neighbor (1NN) FCs, which vary in a nonlinear fashion with an increasing magnitude. This nonlinear phenomenon is also confirmed in the SQS modeling, where the unfolded phonon dispersions are consistent with those in the VCA modeling. With increased branches, hundreds of Weyl phonons are present accompanied by significant disorders in normalized FCs, which mainly occur for N atoms in self-terms and for all components in normalized 1NN FCs.


title: Forecaster: Towards Temporally Abstract Tree-Search Planning from Pixels

abstract: The ability to plan at many different levels of abstraction enables agents to envision the long-term repercussions of their decisions and thus enables sample-efficient learning. This becomes particularly beneficial in complex environments from high-dimensional state space such as pixels, where the goal is distant and the reward sparse. We introduce Forecaster, a deep hierarchical reinforcement learning approach which plans over high-level goals leveraging a temporally abstract world model. Forecaster learns an abstract model of its environment by modelling the transitions dynamics at an abstract level and training a world model on such transition. It then uses this world model to choose optimal high-level goals through a tree-search planning procedure. It additionally trains a low-level policy that learns to reach those goals. Our method not only captures building world models with longer horizons, but also, planning with such models in downstream tasks. We empirically demonstrate Forecaster's potential in both single-task learning and generalization to new tasks in the AntMaze domain.


title: SeUNet-Trans: A Simple yet Effective UNet-Transformer Model for Medical Image Segmentation

abstract: Automated medical image segmentation is becoming increasingly crucial to modern clinical practice, driven by the growing demand for precise diagnosis, the push towards personalized treatment plans, and the advancements in machine learning algorithms, especially the incorporation of deep learning methods. While convolutional neural networks (CNN) have been prevalent among these methods, the remarkable potential of Transformer-based models for computer vision tasks is gaining more acknowledgment. To harness the advantages of both CNN-based and Transformer-based models, we propose a simple yet effective UNet-Transformer (seUNet-Trans) model for medical image segmentation. In our approach, the UNet model is designed as a feature extractor to generate multiple feature maps from the input images, then the maps are propagated into a bridge layer, which is introduced to sequentially connect the UNet and the Transformer. In this stage, we approach the pixel-level embedding technique without position embedding vectors, aiming to make the model more efficient. Moreover, we apply spatial-reduction attention in the Transformer to reduce the computational/memory overhead. By leveraging the UNet architecture and the self-attention mechanism, our model not only retains the preservation of both local and global context information but also is capable of capturing long-range dependencies between input elements. The proposed model is extensively experimented on seven medical image segmentation datasets including polyp segmentation to demonstrate its efficacy. Comparison with several state-of-the-art segmentation models on these datasets shows the superior performance of our proposed seUNet-Trans network.


title: Outlier Detection Using Generative Models with Theoretical Performance Guarantees

abstract: This paper considers the problem of recovering signals modeled by generative models from linear measurements contaminated with sparse outliers. We propose an outlier detection approach for reconstructing the ground-truth signals modeled by generative models under sparse outliers. We establish theoretical recovery guarantees for reconstruction of signals using generative models in the presence of outliers, giving lower bounds on the number of correctable outliers. Our results are applicable to both linear generator neural networks and the nonlinear generator neural networks with an arbitrary number of layers. We propose an iterative alternating direction method of multipliers (ADMM) algorithm for solving the outlier detection problem via $\ell_1$ norm minimization, and a gradient descent algorithm for solving the outlier detection problem via squared $\ell_1$ norm minimization. We conduct extensive experiments using variational auto-encoder and deep convolutional generative adversarial networks, and the experimental results show that the signals can be successfully reconstructed under outliers using our approach. Our approach outperforms the traditional Lasso and $\ell_2$ minimization approach.


title: The Crisscross and the Cup: Two Short 3-Twist Paper Moebius Bands

abstract: We introduce the crisscross and the cup, both of which are immersed $3$-twist polygonal paper Moebius band of aspect ratio $3$. We explain why these two objects are limits of smooth embedded paper Moebius bands having knotted boundary. We conjecture that any smooth embedded paper Moebius band with knotted boundary has aspect ratio greater than $3$. The crisscross is planar but the cup is not.


title: Systematic discrepancies in the delivery of political ads on Facebook and Instagram

abstract: Political advertising on social media has become a central element in election campaigns. However, granular information about political advertising on social media was previously unavailable, thus raising concerns regarding fairness, accountability, and transparency in the electoral process. In this paper, we analyze targeted political advertising on social media via a unique, large-scale dataset of over 80000 political ads from Meta during the 2021 German federal election, with more than 1.1 billion impressions. For each political ad, our dataset records granular information about targeting strategies, spending, and actual impressions. We then study (i) the prevalence of targeted ads across the political spectrum; (ii) the discrepancies between targeted and actual audiences due to algorithmic ad delivery; and (iii) which targeting strategies on social media attain a wide reach at low cost. We find that targeted ads are prevalent across the entire political spectrum. Moreover, there are considerable discrepancies between targeted and actual audiences, and systematic differences in the reach of political ads (in impressions-per-EUR) among parties, where the algorithm favors ads from populists over others.


title: Assessing Encoder-Decoder Architectures for Robust Coronary Artery Segmentation

abstract: Coronary artery diseases are among the leading causes of mortality worldwide. Timely and accurate diagnosis, facilitated by precise coronary artery segmentation, is pivotal in changing patient outcomes. In the realm of biomedical imaging, convolutional neural networks, especially the U-Net architecture, have revolutionised segmentation processes. However, one of the primary challenges remains the lack of benchmarking datasets specific to coronary arteries. However through the use of the recently published public dataset ASOCA, the potential of deep learning for accurate coronary segmentation can be improved. This paper delves deep into examining the performance of 25 distinct encoder-decoder combinations. Through analysis of the 40 cases provided to ASOCA participants, it is revealed that the EfficientNet-LinkNet combination, serving as encoder and decoder, stands out. It achieves a Dice coefficient of 0.882 and a 95th percentile Hausdorff distance of 4.753. These findings not only underscore the superiority of our model in comparison to those presented at the MICCAI 2020 challenge but also set the stage for future advancements in coronary artery segmentation, opening doors to enhanced diagnostic and treatment strategies.


title: Conformal Contextual Robust Optimization

abstract: Data-driven approaches to predict-then-optimize decision-making problems seek to mitigate the risk of uncertainty region misspecification in safety-critical settings. Current approaches, however, suffer from considering overly conservative uncertainty regions, often resulting in suboptimal decisionmaking. To this end, we propose Conformal-Predict-Then-Optimize (CPO), a framework for leveraging highly informative, nonconvex conformal prediction regions over high-dimensional spaces based on conditional generative models, which have the desired distribution-free coverage guarantees. Despite guaranteeing robustness, such black-box optimization procedures alone inspire little confidence owing to the lack of explanation of why a particular decision was found to be optimal. We, therefore, augment CPO to additionally provide semantically meaningful visual summaries of the uncertainty regions to give qualitative intuition for the optimal decision. We highlight the CPO framework by demonstrating results on a suite of simulation-based inference benchmark tasks and a vehicle routing task based on probabilistic weather prediction.


title: Exact Renormalization of the Higgs Field

abstract: Using Wilsonian renormalization, we calculate the quantum correction to observable quantities, rather than the bare parameters, of the Higgs field. A physical parameter, such as a mass-squared or a quartic coupling, at an energy scale $\mu$ is obtained from that at a reference scale by integrating in the degrees of freedom in between. In this process, heavy modes decouple and the ultraviolet scale dependence is canceled in the observables. An exact renormalization group equation is parametrized by the low-energy scale $\mu$.


title: Formulation of downward causation in the brain: whole beats its parts

abstract: Downward causation is the causal effect from the whole at the macro level to its parts at the micro level, and is regarded as a solution to the mind-body problem. However, downward causation is a puzzling phenomenon because the princple by which new causal power is generated but cannot be reduced to the micro level is unclear. Here, we argue that a feedback control of micro-level neural mechanisms using macro-level algebraic structure information between neural network modules, physically composed of the micro neural mechanisms, is a "whole beats its parts" model of downward causation in the brain.


title: Implicit regularization via soft ascent-descent

abstract: As models grow larger and more complex, achieving better off-sample generalization with minimal trial-and-error is critical to the reliability and economy of machine learning workflows. As a proxy for the well-studied heuristic of seeking "flat" local minima, gradient regularization is a natural avenue, and first-order approximations such as Flooding and sharpness-aware minimization (SAM) have received significant attention, but their performance depends critically on hyperparameters (flood threshold and neighborhood radius, respectively) that are non-trivial to specify in advance. In order to develop a procedure which is more resilient to misspecified hyperparameters, with the hard-threshold "ascent-descent" switching device used in Flooding as motivation, we propose a softened, pointwise mechanism called SoftAD that downweights points on the borderline, limits the effects of outliers, and retains the ascent-descent effect. We contrast formal stationarity guarantees with those for Flooding, and empirically demonstrate how SoftAD can realize classification accuracy competitive with SAM and Flooding while maintaining a much smaller loss generalization gap and model norm. Our empirical tests range from simple binary classification on the plane to image classification using neural networks with millions of parameters; the key trends are observed across all datasets and models studied, and suggest a potential new approach to implicit regularization.


title: Broadband radio study of the North Polar Spur: Origin of the spectral turnover with insights into the X-ray and Gamma-ray spectra

abstract: The North Polar Spur (NPS) is a giant structure that is clearly visible in both radio and X-ray all-sky maps. We analyzed broadband radio observations covering a range between 22 MHz and 70 GHz to systematically analyze the thermal/non-thermal emissions associated with the NPS. We demonstrate that the radio emission of the NPS comprises synchrotron, free-free, and dust emission; however, synchrotron emissions dominate over other emissions, especially at high galactic-latitudes. Moreover, the synchrotron spectra exhibit a power-law behavior with $N(\gamma)\propto\gamma^{-s}$ ($s\simeq1.8-2.4$) up to a few GHz moderated by a turnover at $\nu_{\rm brk} \simeq 1$ GHz, above which the spectral index $s$ decrease by one. Assuming that the turnover is due to the electrons cooled by synchrotron radiation before escaping (or advecting) from the emission region, the magnetic field strength can be estimated to be $B\sim 8 \rm\mu G$ if the NPS is a distant structure that is near the Galactic Center (GC). However, an unreasonably strong $B\sim 114\rm\mu G$ is required if the NPS is near the local supernova remnant (SNR). The corresponding non-thermal energy stored in the NPS is $E_{\rm n/th}\simeq 4.4\times 10^{55}$ erg in the GC scenario, whereas $E_{\rm n/th}\simeq 4.1\times 10^{52}$ erg is difficult to explain with a single local SNR. We also estimated the gamma-ray emission associated with the NPS through inverse Comptonization of the cosmic microwave background (CMB), which peaks at 100 - 1000 keV with a flux of $\nu F_{\nu}\sim 10^{-9}$ $\rm erg\,cm^{-2}s^{-1}sr^{-1}$ in the GC model, and may be a good candidate for detection by future X-ray/gamma-ray observatories.


title: Towards Unified and Effective Domain Generalization

abstract: We propose $\textbf{UniDG}$, a novel and $\textbf{Uni}$fied framework for $\textbf{D}$omain $\textbf{G}$eneralization that is capable of significantly enhancing the out-of-distribution generalization performance of foundation models regardless of their architectures. The core idea of UniDG is to finetune models during the inference stage, which saves the cost of iterative training. Specifically, we encourage models to learn the distribution of test data in an unsupervised manner and impose a penalty regarding the updating step of model parameters. The penalty term can effectively reduce the catastrophic forgetting issue as we would like to maximally preserve the valuable knowledge in the original model. Empirically, across 12 visual backbones, including CNN-, MLP-, and Transformer-based models, ranging from 1.89M to 303M parameters, UniDG shows an average accuracy improvement of +5.4% on DomainBed. These performance results demonstrate the superiority and versatility of UniDG. The code is publicly available at https://github.com/invictus717/UniDG


title: Monte Carlo Simulations of Factors Influencing Seasonal Variation of Multiple Muon Events

abstract: In a 2015 study, particle physicists reported the first observed seasonal variation of multiple muon events in the MINOS Far and Near Detectors, where multiple muon events created by cosmic rays were observed to be more numerous in the winter than in the summer. This goes against the initial hypothesis of the researchers, because it has long been measured that single muon rates are higher in the summer. Using Python, I simulated three potential factors that could affect this: muon decay, altitude, and the fate of the leading pion from the first interaction. My data gives support to the conclusion that the seasonal variation is due to the geometry of the underground particle detectors and related to the higher altitude of the atmosphere in the summer.


title: Black-box Targeted Adversarial Attack on Segment Anything (SAM)

abstract: Deep recognition models are widely vulnerable to adversarial examples, which change the model output by adding quasi-imperceptible perturbation to the image input. Recently, Segment Anything Model (SAM) has emerged to become a popular foundation model in computer vision due to its impressive generalization to unseen data and tasks. Realizing flexible attacks on SAM is beneficial for understanding the robustness of SAM in the adversarial context. To this end, this work aims to achieve a targeted adversarial attack (TAA) on SAM. Specifically, under a certain prompt, the goal is to make the predicted mask of an adversarial example resemble that of a given target image. The task of TAA on SAM has been realized in a recent arXiv work in the white-box setup by assuming access to prompt and model, which is thus less practical. To address the issue of prompt dependence, we propose a simple yet effective approach by only attacking the image encoder. Moreover, we propose a novel regularization loss to enhance the cross-model transferability by increasing the feature dominance of adversarial images over random natural images. Extensive experiments verify the effectiveness of our proposed simple techniques to conduct a successful black-box TAA on SAM.


title: A resolvent-based prediction framework for incompressible turbulent channel flow with limited measurements

abstract: A new resolvent-based method is developed to predict the space-time properties of the flow field. To overcome the deterioration of the prediction accuracy with the increasing distance between the measurements and predictions in the Resolvent-Based Estimation (RBE), the newly proposed method utilizes the RBE to estimate the relative energy distribution near the wall rather than the absolute energy directly estimated from the measurements. Using this extra information from RBE, the new method modifies the energy distribution of the spatially uniform and uncorrelated forcing that drives the flow system by minimizing the norm of the cross-spectral density (CSD) tensor of the error matrix in the near-wall region in comparison with the RBE-estimated one, and therefore it is named as the Resolvent-informed White-noise-based Estimation (RWE) method. For validation, three time-resolved direct numerical simulation (DNS) datasets with the friction Reynolds numbers $Re_\tau = 180$, 550, and 950 are generated, with various locations of measurements ranging from the near-wall region ($y^+ = 40$) to the upper bound of the logarithmic region ($y/h \approx 0.2$) for the predictions. Besides the RWE, three existing methods, i.e., the RBE, the $\lambda$-model, and the White-noise-Based Estimation (WBE), are also included for the validation. The performance of the RBE and $\lambda$-model in predicting the energy spectra shows a strong dependence on the measurement locations. The newly proposed RWE shows a low sensitivity on $Re_{\tau}$ and the measurement locations, which may range from the near-wall region to the upper bound of the logarithmic region, and has a high accuracy in predicting the energy spectra.


title: Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?

abstract: Diffusion models for text-to-image (T2I) synthesis, such as Stable Diffusion (SD), have recently demonstrated exceptional capabilities for generating high-quality content. However, this progress has raised several concerns of potential misuse, particularly in creating copyrighted, prohibited, and restricted content, or NSFW (not safe for work) images. While efforts have been made to mitigate such problems, either by implementing a safety filter at the evaluation stage or by fine-tuning models to eliminate undesirable concepts or styles, the effectiveness of these safety measures in dealing with a wide range of prompts remains largely unexplored. In this work, we aim to investigate these safety mechanisms by proposing one novel concept retrieval algorithm for evaluation. We introduce Ring-A-Bell, a model-agnostic red-teaming tool for T2I diffusion models, where the whole evaluation can be prepared in advance without prior knowledge of the target model. Specifically, Ring-A-Bell first performs concept extraction to obtain holistic representations for sensitive and inappropriate concepts. Subsequently, by leveraging the extracted concept, Ring-A-Bell automatically identifies problematic prompts for diffusion models with the corresponding generation of inappropriate content, allowing the user to assess the reliability of deployed safety mechanisms. Finally, we empirically validate our method by testing online services such as Midjourney and various methods of concept removal. Our results show that Ring-A-Bell, by manipulating safe prompting benchmarks, can transform prompts that were originally regarded as safe to evade existing safety mechanisms, thus revealing the defects of the so-called safety mechanisms which could practically lead to the generation of harmful contents. Our codes are available at https://github.com/chiayi-hsu/Ring-A-Bell.


title: Riemannian Residual Neural Networks

abstract: Recent methods in geometric deep learning have introduced various neural networks to operate over data that lie on Riemannian manifolds. Such networks are often necessary to learn well over graphs with a hierarchical structure or to learn over manifold-valued data encountered in the natural sciences. These networks are often inspired by and directly generalize standard Euclidean neural networks. However, extending Euclidean networks is difficult and has only been done for a select few manifolds. In this work, we examine the residual neural network (ResNet) and show how to extend this construction to general Riemannian manifolds in a geometrically principled manner. Originally introduced to help solve the vanishing gradient problem, ResNets have become ubiquitous in machine learning due to their beneficial learning properties, excellent empirical results, and easy-to-incorporate nature when building varied neural networks. We find that our Riemannian ResNets mirror these desirable properties: when compared to existing manifold neural networks designed to learn over hyperbolic space and the manifold of symmetric positive definite matrices, we outperform both kinds of networks in terms of relevant testing metrics and training dynamics.


title: Stability and electronic properties of "4-8"-type ZnSnN$_2$ thin films free of spontaneous polarization for optoelectronic devices

abstract: Ternary nitride ZnSnN$_2$ is a promising photovoltaic absorber material. In this work, using first-principles calculations, we investigate the stability and electronic properties of "4-8"-type ZnSnN$_2$ thin films. We find that below a certain thickness "4-8"-type thin films have lower total energy than polar films. For 4-layer ZnSnN$_2$ thin film, the Pna2$_1$/Pmc2$_1$ $\rightarrow$ 4-8 transition can spontaneously occur at finite temperatures. All "4-8"-type thin films studied are semiconducting and free of spontaneous polarization, the bandgaps of which can be tuned by the thickness of films, ranging from 1.4 eV to 1.8 eV. Furthermore, these films show light electron effective masses, and octet-rule-preserving disorder has insignificant effects on the electronic properties. Our results provide new insights into the structure of ZnSnN$_2$ in the thin film form and guidance for the experimental investigation.


title: The Reality of the Situation: A Survey of Situated Analytics

abstract: The advent of low cost, accessible, and high performance augmented reality (AR) has shed light on a situated form of analytics where in-situ visualizations embedded in the real world can facilitate sensemaking based on the user's physical location. In this work, we identify prior literature in this emerging field with a focus on situated analytics. After collecting 47 relevant situated analytics systems, we classify them using a taxonomy of three dimensions: situating triggers, view situatedness, and data depiction. We then identify four archetypical patterns in our classification using an ensemble cluster analysis. We also assess the level which these systems support the sensemaking process. Finally, we discuss insights and design guidelines that we learned from our analysis.


title: Towards Scalable Cross-Chain Messaging

abstract: Blockchains were originally designed as closed execution environments and lack the ability to communicate directly with external systems. To overcome this limitation, many blockchains employ relayers, external applications capable of transporting data between different blockchains. Typically, the process of relaying data is permissionless and multiple independent relayers work concurrently to transport the same information between two blockchains. While this model increases the reliability of data delivery by providing redundancy, it also introduces challenges that have not been previously discussed. In this work, we bridge this gap by discussing the shortcomings of permissionless cross-chain relaying systems and identifying three issues that adversely impact their performance, scalability and security. We take the first step towards addressing issues that hinder performance and scalability by proposing a novel protocol to enable coordination among independent relayers. Additionally, we provide an in-depth discussion about the trade-offs associated with the design of relayer coordination protocols for permissionless settings. Through this work we provide a foundation for improving cross-chain relaying services.


title: Late-time data require smaller sound horizon at recombination

abstract: The Hubble constant problem is that the values of Hubble constant from the observation of cosmic microwave background assuming the LambdaCDM model disagrees with the values from direct measurements. This problem suggests some new physics beyond the LambdaCDM model. Typically there are two ways of reconciliation: one is the realization of smaller value of sound horizon at recombination, and the other is the modification of the way of expansion of the universe after recombination. In this letter we examine the latter possibility by comparing two typical phenomenological dark energy models with the distance-redshift relation provided by Pantheon catalogue of supernova observations and galaxy surveys by BOSS and eBOSS collaborations. Though these phenomenological dark energy models globally fit observations better than the LambdaCDM model, they are strongly disfavored by the distance-redshift relation as almost the same level as the LambdaCDM model defined by cosmic microwave background observations. The distance-redshift relation strongly suggests some new physics which realize smaller value of sound horizon at recombination.


title: Host Galaxies for Four Nearby CHIME/FRB Sources and the Local Universe FRB Host Galaxy Population

abstract: We present the host galaxies of four apparently non-repeating fast radio bursts (FRBs), FRBs 20181223C, 20190418A, 20191220A, and 20190425A, reported in the first Canadian Hydrogen Intensity Mapping Experiment (CHIME/FRB) catalog. Our selection of these FRBs is based on a planned hypothesis testing framework where we search all CHIME/FRB Catalog-1 events that have low extragalactic dispersion measure (< 100 pc cm$^{-3}$), with high Galactic latitude (|b| > 10$\deg$) and saved baseband data. We associate the selected FRBs to galaxies with moderate to high star-formation rates located at redshifts between 0.027 and 0.071. We also search for possible multi-messenger counterparts, including persistent compact radio and gravitational wave (GW) sources, and find none. Utilizing the four FRB hosts from this study along with the hosts of 14 published local Universe FRBs (z < 0.1) with robust host association, we conduct an FRB host demographics analysis. We find all 18 local Universe FRB hosts in our sample to be spirals (or late-type galaxies), including the host of FRB 20220509G, which was previously reported to be elliptical. Using this observation, we scrutinize proposed FRB source formation channels and argue that core-collapse supernovae are likely the dominant channel to form FRB progenitors. Moreover, we infer no significant difference in the host properties of repeating and apparently non-repeating FRBs in our local Universe FRB host sample. Finally, we find the burst rates of these four apparently non-repeating FRBs to be consistent with those of the sample of localized repeating FRBs observed by CHIME/FRB. Therefore, we encourage further monitoring of these FRBs with more sensitive radio telescopes.


title: KPZ exponents for the half-space log-gamma polymer

abstract: We consider the point-to-point log-gamma polymer of length $2N$ in a half-space with i.i.d. $\operatorname{Gamma}^{-1}(2\theta)$ distributed bulk weights and i.i.d. $\operatorname{Gamma}^{-1}(\alpha+\theta)$ distributed boundary weights for $\theta>0$ and $\alpha>-\theta$. We establish the KPZ exponents ($1/3$ fluctuation and $2/3$ transversal) for this model when $\alpha=N^{-1/3}\mu$ for $\mu\in \mathbb{R}$ fixed (critical regime) and when $\alpha>0$ is fixed (supercritical regime). In particular, in these two regimes, we show that after appropriate centering, the free energy process with spatial coordinate scaled by $N^{2/3}$ and fluctuations scaled by $N^{1/3}$ is tight. These regimes correspond to a polymer measure which is not pinned at the boundary. This is the first instance of establishing the $2/3$ transversal exponent for a positive temperature half-space model, and the first instance of the $1/3$ fluctuation exponent besides precisely at the boundary where recent work of arXiv:2204.08420 applies and also gives the exact one-point fluctuation distribution (our methods do not access exact fluctuation distributions). Our proof relies on two inputs -- the relationship between the half-space log-gamma polymer and half-space Whittaker process (facilitated by the geometric RSK correspondence as initiated in arXiv:1110.3489, arXiv:1210.5126), and an identity in arXiv:2108.08737 which relates the point-to-line half-space partition function to the full-space partition function for the log-gamma polymer. The primary technical contribution of our work is to construct the half-space log-gamma Gibbsian line ensemble and develop, in the spirit of work initiated in arXiv:1108.2291, a toolbox for extracting tightness and absolute continuity results from minimal information about the top curve of such half-space line ensembles. This is the first study of half-space line ensembles.


title: Optimized nanodevice fabrication using clean transfer of graphene by polymer mixture: Experiments and Neural Network based simulations

abstract: In this study, we investigate both experimentally and computationally the molecular interactions of two distinct polymers with graphene. Our experimental findings indicate that the use of a polymer mixture reduces the transfer induced doping and strain in fabricated graphene devices as compared to conventional single polymer wet transfer. We found that such reduction is related to the decreased affinity of mixture of polymethyl methacrylate and angelica lactone polymer for graphene. We investigated changes in binding energy (BE) of polymer mixture and graphene by considering energy decomposition analysis using a pre-trained potential neural network. It was found that numerical simulations accurately predicted two-fold reduction of BE and order of magnitude reduction of electrostatic interaction between polymers.


title: Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidance

abstract: We propose BOSS, an approach that automatically learns to solve new long-horizon, complex, and meaningful tasks by growing a learned skill library with minimal supervision. Prior work in reinforcement learning require expert supervision, in the form of demonstrations or rich reward functions, to learn long-horizon tasks. Instead, our approach BOSS (BOotStrapping your own Skills) learns to accomplish new tasks by performing "skill bootstrapping," where an agent with a set of primitive skills interacts with the environment to practice new skills without receiving reward feedback for tasks outside of the initial skill set. This bootstrapping phase is guided by large language models (LLMs) that inform the agent of meaningful skills to chain together. Through this process, BOSS builds a wide range of complex and useful behaviors from a basic set of primitive skills. We demonstrate through experiments in realistic household environments that agents trained with our LLM-guided bootstrapping procedure outperform those trained with naive bootstrapping as well as prior unsupervised skill acquisition methods on zero-shot execution of unseen, long-horizon tasks in new environments. Website at clvrai.com/boss.


title: An Empirical Study of Super-resolution on Low-resolution Micro-expression Recognition

abstract: Micro-expression recognition (MER) in low-resolution (LR) scenarios presents an important and complex challenge, particularly for practical applications such as group MER in crowded environments. Despite considerable advancements in super-resolution techniques for enhancing the quality of LR images and videos, few study has focused on investigate super-resolution for improving LR MER. The scarcity of investigation can be attributed to the inherent difficulty in capturing the subtle motions of micro-expressions, even in original-resolution MER samples, which becomes even more challenging in LR samples due to the loss of distinctive features. Furthermore, a lack of systematic benchmarking and thorough analysis of super-resolution-assisted MER methods has been noted. This paper tackles these issues by conducting a series of benchmark experiments that integrate both super-resolution (SR) and MER methods, guided by an in-depth literature survey. Specifically, we employ seven cutting-edge state-of-the-art (SOTA) MER techniques and evaluate their performance on samples generated from 13 SOTA SR techniques, thereby addressing the problem of super-resolution in MER. Through our empirical study, we uncover the primary challenges associated with SR-assisted MER and identify avenues to tackle these challenges by leveraging recent advancements in both SR and MER methodologies. Our analysis provides insights for progressing toward more efficient SR-assisted MER.


title: 3D-BBS: Global Localization for 3D Point Cloud Scan Matching Using Branch-and-Bound Algorithm

abstract: This paper presents an accurate and fast 3D global localization method, 3D-BBS, that extends the existing branch-and-bound (BnB)-based 2D scan matching (BBS) algorithm. To reduce memory consumption, we utilize a sparse hash table for storing hierarchical 3D voxel maps. To improve the processing cost of BBS in 3D space, we propose an efficient roto-translational space branching. Furthermore, we devise a batched BnB algorithm to fully leverage GPU parallel processing. Through experiments in simulated and real environments, we demonstrated that the 3D-BBS enabled accurate global localization with only a 3D LiDAR scan roughly aligned in the gravity direction and a 3D pre-built map. This method required only 878 msec on average to perform global localization and outperformed state-of-the-art global registration methods in terms of accuracy and processing speed.


title: Managing Persuasion Robustly: The Optimality of Quota Rules

abstract: We study a sender-receiver model where the receiver can commit to a decision rule before the sender determines the information policy. The decision rule can depend on the signal structure and the signal realization that the sender adopts. This framework captures applications where a decision-maker (the receiver) solicit advice from an interested party (sender). In these applications, the receiver faces uncertainty regarding the sender's preferences and the set of feasible signal structures. Consequently, we adopt a unified robust analysis framework that includes max-min utility, min-max regret, and min-max approximation ratio as special cases. We show that it is optimal for the receiver to sacrifice ex-post optimality to perfectly align the sender's incentive. The optimal decision rule is a quota rule, i.e., the decision rule maximizes the receiver's ex-ante payoff subject to the constraint that the marginal distribution over actions adheres to a consistent quota, regardless of the sender's chosen signal structure.


title: Dual-Scale Interest Extraction Framework with Self-Supervision for Sequential Recommendation

abstract: In the sequential recommendation task, the recommender generally learns multiple embeddings from a user's historical behaviors, to catch the diverse interests of the user. Nevertheless, the existing approaches just extract each interest independently for the corresponding sub-sequence while ignoring the global correlation of the entire interaction sequence, which may fail to capture the user's inherent preference for the potential interests generalization and unavoidably make the recommended items homogeneous with the historical behaviors. In this paper, we propose a novel Dual-Scale Interest Extraction framework (DSIE) to precisely estimate the user's current interests. Specifically, DSIE explicitly models the user's inherent preference with contrastive learning by attending over his/her entire interaction sequence at the global scale and catches the user's diverse interests in a fine granularity at the local scale. Moreover, we develop a novel interest aggregation module to integrate the multi-interests according to the inherent preference to generate the user's current interests for the next-item prediction. Experiments conducted on three real-world benchmark datasets demonstrate that DSIE outperforms the state-of-the-art models in terms of recommendation preciseness and novelty.


title: Real-time Speech Enhancement and Separation with a Unified Deep Neural Network for Single/Dual Talker Scenarios

abstract: This paper introduces a practical approach for leveraging a real-time deep learning model to alternate between speech enhancement and joint speech enhancement and separation depending on whether the input mixture contains one or two active speakers. Scale-invariant signal-to-distortion ratio (SI-SDR) has shown to be a highly effective training measure in time-domain speech separation. However, the SI-SDR metric is ill-defined for zero-energy target signals, which is a problem when training a speech separation model using utterances with varying numbers of talkers. Unlike existing solutions that focus on modifying the loss function to accommodate zero-energy target signals, the proposed approach circumvents this problem by training the model to extract speech on both its output channels regardless if the input is a single or dual-talker mixture. A lightweight speaker overlap detection (SOD) module is also introduced to differentiate between single and dual-talker segments in real-time. The proposed module takes advantage of the new formulation by operating directly on the separated masks, given by the separation model, instead of the original mixture, thus effectively simplifying the detection task. Experimental results show that the proposed training approach outperforms existing solutions, and the SOD module exhibits high accuracy.


title: RoomDesigner: Encoding Anchor-latents for Style-consistent and Shape-compatible Indoor Scene Generation

abstract: Indoor scene generation aims at creating shape-compatible, style-consistent furniture arrangements within a spatially reasonable layout. However, most existing approaches primarily focus on generating plausible furniture layouts without incorporating specific details related to individual furniture pieces. To address this limitation, we propose a two-stage model integrating shape priors into the indoor scene generation by encoding furniture as anchor latent representations. In the first stage, we employ discrete vector quantization to encode furniture pieces as anchor-latents. Based on the anchor-latents representation, the shape and location information of the furniture was characterized by a concatenation of location, size, orientation, class, and our anchor latent. In the second stage, we leverage a transformer model to predict indoor scenes autoregressively. Thanks to incorporating the proposed anchor-latents representations, our generative model produces shape-compatible and style-consistent furniture arrangements and synthesis furniture in diverse shapes. Furthermore, our method facilitates various human interaction applications, such as style-consistent scene completion, object mismatch correction, and controllable object-level editing. Experimental results on the 3D-Front dataset demonstrate that our approach can generate more consistent and compatible indoor scenes compared to existing methods, even without shape retrieval. Additionally, extensive ablation studies confirm the effectiveness of our design choices in the indoor scene generation model.


title: First-Principle Investigation Of Near-Field Energy Transfer Between Localized Quantum Emitters in Solids

abstract: We present a predictive and general approach to investigate near-field energy transfer processes between localized defects in semiconductors, which couples first principle electronic structure calculations and a nonrelativistic quantum electrodynamics description of photons in the weak-coupling regime. We apply our approach to investigate an exemplar point defect in an oxide, the F center in MgO, and we show that the energy transfer from a magnetic source, e.g., a rare earth impurity, to the vacancy can lead to spin non conserving long-lived excitation that are dominant processes in the near field, at distances relevant to the design of photonic devices and ultra-high dense memories. We also define a descriptor for coherent energy transfer to predict geometrical configurations of emitters to enable long-lived excitations, that are useful to design optical memories in semiconductor and insulators.


title: A Human Motion Compensation Framework for a Supernumerary Robotic Arm

abstract: Supernumerary robotic arms (SRAs) can be used as the third arm to complement and augment the abilities of human users. The user carrying a SRA forms a connected kinodynamic chain, which can be viewed as a special class of floating-base robot systems. However, unlike general floating-base robot systems, human users are the bases of SRAs and they have their subjective behaviors/motions. This implies that human body motions can unintentionally affect the SRA's end-effector movements. To address this challenge, we propose a framework to compensate for the human whole-body motions that interfere with the SRA's end-effector trajectories. The SRA system in this study consists of a 6-degree-of-freedom lightweight arm and a wearable interface. The wearable interface allows users to adjust the installation position of the SRA to fit different body shapes. An inertial measurement unit (IMU)-based sensory interface can provide the body skeleton motion feedback of the human user in real time. By simplifying the floating-base kinematics model, we design an effective motion planner by reconstructing the Jacobian matrix of the SRA. Under the proposed framework, the performance of the reconstructed Jacobian method is assessed by comparing the results obtained with the classical nullspace-based method through two sets of experiments.


title: Unraveling Fundamental Properties of Power System Resilience Curves using Unsupervised Machine Learning

abstract: The standard model of infrastructure resilience, the resilience triangle, has been the primary way of characterizing and quantifying infrastructure resilience. However, the theoretical model merely provides a one-size-fits-all framework for all infrastructure systems. Most of the existing studies examine the characteristics of infrastructure resilience curves based on analytical models constructed upon simulated system performance. Limited empirical studies hindered our ability to fully understand and predict resilience characteristics in infrastructure systems. To address this gap, this study examined over 200 resilience curves related to power outages in three major extreme weather events. Using unsupervised machine learning, we examined different curve archetypes, as well as the fundamental properties of each resilience curve archetype. The results show two primary archetypes for power system resilience curves, triangular, and trapezoidal curves. Triangular curves characterize resilience behavior based on 1. critical functionality threshold, 2. critical functionality recovery rate, and 3. recovery pivot point. Trapezoidal archetypes explain resilience curves based on 1. duration of sustained function loss and 2. constant recovery rate. The longer the duration of sustained function loss, the slower the constant rate of recovery. The findings of this study provide novel perspectives enabling better understanding and prediction of resilience performance of power system infrastructures.


title: Density Properties of Sets in Finite-Dimensional, Strictly Convex Banach Spaces

abstract: In this article, we examine the theorem of Mattila establishing rectifiability for regular sets in the setting of strictly convex, finite-dimensional Banach spaces.


title: Magnetic response of a two-dimensional viscous electron fluid

abstract: It has been established that the Coulomb interactions can transform the electron gas into a viscous fluid. This fluid is realized in a number of platforms, including graphene and two-dimensional semiconductor heterostructures. The defining characteristic of the electron fluid is the formation of layers of charge carriers that are in local thermodynamic equilibrium, as in classical fluids. In the presence of nonuniformities, whirlpools and nontrivial flow profiles are formed, which have been directly imaged in recent experiments. In this paper, we theoretically study the response of the electron fluid to localized magnetic fields. We find that the electric current is suppressed by viscous vortices in regions where magnetic field is sharply varying, causing strong transport signatures. Experimentally, our considerations are relevant since local magnetic fields can be applied to the system through implanting adatoms or embedding micromagnets in the top-gate. Our theory is essential for the characterization and future applications of electron fluids in hydrodynamic spin transport.


title: Deep Unfolding Network for Image Compressed Sensing by Content-adaptive Gradient Updating and Deformation-invariant Non-local Modeling

abstract: Inspired by certain optimization solvers, the deep unfolding network (DUN) has attracted much attention in recent years for image compressed sensing (CS). However, there still exist the following two issues: 1) In existing DUNs, most hyperparameters are usually content independent, which greatly limits their adaptability for different input contents. 2) In each iteration, a plain convolutional neural network is usually adopted, which weakens the perception of wider context prior and therefore depresses the expressive ability. In this paper, inspired by the traditional Proximal Gradient Descent (PGD) algorithm, a novel DUN for image compressed sensing (dubbed DUN-CSNet) is proposed to solve the above two issues. Specifically, for the first issue, a novel content adaptive gradient descent network is proposed, in which a well-designed step size generation sub-network is developed to dynamically allocate the corresponding step sizes for different textures of input image by generating a content-aware step size map, realizing a content-adaptive gradient updating. For the second issue, considering the fact that many similar patches exist in an image but have undergone a deformation, a novel deformation-invariant non-local proximal mapping network is developed, which can adaptively build the long-range dependencies between the nonlocal patches by deformation-invariant non-local modeling, leading to a wider perception on context priors. Extensive experiments manifest that the proposed DUN-CSNet outperforms existing state-of-the-art CS methods by large margins.


title: $N_{\rm eff}$ as a new physics probe in the precision era of cosmology

abstract: We perform a global fit to the electroweak vertices and 4-fermion operators of the standard model effective field theory in this work using $N_{\rm eff}$ from cosmological probes, as well as data sets from colliders and low-energy experiments. We find $N_{\rm eff}$, both its current measurement and future projections, can only marginally improve the fit in both the flavor universal and the most general flavor scenarios. The resulting $1\sigma$ bound on $N_{\rm eff}$ is significantly improved from the global fit and becomes comparable to its current theoretical uncertainty, such that the latter will become important for this study at next generation experiments like future lepton colliders. $N_{\rm eff}$ from the global fit is also adopted to predict the primordial helium abundance $Y_P$, which significantly reduces the parameter space on the $Y_P$-$N_{\rm eff}$ plane. Through error propagation, we also conclude that reducing the experimental uncertainty of $Y_P$ from metal-poor galaxies down below 0.2% could play an important role in deepening our understanding on the free neutron lifetime anomaly.


title: Empirical Study of Zero-Shot NER with ChatGPT

abstract: Large language models (LLMs) exhibited powerful capability in various natural language processing tasks. This work focuses on exploring LLM performance on zero-shot information extraction, with a focus on the ChatGPT and named entity recognition (NER) task. Inspired by the remarkable reasoning capability of LLM on symbolic and arithmetic reasoning, we adapt the prevalent reasoning methods to NER and propose reasoning strategies tailored for NER. First, we explore a decomposed question-answering paradigm by breaking down the NER task into simpler subproblems by labels. Second, we propose syntactic augmentation to stimulate the model's intermediate thinking in two ways: syntactic prompting, which encourages the model to analyze the syntactic structure itself, and tool augmentation, which provides the model with the syntactic information generated by a parsing tool. Besides, we adapt self-consistency to NER by proposing a two-stage majority voting strategy, which first votes for the most consistent mentions, then the most consistent types. The proposed methods achieve remarkable improvements for zero-shot NER across seven benchmarks, including Chinese and English datasets, and on both domain-specific and general-domain scenarios. In addition, we present a comprehensive analysis of the error types with suggestions for optimization directions. We also verify the effectiveness of the proposed methods on the few-shot setting and other LLMs.


title: Evading Detection Actively: Toward Anti-Forensics against Forgery Localization

abstract: Anti-forensics seeks to eliminate or conceal traces of tampering artifacts. Typically, anti-forensic methods are designed to deceive binary detectors and persuade them to misjudge the authenticity of an image. However, to the best of our knowledge, no attempts have been made to deceive forgery detectors at the pixel level and mis-locate forged regions. Traditional adversarial attack methods cannot be directly used against forgery localization due to the following defects: 1) they tend to just naively induce the target forensic models to flip their pixel-level pristine or forged decisions; 2) their anti-forensics performance tends to be severely degraded when faced with the unseen forensic models; 3) they lose validity once the target forensic models are retrained with the anti-forensics images generated by them. To tackle the three defects, we propose SEAR (Self-supErvised Anti-foRensics), a novel self-supervised and adversarial training algorithm that effectively trains deep-learning anti-forensic models against forgery localization. SEAR sets a pretext task to reconstruct perturbation for self-supervised learning. In adversarial training, SEAR employs a forgery localization model as a supervisor to explore tampering features and constructs a deep-learning concealer to erase corresponding traces. We have conducted largescale experiments across diverse datasets. The experimental results demonstrate that, through the combination of self-supervised learning and adversarial learning, SEAR successfully deceives the state-of-the-art forgery localization methods, as well as tackle the three defects regarding traditional adversarial attack methods mentioned above.


title: Zero-noise Extrapolation Assisted with Purity for Quantum Error Mitigation

abstract: Quantum error mitigation is a technique used to post-process errors occurring in the quantum system, which reduces the expected errors and achieves higher accuracy. One method of quantum error mitigation is zero-noise extrapolation, which involves amplifying the noise and then extrapolating the observable expectation of interest back to a noise-free point. This method usually relies on the error model of the noise, as error rates for different levels of noise are assumed during the noise amplification process. In this paper, we propose that the purity of output states in noisy circuits can assist in the extrapolation process, eliminating the need for assumptions about error rates. We also introduce the quasi-polynomial model from the linearity of quantum channel for extrapolation of experimental data, which can be reduced to other proposed models. Furthermore, we verify our purity-assisted zero-noise extrapolation by performing numerical simulations and experiments on the online public quantum computation platform, Quafu, to compare it with the routine zero-noise extrapolation and virtual distillation methods. Our results demonstrate that this modified method can suppress the random fluctuation of operator expectation measurement, and effectively reduces the bias in extrapolation to a level lower than both the zero-noise extrapolation and virtual distillation methods, especially when the error rate is moderate.


title: Smart City Transportation: Deep Learning Ensemble Approach for Traffic Accident Detection

abstract: The dynamic and unpredictable nature of road traffic necessitates effective accident detection methods for enhancing safety and streamlining traffic management in smart cities. This paper offers a comprehensive exploration study of prevailing accident detection techniques, shedding light on the nuances of other state-of-the-art methodologies while providing a detailed overview of distinct traffic accident types like rear-end collisions, T-bone collisions, and frontal impact accidents. Our novel approach introduces the I3D-CONVLSTM2D model architecture, a lightweight solution tailored explicitly for accident detection in smart city traffic surveillance systems by integrating RGB frames with optical flow information. Our experimental study's empirical analysis underscores our approach's efficacy, with the I3D-CONVLSTM2D RGB + Optical-Flow (Trainable) model outperforming its counterparts, achieving an impressive 87\% Mean Average Precision (MAP). Our findings further elaborate on the challenges posed by data imbalances, particularly when working with a limited number of datasets, road structures, and traffic scenarios. Ultimately, our research illuminates the path towards a sophisticated vision-based accident detection system primed for real-time integration into edge IoT devices within smart urban infrastructures.


title: TpopT: Efficient Trainable Template Optimization on Low-Dimensional Manifolds

abstract: In scientific and engineering scenarios, a recurring task is the detection of low-dimensional families of signals or patterns. A classic family of approaches, exemplified by template matching, aims to cover the search space with a dense template bank. While simple and highly interpretable, it suffers from poor computational efficiency due to unfavorable scaling in the signal space dimensionality. In this work, we study TpopT (TemPlate OPTimization) as an alternative scalable framework for detecting low-dimensional families of signals which maintains high interpretability. We provide a theoretical analysis of the convergence of Riemannian gradient descent for TpopT, and prove that it has a superior dimension scaling to covering. We also propose a practical TpopT framework for nonparametric signal sets, which incorporates techniques of embedding and kernel interpolation, and is further configurable into a trainable network architecture by unrolled optimization. The proposed trainable TpopT exhibits significantly improved efficiency-accuracy tradeoffs for gravitational wave detection, where matched filtering is currently a method of choice. We further illustrate the general applicability of this approach with experiments on handwritten digit data.


title: The Extended Paley-Wiener Theorem over the Hardy-Sobolev Spaces

abstract: We examine how the square-integrable function subspaces are transformed using the holomorphic Fourier transform. On account of this, the extended Paley-Wiener theorem over the Hardy-Sobolev spaces is produced. The theorem also asserts that the reproducing kernel of the Hardy-Sobolev spaces can be found. We discuss the relationship between the disc and the upper half-plane.


title: Convolution quadratures based on block generalized Adams methods

abstract: This paper studies a class of convolution quadratures, well-known numerical methods for calculation of convolution integrals. In contrast to the existing counterpart, which uses the linear multistep formula or Runge-Kutta method, we employ the block generalized Adams method to discretize the underlying initial value problem. Similar to the convolution quadrature method based on the linear multistep formula, the proposed method can also be implemented on an equispaced grid. In addition, the proposed approach is as stable as the convolution quadrature based on the Runge-Kutta method, which indicates that it can accurately solve a wide range of problems without becoming unstable. We provide a detailed convergence analysis for the proposed convolution quadrature method and numerically illustrate our theoretical findings for convolution integrals with smooth and weakly singular kernels.


title: Fabrication of electronically conductive protein-heme nanowires for power harvesting

abstract: Electronically conductive protein-based materials could enable the creation of bioelectronic components and devices from sustainable and nontoxic materials, while also being well-suited to interface with biological systems, such as living cells, for biosensor applications. In addition, protein materials have other desirable properties such as inherent self-assembly and molecular recognition capabilities. However, as proteins are generally electrical insulators, the ability to render protein assemblies electronically conductive in a tailorable manner could usher in a plethora of useful materials. Here, we present an approach to fabricate electronically conductive protein nanowires by incorporating and aligning heme molecules in proximity along an ultrastable protein filament. The heme-incorporated protein nanowires demonstrated electron transfer over micrometer distances, with conductive atomic force microscopy showing individual nanowires having comparable conductance to naturally occurring bacterial nanowires. The heme-incorporated nanowires were also capable of harvesting energy from ambient humidity when deposited as multilayer films. Exposure of films to humidity produced electrical current, presumably through water molecules ionizing carboxy groups in the protein filament and creating an unbalanced total charge distribution that is enhanced by the presence of heme. A wide variety of other porphyrin molecules exist with varying electrochemical behaviors that could enable the electrical properties of protein assemblies to be tailored, paving the way to structurally- and electrically-defined protein-based bioelectronic devices.


title: Evaluating residual acceleration noise for TianQin gravitational waves observatory with an empirical magnetic field model

abstract: TianQin (TQ) project plans to deploy three satellites in space around the Earth to measure the displacement change of test masses caused by gravitational waves via laser interferometry. The requirement of the acceleration noise of the test mass is on the order of $10^{-15}~\,{\rm m}\,{\rm s}^{-2}\,{\rm Hz}^{-1/2}$ in the sensitive frequency range of TQ, %the extremely precise acceleration measurement requirements make it necessary to investigate acceleration noise due to space magnetic fields. which is so stringent that the acceleration noise caused by the interaction of the space magnetic field with the test mass needs to be investigated. In this work, by using the Tsyganenko model, a data-based empirical space magnetic field model, we obtain the magnetic field distribution around TQ's orbit spanning two solar cycles in 23 years from 1998 to 2020. With the obtained space magnetic field, we derive the distribution and amplitude spectral densities (ASDs) of the acceleration noise of TQ in 23 years. Our results reveal that the average values of the ratio of the acceleration noise cauesd by the space magnetic field to the requirements of TQ at 1 mHz ($R_{\rm 1mHz}$) and 6 mHz ($R_{\rm 6mHz}$) are 0.123$\pm$0.052 and 0.027$\pm$0.013, respectively. The occurence probabilities of $R_{\rm 1mHz}>0.2$ and $>0.3$ are only 7.9% and 1.2%, respectively, and $R_{\rm 6mHz}$ never exceeds 0.2.

categories: astro-ph.IM astro-ph.EP astro-ph.SR gr-qc physics.space-ph


title: Hyperspectral Image Fusion via Logarithmic Low-rank Tensor Ring Decomposition

abstract: Integrating a low-spatial-resolution hyperspectral image (LR-HSI) with a high-spatial-resolution multispectral image (HR-MSI) is recognized as a valid method for acquiring HR-HSI. Among the current fusion approaches, the tensor ring (TR) decomposition-based method has received growing attention owing to its superior performance on preserving the spatial-spectral correlation. Furthermore, the low-rank property in some TR factors has been exploited via the matrix nuclear norm regularization along mode-2. On the other hand, the tensor nuclear norm (TNN)-based approaches have recently demonstrated to be more efficient on keeping high-dimensional low-rank structures in tensor recovery. Here, we study the low-rankness of TR factors from the TNN perspective and consider the mode-2 logarithmic TNN (LTNN) on each TR factor. A novel fusion model is proposed by incorporating this LTNN regularization and the weighted total variation which is to promote the continuity of HR-HSI in the spatial-spectral domain. Meanwhile, we have devised a highly efficient proximal alternating minimization algorithm to solve the proposed model. The experimental results indicate that our method improves the visual quality and exceeds the existing state-of-the-art fusion approaches with respect to various quantitative metrics.


title: Symmetrical SyncMap for Imbalanced General Chunking Problems

abstract: Recently, SyncMap pioneered an approach to learn complex structures from sequences as well as adapt to any changes in underlying structures. This is achieved by using only nonlinear dynamical equations inspired by neuron group behaviors, i.e., without loss functions. Here we propose Symmetrical SyncMap that goes beyond the original work to show how to create dynamical equations and attractor-repeller points which are stable over the long run, even dealing with imbalanced continual general chunking problems (CGCPs). The main idea is to apply equal updates from negative and positive feedback loops by symmetrical activation. We then introduce the concept of memory window to allow for more positive updates. Our algorithm surpasses or ties other unsupervised state-of-the-art baselines in all 12 imbalanced CGCPs with various difficulties, including dynamically changing ones. To verify its performance in real-world scenarios, we conduct experiments on several well-studied structure learning problems. The proposed method surpasses substantially other methods in 3 out of 4 scenarios, suggesting that symmetrical activation plays a critical role in uncovering topological structures and even hierarchies encoded in temporal data.


title: TRANSOM: An Efficient Fault-Tolerant System for Training LLMs

abstract: Large language models (LLMs) with hundreds of billions or trillions of parameters, represented by chatGPT, have achieved profound impact on various fields. However, training LLMs with super-large-scale parameters requires large high-performance GPU clusters and long training periods lasting for months. Due to the inevitable hardware and software failures in large-scale clusters, maintaining uninterrupted and long-duration training is extremely challenging. As a result, A substantial amount of training time is devoted to task checkpoint saving and loading, task rescheduling and restart, and task manual anomaly checks, which greatly harms the overall training efficiency. To address these issues, we propose TRANSOM, a novel fault-tolerant LLM training system. In this work, we design three key subsystems: the training pipeline automatic fault tolerance and recovery mechanism named Transom Operator and Launcher (TOL), the training task multi-dimensional metric automatic anomaly detection system named Transom Eagle Eye (TEE), and the training checkpoint asynchronous access automatic fault tolerance and recovery technology named Transom Checkpoint Engine (TCE). Here, TOL manages the lifecycle of training tasks, while TEE is responsible for task monitoring and anomaly reporting. TEE detects training anomalies and reports them to TOL, who automatically enters the fault tolerance strategy to eliminate abnormal nodes and restart the training task. And the asynchronous checkpoint saving and loading functionality provided by TCE greatly shorten the fault tolerance overhead. The experimental results indicate that TRANSOM significantly enhances the efficiency of large-scale LLM training on clusters. Specifically, the pre-training time for GPT3-175B has been reduced by 28%, while checkpoint saving and loading performance have improved by a factor of 20.


title: Improving Large Language Model Fine-tuning for Solving Math Problems

abstract: Despite their success in many natural language tasks, solving math problems remains a significant challenge for large language models (LLMs). A large gap exists between LLMs' pass-at-one and pass-at-N performance in solving math problems, suggesting LLMs might be close to finding correct solutions, motivating our exploration of fine-tuning methods to unlock LLMs' performance. Using the challenging MATH dataset, we investigate three fine-tuning strategies: (1) solution fine-tuning, where we fine-tune to generate a detailed solution for a given math problem; (2) solution-cluster re-ranking, where the LLM is fine-tuned as a solution verifier/evaluator to choose among generated candidate solution clusters; (3) multi-task sequential fine-tuning, which integrates both solution generation and evaluation tasks together efficiently to enhance the LLM performance. With these methods, we present a thorough empirical study on a series of PaLM 2 models and find: (1) The quality and style of the step-by-step solutions used for fine-tuning can make a significant impact on the model performance; (2) While solution re-ranking and majority voting are both effective for improving the model performance when used separately, they can also be used together for an even greater performance boost; (3) Multi-task fine-tuning that sequentially separates the solution generation and evaluation tasks can offer improved performance compared with the solution fine-tuning baseline. Guided by these insights, we design a fine-tuning recipe that yields approximately 58.8% accuracy on the MATH dataset with fine-tuned PaLM 2-L models, an 11.2% accuracy improvement over the few-shot performance of pre-trained PaLM 2-L model with majority voting.


title: Evaluation of transplant benefits with the U.S. Scientific Registry of Transplant Recipients by semiparametric regression of mean residual life

abstract: Kidney transplantation is the most effective renal replacement therapy for end stage renal disease patients. With the severe shortage of kidney supplies and for the clinical effectiveness of transplantation, patient's life expectancy post transplantation is used to prioritize patients for transplantation; however, severe comorbidity conditions and old age are the most dominant factors that negatively impact post-transplantation life expectancy, effectively precluding sick or old patients from receiving transplants. It would be crucial to design objective measures to quantify the transplantation benefit by comparing the mean residual life with and without a transplant, after adjusting for comorbidity and demographic conditions. To address this urgent need, we propose a new class of semiparametric covariate-dependent mean residual life models. Our method estimates covariate effects semiparametrically efficiently and the mean residual life function nonparametrically, enabling us to predict the residual life increment potential for any given patient. Our method potentially leads to a more fair system that prioritizes patients who would have the largest residual life gains. Our analysis of the kidney transplant data from the U.S. Scientific Registry of Transplant Recipients also suggests that a single index of covariates summarize well the impacts of multiple covariates, which may facilitate interpretations of each covariate's effect. Our subgroup analysis further disclosed inequalities in survival gains across groups defined by race, gender and insurance type (reflecting socioeconomic status).


title: FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Models

abstract: Large Language Models (LLMs), such as ChatGPT, LLaMA, GLM, and PaLM, have exhibited remarkable performances across various tasks in recent years. However, LLMs face two main challenges in real-world applications. One challenge is that training LLMs consumes vast computing resources, preventing LLMs from being adopted by small and medium-sized enterprises with limited computing resources. Another is that training LLM requires a large amount of high-quality data, which are often scattered among enterprises. To address these challenges, we propose FATE-LLM, an industrial-grade federated learning framework for large language models. FATE-LLM (1) facilitates federated learning for large language models (coined FedLLM); (2) promotes efficient training of FedLLM using parameter-efficient fine-tuning methods; (3) protects the intellectual property of LLMs; (4) preserves data privacy during training and inference through privacy-preserving mechanisms. We release the code of FATE-LLM at https://github.com/FederatedAI/FATE-LLM to facilitate the research of FedLLM and enable a broad range of industrial applications.


title: EfficientOCR: An Extensible, Open-Source Package for Efficiently Digitizing World Knowledge

abstract: Billions of public domain documents remain trapped in hard copy or lack an accurate digitization. Modern natural language processing methods cannot be used to index, retrieve, and summarize their texts; conduct computational textual analyses; or extract information for statistical analyses, and these texts cannot be incorporated into language model training. Given the diversity and sheer quantity of public domain texts, liberating them at scale requires optical character recognition (OCR) that is accurate, extremely cheap to deploy, and sample-efficient to customize to novel collections, languages, and character sets. Existing OCR engines, largely designed for small-scale commercial applications in high resource languages, often fall short of these requirements. EffOCR (EfficientOCR), a novel open-source OCR package, meets both the computational and sample efficiency requirements for liberating texts at scale by abandoning the sequence-to-sequence architecture typically used for OCR, which takes representations from a learned vision model as inputs to a learned language model. Instead, EffOCR models OCR as a character or word-level image retrieval problem. EffOCR is cheap and sample efficient to train, as the model only needs to learn characters' visual appearance and not how they are used in sequence to form language. Models in the EffOCR model zoo can be deployed off-the-shelf with only a few lines of code. Importantly, EffOCR also allows for easy, sample efficient customization with a simple model training interface and minimal labeling requirements due to its sample efficiency. We illustrate the utility of EffOCR by cheaply and accurately digitizing 20 million historical U.S. newspaper scans, evaluating zero-shot performance on randomly selected documents from the U.S. National Archives, and accurately digitizing Japanese documents for which all other OCR solutions failed.


title: EAR-Net: Pursuing End-to-End Absolute Rotations from Multi-View Images

abstract: Absolute rotation estimation is an important topic in 3D computer vision. Existing works in literature generally employ a multi-stage (at least two-stage) estimation strategy where multiple independent operations (feature matching, two-view rotation estimation, and rotation averaging) are implemented sequentially. However, such a multi-stage strategy inevitably leads to the accumulation of the errors caused by each involved operation, and degrades its final estimation on global rotations accordingly. To address this problem, we propose an End-to-end method for estimating Absolution Rotations from multi-view images based on deep neural Networks, called EAR-Net. The proposed EAR-Net consists of an epipolar confidence graph construction module and a confidence-aware rotation averaging module. The epipolar confidence graph construction module is explored to simultaneously predict pairwise relative rotations among the input images and their corresponding confidences, resulting in a weighted graph (called epipolar confidence graph). Based on this graph, the confidence-aware rotation averaging module, which is differentiable, is explored to predict the absolute rotations. Thanks to the introduced confidences of the relative rotations, the proposed EAR-Net could effectively handle outlier cases. Experimental results on three public datasets demonstrate that EAR-Net outperforms the state-of-the-art methods by a large margin in terms of accuracy and speed.


title: Group-Orthogonal Subsampling for Hierarchical Data Based on Linear Mixed Models

abstract: Hierarchical data analysis is crucial in various fields for making discoveries. The linear mixed model is often used for training hierarchical data, but its parameter estimation is computationally expensive, especially with big data. Subsampling techniques have been developed to address this challenge. However, most existing subsampling methods assume homogeneous data and do not consider the possible heterogeneity in hierarchical data. To address this limitation, we develop a new approach called group-orthogonal subsampling (GOSS) for selecting informative subsets of hierarchical data that may exhibit heterogeneity. GOSS selects subdata with balanced data size among groups and combinatorial orthogonality within each group, resulting in subdata that are $D$- and $A$-optimal for building linear mixed models. Estimators of parameters trained on GOSS subdata are consistent and asymptotically normal. GOSS is shown to be numerically appealing via simulations and a real data application. Theoretical proofs, R codes, and supplementary numerical results are accessible online as Supplementary Materials.


title: Fast and Low-Cost Approximate Multiplier for FPGAs using Dynamic Reconfiguration

abstract: Multipliers are widely-used arithmetic operators in digital signal processing and machine learning circuits. Due to their relatively high complexity, they can have high latency and be a significant source of power consumption. One strategy to alleviate these limitations is to use approximate computing. This paper thus introduces an original FPGA-based approximate multiplier specifically optimized for machine learning computations. It utilizes dynamically reconfigurable lookup table (LUT) primitives in AMD-Xilinx technology to realize the core part of the computations. The paper provides an in-depth analysis of the hardware architecture, implementation outcomes, and accuracy evaluations of the multiplier proposed in INT8 precision. Implementation results on an AMD-Xilinx Kintex Ultrascale+ FPGA demonstrate remarkable savings of 64% and 67% in LUT utilization for signed multiplication and multiply-and-accumulation configurations, respectively, when compared to the standard Xilinx multiplier core. Accuracy measurements on four popular deep learning (DL) benchmarks indicate a minimal average accuracy decrease of less than 0.29% during post-training deployment, with the maximum reduction staying less than 0.33%. The source code of this work is available on GitHub.


title: NASH: A Simple Unified Framework of Structured Pruning for Accelerating Encoder-Decoder Language Models

abstract: Structured pruning methods have proven effective in reducing the model size and accelerating inference speed in various network architectures such as Transformers. Despite the versatility of encoder-decoder models in numerous NLP tasks, the structured pruning methods on such models are relatively less explored compared to encoder-only models. In this study, we investigate the behavior of the structured pruning of the encoder-decoder models in the decoupled pruning perspective of the encoder and decoder component, respectively. Our findings highlight two insights: (1) the number of decoder layers is the dominant factor of inference speed, and (2) low sparsity in the pruned encoder network enhances generation quality. Motivated by these findings, we propose a simple and effective framework, NASH, that narrows the encoder and shortens the decoder networks of encoder-decoder models. Extensive experiments on diverse generation and inference tasks validate the effectiveness of our method in both speedup and output quality.


title: A fiber-type optomechanical array using high-Q microbottle resonators

abstract: We demonstrate a fiber-type optomechanical array consisting of elastically interconnected silica microbottle resonators with high-Q optical and mechanical modes. In total, fifty optomechanical resonators fabricated by fine glass processing are uniformly arrayed on a silica fiber. Evanescent coupling of a tapered optical fiber to an arbitrary resonator allows for highly sensitive readout and efficient actuation of mechanical motion at an arbitrary position in the array. Phonon propagation through the fifty microbottles is achieved by both linearly and parametrically driving a mechanical mode at one end and by detecting it at the other end. This optomechanical array is scalable, tunable, and lithography-free and can be extended to fiber-based sensory applications with structural flexibility and operability in various environments.


title: Latent Conservative Objective Models for Data-Driven Crystal Structure Prediction

abstract: In computational chemistry, crystal structure prediction (CSP) is an optimization problem that involves discovering the lowest energy stable crystal structure for a given chemical formula. This problem is challenging as it requires discovering globally optimal designs with the lowest energies on complex manifolds. One approach to tackle this problem involves building simulators based on density functional theory (DFT) followed by running search in simulation, but these simulators are painfully slow. In this paper, we study present and study an alternate, data-driven approach to crystal structure prediction: instead of directly searching for the most stable structures in simulation, we train a surrogate model of the crystal formation energy from a database of existing crystal structures, and then optimize this model with respect to the parameters of the crystal structure. This surrogate model is trained to be conservative so as to prevent exploitation of its errors by the optimizer. To handle optimization in the non-Euclidean space of crystal structures, we first utilize a state-of-the-art graph diffusion auto-encoder (CD-VAE) to convert a crystal structure into a vector-based search space and then optimize a conservative surrogate model of the crystal energy, trained on top of this vector representation. We show that our approach, dubbed LCOMs (latent conservative objective models), performs comparably to the best current approaches in terms of success rate of structure prediction, while also drastically reducing computational cost.


title: Storage and retrieval of electromagnetic waves in a metasurface based on bound states in the continuum by conductivity modulation

abstract: In this study, we develop a time-varying metasurface based on the bound states in the continuum (BIC) with variable conductors, to store electromagnetic waves. The storage and retrieval of electromagnetic waves are demonstrated numerically through dynamic switching between quasi-BIC and BIC states by modulating the variable conductors. The storage efficiency exhibits oscillatory behavior with respect to the timing of storage and retrieval. These behaviors can be attributed to the interference of a resonant mode and a static mode that is formed by direct current. In addition, the storage efficiency of a single-layer metasurface can reach 35% under ideal conditions.