title: Benchmarking Resource Usage for Efficient Distributed Deep Learning

abstract: Deep learning (DL) workflows demand an ever-increasing budget of compute and energy in order to achieve outsized gains. Neural architecture searches, hyperparameter sweeps, and rapid prototyping consume immense resources that can prevent resource-constrained researchers from experimenting with large models and carry considerable environmental impact. As such, it becomes essential to understand how different deep neural networks (DNNs) and training leverage increasing compute and energy resources -- especially specialized computationally-intensive models across different domains and applications. In this paper, we conduct over 3,400 experiments training an array of deep networks representing various domains/tasks -- natural language processing, computer vision, and chemistry -- on up to 424 graphics processing units (GPUs). During training, our experiments systematically vary compute resource characteristics and energy-saving mechanisms such as power utilization and GPU clock rate limits to capture and illustrate the different trade-offs and scaling behaviors each representative model exhibits under various resource and energy-constrained regimes. We fit power law models that describe how training time scales with available compute resources and energy constraints. We anticipate that these findings will help inform and guide high-performance computing providers in optimizing resource utilization, by selectively reducing energy consumption for different deep learning tasks/workflows with minimal impact on training.


title: On the Stability of a Wormhole in the Maximally-Extended Reissner-Nordstr\"om Solution

abstract: We consider the stability of the maximally-extended Reissner-Nordstr\"om solution in a Minkowski, de Sitter, or anti-de Sitter background. In a broad class of situations, prior work has shown that spherically symmetric perturbations from a massless scalar field cause the inner horizon of an RN black hole to become singular and collapse. Even if this is the case, it may still be possible for an observer to travel through the inner horizon before it fully collapses, thus violating strong cosmic censorship. In this work, we show that the collapse of the inner horizon and the occurrence of a singularity along the inner horizon are sufficient to prevent an observer from accessing the white hole regions and the parallel universe regions of the maximally extended RN space-time. Thus, if an observer passes through the inner horizon, they will inevitably hit the central singularity. Throughout this article, we use natural units where c = G = 4 {\pi} {\epsilon}_0 = 1.


title: CoordX: Accelerating Implicit Neural Representation with a Split MLP Architecture

abstract: Implicit neural representations with multi-layer perceptrons (MLPs) have recently gained prominence for a wide variety of tasks such as novel view synthesis and 3D object representation and rendering. However, a significant challenge with these representations is that both training and inference with an MLP over a large number of input coordinates to learn and represent an image, video, or 3D object, require large amounts of computation and incur long processing times. In this work, we aim to accelerate inference and training of coordinate-based MLPs for implicit neural representations by proposing a new split MLP architecture, CoordX. With CoordX, the initial layers are split to learn each dimension of the input coordinates separately. The intermediate features are then fused by the last layers to generate the learned signal at the corresponding coordinate point. This significantly reduces the amount of computation required and leads to large speedups in training and inference, while achieving similar accuracy as the baseline MLP. This approach thus aims at first learning functions that are a decomposition of the original signal and then fusing them to generate the learned signal. Our proposed architecture can be generally used for many implicit neural representation tasks with no additional memory overheads. We demonstrate a speedup of up to 2.92x compared to the baseline model for image, video, and 3D shape representation and rendering tasks.


title: A Simple Guard for Learned Optimizers

abstract: If the trend of learned components eventually outperforming their hand-crafted version continues, learned optimizers will eventually outperform hand-crafted optimizers like SGD or Adam. Even if learned optimizers (L2Os) eventually outpace hand-crafted ones in practice however, they are still not provably convergent and might fail out of distribution. These are the questions addressed here. Currently, learned optimizers frequently outperform generic hand-crafted optimizers (such as gradient descent) at the beginning of learning but they generally plateau after some time while the generic algorithms continue to make progress and often overtake the learned algorithm as Aesop's tortoise which overtakes the hare. L2Os also still have a difficult time generalizing out of distribution. Heaton et al. proposed Safeguarded L2O (GL2O) which can take a learned optimizer and safeguard it with a generic learning algorithm so that by conditionally switching between the two, the resulting algorithm is provably convergent. We propose a new class of Safeguarded L2O, called Loss-Guarded L2O (LGL2O), which is both conceptually simpler and computationally less expensive. The guarding mechanism decides solely based on the expected future loss value of both optimizers. Furthermore, we show theoretical proof of LGL2O's convergence guarantee and empirical results comparing to GL2O and other baselines showing that it combines the best of both L2O and SGD and that in practice converges much better than GL2O.


title: Towards Safe Reinforcement Learning with a Safety Editor Policy

abstract: We consider the safe reinforcement learning (RL) problem of maximizing utility with extremely low constraint violation rates. Assuming no prior knowledge or pre-training of the environment safety model given a task, an agent has to learn, via exploration, which states and actions are safe. A popular approach in this line of research is to combine a model-free RL algorithm with the Lagrangian method to adjust the weight of the constraint reward relative to the utility reward dynamically. It relies on a single policy to handle the conflict between utility and constraint rewards, which is often challenging. We present SEditor, a two-policy approach that learns a safety editor policy transforming potentially unsafe actions proposed by a utility maximizer policy into safe ones. The safety editor is trained to maximize the constraint reward while minimizing a hinge loss of the utility state-action values before and after an action is edited. SEditor extends existing safety layer designs that assume simplified safety models, to general safe RL scenarios where the safety model can in theory be arbitrarily complex. As a first-order method, it is easy to implement and efficient for both inference and training. On 12 Safety Gym tasks and 2 safe racing tasks, SEditor obtains much a higher overall safety-weighted-utility (SWU) score than the baselines, and demonstrates outstanding utility performance with constraint violation rates as low as once per 2k time steps, even in obstacle-dense environments. On some tasks, this low violation rate is up to 200 times lower than that of an unconstrained RL method with similar utility performance. Code is available at https://github.com/hnyu/seditor.


title: Systematic Training and Testing for Machine Learning Using Combinatorial Interaction Testing

abstract: This paper demonstrates the systematic use of combinatorial coverage for selecting and characterizing test and training sets for machine learning models. The presented work adapts combinatorial interaction testing, which has been successfully leveraged in identifying faults in software testing, to characterize data used in machine learning. The MNIST hand-written digits data is used to demonstrate that combinatorial coverage can be used to select test sets that stress machine learning model performance, to select training sets that lead to robust model performance, and to select data for fine-tuning models to new domains. Thus, the results posit combinatorial coverage as a holistic approach to training and testing for machine learning. In contrast to prior work which has focused on the use of coverage in regard to the internal of neural networks, this paper considers coverage over simple features derived from inputs and outputs. Thus, this paper addresses the case where the supplier of test and training sets for machine learning models does not have intellectual property rights to the models themselves. Finally, the paper addresses prior criticism of combinatorial coverage and provides a rebuttal which advocates the use of coverage metrics in machine learning applications.


title: The Krengel's theorem for compact operators between locally solid vector lattices

abstract: Suppose $X$ is a locally solid vector lattice. It is known that there are several non-equivalent notions for compact operators on $X$. Furthermore, notion of the $AM$-property in $X$ as an extension for the $AM$-spaces in Banach lattices has been considered, recently. In this paper, we establish a variant of the known Krengel's theorem for different types of compact operators between locally solid vector lattices.


title: Bayesian Nonlinear Models for Repeated Measurement Data: An Overview, Implementation, and Applications

abstract: Nonlinear mixed effects models have become a standard platform for analysis when data is in the form of continuous and repeated measurements of subjects from a population of interest, while temporal profiles of subjects commonly follow a nonlinear tendency. While frequentist analysis of nonlinear mixed effects models has a long history, Bayesian analysis of the models has received comparatively little attention until the late 1980s due primarily to the time-consuming nature of Bayesian computation. Since the early 1990s Bayesian approaches for the models began to emerge to leverage rapid developments in computing power, and recently, have received significant attention due to (1) superiority to quantify the uncertainty of parameter estimation; (2) utility to incorporate prior knowledge into the models; and (3) flexibility to match exactly the increasing complexity of scientific research arising from diverse industrial and academic fields. This review article presents an overview of modeling strategies to implement Bayesian approaches for the nonlinear mixed effects models, ranging from designing a scientific question out of real-life problems to practical computations.


title: Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval

abstract: Retrieval-based language models (R-LM) model the probability of natural language text by combining a standard language model (LM) with examples retrieved from an external datastore at test time. While effective, a major bottleneck of using these models in practice is the computationally costly datastore search, which can be performed as frequently as every time step. In this paper, we present RetoMaton - retrieval automaton - which approximates the datastore search, based on (1) saving pointers between consecutive datastore entries, and (2) clustering of entries into "states". This effectively results in a weighted finite automaton built on top of the datastore, instead of representing the datastore as a flat list. The creation of the automaton is unsupervised, and a RetoMaton can be constructed from any text collection: either the original training corpus or from another domain. Traversing this automaton at inference time, in parallel to the LM inference, reduces its perplexity by up to 1.85, or alternatively saves up to 83% of the nearest neighbor searches over $k$NN-LM (Khandelwal et al., 2020) without hurting perplexity. Our code and trained models are available at https://github.com/neulab/retomaton .


title: Vexillary Grothendieck Polynomials via Bumpless Pipe Dreams

abstract: Recent work of Pechenik, Speyer, and Weigandt proved a formula for the degree of any Grothendieck polynomial. A distinct formula for the degree of vexillary Grothendieck polynomials was proven by Rajchgot, Robichaux, and Weigandt. We give a new proof of Pechenik, Speyer, and Weigandt's formula in the special case of vexillary permutations and characterize the set of bumpless pipe dreams which contribute maximal degree terms to the Grothendieck polynomial in this case. Furthermore, we use this characterization to draw connections between the Pechenik-Speyer-Weigandt and Rajchgot-Robichaux-Weigandt formulas. We also use bumpless pipe dreams to prove new results about the support of vexillary Grothendieck polynomials, addressing special cases of conjectures of M\'esz\'aros, Setiabrata, and St. Dizier.


title: FedGCN: Convergence-Communication Tradeoffs in Federated Training of Graph Convolutional Networks

abstract: Methods for training models on graphs distributed across multiple clients have recently grown in popularity, due to the size of these graphs as well as regulations on keeping data where it is generated. However, the cross-client edges naturally exist among clients. Thus, distributed methods for training a model on a single graph incur either significant communication overhead between clients or a loss of available information to the training. We introduce the Federated Graph Convolutional Network (FedGCN) algorithm, which uses federated learning to train GCN models for semi-supervised node classification with fast convergence and little communication. Compared to prior methods that require extra communication among clients at each training round, FedGCN clients only communicate with the central server in one pre-training step, greatly reducing communication costs and allowing the use of homomorphic encryption to further enhance privacy. We theoretically analyze the tradeoff between FedGCN's convergence rate and communication cost under different data distributions. Experimental results show that our FedGCN algorithm achieves better model accuracy with 51.7% faster convergence on average and at least 100X less communication compared to prior work.


title: Do You Need the Entropy Reward (in Practice)?

abstract: Maximum entropy (MaxEnt) RL maximizes a combination of the original task reward and an entropy reward. It is believed that the regularization imposed by entropy, on both policy improvement and policy evaluation, together contributes to good exploration, training convergence, and robustness of learned policies. This paper takes a closer look at entropy as an intrinsic reward, by conducting various ablation studies on soft actor-critic (SAC), a popular representative of MaxEnt RL. Our findings reveal that in general, entropy rewards should be applied with caution to policy evaluation. On one hand, the entropy reward, like any other intrinsic reward, could obscure the main task reward if it is not properly managed. We identify some failure cases of the entropy reward especially in episodic Markov decision processes (MDPs), where it could cause the policy to be overly optimistic or pessimistic. On the other hand, our large-scale empirical study shows that using entropy regularization alone in policy improvement, leads to comparable or even better performance and robustness than using it in both policy improvement and policy evaluation. Based on these observations, we recommend either normalizing the entropy reward to a zero mean (SACZero), or simply removing it from policy evaluation (SACLite) for better practical results.


title: A class of non-reversible hypercube long-range random walks and Bernoulli autoregression

abstract: We study a large class of long-range random walks which take values on the vertices of an N dimensional hypercube. These processes are connected with multivariate Bernoulli autoregression.


title: Any-Play: An Intrinsic Augmentation for Zero-Shot Coordination

abstract: Cooperative artificial intelligence with human or superhuman proficiency in collaborative tasks stands at the frontier of machine learning research. Prior work has tended to evaluate cooperative AI performance under the restrictive paradigms of self-play (teams composed of agents trained together) and cross-play (teams of agents trained independently but using the same algorithm). Recent work has indicated that AI optimized for these narrow settings may make for undesirable collaborators in the real-world. We formalize an alternative criteria for evaluating cooperative AI, referred to as inter-algorithm cross-play, where agents are evaluated on teaming performance with all other agents within an experiment pool with no assumption of algorithmic similarities between agents. We show that existing state-of-the-art cooperative AI algorithms, such as Other-Play and Off-Belief Learning, under-perform in this paradigm. We propose the Any-Play learning augmentation -- a multi-agent extension of diversity-based intrinsic rewards for zero-shot coordination (ZSC) -- for generalizing self-play-based algorithms to the inter-algorithm cross-play setting. We apply the Any-Play learning augmentation to the Simplified Action Decoder (SAD) and demonstrate state-of-the-art performance in the collaborative card game Hanabi.


title: Mobile Robot Manipulation using Pure Object Detection

abstract: This paper addresses the problem of mobile robot manipulation using object detection. Our approach uses detection and control as complimentary functions that learn from real-world interactions. We develop an end-to-end manipulation method based solely on detection and introduce Task-focused Few-shot Object Detection (TFOD) to learn new objects and settings. Our robot collects its own training data and automatically determines when to retrain detection to improve performance across various subtasks (e.g., grasping). Notably, detection training is low-cost, and our robot learns to manipulate new objects using as few as four clicks of annotation. In physical experiments, our robot learns visual control from a single click of annotation and a novel update formulation, manipulates new objects in clutter and other mobile settings, and achieves state-of-the-art results on an existing visual servo control and depth estimation benchmark. Finally, we develop a TFOD Benchmark to support future object detection research for robotics: https://github.com/griffbr/tfod.


title: Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey

abstract: While commonsense knowledge acquisition and reasoning has traditionally been a core research topic in the knowledge representation and reasoning community, recent years have seen a surge of interest in the natural language processing community in developing pre-trained models and testing their ability to address a variety of newly designed commonsense knowledge reasoning and generation tasks. This paper presents a survey of these tasks, discusses the strengths and weaknesses of state-of-the-art pre-trained models for commonsense reasoning and generation as revealed by these tasks, and reflects on future research directions.


title: Discriminating Defense Against DDoS Attacks; a Novel Approach

abstract: A recent paper (circa 2020) by Osterwile et al., entitled "21 Years of Distributed Denial of Service: A Call to Action", states: "We are falling behind in the war against distributed denial-of-service attacks. Unless we act now, the future of the Internet could be at stake." And an earlier (circa 2007) paper by Peng et al. states: "a key challenge for the defense [against DDoS attacks] is how to discriminate legitimate requests for service from malicious access attempts." This challenge has not been met yet, which is, arguably, a major reason for the dire situation described by Osterwile et al. -- thirteen years later. This paper attempts to meet an approximation to this challenge, by enabling a a site to define the kind of messages that it considers important, and by introducing an unambiguous criterion of discrimination between messages that a given site considers important, and all other messages sent to it. Two anti-DDoS mechanisms based on this criterion are introduced in this paper. One of these relies on lightweight support by routers; and the other one does not.


title: Certifying Model Accuracy under Distribution Shifts

abstract: Certified robustness in machine learning has primarily focused on adversarial perturbations of the input with a fixed attack budget for each point in the data distribution. In this work, we present provable robustness guarantees on the accuracy of a model under bounded Wasserstein shifts of the data distribution. We show that a simple procedure that randomizes the input of the model within a transformation space is provably robust to distributional shifts under the transformation. Our framework allows the datum-specific perturbation size to vary across different points in the input distribution and is general enough to include fixed-sized perturbations as well. Our certificates produce guaranteed lower bounds on the performance of the model for any (natural or adversarial) shift of the input distribution within a Wasserstein ball around the original distribution. We apply our technique to: (i) certify robustness against natural (non-adversarial) transformations of images such as color shifts, hue shifts and changes in brightness and saturation, (ii) certify robustness against adversarial shifts of the input distribution, and (iii) show provable lower bounds (hardness results) on the performance of models trained on so-called "unlearnable" datasets that have been poisoned to interfere with model training.


title: Family-wise error rate control in Gaussian graphical model selection via Distributionally Robust Optimization

abstract: Recently, a special case of precision matrix estimation based on a distributionally robust optimization (DRO) framework has been shown to be equivalent to the graphical lasso. From this formulation, a method for choosing the regularization term, i.e., for graphical model selection, was proposed. In this work, we establish a theoretical connection between the confidence level of graphical model selection via the DRO formulation and the asymptotic family-wise error rate of estimating false edges. Simulation experiments and real data analyses illustrate the utility of the asymptotic family-wise error rate control behavior even in finite samples.


title: Ehrhart Theory of Paving and Panhandle Matroids

abstract: We show that the base polytope $P_M$ of any paving matroid $M$ can be systematically obtained from a hypersimplex by slicing off certain subpolytopes, namely base polytopes of lattice path matroids corresponding to panhandle-shaped Ferrers diagrams. We calculate the Ehrhart polynomials of these matroids and consequently write down the Ehrhart polynomial of $P_M$, starting with Katzman's formula for the Ehrhart polynomial of a hypersimplex. The method builds on and generalizes Ferroni's work on sparse paving matroids. Combinatorially, our construction corresponds to constructing a uniform matroid from a paving matroid by iterating the operation of stressed-hyperplane relaxation introduced by Ferroni, Nasr, and Vecchi, which generalizes the standard matroid-theoretic notion of circuit-hyperplane relaxation. We present evidence that panhandle matroids are Ehrhart positive and describe a conjectured combinatorial formula involving chain forests and Eulerian numbers from which Ehrhart positivity of panhandle matroids will follow. As an application of the main result, we calculate the Ehrhart polynomials of matroids associated with Steiner systems and finite projective planes, and show that they depend only on their design-theoretic parameters: for example, while projective planes of the same order need not have isomorphic matroids, their base polytopes must be Ehrhart equivalent.


title: Hyperbolic boundaries vs. hyperbolic groups

abstract: The aim of these notes is to connect the theory of hyperbolic and relatively hyperbolic groups to the theory of manifolds and Kleinian groups. We survey some of the extensive work that has been done in the field, and explain many examples. These notes are based on lectures given by the third author at CIRM in the Summer of 2018.


title: Twin nucleation in Ti: A study using nudged elastic band (NEB) method

abstract: Capturing twin nucleation in full-field crystal plasticity is a long-standing problem in materials science. The challenge resides mainly in the biased regional lattice transformation associated with twin formation in defiance of its obedience to a threshold stress law which could be fulfilled in regions where twinning is deferred. Hence, determining a favorable site for nucleation of a twin variant remains a daunting task. We hypothesized that this site-specific nucleation is sensitive to the prior atomic structure of the lattice so twin embryos form in regions where the lattice transformation energy is minimum. Thus, quantifying the local strain energy required to trigger a stable twin underscores the non-pseudo-slip behavior of twin nucleation and growth. We performed atomistic calculations based on the nudged elastic band method to identify the minimum energy path and activation energy associated with {10-12} twin nucleation in titanium. Results of calculations demonstrate that the role of stress and atomic structure in twin nucleation could be understood in terms of the minimum energy path, energy barrier, and relaxed energy. Remarkably, for symmetric tilt grain boundaries, a linear correlation between the nucleation stress and grain boundary energy was observed.


title: To the geometry of spaces of plurisubharmonic functions on a K\"ahler manifold

abstract: Consider a compact K\"ahler manifold $(X,\omega)$ and the space $\cal E(X,\omega)=\cal E$ of $\omega$--plurisubharmonic functions of full Monge--Amp\`ere mass on it. We introduce a quantity $\rho[u,v]$ to measure the distance between $u, v\in\cal E$; $\rho[u,v]$ is not a number but rather a decreasing function on a certain interval $(0,V)\subset\mathbb R$. We explore properties of $\rho[u,v]$, and using them we study Lagrangians and associated energy spaces of $\omega$--plurisubharmonic functions. Many results here generalize Darvas's findings about his metrics $d_\chi$.


title: Comparison of measurement systems for assessing number- and mass-based particle filtration efficiency

abstract: The particle filtration efficiency (PFE) of a respirator or face mask is one of its key properties. While the physics of particle filtration results in the PFE being size-dependent, measurement standards are specified using a single, integrated PFE, for simplicity. This integrated PFE is commonly defined with respect to either the number (NBFE) or mass (MBFE) distribution of particles as a function of size. This relationship is non-trivial; it is influenced by both the shape of the particle distribution and the fact that multiple practical definitions of particle size are used. This manuscript discusses the relationship between NBFE and MBFE in detail, providing a guide to practitioners. Our discussion begins with a theoretical discussion of the underlying principles. We then present experimental results for a database of size-resolved PFE (SPFE) measurements for over 900 candidate respirators and filter media, including filter media with systematically varied properties and commercial samples that span the 20%-99.8% MBFE. The observed relationships between NBFE and MBFE are discussed in terms of the most-penetrating particle size (MPPS) and charge state of the media. For the NaCl particles used here, we observed that MBFEs were greater than NBFEs for charged materials and vice versa for uncharged materials. This relationship is observed because a shift from NBFE to MBFE weights the distribution towards larger sizes, while charged materials shift the MPPS to smaller sizes. Results are validated by comparing the output of a pair of TSI 8130As, a common instrument used in gauging standards compliance, to that of MBFEs computed from a system capable of measuring SPFE.


title: Piezo-optomechanical cantilever modulators for VLSI visible photonics

abstract: Visible-wavelength very large-scale integration (VLSI) photonic circuits have potential to play important roles in quantum information and sensing technologies. The realization of scalable, high-speed, and low-loss photonic mesh circuits depends on reliable and well-engineered visible photonic components. Here we report a low-voltage optical phase shifter based on piezo-actuated mechanical cantilevers, fabricated on a CMOS compatible, 200 mm wafer-based visible photonics platform. We show linear phase and amplitude modulation with 6 V$_{\pi}$-cm in differential operation, -1.5 dB to -2 dB insertion loss, and up to 40 dB contrast in the 700 nm - 780 nm range. By adjusting selected cantilever parameters, we demonstrate a low-displacement and a high-displacement device, both exhibiting a nearly flat frequency response from DC to a peak mechanical resonance at 23 MHz and 6.8 MHz respectively, which through resonant enhancement of Q~40, further decreases the operating voltage down to 0.15 V$_{\pi}$-cm.


title: Generalized LKF transformations for $N$-point fermion correlators in QED

abstract: Within the worldline approach to quantum electrodynamics (QED), a change of the photon's covariant gauge parameter $\xi$ is investigated to analyse the non-perturbative gauge dependence of the configuration space fermion correlation functions, deriving a generalization of the Landau-Kalatnikov-Fradkin transformations (LKFt). These transformations reveal how the non-perturbative gauge dependence of position space amplitudes can be absorbed into a multiplicative exponential factor.


title: Asymptotic behaviour of penalized robust estimators in logistic regression when dimension increases

abstract: Penalized $M-$estimators for logistic regression models have been previously study for fixed dimension in order to obtain sparse statistical models and automatic variable selection. In this paper, we derive asymptotic results for penalized $M-$estimators when the dimension $p$ grows to infinity with the sample size $n$. Specifically, we obtain consistency and rates of convergence results, for some choices of the penalty function. Moreover, we prove that these estimators consistently select variables with probability tending to 1 and derive their asymptotic distribution.


title: Decoding Merged Color-Surface Codes and Finding Fault-Tolerant Clifford Circuits Using Solvers for Satisfiability Modulo Theories

abstract: Universal fault-tolerant quantum computers will require the use of efficient protocols to implement encoded operations necessary in the execution of algorithms. In this work, we show how solvers for satisfiability modulo theories (SMT solvers) can be used to automate the construction of Clifford circuits with certain fault-tolerance properties and we apply our techniques to a fault-tolerant magic-state-preparation protocol. Part of the protocol requires converting magic states encoded in the color code to magic states encoded in the surface code. Since the teleportation step involves decoding a color code merged with a surface code, we develop a decoding algorithm that is applicable to such codes.


title: Extracting Finite Automata from RNNs Using State Merging

abstract: One way to interpret the behavior of a blackbox recurrent neural network (RNN) is to extract from it a more interpretable discrete computational model, like a finite state machine, that captures its behavior. In this work, we propose a new method for extracting finite automata from RNNs inspired by the state merging paradigm from grammatical inference. We demonstrate the effectiveness of our method on the Tomita languages benchmark, where we find that it is able to extract faithful automata from RNNs trained on all languages in the benchmark. We find that extraction performance is aided by the number of data provided during the extraction process, as well as, curiously, whether the RNN model is trained for additional epochs after perfectly learning its target language. We use our method to analyze this phenomenon, finding that training beyond convergence is useful because it leads to compression of the internal state space of the RNN. This finding demonstrates how our method can be used for interpretability and analysis of trained RNN models.


title: Unfolding Orthotubes with a Dual Hamiltonian Path

abstract: An orthotube consists of orthogonal boxes (e.g., unit cubes) glued face-to-face to form a path. In 1998, Biedl et al. showed that every orthotube has a grid unfolding: a cutting along edges of the boxes so that the surface unfolds into a connected planar shape without overlap. We give a new algorithmic grid unfolding of orthotubes with the additional property that the rectangular faces are attached in a single path -- a Hamiltonian path on the rectangular faces of the orthotube surface.


title: The Arithmetic Partial Derivative

abstract: The arithmetic partial derivative (with respect to a prime $p$) is a function from the set of integers that sends $p$ to 1 and satisfies the Leibniz rule. In this paper, we prove that the $p$-adic valuation of the sequence of higher order partial derivatives is eventually periodic. We also prove a criterion to determine when an integer has integral anti-partial derivatives. As an application, we show that there are infinitely many integers with exactly $n$ integral anti-partial derivatives for any nonnegative integer $n$.


title: The Complexity of Approximate Pattern Matching on De Bruijn Graphs

abstract: Aligning a sequence to a walk in a labeled graph is a problem of fundamental importance to Computational Biology. For finding a walk in an arbitrary graph with $|E|$ edges that exactly matches a pattern of length $m$, a lower bound based on the Strong Exponential Time Hypothesis (SETH) implies an algorithm significantly faster than $O(|E|m)$ time is unlikely [Equi et al., ICALP 2019]. However, for many special graphs, such as de Bruijn graphs, the problem can be solved in linear time [Bowe et al., WABI 2012]. For approximate matching, the picture is more complex. When edits (substitutions, insertions, and deletions) are only allowed to the pattern, or when the graph is acyclic, the problem is again solvable in $O(|E|m)$ time. When edits are allowed to arbitrary cyclic graphs, the problem becomes NP-complete, even on binary alphabets [Jain et al., RECOMB 2019]. These results hold even when edits are restricted to only substitutions. The complexity of approximate pattern matching on de Bruijn graphs remained open. We investigate this problem and show that the properties that make de Bruijn graphs amenable to efficient exact pattern matching do not extend to approximate matching, even when restricted to the substitutions only case with alphabet size four. We prove that determining the existence of a matching walk in a de Bruijn graph is NP-complete when substitutions are allowed to the graph. In addition, we demonstrate that an algorithm significantly faster than $O(|E|m)$ is unlikely for de Bruijn graphs in the case where only substitutions are allowed to the pattern. This stands in contrast to pattern-to-text matching where exact matching is solvable in linear time, like on de Bruijn graphs, but approximate matching under substitutions is solvable in subquadratic $O(n\sqrt{m})$ time, where $n$ is the text's length [Abrahamson, SIAM J. Computing 1987].


title: Ab-initio investigation of Er3+ defects in tungsten disulfide

abstract: We use density functional theory (DFT) to explore the physical properties of an $Er_{ W}$ point defect in monolayer $WS_{ 2}$. Our calculations indicate that electrons localize at the dangling bonds associated with a tungsten vacancy ($V_{W}$) and at the $Er^{ 3+}$ ion site, even in the presence of a net negative charge in the supercell. The system features a set of intra-gap defect states, some of which are reminiscent of those present in isolated $Er^{ 3+}$ ions. In both instances, the level of hybridization is low, i.e., orbitals show either strong Er or W character. Through the calculation of the absorption spectrum as a function of wavelength, we identify a broad set of transitions, including one possibly consistent with the $Er^{ 3+}$ $4I_{ 15/2} \rightarrow 4I_{ 13/2}$ observed in other hosts. Combined with the low native concentration of spin-active nuclei as well as the two-dimensional nature of the host, these properties reveal $Er:WS_{ 2}$ as a potential platform for realizing spin qubits that can be subsequently integrated with other nanoscale optoelectronic devices.


title: A narrow bandwidth extreme ultra-violet light source for time- and angle-resolved photoemission spectroscopy

abstract: Here we present a high repetition rate, narrow band-width, extreme ultraviolet (XUV) photon source for time- and angle-resolved photoemission spectroscopy (tr-ARPES). The narrow band width pulses $\Delta E=9, 14, 18$ meV for photon energies $h\nu=10.8, 18.1, 25.3$ eV are generated through High Harmonic Generation (HHG) using ultra-violet (UV) drive pulses with relatively long pulse lengths (461 fs). The HHG setup employs an annuluar drive beam in a tight focusing geometry at a repetition rate of 250 kHz. Photon energy selection is provided by a series of selectable multilayer bandpass mirrors and thin film filters, thus avoiding any time broadening introduced by single grating monochromators. A two stage optical-parametric amplifier provides $< 100$ fs tunable pump pulses from 0.65 $\mu$m to 9 $\mu$m. The narrow bandwidth performance of the light source is demonstrated through ARPES measurements on a series of quantum materials including the high-temperature superconductor Bi-2212, WSe$_2$ and graphene.


title: One-loop Amplitudes in the Worldline Formalism

abstract: We summarize recent progress in applying the worldline formalism to the analytic calculation of one-loop N-point amplitudes. This string-inspired approach is well-adapted to avoiding some of the calculational inefficiencies of the standard Feynman diagram approach, most notably by providing master formulas that sum over diagrams differing only by the position of external legs and/or internal propagators. We illustrate the mathematical challenge involved with the low-energy limit of the N-photon amplitudes in scalar and spinor QED, and then present an algorithm that, in principle, solves this problem for the much more difficult case of the N-point amplitudes at full momentum in phi^3 theory. The method is based on the algebra of inverse derivatives in the Hilbert space of periodic functions orthogonal to the constant ones, in which the Bernoulli numbers and polynomials play a central role.


title: Regular Interior Solutions to the Solution of Kerr which Satisfy the Weak and the Strong Energy Conditions

abstract: The line element of a class of solutions which match to the solution of Kerr on an oblate spheroid if the two functions $ F(r)$ and $H(r)$ on which it depends satisfy certain matching conditions is presented. The non vanishing components of the Ricci tensor $R_{\mu\nu}$, the Ricci scalar $ R$, the second order curvature invariant $K$, the eigenvalues of the Ricci tensor, the energy density $\mu$, the tangential pressure $P_{\perp}$, and the quantity $\mu+P_{\perp}$ are calculated. A function $F(r)$ is given for which $R$ and $K$ and therefore the solutions are regular. The function $ H(r) $ should be such that the solution it gives satisfies at least the Weak Energy Conditions (WEC). Several $H(r)$ are given explicitly for which the resulting solutions satisfy the WEC and also the Strong Energy Conditions (SEC) and the graphs of their $\mu$, $P_{\perp}$ and $\mu+P_{\perp}$ for certain values of their parameters are presented. It is shown that all solutions of the class are anisotropic fluid solutions and that there are no perfect fluid solutions in the class.


title: Gate-based Quantum Computing for Protein Design

abstract: Protein design is a technique to engineer proteins by modifying their sequence to obtain novel functionalities. In this method, amino acids in the sequence are permutated to find the low energy states satisfying the configuration. However, exploring all possible combinations of amino acids is generally impossible to achieve on conventional computers due to the exponential growth of possibilities with the number of designable sites. Thus, sampling methods are currently used as a conventional approach to address the protein design problems. Recently, quantum computation methods have shown the potential to solve similar types of problems. In the present work, we use the general idea of Grover's algorithm, a pure quantum computation method, to design circuits at the gate-based level and address the protein design problem. In our quantum algorithms, we use custom pair-wise energy tables consisting of eight different amino acids. Also, the distance reciprocals between designable sites are included in calculating energies in the circuits. Due to the noisy state of current quantum computers, we mainly use quantum computer simulators for this study. However, a very simple version of our circuits is implemented on real quantum devices to examine their capabilities to run these algorithms. Our results show that using $\mathcal{O}(\sqrt N)$ iterations, the circuits find the correct results among all $N$ possibilities, providing the expected quadratic speed up of Grover's algorithm over classical methods.


title: On the Global Convergence of Particle Swarm Optimization Methods

abstract: In this paper we provide a rigorous convergence analysis for the renowned particle swarm optimization method by using tools from stochastic calculus and the analysis of partial differential equations. Based on a time-continuous formulation of the particle dynamics as a system of stochastic differential equations, we establish convergence to a global minimizer of a possibly nonconvex and nonsmooth objective function in two steps. First, we prove consensus formation of an associated mean-field dynamics by analyzing the time-evolution of the variance of the particle distribution. We then show that this consensus is close to a global minimizer by employing the asymptotic Laplace principle and a tractability condition on the energy landscape of the objective function. These results allow for the usage of memory mechanisms, and hold for a rich class of objectives provided certain conditions of well-preparation of the hyperparameters and the initial datum. In a second step, at least for the case without memory effects, we provide a quantitative result about the mean-field approximation of particle swarm optimization, which specifies the convergence of the interacting particle system to the associated mean-field limit. Combining these two results allows for global convergence guarantees of the numerical particle swarm optimization method with provable polynomial complexity. To demonstrate the applicability of the method we propose an efficient and parallelizable implementation, which is tested in particular on a competitive and well-understood high-dimensional benchmark problem in machine learning.


title: Matter Wave Analog of a Fiber-Optic Gyroscope

abstract: Confining the propagating wavepackets of an atom interferometer inside a waveguide can substantially reduce the size of the device while preserving high sensitivity. We have realized a two-dimensional Sagnac atom interferometer in which Bose-condensed $^{87}$Rb atoms propagate within a tight waveguide formed by a collimated laser beam, a matter wave analog of the fiber optic gyro (FOG). The condensate is split, reflected, and recombined with a series of Bragg pulses while the waveguide moves transversely so that the wavepacket trajectories enclose an area. Delta-kick cooling is used to prepare low-density atomic wavepackets with a temperature of \SI{3}{\nano \kelvin}. The low density reduces the impact of interatomic interactions, while the low temperature limits the expansion of the wavepacket during the interferometer cycle. The effective enclosed area is $0.8 \text{mm}^2$ with an average fringe contrast of 20\% and underlying contrast up to 60\%. The main source of the reduced average contrast is phase noise caused by mechanical vibrations of the optical components. We present the first measurement of Allan deviation for such an atom rotation sensor, showing that the interferometer phase noise falls with averaging time $\tau$ as $\tau^{-1/2}$ for $\tau$ up to 10,000 seconds. The statistical noise falls below the Earth rotation rate after 30 minutes of averaging.


title: Explaining Reinforcement Learning Policies through Counterfactual Trajectories

abstract: In order for humans to confidently decide where to employ RL agents for real-world tasks, a human developer must validate that the agent will perform well at test-time. Some policy interpretability methods facilitate this by capturing the policy's decision making in a set of agent rollouts. However, even the most informative trajectories of training time behavior may give little insight into the agent's behavior out of distribution. In contrast, our method conveys how the agent performs under distribution shifts by showing the agent's behavior across a wider trajectory distribution. We generate these trajectories by guiding the agent to more diverse unseen states and showing the agent's behavior there. In a user study, we demonstrate that our method enables users to score better than baseline methods on one of two agent validation tasks.


title: Nanoparticle-enhanced Multifunctional Nanocarbons as Metal-ion Battery and Capacitor Anodes and Supercapacitor Electrodes -- Review

abstract: As renewable energy is becoming a critical energy source to meet the global demand, electrochemical energy storage devices become indispensable for the efficient energy storage and reliable supply. The electrode material is the key factor determining the energy storage capacity and the power delivery of the devices. Carbon-based materials are emerging as a viable candidate for electrodes, yet their low energy densities impede the development of advanced energy storage materials. Nanoparticle decoration of the carbon structures is one of the most promising and easy-to-implement a strategy to enhance the charge-storage performance of carbon-based electrodes. Decoration by nanoparticles of metals, metal oxides, nitrides, carbides, phosphides, chalcogenides, and bimetallic components lead to significant enhancements in the structural and electronic properties, pore refinement, charge storage, and charge transfer kinetics of both pristine and doped carbon structures, thereby making their performance promising for next-generation energy storage devices. This review covers the state-of-art nanoparticle decorated nanocarbons for battery, supercapacitor, and metal-ion capacitor applications. A critical analysis of the elemental composition, structure, associated physico-chemical properties and performance relationships of nanoparticle-decorated nanocarbon electrodes is provided as well to inform the future development of the next generation of advanced energy storage materials, devices, and systems.


title: Using Dynamic Binary Instrumentation to Detect Failures in Robotics Software

abstract: Autonomous and Robotics Systems (ARSs) are widespread, complex, and increasingly coming into contact with the public. Many of these systems are safety-critical, and it is vital to detect software errors to protect against harm. We propose a family of novel techniques to detect unusual program executions and incorrect program behavior. We model execution behavior by collecting low-level signals at run time and using those signals to build machine learning models. These models can identify previously-unseen executions that are more likely to exhibit errors. We describe a tractable approach for collecting dynamic binary runtime signals on ARSs, allowing the systems to absorb most of the overhead from dynamic instrumentation. The architecture of ARSs is particularly well-adapted to hiding the overhead from instrumentation. We demonstrate the efficiency of these approaches on ARDUPILOT -- a popular open-source autopilot software system -- and HUSKY -- an unmanned ground vehicle -- in simulation. We instrument executions to gather data from which we build supervised machine learning models of executions and evaluate the accuracy of these models. We also analyze the amount of training data needed to develop models with various degrees of accuracy, measure the overhead added to executions that use the analysis tool, and analyze which runtime signals are most useful for detecting unusual behavior on the program under test. In addition, we analyze the effects of timing delays on the functional behavior of ARSs.


title: Flashlight: Enabling Innovation in Tools for Machine Learning

abstract: As the computational requirements for machine learning systems and the size and complexity of machine learning frameworks increases, essential framework innovation has become challenging. While computational needs have driven recent compiler, networking, and hardware advancements, utilization of those advancements by machine learning tools is occurring at a slower pace. This is in part due to the difficulties involved in prototyping new computational paradigms with existing frameworks. Large frameworks prioritize machine learning researchers and practitioners as end users and pay comparatively little attention to systems researchers who can push frameworks forward -- we argue that both are equally important stakeholders. We introduce Flashlight, an open-source library built to spur innovation in machine learning tools and systems by prioritizing open, modular, customizable internals and state-of-the-art, research-ready models and training setups across a variety of domains. Flashlight allows systems researchers to rapidly prototype and experiment with novel ideas in machine learning computation and has low overhead, competing with and often outperforming other popular machine learning frameworks. We see Flashlight as a tool enabling research that can benefit widely used libraries downstream and bring machine learning and systems researchers closer together. Flashlight is available at https://github.com/flashlight/flashlight .


title: A Fourier method for the determination of focus for telescopes with stars

abstract: We introduce a Fourier method (Fm) for the determination of best focus for telescopes with stars. Our method fits a power function, that we will derive in this paper, to a set of images taken as a function of focuser position. The best focus position is where the power is maximum. Fm was first tested with small refractor and Schmidt-Cassegrain (SCT) telescopes. After the successful small telescope tests, we then tested Fm with a 2 m Ritchey-Chr\'etien-Coud\'e (RCC). Our tests show that Fm is immune to the problems inherent in the popular half-flux diameter method.


title: Improving Federated Learning Face Recognition via Privacy-Agnostic Clusters

abstract: The growing public concerns on data privacy in face recognition can be greatly addressed by the federated learning (FL) paradigm. However, conventional FL methods perform poorly due to the uniqueness of the task: broadcasting class centers among clients is crucial for recognition performances but leads to privacy leakage. To resolve the privacy-utility paradox, this work proposes PrivacyFace, a framework largely improves the federated learning face recognition via communicating auxiliary and privacy-agnostic information among clients. PrivacyFace mainly consists of two components: First, a practical Differentially Private Local Clustering (DPLC) mechanism is proposed to distill sanitized clusters from local class centers. Second, a consensus-aware recognition loss subsequently encourages global consensuses among clients, which ergo results in more discriminative features. The proposed framework is mathematically proved to be differentially private, introducing a lightweight overhead as well as yielding prominent performance boosts (\textit{e.g.}, +9.63\% and +10.26\% for TAR@FAR=1e-4 on IJB-B and IJB-C respectively). Extensive experiments and ablation studies on a large-scale dataset have demonstrated the efficacy and practicability of our method.


title: Symbolic-Numeric Integration of Univariate Expressions based on Sparse Regression

abstract: Most computer algebra systems (CAS) support symbolic integration as core functionality. The majority of the integration packages use a combination of heuristic algebraic and rule-based (integration table) methods. In this paper, we present a hybrid (symbolic-numeric) methodology to calculate the indefinite integrals of univariate expressions. The primary motivation for this work is to add symbolic integration functionality to a modern CAS (the symbolic manipulation packages of SciML, the Scientific Machine Learning ecosystem of the Julia programming language), which is mainly designed toward numerical and machine learning applications and has a different set of features than traditional CAS. The symbolic part of our method is based on the combination of candidate terms generation (borrowed from the Homotopy operators theory) with rule-based expression transformations provided by the underlying CAS. The numeric part is based on sparse-regression, a component of Sparse Identification of Nonlinear Dynamics (SINDy) technique. We show that this system can solve a large variety of common integration problems using only a few dozen basic integration rules.


title: ScaLA: Accelerating Adaptation of Pre-Trained Transformer-Based Language Models via Efficient Large-Batch Adversarial Noise

abstract: In recent years, large pre-trained Transformer-based language models have led to dramatic improvements in many natural language understanding tasks. To train these models with increasing sizes, many neural network practitioners attempt to increase the batch sizes in order to leverage multiple GPUs to improve training speed. However, increasing the batch size often makes the optimization more difficult, leading to slow convergence or poor generalization that can require orders of magnitude more training time to achieve the same model quality. In this paper, we explore the steepness of the loss landscape of large-batch optimization for adapting pre-trained Transformer-based language models to domain-specific tasks and find that it tends to be highly complex and irregular, posing challenges to generalization on downstream tasks. To tackle this challenge, we propose ScaLA, a novel and efficient method to accelerate the adaptation speed of pre-trained transformer networks. Different from prior methods, we take a sequential game-theoretic approach by adding lightweight adversarial noise into large-batch optimization, which significantly improves adaptation speed while preserving model generalization. Experiment results show that ScaLA attains 2.7--9.8$\times$ adaptation speedups over the baseline for GLUE on BERT-base and RoBERTa-large, while achieving comparable and sometimes higher accuracy than the state-of-the-art large-batch optimization methods. Finally, we also address the theoretical aspect of large-batch optimization with adversarial noise and provide a theoretical convergence rate analysis for ScaLA using techniques for analyzing non-convex saddle-point problems.


title: Distributed Dimension Reduction for Distributed Massive MIMO C-RAN with Finite Fronthaul Capacity

abstract: The use of a large excess of service antennas brings a variety of performance benefits to distributed MIMO C-RAN, but the corresponding high fronthaul data loads can be problematic in practical systems with limited fronthaul capacity. In this work we propose the use of lossy dimension reduction, applied locally at each remote radio head (RRH), to reduce this fronthaul traffic. We first consider the uplink, and the case where each RRH applies a linear dimension reduction filter to its multi-antenna received signal vector. It is shown that under a joint mutual information criteria, the optimal dimension reduction filters are given by a variant of the conditional Karhunen-Loeve transform, with a stationary point found using block co-ordinate ascent. These filters are then modified such that each RRH can calculate its own dimension reduction filter in a decentralised manner, using knowledge only of its own instantaneous channel and network slow fading coefficients. We then show that in TDD systems these dimension reduction filters can be re-used as part of a two-stage reduced dimension downlink precoding scheme. Analysis and numerical results demonstrate that the proposed approach can significantly reduce both uplink and downlink fronthaul traffic whilst incurring very little loss in MIMO performance.


title: Chaotic Diffusion of Dissipative Solitons: From Anti-Persistent Random Walk to Hidden Markov Models

abstract: In previous publications, we showed that the incremental process of the chaotic diffusion of dissipative solitons in a prototypical complex Ginzburg-Landau equation, known, e.g., from nonlinear optics, is governed by a simple Markov process leading to an Anti-Persistent Random Walk of motion or by a more complex Hidden Markov Model with continuous output densities. In this article, we reveal the transition between these two models by studying the soliton dynamics in dependence on the main bifurcation parameter of the Ginzburg-Landau equation and identify the underlying hidden Markov processes. These models capture the non-trivial decay of correlations in jump widths and symbol sequences representing the soliton motion, the statistics of anti-persistent walk episodes, and the multimodal density of the jump widths. We demonstrate that there exists a physically meaningful reduction of the dynamics of an infinite-dimensional deterministic system to one of a probabilistic finite state machine and provide a deeper understanding of the soliton dynamics under parameter variation of the underlying nonlinear dynamics.


title: On some topics around the Wadge rank $\omega_2$

abstract: Kechris and Martin showed that the Wadge rank of the $\omega$-th level of the decreasing difference hierarchy of coanalytic sets is $\omega_2$ under the axiom of determinacy. In this article, we give an alternative proof of the Kechris-Martin theorem, by understanding the $\omega$-th level of the decreasing difference hierarchy of coanalytic sets as the (relative) hyperarithmetical processes with finite mind-changes. Based on this viewpiont, we also examine the gap between the increasing and decreasing difference hierarchies of coanalytic sets by relating them to the $\Pi^1_1$- and $\Sigma^1_1$-least number principles, respectively. We also analyze Weihrauch degrees of related principles.


title: Understanding the Effect of Lead Iodide Excess on the Performance of Methylammonium Lead Iodide Perovskite Solar Cells

abstract: The presence of unreacted lead iodide in organic-inorganic lead halide perovskite solar cells is widely correlated with an increase in power conversion efficiency. We investigate the mechanism for this increase by identifying the role of surfaces and interfaces present between methylammonium lead iodide perovskite films and excess lead iodide. We show how type I and II band alignments arising under different conditions result in either passivation of surface defects or hole injection. Through first-principles simulations of solid-solid interfaces, we find that lead iodide captures holes from methylammonium lead iodide and modulates the formation of defects in the perovskite, affecting recombination. Using surface-sensitive optical spectroscopy techniques, such as transient reflectance and time-resolved photoluminescence, we show how excess lead iodide affects the diffusion and surface recombination velocity of charge carriers in methylammonium lead iodide films. Our coupled experimental and theoretical results elucidate the role of excess lead iodide in perovskite solar cells.


title: Diffraction properties of lights with transverse orbital angular momentum

abstract: Spatiotemporal optical vortex (STOV) is a unique optical vortex with phase singularity in the space-time domain and the photons in a STOV can carry transverse orbital angular momentum (OAM). The STOV shows many fantastic properties which are worth exploring. Here, we theoretically and experimentally study the diffraction property of STOV, which is a fundamental wave phenomenon. The diffraction behaviors of STOVs are obviously affected by the transverse OAM. The diffraction patterns of STOV pulses diffracted by a grating show multi-lobe structure with each gap corresponding to 1 topological charge. The diffraction properties of lights with transverse OAM are demonstrated clearly and help us understanding the physical properties of STOV, which will be of special applications, such as the realization of fast detection of STOVs with different topological charges, which may pay the way for STOV based optical communication.


title: Retroformer: Pushing the Limits of Interpretable End-to-end Retrosynthesis Transformer

abstract: Retrosynthesis prediction is one of the fundamental challenges in organic synthesis. The task is to predict the reactants given a core product. With the advancement of machine learning, computer-aided synthesis planning has gained increasing interest. Numerous methods were proposed to solve this problem with different levels of dependency on additional chemical knowledge. In this paper, we propose Retroformer, a novel Transformer-based architecture for retrosynthesis prediction without relying on any cheminformatics tools for molecule editing. Via the proposed local attention head, the model can jointly encode the molecular sequence and graph, and efficiently exchange information between the local reactive region and the global reaction context. Retroformer reaches the new state-of-the-art accuracy for the end-to-end template-free retrosynthesis, and improves over many strong baselines on better molecule and reaction validity. In addition, its generative procedure is highly interpretable and controllable. Overall, Retroformer pushes the limits of the reaction reasoning ability of deep generative models.


title: Deep Learning application for stellar parameters determination: I- Constraining the hyperparameters

abstract: Machine Learning is an efficient method for analyzing and interpreting the increasing amount of astronomical data that is available. In this study, we show, a pedagogical approach that should benefit anyone willing to experiment with Deep Learning techniques in the context of stellar parameters determination. Utilizing the Convolutional Neural Network architecture, we give a step by step overview of how to select the optimal parameters for deriving the most accurate values for the stellar parameters of stars: T$_{\rm{eff}}$, $\log g$, [X/H], and $v_e \sin i$. Synthetic spectra with random noise were used to constrain this method and to mimic the observations. We found that each stellar parameter requires a different combination of network hyperparameters and the maximum accuracy reached depends on this combination, as well as, the Signal to Noise ratio of the observations, and the architecture of the network. We also show that this technique can be applied to other spectral types in different wavelength ranges after the technique has been optimized.


title: An Indirect Rate-Distortion Characterization for Semantic Sources: General Model and the Case of Gaussian Observation

abstract: A new source model, which consists of an intrinsic state part and an extrinsic observation part, is proposed and its information-theoretic characterization, namely its rate-distortion function, is defined and analyzed. Such a source model is motivated by the recent surge of interest in the semantic aspect of information: the intrinsic state corresponds to the semantic feature of the source, which in general is not observable but can only be inferred from the extrinsic observation. There are two distortion measures, one between the intrinsic state and its reproduction, and the other between the extrinsic observation and its reproduction. Under a given code rate, the tradeoff between these two distortion measures is characterized by the rate-distortion function, which is solved via the indirect rate-distortion theory and is termed as the semantic rate-distortion function of the source. As an application of the general model and its analysis, the case of Gaussian extrinsic observation is studied, assuming a linear relationship between the intrinsic state and the extrinsic observation, under a quadratic distortion structure. The semantic rate-distortion function is shown to be the solution of a convex programming programming problem with respect to an error covariance matrix, and a reverse water-filling type of solution is provided when the model further satisfies a diagonalizability condition.


title: Stability of hypercontractivity, the logarithmic Sobolev inequality, and Talagrand's cost inequality

abstract: We provide deficit estimates for Nelson's hypercontractivity inequality, the logarithmic Sobolev inequality, and Talagrand's transportation cost inequality under the restriction that the inputs are semi-log-subharmonic, semi-log-convex, or semi-log-concave. In particular, our result on the logarithmic Sobolev inequality complements a recently obtained result by Eldan, Lehec and Shenfeld concerning a deficit estimate for inputs with small covariance. Similarly, our result on Talagrand's transportation cost inequality complements and, for a large class of semi-log-concave inputs, improves a deficit estimate recently proved by Mikulincer. Our deficit estimates for hypercontractivity will be obtained by using a flow monotonicity scheme built on the Fokker--Planck equation, and our deficit estimates for the logarithmic Sobolev inequality will be derived as a corollary. For Talagrand's inequality, we use an optimal transportation argument. An appealing feature of our framework is robustness and this allows us to derive deficit estimates for the hypercontracivity inequality associated with the Hamilton--Jacobi equation, the Poincar\'e inequality, and for Beckner's inequality.


title: Total positivity and conjugacy classes

abstract: In this paper, we study the interaction between the totally positive monoid $G_{\ge 0}$ attached to a connected reductive group $G$ with a pinning and the conjugacy classes in $G$. In particular, we study how a conjugacy class meets the various cells of $G_{\ge0}$. We also state a conjectural Jordan decomposition for $G_{\ge0}$ and prove it in some special cases.


title: Interconnect Parasitics and Partitioning in Fully-Analog In-Memory Computing Architectures

abstract: Fully-analog in-memory computing (IMC) architectures that implement both matrix-vector multiplication and non-linear vector operations within the same memory array have shown promising performance benefits over conventional IMC systems due to the removal of energy-hungry signal conversion units. However, maintaining the computation in the analog domain for the entire deep neural network (DNN) comes with potential sensitivity to interconnect parasitics. Thus, in this paper, we investigate the effect of wire parasitic resistance and capacitance on the accuracy of DNN models deployed on fully-analog IMC architectures. Moreover, we propose a partitioning mechanism to alleviate the impact of the parasitic while keeping the computation in the analog domain through dividing large arrays into multiple partitions. The SPICE circuit simulation results for a 400 X 120 X 84 X 10 DNN model deployed on a fully-analog IMC circuit show that a 94.84% accuracy could be achieved for MNIST classification application with 16, 8, and 8 horizontal partitions, as well as 8, 8, and 1 vertical partitions for first, second, and third layers of the DNN, respectively, which is comparable to the ~97% accuracy realized by digital implementation on CPU. It is shown that accuracy benefits are achieved at the cost of higher power consumption due to the extra circuitry required for handling partitioning.


title: Effective decorrelation of Hecke eigenforms

abstract: In this paper, we prove effective quantitative decorrelation of values of two Hecke eigenforms as the weight goes to infinity. As consequences, we get an effective version of equidistribution of mass and zeros of certain linear combinations of Hecke eigenforms.


title: Collaborative Learning in General Graphs with Limited Memorization: Complexity, Learnability, and Reliability

abstract: We consider a K-armed bandit problem in general graphs where agents are arbitrarily connected and each of them has limited memorizing capabilities and communication bandwidth. The goal is to let each of the agents eventually learn the best arm. It is assumed in these studies that the communication graph should be complete or well-structured, whereas such an assumption is not always valid in practice. Furthermore, limited memorization and communication bandwidth also restrict the collaborations of the agents, since the agents memorize and communicate very few experiences. Additionally, an agent may be corrupted to share falsified experiences to its peers, while the resource limit in terms of memorization and communication may considerably restrict the reliability of the learning process. To address the above issues, we propose a three-staged collaborative learning algorithm. In each step, the agents share their latest experiences with each other through light-weight random walks in a general communication graph, and then make decisions on which arms to pull according to the recommendations received from their peers. The agents finally update their adoptions (i.e., preferences to the arms) based on the reward obtained by pulling the arms. Our theoretical analysis shows that, when there are a sufficient number of agents participating in the collaborative learning process, all the agents eventually learn the best arm with high probability, even with limited memorizing capabilities and light-weight communications. We also reveal in our theoretical analysis the upper bound on the number of corrupted agents our algorithm can tolerate. The efficacy of our proposed three-staged collaborative learning algorithm is finally verified by extensive experiments on both synthetic and real datasets.


title: Scintillation decay-time constants for alpha particles and electrons in liquid xenon

abstract: Understanding liquid xenon scintillation and ionization processes is of great interest to improve analysis methods in current and future detectors. In this paper, we investigate the dynamics of the scintillation process for excitation by $\mathcal{O}$(10 keV) electrons from a $^{83m}$Kr source and $\mathcal{O}$(6 MeV) $\alpha$-particles from a $^{222}$Rn source, both mixed with the xenon target. The single photon sampling method is used to record photon arrival times in order to obtain the corresponding time distributions for different applied electric fields between about 0.8 V/cm to 1.2 kV/cm. Energy and field dependences of the signals which are observed in the results are discussed.


title: Fair Stable Matching Meets Correlated Preferences

abstract: The stable matching problem sets the economic foundation of several practical applications ranging from school choice and medical residency to ridesharing and refugee placement. It is concerned with finding a matching between two disjoint sets of agents wherein no pair of agents prefer each other to their matched partners. The Deferred Acceptance (DA) algorithm is an elegant procedure that guarantees a stable matching for any input; however, its outcome may be unfair as it always favors one side by returning a matching that is optimal for one side (say men) and pessimal for the other side (say women). A desirable fairness notion is minimizing the sex-equality cost, i.e. the difference between the total rankings of both sides. Computing such stable matchings is a strongly NP-hard problem, which raises the question of what tractable algorithms to adopt in practice. We conduct a series of empirical evaluations on the properties of sex-equal stable matchings when preferences of agents on both sides are correlated. Our empirical results suggest that under correlated preferences, the DA algorithm returns stable matchings with low sex-equality cost, which further confirms its broad use in many practical applications.


title: Temporal Analysis of GRB Precursors in the Third Swift-BAT Catalog

abstract: We select 52 long gamma-ray bursts(GRBs) having the precursor activity in the third Swift-BAT catalog. Each episode shown in both the precursors and the main bursts is fitted by the Norris function. We systematically analyze the temporal properties for both the precursors and the main bursts. We do not find any significant difference between the temporal profile of the precursor and that of the main burst. The photon count of the precursor is related to that of the main burst. It is indicated that the precursor and the main burst might have the same physical origin, as the precursor and the main burst follow the same $\tau_p-\omega$ relation. However, we do not find the explicit relation between the energy release of the precursor and the quiescent time. Some theoretical models, such as the fallback collapsar scenario and the jet-cocoon scenario, may be helpful to explain the GRB-precursor phenomena.


title: Angular topological superfluid and topological vortex in an ultracold Fermi gas

abstract: We show that pairing in an ultracold Fermi gas under spin-orbital-angular-momentum coupling (SOAMC) can acquire topological characters encoded in the quantized angular degrees of freedom. The resulting topological superfluid is the angular analog of its counterpart in a one-dimensional Fermi gas with spin-orbit coupling, but characterized by a Zak phase defined in the angular-momentum space. Upon tuning the SOAMC parameters, a topological phase transition occurs, which is accompanied by the closing of the quasiparticle excitation gap. Remarkably, a topological vortex state can also be stabilized by deforming the Fermi surface, which is topologically non-trivial in both the coordinate and angular-momentum space, offering interesting potentials for applications in quantum information and quantum control. We discuss how the topological phase transition and the exotic vortex state can be detected experimentally.


title: Counterfactual Plans under Distributional Ambiguity

abstract: Counterfactual explanations are attracting significant attention due to the flourishing applications of machine learning models in consequential domains. A counterfactual plan consists of multiple possibilities to modify a given instance so that the model's prediction will be altered. As the predictive model can be updated subject to the future arrival of new data, a counterfactual plan may become ineffective or infeasible with respect to the future values of the model parameters. In this work, we study the counterfactual plans under model uncertainty, in which the distribution of the model parameters is partially prescribed using only the first- and second-moment information. First, we propose an uncertainty quantification tool to compute the lower and upper bounds of the probability of validity for any given counterfactual plan. We then provide corrective methods to adjust the counterfactual plan to improve the validity measure. The numerical experiments validate our bounds and demonstrate that our correction increases the robustness of the counterfactual plans in different real-world datasets.


title: Achieving Efficient Distributed Machine Learning Using a Novel Non-Linear Class of Aggregation Functions

abstract: Distributed machine learning (DML) over time-varying networks can be an enabler for emerging decentralized ML applications such as autonomous driving and drone fleeting. However, the commonly used weighted arithmetic mean model aggregation function in existing DML systems can result in high model loss, low model accuracy, and slow convergence speed over time-varying networks. To address this issue, in this paper, we propose a novel non-linear class of model aggregation functions to achieve efficient DML over time-varying networks. Instead of taking a linear aggregation of neighboring models as most existing studies do, our mechanism uses a nonlinear aggregation, a weighted power-p mean (WPM), as the aggregation function of local models from neighbors. The subsequent optimizing steps are taken using mirror descent defined by a Bregman divergence that maintains convergence to optimality. In this paper, we analyze properties of the WPM and rigorously prove convergence properties of our aggregation mechanism. Additionally, through extensive experiments, we show that when p > 1, our design significantly improves the convergence speed of the model and the scalability of DML under time-varying networks compared with arithmetic mean aggregation functions, with little additional computation overhead.


title: A Context-Integrated Transformer-Based Neural Network for Auction Design

abstract: One of the central problems in auction design is developing an incentive-compatible mechanism that maximizes the auctioneer's expected revenue. While theoretical approaches have encountered bottlenecks in multi-item auctions, recently, there has been much progress on finding the optimal mechanism through deep learning. However, these works either focus on a fixed set of bidders and items, or restrict the auction to be symmetric. In this work, we overcome such limitations by factoring \emph{public} contextual information of bidders and items into the auction learning framework. We propose $\mathtt{CITransNet}$, a context-integrated transformer-based neural network for optimal auction design, which maintains permutation-equivariance over bids and contexts while being able to find asymmetric solutions. We show by extensive experiments that $\mathtt{CITransNet}$ can recover the known optimal solutions in single-item settings, outperform strong baselines in multi-item auctions, and generalize well to cases other than those in training.


title: Random Orthogonalization for Federated Learning in Massive MIMO Systems

abstract: We propose a novel uplink communication method, coined random orthogonalization, for federated learning (FL) in a massive multiple-input and multiple-output (MIMO) wireless system. The key novelty of random orthogonalization comes from the tight coupling of FL model aggregation and two unique characteristics of massive MIMO - channel hardening and favorable propagation. As a result, random orthogonalization can achieve natural over-the-air model aggregation without requiring transmitter side channel state information, while significantly reducing the channel estimation overhead at the receiver. Theoretical analyses with respect to both communication and machine learning performances are carried out. In particular, an explicit relationship among the convergence rate, the number of clients and the number of antennas is established. Experimental results validate the effectiveness and efficiency of random orthogonalization for FL in massive MIMO.


title: Chimeras with uniformly distributed heterogeneity: two coupled populations

abstract: Chimeras occur in networks of two coupled populations of oscillators when the oscillators in one population synchronise while those in the other are asynchronous. We consider chimeras of this form in networks of planar oscillators for which one parameter associated with the dynamics of an oscillator is randomly chosen from a uniform distribution. A generalisation of the approach in [C.R. Laing, Physical Review E, 100, 042211, 2019], which dealt with identical oscillators, is used to investigate the existence and stability of chimeras for these heterogeneous networks in the limit of an infinite number of oscillators. In all cases, making the oscillators more heterogeneous destroys the stable chimera in a saddle-node bifurcation. The results help us understand the robustness of chimeras in networks of general oscillators to heterogeneity.


title: On plasmon modes in multi-layer structures

abstract: In this paper, we consider the plasmon resonance in multi-layer structures. The conductivity problem associated with uniformly distributed background field is considered. We show that the plasmon mode is equivalent to the eigenvalue problem of a matrix, whose order is the same to the number of layers. For any number of layers, the exact characteristic polynomial is derived by a conjecture and is verified by using induction. It is shown that all the roots to the characteristic polynomial are real and exist in the span [-1, 2]. Numerical examples are presented for finding all the plasmon modes, and it is surprisingly to find out that such multi-layer structures may induce so called surface-plasmon-resonance-like band.


title: A new Sparse Auto-encoder based Framework using Grey Wolf Optimizer for Data Classification Problem

abstract: One of the most important properties of deep auto-encoders (DAEs) is their capability to extract high level features from row data. Hence, especially recently, the autoencoders are preferred to be used in various classification problems such as image and voice recognition, computer security, medical data analysis, etc. Despite, its popularity and high performance, the training phase of autoencoders is still a challenging task, involving to select best parameters that let the model to approach optimal results. Different training approaches are applied to train sparse autoencoders. Previous studies and preliminary experiments reveal that those approaches may present remarkable results in same problems but also disappointing results can be obtained in other complex problems. Metaheuristic algorithms have emerged over the last two decades and are becoming an essential part of contemporary optimization techniques. Gray wolf optimization (GWO) is one of the current of those algorithms and is applied to train sparse auto-encoders for this study. This model is validated by employing several popular Gene expression databases. Results are compared with previous state-of-the art methods studied with the same data sets and also are compared with other popular metaheuristic algorithms, namely, Genetic Algorithms (GA), Particle Swarm Optimization (PSO) and Artificial Bee Colony (ABC). Results reveal that the performance of the trained model using GWO outperforms on both conventional models and models trained with most popular metaheuristic algorithms.


title: Exponential convergence to equilibrium for a two-speed model with variant drift fields via the resolvent estimate

abstract: We study a two-speed model with variant drift fields, which generalizes the Goldstein-Taylor model but the two advection fields are not necessarily in proportion. Due to the lack of a local equilibrium structure of the steady state, prevailing hypocoercivity methods could not be directly applied to such a system. To prove the exponential convergence to equilibrium for this model, we use a recent generalization of the Gearhart-Pr\"uss theorem [Sci. China Math. 64 (2021), no. 3, 507-518] to gain semigroup bounds, where the relative entropy estimate plays a central role in gaining desired resolvent estimates via a compactness argument.


title: Feeding the Accretion Disk from the Dusty Torus in a Reddened Quasar

abstract: We present here a detailed analysis of an unusual absorption line system in the quasar SDSS J122826.79+100532.2. The absorption lines in the system have a common redshifted velocity structure starting from $v\sim0$ and extending to $\sim1,000\ \mathrm{km~s}^{-1}$, and are clearly detected in hydrogen Balmer series up to H$\iota$, in metastable neutral helium triplet, and in optical lines of excited states of single ionized iron. We estimated that the absorber has a density $n_{\mathrm{H}}\approx10^{8.4}\ \mathrm{cm}^{-3}$ and an ionization parameter $U\approx10^{-1.2}$, thereupon located it at $r_{\mathrm{abs}}\approx1.5$ pc from the central supermassive black hole. The inferred distance is remarkably similar to the evaporation radius for dust grains $r_{\mathrm{evap}}\approx1$ pc in the quasar. Thus the absorber may be a probe of an inflow starting from the dusty torus and feeding the accretion disk. Both the featureless continuum and the broad emission lines are heavily reddened with $E(B-V)\approx0.66$, in contrast to the narrow emission lines whose reddening is negligible. The dusty medium could be located in between the broad and narrow emission line regions, and possibly be associated with a 'cold' narrow absorption line system detected in \ion{Ca}{2} and \ion{Na}{1} doublets nearly unshifted from the quasar systemic velocity. SDSS J122826.79+100532.2 might represent such a rare case that both the inflow and the torus could be tracked by absorption lines.


title: A direct and elementary proof of the well-definedness of the interior and exterior polynomials of hypergraphs

abstract: T. K\'{a}lm\'{a}n (A version of Tutte's polynomial for hypergraphs, Adv. Math. 244 (2013) 823-873.) introduced the interior and exterior polynomials which are generalizations of the Tutte polynomial $T(x,y)$ on plane points $(1/x,1)$ and $(1,1/y)$ to hypergraphs. The two polynomials are defined under a fixed ordering of hyperedges, and are proved to be independent of the ordering using techniques of polytopes. In this paper, similar to the Tutte's original proof we provide a direct and elementary proof for the well-definedness of the interior and exterior polynomials of hypergraphs.


title: Irreversible Markov Dynamics and Hydrodynamics for KPZ States in the Stochastic Six Vertex Model

abstract: We introduce a family of Markov growth processes on discrete height functions defined on the 2-dimensional square lattice. Each height function corresponds to a configuration of the six vertex model on the infinite square lattice. We focus on the stochastic six vertex model corresponding to a particular two-parameter family of weights within the ferroelectric regime. It is believed (and partially proven, see Aggarwal, arXiv:2004.13272) that the stochastic six vertex model displays nontrivial pure (i.e., translation invariant and ergodic) Gibbs states of two types, KPZ and liquid. These phases have very different long-range correlation structure. The Markov processes we construct preserve the KPZ pure states in the full plane. We also show that the same processes put on the torus preserve arbitrary Gibbs measures for generic six vertex weights (not necessarily in the ferroelectric regime). Our dynamics arise naturally from the Yang-Baxter equation for the six vertex model via its bijectivisation, a technique first used in Bufetov-Petrov (arXiv:1712.04584). The dynamics we construct are irreversible; in particular, the height function has a nonzero average drift. In each KPZ pure state, we explicitly compute the average drift (also known as the current) as a function of the slope. We use this to analyze the hydrodynamics of a non-stationary version of our process acting on quarter plane stochastic six vertex configurations. The fixed-time limit shapes in the quarter plane model were obtained in Borodin-Corwin-Gorin (arXiv:1407.6729).


title: Investigating Why Contrastive Learning Benefits Robustness Against Label Noise

abstract: Self-supervised Contrastive Learning (CL) has been recently shown to be very effective in preventing deep networks from overfitting noisy labels. Despite its empirical success, the theoretical understanding of the effect of contrastive learning on boosting robustness is very limited. In this work, we rigorously prove that the representation matrix learned by contrastive learning boosts robustness, by having: (i) one prominent singular value corresponding to each sub-class in the data, and significantly smaller remaining singular values; and (ii) {a large alignment between the prominent singular vectors and the clean labels of each sub-class. The above properties enable a linear layer trained on such representations to effectively learn the clean labels without overfitting the noise.} We further show that the low-rank structure of the Jacobian of deep networks pre-trained with contrastive learning allows them to achieve a superior performance initially, when fine-tuned on noisy labels. Finally, we demonstrate that the initial robustness provided by contrastive learning enables robust training methods to achieve state-of-the-art performance under extreme noise levels, e.g., an average of 27.18\% and 15.58\% increase in accuracy on CIFAR-10 and CIFAR-100 with 80\% symmetric noisy labels, and 4.11\% increase in accuracy on WebVision.


title: Reconstruction of Power Lines from Point Clouds

abstract: This paper proposes a novel solution for constructing line features modeling each catenary curve present within a series of points representing multiple catenary curves. The solution can be applied to extract power lines from lidar point clouds, which can then be used in downstream applications like creating digital twin geospatial models and evaluating the encroachment of vegetation. This paper offers an example of how the results obtained by the proposed solution could be used to assess vegetation growth near transmission power lines based on freely available lidar data for the City of Utrecht, Netherlands [1].


title: Analysis of two-component Gibbs samplers using the theory of two projections

abstract: The theory of two projections is utilized to study two-component Gibbs samplers. Through this theory, previously intractable problems regarding the asymptotic variances of two-component Gibbs samplers are reduced to elementary matrix algebra exercises. It is found that in terms of asymptotic variance, the two-component random-scan Gibbs sampler is never much worse, and could be considerably better than its deterministic-scan counterpart, provided that the selection probability is appropriately chosen. This is especially the case when there is a large discrepancy in computation cost between the two components. The result contrasts with the known fact that the deterministic-scan version has a faster convergence rate, which can also be derived from the method herein. On the other hand, a modified version of the deterministic-scan sampler that accounts for computation cost can outperform the random-scan version.


title: Does Transliteration Help Multilingual Language Modeling?

abstract: Script diversity presents a challenge to Multilingual Language Models (MLLM) by reducing lexical overlap among closely related languages. Therefore, transliterating closely related languages that use different writing scripts to a common script may improve the downstream task performance of MLLMs. We empirically measure the effect of transliteration on MLLMs in this context. We specifically focus on the Indic languages, which have the highest script diversity in the world, and we evaluate our models on the IndicGLUE benchmark. We perform the Mann-Whitney U test to rigorously verify whether the effect of transliteration is significant or not. We find that transliteration benefits the low-resource languages without negatively affecting the comparatively high-resource languages. We also measure the cross-lingual representation similarity of the models using centered kernel alignment on parallel sentences from the FLORES-101 dataset. We find that for parallel sentences across different languages, the transliteration-based model learns sentence representations that are more similar.


title: Unsupervised Multi-Granularity Summarization

abstract: Text summarization is a user-preference based task, i.e., for one document, users often have different priorities for summary. As a key aspect of customization in summarization, granularity is used to measure the semantic coverage between the summary and source document. However, developing systems that can generate summaries with customizable semantic coverage is still an under-explored topic. In this paper, we propose the first unsupervised multi-granularity summarization framework, GranuSum. We take events as the basic semantic units of the source documents and propose to rank these events by their salience. We also develop a model to summarize input documents with given events as anchors and hints. By inputting different numbers of events, GranuSum is capable of producing multi-granular summaries in an unsupervised manner. Meanwhile, we annotate a new benchmark GranuDUC that contains multiple summaries at different granularities for each document cluster. Experimental results confirm the substantial superiority of GranuSum on multi-granularity summarization over strong baselines. Further, by exploiting the event information, GranuSum also exhibits state-of-the-art performance under the conventional unsupervised abstractive setting. Dataset for this paper can be found at: https://github.com/maszhongming/GranuDUC


title: Polygon recutting as a cluster integrable system

abstract: Recutting is an operation on planar polygons defined by cutting a polygon along a diagonal to remove a triangle, and then reattaching the triangle along the same diagonal but with opposite orientation. Recuttings along different diagonals generate an action of the affine symmetric group on the space of polygons. We show that this action is given by cluster transformations and is completely integrable. The integrability proof is based on interpretation of recutting as refactorization of quaternionic polynomials.


title: Stability analysis of the Tsallis holographic dark energy model

abstract: Using the generalized Tsallis entropy, the Tsallis holographic dark energy(THDE) was proposed recently. In this paper we analyze the cosmological consequences of the THDE model with an interaction between dark energy and dark matter $Q=H(\alpha\rho_{m}+\beta\rho_{D})$. We find that the THDE model can explain the current accelerated cosmic expansion, and it is stable under certain conditions. Furthermore, through investigating the dynamical analysis, we find that there exists an attractor which represents an accelerated expansion phase of the universe. When $\beta=0$, this attractor corresponds to a dark energy dominated de Sitter solution and the universe can evolve into an era which is depicted by the $\Lambda$CDM model. The age of universe in this model is also explored.


title: On string quasitoric manifolds and their orbit polytopes

abstract: This article mainly aims to give combinatorial characterizations and topological descriptions of quasitoric manifolds with string property. We provide a necessary and sufficient condition for a simple polytope in dimension 2 and 3 to be realizable as the orbit polytope of a string quasitoric manifold. In particular, a complete description of string quasitoric manifolds over prisms is obtained. On the other hand, we characterize string quasitoric manifolds over $n$-dimensional simple polytopes with no more than $2n+2$ facets. Further results are available when the orbit polytope is the connected sum of a cube and another simple polytope. In addtion, a real analogue concerning small cover is briefly discussed.


title: 2D+3D facial expression recognition via embedded tensor manifold regularization

abstract: In this paper, a novel approach via embedded tensor manifold regularization for 2D+3D facial expression recognition (FERETMR) is proposed. Firstly, 3D tensors are constructed from 2D face images and 3D face shape models to keep the structural information and correlations. To maintain the local structure (geometric information) of 3D tensor samples in the low-dimensional tensors space during the dimensionality reduction, the $\ell_0$-norm of the core tensors and a tensor manifold regularization scheme embedded on core tensors are adopted via a low-rank truncated Tucker decomposition on the generated tensors. As a result, the obtained factor matrices will be used for facial expression classification prediction. To make the resulting tensor optimization more tractable, $\ell_1$-norm surrogate is employed to relax $\ell_0$-norm and hence the resulting tensor optimization problem has a nonsmooth objective function due to the $\ell_1$-norm and orthogonal constraints from the orthogonal Tucker decomposition. To efficiently tackle this tensor optimization problem, we establish the first-order optimality condition in terms of stationary points, and then design a block coordinate descent (BCD) algorithm with convergence analysis and the computational complexity. Numerical results on BU-3DFE database and Bosphorus databases demonstrate the effectiveness of our proposed approach.


title: AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models

abstract: Knowledge distillation (KD) methods compress large models into smaller students with manually-designed student architectures given pre-specified computational cost. This requires several trials to find a viable student, and further repeating the process for each student or computational budget change. We use Neural Architecture Search (NAS) to automatically distill several compressed students with variable cost from a large model. Current works train a single SuperLM consisting of millions of subnetworks with weight-sharing, resulting in interference between subnetworks of different sizes. Our framework AutoDistil addresses above challenges with the following steps: (a) Incorporates inductive bias and heuristics to partition Transformer search space into K compact sub-spaces (K=3 for typical student sizes of base, small and tiny); (b) Trains one SuperLM for each sub-space using task-agnostic objective (e.g., self-attention distillation) with weight-sharing of students; (c) Lightweight search for the optimal student without re-training. Fully task-agnostic training and search allow students to be reused for fine-tuning on any downstream task. Experiments on GLUE benchmark against state-of-the-art KD and NAS methods demonstrate AutoDistil to outperform leading compression techniques with upto 2.7x reduction in computational cost and negligible loss in task performance.


title: Upper bound on the colength of the trace of the canonical module in dimension one

abstract: We study the upper bound of the colength of trace of the canonical module in one-dimensional Cohen-Macaulay rings. We answer the two questions posed by Herzog-Hibi-Stamate and Kobayashi.