title: Comparison of Entropy Calculation Methods for Ransomware Encrypted File Identification

abstract: Ransomware is a malicious class of software that utilises encryption to implement an attack on system availability. The target's data remains encrypted and is held captive by the attacker until a ransom demand is met. A common approach used by many crypto-ransomware detection techniques is to monitor file system activity and attempt to identify encrypted files being written to disk, often using a file's entropy as an indicator of encryption. However, often in the description of these techniques, little or no discussion is made as to why a particular entropy calculation technique is selected or any justification given as to why one technique is selected over the alternatives. The Shannon method of entropy calculation is the most commonly-used technique when it comes to file encryption identification in crypto-ransomware detection techniques. Overall, correctly encrypted data should be indistinguishable from random data, so apart from the standard mathematical entropy calculations such as Chi-Square, Shannon Entropy and Serial Correlation, the test suites used to validate the output from pseudo-random number generators would also be suited to perform this analysis. he hypothesis being that there is a fundamental difference between different entropy methods and that the best methods may be used to better detect ransomware encrypted files. The paper compares the accuracy of 53 distinct tests in being able to differentiate between encrypted data and other file types. The testing is broken down into two phases, the first phase is used to identify potential candidate tests, and a second phase where these candidates are thoroughly evaluated. To ensure that the tests were sufficiently robust, the NapierOne dataset is used. This dataset contains thousands of examples of the most commonly used file types, as well as examples of files that have been encrypted by crypto-ransomware.


title: $L^p$ maximal bound and Sobolev regularity of two-parameter averages over tori

abstract: We investigate $L^p$ boundedness of the maximal function defined by the averaging operator $f\to \mathcal{A}_t^s f$ over the two-parameter family of tori $\mathbb{T}_t^{s}:=\{ ( (t+s\cos\theta)\cos\phi,\,(t+s\cos\theta)\sin\phi,\, s\sin\theta ): \theta, \phi \in [0,2\pi) \}$ with $c_0t>s>0$ for some $c_0\in (0,1)$. We prove that the associated (two-parameter) maximal function is bounded on $L^p$ if and only if $p>2$. We also obtain $L^p$--$L^q$ estimates for the local maximal operator on a sharp range of $p,q$. Furthermore, the sharp smoothing estimates are proved including the sharp local smoothing estimates for the operators $f\to \mathcal A_t^s f$ and $f\to \mathcal A_t^{c_0t} f$. For the purpose, we make use of Bourgain--Demeter's decoupling inequality for the cone and Guth--Wang--Zhang's local smoothing estimates for the $2$ dimensional wave operator.


title: ADLight: A Universal Approach of Traffic Signal Control with Augmented Data Using Reinforcement Learning

abstract: Traffic signal control has the potential to reduce congestion in dynamic networks. Recent studies show that traffic signal control with reinforcement learning (RL) methods can significantly reduce the average waiting time. However, a shortcoming of existing methods is that they require model retraining for new intersections with different structures. In this paper, we propose a novel reinforcement learning approach with augmented data (ADLight) to train a universal model for intersections with different structures. We propose a new agent design incorporating features on movements and actions with set current phase duration to allow the generalized model to have the same structure for different intersections. A new data augmentation method named \textit{movement shuffle} is developed to improve the generalization performance. We also test the universal model with new intersections in Simulation of Urban MObility (SUMO). The results show that the performance of our approach is close to the models trained in a single environment directly (only a 5% loss of average waiting time), and we can reduce more than 80% of training time, which saves a lot of computational resources in scalable operations of traffic lights.


title: Ranking nodes in directed networks via continuous-time quantum walks

abstract: Four new centrality measures for directed networks based on unitary, continuous-time quantum walks (CTQW) in $n$ dimensions -- where $n$ is the number of nodes -- are presented, tested and discussed. The main idea behind these methods consists in re-casting the classical HITS and PageRank algorithms as eigenvector problems for symmetric matrices, and using these symmetric matrices as Hamiltonians for CTQWs, in order to obtain a unitary evolution operator. The choice of the initial state is also crucial. Two options were tested: a vector with uniform occupation and a vector weighted w.r.t.~in- or out-degrees (for authority and hub centrality, respectively). Two methods are based on a HITS-derived Hamiltonian, and two use a PageRank-derived Hamiltonian. Centrality scores for the nodes are defined as the average occupation values. All the methods have been tested on a set of small, simple graphs in order to spot possible evident drawbacks, and then on a larger number of artificially generated larger-sized graphs, in order to draw a comparison with classical HITS and PageRank. Numerical results show that, despite some pathologies found in three of the methods when analyzing small graphs, all the methods are effective in finding the first and top ten nodes in larger graphs. We comment on the results and offer some insight into the good accordance between classical and quantum approaches.


title: Averaged Solar Torque Rotational Dynamics for Defunct Satellites

abstract: Spin state predictions for defunct satellites in geosynchronous earth orbit (GEO) are valuable for active debris removal and servicing missions as well as material shedding studies and attitude-dependent solar radiation pressure (SRP) modeling. Previous studies have shown that solar radiation torques can explain the observed spin state evolution of some GEO objects via the Yarkovsky-O'Keefe-Radzievskii-Paddack (YORP) effect. These studies have focused primarily on uniform rotation. Nevertheless, many objects are in non-principal axis rotation (i.e. tumbling). Recent exploration of the tumbling regime for the family of retired GOES 8-12 satellites has shown intriguing YORP-driven behavior including spin-orbit coupling, tumbling cycles, and tumbling period resonances. To better explore and understand the tumbling regime, we develop a semi-analytical tumbling-averaged rotational dynamics model. The derivation requires analytically averaging over the satellite's torque-free rotation, defined by Jacobi elliptic functions. Averaging is facilitated by a second order Fourier series approximation of the facet illumination function. The averaged model is found to capture and explain the general long-term behavior of the full dynamics while reducing computation time by roughly three orders of magnitude. This improved computation efficiency promises to enable rapid exploration of general long-term rotational dynamics for defunct satellites and rocket bodies.


title: Feed rotation corrections for antennas having beam waveguide mounts

abstract: We report on the development of new code to support the beam waveguide antenna mount types in AIPS, which will allow polarisation analysis of observations made using these antennas. Beam Wave-guide antennas in VLBI are common in communication antennas that have been repurposed (e.g. Warkworth, Yamaguchi). The mount type affects the differential phase between the left and the right hand circular polarisations (LHC and RHC) for different points on the sky. We demonstrate that the corrections for the Warkworth beam wave guide antenna can be applied.


title: Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task

abstract: Language models show a surprising range of capabilities, but the source of their apparent competence is unclear. Do these networks just memorize a collection of surface statistics, or do they rely on internal representations of the process that generates the sequences they see? We investigate this question by applying a variant of the GPT model to the task of predicting legal moves in a simple board game, Othello. Although the network has no a priori knowledge of the game or its rules, we uncover evidence of an emergent nonlinear internal representation of the board state. Interventional experiments indicate this representation can be used to control the output of the network and create "latent saliency maps" that can help explain predictions in human terms.


title: Evaluating Long-Term Memory in 3D Mazes

abstract: Intelligent agents need to remember salient information to reason in partially-observed environments. For example, agents with a first-person view should remember the positions of relevant objects even if they go out of view. Similarly, to effectively navigate through rooms agents need to remember the floor plan of how rooms are connected. However, most benchmark tasks in reinforcement learning do not test long-term memory in agents, slowing down progress in this important research direction. In this paper, we introduce the Memory Maze, a 3D domain of randomized mazes specifically designed for evaluating long-term memory in agents. Unlike existing benchmarks, Memory Maze measures long-term memory separate from confounding agent abilities and requires the agent to localize itself by integrating information over time. With Memory Maze, we propose an online reinforcement learning benchmark, a diverse offline dataset, and an offline probing evaluation. Recording a human player establishes a strong baseline and verifies the need to build up and retain memories, which is reflected in their gradually increasing rewards within each episode. We find that current algorithms benefit from training with truncated backpropagation through time and succeed on small mazes, but fall short of human performance on the large mazes, leaving room for future algorithmic designs to be evaluated on the Memory Maze.


title: On the Smoothness and Regularity of the Chess Billiard Flow and the Poincar\'e Problem

abstract: The Poincar\'e problem is a model of two-dimensional internal waves in stable-stratified fluid. The chess billiard flow, a variation of a typical billiard flow, drives the formation behind and describes the evolution of these internal waves, and its trajectories can be represented as rotations around the boundary of a given domain. We find that for sufficiently irrational rotation in the square, or when the rotation number $r(\lambda)$ is Diophantine, the regularity of the solution $u(t)$ of the evolution problem correlates directly to the regularity of the forcing function $f(x)$. Additionally, we show that when $f$ is smooth, then $u$ is also smooth. These results extend studies that have examined singularity points, or the lack of regularity, in rational rotations of the chess billiard flow. We also present numerical simulations in various geometries that analyze plateau formation and fractal dimension in $r(\lambda)$ and conjecture an extension of our results. Our results can be applied in modeling two dimensional oceanic waves, and they also relate the classical quantum correspondence to fluid study.


title: Resilience and Load Balancing in Fog Networks: A Multi-Criteria Decision Analysis Approach

abstract: The advent of Cloud Computing enabled the proliferation of IoT applications for smart environments. However, the distance of these resources makes them unsuitable for delay-sensitive applications. Hence, Fog Computing has emerged to provide such capabilities in proximity to end devices through distributed resources. These limited resources can collaborate to serve distributed IoT application workflows using the concept of stateless micro Fog service replicas, which provides resiliency and maintains service availability in the face of failures. Load balancing supports this collaboration by optimally assigning workloads to appropriate services, i.e., distributing the load among Fog nodes to fairly utilize compute and network resources and minimize execution delays. In this paper, we propose using ELECTRE, a Multi-Criteria Decision Analysis (MCDA) approach, to efficiently balance the load in Fog environments. We considered multiple objectives to make service selection decisions, including compute and network load information. We evaluate our approach in a realistic unbalanced topological setup with heterogeneous workload requirements. To the best of our knowledge, this is the first time ELECTRE-based methods are used to balance the load in Fog environments. Through simulations, we compared the performance of our proposed approach with traditional baseline methods that are commonly used in practice, namely random, Round-Robin, nearest node, and fastest service selection algorithms. In terms of the overall system performance, our approach outperforms these methods with up to 67% improvement.


title: Contraction of Locally Differentially Private Mechanisms

abstract: We investigate the contraction properties of locally differentially private mechanisms. More specifically, we derive tight upper bounds on the divergence between $PK$ and $QK$ output distributions of an $\epsilon$-LDP mechanism $K$ in terms of a divergence between the corresponding input distributions $P$ and $Q$, respectively. Our first main technical result presents a sharp upper bound on the $\chi^2$-divergence $\chi^2(PK}\|QK)$ in terms of $\chi^2(P\|Q)$ and $\varepsilon$. We also show that the same result holds for a large family of divergences, including KL-divergence and squared Hellinger distance. The second main technical result gives an upper bound on $\chi^2(PK\|QK)$ in terms of total variation distance $\mathsf{TV}(P, Q)$ and $\epsilon$. We then utilize these bounds to establish locally private versions of the van Trees inequality, Le Cam's, Assouad's, and the mutual information methods, which are powerful tools for bounding minimax estimation risks. These results are shown to lead to better privacy analyses than the state-of-the-arts in several statistical problems such as entropy and discrete distribution estimation, non-parametric density estimation, and hypothesis testing.

categories: cs.IT cs.CR math.IT math.ST stat.ML stat.TH


title: Towards a Higher-Order Mathematical Operational Semantics

abstract: Compositionality proofs in higher-order languages are notoriously involved, and general semantic frameworks guaranteeing compositionality are hard to come by. In particular, Turi and Plotkin's bialgebraic abstract GSOS framework, which has been successfully applied to obtain off-the-shelf compositionality results for first-order languages, so far does not apply to higher-order languages. In the present work, we develop a theory of abstract GSOS specifications for higher-order languages, in effect transferring the core principles of Turi and Plotkin's framework to a higher-order setting. In our theory, the operational semantics of higher-order languages is represented by certain dinatural transformations that we term pointed higher-order GSOS laws. We give a general compositionality result that applies to all systems specified in this way and discuss how compositionality of the SKI calculus and the $\lambda$-calculus w.r.t. a strong variant of Abramsky's applicative bisimilarity are obtained as instances.


title: Focused Concatenation for Context-Aware Neural Machine Translation

abstract: A straightforward approach to context-aware neural machine translation consists in feeding the standard encoder-decoder architecture with a window of consecutive sentences, formed by the current sentence and a number of sentences from its context concatenated to it. In this work, we propose an improved concatenation approach that encourages the model to focus on the translation of the current sentence, discounting the loss generated by target context. We also propose an additional improvement that strengthen the notion of sentence boundaries and of relative sentence distance, facilitating model compliance to the context-discounted objective. We evaluate our approach with both average-translation quality metrics and contrastive test sets for the translation of inter-sentential discourse phenomena, proving its superiority to the vanilla concatenation approach and other sophisticated context-aware systems.


title: A Regularized Conditional GAN for Posterior Sampling in Image Recovery Problems

abstract: In image recovery problems, one seeks to infer an image from distorted, incomplete, and/or noise-corrupted measurements. Such problems arise in magnetic resonance imaging (MRI), computed tomography, deblurring, super-resolution, inpainting, phase retrieval, image-to-image translation, and other applications. Given a training set of signal/measurement pairs, we seek to do more than just produce one good image estimate. Rather, we aim to rapidly and accurately sample from the posterior distribution. To do this, we propose a regularized conditional Wasserstein GAN that generates dozens of high-quality posterior samples per second. Our regularization comprises an $\ell_1$ penalty and an adaptively weighted standard-deviation reward. Using quantitative evaluation metrics like conditional Fr\'{e}chet inception distance, we demonstrate that our method produces state-of-the-art posterior samples in both multicoil MRI and large-scale inpainting applications. The code for our model can be found here: https://github.com/matt-bendel/rcGAN


title: On the failure of variational score matching for VAE models

abstract: Score matching (SM) is a convenient method for training flexible probabilistic models, which is often preferred over the traditional maximum-likelihood (ML) approach. However, these models are less interpretable than normalized models; as such, training robustness is in general difficult to assess. We present a critical study of existing variational SM objectives, showing catastrophic failure on a wide range of datasets and network architectures. Our theoretical insights on the objectives emerge directly from their equivalent autoencoding losses when optimizing variational autoencoder (VAE) models. First, we show that in the Fisher autoencoder, SM produces far worse models than maximum-likelihood, and approximate inference by Fisher divergence can lead to low-density local optima. However, with important modifications, this objective reduces to a regularized autoencoding loss that resembles the evidence lower bound (ELBO). This analysis predicts that the modified SM algorithm should behave very similarly to ELBO on Gaussian VAEs. We then review two other FD-based objectives from the literature and show that they reduce to uninterpretable autoencoding losses, likely leading to poor performance. The experiments verify our theoretical predictions and suggest that only ELBO and the baseline objective robustly produce expected results, while previously proposed SM methods do not.


title: Explaining Translationese: why are Neural Classifiers Better and what do they Learn?

abstract: Recent work has shown that neural feature- and representation-learning, e.g. BERT, achieves superior performance over traditional manual feature engineering based approaches, with e.g. SVMs, in translationese classification tasks. Previous research did not show $(i)$ whether the difference is because of the features, the classifiers or both, and $(ii)$ what the neural classifiers actually learn. To address $(i)$, we carefully design experiments that swap features between BERT- and SVM-based classifiers. We show that an SVM fed with BERT representations performs at the level of the best BERT classifiers, while BERT learning and using handcrafted features performs at the level of an SVM using handcrafted features. This shows that the performance differences are due to the features. To address $(ii)$ we use integrated gradients and find that $(a)$ there is indication that information captured by hand-crafted features is only a subset of what BERT learns, and $(b)$ part of BERT's top performance results are due to BERT learning topic differences and spurious correlations with translationese.


title: Nanomolding of Metastable Mo$_{4}$P$_{3}$

abstract: Reduced dimensionality leads to emergent phenomena in quantum materials and there is a need for accelerated materials discovery of nanoscale quantum materials in reduced dimensions. Thermomechanical nanomolding is a rapid synthesis method that produces high quality single-crystalline quantum nanowires with controlled dimensions over wafer-scale sizes. Herein, we apply nanomolding to fabricate nanowires from bulk feedstock of MoP, a triple-point topological metal with extremely high conductivity that is promising for low-resistance interconnects. Surprisingly, we obtained single-crystalline Mo$_{4}$P$_{3}$ nanowires, a metastable phase at room temperature in atmospheric pressure. We thus demonstrate nanomolding can create metastable phases inaccessible by other nanomaterial syntheses and can explore a previously inaccessible synthesis space at high temperatures and pressures. Furthermore, our results suggest that the current understanding of interfacial solid diffusion for nanomolding is incomplete, providing opportunities to explore solid-state diffusion at high-pressure and high-temperature regimes in confined dimensions.


title: We need to talk about random seeds

abstract: Modern neural network libraries all take as a hyperparameter a random seed, typically used to determine the initial state of the model parameters. This opinion piece argues that there are some safe uses for random seeds: as part of the hyperparameter search to select a good model, creating an ensemble of several models, or measuring the sensitivity of the training algorithm to the random seed hyperparameter. It argues that some uses for random seeds are risky: using a fixed random seed for "replicability" and varying only the random seed to create score distributions for performance comparison. An analysis of 85 recent publications from the ACL Anthology finds that more than 50% contain risky uses of random seeds.


title: Existence of a tricritical point for the Blume-Capel model on $\mathbb{Z}^d$

abstract: We prove the existence of a tricritical point for the Blume-Capel model on $\mathbb{Z}^d$ for every $d\geq 2$. The proof in $d\geq 3$ relies on a novel combinatorial mapping to an Ising model on a larger graph, the techniques of Aizenman, Duminil-Copin, and Sidoravicious (Comm. Math. Phys, 2015), and the celebrated infrared bound. In $d=2$, the proof relies on a quantitative analysis of crossing probabilities of the dilute random cluster representation of the Blume-Capel. In particular, we develop a quadrichotomy result in the spirit of Duminil-Copin and Tassion (Moscow Math. J., 2020), which allows us to obtain a fine picture of the phase diagram in $d=2$, including asymptotic behaviour of correlations in all regions. Finally, we show that the techniques used to establish subcritical sharpness for the dilute random cluster model extend to any $d\geq 2$.


title: Improved Bi-point Rounding Algorithms and a Golden Barrier for $k$-Median

abstract: The current best approximation algorithms for $k$-median rely on first obtaining a structured fractional solution known as a bi-point solution, and then rounding it to an integer solution. We improve this second step by unifying and refining previous approaches. We describe a hierarchy of increasingly-complex partitioning schemes for the facilities, along with corresponding sets of algorithms and factor-revealing non-linear programs. We prove that the third layer of this hierarchy is a $2.613$-approximation, improving upon the current best ratio of $2.675$, while no layer can be proved better than $2.588$ under the proposed analysis. On the negative side, we give a family of bi-point solutions which cannot be approximated better than the square root of the golden ratio, even if allowed to open $k+o(k)$ facilities. This gives a barrier to current approaches for obtaining an approximation better than $2 \sqrt{\phi} \approx 2.544$. Altogether we reduce the approximation gap of bi-point solutions by two thirds.


title: Offline congestion games: How feedback type affects data coverage requirement

abstract: This paper investigates when one can efficiently recover an approximate Nash Equilibrium (NE) in offline congestion games.The existing dataset coverage assumption in offline general-sum games inevitably incurs a dependency on the number of actions, which can be exponentially large in congestion games. We consider three different types of feedback with decreasing revealed information. Starting from the facility-level (a.k.a., semi-bandit) feedback, we propose a novel one-unit deviation coverage condition and give a pessimism-type algorithm that can recover an approximate NE. For the agent-level (a.k.a., bandit) feedback setting, interestingly, we show the one-unit deviation coverage condition is not sufficient. On the other hand, we convert the game to multi-agent linear bandits and show that with a generalized data coverage assumption in offline linear bandits, we can efficiently recover the approximate NE. Lastly, we consider a novel type of feedback, the game-level feedback where only the total reward from all agents is revealed. Again, we show the coverage assumption for the agent-level feedback setting is insufficient in the game-level feedback setting, and with a stronger version of the data coverage assumption for linear bandits, we can recover an approximate NE. Together, our results constitute the first study of offline congestion games and imply formal separations between different types of feedback.


title: Development of Hybrid ASR Systems for Low Resource Medical Domain Conversational Telephone Speech

abstract: Language barriers present a great challenge in our increasingly connected and global world. Especially within the medical domain, e.g. hospital or emergency room, communication difficulties and delays may lead to malpractice and non-optimal patient care. In the HYKIST project, we consider patient-physician communication, more specifically between a German-speaking physician and an Arabic- or Vietnamese-speaking patient. Currently, a doctor can call the Triaphon service to get assistance from an interpreter in order to help facilitate communication. The HYKIST goal is to support the usually non-professional bilingual interpreter with an automatic speech translation system to improve patient care and help overcome language barriers. In this work, we present our ASR system development efforts for this conversational telephone speech translation task in the medical domain for two languages pairs, data collection, various acoustic model architectures and dialect-induced difficulties.


title: Does a portion of dimer configuration determines its domain of definition?

abstract: Critical models are, almost by definition, supposed to feature both slow decay of correlations for local observables while retaining some mixing even for macroscopic observables. A strong version of the latter property is that changing boundary conditions cannot have a singular (in the measure theoretic sense) effect on the model away from the boundary, even asymptotically. In this paper we prove that statement for the wired uniform spanning tree and temperleyan dimer model.


title: Does Mode of Digital Contact Tracing Affect User Willingness to Share Information? A Quantitative Study

abstract: Digital contact tracing can limit the spread of infectious diseases. Nevertheless, there remain barriers to attaining sufficient adoption. In this study, we investigate how willingness to participate in contact tracing is affected by two critical factors: the modes of data collection and the type of data collected. We conducted a scenario-based survey study among 220 respondents in the United States (U.S.) to understand their perceptions about contact tracing associated with automated and manual contact tracing methods. The findings indicate a promising use of smartphones and a combination of public health officials and medical health records as information sources. Through a quantitative analysis, we describe how different modalities and individual demographic factors may affect user compliance in providing four key pieces of information to contact tracing.


title: Sampling with Mollified Interaction Energy Descent

abstract: Sampling from a target measure whose density is only known up to a normalization constant is a fundamental problem in computational statistics and machine learning. In this paper, we present a new optimization-based method for sampling called mollified interaction energy descent (MIED). MIED minimizes a new class of energies on probability measures called mollified interaction energies (MIEs). These energies rely on mollifier functions -- smooth approximations of the Dirac delta originated from PDE theory. We show that as the mollifier approaches the Dirac delta, the MIE converges to the chi-square divergence with respect to the target measure and the gradient flow of the MIE agrees with that of the chi-square divergence. Optimizing this energy with proper discretization yields a practical first-order particle-based algorithm for sampling in both unconstrained and constrained domains. We show experimentally that for unconstrained sampling problems our algorithm performs on par with existing particle-based algorithms like SVGD, while for constrained sampling problems our method readily incorporates constrained optimization techniques to handle more flexible constraints with strong performance compared to alternatives.


title: Entity-level Sentiment Analysis in Contact Center Telephone Conversations

abstract: Entity-level sentiment analysis predicts the sentiment about entities mentioned in a given text. It is very useful in a business context to understand user emotions towards certain entities, such as products or companies. In this paper, we demonstrate how we developed an entity-level sentiment analysis system that analyzes English telephone conversation transcripts in contact centers to provide business insight. We present two approaches, one entirely based on the transformer-based DistilBERT model, and another that uses a convolutional neural network supplemented with some heuristic rules.


title: Explaining the $R_{K^{(*)}}$ anomalies and the CDF $M_W$ in Flavorful Top Seesaw Models with Gauged $U(1)_{L(-R)}$

abstract: A new heavy $Z'$ vector boson provides a possible explanation for the neutral current B anomalies and points to the scale where we expect the solution to the hierarchy problem to appear. In this paper, we explore a modified Top Seesaw model based on the global symmetry breaking of $U(3)_L$ to $U(2)_L$. The symmetry is partially gauged such that the breaking introduces not only a composite Higgs doublet but also a TeV-scale $Z'$ boson. The $U(3)_L/U(2)_L$ Top Seesaw model predicts a light Higgs boson as well as a large top Yukawa coupling. Additional $U(1)'$ symmetry is gauged such that a flavorful TeV-scale $Z'$ boson is introduced to address the B anomalies. The existence of a heavy vector-like top quark and $Z-Z'$ mixing breaks the custodial symmetry, which predicts a heavier $W$ mass as observed by the CDF collaboration. Two possible candidates for the $U(1)'$ symmetry corresponding to the $Z'$ boson, $U(1)_L$ and $U(1)_{L-R}$, are discussed in detail. The parameter space for the models to address both anomalies is presented. The direct searches of the $Z'$ boson from dilepton channels are also studied. Based on the assumptions, a TeV-scale $Z'$ boson is still viable and can be probed in the near future.


title: In-Hand Manipulation of Unknown Objects with Tactile Sensing for Insertion

abstract: In this paper, we present a method to manipulate unknown objects in-hand using tactile sensing without relying on a known object model. In many cases, vision-only approaches may not be feasible; for example, due to occlusion in cluttered spaces. We address this limitation by introducing a method to reorient unknown objects using tactile sensing. It incrementally builds a probabilistic estimate of the object shape and pose during task-driven manipulation. Our approach uses Bayesian optimization to balance exploration of the global object shape with efficient task completion. To demonstrate the effectiveness of our method, we apply it to a simulated Tactile-Enabled Roller Grasper, a gripper that rolls objects in hand while collecting tactile data. We evaluate our method on an insertion task with randomly generated objects and find that it reliably reorients objects while significantly reducing the exploration time.


title: Contrastive Representation Learning for Gaze Estimation

abstract: Self-supervised learning (SSL) has become prevalent for learning representations in computer vision. Notably, SSL exploits contrastive learning to encourage visual representations to be invariant under various image transformations. The task of gaze estimation, on the other hand, demands not just invariance to various appearances but also equivariance to the geometric transformations. In this work, we propose a simple contrastive representation learning framework for gaze estimation, named Gaze Contrastive Learning (GazeCLR). GazeCLR exploits multi-view data to promote equivariance and relies on selected data augmentation techniques that do not alter gaze directions for invariance learning. Our experiments demonstrate the effectiveness of GazeCLR for several settings of the gaze estimation task. Particularly, our results show that GazeCLR improves the performance of cross-domain gaze estimation and yields as high as 17.2% relative improvement. Moreover, the GazeCLR framework is competitive with state-of-the-art representation learning methods for few-shot evaluation. The code and pre-trained models are available at https://github.com/jswati31/gazeclr.


title: An extension of Seliger's wave breaking condition for the nonlocal Whitham type equation

abstract: We extend the wave breaking condition in Seliger's work [Proc. R. Soc. Lond. Ser. A., 303 (1968)], which has been used widely to prove wave breaking phenomena for nonlinear nonlocal shallow water equations.


title: Autonomous quantum error correction and fault-tolerant quantum computation with squeezed cat qubits

abstract: We propose an autonomous quantum error correction scheme using squeezed cat (SC) code against the dominant error source, excitation loss, in continuous-variable systems. Through reservoir engineering, we show that a structured dissipation can stabilize a two-component SC while autonomously correcting the errors. The implementation of such dissipation only requires low-order nonlinear couplings among three bosonic modes or between a bosonic mode and a qutrit. While our proposed scheme is device independent, it is readily implementable with current experimental platforms such as superconducting circuits and trapped-ion systems. Compared to the stabilized cat, the stabilized SC has a much lower dominant error rate and a significantly enhanced noise bias. Furthermore, the bias-preserving operations for the SC have much lower error rates. In combination, the stabilized SC leads to substantially better logical performance when concatenating with an outer discrete-variable code. The surface-SC scheme achieves more than one order of magnitude increase in the threshold ratio between the loss rate $\kappa_1$ and the engineered dissipation rate $\kappa_2$. Under a practical noise ratio $\kappa_1/\kappa_2 = 10^{-3}$, the repetition-SC scheme can reach a $10^{-15}$ logical error rate even with a small mean excitation number of 4, which already suffices for practically useful quantum algorithms.


title: Universality of the muon component of extensive air showers

abstract: In extensive air shower experiments, the number of muons crossing a detector at a given position, as well as their arrival time, arrival direction, and energy, are determined by a more fundamental 3-dimensional distribution linked to the hadronic core of the shower. Muons are produced high up in the atmosphere after the decay of mesons in the hadronic cascade. The distributions of production depth, energy, and transverse momentum of muons are enough to fully predict the muon component of air showers in any particular observational condition. By using air-shower simulations with the state-of-the-art hadronic interaction models, the mentioned distributions at production are analyzed as a function of zenith angle, primary mass, and hadronic interaction model, and their level of universality is studied and assessed in an exhaustive manner for the first time.


title: Hubbard subbands and superconductivity in the infinite-layer nickelate

abstract: An effective two-dimensional two-band model for infinite-layer nickelates consists of bands obtained from $d_{x^2-y^2}$ and $s$--like orbitals. We investigate whether it could be mapped onto a single-band Hubbard model and the filling of Hubbard bands. We find that both one-band physics and a Kondo-lattice regime emerge from the same two-orbital model, depending on the strength of electronic correlations and the filling of the itinerant $s$-band. Next we investigate one-particle excitations by changing the screening. First, for weak screening the strong correlations push electrons out of the $s$-band so that the undoped nickelate is similar to a cuprate. Second, for strong screening the $s$ and $d_{x^2-y^2}$ bands are both partly filled and weakly couple. Particularly in this latter regime mapping to a one-band model gives significant spectral weight transfer between the Hubbard subbands. Finally we point out that the superconducting phases may have either $d$-wave or $s$-wave symmetry.


title: A reduced model for droplet dynamics in shear flows at finite capillary numbers

abstract: We propose an extension of the Maffettone-Minale (MM) model to predict the dynamics of an ellipsoidal droplet in a shear flow. The parameters of the MM model are traditionally retrieved in the framework of the perturbation theory for small deformations, i.e., small capillary numbers ($\mbox{Ca} \ll 1$) applied to Stokes equations. In this work, we take a novel route, in that we determine the model parameters at finite capillary numbers ($\mbox{Ca}\sim {\cal O}(1)$) without relying on perturbation theory results, while retaining a realistic representation in creeping time and steady deformation attained by the droplet for different realizations of the viscosity ratio $\lambda$ between the inner and the outer fluids. This extended MM (EMM) model hinges on an independent characterization of the process of droplet deformation via numerical simulations of Stokes equations employing the Immersed Boundary - Lattice Boltzmann (IB-LB) numerical techniques. Issues on droplet breakup are also addressed and discussed within the EMM model.


title: Adjusting for informative cluster size in pseudo-value based regression approaches with clustered time to event data

abstract: Informative cluster size (ICS) arises in situations with clustered data where a latent relationship exists between the number of participants in a cluster and the outcome measures. Although this phenomenon has been sporadically reported in statistical literature for nearly two decades now, further exploration is needed in certain statistical methodologies to avoid potentially misleading inferences. For inference about population quantities without covariates, inverse cluster size reweightings are often employed to adjust for ICS. Further, to study the effect of covariates on disease progression described by a multistate model, the pseudo-value regression technique has gained popularity in time-to-event data analysis. We seek to answer the question: "How to apply pseudo-value regression to clustered time-to-event data when cluster size is informative?" ICS adjustment by the reweighting method can be performed in two steps; estimation of marginal functions of the multistate model and fitting the estimating equations based on pseudo-value responses, leading to four possible strategies. We present theoretical arguments and thorough simulation experiments to ascertain the correct strategy for adjusting for ICS. A further extension of our methodology is implemented to include informativeness induced by the intra-cluster group size. We demonstrate the methods in two real-world applications: (i) to determine predictors of tooth survival in a periodontal study, and (ii) to identify indicators of ambulatory recovery in spinal cord injury patients who participated in locomotor-training rehabilitation.


title: Castelnuovo bound and higher genus Gromov-Witten invariants of quintic 3-folds

abstract: We prove a conjectural vanishing result for Gopakumar--Vafa invariants of quintic 3-folds, referred to as Castelnuovo bound in the literature. Furthermore, we calculate Gopakumar--Vafa invariants at Castelnuovo bound $g=\frac{d^2+5d+10}{10}$. As physicists showed, these two properties allow us to compute all Gromov--Witten invariants of quintic 3-folds up to genus $53$, provided that the conifold gap condition holds. We also give a bound for the genus of any one-dimensional closed subscheme in a smooth hypersurface of degree $\leq 5$, which may be of independent interest.


title: A supplement to Chebotarev's density theorem

abstract: Let $L/K$ be a Galois extension of number fields with Galois group $G$. We show that if the density of prime ideals in $K$ that split totally in $L$ tends to $1/|G|$ with a power saving error term, then the density of prime ideals in $K$ whose Frobenius is a given conjugacy class $C\subset G$ tends to $|C|/|G|$ with the same power saving error term. We deduce this by relating the poles of the corresponding Dirichlet series to the zeros of $\zeta_L(s)/\zeta_K(s)$.


title: Razor-thin dust layers in protoplanetary disks: Limits on the vertical shear instability

abstract: Context: Recent observations with the Atacama Large Millimeter Array (ALMA) have shown that the large dust aggregates observed at millimeter wavelengths settle to the midplane into a remarkably thin layer. Aims: We intend to find out if the geometric thinness of these layers is evidence against the vertical shear instability (VSI) operating in these disks. Methods: We performed hydrodynamic simulations of a protoplanetary disk with a locally isothermal equation of state, and let the VSI fully develop. We sprinkled dust particles and followed their motion as they got stirred up by the VSI. We determined for which grain size the layer becomes geometrically thin enough to be consistent with ALMA observations. We then verified if, with these grain sizes, it is still possible to generate a moderately optically thick layer at millimeter wavelengths, as observations appear to indicate. Results: We found that even very large dust aggregates with Stokes numbers close to unity get stirred up to relatively large heights above the midplane by the VSI, which is in conflict with the observed geometric thinness. For grains so large that the Stokes number exceeds unity, the layer can be made to remain thin, but we show that it is hard to make dust layers optically thick at ALMA wavelengths (e.g., tau(1.3mm)>=1) with such large dust aggregates. Conclusions: We conclude that protoplanetary disks with geometrically thin midplane dust layers cannot be VSI unstable, at least not down to the disk midplane. Explanations for the inhibition of the VSI include a reduced dust-to-gas ratio of the small dust grains that are responsible for the radiative cooling of the disk. A reduction of small grains by a factor of between 10 and 100 is sufficient to quench the VSI. Such a reduction is plausible in dust growth models, and still consistent with observations at optical and infrared wavelengths.


title: Thermodynamics-informed neural networks for physically realistic mixed reality

abstract: The imminent impact of immersive technologies in society urges for active research in real-time and interactive physics simulation for virtual worlds to be realistic. In this context, realistic means to be compliant to the laws of physics. In this paper we present a method for computing the dynamic response of (possibly non-linear and dissipative) deformable objects induced by real-time user interactions in mixed reality using deep learning. The graph-based architecture of the method ensures the thermodynamic consistency of the predictions, whereas the visualization pipeline allows a natural and realistic user experience. Two examples of virtual solids interacting with virtual or physical solids in mixed reality scenarios are provided to prove the performance of the method.


title: Deep Learning Approach for Dynamic Sampling for Multichannel Mass Spectrometry Imaging

abstract: Mass Spectrometry Imaging (MSI), using traditional rectilinear scanning, takes hours to days for high spatial resolution acquisitions. Given that most pixels within a sample's field of view are often neither relevant to underlying biological structures nor chemically informative, MSI presents as a prime candidate for integration with sparse and dynamic sampling algorithms. During a scan, stochastic models determine which locations probabilistically contain information critical to the generation of low-error reconstructions. Decreasing the number of required physical measurements thereby minimizes overall acquisition times. A Deep Learning Approach for Dynamic Sampling (DLADS), utilizing a Convolutional Neural Network (CNN) and encapsulating molecular mass intensity distributions within a third dimension, demonstrates a simulated 70% throughput improvement for Nanospray Desorption Electrospray Ionization (nano-DESI) MSI tissues. Evaluations are conducted between DLADS and a Supervised Learning Approach for Dynamic Sampling, with Least-Squares regression (SLADS-LS) and a Multi-Layer Perceptron (MLP) network (SLADS-Net). When compared with SLADS-LS, limited to a single m/z channel, as well as multichannel SLADS-LS and SLADS-Net, DLADS respectively improves regression performance by 36.7%, 7.0%, and 6.2%, resulting in gains to reconstruction quality of 6.0%, 2.1%, and 3.4% for acquisition of targeted m/z.


title: A Continuous Convolutional Trainable Filter for Modelling Unstructured Data

abstract: Convolutional Neural Network (CNN) is one of the most important architectures in deep learning. The fundamental building block of a CNN is a trainable filter, represented as a discrete grid, used to perform convolution on discrete input data. In this work, we propose a continuous version of a trainable convolutional filter able to work also with unstructured data. This new framework allows exploring CNNs beyond discrete domains, enlarging the usage of this important learning technique for many more complex problems. Our experiments show that the continuous filter can achieve a level of accuracy comparable to the state-of-the-art discrete filter, and that it can be used in current deep learning architectures as a building block to solve problems with unstructured domains as well.


title: Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worlds

abstract: Despite impressive successes, deep reinforcement learning (RL) systems still fall short of human performance on generalization to new tasks and environments that differ from their training. As a benchmark tailored for studying RL generalization, we introduce Avalon, a set of tasks in which embodied agents in highly diverse procedural 3D worlds must survive by navigating terrain, hunting or gathering food, and avoiding hazards. Avalon is unique among existing RL benchmarks in that the reward function, world dynamics, and action space are the same for every task, with tasks differentiated solely by altering the environment; its 20 tasks, ranging in complexity from eat and throw to hunt and navigate, each create worlds in which the agent must perform specific skills in order to survive. This setup enables investigations of generalization within tasks, between tasks, and to compositional tasks that require combining skills learned from previous tasks. Avalon includes a highly efficient simulator, a library of baselines, and a benchmark with scoring metrics evaluated against hundreds of hours of human performance, all of which are open-source and publicly available. We find that standard RL baselines make progress on most tasks but are still far from human performance, suggesting Avalon is challenging enough to advance the quest for generalizable RL.


title: Standardly embedded train tracks and pseudo-Anosov maps with minimum expansion factor

abstract: We show that given a fully-punctured pseudo-Anosov map $f:S \to S$ whose punctures lie in at least two orbits under the action of $f$, the expansion factor $\lambda(f)$ satisfies the inequality $\lambda(f)^{|\chi(S)|} \ge \mu^4 \approx 6.85408$, where $\mu = \frac{1 + \sqrt{5}}{2} \approx 1.61803$ is the golden ratio. The proof involves a study of standardly embedded train tracks, and the Thurston symplectic form defined on their weight space.


title: Chiral and trace anomalies in Deeply Virtual Compton Scattering

abstract: Inspired by recent work by Tarasov and Venugopalan, we calculate the one-loop quark box diagrams relevant to polarized and unpolarized Deep Inelastic Scattering (DIS) by introducing off-forward momentum $l^\mu$ as an infrared regulator. In the polarized case, we rederive the pole $1/l^2$ related to the axial (chiral) anomaly. In addition, we obtain the usual logarithmic term and the DIS coefficient function. We interpret the result in terms of the generalized parton distributions (GPDs) $\tilde{H}$ and $\tilde{E}$ and discuss the possible violation of QCD factorization for the Compton scattering amplitude. Remarkably, we also find poles in the unpolarized case which are remnants of the trace anomaly. We argue that these poles are cancelled by the would-be massless glueball poles in the GPDs $H$ and $E$ as well as in their moments, the nucleon gravitational form factors $A,B$ and $D$. This mechanism sheds light on the connection between the gravitational form factors and the gluon condensate operator $F^{\mu\nu}F_{\mu\nu}$.


title: Twitter Users' Behavioral Response to Toxic Replies

abstract: Online toxic attacks, such as harassment, trolling, and hate speech have been linked to an increase in offline violence and negative psychological effects on victims. In this paper, we studied the impact of toxicity on users' online behavior. We collected a sample of 79.8k Twitter conversations. Then, through a longitudinal study, for nine weeks, we tracked and compared the behavioral reactions of authors, who were toxicity victims, with those who were not. We found that toxicity victims show a combination of the following behavioral reactions: avoidance, revenge, countermeasures, and negotiation. We performed statistical tests to understand the significance of the contribution of toxic replies toward user behaviors while considering confounding factors, such as the structure of conversations and the user accounts' visibility, identifiability, and activity level. Interestingly, we found that compared to other random authors, victims are more likely to engage in conversations, reply in a toxic way, and unfollow toxicity instigators. Even if the toxicity is directed at other participants, the root authors are more likely to engage in the conversations and reply in a toxic way. However, victims who have verified accounts are less likely to participate in conversations or respond by posting toxic comments. In addition, replies are more likely to be removed in conversations with a larger percentage of toxic nested replies and toxic replies directed at other users. Our results can assist further studies in developing more effective detection and intervention methods for reducing the negative consequences of toxicity on social media.


title: Evaluation of Position and Velocity Based Forward Dynamics Compliance Control (FDCC) for Robotic Interactions in Position Controlled Robots

abstract: In robotic manipulation, end-effector compliance is an essential precondition for performing contact-rich tasks, such as machining, assembly, and human-robot interaction. Most robotic arms are position-controlled stiff systems at a hardware level. Thus, adding compliance becomes essential. Compliance in those systems has been recently achieved using Forward dynamics compliance control (FDCC), which, owing to its virtual forward dynamics model, can be implemented on both position and velocity-controlled robots. This paper evaluates the choice of control interface (and hence the control domain), which, although considered trivial, is essential due to differences in their characteristics. In some cases, the choice is restricted to the available hardware interface. However, given the option to choose, the velocity-based control interface makes a better candidate for compliance control because of smoother compliant behaviour, reduced interaction forces, and work done. To prove these points, in this paper FDCC is evaluated on the UR10e six-DOF manipulator with velocity and position control modes. The evaluation is based on force-control benchmarking metrics using 3D-printed artefacts. Real experiments favour the choice of velocity control over position control.


title: Confronting the vector leptoquark hypothesis with new low- and high-energy data

abstract: In light of new data we present an updated phenomenological analysis of the simplified $U_1$-leptoquark model addressing charged-current $B$-meson anomalies. The analysis shows a good compatibility of low-energy data (dominated by the lepton flavor universality ratios $R_D$ and $R_{D^*}$) with the high-energy constraints posed by $pp\to \tau\bar\tau$ Drell-Yan data. We also show that present data are well compatible with a framework where the leptoquark couples with similar strength to both left- and right-handed third-generation fermions, a scenario that is well-motivated from a model building perspective. We find that the high-energy implications of this setup will be probed at the 95% confidence level in the high-luminosity phase of the LHC.


title: Experimental design for causal query estimation in partially observed biomolecular networks

abstract: Estimating a causal query from observational data is an essential task in the analysis of biomolecular networks. Estimation takes as input a network topology, a query estimation method, and observational measurements on the network variables. However, estimations involving many variables can be experimentally expensive, and computationally intractable. Moreover, using the full set of variables can be detrimental, leading to bias, or increasing the variance in the estimation. Therefore, designing an experiment based on a well-chosen subset of network components can increase estimation accuracy, and reduce experimental and computational costs. We propose a simulation-based algorithm for selecting sub-networks that support unbiased estimators of the causal query under a constraint of cost, ranked with respect to the variance of the estimators. The simulations are constructed based on historical experimental data, or based on known properties of the biological system. Three case studies demonstrated the effectiveness of well-chosen network subsets for estimating causal queries from observational data. All the case studies are reproducible and available at https://github.com/srtaheri/Simplified_LVM.


title: Voltage-controlled topological interface states for bending waves in soft dielectric phononic crystal plates

abstract: The operating frequency range of passive topological phononic crystals is generally fixed and narrow, limiting their practical applications. To overcome this difficulty, here we design and investigate a one-dimensional soft dielectric phononic crystal (PC) plate system with actively tunable topological interface states via the mechanical and electric loads. We use nonlinear electroelasticity theory and linearized incremental theory to derive the governing equations. First we determine the nonlinear static response of the soft dielectric PC plate subjected to a combination of axial force and electric voltage. Then we study the motion of superimposed incremental bending waves. By adopting the Spectral Element Method, we obtain the dispersion relation for the infinite PC plate and the transmission coefficient for the finite PC plate waveguide. Numerical results show that the low-frequency topological interface state exists at the interface of the finite phononic plate waveguide with two topologically different elements. By simply adjusting the axial force or the electric voltage, an increase or decrease in the frequency of the topological interface state can be realized. Furthermore, applying the electric voltage separately on different elements of the PC plate waveguide is a flexible and smart method to tune the topological interface state in a wide range. These results provide guidance for designing soft smart wave devices with low-frequency tunable topological interface states.


title: Molecular-scale substrate anisotropy and crowding drive long-range nematic order of cell monolayers

abstract: The ability of cells to reorganize in response to external stimuli is important in areas ranging from morphogenesis to tissue engineering. Elongated cells can co-align due to steric effects, forming states with local order. We show that molecular-scale substrate anisotropy can direct cell organization, resulting in the emergence of nematic order on tissue scales. To quantitatively examine the disorder-order transition, we developed a high-throughput imaging platform to analyze velocity and orientational correlations for several thousand cells over days. The establishment of global, seemingly long-ranged order is facilitated by enhanced cell division along the substrate's nematic axis, and associated extensile stresses that restructure the cells' actomyosin networks. Our work, which connects to a class of systems known as active dry nematics, provides a new understanding of the dynamics of cellular remodeling and organization in weakly interacting cell collectives. This enables data-driven discovery of cell-cell interactions and points to strategies for tissue engineering.


title: A revision on Rayleigh capillary jet breakup

abstract: The average Rayleigh capillary breakup length of a cylindrical Newtonian viscous liquid jet moving with homogeneous velocity $\hat{U}$ (negligible external forces) must be determined by the selection of normal modes with time-independent amplitude and wavelength (invariant modes, IMs). Both positive and negative group velocity IMs exist in ample ranges of the parameter domain (Weber and Ohnesorge numbers), which explains (i) the average breakup length independence on ambient conditions (long-term resonance), and (ii) its proportionality to the inverse of the spatial growth rate of the dominant positive group velocity IM. Published experimental results since Grace (1965, PhD Thesis) confirm our proposal.


title: R-matrix electron-impact excitation data for the H- and He-like ions with Z=6-30

abstract: Plasma models built on extensive atomic data are essential to interpreting the observed cosmic spectra. H-like Lyman series and He-like triplets observable in the X-ray band are powerful diagnostic lines to measure the physical properties of various types of astrophysical plasmas. Electron-impact excitation is a fundamental atomic process for the formation of H-like and He-like key diagnostic lines. Electron-impact excitation data adopted by the widely used plasma codes (AtomDB, CHIANTI, and SPEX) do not necessarily agree with each other. Here we present a systematic calculation of electron-impact excitation data of H-like and He-like ions with the atomic number Z=6-30 (i.e., C to Zn). Radiation damped R-matrix intermediate coupling frame transformation calculation was performed for each ion with configurations up to $n=6$. We compare the present work with the above three plasma codes and literature to assess the quality of the new data, which are relevant for current and future high-resolution X-ray spectrometers.


title: PseudoAugment: Learning to Use Unlabeled Data for Data Augmentation in Point Clouds

abstract: Data augmentation is an important technique to improve data efficiency and save labeling cost for 3D detection in point clouds. Yet, existing augmentation policies have so far been designed to only utilize labeled data, which limits the data diversity. In this paper, we recognize that pseudo labeling and data augmentation are complementary, thus propose to leverage unlabeled data for data augmentation to enrich the training data. In particular, we design three novel pseudo-label based data augmentation policies (PseudoAugments) to fuse both labeled and pseudo-labeled scenes, including frames (PseudoFrame), objecta (PseudoBBox), and background (PseudoBackground). PseudoAugments outperforms pseudo labeling by mitigating pseudo labeling errors and generating diverse fused training scenes. We demonstrate PseudoAugments generalize across point-based and voxel-based architectures, different model capacity and both KITTI and Waymo Open Dataset. To alleviate the cost of hyperparameter tuning and iterative pseudo labeling, we develop a population-based data augmentation framework for 3D detection, named AutoPseudoAugment. Unlike previous works that perform pseudo-labeling offline, our framework performs PseudoAugments and hyperparameter tuning in one shot to reduce computational cost. Experimental results on the large-scale Waymo Open Dataset show our method outperforms state-of-the-art auto data augmentation method (PPBA) and self-training method (pseudo labeling). In particular, AutoPseudoAugment is about 3X and 2X data efficient on vehicle and pedestrian tasks compared to prior arts. Notably, AutoPseudoAugment nearly matches the full dataset training results, with just 10% of the labeled run segments on the vehicle detection task.


title: Slow thermalization and subdiffusion in $U(1)$ conserving Floquet random circuits

abstract: Random quantum circuits are paradigmatic models of minimally structured and analytically tractable chaotic dynamics. We study a family of Floquet unitary circuits with Haar random $U(1)$ charge conserving dynamics; the minimal such model has nearest-neighbor gates acting on spin 1/2 qubits, and a single layer of even/odd gates repeated periodically in time. We find that this minimal model is not robustly thermalizing at numerically accessible system sizes, and displays slow subdiffusive dynamics for long times. We map out the thermalization dynamics in a broader parameter space of charge conserving circuits, and understand the origin of the slow dynamics in terms of proximate localized and integrable regimes in parameter space. In contrast, we find that small extensions to the minimal model are sufficient to achieve robust thermalization; these include (i) increasing the interaction range to three-site gates (ii) increasing the local Hilbert space dimension by appending an additional unconstrained qubit to the conserved charge on each site, or (iii) using a larger Floquet period comprised of two independent layers of gates. Our results should inform future numerical studies of charge conserving circuits which are relevant for a wide range of topical theoretical questions.


title: Data-Driven Stabilizing and Robust Control of Discrete-Time Linear Systems with Error in Variables

abstract: This work presents a sum-of-squares (SOS) based framework to perform data-driven stabilization and robust control tasks on discrete-time linear systems where the full-state observations are corrupted by L-infinity bounded input, measurement, and process noise (error in variable setting). Certificates of state-feedback superstability, quadratic stability or positive stability of all plants in a consistency set are provided by solving a feasibility program formed by polynomial nonnegativity constraints. Under mild compactness and data-collection assumptions, SOS tightenings in rising degree will converge to recover the true superstabilizing or positive stabilizing controller, with some conservatism introduced for quadratic stabilizability. The performance of this SOS method is improved through the application of a theorem of alternatives while retaining tightness, in which the unknown noise variables are eliminated from the consistency set description. This SOS feasibility method is extended to provide worst-case-optimal robust controllers under H2 control costs. The consistency set description may be broadened to include cases where the data and process are affected by a combination of L-infinity bounded measurement, process, and input noise.


title: Instruction-Following Agents with Multimodal Transformer

abstract: Humans are excellent at understanding language and vision to accomplish a wide range of tasks. In contrast, creating general instruction-following embodied agents remains a difficult challenge. Prior work that uses pure language-only models lack visual grounding, making it difficult to connect language instructions with visual observations. On the other hand, methods that use pre-trained multimodal models typically come with divided language and visual representations, requiring designing specialized network architecture to fuse them together. We propose a simple yet effective model for robots to solve instruction-following tasks in vision-based environments. Our \ours method consists of a multimodal transformer that encodes visual observations and language instructions, and a transformer-based policy that predicts actions based on encoded representations. The multimodal transformer is pre-trained on millions of image-text pairs and natural language text, thereby producing generic cross-modal representations of observations and instructions. The transformer-based policy keeps track of the full history of observations and actions, and predicts actions autoregressively. Despite its simplicity, we show that this unified transformer model outperforms all state-of-the-art pre-trained or trained-from-scratch methods in both single-task and multi-task settings. Our model also shows better model scalability and generalization ability than prior work.


title: Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models

abstract: Large language models (LLM) trained using the next-token-prediction objective, such as GPT3 and PaLM, have revolutionized natural language processing in recent years by showing impressive zero-shot and few-shot capabilities across a wide range of tasks. In this work, we propose a simple technique that significantly boosts the performance of LLMs without adding computational cost. Our key observation is that, by performing the next token prediction task with randomly selected past tokens masked out, we can improve the quality of the learned representations for downstream language understanding tasks. We hypothesize that randomly masking past tokens prevents over-attending to recent tokens and encourages attention to tokens in the distant past. We find that our method, Forgetful Causal Masking (FCM), significantly improves both few-shot and finetuning performance of PaLM. We further consider a simple extension, T-FCM, which introduces bidirectional context to causal language model without altering the sequence order, and further improves finetuning performance.


title: Vibrations and waves in soft dielectric elastomer structures

abstract: Dielectric elastomers (DEs) are a type of multifunctional materials with salient features that are very attractive in developing soft, lightweight, and small-scale transducers and robotics. This paper reviews the mechanics of soft DE structures, focusing on their dynamic responses, including vibrations and waves in typical structures. The basic formulations of nonlinear electroelasticity are briefly summarized by following the descriptions of Dorfmann and Ogden. The recent advances in the study on linearized dynamic problems of DE plates and shells are then outlined according to the geometric configurations as well as the coordinates thereof adopted. Some interesting observations from the numerical results are highlighted, and the underlying physics is discussed. For completeness, an account of the nonlinear dynamic characteristics of DE structures is also given. For DE structures with a periodic arrangement, i.e., DE phononic crystals (PCs) or metamaterials, this paper reviews the designs, solutions, and results to illustrate the wide-range tunability of band structure that is realized by large reversible deformation and electromechanical coupling. The present literature survey could be a useful guide for future studies on the dynamic topics of DE structures.


title: Combining cosmic shear data with correlated photo-$z$ uncertainties: constraints from DESY1 and HSC-DR1

abstract: An accurate calibration of the source redshift distribution $p(z)$ is a key aspect in the analysis of cosmic shear data. This, one way or another, requires the use of spectroscopic or high-quality photometric samples. However, the difficulty to obtain colour-complete spectroscopic samples matching the depth of weak lensing catalogs means that the analyses of different cosmic shear datasets often use the same samples for redshift calibration. This introduces a source of statistical and systematic uncertainty that is highly correlated across different weak lensing datasets, and which must be accurately characterised and propagated in order to obtain robust cosmological constraints from their combination. In this paper we introduce a method to quantify and propagate the uncertainties on the source redshift distribution in two different surveys sharing the same calibrating sample. The method is based on an approximate analytical marginalisation of the $p(z)$ statistical uncertainties and the correlated marginalisation of residual systematics. We apply this method to the combined analysis of cosmic shear data from the DESY1 data release and the HSC-DR1 data, using the COSMOS 30-band catalog as a common redshift calibration sample. We find that, although there is significant correlation in the uncertainties on the redshift distributions of both samples, this does not change the final constraints on cosmological parameters significantly. The same is true also for the impact of residual systematic uncertainties from the errors in the COSMOS 30-band photometric redshifts. Additionally, we show that these effects will still be negligible in Stage-IV datasets. Finally, the combination of DESY1 and HSC-DR1 allows us to constrain the ``clumpiness'' parameter to $S_8 = 0.768^{+0.021}_{-0.017}$. This corresponds to a $\sim\sqrt{2}$ improvement in uncertainties with respect to either DES or HSC alone.


title: Dichotomy of Control: Separating What You Can Control from What You Cannot

abstract: Future- or return-conditioned supervised learning is an emerging paradigm for offline reinforcement learning (RL), where the future outcome (i.e., return) associated with an observed action sequence is used as input to a policy trained to imitate those same actions. While return-conditioning is at the heart of popular algorithms such as decision transformer (DT), these methods tend to perform poorly in highly stochastic environments, where an occasional high return can arise from randomness in the environment rather than the actions themselves. Such situations can lead to a learned policy that is inconsistent with its conditioning inputs; i.e., using the policy to act in the environment, when conditioning on a specific desired return, leads to a distribution of real returns that is wildly different than desired. In this work, we propose the dichotomy of control (DoC), a future-conditioned supervised learning framework that separates mechanisms within a policy's control (actions) from those beyond a policy's control (environment stochasticity). We achieve this separation by conditioning the policy on a latent variable representation of the future, and designing a mutual information constraint that removes any information from the latent variable associated with randomness in the environment. Theoretically, we show that DoC yields policies that are consistent with their conditioning inputs, ensuring that conditioning a learned policy on a desired high-return future outcome will correctly induce high-return behavior. Empirically, we show that DoC is able to achieve significantly better performance than DT on environments that have highly stochastic rewards and transition


title: Microlensing of gravitational waves by dark matter structures

abstract: Gravitational lensing of gravitational waves provides a potential new probe of dark matter structures. In this work, we consider the microlensing effect on gravitational wave signals from black hole binaries induced by low-mass dark matter halos that do not retain enough baryonic matter to hold stars. We clarify systematically when this microlensing effect is relevant and study in detail its detectability by future gravitational wave observatories. We consider lensing by cold dark matter halos and by solitonic cores that reside in fuzzy dark matter halos. Our results show that although the effect can be detectable at relatively large impact parameters, the probability of detecting such lensed events is low. In particular, we find that the expected number of events lensed by cold dark matter halos is $O(0.01)$ per year for BBO and the expected number of events lensed by solitonic cores inside fuzzy dark matter halos is $O(0.01)$ per year for ET. In the case that a significant fraction of dark matter consists of $=O(100 M_\odot)$ objects that are relatively compact, $R < O(0.1\,{\rm pc})$, we show that the expected number of lensed events per year ET can be very large, $0(1000)$.


title: Logical convergence laws via stochastic approximation and Markov processes

abstract: Since the paper of Kleinberg and Kleinberg, SODA'05, where it was proven that the preferential attachment random graph with degeneracy at least 3 does not obey the first order 0-1 law, no general methods were developed to study logical limit laws for recursive random graph models with arbitrary degeneracy. Even in the (possibly) simplest case of the uniform attachment, it is still not known whether the first order convergence law holds in this model. We prove that the uniform attachment random graph with bounded degrees obeys the first order convergence law. To prove the law, we describe dynamics of first order equivalence classes of the random graph using Markov chains. The convergence law follows from the existence of a limit distribution of the considered Markov chain. To show the latter convergence, we use stochastic approximation processes.


title: High Fidelity Neural Audio Compression

abstract: We introduce a state-of-the-art real-time, high-fidelity, audio codec leveraging neural networks. It consists in a streaming encoder-decoder architecture with quantized latent space trained in an end-to-end fashion. We simplify and speed-up the training by using a single multiscale spectrogram adversary that efficiently reduces artifacts and produce high-quality samples. We introduce a novel loss balancer mechanism to stabilize training: the weight of a loss now defines the fraction of the overall gradient it should represent, thus decoupling the choice of this hyper-parameter from the typical scale of the loss. Finally, we study how lightweight Transformer models can be used to further compress the obtained representation by up to 40%, while staying faster than real time. We provide a detailed description of the key design choices of the proposed model including: training objective, architectural changes and a study of various perceptual loss functions. We present an extensive subjective evaluation (MUSHRA tests) together with an ablation study for a range of bandwidths and audio domains, including speech, noisy-reverberant speech, and music. Our approach is superior to the baselines methods across all evaluated settings, considering both 24 kHz monophonic and 48 kHz stereophonic audio. Code and models are available at github.com/facebookresearch/encodec.


title: Cascading Biases: Investigating the Effect of Heuristic Annotation Strategies on Data and Models

abstract: Cognitive psychologists have documented that humans use cognitive heuristics, or mental shortcuts, to make quick decisions while expending less effort. While performing annotation work on crowdsourcing platforms, we hypothesize that such heuristic use among annotators cascades on to data quality and model robustness. In this work, we study cognitive heuristic use in the context of annotating multiple-choice reading comprehension datasets. We propose tracking annotator heuristic traces, where we tangibly measure low-effort annotation strategies that could indicate usage of various cognitive heuristics. We find evidence that annotators might be using multiple such heuristics, based on correlations with a battery of psychological tests. Importantly, heuristic use among annotators determines data quality along several dimensions: (1) known biased models, such as partial input models, more easily solve examples authored by annotators that rate highly on heuristic use, (2) models trained on annotators scoring highly on heuristic use don't generalize as well, and (3) heuristic-seeking annotators tend to create qualitatively less challenging examples. Our findings suggest that tracking heuristic usage among annotators can potentially help with collecting challenging datasets and diagnosing model biases.


title: Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval

abstract: Current person image retrieval methods have achieved great improvements in accuracy metrics. However, they rarely describe the reliability of the prediction. In this paper, we propose an Uncertainty-Aware Learning (UAL) method to remedy this issue. UAL aims at providing reliability-aware predictions by considering data uncertainty and model uncertainty simultaneously. Data uncertainty captures the ``noise" inherent in the sample, while model uncertainty depicts the model's confidence in the sample's prediction. Specifically, in UAL, (1) we propose a sampling-free data uncertainty learning method to adaptively assign weights to different samples during training, down-weighting the low-quality ambiguous samples. (2) we leverage the Bayesian framework to model the model uncertainty by assuming the parameters of the network follow a Bernoulli distribution. (3) the data uncertainty and the model uncertainty are jointly learned in a unified network, and they serve as two fundamental criteria for the reliability assessment: if a probe is high-quality (low data uncertainty) and the model is confident in the prediction of the probe (low model uncertainty), the final ranking will be assessed as reliable. Experiments under the risk-controlled settings and the multi-query settings show the proposed reliability assessment is effective. Our method also shows superior performance on three challenging benchmarks under the vanilla single query settings.


title: Bridging Machine Learning and Sciences: Opportunities and Challenges

abstract: The application of machine learning in sciences has seen exciting advances in recent years. As a widely applicable technique, anomaly detection has been long studied in the machine learning community. Especially, deep neural nets-based out-of-distribution detection has made great progress for high-dimensional data. Recently, these techniques have been showing their potential in scientific disciplines. We take a critical look at their applicative prospects including data universality, experimental protocols, model robustness, etc. We discuss examples that display transferable practices and domain-specific challenges simultaneously, providing a starting point for establishing a novel interdisciplinary research paradigm in the near future.

categories: stat.ML cs.LG hep-ex hep-ph physics.data-an


title: Protocols for classically training quantum generative models on probability distributions

abstract: Quantum Generative Modelling (QGM) relies on preparing quantum states and generating samples from these states as hidden - or known - probability distributions. As distributions from some classes of quantum states (circuits) are inherently hard to sample classically, QGM represents an excellent testbed for quantum supremacy experiments. Furthermore, generative tasks are increasingly relevant for industrial machine learning applications, and thus QGM is a strong candidate for demonstrating a practical quantum advantage. However, this requires that quantum circuits are trained to represent industrially relevant distributions, and the corresponding training stage has an extensive training cost for current quantum hardware in practice. In this work, we propose protocols for classical training of QGMs based on circuits of the specific type that admit an efficient gradient computation, while remaining hard to sample. In particular, we consider Instantaneous Quantum Polynomial (IQP) circuits and their extensions. Showing their classical simulability in terms of the time complexity, sparsity and anti-concentration properties, we develop a classically tractable way of simulating their output probability distributions, allowing classical training to a target probability distribution. The corresponding quantum sampling from IQPs can be performed efficiently, unlike when using classical sampling. We numerically demonstrate the end-to-end training of IQP circuits using probability distributions for up to 30 qubits on a regular desktop computer. When applied to industrially relevant distributions this combination of classical training with quantum sampling represents an avenue for reaching advantage in the NISQ era.


title: Module categories, internal bimodules and Tambara modules

abstract: We use the theory of Tambara modules to extend and generalize the reconstruction theorem for module categories over a rigid monoidal category to the non-rigid case. We show a biequivalence between the $2$-category of cyclic module categories over a monoidal category $\mathscr{C}$ and the bicategory of algebra and bimodule objects in the category of Tambara modules on $\mathscr{C}$. Using it, we prove that a cyclic module category can be reconstructed as the category of certain free module objects in the category of Tambara modules on $\mathscr{C}$, and give a sufficient condition for its reconstructability as module objects in $\mathscr{C}$. To that end, we extend the definition of the Cayley functor to the non-closed case, and show that Tambara modules give a proarrow equipment for $\mathscr{C}$-module categories, in which $\mathscr{C}$-module functors are characterized as $1$-morphisms admitting a right adjoint. Finally, we show that the $2$-category of all $\mathscr{C}$-module categories embeds into the $2$-category of categories enriched in Tambara modules on $\mathscr{C}$, giving an ''action via enrichment'' result.


title: Universality classes of thermalization for mesoscopic Floquet systems

abstract: We identify several phases of thermalization that describe regimes of behavior in isolated, periodically driven (Floquet), mesoscopic quantum chaotic systems. We also identify a new Floquet thermal ensemble -- the ladder ensemble -- that is qualitatively distinct from the featureless infinite-temperature state that is often assumed to describe the equilibrium of driven systems. The phases can be coarsely classified by (i) whether or not the system irreversibly exchanges energy of order $\omega$ with the drive, i.e., Floquet thermalizes, and (ii) the ensemble describing the final equilibrium in systems that do Floquet thermalize. These phases represent regimes of behavior in mesoscopic systems, but they are sharply defined in a large-system limit where the drive frequency $\omega$ scales up with system size $N$ as the $N\to\infty$ limit is taken: we examine frequency scalings ranging from a weak $\omega \sim \log N$, to stronger scalings ranging from $\omega \sim \sqrt{N}$ to $\omega \sim N$. We show that the transition where Floquet thermalization breaks down occurs at $\omega\sim N$ and, beyond that, systems that do not Floquet thermalize are distinguished based on the presence or absence of rare resonances across Floquet zones. We produce a thermalization phase diagram that is relevant for numerical studies of Floquet systems and experimental studies on small-scale quantum simulators, both of which lack a separation of scales between $N$ and $\omega$. A striking prediction of our work is that, under perfect isolation, certain realistic quench protocols from simple pure initial states can show Floquet thermalization to a novel type of Schrodinger-cat state that is a global superposition of states at distinct temperatures. Our work extends and organizes the theory of Floquet thermalization, heating, and equilibrium into the setting of mesoscopic quantum systems.


title: Monocular Dynamic View Synthesis: A Reality Check

abstract: We study the recent progress on dynamic view synthesis (DVS) from monocular video. Though existing approaches have demonstrated impressive results, we show a discrepancy between the practical capture process and the existing experimental protocols, which effectively leaks in multi-view signals during training. We define effective multi-view factors (EMFs) to quantify the amount of multi-view signal present in the input capture sequence based on the relative camera-scene motion. We introduce two new metrics: co-visibility masked image metrics and correspondence accuracy, which overcome the issue in existing protocols. We also propose a new iPhone dataset that includes more diverse real-life deformation sequences. Using our proposed experimental protocol, we show that the state-of-the-art approaches observe a 1-2 dB drop in masked PSNR in the absence of multi-view cues and 4-5 dB drop when modeling complex motion. Code and data can be found at https://hangg7.com/dycheck.


title: Flying Trot Control Method for Quadruped Robot Based on Trajectory Planning

abstract: An intuitive control method for the flying trot, which combines offline trajectory planning with real-time balance control, is presented. The motion features of running animals in the vertical direction were analysed using the spring-load-inverted-pendulum (SLIP) model, and the foot trajectory of the robot was planned, so the robot could run similar to an animal capable of vertical flight, according to the given height and speed of the trunk. To improve the robustness of running, a posture control method based on a foot acceleration adjustment is proposed. A novel kinematic based CoM observation method and CoM regulation method is present to enhance the stability of locomotion. To reduce the impact force when the robot interacts with the environment, the virtual model control method is used in the control of the foot trajectory to achieve active compliance. By selecting the proper parameters for the virtual model, the oscillation motion of the virtual model and the planning motion of the support foot are synchronized to avoid the large disturbance caused by the oscillation motion of the virtual model in relation to the robot motion. The simulation and experiment using the quadruped robot Billy are reported. In the experiment, the maximum speed of the robot could reach 4.73 times the body length per second, which verified the feasibility of the control method.


title: Precision Machine Learning

abstract: We explore unique considerations involved in fitting ML models to data with very high precision, as is often required for science applications. We empirically compare various function approximation methods and study how they scale with increasing parameters and data. We find that neural networks can often outperform classical approximation methods on high-dimensional examples, by auto-discovering and exploiting modular structures therein. However, neural networks trained with common optimizers are less powerful for low-dimensional cases, which motivates us to study the unique properties of neural network loss landscapes and the corresponding optimization challenges that arise in the high precision regime. To address the optimization issue in low dimensions, we develop training tricks which enable us to train neural networks to extremely low loss, close to the limits allowed by numerical precision.


title: EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domain

abstract: Existing summarization datasets come with two main drawbacks: (1) They tend to focus on overly exposed domains, such as news articles or wiki-like texts, and (2) are primarily monolingual, with few multilingual datasets. In this work, we propose a novel dataset, called EUR-Lex-Sum, based on manually curated document summaries of legal acts from the European Union law platform (EUR-Lex). Documents and their respective summaries exist as cross-lingual paragraph-aligned data in several of the 24 official European languages, enabling access to various cross-lingual and lower-resourced summarization setups. We obtain up to 1,500 document/summary pairs per language, including a subset of 375 cross-lingually aligned legal acts with texts available in all 24 languages. In this work, the data acquisition process is detailed and key characteristics of the resource are compared to existing summarization resources. In particular, we illustrate challenging sub-problems and open questions on the dataset that could help the facilitation of future research in the direction of domain-specific cross-lingual summarization. Limited by the extreme length and language diversity of samples, we further conduct experiments with suitable extractive monolingual and cross-lingual baselines for future work. Code for the extraction as well as access to our data and baselines is available online at: https://github.com/achouhan93/eur-lex-sum.


title: Controlled Text Reduction

abstract: Producing a reduced version of a source text, as in generic or focused summarization, inherently involves two distinct subtasks: deciding on targeted content and generating a coherent text conveying it. While some popular approaches address summarization as a single end-to-end task, prominent works support decomposed modeling for individual subtasks. Further, semi-automated text reduction is also very appealing, where users may identify targeted content while models would generate a corresponding coherent summary. In this paper, we focus on the second subtask, of generating coherent text given pre-selected content. Concretely, we formalize \textit{Controlled Text Reduction} as a standalone task, whose input is a source text with marked spans of targeted content ("highlighting"). A model then needs to generate a coherent text that includes all and only the target information. We advocate the potential of such models, both for modular fully-automatic summarization, as well as for semi-automated human-in-the-loop use cases. Facilitating proper research, we crowdsource high-quality dev and test datasets for the task. Further, we automatically generate a larger "silver" training dataset from available summarization benchmarks, leveraging a pretrained summary-source alignment model. Finally, employing these datasets, we present a supervised baseline model, showing promising results and insightful analyses.


title: Two-point correlation function studies for the Milky Way: discovery of spatial clustering from disk excitations and substructure

abstract: We introduce a two-particle correlation function (2PCF) for the Milky Way, constructed to probe spatial correlations in the orthogonal directions of the stellar disk in the Galactic cylindrical coordinates of $R$, $\phi$, and $z$. We use this new tool to probe the structure and dynamics of the Galaxy using the carefully selected set of solar neighborhood stars ($d \lesssim 3\, \rm kpc$) from Gaia Data Release 2 we previously employed for studies of axial symmetry breaking in stellar number counts. We make additional, extensive tests, comparing to reference numerical simulations, to ensure our control over possibly confounding systematic effects. Supposing either axial or North-South symmetry we divide this data set into two nominally symmetric sectors and construct the 2PCF, in the manner of the Landy-Szalay estimator, from the Gaia data. In so doing, working well away from the mid-plane region in which the spiral arms appear, we have discovered distinct symmetry-breaking patterns in the 2PCF in its orthogonal directions, thus establishing the existence of correlations in stellar number counts at sub-kiloparsec length scales for the very first time. Particularly, we observe extensive wave-like structures of amplitude greatly in excess of what we would estimate if the system were in a steady state. We study the variations in these patterns across the Galactic disk, and with increasing $|z|$, and we show how our results complement other observations of non-steady-state effects near the Sun, such as vertical asymmetries in stellar number counts and the Gaia snail.


title: Superconducting microsphere magnetically levitated in an anharmonic potential with integrated magnetic readout

abstract: Magnetically levitated superconducting microparticles offer a promising path to quantum experiments with picogram to microgram objects. In this work, we levitate a 700ng $\sim 10^{17}$amu superconducting microsphere in a magnetic chip trap in which detection is integrated. We measure the particle's center-of-mass motion using a DC-SQUID magnetometer. The trap frequencies are continuously tunable between 30 and 160 Hz and the particle remains stably trapped over days in a dilution refrigerator environment. We characterize motional-amplitude-dependent frequency shifts, which arise from trap anharmonicities, namely Duffing nonlinearities and mode couplings. We explain this nonlinear behavior using finite element modelling of the chip-based trap potential. This work constitutes a first step towards quantum experiments and ultrasensitive inertial sensors with magnetically levitated superconducting microparticles.


title: MetaFormer Baselines for Vision

abstract: MetaFormer, the abstracted architecture of Transformer, has been found to play a significant role in achieving competitive performance. In this paper, we further explore the capacity of MetaFormer, again, without focusing on token mixer design: we introduce several baseline models under MetaFormer using the most basic or common mixers, and summarize our observations as follows: (1) MetaFormer ensures solid lower bound of performance. By merely adopting identity mapping as the token mixer, the MetaFormer model, termed IdentityFormer, achieves >80% accuracy on ImageNet-1K. (2) MetaFormer works well with arbitrary token mixers. When specifying the token mixer as even a random matrix to mix tokens, the resulting model RandFormer yields an accuracy of >81%, outperforming IdentityFormer. Rest assured of MetaFormer's results when new token mixers are adopted. (3) MetaFormer effortlessly offers state-of-the-art results. With just conventional token mixers dated back five years ago, the models instantiated from MetaFormer already beat state of the art. (a) ConvFormer outperforms ConvNeXt. Taking the common depthwise separable convolutions as the token mixer, the model termed ConvFormer, which can be regarded as pure CNNs, outperforms the strong CNN model ConvNeXt. (b) CAFormer sets new record on ImageNet-1K. By simply applying depthwise separable convolutions as token mixer in the bottom stages and vanilla self-attention in the top stages, the resulting model CAFormer sets a new record on ImageNet-1K: it achieves an accuracy of 85.5% at 224x224 resolution, under normal supervised training without external data or distillation. In our expedition to probe MetaFormer, we also find that a new activation, StarReLU, reduces 71% FLOPs of activation compared with GELU yet achieves better performance. We expect StarReLU to find great potential in MetaFormer-like models alongside other neural networks.


title: A multi-category inverse design neural network and its application to diblock copolymers

abstract: In this work, we design a multi-category inverse design neural network to map ordered periodic structure to physical parameters. The neural network model consists of two parts, a classifier and Structure-Parameter-Mapping (SPM) subnets. The classifier is used to identify structure, and the SPM subnets are used to predict physical parameters for desired structures. We also present an extensible reciprocal-space data augmentation method to guarantee the rotation and translation invariant of periodic structures. We apply the proposed network model and data augmentation method to two-dimensional diblock copolymers based on the Landau-Brazovskii model. Results show that the multi-category inverse design neural network is high accuracy in predicting physical parameters for desired structures. Moreover, the idea of multi-categorization can also be extended to other inverse design problems.


title: A Novel Block-Wise Index Modulation Scheme for High-Mobility OTFS Communications

abstract: As a promising technique for high-mobility wireless communications, orthogonal time frequency space (OTFS) has been proved to enjoy excellent advantages with respect to traditional orthogonal frequency division multiplexing (OFDM). However, a challenging problem is to design efficient systems to further improve the performance. In this paper, we propose a novel block-wise index modulation (IM) scheme for OTFS systems, named Doppler-IM with OTFS (DoIM-OTFS), where a block of Doppler resource bins are activated simultaneously. For practical implementation, we develop a low complexity customized message passing (CMP) algorithm for our proposed DoIM-OTFS scheme. Simulation results demonstrate our proposed DoIM-OTFS system outperforms traditional OTFS system without IM. The proposed CMP algorithm can achieve desired performance and robustness to the imperfect channel state information (CSI).