title: On the calculation of neutron sources generating steady prescribed power distributions in subcritical systems using multigroup X,Y-geometry discrete ordinates models

abstract: In this paper a methodology is described to estimate multigroup neutron source distributions which must be added into a subcritical system to drive it to a steady state prescribed power distribution. This work has been motivated by the principle of operation of the ADS (Accelerator Driven System) reactors, which have subcritical cores stabilized by the action of external sources. We use the energy multigroup two-dimensional neutron transport equation in the discrete ordinates formulation (SN) and the equation which is adjoint to it, whose solution is interpreted here as a distribution measuring the importance of the angular flux of neutrons to a linear functional. These equations are correlated through a reciprocity relation, leading to a relationship between the interior sources of neutrons and the power produced by unit length of height of the domain. A coarse-mesh numerical method of the spectral nodal class, referred to as adjoint response matrix constant-nodal method, is applied to numerically solve the adjoint SN equations. Numerical experiments are performed to analyze the accuracy of the present methodology so as to illustrate its potential practical applications.


title: Dynamics of nanoparticles in polydisperse polymer networks: From free diffusion to hopping

abstract: Using molecular dynamics simulations we study the static and dynamic properties of spherical nanoparticles (NPs) embedded in a disordered and polydisperse polymer network. Purely repulsive (RNP) as well as weakly attractive (ANP) polymer-NP interactions are considered. It is found that for both types of particles the NP dynamics at intermediate and at long times is controlled by the confinement parameter $C=\sigma_N/\lambda$, where $\sigma_N$ is the NP diameter and $\lambda$ is the dynamic localization length of the crosslinks. Three dynamical regimes are identified: i) For weak confinement ($C \lesssim 1$) the NPs can freely diffuse through the mesh; ii) For strong confinement ($C \gtrsim 1$) NPs proceed by means of activated hopping; iii) For extreme confinement ($C \gtrsim 3$) the mean squared displacement shows on intermediate time scales a quasi-plateau since the NPs are trapped by the mesh for very long times. Escaping from this local cage is a process that depends strongly on the local environment, thus giving rise to an extremely heterogeneous relaxation dynamics. The simulation data are compared with the two main theories for the diffusion process of NPs in gels. Both theories give a very good description of the $C-$dependence of the NP diffusion constant, but fail to reproduce the heterogeneous dynamics at intermediate time scales.


title: Boundary Green's functions and Minkowski content measure of multi-force-point SLE$_\kappa(\underline\rho)$

abstract: We consider a transient chordal SLE$_\kappa(\rho_1,\dots,\rho_m)$ curve $\eta$ in $\mathbb{H}$ from $w$ to $\infty$ with force points $ v_1> \cdots >v_m$ in $(-\infty,w^-]$, which intersects and is not boundary-filling on $(-\infty,v_m)$. The main result is that there is an atomless locally finite Borel measure $\mu_\eta$ on $\eta\cap (-\infty,v_m]$ such that for any $v<v_m$, the $d$-dimensional Minkowski content of $\eta\cap [v,v_m]$ exists and equals $\mu_\eta [v,v_m]$, where $d=\frac{(\sum \rho_j+4)(\kappa-4-2\sum \rho_j)}{2\kappa}$ is the Hausdorff dimension of $\eta\cap [v,v_m]$. In the case that all $\rho_j=0$, this measure agrees with the covariant measure derived in [Alberts-Sheffield, 2011] for chordal SLE$_\kappa$ up to a multiplicative constant. %Such measure, called Minkowski content measure, satisfies conformal covariance properties. We call such measure a Minkowski content measure, extend it to a class of subsets of ${\mathbb{R}}^n$, and prove that they satisfy conformal covariance. To construct the Minkowski content measure on $\eta\cap [v,v_m]$, we follow the standard approach to derive the existence and estimates of the one- and two-point boundary Green's functions of $\eta$ on $(-\infty,v_m)$, which are the limits of the rescaled probability that $\eta$ passes through small discs or open real intervals centered at points on $(-\infty,v_m)$.


title: What makes visual place recognition easy or hard?

abstract: Visual place recognition is a fundamental capability for the localization of mobile robots. It places image retrieval in the practical context of physical agents operating in a physical world. It is an active field of research and many different approaches have been proposed and evaluated in many different experiments. In the following, we argue that due to variations of this practical context and individual design decisions, place recognition experiments are barely comparable across different papers and that there is a variety of properties that can change from one experiment to another. We provide an extensive list of such properties and give examples how they can be used to setup a place recognition experiment easier or harder. This might be interesting for different involved parties: (1) people who just want to select a place recognition approach that is suitable for the properties of their particular task at hand, (2) researchers that look for open research questions and are interested in particularly difficult instances, (3) authors that want to create reproducible papers on this topic, and (4) also reviewers that have the task to identify potential problems in papers under review.


title: Charformer: Fast Character Transformers via Gradient-based Subword Tokenization

abstract: State-of-the-art models in natural language processing rely on separate rigid subword tokenization algorithms, which limit their generalization ability and adaptation to new settings. In this paper, we propose a new model inductive bias that learns a subword tokenization end-to-end as part of the model. To this end, we introduce a soft gradient-based subword tokenization module (GBST) that automatically learns latent subword representations from characters in a data-driven fashion. Concretely, GBST enumerates candidate subword blocks and learns to score them in a position-wise fashion using a block scoring network. We additionally introduce Charformer, a deep Transformer model that integrates GBST and operates on the byte level. Via extensive experiments on English GLUE, multilingual, and noisy text datasets, we show that Charformer outperforms a series of competitive byte-level baselines while generally performing on par and sometimes outperforming subword-based models. Additionally, Charformer is fast, improving the speed of both vanilla byte-level and subword-level Transformers by 28%-100% while maintaining competitive quality. We believe this work paves the way for highly performant token-free models that are trained completely end-to-end.


title: Conditional Deformable Image Registration with Convolutional Neural Network

abstract: Recent deep learning-based methods have shown promising results and runtime advantages in deformable image registration. However, analyzing the effects of hyperparameters and searching for optimal regularization parameters prove to be too prohibitive in deep learning-based methods. This is because it involves training a substantial number of separate models with distinct hyperparameter values. In this paper, we propose a conditional image registration method and a new self-supervised learning paradigm for deep deformable image registration. By learning the conditional features that are correlated with the regularization hyperparameter, we demonstrate that optimal solutions with arbitrary hyperparameters can be captured by a single deep convolutional neural network. In addition, the smoothness of the resulting deformation field can be manipulated with arbitrary strength of smoothness regularization during inference. Extensive experiments on a large-scale brain MRI dataset show that our proposed method enables the precise control of the smoothness of the deformation field without sacrificing the runtime advantage or registration accuracy.


title: Fairness via Representation Neutralization

abstract: Existing bias mitigation methods for DNN models primarily work on learning debiased encoders. This process not only requires a lot of instance-level annotations for sensitive attributes, it also does not guarantee that all fairness sensitive information has been removed from the encoder. To address these limitations, we explore the following research question: Can we reduce the discrimination of DNN models by only debiasing the classification head, even with biased representations as inputs? To this end, we propose a new mitigation technique, namely, Representation Neutralization for Fairness (RNF) that achieves fairness by debiasing only the task-specific classification head of DNN models. To this end, we leverage samples with the same ground-truth label but different sensitive attributes, and use their neutralized representations to train the classification head of the DNN model. The key idea of RNF is to discourage the classification head from capturing spurious correlation between fairness sensitive information in encoder representations with specific class labels. To address low-resource settings with no access to sensitive attribute annotations, we leverage a bias-amplified model to generate proxy annotations for sensitive attributes. Experimental results over several benchmark datasets demonstrate our RNF framework to effectively reduce discrimination of DNN models with minimal degradation in task-specific performance.


title: HyPhy: Deep Generative Conditional Posterior Mapping of Hydrodynamical Physics

abstract: Generating large volume hydrodynamical simulations for cosmological observables is a computationally demanding task necessary for next generation observations. In this work, we construct a novel fully convolutional variational auto-encoder (VAE) to synthesize hydrodynamic fields conditioned on dark matter fields from N-body simulations. After training the model on a single hydrodynamical simulation, we are able to probabilistically map new dark matter only simulations to corresponding full hydrodynamical outputs. By sampling over the latent space of our VAE, we can generate posterior samples and study the variance of the mapping. We find that our reconstructed field provides an accurate representation of the target hydrodynamical fields as well as a reasonable variance estimates. This approach has promise for the rapid generation of mocks as well as for implementation in a full Bayesian inverse model of observed data.


title: Plasmonic drag photocurrent in graphene at extreme nonlocality

abstract: It is commonly assumed that photocurrent in two-dimensional systems with centrosymmetric lattice is generated at structural inhomogenities, such as p-n junctions. Here, we study an alternative mechanism of photocurrent generation associated with inhomogenity of the driving electromagnetic field, termed as 'plasmonic drag'. It is associated with direct momentum transfer from field to conduction electrons, and can be characterized by a non-local non-linear conductivity $\sigma^{(2)}({\bf q},\omega)$. By constructing a classical kinetic model of non-linear conductivity with full account of non-locality, we show that it is resonantly enhanced for wave phase velocity coinciding with electron Fermi velocity. The enhancement is interpreted as phase locking between electrons and the wave. We discuss a possible experiment where non-uniform field is created by a propagating graphene plasmon, and find an upper limit of the current responsivity vs plasmon velocity. This limit is set by a competition between resonantly growing $\sigma^{(2)}({\bf q},\omega)$ and diverging kinetic energy of electrons as the wave velocity approaches Fermi velocity.


title: Optimal estimation of coarse structural nested mean models with application to initiating ART in HIV infected patients

abstract: Coarse structural nested mean models are used to estimate treatment effects from longitudinal observational data. Coarse structural nested mean models lead to a large class of estimators. It turns out that estimates and standard errors may differ considerably within this class. We prove that, under additional assumptions, there exists an explicit solution for the optimal estimator within the class of coarse structural nested mean models. Moreover, we show that even if the additional assumptions do not hold, this optimal estimator is doubly-robust: it is consistent and asymptotically normal not only if the model for treatment initiation is correct, but also if a certain outcome-regression model is correct. We compare the optimal estimator to some naive choices within the class of coarse structural nested mean models in a simulation study. Furthermore, we apply the optimal and naive estimators to study how the CD4 count increase due to one year of antiretroviral treatment (ART) depends on the time between HIV infection and ART initiation in recently infected HIV infected patients. Both in the simulation study and in the application, the use of optimal estimators leads to substantial increases in precision.


title: Native Implementation of Mutable Value Semantics

abstract: Unrestricted mutation of shared state is a source of many well-known problems. The predominant safe solutions are pure functional programming, which bans mutation outright, and flow sensitive type systems, which depend on sophisticated typing rules. Mutable value semantics is a third approach that bans sharing instead of mutation, thereby supporting part-wise in-place mutation and local reasoning, while maintaining a simple type system. In the purest form of mutable value semantics, references are second-class: they are only created implicitly, at function boundaries, and cannot be stored in variables or object fields. Hence, variables can never share mutable state. Because references are often regarded as an indispensable tool to write efficient programs, it is legitimate to wonder whether such a discipline can compete other approaches. As a basis for answering that question, we demonstrate how a language featuring mutable value semantics can be compiled to efficient native code. This approach relies on stack allocation for static garbage collection and leverages runtime knowledge to sidestep unnecessary copies.


title: Correlation Between Spin and Orbital Dynamics During Laser-Induced Femtosecond Demagnetization

abstract: Spin and orbital angular momenta are two intrinsic properties of an electron and are responsible for the physics of a solid. How the spin and orbital evolve with respect to each other on several hundred femtoseconds is largely unknown, but it is at the center of laser-induced ultrafast demagnetization. In this paper, we introduce a concept of the spin-orbital correlation diagram, where spin angular momentum is plotted against orbital angular momentum, much like the position-velocity phase diagram in classical mechanics. We use four sets of highly accurate time-resolved x-ray magnetic circular dichroism (TR-XMCD) data to construct four correlation diagrams for iron and cobalt. To our surprise, a pattern emerges. The trace on the correlation diagram for iron is an arc, and at the end of demagnetization, it has a pronounced cusp. The correlation diagram for cobalt is different and appears more linear, but with kinks. We carry out first-principles calculations with two different methods: time-dependent density functional theory (TDDFT) and time-dependent Liouville density functional theory (TDLDFT). These two methods agree that the experimental findings for both Fe and Co are not due t experimental errors. It is the spin-orbit coupling that correlates the spin dynamics to the orbital dynamics.Microscopically, Fe and Co have different orbital occupations, which leads to distinctive correlation diagrams. We believe that this correlation diagram presents a useful tool to better understand spin and orbital dynamics on an ultrafast time scale. A brief discussion on the magnetic anisotropy energy is also provided.


title: Non-diffusive Variational Problems with Distributional and Weak Gradient Constraints

abstract: In this paper, we consider non-diffusive variational problems with mixed boundary conditions and (distributional and weak) gradient constraints. The upper bound in the constraint is either a function or a Borel measure, leading to the state space being a Sobolev one or the space of functions of bounded variation. We address existence and uniqueness of the model under low regularity assumptions, and rigorously identify its Fenchel pre-dual problem. The latter in some cases is posed on a non-standard space of Borel measures with square integrable divergences. We also establish existence and uniqueness of solutions to this pre-dual problem under some assumptions. We conclude the paper by introducing a mixed finite-element method to solve the primal-dual system. The numerical examples confirm our theoretical findings.


title: Geodesic bicombings on some hyperspaces

abstract: We show that if $(X,d)$ is a metric space which admits a consistent convex geodesic bicombing, then we can construct a conical bicombing on $CB(X)$, the hyperspace of nonempty, closed, bounded, and convex subsets of $X$ (with the Hausdorff metric). If $X$ is a normed space or an $\mathbb{R}$-tree, this same method produces a consistent convex bicombing on $CB(X)$. We follow this by examining a geodesic bicombing on the nonempty compact subsets of $X$, assuming $X$ is a proper metric space.


title: Atomic-scale Visualization of Electronic Fluid Flow

abstract: The most essential characteristic of any fluid is the velocity field v(r) and this is particularly true for macroscopic quantum fluids. Although rapid advances have occurred in quantum fluid v(r) imaging, the velocity field of a charged superfluid - a superconductor - has never been visualized. Here we use superconductive-tip scanning tunneling microscopy to image the electron-pair density \r{ho}_S(r) and velocity v_S(r) fields of the flowing electron-pair fluid in superconducting NbSe2. Imaging v_S(r) surrounding a quantized vortex finds speeds reaching 10,000 km/hr. Together with independent imaging of \r{ho}_S(r) via Josephson tunneling, we visualize the supercurrent density j_S(r)=\r{ho}_S(r)v_S(r), which peaks above 3 x 10^7 A/cm^2. The spatial patterns in electronic fluid flow and magneto-hydrodynamics reveal hexagonal structures co-aligned to the crystal lattice and quasiparticle bound states, as long anticipated. These novel techniques pave the way for electronic fluid flow visualization in many other quantum fluids.


title: Resolving the formation of cold HI filaments in the high velocity cloud complex C

abstract: The physical properties of galactic halo gas have a profound impact on the life cycle of galaxies. As gas travels through a galactic halo, it undergoes dynamical interactions, influencing its impact on star formation and the chemical evolution of the galactic disk. In the Milky-Way halo, considerable effort has been made to understand the spatial distribution of neutral gas, which are mostly in the form of large complexes. However, the internal variations of their physical properties remains unclear. In this study, we investigate the thermal and dynamical state of the neutral gas in HVCs. High-resolution observations (1.'1) of the 21 cm line emission in the EN field of the DHIGLS HI survey are used to analyze the physical properties of the bright concentration C I B located at an edge of complex C. We use the Gaussian decomposition code ROHSA to model its multiphase content, and perform a power spectrum analysis to analyze its multi-scale structure. Physical properties of some 200 structures extracted using dendrograms are examined. We identify two distinct regions, one of which has a prominent protrusion extending from the edge of complex C that exhibits an ongoing phase transition from warm diffuse gas to cold dense gas and filaments. The scale at which the warm gas becomes unstable and undergoes a thermal condensation is about 15 pc, corresponding to a cooling time about 1.5 Myr. We find that a transition from subsonic to trans-sonic turbulence is associated with the thermal condensation. A large scale perspective of complex C suggests that hydrodynamic instabilities are involved in creating the structured concentration C I B and the phase transition therein. However, the details of the dynamical and thermal processes remain unclear and will require further investigation, through both observations and numerical simulations. (Shortened for arxiv)


title: A Pure HTTP/3 Alternative to MQTT-over-QUIC in Resource-Constrained IoT

abstract: In this paper, we address the issue of scalable, interoperable, and timely dissemination of information in resource-constrained IoT. Scalability is addressed by adopting a publish-subscribe architecture. To address interoperable and timely dissemination, we propose an HTTP/3 (H3) solution that exploits the wide-ranging improvements made over H2. We evaluated our solution by comparing it to a state-of-the-art work: MQTT-over-QUIC. Because QUIC and H3 have undergone standardization in tandem, we hypothesized that H3 would take better advantage of QUIC transport than an MQTT mapping would. Performance, network overhead, and device overhead were investigated for both protocols. Our H3-based solution satisfied our timely dissemination requirement by offering a key performance savings of 1 RoundTrip Time (RTT) for publish messages to arrive at the broker. In IoT networks, with typically high RTT, this savings is significant. On the other hand, we found that MQTT-over-QUIC put marginally less strain over the network.


title: Efficient ab initio many-body calculations based on sparse modeling of Matsubara Green's function

abstract: This lecture note reviews recently proposed sparse-modeling approaches for efficient ab initio many-body calculations based on the data compression of Green's functions. The sparse-modeling techniques are based on a compact orthogonal basis, an intermediate representation (IR) basis, for imaginary-time and Matsubara Green's functions. A sparse sampling method based on the IR basis enables solving diagrammatic equations efficiently. We describe the basic properties of the IR basis, the sparse sampling method and its applications to ab initio calculations based on the GW approximation and the Migdal-Eliashberg theory. We also describe a numerical library for the IR basis and the sparse sampling method, sparse-ir, and provide its sample codes. This lecture note follows the Japanese review article with major revisions [H. Shinaoka et al., Solid State Physics 56(6), 301 (2021)].

categories: cond-mat.str-el cond-mat.mtrl-sci cond-mat.supr-con physics.comp-ph


title: Revisiting Gini for equitable humanitarian logistics

abstract: Modeling equity in the allocation of scarce resources is a fast-growing concern in the humanitarian logistics field. The Gini coefficient is one of the most widely recognized measures of inequity and it was originally characterized by means of the Lorenz curve, which is a mathematical function that links the cumulative share of income to rank-ordered groups in a population. So far, humanitarian logistics models that have approached equity using the Gini coefficient do not actually optimize its original formulation, but use alternative definitions that do not necessarily replicate that original Gini measure. In this paper, we derive the original Gini coefficient via the Lorenz curve to optimize the effectiveness-equity trade-off in a humanitarian location-allocation problem. We also propose new valid inequalities based on an upper-bounding Lorenz curve to tighten the linear relaxation of our model and develop a clustering-based construction of the Lorenz curve that requires fewer additional constraints and variables than the original one. The computational study, based on the floods and landslides in Rio de Janeiro state, Brazil, reveals that while alternative Gini definitions have interesting properties, they can generate vastly different decisions compared to the original Gini coefficient. In addition, viewed from the perspective of the original Gini coefficient, these decisions can be significantly less equitable.


title: Transmission and Capacity Expansion Planning Against Rising Temperatures: A Case Study in Arizona

abstract: The stable and efficient operation of the transmission network is fundamental to the power system's ability to deliver electricity reliably and cheaply. As average temperatures continue to rise, the ability of the transmission network to meet demand is diminished. Higher temperatures lead to congestion by reducing thermal limits of lines while simultaneously reducing generation potential. Due to prohibitive costs and limited real estate for building new lines, it is necessary to consider capacity expansion as well to improve the functioning and efficiency of the grid. Optimal control, however, requires many discrete choices, rendering fully accurate models intractable. Furthermore, temperature changes will impact different regions and climate differently. As such, it is necessary to model both temperature changes and transmission flows with high spatial resolution. This work proposes a case study of the transmission grid centered in Arizona, using a DC optimal power flow mathematical formulation to plan for future transmission expansion and capacity expansion to efficiently meet demand. The effects of rising temperatures on transmission and generation are modeled at the regional level. Several classes of valid inequalities are employed to speed up the solution process. Multiple experiments considering different temperature and demand trends are considered which include each of the above technologies.


title: Best-Case Lower Bounds in Online Learning

abstract: Much of the work in online learning focuses on the study of sublinear upper bounds on the regret. In this work, we initiate the study of best-case lower bounds in online convex optimization, wherein we bound the largest improvement an algorithm can obtain relative to the single best action in hindsight. This problem is motivated by the goal of better understanding the adaptivity of a learning algorithm. Another motivation comes from fairness: it is known that best-case lower bounds are instrumental in obtaining algorithms for decision-theoretic online learning (DTOL) that satisfy a notion of group fairness. Our contributions are a general method to provide best-case lower bounds in Follow The Regularized Leader (FTRL) algorithms with time-varying regularizers, which we use to show that best-case lower bounds are of the same order as existing upper regret bounds: this includes situations with a fixed learning rate, decreasing learning rates, timeless methods, and adaptive gradient methods. In stark contrast, we show that the linearized version of FTRL can attain negative linear regret. Finally, in DTOL with two experts and binary predictions, we fully characterize the best-case sequences, which provides a finer understanding of the best-case lower bounds.


title: A Unifying Modeling Abstraction for Infinite-Dimensional Optimization

abstract: Infinite-dimensional optimization (InfiniteOpt) problems involve modeling components (variables, objectives, and constraints) that are functions defined over infinite-dimensional domains. Examples include continuous-time dynamic optimization (time is an infinite domain and components are a function of time), PDE optimization problems (space and time are infinite domains and components are a function of space-time), as well as stochastic and semi-infinite optimization (random space is an infinite domain and components are a function of such random space). InfiniteOpt problems also arise from combinations of these problem classes (e.g., stochastic PDE optimization). Given the infinite-dimensional nature of objectives and constraints, one often needs to define appropriate quantities (measures) to properly pose the problem. Moreover, InfiniteOpt problems often need to be transformed into a finite dimensional representation so that they can be solved numerically. In this work, we present a unifying abstraction that facilitates the modeling, analysis, and solution of InfiniteOpt problems. The proposed abstraction enables a general treatment of infinite-dimensional domains and provides a measure-centric paradigm to handle associated variables, objectives, and constraints. This abstraction allows us to transfer techniques across disciplines and with this identify new, interesting, and useful modeling paradigms (e.g., event constraints and risk measures defined over time domains). Our abstraction serves as the backbone of an intuitive Julia-based modeling package that we call InfiniteOpt.jl. We demonstrate the developments using diverse case studies arising in engineering.


title: Absolute instability in shock-containing jets

abstract: We present an analysis of the linear stability characteristics of shock-containing jets. The flow is linearised around a spatially periodic mean, which acts as a surrogate for a mean flow with a shock-cell structure, leading to a set of partial differential equations with periodic coefficients in space. Disturbances are written using the Floquet ansatz and Fourier modes in the streamwise direction, leading to an eigenvalue problem for the Floquet exponent. The characteristics of the solution are directly compared to the locally parallel case, and some of the features are similar. The inclusion of periodicity induces minor changes in the growth rate and phase velocity of the relevant modes for small shock amplitudes. On the other hand, the eigenfunctions are now subject to modulation related to the periodicity of the flow. Analysis of the spatio-temporal growth rates led to the identification of a saddle point between the Kelvin-Helmholtz mode and the guided jet mode, characterising an absolute instability mechanism. Frequencies and mode shapes related to the saddle points for two conditions (associated with axisymmetric and helical modes) are compared with screech frequencies and the most energetic coherent structures of screeching jets, resulting in a good agreement for both. The analysis shows that a periodic shock-cell structure has an impulse response that grows upstream, leading to oscillator behaviour. The results suggest that screech can occur in the absence of a nozzle, and that the upstream reflection condition is not essential for screech frequency selection. Connections to previous models are also discussed.


title: Damping of the Franz-Keldysh oscillations in the presence of disorder

abstract: Franz-Keldysh oscillations of the optical absorption in the presence of short-range disorder are studied theoretically. The magnitude of the effect depends on the relation between the mean-free path in a zero field and the distance between the turning points in electric field. Damping of the Franz-Keldysh oscillations by the disorder develops at high absorption frequency. Effect of damping is amplified by the fact that, that electron and hole are most sensitive to the disorder near the turning points. This is because, near the turning points, velocities of electron and hole turn to zero.


title: Dungeon and Platformer Level Blending and Generation using Conditional VAEs

abstract: Variational autoencoders (VAEs) have been used in prior works for generating and blending levels from different games. To add controllability to these models, conditional VAEs (CVAEs) were recently shown capable of generating output that can be modified using labels specifying desired content, albeit working with segments of levels and platformers exclusively. We expand these works by using CVAEs for generating whole platformer and dungeon levels, and blending levels across these genres. We show that CVAEs can reliably control door placement in dungeons and progression direction in platformer levels. Thus, by using appropriate labels, our approach can generate whole dungeons and platformer levels of interconnected rooms and segments respectively as well as levels that blend dungeons and platformers. We demonstrate our approach using The Legend of Zelda, Metroid, Mega Man and Lode Runner.


title: Deep Learning for Network Traffic Classification

abstract: Monitoring network traffic to identify content, services, and applications is an active research topic in network traffic control systems. While modern firewalls provide the capability to decrypt packets, this is not appealing for privacy advocates. Hence, identifying any information from encrypted traffic is a challenging task. Nonetheless, previous work has identified machine learning methods that may enable application and service identification. The process involves high level feature extraction from network packet data then training a robust machine learning classifier for traffic identification. We propose a classification technique using an ensemble of deep learning architectures on packet, payload, and inter-arrival time sequences. To our knowledge, this is the first time such deep learning architectures have been applied to the Server Name Indication (SNI) classification problem. Our ensemble model beats the state of the art machine learning methods and our up-to-date model can be found on github: \url{https://github.com/niloofarbayat/NetworkClassification}


title: Long short-term relevance learning

abstract: To incorporate prior knowledge as well as measurement uncertainties in the traditional long short term memory (LSTM) neural networks, an efficient sparse Bayesian training algorithm is introduced to the network architecture. The proposed scheme automatically determines relevant neural connections and adapts accordingly, in contrast to the classical LSTM solution. Due to its flexibility, the new LSTM scheme is less prone to overfitting, and hence can approximate time dependent solutions by use of a smaller data set. On a structural nonlinear finite element application we show that the self-regulating framework does not require prior knowledge of a suitable network architecture and size, while ensuring satisfying accuracy at reasonable computational cost.


title: Intrinsic nonlinear Hall effect in antiferromagnetic tetragonal CuMnAs

abstract: Detecting the orientation of the N\'eel vector is a major research topic in antiferromagnetic spintronics. Here we recognize the intrinsic nonlinear Hall effect, which is independent of the relaxation time, as a prominent contribution to the time-reversal-odd second order conductivity and can be used to detect the flipping of the N\'eel vector. In contrast, the Berry-curvature-dipole-induced nonlinear Hall effect depends linear on relaxation time and is time-reversal-even. We study the intrinsic nonlinear Hall effect in an antiferromagnetic metal: tetragonal CuMnAs, and show that its nonlinear Hall conductivity can reach the order of mA/V$^2$. The dependence on the chemical potential of such nonlinear Hall conductivity can be qualitatively explained by a tilted massive Dirac model. Moreover, we demonstrate its strong temperature dependence and briefly discuss its competition with the second order Drude conductivity. Finally, a complete survey of magnetic point groups are presented, providing guidelines for finding more antiferromagnetic materials with the intrinsic nonlinear Hall effect.


title: Comment on Coleman-DeLuccia Instantons

abstract: We complete an old argument that causal diamonds in the crunching region of the Lorentzian continuation of a Coleman-Deluccia instanton for transitions out of de Sitter space have finite area, and provide quantum models consistent with the principle of detailed balance, which can mimic the instanton transition probabilities for the cases where this diamond is larger or smaller than the causal patch of de Sitter space. We review arguments that potentials which do not have a positive energy theorem when the lowest de Sitter minimum is shifted to zero, may not correspond to real models of quantum gravity.


title: Bounded reduction of orthogonal matrices over polynomial rings

abstract: We prove that a matrix from the split orthogonal group over a polynomial ring with coefficients in a small-dimensional ring can be reduced to a smaller matrix by a bounded number of elementary orthogonal transformations. The bound is given explicitly. This result is an effective version of the early stabilisation of the orthogonal K1 functor proven by Suslin and Kopeiko. Since the similar effective results for special linear and symplectic groups were obtained by Vaserstein, the present paper closes the problem for split classical groups.


title: Comparative Error Analysis in Neural and Finite-state Models for Unsupervised Character-level Transduction

abstract: Traditionally, character-level transduction problems have been solved with finite-state models designed to encode structural and linguistic knowledge of the underlying process, whereas recent approaches rely on the power and flexibility of sequence-to-sequence models with attention. Focusing on the less explored unsupervised learning scenario, we compare the two model classes side by side and find that they tend to make different types of errors even when achieving comparable performance. We analyze the distributions of different error classes using two unsupervised tasks as testbeds: converting informally romanized text into the native script of its language (for Russian, Arabic, and Kannada) and translating between a pair of closely related languages (Serbian and Bosnian). Finally, we investigate how combining finite-state and sequence-to-sequence models at decoding time affects the output quantitatively and qualitatively.


title: Distilling the Knowledge from Conditional Normalizing Flows

abstract: Normalizing flows are a powerful class of generative models demonstrating strong performance in several speech and vision problems. In contrast to other generative models, normalizing flows are latent variable models with tractable likelihoods and allow for stable training. However, they have to be carefully designed to represent invertible functions with efficient Jacobian determinant calculation. In practice, these requirements lead to overparameterized and sophisticated architectures that are inferior to alternative feed-forward models in terms of inference time and memory consumption. In this work, we investigate whether one can distill flow-based models into more efficient alternatives. We provide a positive answer to this question by proposing a simple distillation approach and demonstrating its effectiveness on state-of-the-art conditional flow-based models for image super-resolution and speech synthesis.


title: An Efficient Group-based Search Engine Marketing System for E-Commerce

abstract: With the increasing scale of search engine marketing, designing an efficient bidding system is becoming paramount for the success of e-commerce companies. The critical challenges faced by a modern industrial-level bidding system include: 1. the catalog is enormous, and the relevant bidding features are of high sparsity; 2. the large volume of bidding requests induces significant computation burden to both the offline and online serving. Leveraging extraneous user-item information proves essential to mitigate the sparsity issue, for which we exploit the natural language signals from the users' query and the contextual knowledge from the products. In particular, we extract the vector representations of ads via the Transformer model and leverage their geometric relation to building collaborative bidding predictions via clustering. The two-step procedure also significantly reduces the computation stress of bid evaluation and optimization. In this paper, we introduce the end-to-end structure of the bidding system for search engine marketing for Walmart e-commerce, which successfully handles tens of millions of bids each day. We analyze the online and offline performances of our approach and discuss how we find it as a production-efficient solution.


abstract: In the recent decade, electronic technology gets advanced day by day the methodologies too should update. For the purpose of ranging various methods such Radio Detection and Ranging (RADAR), Light Detection and Ranging (LIDAR) and Sonic Navigation and Ranging (SONAR) etc. are used. Later, by adapting the earlier technologies and further modifying the purposes of detection and ranging in navigation, the technology of Sonic Detection and Ranging (SODAR) is used in modern robotics. The SODAR can be defined as a child of SONAR and also a twin of Echo sounder. The echo-sounder is used only for ranging. But the SODAR use the low-frequency wave of 33 kHz to measure the underwater depth and also to detect the objects below the water medium. So, this work comprises the designing of a system to evaluate the Object Detection and Ranging for Autonomous Navigation of Mobile Robots.


title: A Mixed-Integer Conic Programming Formulation for Computing the Flexibility Index under Multivariate Gaussian Uncertainty

abstract: We present a methodology for computing the flexibility index when uncertainty is characterized using multivariate Gaussian random variables. Our approach computes the flexibility index by solving a mixed-integer conic program (MICP). This methodology directly characterizes ellipsoidal sets to capture correlations in contrast to previous methodologies that employ approximations. We also show that, under a Gaussian representation, the flexibility index can be used to obtain a lower bound for the so-called stochastic flexibility index (i.e., the probability of having feasible operation). Our results also show that the methodology can be generalized to capture different types of uncertainty sets.


title: GaN HEMT based high energy particles detection preamplifier

abstract: GaN high electron mobility transistors (HEMT) have gained some foothold in the power electronics industry due to wide frequency bandwidth and power handling. The material offers a wide bandgap and higher critical field strength compared to most wide bandgap semiconductors, resulting in better radiation resistance and theoretically higher speeds as the devices dimensions could be reduced without suffering voltage breakdown. This work consists of the underlying simulation work intended to examine the response of the GaN HEMTs preamlifying circuits for high resolution high energy radiation detectors. The simulation and experimental results illustrate the superior performance of the GaN HEMT in an amplifying circuit. Using a spice model for a commercially available GaN HEMT non distorted output to an input signal of 200 ps was displayed. Real world measurements underscore the fast response of the GaN HEMT with its measured slew rate at approximately 3000 V /{\mu}s a result only 17% lower than the result obtained from the simulation.


title: All unconstrained strongly convex problems are weakly simplicial

abstract: A multi-objective optimization problem is $C^r$ weakly simplicial if there exists a $C^r$ surjection from a simplex onto the Pareto set/front such that the image of each subsimplex is the Pareto set/front of a subproblem, where $0\leq r\leq \infty$. This property is helpful to compute a parametric-surface approximation of the entire Pareto set and Pareto front. It is known that all unconstrained strongly convex $C^r$ problems are $C^{r-1}$ weakly simplicial for $1\leq r \leq \infty$. In this paper, we show that all unconstrained strongly convex problems are $C^0$ weakly simplicial. The usefulness of this theorem is demonstrated in a sparse modeling application: we reformulate the elastic net as a non-differentiable multi-objective strongly convex problem and approximate its Pareto set (the set of all trained models with different hyper-parameters) and Pareto front (the set of performance metrics of the trained models) by using a B\'ezier simplex fitting method, which accelerates hyper-parameter search.


title: Alternative Microfoundations for Strategic Classification

abstract: When reasoning about strategic behavior in a machine learning context it is tempting to combine standard microfoundations of rational agents with the statistical decision theory underlying classification. In this work, we argue that a direct combination of these standard ingredients leads to brittle solution concepts of limited descriptive and prescriptive value. First, we show that rational agents with perfect information produce discontinuities in the aggregate response to a decision rule that we often do not observe empirically. Second, when any positive fraction of agents is not perfectly strategic, desirable stable points -- where the classifier is optimal for the data it entails -- cease to exist. Third, optimal decision rules under standard microfoundations maximize a measure of negative externality known as social burden within a broad class of possible assumptions about agent behavior. Recognizing these limitations we explore alternatives to standard microfoundations for binary classification. We start by describing a set of desiderata that help navigate the space of possible assumptions about how agents respond to a decision rule. In particular, we analyze a natural constraint on feature manipulations, and discuss properties that are sufficient to guarantee the robust existence of stable points. Building on these insights, we then propose the noisy response model. Inspired by smoothed analysis and empirical observations, noisy response incorporates imperfection in the agent responses, which we show mitigates the limitations of standard microfoundations. Our model retains analytical tractability, leads to more robust insights about stable points, and imposes a lower social burden at optimality.


title: A Computational Framework for Quantifying and Analyzing System Flexibility

abstract: We present a computational framework for analyzing and quantifying system flexibility. Our framework incorporates new features that include: general uncertainty characterizations that are constructed using composition of sets, procedures for computing well-centered nominal points, and a procedure for identifying and ranking flexibility-limiting constraints and critical parameter values. These capabilities allow us to analyze the flexibility of complex systems such as distribution networks.


title: Multiple orthogonal polynomials: Pearson equations and Christoffel formulas

abstract: Multiple orthogonal polynomials with respect to two weights on the step-line are considered. A connection between different dual spectral matrices, one banded (recursion matrix) and one Hessenberg, respectively, and the Gauss-Borel factorization of the moment matrix is given. It is shown a hidden freedom exhibited by the spectral system related to the multiple orthogonal polynomials. Pearson equations are discussed, a Laguerre-Freud matrix is considered, and differential equations for type I and II multiple orthogonal polynomials, as well as for the corresponding linear forms are given. The Jacobi-Pi\~neiro multiple orthogonal polynomials of type I and type II are used as an illustrating case and the corresponding differential relations are presented. A permuting Christoffel transformation is discussed, finding the connection between the different families of multiple orthogonal polynomials. The Jacobi-Pi\~neiro case provides a convenient illustration of these symmetries, giving linear relations between different polynomials with shifted and permuted parameters. We also present the general theory for the perturbation of each weight by a different polynomial or rational function aka called Christoffel and Geronimus transformations. The connections formulas between the type II multiple orthogonal polynomials, the type I linear forms, as well as the vector Stieltjes-Markov vector functions is also presented. We illustrate these findings by analyzing the special case of modification by an even polynomial.


title: A Scalable Stochastic Programming Approach for the Design of Flexible Systems

abstract: We study the problem of designing systems in order to minimize cost while meeting a given flexibility target. Flexibility is attained by enforcing a joint chance constraint, which ensures that the system will exhibit feasible operation with a given target probability level. Unfortunately, joint chance constraints are complicated mathematical objects that often need to be reformulated using mixed-integer programming (MIP) techniques. In this work, we cast the design problem as a conflict resolution problem that seeks to minimize cost while maximizing flexibility. We propose a purely continuous relaxation of this problem that provides a significantly more scalable approach relative to MIP methods and show that the formulation delivers solutions that closely approximate the Pareto set of the original joint chance-constrained problem.


title: Topological Semantic Mapping by Consolidation of Deep Visual Features

abstract: Many works in the recent literature introduce semantic mapping methods that use CNNs (Convolutional Neural Networks) to recognize semantic properties in images. The types of properties (eg.: room size, place category, and objects) and their classes (eg.: kitchen and bathroom, for place category) are usually predefined and restricted to a specific task. Thus, all the visual data acquired and processed during the construction of the maps are lost and only the recognized semantic properties remain on the maps. In contrast, this work introduces a topological semantic mapping method that uses deep visual features extracted by a CNN (GoogLeNet), from 2D images captured in multiple views of the environment as the robot operates, to create, through averages, consolidated representations of the visual features acquired in the regions covered by each topological node. These representations allow flexible recognition of semantic properties of the regions and use in other visual tasks. Experiments with a real-world indoor dataset showed that the method is able to consolidate the visual features of regions and use them to recognize objects and place categories as semantic properties, and to indicate the topological location of images, with very promising results.


title: Certifying solution geometry in random CSPs: counts, clusters and balance

abstract: An active topic in the study of random constraint satisfaction problems (CSPs) is the geometry of the space of satisfying or almost satisfying assignments as the function of the density, for which a precise landscape of predictions has been made via statistical physics-based heuristics. In parallel, there has been a recent flurry of work on refuting random constraint satisfaction problems, via nailing refutation thresholds for spectral and semidefinite programming-based algorithms, and also on counting solutions to CSPs. Inspired by this, the starting point for our work is the following question: what does the solution space for a random CSP look like to an efficient algorithm? In pursuit of this inquiry, we focus on the following problems about random Boolean CSPs at the densities where they are unsatisfiable but no refutation algorithm is known. 1. Counts. For every Boolean CSP we give algorithms that with high probability certify a subexponential upper bound on the number of solutions. We also give algorithms to certify a bound on the number of large cuts in a Gaussian-weighted graph, and the number of large independent sets in a random $d$-regular graph. 2. Clusters. For Boolean $3$CSPs we give algorithms that with high probability certify an upper bound on the number of clusters of solutions. 3. Balance. We also give algorithms that with high probability certify that there are no "unbalanced" solutions, i.e., solutions where the fraction of $+1$s deviates significantly from $50\%$. Finally, we also provide hardness evidence suggesting that our algorithms for counting are optimal.


title: Characterisation of quantum betting tasks in terms of Arimoto mutual information

abstract: We introduce operational quantum tasks based on betting with risk-aversion -- or quantum betting tasks for short -- inspired by standard quantum state discrimination and classical horse betting with risk-aversion and side information. In particular, we introduce the operational tasks of quantum state betting (QSB), noisy quantum state betting (nQSB), and quantum channel betting (QCB) played by gamblers with different risk tendencies. We prove that the advantage that informative measurements (non-constant channels) provide in QSB (nQSB) is exactly characterised by Arimoto's $\alpha$-mutual information, with the order $\alpha$ determining the risk aversion of the gambler. More generally, we show that Arimoto-type information-theoretic quantities characterise the advantage that resourceful objects offer at playing quantum betting tasks when compared to resourceless objects, for general quantum resource theories (QRTs) of measurements, channels, states, and state-measurement pairs, with arbitrary resources. In limiting cases, we show that QSB (QCB) recovers the known tasks of quantum state (channel) discrimination when $\alpha \rightarrow \infty$, and quantum state (channel) exclusion when $\alpha \rightarrow -\infty$. Inspired by these connections, we also introduce new quantum R\'enyi divergences for measurements, and derive a new family of resource monotones for the QRT of measurement informativeness. This family of resource monotones recovers in the same limiting cases as above, the generalised robustness and the weight of informativeness. Altogether, these results establish a broad and continuous family of four-way correspondences between operational tasks, mutual information measures, quantum R\'enyi divergences, and resource monotones, that can be seen to generalise two limiting correspondences that were recently discovered for the QRT of measurement informativeness.


title: Measuring and Optimizing System Reliability: A Stochastic Programming Approach

abstract: We propose a computational framework to quantify (measure) and to optimize the reliability of complex systems. The approach uses a graph representation of the system that is subject to random failures of its components (nodes and edges). Under this setting, reliability is defined as the probability of finding a path between sources and sink nodes under random component failures and we show that this measure can be computed by solving a stochastic mixed-integer program. The stochastic programming setting allows us to account for system constraints and general probability distributions to characterize failures and allows us to derive optimization formulations that identify designs of maximum reliability. We also propose a strategy to approximately solve these problems in a scalable manner by using purely continuous formulations.


title: Varifold solutions of the two-phase three-dimensional magnetohydrodynamic equations

abstract: In this paper we study the three-dimensional two-phase magnetohydrodynamic interface problem in a bounded domain. The two incompressible fluids are both Newtonian and the surface tension is considered. We shall use the Galerkin method to construct the approximate solutions in a bounded domain. Due to the magnetic field in the magnetohydrodynamic equations, we cannot use the method of monotone operators to solve the approximate equations. Instead, we will construct an iterating operator and solve the equations by finding the fixed-point of the operator. To deal with the free interface, we shall prove the compactness of the iterating operator and then use the Schauder fixed-point theorem. The existence of the varifold solution is established by the weak convergence method.


title: Circuit Masking: From Theory to Standardization, A Comprehensive Survey for Hardware Security Researchers and Practitioners

abstract: Side-channel attacks extracting sensitive data from implementations have been considered a major threat to the security of cryptographic schemes. This has elevated the need for improved designs by embodying countermeasures, with masking being the most prominent example. To formally verify the security of a masking scheme, numerous attack models have been developed to capture the physical properties of the information leakage as well as the capabilities of the adversary. With regard to these models, extensive research has been performed to realize masking schemes. These research efforts have led to significant progress in the development of security assessment methodologies and further initiated standardization activities. However, since the majority of this work is theoretical, it is challenging for the more practice-oriented hardware security community to fully grasp and contribute to. To bridge the gap, these advancements are reviewed and discussed in this survey, mainly from the perspective of hardware security. In doing so, a clear taxonomy is provided that is helpful for a systematic treatment of the masking-related topics. By giving an extensive overview of the existing methods, this survey (1) provides a research landscape of circuit masking for newcomers to the field, (2) offers guidelines on which attack model and verification tool to choose when designing masking schemes, and (3) identifies interesting new research directions where masking models and assessment tools can be applied. Thus, this survey serves as an essential reference for hardware security practitioners interested in the theory behind masking techniques, the tools useful to verify the security of masked circuits, and their potential applications.


title: The First Data Release of LAMOST Low Resolution Single Epoch Spectra

abstract: LAMOST Data Release 5, covering $\sim$17,000 $deg^2$ from $-10^{\circ}$ to $80^{\circ}$ in declination, contains 9 millions co-added low resolution spectra of celestial objects, each spectrum combined from repeat exposure of two to tens of times during Oct 2011 to Jun 2017. In this paper, We present the spectra of individual exposures for all the objects in LAMOST Data Release 5. For each spectrum, equivalent width of 60 lines from 11 different elements are calculated with a new method combining the actual line core and fitted line wings. For stars earlier than F type, the Balmer lines are fitted with both emission and absorption profiles once two components are detected. Radial velocity of each individual exposure is measured by minimizing ${\chi}^2$ between the spectrum and its best template. Database for equivalent widths of spectral lines and radial velocities of individual spectra are available online. Radial velocity uncertainties with different stellar type and signal-to-noise ratio are quantified by comparing different exposure of the same objects. We notice that the radial velocity uncertainty depends on the time lag between observations. For stars observed in the same day and with signal-to-noise ratio higher than 20, the radial velocity uncertainty is below 5km/s, and increase to 10km/s for stars observed in different nights.


title: Na Site Doping a Pathway for Enhanced Thermoelectric Performance in $Na_{1-x}CoO_2$; The Case of Gd and Yb Dopants

abstract: Doping is considered to be the main method for improving the thermoelectric performance of layered sodium cobaltate (Na$_{1-x}$CoO$_2$). However, in the vast majority of past reports, the equilibrium location of the dopant in the Na$_{1-x}$CoO$_2$'s complex layered lattice has not been confidently identified. Consequently, a universal strategy for choosing a suitable dopant for enhancing Na$_{1-x}$CoO$_2$'s figure of merit is yet to be established. Here, by examining the formation energy of Gd and Yb dopants in Na$_{0.75}$CoO$_2$ and Na$_{0.50}$CoO$_2$, we demonstrate that in an oxygen poor environment, Gd and Yb dopants reside in the Na layer while in an oxygen rich environment these dopants replace a Co in CoO$_2$ layer. When at Na layer, Gd and Yb dopants reduce the carrier concentration via electron-hole recombination, simultaneously increasing the Seebeck coefficient ($S$) and reducing electric conductivity ($\sigma$). Na site doping, however, improves the thermoelectric power factor (PF) only in Na$_{0.50}$CoO$_2$. When replacing a Co, these dopants reduce $S$ and PF. The results demonstrate how thermoelectric performance critically depends on the synthesis environment that must be fine-tuned for achieving any thermoelectric enhancement.


title: On the continuity of half-plane capacity with respect to Carath\'eodory convergence

abstract: We study the continuity of half-plane capacity as a function of boundary hulls with respect to the Carath\'eodory convergence. In particular, our interest lies in the case that hulls are unbounded. Under the assumption that every hull is contained in a fixed hull with finite imaginary part and finite half-plane capacity, we show that the half-plane capacity is indeed continuous. We also discuss the extension of this result to the case that the underlying domain is finitely connected.


title: Sparse Flows: Pruning Continuous-depth Models

abstract: Continuous deep learning architectures enable learning of flexible probabilistic models for predictive modeling as neural ordinary differential equations (ODEs), and for generative modeling as continuous normalizing flows. In this work, we design a framework to decipher the internal dynamics of these continuous depth models by pruning their network architectures. Our empirical results suggest that pruning improves generalization for neural ODEs in generative modeling. We empirically show that the improvement is because pruning helps avoid mode-collapse and flatten the loss surface. Moreover, pruning finds efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy. We hope our results will invigorate further research into the performance-size trade-offs of modern continuous-depth models.


title: FDR Controlled Multiple Testing for Union Null Hypotheses: A Knockoff-based Approach

abstract: False discovery rate (FDR) controlling procedures provide important statistical guarantees for the replicability in signal identification based on multiple hypotheses testing. In many fields of study, FDR controlling procedures are used in high-dimensional (HD) analyses to discover features that are truly associated with the outcome. In some recent applications, data on the same set of candidate features are independently collected in multiple different studies. For example, gene expression data are collected at different facilities and with different cohorts, to identify the genetic biomarkers of multiple types of cancers. These studies provide us opportunities to identify signals by considering information from different sources (with potential heterogeneity) jointly. This paper is about how to provide FDR control guarantees for the tests of union null hypotheses of conditional independence. We present a knockoff-based variable selection method (\textit{Simultaneous knockoffs}) to identify mutual signals from multiple independent data sets, providing exact FDR control guarantees under finite sample settings. This method can work with very general model settings and test statistics. We demonstrate the performance of this method with extensive numerical studies and two real data examples.


title: All You Need is a Second Look: Towards Arbitrary-Shaped Text Detection

abstract: Arbitrary-shaped text detection is a challenging task since curved texts in the wild are of the complex geometric layouts. Existing mainstream methods follow the instance segmentation pipeline to obtain the text regions. However, arbitraryshaped texts are difficult to be depicted through one single segmentation network because of the varying scales. In this paper, we propose a two-stage segmentation-based detector, termed as NASK (Need A Second looK), for arbitrary-shaped text detection. Compared to the traditional single-stage segmentation network, our NASK conducts the detection in a coarse-to-fine manner with the first stage segmentation spotting the rectangle text proposals and the second one retrieving compact representations. Specifically, NASK is composed of a Text Instance Segmentation (TIS) network (1st stage), a Geometry-aware Text RoI Alignment (GeoAlign) module, and a Fiducial pOint eXpression (FOX) module (2nd stage). Firstly, TIS extracts the augmented features with a novel Group Spatial and Channel Attention (GSCA) module and conducts instance segmentation to obtain rectangle proposals. Then, GeoAlign converts these rectangles into the fixed size and encodes RoI-wise feature representation. Finally, FOX disintegrates the text instance into serval pivotal geometrical attributes to refine the detection results. Extensive experimental results on three public benchmarks including Total-Text, SCUTCTW1500, and ICDAR 2015 verify that our NASK outperforms recent state-of-the-art methods.


title: Hourly Warning for Strong Earthquakes

abstract: A promising perspective is presented that humans can provide hourly warning for strong land earthquakes (EQs, Ms6). Two important atmospheric electrostatic signal features are described. A table that lists 9 strong land EQs with shock time, epicenter, magnitude, weather in the region near the epicenter, precursor beginning time, and precursor duration demonstrates that at approximately several hours to one day before a strong land EQ, the weather conditions are fair near the epicenter, and an abnormal negative atmospheric electrostatic signal is very obvious. Moreover, the mechanism is explained. A method by which someone could determine the epicenter and the magnitude of a forthcoming strong EQ is suggested. Finally, the possibility of realizing hourly warning for strong land EQs in the near future is pointed out.


title: Polymers' surface interactions with molten iron: a theoretical study

abstract: Environmental concerns are the chief drive for more innovative recycling techniques for end-of-life polymeric products. One attractive option is taking advantage of C and H content of polymeric waste in steelmaking industry. In this work, we examined the interaction of two high production polymers, i.e., polyurethane and polysulfide with molten iron using ab initio molecular dynamics simulation. We demonstrate that both polymers can be used as carburizers for molten iron. Additionally, we found that light weight H$_2$ and CH$_x$ molecules were released as by-products of the polymer-molten iron interaction. The outcomes of this study will have applications in the carburization of molten iron during ladle metallurgy and waste plastic injection in electric arc furnace.


title: Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanations

abstract: Understanding and explaining the mistakes made by trained models is critical to many machine learning objectives, such as improving robustness, addressing concept drift, and mitigating biases. However, this is often an ad hoc process that involves manually looking at the model's mistakes on many test samples and guessing at the underlying reasons for those incorrect predictions. In this paper, we propose a systematic approach, conceptual counterfactual explanations (CCE), that explains why a classifier makes a mistake on a particular test sample(s) in terms of human-understandable concepts (e.g. this zebra is misclassified as a dog because of faint stripes). We base CCE on two prior ideas: counterfactual explanations and concept activation vectors, and validate our approach on well-known pretrained models, showing that it explains the models' mistakes meaningfully. In addition, for new models trained on data with spurious correlations, CCE accurately identifies the spurious correlation as the cause of model mistakes from a single misclassified test sample. On two challenging medical applications, CCE generated useful insights, confirmed by clinicians, into biases and mistakes the model makes in real-world settings.


title: Superconductivity-induced spectral weight transfer due to quantum geometry

abstract: Optical spectral weight transfer associated with the onset of superconductivity at high energy scales compared with the superconducting gap has been observed in several systems such as high-$T_c$ cuprates. While there are still debates on the origin of this phenomenon, a consensus is that it is due to strong correlation effects beyond the BCS theory. Here we show that there is another route to a nonzero spectral weight transfer based on the quantum geometry of the conduction band in multiband systems. We discuss applying this idea to the cuprates and twisted multilayer graphene.


title: Breaking the $O(n)$-Barrier in the Construction of Compressed Suffix Arrays and Suffix Trees

abstract: The suffix array and the suffix tree are the two most fundamental data structures for string processing. For a length-$n$ text, however, they use $\Theta(n \log n)$ bits of space, which is often too costly. To address this, Grossi and Vitter [STOC 2000] and, independently, Ferragina and Manzini [FOCS 2000] introduced space-efficient versions of the suffix array, known as the compressed suffix array (CSA) and the FM-index. Sadakane [SODA 2002] then showed how to augment them to obtain the compressed suffix tree (CST). For a length-$n$ text over an alphabet of size $\sigma$, these structures use only $O(n\log\sigma)$ bits. The biggest remaining open question is how efficiently they can be constructed. After two decades, the fastest algorithms still run in $O(n)$ time [Hon et al., FOCS 2003], which is $\Theta(\log_{\sigma} n)$ factor away from the lower bound of $\Omega(n/\log_{\sigma}n)$. In this paper, we make the first in 20 years improvement in $n$ for this problem by proposing a new compressed suffix array and a new compressed suffix tree which admit $o(n)$-time construction algorithms while matching the space bounds and the query times of the original CSA/CST and the FM-index. More precisely, our structures take $O(n\log\sigma)$ bits, support SA queries and full suffix tree functionality in $O(\log^{\epsilon}n)$ time per operation, and can be constructed in $O(n \min(1,\log\sigma/\sqrt{\log n}))$ time using $O(n\log\sigma)$ bits of working space. We derive this result as a corollary from a much more general reduction: We prove that all parameters of a compressed suffix array/tree (query time, space, construction time, and construction working space) can essentially be reduced to those of a data structure answering new query types that we call prefix rank and prefix selection. Using the novel techniques, we also develop a new index for pattern matching.


title: Images of multilinear polynomials on $n\times n$ upper triangular matrices over infinite fields

abstract: In this paper we prove that the image of multilinear polynomials evaluated on the algebra $UT_n(K)$ of $n\times n$ upper triangular matrices over an infinite field $K$ equals $J^r$, a power of its Jacobson ideal $J=J(UT_n(K))$. In particular, this shows that the analogue of the Lvov-Kaplansky conjecture for $UT_n(K)$ is true, solving a conjecture of Fagundes and de Mello. To prove that fact, we introduce the notion of commutator-degree of a polynomial and characterize the multilinear polynomials of commutator-degree $r$ in terms of its coefficients. It turns out that the image of a multilinear polynomial $f$ on $UT_n(K)$ is $J^r$ if and only if $f$ has commutator degree $r$.


title: Robust Misspecified Models and Paradigm Shifts

abstract: Individuals use models to guide decisions, but many models are wrong. This paper studies which misspecified models are likely to persist when individuals also entertain alternative models. Consider an agent who uses her model to learn the relationship between action choices and outcomes. The agent exhibits sticky model switching, captured by a threshold rule such that she switches to an alternative model when it is a sufficiently better fit for the data she observes. The main result provides a characterization of whether a model persists based on two key features that are straightforward to derive from the primitives of the learning environment, namely, the model's asymptotic accuracy in predicting the equilibrium pattern of observed outcomes and the 'tightness' of the prior around this equilibrium. I show that misspecified models can be robust in that they persist against a wide range of competing models -- including the correct model -- despite individuals observing an infinite amount of data. Moreover, simple misspecified models with entrenched priors can be even more robust than correctly specified models. I use this characterization to provide a learning foundation for the persistence of systemic biases in two applications. First, in an effort-choice problem, I show that overconfidence in one's ability is more robust than underconfidence. Second, a simplistic binary view of politics is more robust than the more complex correct view when individuals consume media without fully recognizing the reporting bias.


title: ATP-Net: An Attention-based Ternary Projection Network For Compressed Sensing

abstract: Compressed Sensing (CS) theory simultaneously realizes the signal sampling and compression process, and can use fewer observations to achieve accurate signal recovery, providing a solution for better and faster transmission of massive data. In this paper, a ternary sampling matrix-based method with attention mechanism is proposed with the purpose to solve the problem that the CS sampling matrices in most cases are random matrices, which are irrelative to the sampled signal and need a large storage space. The proposed method consists of three components, i.e., ternary sampling, initial reconstruction and deep reconstruction, with the emphasis on the ternary sampling. The main idea of the ternary method (-1, 0, +1) is to introduce the attention mechanism to evaluate the importance of parameters at the sampling layer after the sampling matrix is binarized (-1, +1), followed by pruning weight of parameters, whose importance is below a predefined threshold, to achieve ternarization. Furthermore, a compressed sensing algorithm especially for image reconstruction is implemented, on the basis of the ternary sampling matrix, which is called ATP-Net, i.e., Attention-based ternary projection network. Experimental results show that the quality of image reconstruction by means of ATP-Net maintains a satisfactory level with the employment of the ternary sampling matrix, i.e., the average PSNR on Set11 is 30.4 when the sampling rate is 0.25, approximately 6% improvement compared with that of DR2-Net.


title: Finite-Sample Analysis of Off-Policy TD-Learning via Generalized Bellman Operators

abstract: In temporal difference (TD) learning, off-policy sampling is known to be more practical than on-policy sampling, and by decoupling learning from data collection, it enables data reuse. It is known that policy evaluation (including multi-step off-policy importance sampling) has the interpretation of solving a generalized Bellman equation. In this paper, we derive finite-sample bounds for any general off-policy TD-like stochastic approximation algorithm that solves for the fixed-point of this generalized Bellman operator. Our key step is to show that the generalized Bellman operator is simultaneously a contraction mapping with respect to a weighted $\ell_p$-norm for each $p$ in $[1,\infty)$, with a common contraction factor. Off-policy TD-learning is known to suffer from high variance due to the product of importance sampling ratios. A number of algorithms (e.g. $Q^\pi(\lambda)$, Tree-Backup$(\lambda)$, Retrace$(\lambda)$, and $Q$-trace) have been proposed in the literature to address this issue. Our results immediately imply finite-sample bounds of these algorithms. In particular, we provide first-known finite-sample guarantees for $Q^\pi(\lambda)$, Tree-Backup$(\lambda)$, and Retrace$(\lambda)$, and improve the best known bounds of $Q$-trace in [19]. Moreover, we show the bias-variance trade-offs in each of these algorithms.


title: Deep Learning and Crystal Plasticity: A Preconditioning Approach for Accurate Orientation Evolution Prediction

abstract: Efficient and precise prediction of plasticity by data-driven models relies on appropriate data preparation and a well-designed model. Here we introduce an unsupervised machine learning-based data preparation method to maximize the trainability of crystal orientation evolution data during deformation. For Taylor model crystal plasticity data, the preconditioning procedure improves the test score of an artificial neural network from 0.831 to 0.999, while decreasing the training iterations by an order of magnitude. The efficacy of the approach was further improved with a recurrent neural network. Electron backscattered (EBSD) lab measurements of crystal rotation during rolling were compared with the results of the surrogate model, and despite error introduced by Taylor model simplifying assumptions, very reasonable agreement between the surrogate model and experiment was observed. Our method is foundational for further data-driven studies, enabling the efficient and precise prediction of texture evolution from experimental and simulated crystal plasticity results.


title: Can Perceptual Guidance Lead to Semantically Explainable Adversarial Perturbations?

abstract: It is well known that carefully crafted imperceptible perturbations can cause state-of-the-art deep learning classification models to misclassify. Understanding and analyzing these adversarial perturbations play a crucial role in the design of robust convolutional neural networks. However, their mechanics are not well understood. In this work, we attempt to understand the mechanics by systematically answering the following question: do imperceptible adversarial perturbations focus on changing the regions of the image that are important for classification? In other words, are imperceptible adversarial perturbations semantically explainable? Most current methods use $l_p$ distance to generate and characterize the imperceptibility of the adversarial perturbations. However, since $l_p$ distances only measure the pixel to pixel distances and do not consider the structure in the image, these methods do not provide a satisfactory answer to the above question. To address this issue, we propose a novel framework for generating adversarial perturbations by explicitly incorporating a "perceptual quality ball" constraint in our formulation. Specifically, we pose the adversarial example generation problem as a tractable convex optimization problem, with constraints taken from a mathematically amenable variant of the popular SSIM index. We use the MobileNetV2 network trained on the ImageNet dataset for our experiments. By comparing the SSIM maps generated by our method with class activation maps, we show that the perceptually guided perturbations introduce changes specifically in the regions that contribute to classification decisions i.e., these perturbations are semantically explainable.


title: Online Verification of Deep Neural Networks under Domain Shift or Network Updates

abstract: Although neural networks are widely used, it remains challenging to formally verify the safety and robustness of neural networks in real-world applications. Existing methods are designed to verify the network before deployment, which are limited to relatively simple specifications and fixed networks. These methods are not ready to be applied to real-world problems with complex and/or dynamically changing specifications and networks. To effectively handle such problems, verification needs to be performed online when these changes take place. However, it is still challenging to run existing verification algorithms online. Our key insight is that we can leverage the temporal dependencies of these changes to accelerate the verification process. This paper establishes a novel framework for scalable online verification to solve real-world verification problems with dynamically changing specifications and/or networks. We propose three types of acceleration algorithms: Branch Management to reduce repetitive computation, Perturbation Tolerance to tolerate changes, and Incremental Computation to reuse previous results. Experiment results show that our algorithms achieve up to $100\times$ acceleration, and thus show a promising way to extend neural network verification to real-world applications.


title: Feature Completion for Occluded Person Re-Identification

abstract: Person re-identification (reID) plays an important role in computer vision. However, existing methods suffer from performance degradation in occluded scenes. In this work, we propose an occlusion-robust block, Region Feature Completion (RFC), for occluded reID. Different from most previous works that discard the occluded regions, RFC block can recover the semantics of occluded regions in feature space. Firstly, a Spatial RFC (SRFC) module is developed. SRFC exploits the long-range spatial contexts from non-occluded regions to predict the features of occluded regions. The unit-wise prediction task leads to an encoder/decoder architecture, where the region-encoder models the correlation between non-occluded and occluded region, and the region-decoder utilizes the spatial correlation to recover occluded region features. Secondly, we introduce Temporal RFC (TRFC) module which captures the long-term temporal contexts to refine the prediction of SRFC. RFC block is lightweight, end-to-end trainable and can be easily plugged into existing CNNs to form RFCnet. Extensive experiments are conducted on occluded and commonly holistic reID benchmarks. Our method significantly outperforms existing methods on the occlusion datasets, while remains top even superior performance on holistic datasets. The source code is available at https://github.com/blue-blue272/OccludedReID-RFCnet.


title: Combining Astrometry and Elemental Abundances: The Case of the Candidate Pre-Gaia Halo Moving Groups G03-37, G18-39, and G21-22

abstract: While most moving groups are young and nearby, a small number have been identified in the Galactic halo. Understanding the origin and evolution of these groups is an important piece of reconstructing the formation history of the halo. Here we report on our analysis of three putative halo moving groups: G03-37, G18-39, and G21-22. Based on Gaia EDR3 data, the stars associated with each group show some scatter in velocity (e.g., Toomre diagram) and integrals of motion (energy, angular momentum) spaces, counter to expectations of moving-group stars. We choose the best candidate of the three groups, G21-22, for follow-up chemical analysis based on high-resolution spectroscopy of six presumptive members. Using a new Python code that uses a Bayesian method to self-consistently propagate uncertainties from stellar atmosphere solutions in calculating individual abundances and spectral synthesis, we derive the abundances of $\alpha$- (Mg, Si, Ca, Ti), Fe-peak (Cr, Sc, Mn, Fe, Ni), odd-$Z$ (Na, Al, V), and neutron-capture (Ba, Eu) elements for each star. We find that the G21-22 stars are not chemically homogeneous. Based on the kinematic analysis for all three groups and the chemical analysis for G21-22, we conclude the three are not genuine moving groups. The case for G21-22 demonstrates the benefit of combining kinematic and chemical information in identifying conatal populations when either alone may be insufficient. Comparing the integrals of motion and velocities of the six G21-22 stars with those of known structures in the halo, we tentatively associate them with the Gaia-Enceladus accretion event.


title: Multi-Modal 3D Object Detection in Autonomous Driving: a Survey

abstract: In this survey, we first introduce the background of popular sensors used for self-driving, their data properties, and the corresponding object detection algorithms. Next, we discuss existing datasets that can be used for evaluating multi-modal 3D object detection algorithms. Then we present a review of multi-modal fusion based 3D detection networks, taking a close look at their fusion stage, fusion input and fusion granularity, and how these design choices evolve with time and technology. After the review, we discuss open challenges as well as possible solutions. We hope that this survey can help researchers to get familiar with the field and embark on investigations in the area of multi-modal 3D object detection.


title: Frequency Domain Convolutional Neural Network: Accelerated CNN for Large Diabetic Retinopathy Image Classification

abstract: The conventional spatial convolution layers in the Convolutional Neural Networks (CNNs) are computationally expensive at the point where the training time could take days unless the number of layers, the number of training images or the size of the training images are reduced. The image size of 256x256 pixels is commonly used for most of the applications of CNN, but this image size is too small for applications like Diabetic Retinopathy (DR) classification where the image details are important for accurate classification. This research proposed Frequency Domain Convolution (FDC) and Frequency Domain Pooling (FDP) layers which were built with RFFT, kernel initialization strategy, convolution artifact removal and Channel Independent Convolution (CIC) to replace the conventional convolution and pooling layers. The FDC and FDP layers are used to build a Frequency Domain Convolutional Neural Network (FDCNN) to accelerate the training of large images for DR classification. The Full FDC layer is an extension of the FDC layer to allow direct use in conventional CNNs, it is also used to modify the VGG16 architecture. FDCNN is shown to be at least 54.21% faster and 70.74% more memory efficient compared to an equivalent CNN architecture. The modified VGG16 architecture with Full FDC layer is reported to achieve a shorter training time and a higher accuracy at 95.63% compared to the original VGG16 architecture for DR classification.


title: Distribution Dependent Reflecting Stochastic Differential Equations

abstract: To characterize the Neumann problem for nonlinear Fokker-Planck equations, we investigate distribution dependent reflecting SDEs (DDRSDEs) in a domain. We first prove the well-posedness and establish functional inequalities for reflecting SDEs with singular drifts, then extend these results to DDRSDEs with singular or monotone coefficients, for which a general criterion deducing the well-posedness of DDRSDEs from that of reflecting SDEs is established.


title: Planetary UAV localization based on Multi-modal Registration with Pre-existing Digital Terrain Model

abstract: The autonomous real-time optical navigation of planetary UAV is of the key technologies to ensure the success of the exploration. In such a GPS denied environment, vision-based localization is an optimal approach. In this paper, we proposed a multi-modal registration based SLAM algorithm, which estimates the location of a planet UAV using a nadir view camera on the UAV compared with pre-existing digital terrain model. To overcome the scale and appearance difference between on-board UAV images and pre-installed digital terrain model, a theoretical model is proposed to prove that topographic features of UAV image and DEM can be correlated in frequency domain via cross power spectrum. To provide the six-DOF of the UAV, we also developed an optimization approach which fuses the geo-referencing result into a SLAM system via LBA (Local Bundle Adjustment) to achieve robust and accurate vision-based navigation even in featureless planetary areas. To test the robustness and effectiveness of the proposed localization algorithm, a new cross-source drone-based localization dataset for planetary exploration is proposed. The proposed dataset includes 40200 synthetic drone images taken from nine planetary scenes with related DEM query images. Comparison experiments carried out demonstrate that over the flight distance of 33.8km, the proposed method achieved average localization error of 0.45 meters, compared to 1.31 meters by ORB-SLAM, with the processing speed of 12hz which will ensure a real-time performance. We will make our datasets available to encourage further work on this promising topic.


title: Machine Learning-based Orchestration of Containers: A Taxonomy and Future Directions

abstract: Containerization is a lightweight application virtualization technology, providing high environmental consistency, operating system distribution portability, and resource isolation. Existing mainstream cloud service providers have prevalently adopted container technologies in their distributed system infrastructures for automated application management. To handle the automation of deployment, maintenance, autoscaling, and networking of containerized applications, container orchestration is proposed as an essential research problem. However, the highly dynamic and diverse feature of cloud workloads and environments considerably raises the complexity of orchestration mechanisms. Machine learning algorithms are accordingly employed by container orchestration systems for behavior modelling and prediction of multi-dimensional performance metrics. Such insights could further improve the quality of resource provisioning decisions in response to the changing workloads under complex environments. In this paper, we present a comprehensive literature review of existing machine learning-based container orchestration approaches. Detailed taxonomies are proposed to classify the current researches by their common features. Moreover, the evolution of machine learning-based container orchestration technologies from the year 2016 to 2021 has been designed based on objectives and metrics. A comparative analysis of the reviewed techniques is conducted according to the proposed taxonomies, with emphasis on their key characteristics. Finally, various open research challenges and potential future directions are highlighted.


title: Coronal Properties of Low-mass Population III Stars and the Radiative Feedback in the Early Universe

abstract: We systematically investigated the heating of coronal loops on metal-free stars with various stellar masses and magnetic fields by magnetohydrodynamic simulations. It is found that the coronal property is dependent on the coronal magnetic field strength $B_{\rm c}$ because it affects the difference of the nonlinearity of the Alfv\'{e}nic waves. Weaker $B_{\rm c}$ leads to cooler and less dense coronae because most of the input waves dissipate in the lower atmosphere on account of the larger nonlinearity. Accordingly EUV and X-ray luminosities also correlate with $B_{\rm c}$, while they are emitted in a wide range of the field strength. Finally we extend our results to evaluating the contribution from low-mass Population III coronae to the cosmic reionization. Within the limited range of our parameters on magnetic fields and loop lengths, the EUV and X-ray radiations give a weak impact on the ionization and heating of the gas at high redshifts. However, there still remains a possibility of the contribution to the reionization from energetic flares involving long magnetic loops.


title: Discovering novel drug-supplement interactions using a dietary supplements knowledge graph generated from the biomedical literature

abstract: OBJECTIVE: Leverage existing biomedical NLP tools and DS domain terminology to produce a novel and comprehensive knowledge graph containing dietary supplement (DS) information for discovering interactions between DS and drugs, or Drug-Supplement Interactions (DSI). MATERIALS AND METHODS: We created SemRepDS (an extension of SemRep), capable of extracting semantic relations from abstracts by leveraging a DS-specific terminology (iDISK) containing 28,884 DS terms not found in the UMLS. PubMed abstracts were processed using SemRepDS to generate semantic relations, which were then filtered using a PubMedBERT-based model to remove incorrect relations before generating our knowledge graph (SuppKG). Two pathways are used to identify potential DS-Drug interactions which are then evaluated by medical professionals for mechanistic plausibility. RESULTS: Comparison analysis found that SemRepDS returned 206.9% more DS relations and 158.5% more DS entities than SemRep. The fine-tuned BERT model obtained an F1 score of 0.8605 and removed 43.86% of the relations, improving the precision of the relations by 26.4% compared to pre-filtering. SuppKG consists of 2,928 DS-specific nodes. Manual review of findings identified 44 (88%) proposed DS-Gene-Drug and 32 (64%) proposed DS-Gene1-Function-Gene2-Drug pathways to be mechanistically plausible. DISCUSSION: The additional relations extracted using SemRepDS generated SuppKG that was used to find plausible DSI not found in the current literature. By the nature of the SuppKG, these interactions are unlikely to have been found using SemRep without the expanded DS terminology. CONCLUSION: We successfully extend SemRep to include DS information and produce SuppKG which can be used to find potential DS-Drug interactions.


title: Relativistic kinetic approach to light nuclei production in high-energy nuclear collisions

abstract: Understanding the production mechanism of light (anti-)nuclei in high-energy nuclear collisions and cosmic rays has been a long-standing problem in nuclear physics. In the present study, we develop a stochastic method to solve the relativistic kinetic equations for light nuclei production from many-body reactions with the inclusion of their finite sizes. The present approach gives an excellent description of the deuteron and helium-3 data from central Au+Au (Pb+Pb) collisions at $\sqrt{s_{\rm NN}}$ $=$ $200~\rm GeV$ ($2.76~\rm TeV$). It can also naturally explain their suppressed production in $pp$ collisions at 7 TeV as a result of their finite sizes.


title: A Simultaneous Denoising and Dereverberation Framework with Target Decoupling

abstract: Background noise and room reverberation are regarded as two major factors to degrade the subjective speech quality. In this paper, we propose an integrated framework to address simultaneous denoising and dereverberation under complicated scenario environments. It adopts a chain optimization strategy and designs four sub-stages accordingly. In the first two stages, we decouple the multi-task learning w.r.t. complex spectrum into magnitude and phase, and only implement noise and reverberation removal in the magnitude domain. Based on the estimated priors above, we further polish the spectrum in the third stage, where both magnitude and phase information are explicitly repaired with the residual learning. Due to the data mismatch and nonlinear effect of DNNs, the residual noise often exists in the DNN-processed spectrum. To resolve the problem, we adopt a light-weight algorithm as the post-processing module to capture and suppress the residual noise in the non-active regions. In the Interspeech 2021 Deep Noise Suppression (DNS) Challenge, our submitted system ranked top-1 for the real-time track in terms of Mean Opinion Score (MOS) with ITU-T P.835 framework


title: An Automated Knowledge Mining and Document Classification System with Multi-model Transfer Learning

abstract: Service manual documents are crucial to the engineering company as they provide guidelines and knowledge to service engineers. However, it has become inconvenient and inefficient for service engineers to retrieve specific knowledge from documents due to the complexity of resources. In this research, we propose an automated knowledge mining and document classification system with novel multi-model transfer learning approaches. Particularly, the classification performance of the system has been improved with three effective techniques: fine-tuning, pruning, and multi-model method. The fine-tuning technique optimizes a pre-trained BERT model by adding a feed-forward neural network layer and the pruning technique is used to retrain the BERT model with new data. The multi-model method initializes and trains multiple BERT models to overcome the randomness of data ordering during the fine-tuning process. In the first iteration of the training process, multiple BERT models are being trained simultaneously. The best model is then selected for the next phase of the training process with another two iterations and the training processes for other BERT models will be terminated. The performance of the proposed system has been evaluated by comparing with two robust baseline methods, BERT and BERT-CNN. Experimental results on a widely used Corpus of Linguistic Acceptability (CoLA) dataset have shown that the proposed techniques perform better than these baseline methods in terms of accuracy and MCC score.


title: Artificial neural network molecular mechanics of iron grain boundaries

abstract: This study reports grain boundary (GB) energy calculations for 46 symmetric-tilt GBs in alpha-iron using molecular mechanics based on an artificial neural network (ANN) potential and compares the results with calculations based on the density functional theory (DFT), the embedded atom method (EAM), and the modified EAM (MEAM). The results by the ANN potential are in excellent agreement with those of the DFT (5% on average), while the EAM and MEAM significantly differ from the DFT results (about 27% on average). In a uniaxial tensile calculation of Sigma 3 (1-12) GB, the ANN potential reproduced the brittle fracture tendency of the GB observed in the DFT while the EAM and MEAM showed mistakenly showed ductile behaviors. These results demonstrate the effectiveness of the ANN potential in grain boundary calculations of iron as a fast and accurate simulation highly in demand in the modern industrial world.