title: Theory of colloidal stabilization by unattached polymers

abstract: Stable colloidal dispersions with evenly distributed particles are important for many technological applications. Due to Brownian motion colloidal particles have constant collisions with each other which often lead to their aggregation driven by the long range van der Waals attraction. As a result the colloidal systems often tend to precipitate. A number of methods have been devised to minimize the effect of long-range van der Waals attraction between colloidal particles or to override the influence of the attraction in order to provide the colloidal stability.In the PhD thesis we investigated the colloidal stabilization in solutions of free polymers which is commonly referred to as depletion stabilization. Previous theoretical studies of free-polymer induced (FPI) stabilization were based on oversimplified models involving uncontrolled approximations. Even the most basic features of the depletion stabilization phenomenon were unknown. It was unclear how the PI repulsion depends on the solution parameters, polymer structure and monomer/surface interactions.The free polymer chains were modeled as random walks in a self-consistent molecular field that satisfied to diffusion-like integro-differential equation. As the molecular field we used the chemical potential that for semi-dilute polymer solution can be represented as a virial expansion where we took into account only second and third virial coefficients of the polymer solution. Varying the parameters like polymer stiffness, polymer length, polymer concentration and solvent regime (like theta solvent) whether it is for purely repulsive colloidal surface, adsorbed surface or surface with grafted polymer layer we were able to enhance the repulsive barrier due to the free polymers between the particles and therefore found conditions for kinetic stabilization of the system.


title: Lagrangian Mechanics of Active Systems

abstract: We present a multi-scale modeling and simulation framework for low-Reynolds number hydrodynamics of shape-changing immersed objects, e.g., biological microswimmers and active surfaces. The key idea is to consider principal shape changes as generalized coordinates, and define conjugate generalized hydrodynamic friction forces. Conveniently, the corresponding generalized friction coefficients can be pre-computed and subsequently re-used to solve dynamic equations of motion fast. This framework extends Lagrangian mechanics of dissipative systems to active surfaces and active microswimmers, whose shape dynamics is driven by internal forces. As an application case, we predict in-phase and anti-phase synchronization in pairs of cilia for an experimentally measured cilia beat pattern.


title: Persistent fluid flows defined by active matter boundaries

abstract: Biological systems achieve precise control over ambient fluids through the self-organization of active protein structures including flagella, cilia, and cytoskeletal networks. In active structures individual proteins consume chemical energy to generate force and motion at molecular length scales. Self-organization of protein components enables the control and modulation of fluid flow fields on micron scales. The physical principles underlying the organization and control of active-matter driven fluid flows are poorly understood. Here, we apply an optically-controlled active-matter system composed of microtubule filaments and light-switchable kinesin motor proteins to analyze the emergence of persistent flow fields in a model active matter system. Using light, we form contractile microtubule networks of varying shape. We analyze the fluid flow fields generated by a wide range of microtubule network geometries and explain the resulting flow fields within a unified theoretical framework. We specifically demonstrate that the geometry of microtubule flux at the boundary of contracting microtubule networks predicts the steady-state fluid flow fields across polygonal network geometries through finite-element simulations. Our work provides a foundation for programming microscopic fluid-flows with controllable active matter and could enable the engineering of versatile and dynamic microfluidic devices.


title: Measures of Model Risk in Continuous-time Finance Models

abstract: Measuring model risk is required by regulators on financial and insurance markets. We separate model risk into parameter estimation risk and model specification risk, and we propose expected shortfall type model risk measures applied to Levy jump models and affine jump-diffusion models. We investigate the impact of parameter estimation risk and model specification risk on the models' ability to capture the joint dynamics of stock and option prices. We estimate the parameters using Markov chain Monte Carlo techniques, under the risk-neutral probability measure and the real-world probability measure jointly. We find strong evidence supporting modeling of price jumps.


title: Distributed Representations of Entities in Open-World Knowledge Graphs

abstract: Graph neural network (GNN)-based methods have demonstrated remarkable performance in various knowledge graph (KG) tasks. However, most existing approaches rely on observing all entities during training, posing a challenge in real-world knowledge graphs where new entities emerge frequently. To address this limitation, we introduce Decentralized Attention Network (DAN). DAN leverages neighbor context as the query vector to score the neighbors of an entity, thereby distributing the entity semantics only among its neighbor embeddings. To effectively train a DAN, we introduce self-distillation, a technique that guides the network in generating desired representations. Theoretical analysis validates the effectiveness of our approach. We implement an end-to-end framework and conduct extensive experiments to evaluate our method, showcasing competitive performance on conventional entity alignment and entity prediction tasks. Furthermore, our method significantly outperforms existing methods in open-world settings.


title: Pinball-OCSVM for early-stage COVID-19 diagnosis with limited posteroanterior chest X-ray images

abstract: The infection of respiratory coronavirus disease 2019 (COVID-19) starts with the upper respiratory tract and as the virus grows, the infection can progress to lungs and develop pneumonia. The conventional way of COVID-19 diagnosis is reverse transcription polymerase chain reaction (RT-PCR), which is less sensitive during early stages; especially if the patient is asymptomatic, which may further cause more severe pneumonia. In this context, several deep learning models have been proposed to identify pulmonary infections using publicly available chest X-ray (CXR) image datasets for early diagnosis, better treatment and quick cure. In these datasets, presence of less number of COVID-19 positive samples compared to other classes (normal, pneumonia and Tuberculosis) raises the challenge for unbiased learning of deep learning models. All deep learning models opted class balancing techniques to solve this issue; which however should be avoided in any medical diagnosis process. Moreover, the deep learning models are also data hungry and need massive computation resources. Therefore for quicker diagnosis, this research proposes a novel pinball loss function based one-class support vector machine (PB-OCSVM), that can work in presence of limited COVID-19 positive CXR samples with objectives to maximize the learning efficiency and to minimize the false predictions. The performance of the proposed model is compared with conventional OCSVM and existing deep learning models, and the experimental results prove that the proposed model outperformed over state-of-the-art methods. To validate the robustness of the proposed model, experiments are also performed with noisy CXR images and UCI benchmark datasets.


title: Four-Fluid Axisymmetric Plasma Equilibrium Model Including Relativistic Electrons and Computational Method and Results

abstract: A non-relativistic multi-fluid plasma axisymmetric equilibrium model was developed recently to account for the presence of an energetic electron fluid in addition to thermal electron and ion fluids. The equilibrium formulation of a multi-fluid plasma with relativistic energetic electrons is developed and reported in this paper. Relativistic effects in a fluid model approximation can appear in two ways: due to a large macroscopic fluid velocity comparable to the speed of light and large particle's microscopic random motion which becomes significant if the temperature becomes comparable to or larger than the electron rest mass-energy. It is found that the axial component of relativistic generalized angular momentum can be used to describe relativistic axisymmetric equilibrium. The formulation is applied to a four-fluid plasma composed of a relativistic energetic electron fluid, a thermal electron fluid, and fluids of two thermal ion species (e.g. proton and boron ions). The four-fluid density expression which is consistent with the electrostatic potential is obtained and applied in the computation. An example equilibrium approximating a four-fluid plasma recently observed in a solenoid-free ECRH sustained spherical torus plasma is calculated and presented. A second equilibrium that extends the energetic electron temperature of the first example to 679keV is calculated revealing significant relativistic effects.


title: Enrichment paradox and applications

abstract: We introduced a more general predator-prey model to analyze the paradox of enrichment. We hope the results obtained for the model can guide us on identifying real field paradox of enrichment.


title: Sadovskii vortex in a wedge and the associated Riemann-Hilbert problem on a torus

abstract: Reconstruction of conformal mappings from canonical slit domains onto multiply-connected physical domains with a free boundary is of interest in many different models arising in fluid mechanics. In the present paper, an exact formula for the conformal map from the exterior of two slits onto the doubly connected flow domain is obtained when a fluid flows in a wedge about a Sadovskii vortex. The map is employed to determine the potential flow outside the vortex and the vortex domain boundary provided the circulation $\Gamma$ around the vortex and constant speed $U$ on the vortex boundary are prescribed, and there are no stagnation points on the walls. The map is expressed in terms of a rational function on an elliptic surface topologically equivalent to a torus and the solution to a symmetric Riemann-Hilbert problem on a finite and a semi-infinite segments on the same genus-1 Riemann surface. Owing to its special features, the Riemann-Hilbert problem requires a novel analogue of the Cauchy kernel on an elliptic surface. Such a kernel is proposed and employed to derive a closed-form solution to the Riemann-Hilbert problem and the associated Jacobi inversion problem. The final formula for the conformal map possesses a free geometric parameter and two model parameters, the wedge angle $\alpha$ and $\Gamma/U$. It is shown that when $\alpha<\pi$ the solution exists and the vortex has two cusps, while the solution does not exist when the wedge angle exceeds $\pi$.


title: Revenue and Energy Efficiency-Driven Delay Constrained Computing Task Offloading and Resource Allocation in a Vehicular Edge Computing Network: A Deep Reinforcement Learning Approach

abstract: For in-vehicle application,task type and vehicle state information, i.e., vehicle speed, bear a significant impact on the task delay requirement. However, the joint impact of task type and vehicle speed on the task delay constraint has not been studied, and this lack of study may cause a mismatch between the requirement of the task delay and allocated computation and wireless resources. In this paper, we propose a joint task type and vehicle speed-aware task offloading and resource allocation strategy to decrease the vehicl's energy cost for executing tasks and increase the revenue of the vehicle for processing tasks within the delay constraint. First, we establish the joint task type and vehicle speed-aware delay constraint model. Then, the delay, energy cost and revenue for task execution in the vehicular edge computing (VEC) server, local terminal and terminals of other vehicles are calculated. Based on the energy cost and revenue from task execution,the utility function of the vehicle is acquired. Next, we formulate a joint optimization of task offloading and resource allocation to maximize the utility level of the vehicles subject to the constraints of task delay, computation resources and wireless resources. To obtain a near-optimal solution of the formulated problem, a joint offloading and resource allocation based on the multi-agent deep deterministic policy gradient (JORA-MADDPG) algorithm is proposed to maximize the utility level of vehicles. Simulation results show that our algorithm can achieve superior performance in task completion delay, vehicles' energy cost and processing revenue.


title: Joint Inference of Multiple Graphs from Matrix Polynomials

abstract: Inferring graph structure from observations on the nodes is an important and popular network science task. Departing from the more common inference of a single graph and motivated by social and biological networks, we study the problem of jointly inferring multiple graphs from the observation of signals at their nodes (graph signals), which are assumed to be stationary in the sought graphs. From a mathematical point of view, graph stationarity implies that the mapping between the covariance of the signals and the sparse matrix representing the underlying graph is given by a matrix polynomial. A prominent example is that of Markov random fields, where the inverse of the covariance yields the sparse matrix of interest. From a modeling perspective, stationary graph signals can be used to model linear network processes evolving on a set of (not necessarily known) networks. Leveraging that matrix polynomials commute, a convex optimization method along with sufficient conditions that guarantee the recovery of the true graphs are provided when perfect covariance information is available. Particularly important from an empirical viewpoint, we provide high-probability bounds on the recovery error as a function of the number of signals observed and other key problem parameters. Numerical experiments using synthetic and real-world data demonstrate the effectiveness of the proposed method with perfect covariance information as well as its robustness in the noisy regime.


title: Joint Optimization for Coordinated Charging Control of Commercial Electric Vehicles Under Distributed Hydrogen Energy Supply

abstract: The transition to the zero-carbon power system is underway accelerating recently. Hydrogen energy and electric vehicles (EVs) are promising solutions on the supply and demand sides. This paper presents a novel architecture that includes hydrogen production stations (HPSs), fast charging stations (FCSs), and commercial EVs. The proposed architecture jointly optimizes the distributed hydrogen energy dispatch and the EV charging location selection, and is formulated by a time-varying bi-level bipartite graph (T-BBG) model for real-time operation. We develop a bi-level iteration optimization method combining linear programming (LP) and Kuhn-Munkres (KM) algorithm to solve the joint problem whose optimality is proved theoretically. The effectiveness of the proposed architecture on reducing the operating cost is verified via case studies in Shanghai. The proposed method outperforms other strategies and improves the performance by at least 13% which shows the potential economic benefits of the joint architecture. The convergence and impact of the pile number, battery capacity, EV speed and penalty factor are assessed.


title: An Application of H\"older's Inequality to Economics

abstract: We use H\"older's inequality to get simple derivations of certain economic formulas involving CES, Armington, or $n$-stage Armington functions.


title: Melody Classifier with Stacked-LSTM

abstract: Attempts to use generative models for music generation have been common in recent years, and some of them have achieved good results. Pieces generated by some of these models are almost indistinguishable from those being composed by human composers. However, the research on the evaluation system for machine-generated music is still at a relatively early stage, and there is no uniform standard for such tasks. This paper proposes a stacked-LSTM binary classifier based on a language model, which can be used to distinguish the human composer's work from the machine-generated melody by learning the MIDI file's pitch, position, and duration.


title: Risk-Aware Decision Making in Service Robots to Minimize Risk of Patient Falls in Hospitals

abstract: Planning under uncertainty is a crucial capability for autonomous systems to operate reliably in uncertain and dynamic environments. The concern of safety becomes even more critical in healthcare settings where robots interact with human patients. In this paper, we propose a novel risk-aware planning framework to minimize the risk of falls by providing a patient with an assistive device. Our approach combines learning-based prediction with model-based control to plan for the fall prevention task. This provides advantages compared to end-to-end learning methods in which the robot's performance is limited to specific scenarios, or purely model-based approaches that use relatively simple function approximators and are prone to high modeling errors. We compare various risk metrics and the results from simulated scenarios show that using the proposed cost function, the robot can plan interventions to avoid high fall score events.


title: Testing the Quantitative Spacetime Hypothesis using Artificial Narrative Comprehension (II) : Establishing the Geometry of Invariant Concepts, Themes, and Namespaces

abstract: Given a pool of observations selected from a sensor stream, input data can be robustly represented, via a multiscale process, in terms of invariant concepts, and themes. Applying this to episodic natural language data, one may obtain a graph geometry associated with the decomposition, which is a direct encoding of spacetime relationships for the events. This study contributes to an ongoing application of the Semantic Spacetime Hypothesis, and demonstrates the unsupervised analysis of narrative texts using inexpensive computational methods without knowledge of linguistics. Data streams are parsed and fractionated into small constituents, by multiscale interferometry, in the manner of bioinformatic analysis. Fragments may then be recombined to construct original sensory episodes---or form new narratives by a chemistry of association and pattern reconstruction, based only on the four fundamental spacetime relationships. There is a straightforward correspondence between bioinformatic processes and this cognitive representation of natural language. Features identifiable as `concepts' and `narrative themes' span three main scales (micro, meso, and macro). Fragments of the input act as symbols in a hierarchy of alphabets that define new effective languages at each scale.


title: Testing the Quantitative Spacetime Hypothesis using Artificial Narrative Comprehension (I) : Bootstrapping Meaning from Episodic Narrative viewed as a Feature Landscape

abstract: The problem of extracting important and meaningful parts of a sensory data stream, without prior training, is studied for symbolic sequences, by using textual narrative as a test case. This is part of a larger study concerning the extraction of concepts from spacetime processes, and their knowledge representations within hybrid symbolic-learning `Artificial Intelligence'. Most approaches to text analysis make extensive use of the evolved human sense of language and semantics. In this work, streams are parsed without knowledge of semantics, using only measurable patterns (size and time) within the changing stream of symbols -- as an event `landscape'. This is a form of interferometry. Using lightweight procedures that can be run in just a few seconds on a single CPU, this work studies the validity of the Semantic Spacetime Hypothesis, for the extraction of concepts as process invariants. This `semantic preprocessor' may then act as a front-end for more sophisticated long-term graph-based learning techniques. The results suggest that what we consider important and interesting about sensory experience is not solely based on higher reasoning, but on simple spacetime process cues, and this may be how cognitive processing is bootstrapped in the beginning.


title: The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers

abstract: We propose a new framework for reasoning about generalization in deep learning. The core idea is to couple the Real World, where optimizers take stochastic gradient steps on the empirical loss, to an Ideal World, where optimizers take steps on the population loss. This leads to an alternate decomposition of test error into: (1) the Ideal World test error plus (2) the gap between the two worlds. If the gap (2) is universally small, this reduces the problem of generalization in offline learning to the problem of optimization in online learning. We then give empirical evidence that this gap between worlds can be small in realistic deep learning settings, in particular supervised image classification. For example, CNNs generalize better than MLPs on image distributions in the Real World, but this is "because" they optimize faster on the population loss in the Ideal World. This suggests our framework is a useful tool for understanding generalization in deep learning, and lays a foundation for future research in the area.

categories: cs.LG cs.CV cs.NE math.ST stat.ML stat.TH


title: Semantic Editing On Segmentation Map Via Multi-Expansion Loss

abstract: Semantic editing on segmentation map has been proposed as an intermediate interface for image generation, because it provides flexible and strong assistance in various image generation tasks. This paper aims to improve quality of edited segmentation map conditioned on semantic inputs. Even though recent studies apply global and local adversarial losses extensively to generate images for higher image quality, we find that they suffer from the misalignment of the boundary area in the mask area. To address this, we propose MExGAN for semantic editing on segmentation map, which uses a novel Multi-Expansion (MEx) loss implemented by adversarial losses on MEx areas. Each MEx area has the mask area of the generation as the majority and the boundary of original context as the minority. To boost convenience and stability of MEx loss, we further propose an Approximated MEx (A-MEx) loss. Besides, in contrast to previous model that builds training data for semantic editing on segmentation map with part of the whole image, which leads to model performance degradation, MExGAN applies the whole image to build the training data. Extensive experiments on semantic editing on segmentation map and natural image inpainting show competitive results on four datasets.


title: One-parameter groups of orthogonality preservers on JB$^*$-algebras

abstract: In a first objective we improve our understanding about surjective and bijective bounded linear operators preserving orthogonality from a JB$^*$-algebra $\mathcal{A}$ into a JB$^*$-triple $E$. Among many other conclusions, it is shown that a bounded linear bijection $T: \mathcal{A}\to E$ is orthogonality preserving if, and only if, it is biorthogonality preserving if, and only if, it preserves zero-triple-products in both directions (i.e., $\{a,b,c\}=0 \Leftrightarrow \{T(a),T(b),T(c)\}=0$). In the second main result we establish a complete characterization of all one-parameter groups of orthogonality preserving operators on a JB$^*$-algebra.


title: Offer Personalization using Temporal Convolution Network and Optimization

abstract: Lately, personalized marketing has become important for retail/e-retail firms due to significant rise in online shopping and market competition. Increase in online shopping and high market competition has led to an increase in promotional expenditure for online retailers, and hence, rolling out optimal offers has become imperative to maintain balance between number of transactions and profit. In this paper, we propose our approach to solve the offer optimization problem at the intersection of consumer, item and time in retail setting. To optimize offer, we first build a generalized non-linear model using Temporal Convolutional Network to predict the item purchase probability at consumer level for the given time period. Secondly, we establish the functional relationship between historical offer values and purchase probabilities obtained from the model, which is then used to estimate offer-elasticity of purchase probability at consumer item granularity. Finally, using estimated elasticities, we optimize offer values using constraint based optimization technique. This paper describes our detailed methodology and presents the results of modelling and optimization across categories.


title: [Rb/Zr] ratio in Ba stars as diagnostics of the companion AGB stellar mass

abstract: Understanding the nucleosynthesis and evolution of Asymptotic Giant Branch (AGB) stars is of primary importance as they are the main producers of some of the key elements in the Universe. They are the predominant sites for the slow neutron-capture nucleosynthesis. The exact physical conditions and nucleosynthetic processes occuring at the interior of AGB stars are not clearly understood, that hinders a better understanding of the contribution of these stars to the Galactic chemical enrichment. The extrinsic stars, which are known to have received products of AGB phase of evolution via binary mass transfer mechanisms, form vital tools to trace the AGB nucleosynthesis. The [Rb/Zr] ratio is an important diagnostic to understand the average neutron density at the s-process site and provide important clues to the mass of the companion AGB stars. In this work we have presented the estimate of [Rb/Zr] ratios, based on the high resolution spectroscopic analysis for a sample of Ba stars, and discussed how it can be used to understand the characteristics of the AGB star. Results from an analysis based on parametric model to confirm the mass of the companion AGB star are also presented.


title: Power of Knockoff: The Impact of Ranking Algorithm, Augmented Design, and Symmetric Statistic

abstract: The knockoff filter is a recent false discovery rate (FDR) control method for high-dimensional linear models. We point out that knockoff has three key components: ranking algorithm, augmented design, and symmetric statistic, and each component admits multiple choices. By considering various combinations of the three components, we obtain a collection of variants of knockoff. All these variants guarantee finite-sample FDR control, and our goal is to compare their power. We assume a Rare and Weak signal model on regression coefficients and compare the power of different variants of knockoff by deriving explicit formulas of false positive rate and false negative rate. Our results provide new insights on how to improve power when controlling FDR at a targeted level. We also compare the power of knockoff with its propotype - a method that uses the same ranking algorithm but has access to an ideal threshold. The comparison reveals the additional price one pays by finding a data-driven threshold to control FDR.


title: A quartic diophantine equation inspired by Brahmagupta's identity

abstract: In this paper we obtain several parametric solutions of the quartic diophantine equation $(x_1^4+x_2^4)(y_1^4+y_2^4)=z_1^4+z_2^4$. We also show how infinitely many parametric solutions of this equation may be obtained by using elliptic curves.


title: Generalized Co-sparse Factor Regression

abstract: Multivariate regression techniques are commonly applied to explore the associations between large numbers of outcomes and predictors. In real-world applications, the outcomes are often of mixed types, including continuous measurements, binary indicators, and counts, and the observations may also be incomplete. Building upon the recent advances in mixed-outcome modeling and sparse matrix factorization, generalized co-sparse factor regression (GOFAR) is proposed, which utilizes the flexible vector generalized linear model framework and encodes the outcome dependency through a sparse singular value decomposition (SSVD) of the integrated natural parameter matrix. To avoid the estimation of the notoriously difficult joint SSVD, GOFAR proposes both sequential and parallel unit-rank estimation procedures. By combining the ideas of alternating convex search and majorization-minimization, an efficient algorithm with guaranteed convergence is developed to solve the sparse unit-rank problem and implemented in the R package gofar. Extensive simulation studies and two real-world applications demonstrate the effectiveness of the proposed approach.


title: Model-based Decentralized Bayesian Algorithm for Distributed Compressed Sensing

abstract: In this paper, a novel model-based distributed compressive sensing (DCS) algorithm is proposed. DCS exploits the inter-signal correlations and has the capability to jointly recover multiple sparse signals. Proposed approach is a Bayesian decentralized algorithm which uses the type 1 joint sparsity model (JSM-1) and exploits the intra-signal correlations, as well as the inter-signal correlations. Compared to the conventional DCS algorithm, which only exploit the joint sparsity of the signals, the proposed approach takes the intra- and inter-scale dependencies among the wavelet coefficients into account to enable the utilization of the individual signal structure. Furthermore, the Bessel K-form (BKF) is used as the prior distribution which has a sharper peak at zero and heavier tails than the Gaussian distribution. The variational Bayesian (VB) inference is employed to perform the posterior distributions and acquire a closed-form solution for model parameters. Simulation results demonstrate that the proposed algorithm have good recovery performance in comparison with state-of the-art techniques.


title: Towards Natural Bilingual and Code-Switched Speech Synthesis Based on Mix of Monolingual Recordings and Cross-Lingual Voice Conversion

abstract: Recent state-of-the-art neural text-to-speech (TTS) synthesis models have dramatically improved intelligibility and naturalness of generated speech from text. However, building a good bilingual or code-switched TTS for a particular voice is still a challenge. The main reason is that it is not easy to obtain a bilingual corpus from a speaker who achieves native-level fluency in both languages. In this paper, we explore the use of Mandarin speech recordings from a Mandarin speaker, and English speech recordings from another English speaker to build high-quality bilingual and code-switched TTS for both speakers. A Tacotron2-based cross-lingual voice conversion system is employed to generate the Mandarin speaker's English speech and the English speaker's Mandarin speech, which show good naturalness and speaker similarity. The obtained bilingual data are then augmented with code-switched utterances synthesized using a Transformer model. With these data, three neural TTS models -- Tacotron2, Transformer and FastSpeech are applied for building bilingual and code-switched TTS. Subjective evaluation results show that all the three systems can produce (near-)native-level speech in both languages for each of the speaker.


title: Statistical Graph Signal Recovery Using Variational Bayes

abstract: This paper investigates the problem of graph signal recovery (GSR) when the topology of the graph is not known in advance. In this paper, the elements of the weighted adjacency matrix is statistically related to normal distribution and the graph signal is assumed to be Gaussian Markov Random Field (GMRF). Then, the problem of GSR is solved by a Variational Bayes (VB) algorithm in a Bayesian manner by computing the posteriors in a closed form. The posteriors of the elements of weighted adjacency matrix are proved to have a new distribution which we call it generalized compound confluent hypergeometric (GCCH) distribution. Moreover, the variance of the noise is estimated by calculating its posterior via VB. The simulation results on synthetic and real-world data shows the superiority of the proposed Bayesian algorithm over some state-of-the-art algorithms in recovering the graph signal.


title: Input-Aware Dynamic Backdoor Attack

abstract: In recent years, neural backdoor attack has been considered to be a potential security threat to deep learning systems. Such systems, while achieving the state-of-the-art performance on clean data, perform abnormally on inputs with predefined triggers. Current backdoor techniques, however, rely on uniform trigger patterns, which are easily detected and mitigated by current defense methods. In this work, we propose a novel backdoor attack technique in which the triggers vary from input to input. To achieve this goal, we implement an input-aware trigger generator driven by diversity loss. A novel cross-trigger test is applied to enforce trigger nonreusablity, making backdoor verification impossible. Experiments show that our method is efficient in various attack scenarios as well as multiple datasets. We further demonstrate that our backdoor can bypass the state of the art defense methods. An analysis with a famous neural network inspector again proves the stealthiness of the proposed attack. Our code is publicly available at https://github.com/VinAIResearch/input-aware-backdoor-attack-release.


title: A non-intrusive data-driven ROM framework for hemodynamics problems

abstract: Reduced order modeling (ROM) techniques are numerical methods that approximate the solution of parametric partial differential equation (PDE) by properly combining the high-fidelity solutions of the problem obtained for several configurations, i.e. for several properly chosen values of the physical/geometrical parameters characterizing the problem. In this contribution, we propose an efficient non-intrusive data-driven framework involving ROM techniques in computational fluid dynamics (CFD) for hemodynamics applications. By starting from a database of high-fidelity solutions related to a certain values of the parameters, we apply the proper orthogonal decomposition with interpolation (PODI) and then reconstruct the variables of interest for new values of the parameters, i.e. different values from the ones included in the database. Furthermore, we present a preliminary web application through which one can run the ROM with a very user-friendly approach, without the need of having expertise in the numerical analysis and scientific computing field. The case study we have chosen to test the efficiency of our algorithm is represented by the aortic blood flow pattern in presence of a Left Ventricular Assist Device (LVAD) when varying the pump flow rate.


title: Monitoring Trust in Human-Machine Interactions for Public Sector Applications

abstract: The work reported here addresses the capacity of psychophysiological sensors and measures using Electroencephalogram (EEG) and Galvanic Skin Response (GSR) to detect levels of trust for humans using AI-supported Human-Machine Interaction (HMI). Improvements to the analysis of EEG and GSR data may create models that perform as well, or better than, traditional tools. A challenge to analyzing the EEG and GSR data is the large amount of training data required due to a large number of variables in the measurements. Researchers have routinely used standard machine-learning classifiers like artificial neural networks (ANN), support vector machines (SVM), and K-nearest neighbors (KNN). Traditionally, these have provided few insights into which features of the EEG and GSR data facilitate the more and least accurate predictions - thus making it harder to improve the HMI and human-machine trust relationship. A key ingredient to applying trust-sensor research results to practical situations and monitoring trust in work environments is the understanding of which key features are contributing to trust and then reducing the amount of data needed for practical applications. We used the Local Interpretable Model-agnostic Explanations (LIME) model as a process to reduce the volume of data required to monitor and enhance trust in HMI systems - a technology that could be valuable for governmental and public sector applications. Explainable AI can make HMI systems transparent and promote trust. From customer service in government agencies and community-level non-profit public service organizations to national military and cybersecurity institutions, many public sector organizations are increasingly concerned to have effective and ethical HMI with services that are trustworthy, unbiased, and free of unintended negative consequences.


title: Autonomous Control of a Particle Accelerator using Deep Reinforcement Learning

abstract: We describe an approach to learning optimal control policies for a large, linear particle accelerator using deep reinforcement learning coupled with a high-fidelity physics engine. The framework consists of an AI controller that uses deep neural nets for state and action-space representation and learns optimal policies using reward signals that are provided by the physics simulator. For this work, we only focus on controlling a small section of the entire accelerator. Nevertheless, initial results indicate that we can achieve better-than-human level performance in terms of particle beam current and distribution. The ultimate goal of this line of work is to substantially reduce the tuning time for such facilities by orders of magnitude, and achieve near-autonomous control.


title: Improved Approximation Algorithms for Stochastic-Matching Problems

abstract: We consider the Stochastic Matching problem, which is motivated by applications in kidney exchange and online dating. In this problem, we are given an undirected graph. Each edge is assigned a known, independent probability of existence and a positive weight (or profit). We must probe an edge to discover whether or not it exists. Each node is assigned a positive integer called a timeout (or a patience). On this random graph we are executing a process, which probes the edges one-by-one and gradually constructs a matching. The process is constrained in two ways. First, if a probed edge exists, it must be added irrevocably to the matching (the query-commit model). Second, the timeout of a node $v$ upper-bounds the number of edges incident to $v$ that can be probed. The goal is to maximize the expected weight of the constructed matching. For this problem, Bansal et al. (Algorithmica 2012) provided a $0.33$-approximation algorithm for bipartite graphs and a $0.25$-approximation for general graphs. We improve the approximation factors to $0.39$ and $0.269$, respectively. The main technical ingredient in our result is a novel way of probing edges according to a not-uniformly-random permutation. Patching this method with an algorithm that works best for large-probability edges (plus additional ideas) leads to our improved approximation factors.


title: Equilibrium States for Open Zooming Systems

abstract: In this work, we construct Markov structures for zooming systems adapted to holes of a special type. Our construction is based on backward contractions provided by zooming times. These Markov structures may be used to code the open zooming systems. In the context of open zooming systems, possibly with the presence of a critical/singular set, we prove the existence of finitely many ergodic zooming equilibrium states for zooming potentials that are also H\"older continuous. Among the examples of zooming potentials are the so-called hyperbolic potentials and also what we call pseudo-geometric potentials $\phi_{t} = -t \log J_{\mu}f $, where $J_{\mu}f$ is a Jacobian of the reference zooming measure. Moreover, for the same class of potentials, we show the existence of what we call pseudo-conformal measures. We prove uniqueness of equilibrium state when the system is also strongly topologically transitive and backward separated. The technique consists in using an inducing scheme in a finite Markov structure with infinitely many symbols to code the dynamics to obtain an equilibrium state for the associated symbolic dynamics and then projecting it to obtain an equilibrium state for the original map. To obtain a pseudo-conformal measure, we "spread" the conformal one which exists for the symbolic dynamics. The uniqueness is obtained by showing that the equilibrium states are liftable to the same inducing scheme. Finally, we show that the class of hyperbolic potentials is equivalent to the class of continuous zooming potentials. Also, we give a class of examples of hyperbolic potentials (including the null one) for zooming systems with exponential contractions. It implies the existence and uniqueness of equilibrium state (in particular, of maximal entropy measure). Among the maps considered is the important class known as Viana maps.


title: Isabelle Stone: breaking the glass ceiling with thin films and teaching

abstract: Physicist Isabelle Stone forged ahead in what was largely a men's profession in the early 1900s. She is credited as the first woman in the United States to obtain a Ph.D. in physics, one of the two women founders of the American Physical Society, and one of only two women to attend the first International Congress of Physics. Hers is the story of an unmarried, educated woman doing scientific research, living and working abroad, and providing educational stepping stones for more women to go to college. It starts with the electric resistance of thin films in Chicago and becomes a journey in teaching women up and down the east coast, across the Atlantic, and back again.


title: Taking A Closer Look at Synthesis: Fine-grained Attribute Analysis for Person Re-Identification

abstract: Person re-identification (re-ID) plays an important role in applications such as public security and video surveillance. Recently, learning from synthetic data, which benefits from the popularity of synthetic data engine, has achieved remarkable performance. However, in pursuit of high accuracy, researchers in the academic always focus on training with large-scale datasets at a high cost of time and label expenses, while neglect to explore the potential of performing efficient training from millions of synthetic data. To facilitate development in this field, we reviewed the previously developed synthetic dataset GPR and built an improved one (GPR+) with larger number of identities and distinguished attributes. Based on it, we quantitatively analyze the influence of dataset attribute on re-ID system. To our best knowledge, we are among the first attempts to explicitly dissect person re-ID from the aspect of attribute on synthetic dataset. This research helps us have a deeper understanding of the fundamental problems in person re-ID, which also provides useful insights for dataset building and future practical usage.


title: Online Decision Trees with Fairness

abstract: While artificial intelligence (AI)-based decision-making systems are increasingly popular, significant concerns on the potential discrimination during the AI decision-making process have been observed. For example, the distribution of predictions is usually biased and dependents on the sensitive attributes (e.g., gender and ethnicity). Numerous approaches have therefore been proposed to develop decision-making systems that are discrimination-conscious by-design, which are typically batch-based and require the simultaneous availability of all the training data for model learning. However, in the real-world, the data streams usually come on the fly which requires the model to process each input data once "on arrival" and without the need for storage and reprocessing. In addition, the data streams might also evolve over time, which further requires the model to be able to simultaneously adapt to non-stationary data distributions and time-evolving bias patterns, with an effective and robust trade-off between accuracy and fairness. In this paper, we propose a novel framework of online decision tree with fairness in the data stream with possible distribution drifting. Specifically, first, we propose two novel fairness splitting criteria that encode the data as well as possible, while simultaneously removing dependence on the sensitive attributes, and further adapts to non-stationary distribution with fine-grained control when needed. Second, we propose two fairness decision tree online growth algorithms that fulfills different online fair decision-making requirements. Our experiments show that our algorithms are able to deal with discrimination in massive and non-stationary streaming environments, with a better trade-off between fairness and predictive performance.


title: Attosecond pulse formation in the "water window" range by optically dressed hydrogen-like plasma-based C5+ X-ray laser

abstract: In this paper, we present the analytical theory of attosecond pulse formation via optical modulation of an active medium of the hydrogen-like C5+ plasma-based X-ray laser at 3.4 nm wavelength in the "water window" range, taking into account a variation of the population inversion caused by radiative decay of the upper lasing states. We derive an analytical solution for the X-ray field amplified by an X-ray laser with time-dependent population inversion, which is simultaneously irradiated by a strong optical laser field, and use it to find the optimal conditions for the attosecond pulse formation from a narrowband seeding X-ray field. We show that the shape of pulses can be improved at the cost of reduced pulse peak intensity (i) via external attenuation of the resonant spectral component of the amplified X-ray field or (ii) by using a resonantly absorbing medium (the active medium of the X-ray laser after the change of sign of the population inversion) for the pulse formation. The results of the analytical theory are in a good agreement with the numerical solutions of the Maxwell-Bloch equations which account for the nonlinearity, as well as the amplified spontaneous emission, of the active medium. Both analytically and numerically we show the possibility to produce a train of attosecond pulses with sub-200 as duration and the peak intensity exceeding 10^12 W/cm^2 at the carrier wavelength 3.4 nm in the "water window" range, which makes them attractive for the biological and medical applications.


title: Consistency of archetypal analysis

abstract: Archetypal analysis is an unsupervised learning method that uses a convex polytope to summarize multivariate data. For fixed $k$, the method finds a convex polytope with $k$ vertices, called archetype points, such that the polytope is contained in the convex hull of the data and the mean squared distance between the data and the polytope is minimal. In this paper, we prove a consistency result that shows if the data is independently sampled from a probability measure with bounded support, then the archetype points converge to a solution of the continuum version of the problem, of which we identify and establish several properties. We also obtain the convergence rate of the optimal objective values under appropriate assumptions on the distribution. If the data is independently sampled from a distribution with unbounded support, we also prove a consistency result for a modified method that penalizes the dispersion of the archetype points. Our analysis is supported by detailed computational experiments of the archetype points for data sampled from the uniform distribution in a disk, the normal distribution, an annular distribution, and a Gaussian mixture model.


title: Mixed-hybrid and mixed-discontinuous Galerkin methods for linear dynamical elastic-viscoelastic composite structures

abstract: We introduce and analyze a stress-based formulation for Zener's model in linear viscoelasticity. The method is aimed to tackle efficiently heterogeneous materials that admit purely elastic and viscoelastic parts in their composition. We write the mixed variational formulation of the problem in terms of a class of tensorial wave equation and obtain an energy estimate that guaranties the well-posedness of the problem through a standard Galerkin procedure. We propose and analyze mixed continuous and discontinuous Galerkin space discretizations of the problem and derive optimal error bounds for each semidiscrete solution in the corresponding energy norm.


title: Microscopic coupled-channel calculation of proton and alpha inelastic scattering to the $4^+_1$ and $4^+_2$ states of $^{24}$Mg

abstract: Background: The triaxial and hexadecapole deformations of the K=0+ and K=2+ bands of 24Mg have been investigated by the inelastic scatterings of various probes, including electrons, protons, and alpha particles, for a prolonged time. However, it has been challenging to explain the unique properties of the scatterings observed for the $4^+_1$ state through reaction calculations. Purpose: To investigate the structure and transition properties of the K=0+ and K=2+ bands of 24Mg employing the microscopic structure and reaction calculations via inelastic proton and alpha-scattering. Particularly, the E4 transitions to the $4^+_1$ and $4^+_2$ states were reexamined. Method: The structure of 24Mg was calculated employing the variation after the parity and total-angular momentum projections in the framework of the antisymmetrized molecular dynamics(AMD). The inelastic proton and alpha reactions were calculated by the microscopic coupled-channel (MCC) approach by folding the Melbourne g-matrix NN interaction with the AMD densities of 24Mg. Results: Reasonable results were obtained on the properties of the structure, including the energy spectra and E2 and E4 transitions of the K=0+ and K=2+ bands owing to the enhanced collectivity of triaxial deformation. The MCC+AMD calculation successfully reproduced the angular distributions of the $4^+_1$ and $4^+_2$ cross sections of proton scattering at incident energies of $E_p$=40--100MeV and alpha-scattering at $E_\alpha$=100--400MeV. Conclusions: This is the first microscopic calculation that described the unique properties of the $0^+_1\to 4^+_1$ transition. In the inelastic scattering to the $4^+_1$ state, the dominant two-step process of the $0^+_1\to 2^+_1\to 4^+_1$ transitions and the deconstructive interference is the weak one-step process were essential.


title: Physics-based Machine Learning Discovered Nano-circuitry for Nonlinear Ion Transport in Nanoporous Electrodes

abstract: Confined ion transport is involved in nanoporous ionic systems. However, it is challenging to mechanistically predict its electrical characteristics for rational system design and performance evaluation using electrical circuit model due to the gap between the circuit theory and the underlying physical chemistry. Here we demonstrate that machine learning can bridge this gap and produce physics-based nano-circuitry, based on equation discovery from the modified Poisson-Nernst-Planck simulation results where an anomalous constructive diffusion-migration interplay of confined ions is unveiled. This bridging technique allows us to gain physical insights of ion dynamics in nanoporous electrodes, such as the non-ideal cyclic voltammetry.

categories: cond-mat.mes-hall physics.app-ph physics.chem-ph physics.comp-ph physics.flu-dyn


title: A multinomial truncated D-vine copula mixed model for the joint meta-analysis of multiple diagnostic tests

abstract: There is an extensive literature on methods for meta-analysis of diagnostic studies, but it mainly focuses on a single test. However, the better understanding of a particular disease has led to the development of multiple tests. A multinomial generalized linear mixed model (GLMM) is recently proposed for the joint meta-analysis of studies comparing multiple tests. We propose a novel model for the joint meta-analysis of multiple tests, which assumes independent multinomial distributions for the counts of each combination of test results in diseased and non-diseased patients, conditional on the latent vector of probabilities of each combination of test results in diseased and non-diseased patients. For the random effects distribution of the latent proportions, we employ a truncated drawable vine copula that can cover flexible dependence structures. The proposed model includes the multinomial GLMM as a special case, but can also operate on the original scale of the latent proportions. Our methodology is demonstrated with a simulation study and using a meta-analysis of screening for Down syndrome with two tests: shortened humerus and shortened femur. The comparison of our method with the multinomial GLMM yields findings in the real data meta-analysis that change the current conclusions.


title: On the exact computation of linear frequency principle dynamics and its generalization

abstract: Recent works show an intriguing phenomenon of Frequency Principle (F-Principle) that deep neural networks (DNNs) fit the target function from low to high frequency during the training, which provides insight into the training and generalization behavior of DNNs in complex tasks. In this paper, through analysis of an infinite-width two-layer NN in the neural tangent kernel (NTK) regime, we derive the exact differential equation, namely Linear Frequency-Principle (LFP) model, governing the evolution of NN output function in the frequency domain during the training. Our exact computation applies for general activation functions with no assumption on size and distribution of training data. This LFP model unravels that higher frequencies evolve polynomially or exponentially slower than lower frequencies depending on the smoothness/regularity of the activation function. We further bridge the gap between training dynamics and generalization by proving that LFP model implicitly minimizes a Frequency-Principle norm (FP-norm) of the learned function, by which higher frequencies are more severely penalized depending on the inverse of their evolution rate. Finally, we derive an \textit{a priori} generalization error bound controlled by the FP-norm of the target function, which provides a theoretical justification for the empirical results that DNNs often generalize well for low frequency functions.


title: PoSAT: Proof-of-Work Availability and Unpredictability, without the Work

abstract: An important feature of Proof-of-Work (PoW) blockchains is full dynamic availability, allowing miners to go online and offline while requiring only 50% of the online miners to be honest. Existing Proof-of-stake (PoS), Proof-of-Space and related protocols are able to achieve this property only partially, either putting the additional assumption that adversary nodes to be online from the beginning and no new adversary nodes come online afterwards, or use additional trust assumptions for newly joining nodes.We propose a new PoS protocol PoSAT which can provably achieve dynamic availability fully without any additional assumptions. The protocol is based on the longest chain and uses a Verifiable Delay Function for the block proposal lottery to provide an arrow of time. The security analysis of the protocol draws on the recently proposed technique of Nakamoto blocks as well as the theory of branching random walks. An additional feature of PoSAT is the complete unpredictability of who will get to propose a block next, even by the winner itself. This unpredictability is at the same level of PoW protocols, and is stronger than that of existing PoS protocols using Verifiable Random Functions.


title: Guided Data Discovery in Interactive Visualizations via Active Search

abstract: Recent advances in visual analytics have enabled us to learn from user interactions and uncover analytic goals. These innovations set the foundation for actively guiding users during data exploration. Providing such guidance will become more critical as datasets grow in size and complexity, precluding exhaustive investigation. Meanwhile, the machine learning community also struggles with datasets growing in size and complexity, precluding exhaustive labeling. Active learning is a broad family of algorithms developed for actively guiding models during training. We will consider the intersection of these analogous research thrusts. First, we discuss the nuances of matching the choice of an active learning algorithm to the task at hand. This is critical for performance, a fact we demonstrate in a simulation study. We then present results of a user study for the particular task of data discovery guided by an active learning algorithm specifically designed for this task.


title: An Involution on Semistandard Skyline Fillings

abstract: Non-attacking skyline fillings were used by Haglund, Haiman and Loehr to establish a combinatorial formula for nonsymmetric Macdonald polynomials. Semistandard skyline fillings are non-attacking skyline fillings with both major index and coinversion number equal to zero, which serve as a combinatorial model for key polynomials. In this paper, we construct an involution on semistandard skyline fillings. This involution can be viewed as a vast generalization of the classical Bender--Knuth involution. As an application, we obtain that semistandard skyline fillings are compatible with the Demazure operators, offering a new combinatorial proof that nonsymmetric Macdonald polynomials specialize to key polynomials.


title: Predicting the popularity of scientific publications by an age-based diffusion model

abstract: Predicting the popularity of scientific publications has attracted many attentions from various disciplines. In this paper, we focus on the popularity prediction problem of scientific papers, and propose an age-based diffusion (AD) model to identify which paper will receive more citations in the near future and will be popular. The AD model is a mimic of the attention diffusion process along the citation networks. The experimental study shows that the AD model can achieve better prediction accuracy than other networkbased methods. For some newly published papers that have not accumulated many citations but will be popular in the near future, the AD model can substantially improve their rankings. This is really critical, because identifying the future highly cited papers from massive numbers of new papers published each month would provide very valuable references for researchers.


title: An Accurate and Fully-Automated Ensemble Model for Weekly Time Series Forecasting

abstract: Many businesses and industries require accurate forecasts for weekly time series nowadays. However, the forecasting literature does not currently provide easy-to-use, automatic, reproducible and accurate approaches dedicated to this task. We propose a forecasting method in this domain to fill this gap, leveraging state-of-the-art forecasting techniques, such as forecast combination, meta-learning, and global modelling. We consider different meta-learning architectures, algorithms, and base model pools. Based on all considered model variants, we propose to use a stacking approach with lasso regression which optimally combines the forecasts of four base models: a global Recurrent Neural Network model (RNN), Theta, Trigonometric Box-Cox ARMA Trend Seasonal (TBATS) and Dynamic Harmonic Regression ARIMA (DHR-ARIMA), as it shows the overall best performance across seven experimental weekly datasets on four evaluation metrics. Our proposed method also consistently outperforms a set of benchmarks and state-of-the-art weekly forecasting models by a considerable margin with statistical significance. Our method can produce the most accurate forecasts, in terms of mean sMAPE, for the M4 weekly dataset among all benchmarks and all original competition participants.


title: A boundary penalization technique to remove outliers from isogeometric analysis on tensor-product meshes

abstract: We introduce a boundary penalization technique to improve the spectral approximation of isogeometric analysis (IGA). The technique removes the outliers appearing in the high-frequency region of the approximate spectrum when using the $C^{p-1}, p$-th ($p\ge3$) order isogeometric elements. We focus on the classical Laplacian (Dirichlet) eigenvalue problem in 1D to illustrate the idea and then use the tensor-product structure to generate the stiffness and mass matrices for multiple dimensional problems. To remove the outliers, we penalize the product of the higher-order derivatives from both the solution and test spaces at the domain boundary. Intuitively, we construct a better approximation by weakly imposing features of the exact solution. Effectively, we add terms to the variational formulation at the boundaries with minimal extra computational cost. We then generalize the idea to remove the outliers for the isogeometric analysis to the Neumann eigenvalue problem (for $p\ge2$). The boundary penalization does not change the test and solution spaces. In the limiting case when the penalty goes to infinity, we perform the dispersion analysis of $C^2$ cubic elements for Dirichlet eigenvalue problem and $C^1$ quadratic elements for Neumann eigenvalue problem. We obtain the analytical eigenpairs for the resulting matrix eigenvalue problems. Numerical experiments show optimal convergence rates for the eigenvalues and eigenfunctions of the discrete operator.


title: Ultra-broadband acoustic ventilation barriers via hybrid-functional metasurfaces

abstract: Ventilation barriers allowing simultaneous sound blocking and free airflow passage are of great challenge but necessary for particular scenarios calling for sound-proofing ventilation. Previous works based on local resonance or Fano-like interference serve a narrow working range around the resonant or destructive-interference frequency. Efforts made on broadband designs show a limited bandwidth typically smaller than half an octave. Here, we theoretically design an ultra-broadband ventilation barrier via hybridizing dissipation and interference. Confirmed by experiments, the synergistic effect from our hybrid-functional metasurface significantly expand the scope of its working frequencies, leading to an effective blocking of more than 90% of incident energy in the range of 650-2000 Hz, while its structural thickness is only 53 mm $(\sim \lambda / 10)$. Our design shows great flexibility in customizing the broadband and is capable of handling sound coming from various directions, which has potential in air-permeable yet sound-proofing applications.


title: ALdataset: a benchmark for pool-based active learning

abstract: Active learning (AL) is a subfield of machine learning (ML) in which a learning algorithm could achieve good accuracy with less training samples by interactively querying a user/oracle to label new data points. Pool-based AL is well-motivated in many ML tasks, where unlabeled data is abundant, but their labels are hard to obtain. Although many pool-based AL methods have been developed, the lack of a comparative benchmarking and integration of techniques makes it difficult to: 1) determine the current state-of-the-art technique; 2) evaluate the relative benefit of new methods for various properties of the dataset; 3) understand what specific problems merit greater attention; and 4) measure the progress of the field over time. To conduct easier comparative evaluation among AL methods, we present a benchmark task for pool-based active learning, which consists of benchmarking datasets and quantitative metrics that summarize overall performance. We present experiment results for various active learning strategies, both recently proposed and classic highly-cited methods, and draw insights from the results.


title: SidechainNet: An All-Atom Protein Structure Dataset for Machine Learning

abstract: Despite recent advancements in deep learning methods for protein structure prediction and representation, little focus has been directed at the simultaneous inclusion and prediction of protein backbone and sidechain structure information. We present SidechainNet, a new dataset that directly extends the ProteinNet dataset. SidechainNet includes angle and atomic coordinate information capable of describing all heavy atoms of each protein structure. In this paper, we provide background information on the availability of protein structure data and the significance of ProteinNet. Thereafter, we argue for the potentially beneficial inclusion of sidechain information through SidechainNet, describe the process by which we organize SidechainNet, and provide a software package (https://github.com/jonathanking/sidechainnet) for data manipulation and training with machine learning models.


title: Global asymptotic stability of the active disassembly model of flagellar length control

abstract: Organelle size control is a fundamental question in biology that demonstrates the fascinating ability of cells to maintain homeostasis within their highly variable environments. Theoretical models describing cellular dynamics have the potential to help elucidate the principles underlying size control. Here, we perform a detailed study of the active disassembly model proposed in [Fai et al, Length regulation of multiple flagella that self-assemble from a shared pool of components, eLife, 8, (2019): e42599]. We construct a hybrid system which is shown to be well-behaved throughout the domain. We rule out the possibility of oscillations arising in the model and prove global asymptotic stability in the case of two flagella by the construction of a suitable Lyapunov function. Finally, we generalize the model to the case of arbitrary flagellar number in order to study olfactory sensory neurons, which have up to twenty cilia per cell. We show that our theoretical results may be extended to this case and explore the implications of this universal mechanism of size control.


title: Pose And Joint-Aware Action Recognition

abstract: Recent progress on action recognition has mainly focused on RGB and optical flow features. In this paper, we approach the problem of joint-based action recognition. Unlike other modalities, constellation of joints and their motion generate models with succinct human motion information for activity recognition. We present a new model for joint-based action recognition, which first extracts motion features from each joint separately through a shared motion encoder before performing collective reasoning. Our joint selector module re-weights the joint information to select the most discriminative joints for the task. We also propose a novel joint-contrastive loss that pulls together groups of joint features which convey the same action. We strengthen the joint-based representations by using a geometry-aware data augmentation technique which jitters pose heatmaps while retaining the dynamics of the action. We show large improvements over the current state-of-the-art joint-based approaches on JHMDB, HMDB, Charades, AVA action recognition datasets. A late fusion with RGB and Flow-based approaches yields additional improvements. Our model also outperforms the existing baseline on Mimetics, a dataset with out-of-context actions.


title: Thermal conductivity of the quasi-1D materials TaSe3 and ZrTe3

abstract: The high breakdown current densities and resilience to scaling of the metallic transition metal trichalcogenides TaSe3 and ZrTe3 make them of interest for possible interconnect applications, and it motivates this study of their thermal conductivities and phonon properties. These crystals consist of planes of strongly bonded one-dimensional chains more weakly bonded to neighboring chains. Phonon dispersions and the thermal conductivity tensors are calculated using density functional theory combined with an iterative solution of the phonon Boltzmann transport equation. The phonon velocities and the thermal conductivities of TaSe3 are considerably more anisotropic than those of ZrTe3. The maximum LA velocity in ZrTe3 occurs in the cross-chain direction, and this is consistent with the strong cross-chain bonding that gives rise to large Fermi velocities in that direction. The thermal conductivities are similar to those of other metallic two-dimensional transition metal dichalcogenides. At room temperature, a significant portion of the heat is carried by the optical modes. In the low frequency range, the phonon lifetimes and mean free paths in TaSe3 are considerably shorter than those in ZrTe3. The shorter lifetimes in TaSe3 are consistent with the presence of lower frequency optical branches and zone-folding features in the acoustic branches that arise due to the doubling of the TaSe3 unit cell within the plane.


title: Scaling Limits of Fluctuations of Extended-Source Internal DLA

abstract: In a previous work, we showed that the 2D, extended-source internal DLA (IDLA) of Levine and Peres is $\delta^{3/5}$-close to its scaling limit, if $\delta$ is the lattice size. In this paper, we investigate the scaling limits of the fluctuations themselves. Namely, we show that two naturally defined error functions, which measure the "lateness" of lattice points at one time and at all times, respectively, converge to geometry-dependent Gaussian random fields. We use these results to calculate point-correlation functions associated with the fluctuations of the flow. Along the way, we demonstrate similar $\delta^{3/5}$ bounds on the fluctuations of the related divisible sandpile model of Levine and Peres.


title: Piecewise-Linear Motion Planning amidst Static, Moving, or Morphing Obstacles

abstract: We propose a novel method for planning shortest length piecewise-linear motions through complex environments punctured with static, moving, or even morphing obstacles. Using a moment optimization approach, we formulate a hierarchy of semidefinite programs that yield increasingly refined lower bounds converging monotonically to the optimal path length. For computational tractability, our global moment optimization approach motivates an iterative motion planner that outperforms competing sampling-based and nonlinear optimization baselines. Our method natively handles continuous time constraints without any need for time discretization, and has the potential to scale better with dimensions compared to popular sampling-based methods.


title: A Generalizable and Accessible Approach to Machine Learning with Global Satellite Imagery

abstract: Combining satellite imagery with machine learning (SIML) has the potential to address global challenges by remotely estimating socioeconomic and environmental conditions in data-poor regions, yet the resource requirements of SIML limit its accessibility and use. We show that a single encoding of satellite imagery can generalize across diverse prediction tasks (e.g. forest cover, house price, road length). Our method achieves accuracy competitive with deep neural networks at orders of magnitude lower computational cost, scales globally, delivers label super-resolution predictions, and facilitates characterizations of uncertainty. Since image encodings are shared across tasks, they can be centrally computed and distributed to unlimited researchers, who need only fit a linear regression to their own ground truth data in order to achieve state-of-the-art SIML performance.


title: Uncertainty-aware Contact-safe Model-based Reinforcement Learning

abstract: This letter presents contact-safe Model-based Reinforcement Learning (MBRL) for robot applications that achieves contact-safe behaviors in the learning process. In typical MBRL, we cannot expect the data-driven model to generate accurate and reliable policies to the intended robotic tasks during the learning process due to sample scarcity. Operating these unreliable policies in a contact-rich environment could cause damage to the robot and its surroundings. To alleviate the risk of causing damage through unexpected intensive physical contacts, we present the contact-safe MBRL that associates the probabilistic Model Predictive Control's (pMPC) control limits with the model uncertainty so that the allowed acceleration of controlled behavior is adjusted according to learning progress. Control planning with such uncertainty-aware control limits is formulated as a deterministic MPC problem using a computation-efficient approximated GP dynamics and an approximated inference technique. Our approach's effectiveness is evaluated through bowl mixing tasks with simulated and real robots, scooping tasks with a real robot as examples of contact-rich manipulation skills. (video: https://youtu.be/sdhHP3NhYi0)


title: Hyperspectral interference tomography of nacre

abstract: Structural characterization of biologically formed materials is essential for understanding biological phenomena and their environment, and generating new bio-inspired engineering concepts. For example, nacre -- formed by mollusks in the ocean -- encodes local environmental conditions throughout its formation and has exceptional strength due to its nanoscale brick-and-mortar structure. This layered structure, comprising transparent aragonite tablets bonded with an ultra-thin organic polymer, also results in stunning interference colors. Existing methods of structural characterization of nacre rely on some form of cross-sectional analysis, such as scanning electron microscopy or polarization-dependent imaging contrast (PIC) mapping. However, these techniques are destructive and too time- and resource-intensive to analyze large sample areas. Here we present an all-optical, rapid, and non-destructive imaging technique -- hyperspectral interference tomography (HIT) -- to spatially map the structural parameters of nacre and other disordered layered materials. We combined hyperspectral imaging with optical-interference modeling to infer the mean tablet thickness and disordering of nacre layers across entire mollusk shells at various stages of development, observing a previously unknown relationship between the growth of the mollusk and tablet thickness. Our rapid, inexpensive, and nondestructive method can be readily applied to in-field studies.

categories: physics.optics physics.app-ph physics.bio-ph physics.ins-det


title: Light and colour variations of Mira variables in the Small Magellanic Cloud

abstract: The goal of this paper is to characterise the light variation properties of Mira variables in the Small Magellanic Cloud. We have investigated a combined optical and near infrared multi-epoch dataset of Mira variables based on our monitoring data obtained over 15 years. Bolometric correction relations are formulated for various near-infrared colours. We find that the same bolometric correction equation holds for both the bolometricly brightest and faintest pulsation phases. Period-bolometric magnitude relations and period-colour relations were derived using time-averaged values. Phase lags between bolometric phase and optical and near-infrared phases were detected from the O-rich (the surface C/O number ratio is below unity) Mira variables, while no significant systematic lags were observed in most of the C-rich (the C/O ratio is over unity) ones. Some Miras show colour phase inversions, e.g., $H-K_{\rm s}$ at its bluest while $J-H$ and $J-K_{\rm s}$ are at their reddest values at about the bolometricly brightest phase. Their occurrence conditions were studied but no clear direct or indirect trigger was found. A large NIR colour change unassociated with stellar pulsation was observed in Miras with long secondary periods, and its possible explanation is described.


title: Central Limit Theorem for Majority Dynamics: Bribing Three Voters Suffices

abstract: Given a graph $G$ and some initial labelling $\sigma : V(G) \to \{Red, Blue\}$ of its vertices, the \textit{majority dynamics model} is the deterministic process where at each stage, every vertex simultaneously replaces its label with the majority label among its neighbors (remaining unchanged in the case of a tie). We prove---for a wide range of parameters---that if an initial assignment is fixed and we independently sample an Erd\H{o}s--R\'enyi random graph, $G_{n,p}$, then after one step of majority dynamics, the number of vertices of each label follows a central limit law. As a corollary, we provide a strengthening of a theorem of Benjamini, Chan, O'Donnell, Tamuz, and Tan about the number of steps required for the process to reach unanimity when the initial assignment is also chosen randomly. Moreover, suppose there are initially three more red vertices than blue. In this setting, we prove that if we independently sample the graph $G_{n,1/2}$, then with probability at least $51\%$, the majority dynamics process will converge to every vertex being red. This improves a result of Tran and Vu who addressed the case that the initial lead is at least 10.


title: The specific star formation rate function at different mass scales and quenching: A comparison between cosmological models and SDSS

abstract: We present the eddington bias corrected Specific Star Formation Rate Function (sSFRF) at different stellar mass scales from a sub-sample of the Sloan Digital Sky Survey Data Release DR7 (SDSS), which is considered complete both in terms of stellar mass (${\rm M_{\star}}$) and star formation rate (SFR). The above enable us to study qualitatively and quantitatively quenching, the distribution of passive/star-forming galaxies and perform comparisons with the predictions from state-of-the-art cosmological models, within the same ${\rm M_{\star}}$ and SFR limits. We find that at the low mass end (${\rm M_{\star}} = 10^{9.5} - 10^{10} \, {\rm M_{\odot}}$) the sSFRF is mostly dominated by star-forming objects. However, moving to the two more massive bins (${\rm M_{\star}} = 10^{10} - 10^{10.5} \, {\rm M_{\odot}}$ and ${\rm M_{\star}} = 10^{10.5} - 10^{11} \, {\rm M_{\odot}}$) a bi-modality with two peaks emerges. One peak represents the star-forming population, while the other describes a rising passive population. The bi-modal form of the sSFRFs is not reproduced by a range of cosmological simulations (e.g. Illustris, EAGLE, Mufasa, IllustrisTNG) which instead generate mostly the star-forming population, while a bi-modality emerges in others (e.g. L-Galaxies, Shark, Simba). Our findings reflect the need for the employed quenching schemes in state-of-the-art models to be reconsidered, involving prescriptions that allow "quenched galaxies" to retain a small level of SF activity (sSFR $=$ ${\rm 10^{-11} {\rm yr^{-1}}}$-${\rm 10^{-12} {\rm yr^{-1}}}$) and generate an adequate passive population/bi-modality even at intermediate masses (${\rm M_{\star}} = 10^{10} - 10^{10.5} \, {\rm M_{\odot}}$).


title: RAT iLQR: A Risk Auto-Tuning Controller to Optimally Account for Stochastic Model Mismatch

abstract: Successful robotic operation in stochastic environments relies on accurate characterization of the underlying probability distributions, yet this is often imperfect due to limited knowledge. This work presents a control algorithm that is capable of handling such distributional mismatches. Specifically, we propose a novel nonlinear MPC for distributionally robust control, which plans locally optimal feedback policies against a worst-case distribution within a given KL divergence bound from a Gaussian distribution. Leveraging mathematical equivalence between distributionally robust control and risk-sensitive optimal control, our framework also provides an algorithm to dynamically adjust the risk-sensitivity level online for risk-sensitive control. The benefits of the distributional robustness as well as the automatic risk-sensitivity adjustment are demonstrated in a dynamic collision avoidance scenario where the predictive distribution of human motion is erroneous.


title: Anisotropic Stroke Control for Multiple Artists Style Transfer

abstract: Though significant progress has been made in artistic style transfer, semantic information is usually difficult to be preserved in a fine-grained locally consistent manner by most existing methods, especially when multiple artists styles are required to transfer within one single model. To circumvent this issue, we propose a Stroke Control Multi-Artist Style Transfer framework. On the one hand, we develop a multi-condition single-generator structure which first performs multi-artist style transfer. On the one hand, we design an Anisotropic Stroke Module (ASM) which realizes the dynamic adjustment of style-stroke between the non-trivial and the trivial regions. ASM endows the network with the ability of adaptive semantic-consistency among various styles. On the other hand, we present an novel Multi-Scale Projection Discriminator} to realize the texture-level conditional generation. In contrast to the single-scale conditional discriminator, our discriminator is able to capture multi-scale texture clue to effectively distinguish a wide range of artistic styles. Extensive experimental results well demonstrate the feasibility and effectiveness of our approach. Our framework can transform a photograph into different artistic style oil painting via only ONE single model. Furthermore, the results are with distinctive artistic style and retain the anisotropic semantic information. The code is already available on github: https://github.com/neuralchen/ASMAGAN.


title: Flexible, Decentralized Access Control for Smart Buildings with Smart Contracts

abstract: Large commercial buildings are complex cyber-physical systems containing expensive and critical equipment that ensure the safety and comfort of their numerous occupants. Yet occupant and visitor access to spaces and equipment within these buildings are still managed through unsystematic, inefficient, and human-intensive processes. As a standard practice, long-term building occupants are given access privileges to rooms and equipment based on their organizational roles, while visitors have to be escorted by their hosts. This approach is conservative and inflexible. In this paper, we describe a methodology that can flexibly and securely manage building access privileges for long-term occupants and short-term visitors alike, taking into account the risk associated with accessing each space within the building. Our methodology relies on blockchain smart contracts to describe, grant, audit, and revoke fine-grained permissions for building occupants and visitors, in a decentralized fashion. The smart contracts are specified through a process that leverages the information compiled from Brick and BOT models of the building. We illustrate the proposed method through a typical application scenario in the context of a real office building and argue that it can greatly reduce the administration overhead, while, at the same time, providing fine-grained, auditable access control.


title: Projection-free Online Learning over Strongly Convex Sets

abstract: To efficiently solve online problems with complicated constraints, projection-free algorithms including online frank-wolfe (OFW) and its variants have received significant interest recently. However, in the general case, existing efficient projection-free algorithms only achieved the regret bound of $O(T^{3/4})$, which is worse than the regret of projection-based algorithms, where $T$ is the number of decision rounds. In this paper, we study the special case of online learning over strongly convex sets, for which we first prove that OFW can enjoy a better regret bound of $O(T^{2/3})$ for general convex losses. The key idea is to refine the decaying step-size in the original OFW by a simple line search rule. Furthermore, for strongly convex losses, we propose a strongly convex variant of OFW by redefining the surrogate loss function in OFW. We show that it achieves a regret bound of $O(T^{2/3})$ over general convex sets and a better regret bound of $O(\sqrt{T})$ over strongly convex sets.


title: Generating Diverse Translation from Model Distribution with Dropout

abstract: Despite the improvement of translation quality, neural machine translation (NMT) often suffers from the lack of diversity in its generation. In this paper, we propose to generate diverse translations by deriving a large number of possible models with Bayesian modelling and sampling models from them for inference. The possible models are obtained by applying concrete dropout to the NMT model and each of them has specific confidence for its prediction, which corresponds to a posterior model distribution under specific training data in the principle of Bayesian modeling. With variational inference, the posterior model distribution can be approximated with a variational distribution, from which the final models for inference are sampled. We conducted experiments on Chinese-English and English-German translation tasks and the results shows that our method makes a better trade-off between diversity and accuracy.


title: Tongji University Team for the VoxCeleb Speaker Recognition Challenge 2020

abstract: In this report, we describe the submission of Tongji University team to the CLOSE track of the VoxCeleb Speaker Recognition Challenge (VoxSRC) 2020 at Interspeech 2020. We investigate different speaker recognition systems based on the popular ResNet-34 architecture, and train multiple variants via various loss functions. Both Offline and online data augmentation are introduced to improve the diversity of the training set, and score normalization with the exhaustive grid search is applied in the post-processing. Our best fusion of five selected systems for the CLOSE track achieves 0.2800 DCF and 4.7770% EER on the challenge.


title: Who's in the Gang? Revealing Coordinating Communities in Social Media

abstract: Political astroturfing and organised trolling are online malicious behaviours with significant real-world effects. Common approaches examining these phenomena focus on broad campaigns rather than the small groups responsible. To reveal latent networks of cooperating accounts, we propose a novel temporal window approach that relies on account interactions and metadata alone. It detects groups of accounts engaging in behaviours that, in concert, execute different goal-based strategies, which we describe. Our approach is validated against two relevant datasets with ground truth data.


title: Spinors and Scalars in curved spacetime: neutrino dark energy (DE$_{\nu}$)

abstract: We study the interaction, in general curved spacetime, between a spinor and a scalar field describing dark energy; the so-called DE$_{\nu}$ model in curved space. The dominant term is the dimension 5 operator, which results in different energy shifts for the neutrino states: an Aharonov-Bohm-like effect. We study the phenomenology of this term and make observational predictions to detect dark energy interactions in the laboratory due to its effect on neutrino oscillation experiments, which opens up the possibility of designing underground experiments to detect dark energy. This dimension 5 operator beyond the Standard Model interaction is less suppressed than the widely discussed dimension 6 operator, which corresponds to mass varying neutrinos; the dimension 5 operator does not suffer from gravitational instabilities.


title: Measuring the Dynamic Impact of High-Speed Railways on Urban Interactions in China

abstract: High-speed rail (HSR) has become an important mode of inter-city transportation between large cities. Inter-city interaction facilitated by HSR tends to play a more prominent role in promoting urban and regional economic integration and development. Quantifying the impact of HSR's interaction on cities and people is therefore crucial for long-term urban and regional development planning and policy making. We develop an evaluation framework using toponym information from social media as a proxy to estimate the dynamics of such interactions. This paper adopts two types of spatial information: toponyms from social media posts, and the geographical location information embedded in social media posts. The framework highlights the asymmetric nature of social interaction among cities, and proposes a series of metrics to quantify such impact from multiple perspectives, including interaction strength, spatial decay, and channel effect. The results show that HSRs not only greatly expand the uneven distribution of inter-city connections, but also significantly reshape the interactions that occur along HSR routes through the channel effect.


title: Analysis of Twitter and YouTube during USelections 2020

abstract: The presidential elections in the United States on 3 November 2020 have caused extensive discussions on social media. A part of the content on US elections is organic, coming from users discussing their opinions of the candidates, political positions, or relevant content presented on television. Another significant part of the content generated originates from organized campaigns, both official and by astroturfing. In this study, we obtain approximately 17.5M tweets containing 3M users, based on prevalent hashtags related to US election 2020, as well as the related YouTube links, contained in the Twitter dataset, likes, dislikes and comments of the videos and conduct volume, sentiment and graph analysis on the communities formed. Particularly, we study the daily traffic per prevalent hashtags, plot the retweet graph from July to September 2020, show how its main connected component becomes denser in the period closer to the elections and highlight the two main entities ('Biden' and 'Trump'). Additionally, we gather the related YouTube links contained in the previous dataset and perform sentiment analysis. The results on sentiment analysis on the Twitter corpus and the YouTube metadata gathered, show the positive and negative sentiment for the two entities throughout this period. The results of sentiment analysis indicate that 45.7% express positive sentiment towards Trump in Twitter and 33.8% positive sentiment towards Biden, while 14.55% of users express positive sentiment in YouTube metadata gathered towards Trump and 8.7% positive sentiment towards Biden. Our analysis fill the gap between the connection of offline events and their consequences in social media by monitoring important events in real world and measuring public volume and sentiment before and after the event in social media.


title: PRIMAL2: Pathfinding via Reinforcement and Imitation Multi-Agent Learning -- Lifelong

abstract: Multi-agent path finding (MAPF) is an indispensable component of large-scale robot deployments in numerous domains ranging from airport management to warehouse automation. In particular, this work addresses lifelong MAPF (LMAPF) - an online variant of the problem where agents are immediately assigned a new goal upon reaching their current one - in dense and highly structured environments, typical of real-world warehouse operations. Effectively solving LMAPF in such environments requires expensive coordination between agents as well as frequent replanning abilities, a daunting task for existing coupled and decoupled approaches alike. With the purpose of achieving considerable agent coordination without any compromise on reactivity and scalability, we introduce PRIMAL2, a distributed reinforcement learning framework for LMAPF where agents learn fully decentralized policies to reactively plan paths online in a partially observable world. We extend our previous work, which was effective in low-density sparsely occupied worlds, to highly structured and constrained worlds by identifying behaviors and conventions which improve implicit agent coordination, and enable their learning through the construction of a novel local agent observation and various training aids. We present extensive results of PRIMAL2 in both MAPF and LMAPF environments and compare its performance to state-of-the-art planners in terms of makespan and throughput. We show that PRIMAL2 significantly surpasses our previous work and performs comparably to these baselines, while allowing real-time re-planning and scaling up to 2048 agents.


title: DiDi's Machine Translation System for WMT2020

abstract: This paper describes DiDi AI Labs' submission to the WMT2020 news translation shared task. We participate in the translation direction of Chinese->English. In this direction, we use the Transformer as our baseline model, and integrate several techniques for model enhancement, including data filtering, data selection, back-translation, fine-tuning, model ensembling, and re-ranking. As a result, our submission achieves a BLEU score of $36.6$ in Chinese->English.


title: How many images do I need? Understanding how sample size per class affects deep learning model performance metrics for balanced designs in autonomous wildlife monitoring

abstract: Deep learning (DL) algorithms are the state of the art in automated classification of wildlife camera trap images. The challenge is that the ecologist cannot know in advance how many images per species they need to collect for model training in order to achieve their desired classification accuracy. In fact there is limited empirical evidence in the context of camera trapping to demonstrate that increasing sample size will lead to improved accuracy. In this study we explore in depth the issues of deep learning model performance for progressively increasing per class (species) sample sizes. We also provide ecologists with an approximation formula to estimate how many images per animal species they need for certain accuracy level a priori. This will help ecologists for optimal allocation of resources, work and efficient study design. In order to investigate the effect of number of training images; seven training sets with 10, 20, 50, 150, 500, 1000 images per class were designed. Six deep learning architectures namely ResNet-18, ResNet-50, ResNet-152, DnsNet-121, DnsNet-161, and DnsNet-201 were trained and tested on a common exclusive testing set of 250 images per class. The whole experiment was repeated on three similar datasets from Australia, Africa and North America and the results were compared. Simple regression equations for use by practitioners to approximate model performance metrics are provided. Generalized additive models (GAM) are shown to be effective in modelling DL performance metrics based on the number of training images per class, tuning scheme and dataset. Key-words: Camera Traps, Deep Learning, Ecological Informatics, Generalised Additive Models, Learning Curves, Predictive Modelling, Wildlife.


title: PrivNet: Safeguarding Private Attributes in Transfer Learning for Recommendation

abstract: Transfer learning is an effective technique to improve a target recommender system with the knowledge from a source domain. Existing research focuses on the recommendation performance of the target domain while ignores the privacy leakage of the source domain. The transferred knowledge, however, may unintendedly leak private information of the source domain. For example, an attacker can accurately infer user demographics from their historical purchase provided by a source domain data owner. This paper addresses the above privacy-preserving issue by learning a privacy-aware neural representation by improving target performance while protecting source privacy. The key idea is to simulate the attacks during the training for protecting unseen users' privacy in the future, modeled by an adversarial game, so that the transfer learning model becomes robust to attacks. Experiments show that the proposed PrivNet model can successfully disentangle the knowledge benefitting the transfer from leaking the privacy.


title: Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Differential Equation

abstract: Video generation models often operate under the assumption of fixed frame rates, which leads to suboptimal performance when it comes to handling flexible frame rates (e.g., increasing the frame rate of the more dynamic portion of the video as well as handling missing video frames). To resolve the restricted nature of existing video generation models' ability to handle arbitrary timesteps, we propose continuous-time video generation by combining neural ODE (Vid-ODE) with pixel-level video processing techniques. Using ODE-ConvGRU as an encoder, a convolutional version of the recently proposed neural ODE, which enables us to learn continuous-time dynamics, Vid-ODE can learn the spatio-temporal dynamics of input videos of flexible frame rates. The decoder integrates the learned dynamics function to synthesize video frames at any given timesteps, where the pixel-level composition technique is used to maintain the sharpness of individual frames. With extensive experiments on four real-world video datasets, we verify that the proposed Vid-ODE outperforms state-of-the-art approaches under various video generation settings, both within the trained time range (interpolation) and beyond the range (extrapolation). To the best of our knowledge, Vid-ODE is the first work successfully performing continuous-time video generation using real-world videos.


title: New Ideas and Trends in Deep Multimodal Content Understanding: A Review

abstract: The focus of this survey is on the analysis of two modalities of multimodal deep learning: image and text. Unlike classic reviews of deep learning where monomodal image classifiers such as VGG, ResNet and Inception module are central topics, this paper will examine recent multimodal deep models and structures, including auto-encoders, generative adversarial nets and their variants. These models go beyond the simple image classifiers in which they can do uni-directional (e.g. image captioning, image generation) and bi-directional (e.g. cross-modal retrieval, visual question answering) multimodal tasks. Besides, we analyze two aspects of the challenge in terms of better content understanding in deep multimodal applications. We then introduce current ideas and trends in deep multimodal feature learning, such as feature embedding approaches and objective function design, which are crucial in overcoming the aforementioned challenges. Finally, we include several promising directions for future research.