title: Symmetry-protected intermediate trivial phases in quantum spin chains

abstract: Symmetry-protected trivial (SPt) phases of matter are the product-state analogue of symmetry-protected topological (SPT) phases. This means, SPt phases can be adiabatically connected to a product state by some path that preserves the protecting symmetry. Moreover, SPt and SPT phases can be adiabatically connected to each other when interaction terms that break the symmetries protecting the SPT order are added in the Hamiltonian. It is also known that spin-1 SPT phases in quantum spin chains can emerge as effective intermediate phases of spin-2 Hamiltonians. In this paper we show that a similar scenario is also valid for SPt phases. More precisely, we show that for a given spin-2 quantum chain, effective intermediate spin-1 SPt phases emerge in some regions of the phase diagram, these also being adiabatically connected to non-trivial intermediate SPT phases. We characterize the phase diagram of our model by studying quantities such as the entanglement entropy, symmetry-related order parameters, and 1-site fidelities. Our numerical analysis uses Matrix Product States (MPS) and the infinite Time-Evolving Block Decimation (iTEBD) method to approximate ground states of the system in the thermodynamic limit. Moreover, we provide a field theory description of the possible quantum phase transitions between the SPt phases. Together with the numerical results, such a description shows that the transitions may be described by Conformal Field Theories (CFT) with central charge c=1. Our results are in agreement, and further generalize, those in [Y. Fuji, F. Pollmann, M. Oshikawa, Phys. Rev. Lett. 114, 177204 (2015)].


title: Bi-Hamiltonian geometry and canonical spectral coordinates for the Rational Calogero-Moser system

abstract: We reconsider the (rational) Calogero-Moser system from the point of view of bi-Hamiltonian geometry. By using geometrical tools of the latter, we explicitly construct set(s) of spectral canonical coordinates, that is, complete sets of Darboux coordinates defined by the eigenvalues and the eigenvectors of the Lax matrix.


title: Robust Classification by Pre-conditioned LASSO and Transductive Diffusion Component Analysis

abstract: Modern machine learning-based recognition approaches require large-scale datasets with large number of labelled training images. However, such datasets are inherently difficult and costly to collect and annotate. Hence there is a great and growing interest in automatic dataset collection methods that can leverage the web. % which are collected % in a cheap, efficient and yet unreliable way. Collecting datasets in this way, however, requires robust and efficient ways for detecting and excluding outliers that are common and prevalent. % Outliers are thus a % prominent treat of using these dataset. So far, there have been a limited effort in machine learning community to directly detect outliers for robust classification. Inspired by the recent work on Pre-conditioned LASSO, this paper formulates the outlier detection task using Pre-conditioned LASSO and employs \red{unsupervised} transductive diffusion component analysis to both integrate the topological structure of the data manifold, from labeled and unlabeled instances, and reduce the feature dimensionality. Synthetic experiments as well as results on two real-world classification tasks show that our framework can robustly detect the outliers and improve classification.


title: Communicating Semantics: Reference by Description

abstract: Messages often refer to entities such as people, places and events. Correct identification of the intended reference is an essential part of communication. Lack of shared unique names often complicates entity reference. Shared knowledge can be used to construct uniquely identifying descriptive references for entities with ambiguous names. We introduce a mathematical model for `Reference by Description', derive results on the conditions under which, with high probability, programs can construct unambiguous references to most entities in the domain of discourse and provide empirical validation of these results.


title: Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning

abstract: The ability to act in multiple environments and transfer previous knowledge to new situations can be considered a critical aspect of any intelligent agent. Towards this goal, we define a novel method of multitask and transfer learning that enables an autonomous agent to learn how to behave in multiple tasks simultaneously, and then generalize its knowledge to new domains. This method, termed "Actor-Mimic", exploits the use of deep reinforcement learning and model compression techniques to train a single policy network that learns how to act in a set of distinct tasks by using the guidance of several expert teachers. We then show that the representations learnt by the deep policy network are capable of generalizing to new tasks with no prior expert guidance, speeding up learning in novel environments. Although our method can in general be applied to a wide range of problems, we use Atari games as a testing environment to demonstrate these methods.


title: Online Batch Selection for Faster Training of Neural Networks

abstract: Deep neural networks are commonly trained using stochastic non-convex optimization procedures, which are driven by gradient information estimated on fractions (batches) of the dataset. While it is commonly accepted that batch size is an important parameter for offline tuning, the benefits of online selection of batches remain poorly understood. We investigate online batch selection strategies for two state-of-the-art methods of stochastic gradient-based optimization, AdaDelta and Adam. As the loss function to be minimized for the whole dataset is an aggregation of loss functions of individual datapoints, intuitively, datapoints with the greatest loss should be considered (selected in a batch) more frequently. However, the limitations of this intuition and the proper control of the selection pressure over time are open questions. We propose a simple strategy where all datapoints are ranked w.r.t. their latest known loss value and the probability to be selected decays exponentially as a function of rank. Our experimental results on the MNIST dataset suggest that selecting batches speeds up both AdaDelta and Adam by a factor of about 5.


title: Channel-Adaptive Packetization Policy for Minimal Latency and Maximal Energy Efficiency

abstract: This article considers the problem of delay optimal bundling of the input symbols into transmit packets in the entry point of a wireless sensor network such that the link delay is minimized under an arbitrary arrival rate and a given channel error rate. The proposed policy exploits the variable packet length feature of contemporary communications protocols in order to minimize the link delay via packet length regularization. This is performed through concrete characterization of the end-to-end link delay for zero error tolerance system with First Come First Serve (FCFS)queuing discipline and Automatic Repeat Request (ARQ) retransmission mechanism. The derivations are provided for an uncoded system as well as a coded system with a given bit error rate. The proposed packetization policy provides an optimal packetization interval that minimizes the end-to-end delay for a given channel with certain bit error probability. This algorithm can also be used for near-optimal bundling of input symbols for dynamic channel conditions provided that the channel condition varies slowly over time with respect to symbol arrival rate. This algorithm complements the current network-based delay-optimal routing and scheduling algorithms in order to further reduce the end-to-end delivery time. Moreover, the proposed method is employed to solve the problem of energy efficiency maximization under an average delay constraint by recasting it as a convex optimization problem.


title: Photon Detection System Designs for the Deep Underground Neutrino Experiment

abstract: The Deep Underground Neutrino Experiment (DUNE) will be a premier facility for exploring long-standing questions about the boundaries of the standard model. Acting in concert with the liquid argon time projection chambers underpinning the far detector design, the DUNE photon detection system will capture ultraviolet scintillation light in order to provide valuable timing information for event reconstruction. To maximize the active area while maintaining a small photocathode coverage, the experiment will utilize a design based on plastic light guides coated with a wavelength-shifting compound, along with silicon photomultipliers, to collect and record scintillation light from liquid argon. This report presents recent preliminary performance measurements of this baseline design and several alternative designs which promise significant improvements in sensitivity to low-energy interactions.


title: The prompt atmospheric neutrino flux in the light of LHCb

abstract: The recent observation of very high energy cosmic neutrinos by IceCube heralds the beginning of neutrino astronomy. At these energies, the dominant background to the astrophysical signal is the flux of `prompt' neutrinos, arising from the decay of charmed mesons produced by cosmic ray collisions in the atmosphere. In this work we provide predictions for the prompt atmospheric neutrino flux in the framework of perturbative QCD, using state-of-the-art Monte Carlo event generators. Our calculation includes the constraints set by charm production measurements from the LHCb experiment at 7 TeV, and has been recently validated with the corresponding 13 TeV data. Our results for the prompt flux are a factor of about 2 below the previous benchmark calculation, in general agreement with two other recent estimates, and with an improved estimate of the uncertainty. This alleviates the existing tension between the theoretical prediction and IceCube limits, and suggests that a direct direction of the prompt flux is imminent.


title: Axion Isocurvature and Magnetic Monopoles

abstract: We propose a simple mechanism to suppress axion isocurvature fluctuations using hidden sector magnetic monopoles. This allows for the Peccei-Quinn scale to be of order the unification scale consistently with high scale inflation.


title: How much data is needed to train a medical image deep learning system to achieve necessary high accuracy?

abstract: The use of Convolutional Neural Networks (CNN) in natural image classification systems has produced very impressive results. Combined with the inherent nature of medical images that make them ideal for deep-learning, further application of such systems to medical image classification holds much promise. However, the usefulness and potential impact of such a system can be completely negated if it does not reach a target accuracy. In this paper, we present a study on determining the optimum size of the training data set necessary to achieve high classification accuracy with low variance in medical image classification systems. The CNN was applied to classify axial Computed Tomography (CT) images into six anatomical classes. We trained the CNN using six different sizes of training data set (5, 10, 20, 50, 100, and 200) and then tested the resulting system with a total of 6000 CT images. All images were acquired from the Massachusetts General Hospital (MGH) Picture Archiving and Communication System (PACS). Using this data, we employ the learning curve approach to predict classification accuracy at a given training sample size. Our research will present a general methodology for determining the training data set size necessary to achieve a certain target classification accuracy that can be easily applied to other problems within such systems.


title: Generating Sentences from a Continuous Space

abstract: The standard recurrent neural network language model (RNNLM) generates sentences one word at a time and does not work from an explicit global sentence representation. In this work, we introduce and study an RNN-based variational autoencoder generative model that incorporates distributed latent representations of entire sentences. This factorization allows it to explicitly model holistic properties of sentences such as style, topic, and high-level syntactic features. Samples from the prior over these sentence representations remarkably produce diverse and well-formed sentences through simple deterministic decoding. By examining paths through this latent space, we are able to generate coherent novel sentences that interpolate between known sentences. We present techniques for solving the difficult learning problem presented by this model, demonstrate its effectiveness in imputing missing words, explore many interesting properties of the model's latent sentence space, and present negative results on the use of the model in language modeling.


title: Structured Prediction Energy Networks

abstract: We introduce structured prediction energy networks (SPENs), a flexible framework for structured prediction. A deep architecture is used to define an energy function of candidate labels, and then predictions are produced by using back-propagation to iteratively optimize the energy with respect to the labels. This deep architecture captures dependencies between labels that would lead to intractable graphical models, and performs structure learning by automatically learning discriminative features of the structured output. One natural application of our technique is multi-label classification, which traditionally has required strict prior assumptions about the interactions between labels to ensure tractable learning and prediction. We are able to apply SPENs to multi-label problems with substantially larger label sets than previous applications of structured prediction, while modeling high-order interactions using minimal structural assumptions. Overall, deep learning provides remarkable tools for learning features of the inputs to a prediction problem, and this work extends these techniques to learning features of structured outputs. Our experiments provide impressive performance on a variety of benchmark multi-label classification tasks, demonstrate that our technique can be used to provide interpretable structure learning, and illuminate fundamental trade-offs between feed-forward and iterative structured prediction.


title: Learning Representations Using Complex-Valued Nets

abstract: Complex-valued neural networks (CVNNs) are an emerging field of research in neural networks due to their potential representational properties for audio, image, and physiological signals. It is common in signal processing to transform sequences of real values to the complex domain via a set of complex basis functions, such as the Fourier transform. We show how CVNNs can be used to learn complex representations of real valued time-series data. We present methods and results using a framework that can compose holomorphic and non-holomorphic functions in a multi-layer network using a theoretical result called the Wirtinger derivative. We test our methods on a representation learning task for real-valued signals, recurrent complex-valued networks and their real-valued counterparts. Our results show that recurrent complex-valued networks can perform as well as their real-valued counterparts while learning filters that are representative of the domain of the data.


title: Dynamic fluctuations coincide with periods of high and low modularity in resting-state functional brain networks

abstract: We investigate the relationship of resting-state fMRI functional connectivity estimated over long periods of time with time-varying functional connectivity estimated over shorter time intervals. We show that using Pearson's correlation to estimate functional connectivity implies that the range of fluctuations of functional connections over short time scales is subject to statistical constraints imposed by their connectivity strength over longer scales. We present a method for estimating time-varying functional connectivity that is designed to mitigate this issue and allows us to identify episodes where functional connections are unexpectedly strong or weak. We apply this method to data recorded from $N=80$ participants, and show that the number of unexpectedly strong/weak connections fluctuates over time, and that these variations coincide with intermittent periods of high and low modularity in time-varying functional connectivity. We also find that during periods of relative quiescence regions associated with default mode network tend to join communities with attentional, control, and primary sensory systems. In contrast, during periods where many connections are unexpectedly strong/weak, default mode regions dissociate and form distinct modules. Finally, we go on to show that, while all functional connections can at times manifest stronger (more positively correlated) or weaker (more negatively correlated) than expected, a small number of connections, mostly within the visual and somatomotor networks, do so a disproportional number of times. Our statistical approach allows the detection of functional connections that fluctuate more or less than expected based on their long-time averages and may be of use in future studies characterizing the spatio-temporal patterns of time-varying functional connectivity


title: Evolution of public cooperation in a monitored society with implicated punishment and within-group enforcement

abstract: Monitoring with implicated punishment is common in human societies to avert freeriding on common goods. But is it effective in promoting public cooperation? We show that the introduction of monitoring and implicated punishment is indeed effective, as it transforms the public goods game to a coordination game, thus rendering cooperation viable in infinite and finite well-mixed populations. We also show that the addition of within-group enforcement further promotes the evolution of public cooperation. However, although the group size in this context has nonlinear effects on collective action, an intermediate group size is least conductive to cooperative behaviour. This contradicts recent field observations, where an intermediate group size was declared optimal with the conjecture that group-size effects and within-group enforcement are responsible. Our theoretical research thus clarifies key aspects of monitoring with implicated punishment in human societies, and additionally, it reveals fundamental group-size effects that facilitate prosocial collective action.


title: GeMs/GSAOI observations of La Serena 94: an old and far open cluster inside the solar circle

abstract: Physical properties were derived for the candidate open cluster La Serena 94, recently unveiled by the VVV collaboration. Thanks to the exquisite angular resolution provided by GeMS/GSAOI, we could characterize this system in detail, for the first time, with deep photometry in JHK$_{s}$ - bands. Decontaminated JHK$_{s}$ diagrams reach about 5 mag below the cluster turnoff in H. The locus of red clump giants in the colour - colour diagram, together with an extinction law, was used to obtain an average extinction of $A_V =14.18 \pm 0.71$. The same stars were considered as standard - candles to derive the cluster distance, $8.5 \pm 1.0$ kpc. Isochrones were matched to the cluster colour - magnitude diagrams to determine its age, $\log{t(yr)}=9.12\pm 0.06$, and metallicity, $Z=0.02\pm0.01$. A core radius of $r_{c}=0.51\pm 0.04$ pc was found by fitting King models to the radial density profile. By adding up the visible stellar mass to an extrapolated mass function, the cluster mass was estimated as $M=(2.65\pm0.57) \times 10^3$ M$_{\odot}$, consistent with an integrated magnitude of $M_{K}=-5.82\pm0.16$ and a tidal radius of $r_{t}=17.2\pm2.1$ pc. The overall characteristics of La Serena 94 confirm that it is an old open cluster located in the Crux spiral arm towards the fourth Galactic quadrant and distant $7.30\pm 0.49$ kpc from the Galactic centre. The cluster distorted structure, mass segregation and age indicate that it is a dynamically evolved stellar system.


title: OGLE Atlas of Classical Novae. II. Magellanic Clouds

abstract: The population of classical novae in the Magellanic Clouds was poorly known because of a lack of systematic studies. There were some suggestions that nova rates per unit mass in the Magellanic Clouds were higher than in any other galaxy. Here, we present an analysis of data collected over 16 years by the OGLE survey with the aim of characterizing the nova population in the Clouds. We found 20 eruptions of novae, half of which are new discoveries. We robustly measure nova rates of $2.4 \pm 0.8$ yr$^{-1}$ (LMC) and $0.9 \pm 0.4$ yr$^{-1}$ (SMC) and confirm that the K-band luminosity-specific nova rates in both Clouds are 2-3 times higher than in other galaxies. This can be explained by the star formation history in the Magellanic Clouds, specifically the re-ignition of the star formation rate a few Gyr ago. We also present the discovery of the intriguing system OGLE-MBR133.25.1160 which mimics recurrent nova eruptions.


title: Deep factorisation of the stable process II; potentials and applications

abstract: Here we propose a different perspective of the deep factorisation in Kyprianou (2015) based on determining potentials. Indeed, we factorise the inverse of the MAP-exponent associated to a stable process via the Lamperti-Kiu transform. Here our factorisation is completely independent from the derivation in Kyprianou (2015) , moreover there is no clear way to invert the factors in Kyprianou (2015) to derive our results. Our method gives direct access to the potential densities of the ascending and descending ladder MAP of the Lamperti-stable MAP in closed form. In the spirit of the interplay between the classical Wiener-Hopf factorisation and fluctuation theory of the underlying Levy process, our analysis will produce a collection of of new results for stable processes. We give an identity for the point of closest reach to the origin for a stable process with index $\alpha\in (0,1)$ as well as and identity for the point of furthest reach before absorption at the origin for a stable process with index $\alpha\in (1,2)$. Moreover, we show how the deep factorisation allows us to compute explicitly the stationary distribution of stable processes multiplicatively reflected in such a way that it remains in the strip [-1,1].


title: Non-Standard Interactions in propagation at the Deep Underground Neutrino Experiment

abstract: We study the sensitivity of current and future long-baseline neutrino oscillation experiments to the effects of dimension six operators affecting neutrino propagation through Earth, commonly referred to as Non-Standard Interactions (NSI). All relevant parameters entering the oscillation probabilities (standard and non-standard) are considered at once, in order to take into account possible cancellations and degeneracies between them. We find that the Deep Underground Neutrino Experiment will significantly improve over current constraints for most NSI parameters. Most notably, it will be able to rule out the so-called LMA-dark solution, still compatible with current oscillation data, and will be sensitive to off-diagonal NSI parameters at the level of $\varepsilon \sim \mathcal{O}(0.05 - 0.5)$. We also identify two degeneracies among standard and non-standard parameters, which could be partially resolved by combining T2HK and DUNE data.


title: Resurgence in Extended Hydrodynamics

abstract: It has recently been understood that the hydrodynamic series generated by the M\"uller-Israel-Stewart theory is divergent, and that this large order behaviour is consistent with the theory of resurgence. Furthermore, it was observed, that the physical origin of this is the presence of a purely damped nonhydrodynamic mode. It is very interesting to ask whether this picture persists in cases where the spectrum of nonhydrodynamic modes is richer. We take the first step in this direction by considering the simplest hydrodynamic theory which, instead of the purely damped mode, contains a pair of nonhydrodynamic modes of complex conjugate frequencies. This mimics the pattern of black brane quasinormal modes which appear on the gravity side of the AdS/CFT description of \symm\ plasma. We find that the resulting hydrodynamic series is divergent in a way consistent with resurgence and precisely encodes information about the nonhydrodynamic modes of the theory.


title: FRIST - Flipping and Rotation Invariant Sparsifying Transform Learning and Applications

abstract: Features based on sparse representation, especially using the synthesis dictionary model, have been heavily exploited in signal processing and computer vision. However, synthesis dictionary learning typically involves NP-hard sparse coding and expensive learning steps. Recently, sparsifying transform learning received interest for its cheap computation and its optimal updates in the alternating algorithms. In this work, we develop a methodology for learning Flipping and Rotation Invariant Sparsifying Transforms, dubbed FRIST, to better represent natural images that contain textures with various geometrical directions. The proposed alternating FRIST learning algorithm involves efficient optimal updates. We provide a convergence guarantee, and demonstrate the empirical convergence behavior of the proposed FRIST learning approach. Preliminary experiments show the promising performance of FRIST learning for sparse image representation, segmentation, denoising, robust inpainting, and compressed sensing-based magnetic resonance image reconstruction.


title: On pro-isomorphic zeta functions of $D^*$-groups of even Hirsch length

abstract: The pro-isomorphic zeta function of a finitely generated nilpotent group is a Dirichlet generating series that enumerates all finite-index subgroups whose profinite completion is isomorphic to that of the ambient group. We study the pro-isomorphic zeta functions of $\mathbb{Q}$-indecomposable $D^*$-groups of even Hirsch length. These groups are building blocks of finitely generated class-two nilpotent groups with rank-two centre, up to commensurability. Due to a classification by Grunewald and Segal, they are parameterised by primary polynomials whose companion matrices define commutator relations for an explicit presentation. For Grunewald-Segal representatives of even Hirsch length of type $f(t)=t^m$, we give a complete description of the algebraic automorphism groups of associated Lie lattices. Utilising the automorphism groups, we determine the local pro-isomorphic zeta functions of groups associated to $t^2$ and $t^3$. In both cases, the local zeta functions are uniform in the prime $p$ and satisfy functional equations. The functional equations for these groups, not predicted by the currently available theory, prompt us to formulate a conjecture which prescribes, in particular, information about the symmetry factor appearing in local functional equations for pro-isomorphic zeta functions of nilpotent groups. Our description of the local zeta functions also yields information about the analytic properties of the corresponding global pro-isomorphic zeta functions. Some of our results for the $D^*$-groups associated to $t^2$ and $t^3$ generalise to two infinite families of class-two nilpotent groups that result naturally from the initial groups via `base extensions'.


title: Order-Embeddings of Images and Language

abstract: Hypernymy, textual entailment, and image captioning can be seen as special cases of a single visual-semantic hierarchy over words, sentences, and images. In this paper we advocate for explicitly modeling the partial order structure of this hierarchy. Towards this goal, we introduce a general method for learning ordered representations, and show how it can be applied to a variety of tasks involving images and language. We show that the resulting representations improve performance over current approaches for hypernym prediction and image-caption retrieval.


title: Efficient inference in occlusion-aware generative models of images

abstract: We present a generative model of images based on layering, in which image layers are individually generated, then composited from front to back. We are thus able to factor the appearance of an image into the appearance of individual objects within the image --- and additionally for each individual object, we can factor content from pose. Unlike prior work on layered models, we learn a shape prior for each object/layer, allowing the model to tease out which object is in front by looking for a consistent shape, without needing access to motion cues or any labeled data. We show that ordinary stochastic gradient variational bayes (SGVB), which optimizes our fully differentiable lower-bound on the log-likelihood, is sufficient to learn an interpretable representation of images. Finally we present experiments demonstrating the effectiveness of the model for inferring foreground and background objects in images.


title: Synchronization of two memristive coupled van der Pol oscillators

abstract: The objective of this paper is to explore the possibility to couple two van der Pol (vdP) oscillators via a resistance-capacitance (RC) network comprising a Ag-TiOx-Al memristive device. The coupling was mediated by connecting the gate terminals of two programmable unijunction transistors (PUTs) through the network. In the high resistance state (HRS) the memresistance was in the order of MOhm leading to two independent selfsustained oscillators characterized by the different frequencies f1 and f2 and no phase relation between the oscillations. After a few cycles and in dependency of the mediated pulse amplitude the memristive device switched to the low resistance state (LRS) and a frequency adaptation and phase locking was observed. The experimental results are underlined by theoretically considering a system of two coupled vdP equations. The presented neuromorphic circuitry conveys two essentials principle of interacting neuronal ensembles: synchronization and memory. The experiment may path the way to larger neuromorphic networks in which the coupling parameters can vary in time and strength and are realized by memristive devices.


title: Spatially localized solutions of the Hammerstein equation with sigmoid type of nonlinearity

abstract: We study the existence of fixed points to a parameterized Hammertstain operator $\cH_\beta,$ $\beta\in (0,\infty],$ with sigmoid type of nonlinearity. The parameter $\beta<\infty$ indicates the steepness of the slope of a nonlinear smooth sigmoid function and the limit case $\beta=\infty$ corresponds to a discontinuous unit step function. We prove that spatially localized solutions to the fixed point problem for large $\beta$ exist and can be approximated by the fixed points of $\cH_\infty.$ These results are of a high importance in biological applications where one often approximates the smooth sigmoid by discontinuous unit step function. Moreover, in order to achieve even better approximation than a solution of the limit problem, we employ the iterative method that has several advantages compared to other existing methods. For example, this method can be used to construct non-isolated homoclinic orbit of a Hamiltionian system of equations. We illustrate the results and advantages of the numerical method for stationary versions of the FitzHugh-Nagumo reaction-diffusion equation and a neural field model.


title: Spirality: A Novel Way to Measure Spiral Arm Pitch Angle

abstract: We present the MATLAB code Spirality, a novel method for measuring spiral arm pitch angles by fitting galaxy images to spiral templates of known pitch. Computation time is typically on the order of 2 minutes per galaxy, assuming at least 8 GB of working memory. We tested the code using 117 synthetic spiral images with known pitches, varying both the spiral properties and the input parameters. The code yielded correct results for all synthetic spirals with galaxy-like properties. We also compared the code's results to two-dimensional Fast Fourier Transform (2DFFT) measurements for the sample of nearby galaxies defined by DMS PPak. Spirality's error bars overlapped 2DFFT's error bars for 26 of the 30 galaxies. The two methods' agreement correlates strongly with galaxy radius in pixels and also with i-band magnitude, but not with redshift, a result that is consistent with at least some galaxies' spiral structure being fully formed by z=1.2, beyond which there are few galaxies in our sample. The Spirality code package also includes GenSpiral, which produces FITS images of synthetic spirals, and SpiralArmCount, which uses a one-dimensional Fast Fourier Transform to count the spiral arms of a galaxy after its pitch is determined. The code package is freely available online; see Comments for URL.


title: Site-resolved imaging of a fermionic Mott insulator

abstract: The complexity of quantum many-body systems originates from the interplay of strong interactions, quantum statistics, and the large number of quantum-mechanical degrees of freedom. Probing these systems on a microscopic level with single-site resolution offers important insights. Here we report site-resolved imaging of two-component fermionic Mott insulators, metals, and band insulators using ultracold atoms in a square lattice. For strong repulsive interactions we observe two-dimensional Mott insulators containing over 400 atoms. For intermediate interactions, we observe a coexistence of phases. From comparison to theory we find trap-averaged entropies per particle of $1.0\,k_{\mathrm{B}}$. In the band-insulator we find local entropies as low as $0.5\,k_{\mathrm{B}}$. Access to local observables will aid the understanding of fermionic many-body systems in regimes inaccessible by modern theoretical methods.


title: Quantum phases of interacting electrons in three-dimensional dirty Dirac semimetals

abstract: We theoretically study the stability of three dimensional Dirac semimetals against short-range electron-electron interaction and quenched time-reversal symmetric disorder (but excluding mass disorder). First we focus on the clean interacting and the noninteracting dirty Dirac semimetal separately, and show that they support two distinct quantum critical points. Using renormalization group techniques, we find that while interaction driven quantum critical points are \emph{Gaussian} (mean-field) in nature, describing quantum phase transitions into various broken symmetry phases, the ones controlled by disorder are \emph{non-Gaussian}, capturing the transition to a metallic phase. We classify such diffusive quantum critical points based on the transformation of disorder vertices under a \emph{continuous} chiral rotation. Our wek coupling renormalization group analysis suggests that two distinct quantum critical points are stable in an interacting dirty Dirac semimetal (with chiral symmetric randomness), and a multicritical point (at finite interaction and disorder) results from their interplay. By contrast, the chiral symmetry breaking disorder driven critical point is unstable against weak interactions. Effects of weak disorder on the ordering tendencies in Dirac semimetal are analyzed. The clean interacting critical points, however, satisfy the \emph{Harris criterion}, and are therefore expected to be unstable against bond disorder. Although our weak coupling analysis is inadequate to establish the ultimate stability of these fixed points in the strong coupling regime (when both interaction and disorder are strong), they can still govern crossover behaviors in Dirac semimetals over a large length scale, when either interaction or randomness is sufficiently weak.


title: Accretion disc dynamo activity in local simulations spanning weak-to-strong net vertical magnetic flux regimes

abstract: Strongly magnetized accretion discs around black holes have attractive features that may explain enigmatic aspects of X-ray binary behaviour. The structure and evolution of these discs are governed by a dynamo-like mechanism, which channels part of the accretion power liberated by the magnetorotational instability (MRI) into an ordered toroidal magnetic field. To study dynamo activity, we performed three-dimensional, stratified, isothermal, ideal magnetohydrodynamic shearing box simulations. The strength of the self-sustained toroidal magnetic field depends on the net vertical magnetic flux, which we vary across almost the entire range over which the MRI is linearly unstable. We quantify disc structure and dynamo properties as a function of the initial ratio of mid-plane gas pressure to vertical magnetic field pressure, $\beta_0^{\rm mid} = p_{\rm gas} / p_B$. For $10^5 \geq \beta_0^{\rm mid} \geq 10$ the effective $\alpha$-viscosity parameter scales as a power-law. Dynamo activity persists up to and including $\beta_0^{\rm mid} = 10^2$, at which point the entire vertical column of the disc is magnetic pressure-dominated. Still stronger fields result in a highly inhomogeneous disc structure, with large density fluctuations. We show that the turbulent steady state $\beta^{\rm mid}$ in our simulations is well-matched by the analytic model of Begelman et al. (2015) describing the creation and buoyant escape of toroidal field, while the vertical structure of the disc can be broadly reproduced using this model. Finally, we discuss the implications of our results for observed properties of X-ray binaries.


title: Imprints of radial migration on the Milky Way's metallicity distribution functions

abstract: Recent analysis of the SDSS-III/APOGEE Data Release 12 stellar catalogue has revealed that the Milky Way's metallicity distribution function (MDF) changes shape as a function of radius, transitioning from being negatively skewed at small Galactocentric radii to positively skewed at large Galactocentric radii. Using a high resolution, N-body+SPH simulation, we show that the changing skewness arises from radial migration - metal-rich stars form in the inner disk and subsequently migrate to the metal-poorer outer disk. These migrated stars represent a large fraction (> 50%) of the stars in the outer disk; they populate the high metallicity tail of the MDFs and are, in general, more metal-rich than the surrounding outer disk gas. The simulation also reproduces another surprising APOGEE result: the spatially invariant high-[alpha/Fe] MDFs. This arises in the simulation from the migration of a population formed within a narrow range of radii (3.2+/-1.2 kpc) and time (8.8+/-0.6 Gyr ago), rather than from spatially extended star formation in a homogeneous medium at early times. These results point toward the crucial role radial migration has played in shaping our Milky Way.


title: The Musca cloud: A 6 pc-long velocity-coherent, sonic filament

abstract: Filaments play a central role in the molecular clouds' evolution, but their internal dynamical properties remain poorly characterized. To further explore the physical state of these structures, we have investigated the kinematic properties of the Musca cloud. We have sampled the main axis of this filamentary cloud in $^{13}$CO and C$^{18}$O (2--1) lines using APEX observations. The different line profiles in Musca shows that this cloud presents a continuous and quiescent velocity field along its $\sim$6.5 pc of length. With an internal gas kinematics dominated by thermal motions (i.e., $\sigma_{NT}/c_s\lesssim1$) and large-scale velocity gradients, these results reveal Musca as the longest velocity-coherent, sonic-like object identified so far in the ISM. The transonic properties of Musca present a clear departure from the predicted supersonic velocity dispersions expected in the Larson's velocity dispersion-size relationship, and constitute the first observational evidence of a filament fully decoupled from the turbulent regime over multi-parsec scales.


title: Neutrino Anarchy and Renormalization Group Evolution

abstract: The observed pattern of neutrino mixing angles is in good agreement with the hypothesis of neutrino anarchy, which posits that Nature has chosen the entries of the leptonic mixing matrix at random. In this paper we investigate how stable this conclusion is under renormalization group effects. Working in the simplest type-I seesaw model and two variants of the inverse seesaw model we study how the statistical distributions of the neutrino mixing parameters evolve between the Grand Unification scale and the electroweak scale. Especially in the inverse seesaw case we find significant distortions: mixing angles tend to be smaller after RG running, and the Dirac CP phase tends to be closer to zero. The p-value describing the compatibility between the observed mixing angles and the anarchy hypothesis increases by 10-20%. This illustrates that RG effects are highly relevant for quantitative studies of the anarchy scenario.


title: PS1-10jh Continues to Follow the Fallback Accretion Rate of a Tidally Disrupted Star

abstract: We present late-time observations of the tidal disruption event candidate PS1-10jh. UV and optical imaging with HST/WFC3 localize the transient to be coincident with the host galaxy nucleus to an accuracy of 0.023 arcsec, corresponding to 66 pc. The UV flux in the F225W filter, measured 3.35 rest-frame years after the peak of the nuclear flare, is consistent with a decline that continues to follow a $t^{-5/3}$ power-law with no spectral evolution. Late epochs of optical spectroscopy obtained with MMT ~ 2 and 4 years after the peak, enable a clean subtraction of the host galaxy from the early spectra, revealing broad helium emission lines on top of a hot continuum, and placing stringent upper limits on the presence of hydrogen line emission. We do not measure Balmer H\delta absorption in the host galaxy strong enough to be indicative of a rare, post-starburst "E+A" galaxy as reported by Arcavi et al. (2014). The light curve of PS1-10jh over a baseline of 3.5 yr is best modeled by fallback accretion of a tidally disrupted star. Its strong broad helium emission relative to hydrogen (He II \lambda 4686/H\alpha > 5) could be indicative of either the hydrogen-poor chemical composition of the disrupted star, or certain conditions in the tidal debris of a solar-composition star in the presence of an optically-thick, extended reprocessing envelope.


title: Holographic model for heavy vector meson masses

abstract: The experimentally observed spectra of heavy vector meson radial excitations show a dependence on two different energy parameters. One is associated with the quark mass and the other with the binding energy levels of the quark anti-quark pair. The first is present in the large mass of the first state while the other corresponds to the small mass splittings between radial excitations. In this article we show how to reproduce such a behavior with reasonable precision using a holographic model. In the dual picture, the large energy scale shows up from a bulk mass and the small scale comes from the position of anti-de Sitter (AdS) space where field correlators are calculated. The model determines the masses of four observed S-wave states of charmonium and six S-wave states of bottomonium with , 6.1 % rms error. In consistency with the physical picture, the large energy parameter is flavor dependent, while the small parameter, associated with quark anti-quark interaction is the same for charmonium and bottomonium states.


title: Affine transformation with zero entropy and nilsystems

abstract: In this paper, we study affine transformations on tori, nilmanifolds and compact abelian groups. For these systems, we show that an equivalent condition for zero entropy is the orbit closure of each point has a nice structure. To be precise, the affine systems on those spaces are zero entropy if and only if the orbit closure of each point is isomorphic to an inverse limit of nilsystems.


title: Type II Radiative Seesaw Model of Neutrino Mass with Dark Matter

abstract: We consider a model of neutrino mass with a scalar triplet $(\xi^{++},\xi^+,\xi^0)$ assigned lepton number $L=0$, so that the tree-level Yukawa coupling $\xi^0 \nu_i \nu_j$ is not allowed. It is generated instead through the interaction of $\xi$ and $\nu$ with dark matter and the soft breaking of $L$ to $(-1)^L$. We discuss the phenomenological implications of this model, including $\xi^{++}$ decay and the prognosis of discovering the dark sector at the Large Hadron Collider.


title: "'Formal' vs. 'Empirical' Approaches to Quantum-Classical Reduction"

abstract: I distinguish two types of reduction within the context of quantum-classical relations, which I designate "formal" and "empirical". Formal reduction holds or fails to hold solely by virtue of the mathematical relationship between two theories; it is therefore a two-place, a priori relation between theories. Empirical reduction requires one theory to encompass the range of physical behaviors that are well-modeled in another theory; in a certain sense, it is a three-place, a posteriori relation connecting the theories and the domain of physical reality that both serve to describe. Focusing on the relationship between classical and quantum mechanics, I argue that while certain formal results concerning singular limits as Planck's constant goes to zero have been taken to preclude the possibility of reduction between these theories, such results at most block reduction in the formal sense; little if any reason has been given for thinking that they block reduction in the empirical sense. I then briefly outline a strategy for empirical reduction that is suggested by work on decoherence theory, arguing that this sort of account remains a fully viable route to the empirical reduction of classical to quantum mechanics and is unaffected by such singular limits.


title: Novel considerations about the non-equilibrium regime of the tricritical point in a metamagnetic model: localization and tricritical exponents

abstract: We have investigated the time-dependent regime of a two-dimensional metamagnetic model at its tricritical point via Monte Carlo simulations. First of all, we obtained the temperature and magnetic field corresponding to the tricritical point of the model by using a refinement process based on optimization of the coefficient of determination in the log-log fit of magnetization decay as function of time. With these estimates in hand, we obtained the dynamic tricritical exponents $\theta $ and $z$ and the static tricritical exponents $\nu $ and $\beta $ by using the universal power-law scaling relations for the staggered magnetization and its moments at early stage of the dynamic evolution. Our results at tricritical point confirm that this model belongs to the two-dimensional Blume-Capel model universality class for both static and dynamic behaviors, and also they corroborate the conjecture of Janssen and Oerding for the dynamics of tricritical points.


title: Characterizing graphs of maximum principal ratio

abstract: The principal ratio of a connected graph, denoted $\gamma(G)$, is the ratio of the maximum and minimum entries of its first eigenvector. Cioab\u{a} and Gregory conjectured that the graph on $n$ vertices maximizing $\gamma(G)$ is a kite graph: a complete graph with a pendant path. In this paper we prove their conjecture.


title: Dynamic Adaptive Network Intelligence

abstract: Accurate representational learning of both the explicit and implicit relationships within data is critical to the ability of machines to perform more complex and abstract reasoning tasks. We describe the efficient weakly supervised learning of such inferences by our Dynamic Adaptive Network Intelligence (DANI) model. We report state-of-the-art results for DANI over question answering tasks in the bAbI dataset that have proved difficult for contemporary approaches to learning representation (Weston et al., 2015).


title: Unsupervised Learning of Visual Structure using Predictive Generative Networks

abstract: The ability to predict future states of the environment is a central pillar of intelligence. At its core, effective prediction requires an internal model of the world and an understanding of the rules by which the world changes. Here, we explore the internal models developed by deep neural networks trained using a loss based on predicting future frames in synthetic video sequences, using a CNN-LSTM-deCNN framework. We first show that this architecture can achieve excellent performance in visual sequence prediction tasks, including state-of-the-art performance in a standard 'bouncing balls' dataset (Sutskever et al., 2009). Using a weighted mean-squared error and adversarial loss (Goodfellow et al., 2014), the same architecture successfully extrapolates out-of-the-plane rotations of computer-generated faces. Furthermore, despite being trained end-to-end to predict only pixel-level information, our Predictive Generative Networks learn a representation of the latent structure of the underlying three-dimensional objects themselves. Importantly, we find that this representation is naturally tolerant to object transformations, and generalizes well to new tasks, such as classification of static images. Similar models trained solely with a reconstruction loss fail to generalize as effectively. We argue that prediction can serve as a powerful unsupervised loss for learning rich internal representations of high-level object features.


title: Manifold Regularized Deep Neural Networks using Adversarial Examples

abstract: Learning meaningful representations using deep neural networks involves designing efficient training schemes and well-structured networks. Currently, the method of stochastic gradient descent that has a momentum with dropout is one of the most popular training protocols. Based on that, more advanced methods (i.e., Maxout and Batch Normalization) have been proposed in recent years, but most still suffer from performance degradation caused by small perturbations, also known as adversarial examples. To address this issue, we propose manifold regularized networks (MRnet) that utilize a novel training objective function that minimizes the difference between multi-layer embedding results of samples and those adversarial. Our experimental results demonstrated that MRnet is more resilient to adversarial examples and helps us to generalize representations on manifolds. Furthermore, combining MRnet and dropout allowed us to achieve competitive classification performances for three well-known benchmarks: MNIST, CIFAR-10, and SVHN.


title: Iterative Refinement of the Approximate Posterior for Directed Belief Networks

abstract: Variational methods that rely on a recognition network to approximate the posterior of directed graphical models offer better inference and learning than previous methods. Recent advances that exploit the capacity and flexibility in this approach have expanded what kinds of models can be trained. However, as a proposal for the posterior, the capacity of the recognition network is limited, which can constrain the representational power of the generative model and increase the variance of Monte Carlo estimates. To address these issues, we introduce an iterative refinement procedure for improving the approximate posterior of the recognition network and show that training with the refined posterior is competitive with state-of-the-art methods. The advantages of refinement are further evident in an increased effective sample size, which implies a lower variance of gradient estimates.


title: Interpretation Neutrality in the Classical Domain of Quantum Theory

abstract: I show explicitly how concerns about wave function collapse and ontology can be decoupled from the bulk of technical analysis necessary to recover localized, approximately Newtonian trajectories from quantum theory. In doing so, I demonstrate that the account of classical behavior provided by decoherence theory can be straightforwardly tailored to give accounts of classical behavior on multiple interpretations of quantum theory, including the Everett, de Broglie-Bohm and GRW interpretations. I further show that this interpretation-neutral, decoherence-based account conforms to a general view of inter-theoretic reduction in physics that I have elaborated elsewhere, which differs from the oversimplified picture that treats reduction as a matter of simply taking limits. This interpretation-neutral account rests on a general three-pronged strategy for reduction between quantum and classical theories that combines decoherence, an appropriate form of Ehrenfest's Theorem, and a decoherence-compatible mechanism for collapse. It also incorporates a novel argument as to why branch-relative trajectories should be approximately Newtonian, which is based on a little-discussed extension of Ehrenfest's Theorem to open systems, rather than on the more commonly cited but less germane closed-systems version. In the Conclusion, I briefly suggest how the strategy for quantum-classical reduction described here might be extended to reduction between other classical and quantum theories, including classical and quantum field theory and classical and quantum gravity.


title: Decentralization of a Machine: Some Definitions

abstract: We define some notions of the decentralization of a deterministic input-output machine. This opens the possibility for introducing game-theoretic elements -- such as strategic players -- inside the machine, as part of its design.


title: A Unified Gradient Regularization Family for Adversarial Examples

abstract: Adversarial examples are augmented data points generated by imperceptible perturbation of input samples. They have recently drawn much attention with the machine learning and data mining community. Being difficult to distinguish from real examples, such adversarial examples could change the prediction of many of the best learning models including the state-of-the-art deep learning models. Recent attempts have been made to build robust models that take into account adversarial examples. However, these methods can either lead to performance drops or lack mathematical motivations. In this paper, we propose a unified framework to build robust machine learning models against adversarial examples. More specifically, using the unified framework, we develop a family of gradient regularization methods that effectively penalize the gradient of loss function w.r.t. inputs. Our proposed framework is appealing in that it offers a unified view to deal with adversarial examples. It incorporates another recently-proposed perturbation based approach as a special case. In addition, we present some visual effects that reveals semantic meaning in those perturbations, and thus support our regularization method and provide another explanation for generalizability of adversarial examples. By applying this technique to Maxout networks, we conduct a series of experiments and achieve encouraging results on two benchmark datasets. In particular,we attain the best accuracy on MNIST data (without data augmentation) and competitive performance on CIFAR-10 data.


title: The effect of gait on swimming in viscoelastic fluids

abstract: In this paper, we give formulas for the swimming of simplified two-dimensional bodies in complex fluids using the reciprocal theorem. By way of these formulas we calculate the swimming velocity due to small-amplitude deformations on the simplest of these bodies, a two-dimensional sheet, to explore general conditions on the swimming gait under which the sheet may move faster, or slower, in a viscoelastic fluid compared to a Newtonian fluid. We show that in general, for small amplitude deformations, a speed increase can only be realized by multiple deformation modes in contrast to slip flows. Additionally, we show that a change in swimming speed is directly due to a change in thrust generated by the swimmer.


title: The Surface Density Profile of the Galactic Disk from the Terminal Velocity Curve

abstract: The mass distribution of the Galactic disk is constructed from the terminal velocity curve and the mass discrepancy-acceleration relation. Mass models numerically quantifying the detailed surface density profiles are tabulated. For $R_0 = 8$ kpc, the models have stellar mass $5 < M_* < 6 \times 10^{10}$ M$_{\odot}$, scale length $2.0 \le R_d \le 2.9$ kpc, LSR circular velocity $222 \le \Theta_0 \le 233$ km s$^{-1}$, and solar circle stellar surface density $34 \le \Sigma_d(R_0) \le 61$ M$_{\odot}$ pc$^{-2}$. The present inter-arm location of the solar neighborhood may have a somewhat lower stellar surface density than average for the solar circle. The Milky Way appears to be a normal spiral galaxy that obeys scaling relations like the Tully-Fisher relation, the size-mass relation, and the disk maximality-surface brightness relation. The stellar disk is maximal, and the spiral arms are massive. The bumps and wiggles in the terminal velocity curve correspond to known spiral features (e.g., the Centaurus Arm is a $\sim 50\%$ overdensity). The rotation curve switches between positive and negative over scales of hundreds of parsecs. The rms amplitude $\langle$$|$$dV/dR$$|^2$$\rangle$$^{1/2} \approx 14$ km s$^{-1}$ kpc$^{-1}$, implying that commonly neglected terms in the Jeans equations may be non-negligible. The spherically averaged local dark matter density is $\rho_{0,DM} \approx 0.009$ M$_{\odot}$ pc$^{-3}$ (0.3 GeV cm$^{-3}$). Adiabatic compression of the dark matter halo may help reconcile the Milky Way with the $c$-$V_{200}$ relation expected in $\Lambda$CDM while also helping to mitigate the too big to fail problem, but it remains difficult to reconcile the inner bulge/bar dominated region with a cuspy halo. We note that NGC 3521 is a near twin to the Milky Way, having a similar luminosity, scale length, and rotation curve.


title: sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings

abstract: Neural word representations have proven useful in Natural Language Processing (NLP) tasks due to their ability to efficiently model complex semantic and syntactic word relationships. However, most techniques model only one representation per word, despite the fact that a single word can have multiple meanings or "senses". Some techniques model words by using multiple vectors that are clustered based on context. However, recent neural approaches rarely focus on the application to a consuming NLP algorithm. Furthermore, the training process of recent word-sense models is expensive relative to single-sense embedding processes. This paper presents a novel approach which addresses these concerns by modeling multiple embeddings for each word based on supervised disambiguation, which provides a fast and accurate way for a consuming NLP model to select a sense-disambiguated embedding. We demonstrate that these embeddings can disambiguate both contrastive senses such as nominal and verbal senses as well as nuanced senses such as sarcasm. We further evaluate Part-of-Speech disambiguated embeddings on neural dependency parsing, yielding a greater than 8% average error reduction in unlabeled attachment scores across 6 languages.


title: Griffiths effects and slow dynamics in nearly many-body localized systems

abstract: The low-frequency response of systems near a many-body localization transition can be dominated by rare regions that are locally critical or "in the other phase". It is known that, in one dimension, these rare regions can cause the d.c. conductivity and diffusion constant to vanish even inside the delocalized thermal phase. Here, we present a general analysis of such Griffiths effects in the thermal phase near the many-body localization transition: we consider both one-dimensional and higher-dimensional systems, subject to quenched randomness, and discuss both linear response (including the frequency- and wavevector-dependent conductivity) and more general dynamics. In all the regimes we consider, we identify observables that are dominated by rare-region effects. In some cases (one-dimensional systems and Floquet systems with no extensive conserved quantities), essentially all long-time local observables are dominated by rare-region effects; in others, generic observables are instead dominated by hydrodynamic long-time tails throughout the thermal phase, and one must look at specific probes, such as spin echo, to see Griffiths behavior.


title: Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks

abstract: In this paper we present a method for learning a discriminative classifier from unlabeled or partially labeled data. Our approach is based on an objective function that trades-off mutual information between observed examples and their predicted categorical class distribution, against robustness of the classifier to an adversarial generative model. The resulting algorithm can either be interpreted as a natural generalization of the generative adversarial networks (GAN) framework or as an extension of the regularized information maximization (RIM) framework to robust classification against an optimal adversary. We empirically evaluate our method - which we dub categorical generative adversarial networks (or CatGAN) - on synthetic data as well as on challenging image classification tasks, demonstrating the robustness of the learned classifiers. We further qualitatively assess the fidelity of samples generated by the adversarial generator that is learned alongside the discriminative classifier, and identify links between the CatGAN objective and discriminative clustering algorithms (such as RIM).


title: Order Matters: Sequence to sequence for sets

abstract: Sequences have become first class citizens in supervised learning thanks to the resurgence of recurrent neural networks. Many complex tasks that require mapping from or to a sequence of observations can now be formulated with the sequence-to-sequence (seq2seq) framework which employs the chain rule to efficiently represent the joint probability of sequences. In many cases, however, variable sized inputs and/or outputs might not be naturally expressed as sequences. For instance, it is not clear how to input a set of numbers into a model where the task is to sort them; similarly, we do not know how to organize outputs when they correspond to random variables and the task is to model their unknown joint probability. In this paper, we first show using various examples that the order in which we organize input and/or output data matters significantly when learning an underlying model. We then discuss an extension of the seq2seq framework that goes beyond sequences and handles input sets in a principled way. In addition, we propose a loss which, by searching over possible orders during training, deals with the lack of structure of output sets. We show empirical evidence of our claims regarding ordering, and on the modifications to the seq2seq framework on benchmark language modeling and parsing tasks, as well as two artificial tasks -- sorting numbers and estimating the joint probability of unknown graphical models.


title: Neural Random-Access Machines

abstract: In this paper, we propose and investigate a new neural network architecture called Neural Random Access Machine. It can manipulate and dereference pointers to an external variable-size random-access memory. The model is trained from pure input-output examples using backpropagation. We evaluate the new model on a number of simple algorithmic tasks whose solutions require pointer manipulation and dereferencing. Our results show that the proposed model can learn to solve algorithmic tasks of such type and is capable of operating on simple data structures like linked-lists and binary trees. For easier tasks, the learned solutions generalize to sequences of arbitrary length. Moreover, memory access during inference can be done in a constant time under some assumptions.


title: Fixed Point Quantization of Deep Convolutional Networks

abstract: In recent years increasingly complex architectures for deep convolution networks (DCNs) have been proposed to boost the performance on image recognition tasks. However, the gains in performance have come at a cost of substantial increase in computation and model storage resources. Fixed point implementation of DCNs has the potential to alleviate some of these complexities and facilitate potential deployment on embedded hardware. In this paper, we propose a quantizer design for fixed point implementation of DCNs. We formulate and solve an optimization problem to identify optimal fixed point bit-width allocation across DCN layers. Our experiments show that in comparison to equal bit-width settings, the fixed point DCNs with optimized bit width allocation offer >20% reduction in the model size without any loss in accuracy on CIFAR-10 benchmark. We also demonstrate that fine-tuning can further enhance the accuracy of fixed point DCNs beyond that of the original floating point model. In doing so, we report a new state-of-the-art fixed point performance of 6.78% error-rate on CIFAR-10 benchmark.


title: Geodesics of learned representations

abstract: We develop a new method for visualizing and refining the invariances of learned representations. Specifically, we test for a general form of invariance, linearization, in which the action of a transformation is confined to a low-dimensional subspace. Given two reference images (typically, differing by some transformation), we synthesize a sequence of images lying on a path between them that is of minimal length in the space of the representation (a "representational geodesic"). If the transformation relating the two reference images is linearized by the representation, this sequence should follow the gradual evolution of this transformation. We use this method to assess the invariance properties of a state-of-the-art image classification network and find that geodesics generated for image pairs differing by translation, rotation, and dilation do not evolve according to their associated transformations. Our method also suggests a remedy for these failures, and following this prescription, we show that the modified representation is able to linearize a variety of geometric image transformations.


title: Analysis of a heterogeneous kinetic model for traffic flow

abstract: In this work we extend a recent kinetic traffic model to the case of more than one class of vehicles, each of which is characterized by few different microscopic features. We consider a Boltzmann-like framework with only binary interactions, which take place among vehicles belonging to the various classes. Our approach differs from the multi-population kinetic model based on a lattice of speeds because here we assume continuous velocity spaces and we introduce a parameter describing the physical velocity jump performed by a vehicle that increases its speed after an interaction. The model is discretized in order to investigate numerically the structure of the resulting fundamental diagrams and the system of equations is analyzed by studying well posedness. Moreover, we compute the equilibria of the discretized model and we show that the exact asymptotic kinetic distributions can be obtained with a small number of velocities in the grid. Finally, we introduce a new probability law in order to attenuate the sharp capacity drop occurring in the diagrams of traffic.


title: Multilingual Relation Extraction using Compositional Universal Schema

abstract: Universal schema builds a knowledge base (KB) of entities and relations by jointly embedding all relation types from input KBs as well as textual patterns expressing relations from raw text. In most previous applications of universal schema, each textual pattern is represented as a single embedding, preventing generalization to unseen patterns. Recent work employs a neural network to capture patterns' compositional semantics, providing generalization to all possible input text. In response, this paper introduces significant further improvements to the coverage and flexibility of universal schema relation extraction: predictions for entities unseen in training and multilingual transfer learning to domains with no annotation. We evaluate our model through extensive experiments on the English and Spanish TAC KBP benchmark, outperforming the top system from TAC 2013 slot-filling using no handwritten patterns or additional annotation. We also consider a multilingual setting in which English training data entities overlap with the seed KB, but Spanish text does not. Despite having no annotation for Spanish data, we train an accurate predictor, with additional improvements obtained by tying word embeddings across languages. Furthermore, we find that multilingual training improves English relation extraction accuracy. Our approach is thus suited to broad-coverage automated knowledge base construction in a variety of languages and domains.


title: Compressing Word Embeddings

abstract: Recent methods for learning vector space representations of words have succeeded in capturing fine-grained semantic and syntactic regularities using vector arithmetic. However, these vector space representations (created through large-scale text analysis) are typically stored verbatim, since their internal structure is opaque. Using word-analogy tests to monitor the level of detail stored in compressed re-representations of the same vector space, the trade-offs between the reduction in memory usage and expressiveness are investigated. A simple scheme is outlined that can reduce the memory footprint of a state-of-the-art embedding by a factor of 10, with only minimal impact on performance. Then, using the same `bit budget', a binary (approximate) factorisation of the same space is also explored, with the aim of creating an equivalent representation with better interpretability.


title: Measurement of W-boson production in p-Pb collisions at the LHC with ALICE

abstract: ALICE (A Large Ion Collider Experiment) is designed and optimized to study ultra-relativistic heavy-ion collisions, in which a hot and dense strongly-interacting medium is created. W bosons are produced in hard scattering processes occurring at the early stage of the collision and, since they are not affected by the strong interaction, they can be used as a benchmark for medium-induced effects. In proton-nucleus collisions the production of W bosons can be used to study the modification of parton distribution functions in the nucleus and to test the validity of binary collision scaling. The latter is studied by measuring the yield of W bosons in different intervals of event activity. In ALICE, the production of W bosons is measured via the contribution of their muonic decays to the inclusive $p_{\rm T}$-differential muon yield reconstructed with the muon spectro-meter at forward ($2.03 < \mathit{y}^{\mu}_{cms} < 3.53$) and backward rapidity ($-4.46< \mathit{y}^{\mu}_{cms} <-2.96$). The recent results from p--Pb collisions at a centre-of-mass energy of $\sqrt {s_{\rm NN}}$ = 5.02 TeV are presented and the measured cross sections are compared to perturbative Quantum Chromodynamics calculations at next-to-leading order.


title: Admissible Region of Large-Scale Uncertain Wind Generation Considering Small-Signal Stability of Power System

abstract: The increasing integration of wind generation has brought great challenges to small-signal stability analysis of bulk power systems, since the volatility and uncertainty nature of wind generation may considerably affect equilibriums of the systems. In this regard, this paper develops a conceptual framework to depict the influence of uncertain wind power injections (WPIs) on small-signal stability of bulk power systems. To do this, a new concept, the admissible region of uncertain wind generation considering small-signal stability (SSAR) is introduced to geometrically measure how much uncertain wind generation can be accommodated by a bulk power system without breaking its small-signal stability. As a generalization of the traditional concept of the small-signal stability region (SSSR), SSAR is derived by extending the SSSR to a higher-dimensional injection space that incorporates both the conventional nodal generation injections and the WPIs, and then mapping it onto the lower-dimensional WPI space. Case studies on the modified New England 39-bus system with multiple wind farms illustrate the SSAR concept and its potential applications.


abstract: Minimum disparity estimation in controlled branching processes is dealt with by assuming that the offspring law belongs to a general parametric family. Under some regularity conditions it is proved that the minimum disparity estimators proposed -based on the nonparametric maximum likelihood estimator of the offspring law when the entire family tree is observed- are consistent and asymptotic normally distributed. Moreover, it is discussed the robustness of the estimators proposed. Through a simulated example, focussing on the minimum Hellinger and negative exponential disparity estimators, it is shown that both are robust against outliers, being the negative exponential one also robust against inliers.


title: Formation probabilities and Shannon information and their time evolution after quantum quench in transverse-field XY-chain

abstract: We first provide a formula to calculate the probability of occurrence of different configurations (formation probabilities) in a generic free fermion system. We then study the scaling of these probabilities with respect to the size in the case of critical transverse-field XY-chain in the $\sigma^z$ bases. In the case of the transverse field Ising model, we show that all the "crystal" configurations follow the formulas expected from conformal field theory (CFT). In the case of critical $XX$ chain, we show that the only configurations that follow the formulas of the CFT are the ones which respect the filling factor of the system. By repeating all the calculations in the presence of open and periodic boundary conditions we find further support to our classification of different configurations. Using the developed technique, we also study Shannon information of a subregion in our system. In this respect we distinguish particular configurations that are more important in the study of the scaling limit of the Shannon information of the subsystem. Finally, we study the evolution of formation probabilities, Shannon information and Shannon mutual information after a quantum quench in free fermion system. In particular, for the intial state considered in this paper, we demonstrate that the Shannon information after quantum quench first increases with the time and then saturates at time $t^*=\frac{l}{2}$, where $l$ is the size of the subsystem.


title: The DEdicated MONitor of EXotransits (DEMONEX): Seven Transits of XO-4b

abstract: The DEdicated MONitor of EXotransits (DEMONEX) was a 20 inch robotic and automated telescope to monitor bright stars hosting transiting exoplanets to discover new planets and improve constraints on the properties of known transiting planetary systems. We present results for the misaligned hot Jupiter XO-4b containing 7 new transits from the DEMONEX telescope, including 3 full and 4 partial transits. We combine these data with archival light curves and archival radial velocity measurements to derive the host star mass $M_{*}=1.293_{-0.029}^{+0.030} M_\odot$ and radius $R_{*}=1.554_{-0.030}^{+0.042} R_\odot$ as well as the planet mass $M_{P}=1.615_{-0.099}^{+0.10} M_{\rm J}$ and radius $R_{P}=1.317_{-0.029}^{+0.040} R_{\rm J}$ and a refined ephemeris of $P=4.1250687\pm0.0000024$ days and $T_{0}=2454758.18978\pm0.00024 \rm {BJD_{TDB}}$. We include archival Rossiter-McLaughlin measurements of XO-4 to infer the stellar spin-planetary orbit alignment $\lambda=-40.0_{-7.5}^{+8.8}$ degrees. We test the effects of including various detrend parameters, theoretical and empirical mass-radius relations, and Rossiter-McLaughlin models. We infer that detrending against CCD position and time or airmass can improve data quality, but can have significant effects on the inferred values of many parameters --- most significantly $R_{P}/R_{*}$ and the observed central transit times $T_{C}$. In the case of $R_{P}/R_{*}$ we find that the systematic uncertainty due to detrending can be three times that of the quoted statistical uncertainties. The choice of mass-radius relation has little effect on our inferred values of the system parameters. The choice of Rossiter-McLaughlin models can have significant effects of the inferred values of $v\sin{I_{*}}$ and the stellar spin-planet orbit angle $\lambda$.


title: Measuring High-Precision Astrometry with the Infrared Array Camera on the Spitzer Space Telescope

abstract: The Infrared Array Camera (IRAC) on the Spitzer Space Telescope currently offers the greatest potential for high-precision astrometry of faint mid-IR sources across arcminute-scale fields, which would be especially valuable for measuring parallaxes of cold brown dwarfs in the solar neighborhood and proper motions of obscured members of nearby star-forming regions. To more fully realize IRAC's astrometric capabilities, we have sought to minimize the largest sources of uncertainty in astrometry with its 3.6 and 4.5 $\mu$m bands. By comparing different routines that estimate stellar positions, we have found that Point Response Function (PRF) fitting with the Spitzer Science Center's Astronomical Point Source Extractor produces both the smallest systematic errors from varying intra-pixel sensitivity and the greatest precision in measurements of positions. In addition, self-calibration has been used to derive new 7$^{\rm th}$ and 8$^{\rm th}$ order distortion corrections for the 3.6 and 4.5 $\mu$m arrays of IRAC, respectively. These corrections are suitable for data throughout the mission of Spitzer when a time-dependent scale factor is applied to the corrections. To illustrate the astrometric accuracy that can be achieved by combining PRF fitting with our new distortion corrections, we have applied them to archival data for a nearby star-forming region, arriving at total astrometric errors of $\sim$20 and 70 mas at signal to noise ratios of 100 and 10, respectively.


title: Bounded tiles in $\mathbb{Q}_p$ are compact open sets

abstract: Any bounded tile of the field $\mathbb{Q}_p$ of $p$-adic numbers is a compact open set up to a zero Haar measure set. In this note, we give a simple and direct proof of this fact.


title: Symmetry-driven atomic rearrangement at a brownmillerite-perovskite interface

abstract: Many of the recent advancements in oxide heterostructures have been attributed to modification of spin, charge, lattice, and orbital order parameters at atomically well-defined interfaces. However, the details on the structural, chemical, and electrostatic evolution of interfaces comprised of materials with different crystallographic symmetries remain to be understood. In this work, we have mapped out the interfacial connectivity of atoms of two dissimilar materials, the perovskite SrTiO3 and the brownmillerite SrCoO2.5, using high resolution scanning transmission electron microscopy and geometric phase analysis. We observed unique symmetry-mismatch driven atomic displacements restricted to only the first few atomic layers, which can critically modify the properties of the system. Provided that SrCoO2.5 is a promising energy material due to its open framework structure, the improved understanding of the interfacial structure on the atomic level can lead to the rational design of novel oxide heterostructures.


title: Denoising Criterion for Variational Auto-Encoding Framework

abstract: Denoising autoencoders (DAE) are trained to reconstruct their clean inputs with noise injected at the input level, while variational autoencoders (VAE) are trained with noise injected in their stochastic hidden layer, with a regularizer that encourages this noise injection. In this paper, we show that injecting noise both in input and in the stochastic hidden layer can be advantageous and we propose a modified variational lower bound as an improved objective function in this setup. When input is corrupted, then the standard VAE lower bound involves marginalizing the encoder conditional distribution over the input noise, which makes the training criterion intractable. Instead, we propose a modified training criterion which corresponds to a tractable bound when input is corrupted. Experimentally, we find that the proposed denoising variational autoencoder (DVAE) yields better average log-likelihood than the VAE and the importance weighted autoencoder on the MNIST and Frey Face datasets.


title: Recurrent Models for Auditory Attention in Multi-Microphone Distance Speech Recognition

abstract: Integration of multiple microphone data is one of the key ways to achieve robust speech recognition in noisy environments or when the speaker is located at some distance from the input device. Signal processing techniques such as beamforming are widely used to extract a speech signal of interest from background noise. These techniques, however, are highly dependent on prior spatial information about the microphones and the environment in which the system is being used. In this work, we present a neural attention network that directly combines multi-channel audio to generate phonetic states without requiring any prior knowledge of the microphone layout or any explicit signal preprocessing for speech enhancement. We embed an attention mechanism within a Recurrent Neural Network (RNN) based acoustic model to automatically tune its attention to a more reliable input source. Unlike traditional multi-channel preprocessing, our system can be optimized towards the desired output in one step. Although attention-based models have recently achieved impressive results on sequence-to-sequence learning, no attention mechanisms have previously been applied to learn potentially asynchronous and non-stationary multiple inputs. We evaluate our neural attention model on the CHiME-3 challenge task, and show that the model achieves comparable performance to beamforming using a purely data-driven method.


title: Feature-based Attention in Convolutional Neural Networks

abstract: Convolutional neural networks (CNNs) have proven effective for image processing tasks, such as object recognition and classification. Recently, CNNs have been enhanced with concepts of attention, similar to those found in biology. Much of this work on attention has focused on effective serial spatial processing. In this paper, I introduce a simple procedure for applying feature-based attention (FBA) to CNNs and compare multiple implementation options. FBA is a top-down signal applied globally to an input image which aides in detecting chosen objects in cluttered or noisy settings. The concept of FBA and the implementation details tested here were derived from what is known (and debated) about biological object- and feature-based attention. The implementations of FBA described here increase performance on challenging object detection tasks using a procedure that is simple, fast, and does not require additional iterative training. Furthermore, the comparisons performed here suggest that a proposed model of biological FBA (the "feature similarity gain model") is effective in increasing performance.


title: Learning to Generate Images with Perceptual Similarity Metrics

abstract: Deep networks are increasingly being applied to problems involving image synthesis, e.g., generating images from textual descriptions and reconstructing an input image from a compact representation. Supervised training of image-synthesis networks typically uses a pixel-wise loss (PL) to indicate the mismatch between a generated image and its corresponding target image. We propose instead to use a loss function that is better calibrated to human perceptual judgments of image quality: the multiscale structural-similarity score (MS-SSIM). Because MS-SSIM is differentiable, it is easily incorporated into gradient-descent learning. We compare the consequences of using MS-SSIM versus PL loss on training deterministic and stochastic autoencoders. For three different architectures, we collected human judgments of the quality of image reconstructions. Observers reliably prefer images synthesized by MS-SSIM-optimized models over those synthesized by PL-optimized models, for two distinct PL measures ($\ell_1$ and $\ell_2$ distances). We also explore the effect of training objective on image encoding and analyze conditions under which perceptually-optimized representations yield better performance on image classification. Finally, we demonstrate the superiority of perceptually-optimized networks for super-resolution imaging. Just as computer vision has advanced through the use of convolutional architectures that mimic the structure of the mammalian visual system, we argue that significant additional advances can be made in modeling images through the use of training objectives that are well aligned to characteristics of human perception.


title: Better Computer Go Player with Neural Network and Long-term Prediction

abstract: Competing with top human players in the ancient game of Go has been a long-term goal of artificial intelligence. Go's high branching factor makes traditional search techniques ineffective, even on leading-edge hardware, and Go's evaluation function could change drastically with one stone change. Recent works [Maddison et al. (2015); Clark & Storkey (2015)] show that search is not strictly necessary for machine Go players. A pure pattern-matching approach, based on a Deep Convolutional Neural Network (DCNN) that predicts the next move, can perform as well as Monte Carlo Tree Search (MCTS)-based open source Go engines such as Pachi [Baudis & Gailly (2012)] if its search budget is limited. We extend this idea in our bot named darkforest, which relies on a DCNN designed for long-term predictions. Darkforest substantially improves the win rate for pattern-matching approaches against MCTS-based approaches, even with looser search budgets. Against human players, the newest versions, darkfores2, achieve a stable 3d level on KGS Go Server as a ranked bot, a substantial improvement upon the estimated 4k-5k ranks for DCNN reported in Clark & Storkey (2015) based on games against other machine players. Adding MCTS to darkfores2 creates a much stronger player named darkfmcts3: with 5000 rollouts, it beats Pachi with 10k rollouts in all 250 games; with 75k rollouts it achieves a stable 5d level in KGS server, on par with state-of-the-art Go AIs (e.g., Zen, DolBaram, CrazyStone) except for AlphaGo [Silver et al. (2016)]; with 110k rollouts, it won the 3rd place in January KGS Go Tournament.


title: Training Deep Neural Networks via Direct Loss Minimization

abstract: Supervised training of deep neural nets typically relies on minimizing cross-entropy. However, in many domains, we are interested in performing well on metrics specific to the application. In this paper we propose a direct loss minimization approach to train deep neural networks, which provably minimizes the application-specific loss function. This is often non-trivial, since these functions are neither smooth nor decomposable and thus are not amenable to optimization with standard gradient-based methods. We demonstrate the effectiveness of our approach in the context of maximizing average precision for ranking problems. Towards this goal, we develop a novel dynamic programming algorithm that can efficiently compute the weight updates. Our approach proves superior to a variety of baselines in the context of action classification and object detection, especially in the presence of label noise.


title: QBDC: Query by dropout committee for training deep supervised architecture

abstract: While the current trend is to increase the depth of neural networks to increase their performance, the size of their training database has to grow accordingly. We notice an emergence of tremendous databases, although providing labels to build a training set still remains a very expensive task. We tackle the problem of selecting the samples to be labelled in an online fashion. In this paper, we present an active learning strategy based on query by committee and dropout technique to train a Convolutional Neural Network (CNN). We derive a commmittee of partial CNNs resulting from batchwise dropout runs on the initial CNN. We evaluate our active learning strategy for CNN on MNIST benchmark, showing in particular that selecting less than 30 % from the annotated database is enough to get similar error rate as using the full training set on MNIST. We also studied the robustness of our method against adversarial examples.


title: Image effects in transport at metal-molecule interfaces

abstract: We present a method for incorporating image-charge effects into the description of charge transport through molecular devices. A simple model allows us to calculate the adjustment of the transport levels, due to the polarization of the electrodes as charge is added to and removed from the molecule. For this, we use the charge distributions of the molecule between two metal electrodes in several charge states, rather than in gas phase, as obtained from a density-functional theory-based transport code. This enables us to efficiently model level shifts and gap renormalization caused by image-charge effects, which are essential for understanding molecular transport experiments. We apply the method to benzene di-amine molecules and compare our results with the standard approach based on gas phase charges. Finally, we give a detailed account of the application of our approach to porphyrin-derivative devices recently studied experimentally by Perrin et al. [Nat. Nanotechnol. 8, 282 (2013)], which demonstrates the importance of accounting for image-charge effects when modeling transport through molecular junctions.


title: An Improved Neoclassical Drift-Magnetohydrodynamical Fluid Model of Helical Magnetic Island Equilibria in Tokamak Plasmas

abstract: The effect of the perturbed ion polarization current on the stability of neoclassical tearing modes is calculated using an improved, neoclassical, four-field, drift-MHD model. The calculation involves the self-consistent determination of the pressure and scalar electric potential profiles in the vicinity of the associated magnetic island chain, which allows the chain's propagation velocity to be fixed. Two regimes are considered. First, a regime in which neoclassical ion poloidal flow damping is not strong enough to enhance the magnitude of the polarization current (relative to that found in slab geometry). Second, a regime in which neoclassical ion poloidal flow damping is strong enough to significantly enhance the magnitude of the polarization current. In both regimes, two types of solution are considered. First, a freely rotating solution (i.e., an island chain that is not interacting with a static, resonant, magnetic perturbation). Second, a locked solution (i.e., an island chain that has been brought to rest in the laboratory frame via interaction with a static, resonant, magnetic perturbation). In all cases, the polarization current is found to be either always stabilizing, or stabilizing provided that eta_i = d ln T_i/d ln n_e does not exceed some threshold value. In certain ranges of eta_i, the polarization current is found to have have a stabilizing effect on a freely rotating island, but a destabilizing effect on a corresponding locked island.


title: The fate of a red nugget: In-situ star formation of satellites around a massive compact galaxy

abstract: To study the accretion phase for local massive galaxies, we search accreting satellites around a massive compact galaxy (M_*~3.9x10^10Msun), spectroscopically confirmed (z_spec-1.9213) in the eXtreme Deep Field, which has been originally reported in Szomoru et al. We detect 1369 satellite candidates within the projected virial radius (rvir~300 kpc) of the compact galaxy in the all-combined ACS image with 5sigma-limiting magnitude of mACS~30.6 ABmag, which corresponds to ~1.6x10^7M_sun at the redshift. The photometric redshift measured with 12 multi-band images confirms 34 satellites out of the candidates. Most of the satellites are found to have the rest-frame colors consistent with star forming galaxies. We investigate the relation between stellar mass and star formation rate (the star formation main sequence), and find the steeper slope at the low-mass end (<10^8M_sun), while more massive satellites are consistently on the sequence reported in previous studies. Within the uncertainties of star formation and photometric redshift, we conjecture possible scenarios for the compact galaxy which evolves to a local massive galaxy by way of significant size and mass growth. While merging of the existing total stellar mass of the satellites is not enough to explain the mass growth predicted by observations and simulations, the contribution by in-situ star formation in the satellites would compensate the deficit. Provided that most satellites keep the observed in-situ star formation and then quench before they accrete by, e.g., environmental quenching, the compact galaxy would become a massive early-type galaxy consistent with the local size-mass relation.


title: Fast Parallel SAME Gibbs Sampling on General Discrete Bayesian Networks

abstract: A fundamental task in machine learning and related fields is to perform inference on Bayesian networks. Since exact inference takes exponential time in general, a variety of approximate methods are used. Gibbs sampling is one of the most accurate approaches and provides unbiased samples from the posterior but it has historically been too expensive for large models. In this paper, we present an optimized, parallel Gibbs sampler augmented with state replication (SAME or State Augmented Marginal Estimation) to decrease convergence time. We find that SAME can improve the quality of parameter estimates while accelerating convergence. Experiments on both synthetic and real data show that our Gibbs sampler is substantially faster than the state of the art sampler, JAGS, without sacrificing accuracy. Our ultimate objective is to introduce the Gibbs sampler to researchers in many fields to expand their range of feasible inference problems.


title: Modelling Spatial Compositional Data: Reconstructions of past land cover and uncertainties

abstract: In this paper, we construct a hierarchical model for spatial compositional data, which is used to reconstruct past land-cover compositions (in terms of coniferous forest, broadleaved forest, and unforested/open land) for five time periods during the past $6\,000$ years over Europe. The model consists of a Gaussian Markov Random Field (GMRF) with Dirichlet observations. A block updated Markov chain Monte Carlo (MCMC), including an adaptive Metropolis adjusted Langevin step, is used to estimate model parameters. The sparse precision matrix in the GMRF provides computational advantages leading to a fast MCMC algorithm. Reconstructions are obtained by combining pollen-based estimates of vegetation cover at a limited number of locations with scenarios of past deforestation and output from a dynamic vegetation model. To evaluate uncertainties in the predictions a novel way of constructing joint confidence regions for the entire composition at each prediction location is proposed. The hierarchical model's ability to reconstruct past land cover is evaluated through cross validation for all time periods, and by comparing reconstructions for the recent past to a present day European forest map. The evaluation results are promising and the model is able to capture known structures in past land-cover compositions.


title: Binding via Reconstruction Clustering

abstract: Disentangled distributed representations of data are desirable for machine learning, since they are more expressive and can generalize from fewer examples. However, for complex data, the distributed representations of multiple objects present in the same input can interfere and lead to ambiguities, which is commonly referred to as the binding problem. We argue for the importance of the binding problem to the field of representation learning, and develop a probabilistic framework that explicitly models inputs as a composition of multiple objects. We propose an unsupervised algorithm that uses denoising autoencoders to dynamically bind features together in multi-object inputs through an Expectation-Maximization-like clustering process. The effectiveness of this method is demonstrated on artificially generated datasets of binary images, showing that it can even generalize to bind together new objects never seen by the autoencoder during training.


title: Canonical Autocorrelation Analysis

abstract: We present an extension of sparse Canonical Correlation Analysis (CCA) designed for finding multiple-to-multiple linear correlations within a single set of variables. Unlike CCA, which finds correlations between two sets of data where the rows are matched exactly but the columns represent separate sets of variables, the method proposed here, Canonical Autocorrelation Analysis (CAA), finds multivariate correlations within just one set of variables. This can be useful when we look for hidden parsimonious structures in data, each involving only a small subset of all features. In addition, the discovered correlations are highly interpretable as they are formed by pairs of sparse linear combinations of the original features. We show how CAA can be of use as a tool for anomaly detection when the expected structure of correlations is not followed by anomalous data. We illustrate the utility of CAA in two application domains where single-class and unsupervised learning of correlation structures are particularly relevant: breast cancer diagnosis and radiation threat detection. When applied to the Wisconsin Breast Cancer data, single-class CAA is competitive with supervised methods used in literature. On the radiation threat detection task, unsupervised CAA performs significantly better than an unsupervised alternative prevalent in the domain, while providing valuable additional insights for threat analysis.