title: Learning Task Grouping and Overlap in Multi-task Learning

abstract: In the paradigm of multi-task learning, mul- tiple related prediction tasks are learned jointly, sharing information across the tasks. We propose a framework for multi-task learn- ing that enables one to selectively share the information across the tasks. We assume that each task parameter vector is a linear combi- nation of a finite number of underlying basis tasks. The coefficients of the linear combina- tion are sparse in nature and the overlap in the sparsity patterns of two tasks controls the amount of sharing across these. Our model is based on on the assumption that task pa- rameters within a group lie in a low dimen- sional subspace but allows the tasks in differ- ent groups to overlap with each other in one or more bases. Experimental results on four datasets show that our approach outperforms competing methods.


title: Learning Invariant Representations with Local Transformations

abstract: Learning invariant representations is an important problem in machine learning and pattern recognition. In this paper, we present a novel framework of transformation-invariant feature learning by incorporating linear transformations into the feature learning algorithms. For example, we present the transformation-invariant restricted Boltzmann machine that compactly represents data by its weights and their transformations, which achieves invariance of the feature representation via probabilistic max pooling. In addition, we show that our transformation-invariant feature learning framework can also be extended to other unsupervised learning methods, such as autoencoders or sparse coding. We evaluate our method on several image classification benchmark datasets, such as MNIST variations, CIFAR-10, and STL-10, and show competitive or superior classification performance when compared to the state-of-the-art. Furthermore, our method achieves state-of-the-art performance on phone classification tasks with the TIMIT dataset, which demonstrates wide applicability of our proposed algorithms to other domains.


title: Cross-Domain Multitask Learning with Latent Probit Models

abstract: Learning multiple tasks across heterogeneous domains is a challenging problem since the feature space may not be the same for different tasks. We assume the data in multiple tasks are generated from a latent common domain via sparse domain transforms and propose a latent probit model (LPM) to jointly learn the domain transforms, and the shared probit classifier in the common domain. To learn meaningful task relatedness and avoid over-fitting in classification, we introduce sparsity in the domain transforms matrices, as well as in the common classifier. We derive theoretical bounds for the estimation error of the classifier in terms of the sparsity of domain transforms. An expectation-maximization algorithm is derived for learning the LPM. The effectiveness of the approach is demonstrated on several real datasets.


title: Distributed Parameter Estimation via Pseudo-likelihood

abstract: Estimating statistical models within sensor networks requires distributed algorithms, in which both data and computation are distributed across the nodes of the network. We propose a general approach for distributed learning based on combining local estimators defined by pseudo-likelihood components, encompassing a number of combination methods, and provide both theoretical and experimental analysis. We show that simple linear combination or max-voting methods, when combined with second-order information, are statistically competitive with more advanced and costly joint optimization. Our algorithms have many attractive properties including low communication and computational cost and "any-time" behavior.


title: Structured Learning from Partial Annotations

abstract: Structured learning is appropriate when predicting structured outputs such as trees, graphs, or sequences. Most prior work requires the training set to consist of complete trees, graphs or sequences. Specifying such detailed ground truth can be tedious or infeasible for large outputs. Our main contribution is a large margin formulation that makes structured learning from only partially annotated data possible. The resulting optimization problem is non-convex, yet can be efficiently solve by concave-convex procedure (CCCP) with novel speedup strategies. We apply our method to a challenging tracking-by-assignment problem of a variable number of divisible objects. On this benchmark, using only 25% of a full annotation we achieve a performance comparable to a model learned with a full annotation. Finally, we offer a unifying perspective of previous work using the hinge, ramp, or max loss for structured learning, followed by an empirical comparison on their practical performance.


title: An Online Boosting Algorithm with Theoretical Justifications

abstract: We study the task of online boosting--combining online weak learners into an online strong learner. While batch boosting has a sound theoretical foundation, online boosting deserves more study from the theoretical perspective. In this paper, we carefully compare the differences between online and batch boosting, and propose a novel and reasonable assumption for the online weak learner. Based on the assumption, we design an online boosting algorithm with a strong theoretical guarantee by adapting from the offline SmoothBoost algorithm that matches the assumption closely. We further tackle the task of deciding the number of weak learners using established theoretical results for online convex programming and predicting with expert advice. Experiments on real-world data sets demonstrate that the proposed algorithm compares favorably with existing online boosting algorithms.


title: A Joint Model of Language and Perception for Grounded Attribute Learning

abstract: As robots become more ubiquitous and capable, it becomes ever more important to enable untrained users to easily interact with them. Recently, this has led to study of the language grounding problem, where the goal is to extract representations of the meanings of natural language tied to perception and actuation in the physical world. In this paper, we present an approach for joint learning of language and perception models for grounded attribute induction. Our perception model includes attribute classifiers, for example to detect object color and shape, and the language model is based on a probabilistic categorial grammar that enables the construction of rich, compositional meaning representations. The approach is evaluated on the task of interpreting sentences that describe sets of objects in a physical workspace. We demonstrate accurate task performance and effective latent-variable concept induction in physical grounded scenes.


title: Anytime Marginal MAP Inference

abstract: This paper presents a new anytime algorithm for the marginal MAP problem in graphical models. The algorithm is described in detail, its complexity and convergence rate are studied, and relations to previous theoretical results for the problem are discussed. It is shown that the algorithm runs in polynomial-time if the underlying graph of the model has bounded tree-width, and that it provides guarantees to the lower and upper bounds obtained within a fixed amount of computational resources. Experiments with both real and synthetic generated models highlight its main characteristics and show that it compares favorably against Park and Darwiche's systematic search, particularly in the case of problems with many MAP variables and moderate tree-width.


title: Sparse Stochastic Inference for Latent Dirichlet allocation

abstract: We present a hybrid algorithm for Bayesian topic models that combines the efficiency of sparse Gibbs sampling with the scalability of online stochastic inference. We used our algorithm to analyze a corpus of 1.2 million books (33 billion words) with thousands of topics. Our approach reduces the bias of variational inference and generalizes to many Bayesian hidden-variable models.


title: A Fast and Simple Algorithm for Training Neural Probabilistic Language Models

abstract: In spite of their superior performance, neural probabilistic language models (NPLMs) remain far less widely used than n-gram models due to their notoriously long training times, which are measured in weeks even for moderately-sized datasets. Training NPLMs is computationally expensive because they are explicitly normalized, which leads to having to consider all words in the vocabulary when computing the log-likelihood gradients. We propose a fast and simple algorithm for training NPLMs based on noise-contrastive estimation, a newly introduced procedure for estimating unnormalized continuous distributions. We investigate the behaviour of the algorithm on the Penn Treebank corpus and show that it reduces the training times by more than an order of magnitude without affecting the quality of the resulting models. The algorithm is also more efficient and much more stable than importance sampling because it requires far fewer noise samples to perform well. We demonstrate the scalability of the proposed approach by training several neural language models on a 47M-word corpus with a 80K-word vocabulary, obtaining state-of-the-art results on the Microsoft Research Sentence Completion Challenge dataset.


title: Convergence of the EM Algorithm for Gaussian Mixtures with Unbalanced Mixing Coefficients

abstract: The speed of convergence of the Expectation Maximization (EM) algorithm for Gaussian mixture model fitting is known to be dependent on the amount of overlap among the mixture components. In this paper, we study the impact of mixing coefficients on the convergence of EM. We show that when the mixture components exhibit some overlap, the convergence of EM becomes slower as the dynamic range among the mixing coefficients increases. We propose a deterministic anti-annealing algorithm, that significantly improves the speed of convergence of EM for such mixtures with unbalanced mixing coefficients. The proposed algorithm is compared against other standard optimization techniques like BFGS, Conjugate Gradient, and the traditional EM algorithm. Finally, we propose a similar deterministic anti-annealing based algorithm for the Dirichlet process mixture model and demonstrate its advantages over the conventional variational Bayesian approach.


title: A Binary Classification Framework for Two-Stage Multiple Kernel Learning

abstract: With the advent of kernel methods, automating the task of specifying a suitable kernel has become increasingly important. In this context, the Multiple Kernel Learning (MKL) problem of finding a combination of pre-specified base kernels that is suitable for the task at hand has received significant attention from researchers. In this paper we show that Multiple Kernel Learning can be framed as a standard binary classification problem with additional constraints that ensure the positive definiteness of the learned kernel. Framing MKL in this way has the distinct advantage that it makes it easy to leverage the extensive research in binary classification to develop better performing and more scalable MKL algorithms that are conceptually simpler, and, arguably, more accessible to practitioners. Experiments on nine data sets from different domains show that, despite its simplicity, the proposed technique compares favorably with current leading MKL approaches.


title: Incorporating Domain Knowledge in Matching Problems via Harmonic Analysis

abstract: Matching one set of objects to another is a ubiquitous task in machine learning and computer vision that often reduces to some form of the quadratic assignment problem (QAP). The QAP is known to be notoriously hard, both in theory and in practice. Here, we investigate if this difficulty can be mitigated when some additional piece of information is available: (a) that all QAP instances of interest come from the same application, and (b) the correct solution for a set of such QAP instances is given. We propose a new approach to accelerate the solution of QAPs based on learning parameters for a modified objective function from prior QAP instances. A key feature of our approach is that it takes advantage of the algebraic structure of permutations, in conjunction with special methods for optimizing functions over the symmetric group Sn in Fourier space. Experiments show that in practical domains the new method can outperform existing approaches.


title: Variational Bayesian Inference with Stochastic Search

abstract: Mean-field variational inference is a method for approximate Bayesian posterior inference. It approximates a full posterior distribution with a factorized set of distributions by maximizing a lower bound on the marginal likelihood. This requires the ability to integrate a sum of terms in the log joint likelihood using this factorized distribution. Often not all integrals are in closed form, which is typically handled by using a lower bound. We present an alternative algorithm based on stochastic optimization that allows for direct optimization of the variational lower bound. This method uses control variates to reduce the variance of the stochastic search gradient, in which existing lower bounds can play an important role. We demonstrate the approach on two non-conjugate models: logistic regression and an approximation to the HDP.


title: Exact Maximum Margin Structure Learning of Bayesian Networks

abstract: Recently, there has been much interest in finding globally optimal Bayesian network structures. These techniques were developed for generative scores and can not be directly extended to discriminative scores, as desired for classification. In this paper, we propose an exact method for finding network structures maximizing the probabilistic soft margin, a successfully applied discriminative score. Our method is based on branch-and-bound techniques within a linear programming framework and maintains an any-time solution, together with worst-case sub-optimality bounds. We apply a set of order constraints for enforcing the network structure to be acyclic, which allows a compact problem representation and the use of general-purpose optimization techniques. In classification experiments, our methods clearly outperform generatively trained network structures and compete with support vector machines.


title: Sparse Support Vector Infinite Push

abstract: In this paper, we address the problem of embedded feature selection for ranking on top of the list problems. We pose this problem as a regularized empirical risk minimization with $p$-norm push loss function ($p=\infty$) and sparsity inducing regularizers. We leverage the issues related to this challenging optimization problem by considering an alternating direction method of multipliers algorithm which is built upon proximal operators of the loss function and the regularizer. Our main technical contribution is thus to provide a numerical scheme for computing the infinite push loss function proximal operator. Experimental results on toy, DNA microarray and BCI problems show how our novel algorithm compares favorably to competitors for ranking on top while using fewer variables in the scoring function.


title: Copula Mixture Model for Dependency-seeking Clustering

abstract: We introduce a copula mixture model to perform dependency-seeking clustering when co-occurring samples from different data sources are available. The model takes advantage of the great flexibility offered by the copulas framework to extend mixtures of Canonical Correlation Analysis to multivariate data with arbitrary continuous marginal densities. We formulate our model as a non-parametric Bayesian mixture, while providing efficient MCMC inference. Experiments on synthetic and real data demonstrate that the increased flexibility of the copula mixture significantly improves the clustering and the interpretability of the results.


title: A Generative Process for Sampling Contractive Auto-Encoders

abstract: The contractive auto-encoder learns a representation of the input data that captures the local manifold structure around each data point, through the leading singular vectors of the Jacobian of the transformation from input to representation. The corresponding singular values specify how much local variation is plausible in directions associated with the corresponding singular vectors, while remaining in a high-density region of the input space. This paper proposes a procedure for generating samples that are consistent with the local structure captured by a contractive auto-encoder. The associated stochastic process defines a distribution from which one can sample, and which experimentally appears to converge quickly and mix well between modes, compared to Restricted Boltzmann Machines and Deep Belief Networks. The intuitions behind this procedure can also be used to train the second layer of contraction that pools lower-level features and learns to be invariant to the local directions of variation discovered in the first layer. We show that this can help learn and represent invariances present in the data and improve classification error.


title: Rethinking Collapsed Variational Bayes Inference for LDA

abstract: We propose a novel interpretation of the collapsed variational Bayes inference with a zero-order Taylor expansion approximation, called CVB0 inference, for latent Dirichlet allocation (LDA). We clarify the properties of the CVB0 inference by using the alpha-divergence. We show that the CVB0 inference is composed of two different divergence projections: alpha=1 and -1. This interpretation will help shed light on CVB0 works.


title: Efficient Structured Prediction with Latent Variables for General Graphical Models

abstract: In this paper we propose a unified framework for structured prediction with latent variables which includes hidden conditional random fields and latent structured support vector machines as special cases. We describe a local entropy approximation for this general formulation using duality, and derive an efficient message passing algorithm that is guaranteed to converge. We demonstrate its effectiveness in the tasks of image segmentation as well as 3D indoor scene understanding from single images, showing that our approach is superior to latent structured support vector machines and hidden conditional random fields.


title: Large Scale Variational Bayesian Inference for Structured Scale Mixture Models

abstract: Natural image statistics exhibit hierarchical dependencies across multiple scales. Representing such prior knowledge in non-factorial latent tree models can boost performance of image denoising, inpainting, deconvolution or reconstruction substantially, beyond standard factorial "sparse" methodology. We derive a large scale approximate Bayesian inference algorithm for linear models with non-factorial (latent tree-structured) scale mixture priors. Experimental results on a range of denoising and inpainting problems demonstrate substantially improved performance compared to MAP estimation or to inference with factorial priors.


title: Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation

abstract: We study the problem of unsupervised domain adaptation, which aims to adapt classifiers trained on a labeled source domain to an unlabeled target domain. Many existing approaches first learn domain-invariant features and then construct classifiers with them. We propose a novel approach that jointly learn the both. Specifically, while the method identifies a feature space where data in the source and the target domains are similarly distributed, it also learns the feature space discriminatively, optimizing an information-theoretic metric as an proxy to the expected misclassification error on the target domain. We show how this optimization can be effectively carried out with simple gradient-based methods and how hyperparameters can be cross-validated without demanding any labeled data from the target domain. Empirical studies on benchmark tasks of object recognition and sentiment analysis validated our modeling assumptions and demonstrated significant improvement of our method over competing ones in classification accuracies.


title: Gap Filling in the Plant Kingdom---Trait Prediction Using Hierarchical Probabilistic Matrix Factorization

abstract: Plant traits are a key to understanding and predicting the adaptation of ecosystems to environmental changes, which motivates the TRY project aiming at constructing a global database for plant traits and becoming a standard resource for the ecological community. Despite its unprecedented coverage, a large percentage of missing data substantially constrains joint trait analysis. Meanwhile, the trait data is characterized by the hierarchical phylogenetic structure of the plant kingdom. While factorization based matrix completion techniques have been widely used to address the missing data problem, traditional matrix factorization methods are unable to leverage the phylogenetic structure. We propose hierarchical probabilistic matrix factorization (HPMF), which effectively uses hierarchical phylogenetic information for trait prediction. We demonstrate HPMF's high accuracy, effectiveness of incorporating hierarchical structure and ability to capture trait correlation through experiments.


title: Predicting Preference Flips in Commerce Search

abstract: Traditional approaches to ranking in web search follow the paradigm of rank-by-score: a learned function gives each query-URL combination an absolute score and URLs are ranked according to this score. This paradigm ensures that if the score of one URL is better than another then one will always be ranked higher than the other. Scoring contradicts prior work in behavioral economics that showed that users' preferences between two items depend not only on the items but also on the presented alternatives. Thus, for the same query, users' preference between items A and B depends on the presence/absence of item C. We propose a new model of ranking, the Random Shopper Model, that allows and explains such behavior. In this model, each feature is viewed as a Markov chain over the items to be ranked, and the goal is to find a weighting of the features that best reflects their importance. We show that our model can be learned under the empirical risk minimization framework, and give an efficient learning algorithm. Experiments on commerce search logs demonstrate that our algorithm outperforms scoring-based approaches including regression and listwise ranking.


title: A Topic Model for Melodic Sequences

abstract: We examine the problem of learning a probabilistic model for melody directly from musical sequences belonging to the same genre. This is a challenging task as one needs to capture not only the rich temporal structure evident in music, but also the complex statistical dependencies among different music components. To address this problem we introduce the Variable-gram Topic Model, which couples the latent topic formalism with a systematic model for contextual information. We evaluate the model on next-step prediction. Additionally, we present a novel way of model evaluation, where we directly compare model samples with data sequences using the Maximum Mean Discrepancy of string kernels, to assess how close is the model distribution to the data distribution. We show that the model has the highest performance under both evaluation measures when compared to LDA, the Topic Bigram and related non-topic models.


title: Minimizing The Misclassification Error Rate Using a Surrogate Convex Loss

abstract: We carefully study how well minimizing convex surrogate loss functions, corresponds to minimizing the misclassification error rate for the problem of binary classification with linear predictors. In particular, we show that amongst all convex surrogate losses, the hinge loss gives essentially the best possible bound, of all convex loss functions, for the misclassification error rate of the resulting linear predictor in terms of the best possible margin error rate. We also provide lower bounds for specific convex surrogates that show how different commonly used losses qualitatively differ from each other.


title: Isoelastic Agents and Wealth Updates in Machine Learning Markets

abstract: Recently, prediction markets have shown considerable promise for developing flexible mechanisms for machine learning. In this paper, agents with isoelastic utilities are considered. It is shown that the costs associated with homogeneous markets of agents with isoelastic utilities produce equilibrium prices corresponding to alpha-mixtures, with a particular form of mixing component relating to each agent's wealth. We also demonstrate that wealth accumulation for logarithmic and other isoelastic agents (through payoffs on prediction of training targets) can implement both Bayesian model updates and mixture weight updates by imposing different market payoff structures. An iterative algorithm is given for market equilibrium computation. We demonstrate that inhomogeneous markets of agents with isoelastic utilities outperform state of the art aggregate classifiers such as random forests, as well as single classifiers (neural networks, decision trees) on a number of machine learning benchmarks, and show that isoelastic combination methods are generally better than their logarithmic counterparts.


title: Statistical Linear Estimation with Penalized Estimators: an Application to Reinforcement Learning

abstract: Motivated by value function estimation in reinforcement learning, we study statistical linear inverse problems, i.e., problems where the coefficients of a linear system to be solved are observed in noise. We consider penalized estimators, where performance is evaluated using a matrix-weighted two-norm of the defect of the estimator measured with respect to the true, unknown coefficients. Two objective functions are considered depending whether the error of the defect measured with respect to the noisy coefficients is squared or unsquared. We propose simple, yet novel and theoretically well-founded data-dependent choices for the regularization parameters for both cases that avoid data-splitting. A distinguishing feature of our analysis is that we derive deterministic error bounds in terms of the error of the coefficients, thus allowing the complete separation of the analysis of the stochastic properties of these errors. We show that our results lead to new insights and bounds for linear value function estimation in reinforcement learning.


title: Deep Lambertian Networks

abstract: Visual perception is a challenging problem in part due to illumination variations. A possible solution is to first estimate an illumination invariant representation before using it for recognition. The object albedo and surface normals are examples of such representations. In this paper, we introduce a multilayer generative model where the latent variables include the albedo, surface normals, and the light source. Combining Deep Belief Nets with the Lambertian reflectance assumption, our model can learn good priors over the albedo from 2D images. Illumination variations can be explained by changing only the lighting latent variable in our model. By transferring learned knowledge from similar objects, albedo and surface normals estimation from a single image is possible in our model. Experiments demonstrate that our model is able to generalize as well as improve over standard baselines in one-shot face recognition.


title: Agglomerative Bregman Clustering

abstract: This manuscript develops the theory of agglomerative clustering with Bregman divergences. Geometric smoothing techniques are developed to deal with degenerate clusters. To allow for cluster models based on exponential families with overcomplete representations, Bregman divergences are developed for nondifferentiable convex functions.


title: Small-sample Brain Mapping: Sparse Recovery on Spatially Correlated Designs with Randomization and Clustering

abstract: Functional neuroimaging can measure the brain?s response to an external stimulus. It is used to perform brain mapping: identifying from these observations the brain regions involved. This problem can be cast into a linear supervised learning task where the neuroimaging data are used as predictors for the stimulus. Brain mapping is then seen as a support recovery problem. On functional MRI (fMRI) data, this problem is particularly challenging as i) the number of samples is small due to limited acquisition time and ii) the variables are strongly correlated. We propose to overcome these difficulties using sparse regression models over new variables obtained by clustering of the original variables. The use of randomization techniques, e.g. bootstrap samples, and clustering of the variables improves the recovery properties of sparse methods. We demonstrate the benefit of our approach on an extensive simulation study as well as two fMRI datasets.


title: Online Alternating Direction Method

abstract: Online optimization has emerged as powerful tool in large scale optimization. In this paper, we introduce efficient online algorithms based on the alternating directions method (ADM). We introduce a new proof technique for ADM in the batch setting, which yields the O(1/T) convergence rate of ADM and forms the basis of regret analysis in the online setting. We consider two scenarios in the online setting, based on whether the solution needs to lie in the feasible set or not. In both settings, we establish regret bounds for both the objective function as well as constraint violation for general and strongly convex functions. Preliminary results are presented to illustrate the performance of the proposed algorithms.


title: Monte Carlo Bayesian Reinforcement Learning

abstract: Bayesian reinforcement learning (BRL) encodes prior knowledge of the world in a model and represents uncertainty in model parameters by maintaining a probability distribution over them. This paper presents Monte Carlo BRL (MC-BRL), a simple and general approach to BRL. MC-BRL samples a priori a finite set of hypotheses for the model parameter values and forms a discrete partially observable Markov decision process (POMDP) whose state space is a cross product of the state space for the reinforcement learning task and the sampled model parameter space. The POMDP does not require conjugate distributions for belief representation, as earlier works do, and can be solved relatively easily with point-based approximation algorithms. MC-BRL naturally handles both fully and partially observable worlds. Theoretical and experimental results show that the discrete POMDP approximates the underlying BRL task well with guaranteed performance.


title: Conditional Sparse Coding and Grouped Multivariate Regression

abstract: We study the problem of multivariate regression where the data are naturally grouped, and a regression matrix is to be estimated for each group. We propose an approach in which a dictionary of low rank parameter matrices is estimated across groups, and a sparse linear combination of the dictionary elements is estimated to form a model within each group. We refer to the method as conditional sparse coding since it is a coding procedure for the response vectors Y conditioned on the covariate vectors X. This approach captures the shared information across the groups while adapting to the structure within each group. It exploits the same intuition behind sparse coding that has been successfully developed in computer vision and computational neuroscience. We propose an algorithm for conditional sparse coding, analyze its theoretical properties in terms of predictive accuracy, and present the results of simulation and brain imaging experiments that compare the new technique to reduced rank regression.


title: The Greedy Miser: Learning under Test-time Budgets

abstract: As machine learning algorithms enter applications in industrial settings, there is increased interest in controlling their cpu-time during testing. The cpu-time consists of the running time of the algorithm and the extraction time of the features. The latter can vary drastically when the feature set is diverse. In this paper, we propose an algorithm, the Greedy Miser, that incorporates the feature extraction cost during training to explicitly minimize the cpu-time during testing. The algorithm is a straightforward extension of stage-wise regression and is equally suitable for regression or multi-class classification. Compared to prior work, it is significantly more cost-effective and scales to larger data sets.


title: Smoothness and Structure Learning by Proxy

abstract: As data sets grow in size, the ability of learning methods to find structure in them is increasingly hampered by the time needed to search the large spaces of possibilities and generate a score for each that takes all of the observed data into account. For instance, Bayesian networks, the model chosen in this paper, have a super-exponentially large search space for a fixed number of variables. One possible method to alleviate this problem is to use a proxy, such as a Gaussian Process regressor, in place of the true scoring function, training it on a selection of sampled networks. We prove here that the use of such a proxy is well-founded, as we can bound the smoothness of a commonly-used scoring function for Bayesian network structure learning. We show here that, compared to an identical search strategy using the network?s exact scores, our proxy-based search is able to get equivalent or better scores on a number of data sets in a fraction of the time.


title: Adaptive Canonical Correlation Analysis Based On Matrix Manifolds

abstract: In this paper, we formulate the Canonical Correlation Analysis (CCA) problem on matrix manifolds. This framework provides a natural way for dealing with matrix constraints and tools for building efficient algorithms even in an adaptive setting. Finally, an adaptive CCA algorithm is proposed and applied to a change detection problem in EEG signals.


title: Hierarchical Exploration for Accelerating Contextual Bandits

abstract: Contextual bandit learning is an increasingly popular approach to optimizing recommender systems via user feedback, but can be slow to converge in practice due to the need for exploring a large feature space. In this paper, we propose a coarse-to-fine hierarchical approach for encoding prior knowledge that drastically reduces the amount of exploration required. Intuitively, user preferences can be reasonably embedded in a coarse low-dimensional feature space that can be explored efficiently, requiring exploration in the high-dimensional space only as necessary. We introduce a bandit algorithm that explores within this coarse-to-fine spectrum, and prove performance guarantees that depend on how well the coarse space captures the user's preferences. We demonstrate substantial improvement over conventional bandit algorithms through extensive simulation as well as a live user study in the setting of personalized news recommendation.


title: Regularizers versus Losses for Nonlinear Dimensionality Reduction: A Factored View with New Convex Relaxations

abstract: We demonstrate that almost all non-parametric dimensionality reduction methods can be expressed by a simple procedure: regularized loss minimization plus singular value truncation. By distinguishing the role of the loss and regularizer in such a process, we recover a factored perspective that reveals some gaps in the current literature. Beyond identifying a useful new loss for manifold unfolding, a key contribution is to derive new convex regularizers that combine distance maximization with rank reduction. These regularizers can be applied to any loss.


title: Lognormal and Gamma Mixed Negative Binomial Regression

abstract: In regression analysis of counts, a lack of simple and efficient algorithms for posterior computation has made Bayesian approaches appear unattractive and thus underdeveloped. We propose a lognormal and gamma mixed negative binomial (NB) regression model for counts, and present efficient closed-form Bayesian inference; unlike conventional Poisson models, the proposed approach has two free parameters to include two different kinds of random effects, and allows the incorporation of prior information, such as sparsity in the regression coefficients. By placing a gamma distribution prior on the NB dispersion parameter r, and connecting a lognormal distribution prior with the logit of the NB probability parameter p, efficient Gibbs sampling and variational Bayes inference are both developed. The closed-form updates are obtained by exploiting conditional conjugacy via both a compound Poisson representation and a Polya-Gamma distribution based data augmentation approach. The proposed Bayesian inference can be implemented routinely, while being easily generalizable to more complex settings involving multivariate dependence structures. The algorithms are illustrated using real examples.


title: Exponential Regret Bounds for Gaussian Process Bandits with Deterministic Observations

abstract: This paper analyzes the problem of Gaussian process (GP) bandits with deterministic observations. The analysis uses a branch and bound algorithm that is related to the UCB algorithm of (Srinivas et al, 2010). For GPs with Gaussian observation noise, with variance strictly greater than zero, Srinivas et al proved that the regret vanishes at the approximate rate of $O(1/\sqrt{t})$, where t is the number of observations. To complement their result, we attack the deterministic case and attain a much faster exponential convergence rate. Under some regularity assumptions, we show that the regret decreases asymptotically according to $O(e^{-\frac{\tau t}{(\ln t)^{d/4}}})$ with high probability. Here, d is the dimension of the search space and tau is a constant that depends on the behaviour of the objective function near its global maximum.


title: Batch Active Learning via Coordinated Matching

abstract: Most prior work on active learning of classifiers has focused on sequentially selecting one unlabeled example at a time to be labeled in order to reduce the overall labeling effort. In many scenarios, however, it is desirable to label an entire batch of examples at once, for example, when labels can be acquired in parallel. This motivates us to study batch active learning, which iteratively selects batches of $k>1$ examples to be labeled. We propose a novel batch active learning method that leverages the availability of high-quality and efficient sequential active-learning policies by attempting to approximate their behavior when applied for $k$ steps. Specifically, our algorithm first uses Monte-Carlo simulation to estimate the distribution of unlabeled examples selected by a sequential policy over $k$ step executions. The algorithm then attempts to select a set of $k$ examples that best matches this distribution, leading to a combinatorial optimization problem that we term "bounded coordinated matching". While we show this problem is NP-hard in general, we give an efficient greedy solution, which inherits approximation bounds from supermodular minimization theory. Our experimental results on eight benchmark datasets show that the proposed approach is highly effective


title: Bayesian Conditional Cointegration

abstract: Cointegration is an important topic for time-series, and describes a relationship between two series in which a linear combination is stationary. Classically, the test for cointegration is based on a two stage process in which first the linear relation between the series is estimated by Ordinary Least Squares. Subsequently a unit root test is performed on the residuals. A well-known deficiency of this classical approach is that it can lead to erroneous conclusions about the presence of cointegration. As an alternative, we present a framework for estimating whether cointegration exists using Bayesian inference which is empirically superior to the classical approach. Finally, we apply our technique to model segmented cointegration in which cointegration may exist only for limited time. In contrast to previous approaches our model makes no restriction on the number of possible cointegration segments.


title: Output Space Search for Structured Prediction

abstract: We consider a framework for structured prediction based on search in the space of complete structured outputs. Given a structured input, an output is produced by running a time-bounded search procedure guided by a learned cost function, and then returning the least cost output uncovered during the search. This framework can be instantiated for a wide range of search spaces and search procedures, and easily incorporates arbitrary structured-prediction loss functions. In this paper, we make two main technical contributions. First, we define the limited-discrepancy search space over structured outputs, which is able to leverage powerful classification learning algorithms to improve the search space quality. Second, we give a generic cost function learning approach, where the key idea is to learn a cost function that attempts to mimic the behavior of conducting searches guided by the true loss function. Our experiments on six benchmark domains demonstrate that using our framework with only a small amount of search is sufficient for significantly improving on state-of-the-art structured-prediction performance.


title: On the Sample Complexity of Reinforcement Learning with a Generative Model

abstract: We consider the problem of learning the optimal action-value function in the discounted-reward Markov decision processes (MDPs). We prove a new PAC bound on the sample-complexity of model-based value iteration algorithm in the presence of the generative model, which indicates that for an MDP with N state-action pairs and the discount factor \gamma\in[0,1) only O(N\log(N/\delta)/((1-\gamma)^3\epsilon^2)) samples are required to find an \epsilon-optimal estimation of the action-value function with the probability 1-\delta. We also prove a matching lower bound of \Theta (N\log(N/\delta)/((1-\gamma)^3\epsilon^2)) on the sample complexity of estimating the optimal action-value function by every RL algorithm. To the best of our knowledge, this is the first matching result on the sample complexity of estimating the optimal (action-) value function in which the upper bound matches the lower bound of RL in terms of N, \epsilon, \delta and 1/(1-\gamma). Also, both our lower bound and our upper bound significantly improve on the state-of-the-art in terms of 1/(1-\gamma).


title: Learning Object Arrangements in 3D Scenes using Human Context

abstract: We consider the problem of learning object arrangements in a 3D scene. The key idea here is to learn how objects relate to human poses based on their affordances, ease of use and reachability. In contrast to modeling object-object relationships, modeling human-object relationships scales linearly in the number of objects. We design appropriate density functions based on 3D spatial features to capture this. We learn the distribution of human poses in a scene using a variant of the Dirichlet process mixture model that allows sharing of the density function parameters across the same object types. Then we can reason about arrangements of the objects in the room based on these meaningful human poses. In our extensive experiments on 20 different rooms with a total of 47 objects, our algorithm predicted correct placements with an average error of 1.6 meters from ground truth. In arranging five real scenes, it received a score of 4.3/5 compared to 3.7 for the best baseline method.


title: An Iterative Locally Linear Embedding Algorithm

abstract: Local Linear embedding (LLE) is a popular dimension reduction method. In this paper, we first show LLE with nonnegative constraint is equivalent to the widely used Laplacian embedding. We further propose to iterate the two steps in LLE repeatedly to improve the results. Thirdly, we relax the kNN constraint of LLE and present a sparse similarity learning algorithm. The final Iterative LLE combines these three improvements. Extensive experiment results show that iterative LLE algorithm significantly improve both classification and clustering results.


title: Estimating the Hessian by Back-propagating Curvature

abstract: In this work we develop Curvature Propagation (CP), a general technique for efficiently computing unbiased approximations of the Hessian of any function that is computed using a computational graph. At the cost of roughly two gradient evaluations, CP can give a rank-1 approximation of the whole Hessian, and can be repeatedly applied to give increasingly precise unbiased estimates of any or all of the entries of the Hessian. Of particular interest is the diagonal of the Hessian, for which no general approach is known to exist that is both efficient and accurate. We show in experiments that CP turns out to work well in practice, giving very accurate estimates of the Hessian of neural networks, for example, with a relatively small amount of work. We also apply CP to Score Matching, where a diagonal of a Hessian plays an integral role in the Score Matching objective, and where it is usually computed exactly using inefficient algorithms which do not scale to larger and more complex models.


title: Bayesian Efficient Multiple Kernel Learning

abstract: Multiple kernel learning algorithms are proposed to combine kernels in order to obtain a better similarity measure or to integrate feature representations coming from different data sources. Most of the previous research on such methods is focused on the computational efficiency issue. However, it is still not feasible to combine many kernels using existing Bayesian approaches due to their high time complexity. We propose a fully conjugate Bayesian formulation and derive a deterministic variational approximation, which allows us to combine hundreds or thousands of kernels very efficiently. We briefly explain how the proposed method can be extended for multiclass learning and semi-supervised learning. Experiments with large numbers of kernels on benchmark data sets show that our inference method is quite fast, requiring less than a minute. On one bioinformatics and three image recognition data sets, our method outperforms previously reported results with better generalization performance.


title: Utilizing Static Analysis and Code Generation to Accelerate Neural Networks

abstract: As datasets continue to grow, neural network (NN) applications are becoming increasingly limited by both the amount of available computational power and the ease of developing high-performance applications. Researchers often must have expert systems knowledge to make their algorithms run efficiently. Although available computing power increases rapidly each year, algorithm efficiency is not able to keep pace due to the use of general purpose compilers, which are not able to fully optimize specialized application domains. Within the domain of NNs, we have the added knowledge that network architecture remains constant during training, meaning the architecture's data structure can be statically optimized by a compiler. In this paper, we present SONNC, a compiler for NNs that utilizes static analysis to generate optimized parallel code. We show that SONNC's use of static optimizations make it able to outperform hand-optimized C++ code by up to 7.8X, and MATLAB code by up to 24X. Additionally, we show that use of SONNC significantly reduces code complexity when using structurally sparse networks.


title: Semi-Supervised Collective Classification via Hybrid Label Regularization

abstract: Many classification problems involve data instances that are interlinked with each other, such as webpages connected by hyperlinks. Techniques for "collective classification" (CC) often increase accuracy for such data graphs, but usually require a fully-labeled training graph. In contrast, we examine how to improve the semi-supervised learning of CC models when given only a sparsely-labeled graph, a common situation. We first describe how to use novel combinations of classifiers to exploit the different characteristics of the relational features vs. the non-relational features. We also extend the ideas of "label regularization" to such hybrid classifiers, enabling them to leverage the unlabeled data to bias the learning process. We find that these techniques, which are efficient and easy to implement, significantly increase accuracy on three real datasets. In addition, our results explain conflicting findings from prior related studies.


title: Variational Inference in Non-negative Factorial Hidden Markov Models for Efficient Audio Source Separation

abstract: The past decade has seen substantial work on the use of non-negative matrix factorization and its probabilistic counterparts for audio source separation. Although able to capture audio spectral structure well, these models neglect the non-stationarity and temporal dynamics that are important properties of audio. The recently proposed non-negative factorial hidden Markov model (N-FHMM) introduces a temporal dimension and improves source separation performance. However, the factorial nature of this model makes the complexity of inference exponential in the number of sound sources. Here, we present a Bayesian variant of the N-FHMM suited to an efficient variational inference algorithm, whose complexity is linear in the number of sound sources. Our algorithm performs comparably to exact inference in the original N-FHMM but is significantly faster. In typical configurations of the N-FHMM, our method achieves around a 30x increase in speed.


title: Inferring Latent Structure From Mixed Real and Categorical Relational Data

abstract: We consider analysis of relational data (a matrix), in which the rows correspond to subjects (e.g., people) and the columns correspond to attributes. The elements of the matrix may be a mix of real and categorical. Each subject and attribute is characterized by a latent binary feature vector, and an inferred matrix maps each row-column pair of binary feature vectors to an observed matrix element. The latent binary features of the rows are modeled via a multivariate Gaussian distribution with low-rank covariance matrix, and the Gaussian random variables are mapped to latent binary features via a probit link. The same type construction is applied jointly to the columns. The model infers latent, low-dimensional binary features associated with each row and each column, as well correlation structure between all rows and between all columns.


title: A Combinatorial Algebraic Approach for the Identifiability of Low-Rank Matrix Completion

abstract: In this paper, we review the problem of matrix completion and expose its intimate relations with algebraic geometry, combinatorics and graph theory. We present the first necessary and sufficient combinatorial conditions for matrices of arbitrary rank to be identifiable from a set of matrix entries, yielding theoretical constraints and new algorithms for the problem of matrix completion. We conclude by algorithmically evaluating the tightness of the given conditions and algorithms for practically relevant matrix sizes, showing that the algebraic-combinatoric approach can lead to improvements over state-of-the-art matrix completion methods.


title: On Causal and Anticausal Learning

abstract: We consider the problem of function estimation in the case where an underlying causal model can be inferred. This has implications for popular scenarios such as covariate shift, concept drift, transfer learning and semi-supervised learning. We argue that causal knowledge may facilitate some approaches for a given problem, and rule out others. In particular, we formulate a hypothesis for when semi-supervised learning can help, and corroborate it with empirical results.


title: An Efficient Approach to Sparse Linear Discriminant Analysis

abstract: We present a novel approach to the formulation and the resolution of sparse Linear Discriminant Analysis (LDA). Our proposal, is based on penalized Optimal Scoring. It has an exact equivalence with penalized LDA, contrary to the multi-class approaches based on the regression of class indicator that have been proposed so far. Sparsity is obtained thanks to a group-Lasso penalty that selects the same features in all discriminant directions. Our experiments demonstrate that this approach generates extremely parsimonious models without compromising prediction performances. Besides prediction, the resulting sparse discriminant directions are also amenable to low-dimensional representations of data. Our algorithm is highly efficient for medium to large number of variables, and is thus particularly well suited to the analysis of gene expression data.


title: Compositional Planning Using Optimal Option Models

abstract: In this paper we introduce a framework for option model composition. Option models are temporal abstractions that, like macro-operators in classical planning, jump directly from a start state to an end state. Prior work has focused on constructing option models from primitive actions, by intra-option model learning; or on using option models to construct a value function, by inter-option planning. We present a unified view of intra- and inter-option model learning, based on a major generalisation of the Bellman equation. Our fundamental operation is the recursive composition of option models into other option models. This key idea enables compositional planning over many levels of abstraction. We illustrate our framework using a dynamic programming algorithm that simultaneously constructs optimal option models for multiple subgoals, and also searches over those option models to provide rapid progress towards other subgoals.


title: Estimation of Simultaneously Sparse and Low Rank Matrices

abstract: The paper introduces a penalized matrix estimation procedure aiming at solutions which are sparse and low-rank at the same time. Such structures arise in the context of social networks or protein interactions where underlying graphs have adjacency matrices which are block-diagonal in the appropriate basis. We introduce a convex mixed penalty which involves $\ell_1$-norm and trace norm simultaneously. We obtain an oracle inequality which indicates how the two effects interact according to the nature of the target matrix. We bound generalization error in the link prediction problem. We also develop proximal descent strategies to solve the optimization problem efficiently and evaluate performance on synthetic and real data sets.


title: A Split-Merge Framework for Comparing Clusterings

abstract: Clustering evaluation measures are frequently used to evaluate the performance of algorithms. However, most measures are not properly normalized and ignore some information in the inherent structure of clusterings. We model the relation between two clusterings as a bipartite graph and propose a general component-based decomposition formula based on the components of the graph. Most existing measures are examples of this formula. In order to satisfy consistency in the component, we further propose a split-merge framework for comparing clusterings of different data sets. Our framework gives measures that are conditionally normalized, and it can make use of data point information, such as feature vectors and pairwise distances. We use an entropy-based instance of the framework and a coreference resolution data set to demonstrate empirically the utility of our framework over other measures.


title: Similarity Learning for Provably Accurate Sparse Linear Classification

abstract: In recent years, the crucial importance of metrics in machine learning algorithms has led to an increasing interest for optimizing distance and similarity functions. Most of the state of the art focus on learning Mahalanobis distances (requiring to fulfill a constraint of positive semi-definiteness) for use in a local k-NN algorithm. However, no theoretical link is established between the learned metrics and their performance in classification. In this paper, we make use of the formal framework of good similarities introduced by Balcan et al. to design an algorithm for learning a non PSD linear similarity optimized in a nonlinear feature space, which is then used to build a global linear classifier. We show that our approach has uniform stability and derive a generalization bound on the classification error. Experiments performed on various datasets confirm the effectiveness of our approach compared to state-of-the-art methods and provide evidence that (i) it is fast, (ii) robust to overfitting and (iii) produces very sparse classifiers.


title: Discovering Support and Affiliated Features from Very High Dimensions

abstract: In this paper, a novel learning paradigm is presented to automatically identify groups of informative and correlated features from very high dimensions. Specifically, we explicitly incorporate correlation measures as constraints and then propose an efficient embedded feature selection method using recently developed cutting plane strategy. The benefits of the proposed algorithm are two-folds. First, it can identify the optimal discriminative and uncorrelated feature subset to the output labels, denoted here as Support Features, which brings about significant improvements in prediction performance over other state of the art feature selection methods considered in the paper. Second, during the learning process, the underlying group structures of correlated features associated with each support feature, denoted as Affiliated Features, can also be discovered without any additional cost. These affiliated features serve to improve the interpretations on the learning tasks. Extensive empirical studies on both synthetic and very high dimensional real-world datasets verify the validity and efficiency of the proposed method.


title: Maximum Margin Output Coding

abstract: In this paper we study output coding for multi-label prediction. For a multi-label output coding to be discriminative, it is important that codewords for different label vectors are significantly different from each other. In the meantime, unlike in traditional coding theory, codewords in output coding are to be predicted from the input, so it is also critical to have a predictable label encoding. To find output codes that are both discriminative and predictable, we first propose a max-margin formulation that naturally captures these two properties. We then convert it to a metric learning formulation, but with an exponentially large number of constraints as commonly encountered in structured prediction problems. Without a label structure for tractable inference, we use overgenerating (i.e., relaxation) techniques combined with the cutting plane method for optimization. In our empirical study, the proposed output coding scheme outperforms a variety of existing multi-label prediction methods for image, text and music classification.


title: The Landmark Selection Method for Multiple Output Prediction

abstract: Conditional modeling x \to y is a central problem in machine learning. A substantial research effort is devoted to such modeling when x is high dimensional. We consider, instead, the case of a high dimensional y, where x is either low dimensional or high dimensional. Our approach is based on selecting a small subset y_L of the dimensions of y, and proceed by modeling (i) x \to y_L and (ii) y_L \to y. Composing these two models, we obtain a conditional model x \to y that possesses convenient statistical properties. Multi-label classification and multivariate regression experiments on several datasets show that this model outperforms the one vs. all approach as well as several sophisticated multiple output prediction methods.


title: A Dantzig Selector Approach to Temporal Difference Learning

abstract: LSTD is a popular algorithm for value function approximation. Whenever the number of features is larger than the number of samples, it must be paired with some form of regularization. In particular, L1-regularization methods tend to perform feature selection by promoting sparsity, and thus, are well-suited for high-dimensional problems. However, since LSTD is not a simple regression algorithm, but it solves a fixed--point problem, its integration with L1-regularization is not straightforward and might come with some drawbacks (e.g., the P-matrix assumption for LASSO-TD). In this paper, we introduce a novel algorithm obtained by integrating LSTD with the Dantzig Selector. We investigate the performance of the proposed algorithm and its relationship with the existing regularized approaches, and show how it addresses some of their drawbacks.


title: Cross Language Text Classification via Subspace Co-Regularized Multi-View Learning

abstract: In many multilingual text classification problems, the documents in different languages often share the same set of categories. To reduce the labeling cost of training a classification model for each individual language, it is important to transfer the label knowledge gained from one language to another language by conducting cross language classification. In this paper we develop a novel subspace co-regularized multi-view learning method for cross language text classification. This method is built on parallel corpora produced by machine translation. It jointly minimizes the training error of each classifier in each language while penalizing the distance between the subspace representations of parallel documents. Our empirical study on a large set of cross language text classification tasks shows the proposed method consistently outperforms a number of inductive methods, domain adaptation methods, and multi-view learning methods.


title: Modeling Images using Transformed Indian Buffet Processes

abstract: Latent feature models are attractive for image modeling, since images generally contain multiple objects. However, many latent feature models ignore that objects can appear at different locations or require pre-segmentation of images. While the transformed Indian buffet process (tIBP) provides a method for modeling transformation-invariant features in unsegmented binary images, its current form is inappropriate for real images because of its computational cost and modeling assumptions. We combine the tIBP with likelihoods appropriate for real images and develop an efficient inference, using the cross-correlation between images and features, that is theoretically and empirically faster than existing inference techniques. Our method discovers reasonable components and achieve effective image reconstruction in natural images.


title: Subgraph Matching Kernels for Attributed Graphs

abstract: We propose graph kernels based on subgraph matchings, i.e. structure-preserving bijections between subgraphs. While recently proposed kernels based on common subgraphs (Wale et al., 2008; Shervashidze et al., 2009) in general can not be applied to attributed graphs, our approach allows to rate mappings of subgraphs by a flexible scoring scheme comparing vertex and edge attributes by kernels. We show that subgraph matching kernels generalize several known kernels. To compute the kernel we propose a graph-theoretical algorithm inspired by a classical relation between common subgraphs of two graphs and cliques in their product graph observed by Levi (1973). Encouraging experimental results on a classification task of real-world graphs are presented.


title: Apprenticeship Learning for Model Parameters of Partially Observable Environments

abstract: We consider apprenticeship learning, i.e., having an agent learn a task by observing an expert demonstrating the task in a partially observable environment when the model of the environment is uncertain. This setting is useful in applications where the explicit modeling of the environment is difficult, such as a dialogue system. We show that we can extract information about the environment model by inferring action selection process behind the demonstration, under the assumption that the expert is choosing optimal actions based on knowledge of the true model of the target environment. Proposed algorithms can achieve more accurate estimates of POMDP parameters and better policies from a short demonstration, compared to methods that learns only from the reaction from the environment.


title: Greedy Algorithms for Sparse Reinforcement Learning

abstract: Feature selection and regularization are becoming increasingly prominent tools in the efforts of the reinforcement learning (RL) community to expand the reach and applicability of RL. One approach to the problem of feature selection is to impose a sparsity-inducing form of regularization on the learning method. Recent work on $L_1$ regularization has adapted techniques from the supervised learning literature for use with RL. Another approach that has received renewed attention in the supervised learning community is that of using a simple algorithm that greedily adds new features. Such algorithms have many of the good properties of the $L_1$ regularization methods, while also being extremely efficient and, in some cases, allowing theoretical guarantees on recovery of the true form of a sparse target function from sampled data. This paper considers variants of orthogonal matching pursuit (OMP) applied to reinforcement learning. The resulting algorithms are analyzed and compared experimentally with existing $L_1$ regularized approaches. We demonstrate that perhaps the most natural scenario in which one might hope to achieve sparse recovery fails; however, one variant, OMP-BRM, provides promising theoretical guarantees under certain assumptions on the feature dictionary. Another variant, OMP-TD, empirically outperforms prior methods both in approximation accuracy and efficiency on several benchmark problems.


title: Flexible Modeling of Latent Task Structures in Multitask Learning

abstract: Multitask learning algorithms are typically designed assuming some fixed, a priori known latent structure shared by all the tasks. However, it is usually unclear what type of latent task structure is the most appropriate for a given multitask learning problem. Ideally, the "right" latent task structure should be learned in a data-driven manner. We present a flexible, nonparametric Bayesian model that posits a mixture of factor analyzers structure on the tasks. The nonparametric aspect makes the model expressive enough to subsume many existing models of latent task structures (e.g, mean-regularized tasks, clustered tasks, low-rank or linear/non-linear subspace assumption on tasks, etc.). Moreover, it can also learn more general task structures, addressing the shortcomings of such models. We present a variational inference algorithm for our model. Experimental results on synthetic and real-world datasets, on both regression and classification problems, demonstrate the effectiveness of the proposed method.


title: An Adaptive Algorithm for Finite Stochastic Partial Monitoring

abstract: We present a new anytime algorithm that achieves near-optimal regret for any instance of finite stochastic partial monitoring. In particular, the new algorithm achieves the minimax regret, within logarithmic factors, for both "easy" and "hard" problems. For easy problems, it additionally achieves logarithmic individual regret. Most importantly, the algorithm is adaptive in the sense that if the opponent strategy is in an "easy region" of the strategy space then the regret grows as if the problem was easy. As an implication, we show that under some reasonable additional assumptions, the algorithm enjoys an O(\sqrt{T}) regret in Dynamic Pricing, proven to be hard by Bartok et al. (2011).


title: The Nonparanormal SKEPTIC

abstract: We propose a semiparametric approach, named nonparanormal skeptic, for estimating high dimensional undirected graphical models. In terms of modeling, we consider the nonparanormal family proposed by Liu et al (2009). In terms of estimation, we exploit nonparametric rank-based correlation coefficient estimators including the Spearman's rho and Kendall's tau. In high dimensional settings, we prove that the nonparanormal skeptic achieves the optimal parametric rate of convergence in both graph and parameter estimation. This result suggests that the nonparanormal graphical models are a safe replacement of the Gaussian graphical models, even when the data are Gaussian.


title: Using Tracker as a Pedagogical Tool for Understanding Projectile Motion

abstract: This paper reports the use of Tracker as a pedagogical tool in the effective learning and teaching of projectile motion in physics. When computer model building learning processes is supported and driven by video analysis data, this free Open Source Physics (OSP) tool can provide opportunities for students to engage in active inquiry-based learning. We discuss the pedagogical use of Tracker to address some common misconceptions of projectile motion by allowing students to test their hypothesis by juxtaposing their mental models against the analysis of real life videos. Initial research findings suggest that allowing learners to relate abstract physics concepts to real life through coupling computer modeling with traditional video analysis could be an innovative and effective way to learn projectile motion. 2015 Resources: http://iwant2study.org/ospsg/index.php/interactive-resources/physics/02-newtonian-mechanics/01-kinematics/174-projectile-motion


title: Living Devices

abstract: The physiological tradition of biological research analyzes biological systems using reduced descriptions much as an engineer uses a 'black box' description of an amplifier. Simple models have been used by physiologists for a very long time. Physiologists have successfully analyzed a broad range of biological systems using a 'device-oriented' approach similar to the approach an engineer would use to investigate her devices. The present generation views biology through the powerful lenses of structural and (molecular) dynamic analysis, understandably enough because of the beauty and power of the analysis, and the ease of using these structures with present freely available software. The problem is that these powerful lenses offer such magnification that the engineering approach cannot be seen. High magnification means limited field of view, because the (spatial) dynamic range cannot cover everything. The function of the structures and molecular dynamics cannot be seen in the work of many biologists, probably because function cannot be immediately seen in the structures and molecular dynamics they compute. It is just as important for biologists to measure the inputs and outputs of their systems as it is to measure their structures. It seems clear, at least to one physiologist, that this research will be catalyzed by assuming that most biological systems are devices that can be analyzed with the same strategies one would use to analyze engineering devices. Thinking today about your biological preparation as a device tells you what experiments to do tomorrow. An important task for many of us is to transmit the physiological tradition to the next generation of biophysicists to help them adapt traditional questions to the new length scales and techniques of molecular and atomic biology.


title: A problem with the Pusey, Barrett, Rudolph analysis of the reality of the quantum state

abstract: The analysis of Pusey, Barrett and Rudolph aims to show there can be no objective physical reality which underlies, and is more general than, the state vector. But there appears to be a gap in their reasoning. To show their result, they use entangled states of independent systems. However, no specific experimental arrangement to detect these entangled states has been proposed. Thus their argument as it stands does not fully show there is a detectable conflict between the predictions of quantum mechanics and the existence of an underlying reality. And it is not clear that one can devise the necessary measuring device.


title: Dispersal of molecular clouds by ionising radiation

abstract: The role of feedback from massive stars is believed to be a key element in the evolution of molecular clouds. We use high-resolution 3D SPH simulations to explore the dynamical effects of a single O7 star located at the centre of a molecular cloud with mass 10^4M_sun and radius 6.4pc. The initial internal structure of the cloud is characterised by its fractal dimension, D=2.0 - 2.8, and its log-normal density PDF. (i) As regards star formation, in the short term ionising feedback is positive, in the sense that star formation occurs much more quickly in gas that is compressed by the high pressure of the ionised gas. However, in the long term ionising feedback is negative, in the sense that most of the cloud is dispersed with an outflow rate of up to ~0.01M_sun/yr, on a timescale comparable with the sound-crossing time for the ionised gas (~1-2Myr), and triggered star formation is therefore limited to a few percent of the cloud's mass. (ii) As regards the morphology of the ionisation fronts (IFs) bounding the HII region and the systematics of outflowing gas, we distinguish two regimes. For low D<=2.2, the initial cloud is dominated by large-scale structures, so the neutral gas tends to be swept up into a few extended coherent shells, and the ionised gas blows out through a few large holes between these shells; we term these HII regions "shell-dominated". Conversely, for high D>=2.6, the initial cloud is dominated by small-scale structures, and these are quickly overrun by the advancing IF, thereby producing neutral pillars whilst the ionised gas blows out through a large number of small holes between the pillars; we term these HII regions "pillar-dominated". (iii) As regards the injection of bulk kinetic energy, by ~1Myr, the expansion of the HII region has delivered a rms velocity of ~6km/s; this represents less than 0.1% of the total energy radiated by the O7 star.


title: Ionization Parameter as a Diagnostic of Radiation and Wind Pressures in H II Regions and Starburst Galaxies

abstract: The ionization parameter U is potentially useful for measuring radiation pressure feedback from massive star clusters, as it reflects the radiation-to-gas-pressure ratio and is readily derived from mid-infrared line ratios. We consider several effects which determine the apparent value of U in HII regions and galaxies. An upper limit is set by the compression of gas by radiation pressure. The pressure from stellar winds and the presence of neutral clumps both reduce U for a given radiation intensity. The most intensely irradiated regions are selectively dimmed by internal dust absorption of ionizing photons, inducing observational bias on galactic scales. We explore these effects analytically and numerically, and use them to interpret previous observational results. We find that radiation confinement sets the upper limit log_10 U = -1 seen in individual regions. Unresolved starbursts display a maximum value of ~ -2.3. While lower, this is also consistent with a large portion of their HII regions being radiation dominated, given the different technique used to interpret unresolved regions, and given the bias caused by dust absorption. We infer that many individual, strongly illuminated regions cannot be dominated by stellar winds, and that even when averaged on galactic scales, shocked wind pressures cannot be large compared to radiation pressure. Therefore, most HII regions cannot be adiabatic wind bubbles. Our models imply a metallicity dependence in the physical structure and dust attenuation of radiation-dominated regions, both of which should vary strongly across a critical metallicity of about one-twentieth solar.


title: Early-time velocity autocorrelation for charged particles diffusion and drift in static magnetic turbulence

abstract: Using test-particle simulations, we investigate the temporal dependence of the two-point velocity correlation function for charged particles scattering in a time-independent spatially fluctuating magnetic field derived from a three-dimensional isotropic turbulence power spectrum. Such a correlation function allowed us to compute the spatial coefficients of diffusion both parallel and perpendicular to the average magnetic field. Our simulations confirm the dependence of the perpendicular diffusion coefficient on turbulence energy density and particle energy predicted previously by a model for early-time charged particle transport. Using the computed diffusion coefficients, we exploit the particle velocity autocorrelation to investigate the time-scale over which the particles "decorrelate" from the solution to the unperturbed equation of motion. Decorrelation time-scales are evaluated for parallel and perpendicular motions, including the drift of the particles from the local magnetic field line. The regimes of strong and weak magnetic turbulence are compared for various values of the ratio of the particle gyroradius to the correlation length of the magnetic turbulence. Our simulation parameters can be applied to energetic particles in the interplanetary space, cosmic rays at the supernova shocks, and cosmic-rays transport in the intergalactic medium.

categories: astro-ph.HE astro-ph.EP physics.plasm-ph physics.space-ph


title: The abundance of (not just) dark matter haloes

abstract: We study the effect of baryons on the abundance of structures and substructures in a Lambda-CDM cosmology, using a pair of high resolution cosmological simulations from the GIMIC project. Both simulations use identical initial conditions, but while one contains only dark matter, the other also includes baryons. We find that gas pressure, reionisation, supernova feedback, stripping, and truncated accretion systematically reduce the total mass and the abundance of structures below ~10^12 solar masses compared to the pure dark matter simulation. Taking this into account and adopting an appropriate detection threshold lowers the abundance of observed galaxies with maximum circular velocities below 100 km/s, significantly reducing the reported discrepancy between Lambda-CDM and the measured HI velocity function of the ALFALFA survey. We also show that the stellar-to-total mass ratios of galaxies with stellar masses of ~10^5 - 10^7 solar masses inferred from abundance matching of the (sub)halo mass function to the observed galaxy mass function increase by a factor of ~2. In addition, we find that an important fraction of low-mass subhaloes are completely devoid of stars. Accounting for the presence of dark subhaloes below 10^10 solar masses further reduces the abundance of observable objects, and leads to an additional increase in the inferred stellar-to-total mass ratio by factors of 2 - 10 for galaxies in haloes of 10^9 - 10^10 solar masses. This largely reconciles the abundance matching results with the kinematics of individual dwarf galaxies in Lambda-CDM. We propose approximate corrections to the masses of objects derived from pure dark matter calculations to account for baryonic effects.


title: The Zurich Environmental Study of Galaxies in Groups along the Cosmic Web. III. Galaxy Photometric Measurements and the Spatially-Resolved Color Properties of Early- and Late-Type Satellites in Diverse Environments

abstract: We present photometric measurements for the galaxies - and when possible their bulges and disks - in the 0.05<z<0.0585 groups of the Zurich Environmental Study (ZENS); these measurements include (B-I) colors, color gradients and maps, color dispersions, as well as stellar masses and star-formation rates. The ZENS galaxies are classified into quenched, moderately star-forming, and strongly star-forming using a combination of spectral features and FUV-to-optical colors; this approach optimally distinguishes quenched systems from dust-reddened star-forming galaxies. The latter contribute up to 50% to the (B-I) "red sequence" at ~10^10Msun. At fixed morphological or spectral type, we find that galaxy stellar masses are largely independent of environment, and especially of halo mass. As a first utilization of our photometric database, we study, at fixed stellar mass and Hubble type, how (B-I) colors, color gradients and color dispersion of disk satellites depend on group mass (M_GROUP), group-centric distance (R/R_200) and large-scale structure overdensity. The strongest environmental trend is found for disk-dominated satellites with M_GROUP and R/R_200. At M<10^10 Msun, disk-dominated satellites are redder in the inner regions of the groups than in the outer parts. At M>10^10 Msun, these satellites have shallower color gradients in higher mass groups and in the cores of groups compared with lower mass groups and the outskirts of groups. Stellar population analyses and semi-analytic models suggest that disk-dominated satellites undergo quenching of star formation in their outer disks, on timescales ~2 Gyr, as they progressively move inside the group potential.


title: Enhancing lepton flavour violation in the supersymmetric inverse seesaw beyond the dipole contribution

abstract: In minimal supersymmetric models the $Z$-penguin usually provides sub-dominant contributions to charged lepton flavour violating observables. In this study, we consider the supersymmetric inverse seesaw in which the non-minimal particle content allows for dominant contributions of the $Z$-penguin to several lepton flavour violating observables. In particular, and due to the low-scale (TeV) seesaw, the penguin contribution to, for instance, $\Br(\mu \to 3e)$ and $\mu-e$ conversion in nuclei, allows to render some of these observables within future sensitivity reach. Moreover, we show that in this framework, the $Z$-penguin exhibits the same non-decoupling behaviour which had previously been identified in flavour violating Higgs decays in the Minimal Supersymmetric Standard Model.


title: Reflection algebras for sl(2) and gl(1|1)

abstract: We present a generalization the G. Letzter's theory of quantum symmetric pairs of semisimple Lie algebras for the case of quantum affine algebras. We then study solutions of the reflection equation for the quantum affine algebras sl(2) and gl(1|1) and their Yangian limit for singlet (diagonal) and vector (non-diagonal) boundary conditions. We construct the corresponding quantum affine coideal subalgebras that are based on the quantum symmetric pairs, and the (generalized) twisted Yangians.


title: Constraints on Anomalous Fluid in Arbitrary Dimensions

abstract: Using the techniques developed in arxiv: 1203.3544 we compute the universal part of the equilibrium partition function characteristic of a theory with multiple abelian U(1) anomalies in arbitrary even spacetime dimensions. This contribution is closely linked to the universal anomaly induced transport coefficients in hydrodynamics which have been studied before using entropy techniques. Equilibrium partition function provides an alternate and a microscopically more transparent way to derive the constraints on these transport coefficients. We re-derive this way all the known results on these transport coefficients including their polynomial structure which has recently been conjectured to be linked to the anomaly polynomial of the theory. Further we link the local description of anomaly induced transport in terms of a Gibbs current to the more global description in terms of the partition function .


title: Chemistry in Infrared Dark Cloud Clumps: a Molecular Line Survey at 3 mm

abstract: We have observed 37 Infrared Dark Clouds (IRDCs), containing a total of 159 clumps, in high-density molecular tracers at 3 mm using the 22-meter ATNF Mopra Telescope located in Australia. After determining kinematic distances, we eliminated clumps that are not located in IRDCs and clumps with a separation between them of less than one Mopra beam. Our final sample consists of 92 IRDC clumps. The most commonly detected molecular lines are (detection rates higher than 8%): N2H+, HNC, HN13C, HCO+, H13CO+, HCN, C2H, HC3N, HNCO, and SiO. We investigate the behavior of the different molecular tracers and look for chemical variations as a function of an evolutionary sequence based on Spitzer IRAC and MIPS emission. We find that the molecular tracers behave differently through the evolutionary sequence and some of them can be used to yield useful relative age information. The presence of HNC and N2H+ lines do not depend on the star formation activity. On the other hand, HC3N, HNCO, and SiO are predominantly detected in later stages of evolution. Optical depth calculations show that in IRDC clumps the N2H+ line is optically thin, the C2H line is moderately optically thick, and HNC and HCO+ are optically thick. The HCN hyperfine transitions are blended, and, in addition, show self-absorbed line profiles and extended wing emission. These factors combined prevent the use of HCN hyperfine transitions for the calculation of physical parameters. Total column densities of the different molecules, except C2H, increase with the evolutionary stage of the clumps. Molecular abundances increase with the evolutionary stage for N2H+ and HCO+. The N2H+/HCO+ and N2H+/HNC abudance ratios act as chemical clocks, increasing with the evolution of the clumps.


title: On the stability criteria for equatorial circular orbits in Galactic Dynamics: I.Newtonian Thin Disks

abstract: We make a revision of the stability criteria for equatorial circular orbits, obtained from the epicyclic approximation, which is widely used in Newtonian models for axisymmetric galaxies. We find that, for the case of thin disk models, the criteria of vertical stability must be reformulated, due to the discontinuity in the gravitational field. We show that, for a model characterized by a surface mass density $\Sigma$, the necessary and sufficient condition to have vertically stable orbits is that $\Sigma>0$. On the other hand, the criteria for radial stability is the same as in thick diks, i.e. that the epicyclic frequency is positive.


title: THC: a new high-order finite-difference high-resolution shock-capturing code for special-relativistic hydrodynamics

abstract: We present THC: a new high-order flux-vector-splitting code for Newtonian and special-relativistic hydrodynamics designed for direct numerical simulations of turbulent flows. Our code implements a variety of different reconstruction algorithms, such as the popular weighted essentially non oscillatory and monotonicity-preserving schemes, or the more specialised bandwidth-optimised WENO scheme that has been specifically designed for the study of compressible turbulence. We show the first systematic comparison of these schemes in Newtonian physics as well as for special-relativistic flows. In particular we will present the results obtained in simulations of grid-aligned and oblique shock waves and nonlinear, large-amplitude, smooth adiabatic waves. We will also discuss the results obtained in classical benchmarks such as the double-Mach shock reflection test in Newtonian physics or the linear and nonlinear development of the relativistic Kelvin-Helmholtz instability in two and three dimensions. Finally, we study the turbulent flow induced by the Kelvin-Helmholtz instability and we show that our code is able to obtain well-converged velocity spectra, from which we benchmark the effective resolution of the different schemes.


title: Graphene p-n junctions with nonuniform Rashba spin-orbit coupling

abstract: Linear conductance of graphene-based p-n junctions with Rashba spin-orbit coupling is considered theoretically. A square potential step is used to model the junctions, while the coupling is introduced in terms of the Kane-Mele model (C.L. Kane and E.J. Mele, Phys.Rev.Lett. 95, 226801(2005)). The main objective is a description of electronic transport in junctions where Rashba parameter is nonuniform. Such a nonuniformity can appear when graphene is asymmetrically covered with atomic layers, or when Rashba coupling is strongly dependent on electric field. It is shown that conductance is significantly modified by the considered nonuniformity, which is most clearly manifested by an anomalous minimum at a certain potential step height.


title: An Abstract Approach to Stratification in Linear Logic

abstract: We study the notion of stratification, as used in subsystems of linear logic with low complexity bounds on the cut-elimination procedure (the so-called light logics), from an abstract point of view, introducing a logical system in which stratification is handled by a separate modality. This modality, which is a generalization of the paragraph modality of Girard's light linear logic, arises from a general categorical construction applicable to all models of linear logic. We thus learn that stratification may be formulated independently of exponential modalities; when it is forced to be connected to exponential modalities, it yields interesting complexity properties. In particular, from our analysis stem three alternative reformulations of Baillot and Mazza's linear logic by levels: one geometric, one interactive, and one semantic.


title: New Local Duals in Eternal Inflation

abstract: Global-local duality is the equivalence of seemingly different regulators in eternal inflation. For example, the light-cone time cutoff (a global measure, which regulates time) makes the same predictions as the causal patch (a local measure that cuts off space). We show that global-local duality is far more general. It rests on a redundancy inherent in any global cutoff: at late times, an attractor regime is reached, characterized by the unlimited exponential self-reproduction of a certain fundamental region of spacetime. An equivalent local cutoff can be obtained by restricting to this fundamental region. We derive local duals to several global cutoffs of interest. The New Scale Factor Cutoff is dual to the Short Fat Geodesic, a geodesic of fixed infinitesimal proper width. Vilenkin's CAH Cutoff is equivalent to the Hubbletube, whose width is proportional to the local Hubble volume. The famous youngness problem of the Proper Time Cutoff can be readily understood by considering its local dual, the Incredible Shrinking Geodesic.


title: Formation of Magnetic Microphases in Ca$_3$Co$_2$O$_6$

abstract: We study a frustrated quantum Ising model relevant for Ca$_3$Co$_2$O$_6$ that consists of a triangular lattice of weakly-coupled ferromagnetic (FM) chains. According to our quantum Monte Carlo simulations, the chains become FM and form a three-sublattice "up-up-down" structure for $T \leq T_\text{CI}$. In contrast, long-period spin-density-wave (SDW) {\it microphases} are stabilized along the chains for $T_\text{CI} < T < T_c$. Our mean field solutions reveal a quasi-continuous temperature dependence of the SDW wavelength, implying the existence of metastable states that explain the very slow dynamics observed in Ca$_3$Co$_2$O$_6$. We also discuss implications of microphases for the related multiferroic compounds Ca$_3$CoMnO$_6$ and Lu$_2$MnCoO$_6$.


title: Correlation functions for Schr\"odinger backgrounds

abstract: We work out the holographic dictionary for the three-dimensional Schr\"odinger spacetimes. The first step in our analysis involves the correct identification of the dual sources from the radial expansion of the bulk fields, which turns out to be surprisingly subtle. We discuss in detail the holographic renormalization procedure at the linearized level and holographically compute the two-point functions of the energy-momentum tensor and an irrelevant vector operator. We discuss the appearance of multi-trace counterterms, parametrize the scheme dependence in our results, identify the non-relativistic Ward identities and compare them with expectations in the literature. Our results lead to valuable general insights regarding holography for spacetimes that are not of an asymptotically AdS form.


title: A classification of spherically symmetric spacetimes

abstract: A complete classification of locally spherically symmetric four-dimensional Lorentzian spacetimes is given in terms of their local conformal symmetries. The general solution is given in terms of canonical metric types and the associated conformal Lie algebras. The analysis is based upon the local conformal decomposition into 2+2 reducible spacetimes and the Petrov type. A variety of physically meaningful example spacetimes are discussed.


title: Polarization of light scattered by large aggregates

abstract: Study of cosmic dust and planetary aerosols indicate that some of them contain a large number of aggregates of the size that significantly exceeds the wavelengths of the visible light. In some cases such large aggregates may dominate in formation of the light scattering characteristics of the dust. In this paper we present the results of computer modelling of light scattering by aggregates that contain more than 1000 monomers of submicron size and study how their light scattering characteristics, specifically polarization, change with phase angle and wavelength. Such a modeling became possible due to development of a new version of MSTM (Multi Sphere T-Matrix) code for parallel computing. The results of the modeling are applied to the results of comet polarimetric observations to check if large aggregates dominate in formation of light scattering by comet dust. We compare aggregates of different structure and porosity. We show that large aggregates of more than 98% porosity (e.g. ballistic cluster-cluster aggregates) have angular dependence of polarization almost identical to the Rayleigh one. Large compact aggregates (less than 80% porosity) demonstrate the curves typical for solid particles. This rules out too porous and too compact aggregates as typical comet dust particles. We show that large aggregates not only can explain phase angle dependence of comet polarization in the near infrared but also may be responsible for the wavelength dependence of polarization, which can be related to their porosity.


title: Gyrating Schrodinger Geometries and Non-Relativistic Field Theories

abstract: We propose homogeneous metrics of Petrov type III that describe gyrating Schrodinger geometries as duals to some non-relativistic field theories, in which the Schrodinger symmetry is broken further so that the phase space has a linear dependence of the momentum in a selected direction. We show that such solutions can arise in four-dimensional Einstein-Weyl supergravity as well as higher-dimensional extended gravities with quadratic curvature terms coupled to a massive vector. In Einstein-Weyl supergravity, the gyrating Schrodinger solutions can be supersymmetric, preserving 1/4 of the supersymmetry. We obtain the exact Green function in the phase space associated with a bulk free massive scalar.


title: A "Twistor String" Inspired Formula For Tree-Level Scattering Amplitudes in N=8 SUGRA

abstract: We propose a new formulation of the complete tree-level S-matrix of N = 8 supergravity. The new formula for n particles in the k R-charge sector is an integral over the Grassmannian G(2,n) and uses the Veronese map into G(k,n). The image of a point in G(2,n) is required to be in the "complement" of a 2|8-plane thus making the SU(8) R-symmetry manifest. The integrand is the ratio of two determinants. The numerator is an analog of Hodges' recent determinant formula for MHV amplitudes. The denominator is a 2(n+k-2) x 2(n+k-2) minor of a 2(n+k) x 2(n+k) matrix of rank 2(n+k-2). Just as Hodges' formula does for MHV amplitudes, our integrand makes the complete invariance under Sn manifest for all sectors. The validity of the new formula follows from two surprising facts. One is the equivalence of Hodges' MHV formula and the Kawai-Lewellen-Tye (KLT) formula when kinematic invariants are allowed to be off-shell in a novel way. We give a proof of this for any number of particles. The second fact is an orthogonality property of the solutions to the polynomial equations defining the Veronese embedding. Explicit proof of the orthogonality is given for all amplitudes in all R-charge sectors with eight or less particles thus providing non-trivial evidence for our proposal.


title: Quasilocal Conservation Laws: Why We Need Them

abstract: We argue that conservation laws based on the local matter-only stress-energy-momentum tensor (characterized by energy and momentum per unit volume) cannot adequately explain a wide variety of even very simple physical phenomena because they fail to properly account for gravitational effects. We construct a general quasi}local conservation law based on the Brown and York total (matter plus gravity) stress-energy-momentum tensor (characterized by energy and momentum per unit area), and argue that it does properly account for gravitational effects. As a simple example of the explanatory power of this quasilocal approach, consider that, when we accelerate toward a freely-floating massive object, the kinetic energy of that object increases (relative to our frame). But how, exactly, does the object acquire this increasing kinetic energy? Using the energy form of our quasilocal conservation law, we can see precisely the actual mechanism by which the kinetic energy increases: It is due to a bona fide gravitational energy flux that is exactly analogous to the electromagnetic Poynting flux, and involves the general relativistic effect of frame dragging caused by the object's motion relative to us.


title: On the retrieval of attenuation from the azimuthally averaged coherency of a diffuse field

abstract: It has been suggested that seismic attenuation \alpha can be inferred from comparisons of empirical coherencies (the cross spectra of pre-whitened ambient seismic noise records) with attenuated Bessel functions Jo(\omega r/c) exp(-\alpha r). Analysis shows here, however, that coherency depends strongly on the directionality of ambient noise intensity. Even if coherency is azimuthally averaged, the suggested attenuation dependence exp(-\alpha r) does not apply. Indeed in highly directional noise fields, coherency is independent of attenuation. It is also argued that spatial and azimuthal averages of coherency can incur phase cancellations related to variations in wavespeed that may mimic factors like exp(-\alpha r). Inference of attenuation from comparison of empirical coherencies to Jo(\omega r/c) exp(-\alpha r) is problematic.