Lexical and Hierarchical Topic Regression
Inspired by a two-level theory from political science that unifies agenda setting and ideological framing, we propose supervised hierarchical latent Dirichlet allocation (SHLDA), which jointly captures documents' multi-level topic structure and their polar response variables. Our model extends the nested Chinese restaurant processes to discover tree-structured topic hierarchies and uses both per-topic hierarchical and per-word lexical regression parameters to model response variables. SHLDA improves prediction on political affiliation and sentiment tasks in addition to providing insight into how topics under discussion are framed.
This paper proposes a hierarchical, supervised topic 
      model that builds on ideas from hierarchical LDA using the nested CRP 
      and from supervised LDA, which associates topics with regression 
      coefficients. The proposed model further extends hierarchical LDA by 
      allowing each sentence in a document to chose its own path through the 
      topic tree, and sharing of paths for sentences within the same 
      document is modelled through a document-level CRP where the customers 
      are sentences and the dishes are paths through the topic hierarchy 
      ("combos"). Another difference to supervised LDA is that each word is 
      also associated with its own regression coefficient to model 
      topic-independent effects. The authors empirically demonstrate the 
      performance of the model on 3 data sets and show that the proposed 
      model outperforms LDA+linear regression as well as supervised LDA. 
      The idea of modelling framing in the proposed way by associating 
      regression coefficients with hierarchically arranged topics is 
      interesting and appears novel. The paper is clearly written and the 
      proposed model and inference procedure seem technically sound. The 
      quantitative evaluation shows that the proposed model (slightly) 
      outperforms simple baselines and the qualitative evaluation suggests that 
      the discovered topic hierarchy has the desired property of separating 
      topics into different aspects or ways of framing them. The 
      main question I had while reading this paper was: Is such a complicated 
      model really necessary for modelling this what/how distinction? Given 
      that the depth of the hierarchy is fixed to 3 in the experiments 
      (why?), it seems that at least an infinitely deep hierarchy is not 
      necessary. If interpretability of the resulting topics is desired, 
      would it make sense to model the binary split (positive/negative, 
      republican/democrat) explicitly, e.g. by explicitly splitting each 
      topic into pos/neutral/neg parts with fixed regression coefficients 
      and then associating with each document a distribution over those 
      parts? In terms of quantitative experiments, it would be nice to 
      tease apart the contributions of the individual changes to existing 
      models: how does sLDA perform if lexical regression parameters are 
      added, or a supervised response variable is added to a plain HLDA 
      model? Also, it would be helpful to see some descriptive statistics 
      about the inference and the learned models, e.g. how many nodes are in 
      the hierarchy, branching factor, iterations until convergence, 
      etc.