Topic Models with Survival Supervision: Archetypal Analysis and Neural Approaches
We introduce two approaches to topic modeling supervised by survival analysis. Both approaches predict time-to-event outcomes while simultaneously learning topics over features that help prediction. The high-level idea is to represent each data point as a distribution over topics using some underlying topic model. Then each data point*s distribution over topics is fed as input to a survival model. The topic and survival models are jointly learned. The two approaches we propose differ in the generality of topic models they can learn. The first approach finds topics via archetypal analysis, a nonnegative matrix factorization method that optimizes over a wide class of topic models encompassing latent Dirichlet allocation (LDA), correlated topic models, and topic models based on the ``anchor word** assumption; the resulting survival-supervised variant solves an alternating minimization problem. Our second approach builds on recent work that approximates LDA in a neural net framework. We add a survival loss layer to this neural net to form an approximation to survival-supervised LDA. Both of our approaches can be combined with a variety of survival models. We demonstrate our approach on two survival datasets, showing that survival-supervised topic models can achieve competitive time-to-event prediction accuracy while outputting clinically interpretable topics.
This paper explores the problem of simultaneous learning for topic modeling and survival prediction. Their contributions are: (1) integrating two different topic modeling approaches with survival models for joint learning and (2) showing results on two medical datasets with some brief analysis of what topics are recovered. This is an interesting task to be using with topic modeling. I appreciate that the approach could be used in medical systems where interpretability of models is very important. I think the high-level idea of the algorithms is fine, but the main drawback is that the experimental section needs to be expanded on, perhaps with larger datasets and more analysis. Unless more experiments are provided, I might lean towards rejection. I think it*s ok that some of the empirical results are inconclusive, but it should lead to more quantitative error analysis. I definitely think the smaller size of the datasets could be one factor in the negative results, which is why I think the authors should try experimenting with a larger dataset. More specific comments and suggestions: - Each dataset is small (371 and 1981 data points respectively). This may be too small to effectively train some of these neural models, which I believe may be one reason why they underperform, especially on the pancreatitis data. In order for the experiments to be more conclusive, maybe you can switch to a different dataset. - Related to the size of the data, I suspect that many of the differences in the table are statistically insignificant. Can the authors please specify which of the results are significantly better than the others? - While I liked the examples in Section F of the appendix, they are difficult to compare between models, and not all of the models are included. More quantitative analysis is needed to really understand how the models’ learned topics differ in terms of coherency, cohesiveness, interpretability, etc. Providing really detailed quantitative comparisons might strengthen the analysis section of the paper. - I liked the preciseness of the descriptions of different algorithms in the background section. They were all explained nicely and easy to understand. However, the background section is a bit too detailed at times. Authors may consider condensing a bit more.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Topic Models with Survival Supervision: Archetypal Analysis and Neural Approaches
We introduce two approaches to topic modeling supervised by survival analysis. Both approaches predict time-to-event outcomes while simultaneously learning topics over features that help prediction. The high-level idea is to represent each data point as a distribution over topics using some underlying topic model. Then each data point*s distribution over topics is fed as input to a survival model. The topic and survival models are jointly learned. The two approaches we propose differ in the generality of topic models they can learn. The first approach finds topics via archetypal analysis, a nonnegative matrix factorization method that optimizes over a wide class of topic models encompassing latent Dirichlet allocation (LDA), correlated topic models, and topic models based on the ``anchor word** assumption; the resulting survival-supervised variant solves an alternating minimization problem. Our second approach builds on recent work that approximates LDA in a neural net framework. We add a survival loss layer to this neural net to form an approximation to survival-supervised LDA. Both of our approaches can be combined with a variety of survival models. We demonstrate our approach on two survival datasets, showing that survival-supervised topic models can achieve competitive time-to-event prediction accuracy while outputting clinically interpretable topics.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper explores the problem of simultaneous learning for topic modeling and survival prediction. Their contributions are: (1) integrating two different topic modeling approaches with survival models for joint learning and (2) showing results on two medical datasets with some brief analysis of what topics are recovered. This is an interesting task to be using with topic modeling. I appreciate that the approach could be used in medical systems where interpretability of models is very important. I think the high-level idea of the algorithms is fine, but the main drawback is that the experimental section needs to be expanded on, perhaps with larger datasets and more analysis. Unless more experiments are provided, I might lean towards rejection. I think it*s ok that some of the empirical results are inconclusive, but it should lead to more quantitative error analysis. I definitely think the smaller size of the datasets could be one factor in the negative results, which is why I think the authors should try experimenting with a larger dataset. More specific comments and suggestions: - Each dataset is small (371 and 1981 data points respectively). This may be too small to effectively train some of these neural models, which I believe may be one reason why they underperform, especially on the pancreatitis data. In order for the experiments to be more conclusive, maybe you can switch to a different dataset. - Related to the size of the data, I suspect that many of the differences in the table are statistically insignificant. Can the authors please specify which of the results are significantly better than the others? - While I liked the examples in Section F of the appendix, they are difficult to compare between models, and not all of the models are included. More quantitative analysis is needed to really understand how the models’ learned topics differ in terms of coherency, cohesiveness, interpretability, etc. Providing really detailed quantitative comparisons might strengthen the analysis section of the paper. - I liked the preciseness of the descriptions of different algorithms in the background section. They were all explained nicely and easy to understand. However, the background section is a bit too detailed at times. Authors may consider condensing a bit more.
assistant
This paper explores the problem of simultaneous learning for topic modeling and survival prediction. Their contributions are: (1) integrating two different topic modeling approaches with survival models for joint learning and (2) showing results on two medical datasets with some brief analysis of what topics are recovered. This is an interesting task to be using with topic modeling. I appreciate that the approach could be used in medical systems where interpretability of models is very important. I think the high-level idea of the algorithms is fine, but the main drawback is that the experimental section needs to be expanded on, perhaps with larger datasets and more analysis. Unless more experiments are provided, I might lean towards rejection. I think it*s ok that some of the empirical results are inconclusive, but it should lead to more quantitative error analysis. I definitely think the smaller size of the datasets could be one factor in the negative results, which is why I think the authors should try experimenting with a larger dataset. More specific comments and suggestions: - Each dataset is small (371 and 1981 data points respectively). This may be too small to effectively train some of these neural models, which I believe may be one reason why they underperform, especially on the pancreatitis data. In order for the experiments to be more conclusive, maybe you can switch to a different dataset. - Related to the size of the data, I suspect that many of the differences in the table are statistically insignificant. Can the authors please specify which of the results are significantly better than the others? - While I liked the examples in Section F of the appendix, they are difficult to compare between models, and not all of the models are included. More quantitative analysis is needed to really understand how the models’ learned topics differ in terms of coherency, cohesiveness, interpretability, etc. Providing really detailed quantitative comparisons might strengthen the analysis section of the paper. - I liked the preciseness of the descriptions of different algorithms in the background section. They were all explained nicely and easy to understand. However, the background section is a bit too detailed at times. Authors may consider condensing a bit more.