Recent deep multi-task learning (MTL) has been witnessed its success in alleviating data scarcity of some task by utilizing domain-specific knowledge from related tasks. Nonetheless, several major issues of deep MTL, including the effectiveness of sharing mechanisms, the efficiency of model complexity and the flexibility of network architectures, still remain largely unaddressed. To this end, we propose a novel generalized latent-subspace based knowledge sharing mechanism for linking task-specific models, namely tensor ring multi-task learning (TRMTL). TRMTL has a highly compact representation, and it is very effective in transferring task-invariant knowledge while being super flexible in learning task-specific features, successfully mitigating the dilemma of both negative-transfer in lower layers and under-transfer in higher layers. Under our TRMTL, it is feasible for each task to have heterogenous input data dimensionality or distinct feature sizes at different hidden layers. Experiments on a variety of datasets demonstrate our model is capable of significantly improving each single task's performance, particularly favorable in scenarios where some of the tasks have insufficient data.
Summary:  This paper studies deep multi-task learning. Prior papers have studied various knowledge sharing approaches for deep multi-task learning including hard and soft sharing schemes. And some soft sharing schemes have used tensor decompositions (including TT, and Tucker). This paper fuses this line of work with the recently proposed Tensor-Ring decomposition in order to obtain Tensor Ring (TR)-based soft sharing for multi-task learning. The results show some improvement over prior deep MTL methods based on other tensor factorisation methods.
Strengths:
+ Nice extension of existing line of work tensor-factorisation based MTL.
+ More flexibility for controlling shared/unshared portions of weights compared to DMTRL. 
+ Improves on previous methods results.
+ Experiments evaluate how MTL methods relate with various amounts of training data on each task.
Weaknesses:
- Novelty/significance is limited. 
- Writing. Many things are not clearly and intuitively explained. Some claims are not adequately justified. 
- Introduces more hyper parameters to tune.
- Results may rely on hyper parameter tuning. 
Comments:
1. Novelty.  Existing studies already established the template of different tensor factorisation methods (TT, Tucker) being possible to plug into deep networks for different kinds of soft-sharing MTL. Meanwhile, TR decomposition is taken off the shelf; (and as it’s been applied for compression before, this is not the first time TR decomposition has been used in a CNN context either). Therefore this is an A+B paper and a high bar should be met for the additional analysis, insight, or performance improvements that should provided.
2. Lots of writing issues:
2.1 Many things are not explained transparently enough at best (or major over-claim at worst). For example: 
2.1.1 Paper claims the benefit that each task can have its own I/O dimensionality. However if TR-decomp is “circularly connected” TT-decomp (Fig 1), then this seems not to happen automatically. So it should be unpacked more clearly how this is achieved. 
2.1.2 Paper claims favourable ability to use more private cores than TT, where only one core is private. However circular TT would also seem to have one private core by default (the core with a task axis). So I suspect something else is going on, but this is completely unclear and should be explained more transparently. Furthermore it should be justified if whatever modifications do enable these properties are definitely a unique property of TR-decomp, or could also be applied to TT-decomp. 
2.1.3 Statement “TRMTL generalizes to allow the layer-wise weight to be represented by a relatively lager number of latent cores” unclear: generalises what? larger number of cores than what? Than TT? The previous presentation suggests TT and TR should have same number of cores.  
2.1.4 Statements like “TR enjoys the property of circular dimensional permutation invariance” are made without any explanation about what is the implication of this for neural networks and multi-task learning.  
2.2 Many claims are inaccurate or not adequately backed up by theory or experiment. EG: (i) Paper claims to include DMTRL as a special case. But it only subsumes DMTRL-TT, not DMTRL-Tucker. Because TR-decomp does not include Tucker-decomp as an exact special case.  (ii) Sentences “TR-ranks are usually smaller than TT-ranks” are assertions without verification. 
2.3 Sentences are taken verbatim from other papers, plagiarism. For example: “TR model is more flexible than TT, because TR-ranks can be equally distributed in the cores, but TT-ranks have a relatively fixed pattern”  is verbatim from Zhao’16 TR-decomp paper. 
3. Hyperparameters: This paper apparently gains some practical benefit due to the notion of shared/unshared cores. However, this also introduces  additional hyper parameters (E.g., each layers private proportion “c”) to tune besides the ranks. Unlike the rank that can be pre-estimated by reconstruction error, this one seems to require tuning by cross-validation. This is not scalable. 
4. Hyperparameters+Tuning: Hyperparameters Private proportion, “sharing pattern”, IO dimension seem to be tuned by accuracy.( “We test different sharing patterns and report the ones with the best accuracies”). This is even less scalable, and additional tuning makes it unsurprising it surpasses other models performance.
5. Insight & Analysis. All the core selection & public/private core selection are treated as black box optimisation. No insight is given about what turns out to be useful to share or not, and how consistent this is, etc.