Action segmentation as a milestone towards building automatic systems to understand untrimmed videos has received considerable attention in the recent years. It is typically being modeled as a sequence labeling problem but contains intrinsic and sufficient differences than text parsing or speech processing. In this paper, we introduce a novel hybrid temporal convolutional and recurrent network (TricorNet), which has an encoder-decoder architecture: the encoder consists of a hierarchy of temporal convolutional kernels that capture the local motion changes of different actions; the decoder is a hierarchy of recurrent neural networks that are able to learn and memorize long-term action dependencies after the encoding stage. Our model is simple but extremely effective in terms of video sequence labeling. The experimental results on three public action segmentation datasets have shown that the proposed model achieves superior performance over the state of the art.
I will be upfront: I have already reviewed this paper when it was submitted to NIPS 2017, so this review is based heavily on the NIPS submission. 
I am quite concerned that this paper has been resubmitted as it is, word by word, character by character. The authors could have benefited from the feedback they obtained from the reviewers of their last submissions to improved their paper, but nothing has been done. Even very easy remarks, like bolding errors (see below) have been kept in the paper.
The proposed paper describes a method for video action segmentation, a task where the video must be temporally densely labeled by assigned an action (sub) class to each frame. The method proceeds by extracting frame level features using convolutional networks and then passing a temporal encoder-decoder in 1D over the video, using fully supervised training.
On the positive side, the method has been tested on 3 different datasets, outperforming the baselines (recent methods from 2016) on 2 of them.
My biggest concern with the paper is novelty. A significant part of the paper is based on reference [Lea et al. 2017], the differences being quite incremental. The frame-level features are the same as in [Lea et al. 2017], and the basic encoder-decoder strategy is also taken from [Lea et al. 2017]. The encoder is also the same. Even details are reproduced, as the choice of normalized Relu activations.
The main difference seems to me that the decoder is not convolutional, but a recurrent network.
The encoder-decoder architecture seems to be surprisingly shallow, with only K=2 layers at each side.
The paper is well written and can be easily understood. However, a quite large amount of space is wasted on obvious and known content, as for example the basic equation for a convolutional layer (equation (1)) and the following half page of text and equations of LSTM and Bi-directional LSTM networks. This is very well known and the space can be used for more details on the paper's contributions.
While the paper is generally well written, there are a couple of exceptions in the form of ambiguous sentences, for example the lines before section 3.
There is a bolding error in table 2, where the proposed method is not state of the art (as indicated) w.r.t. to the accuracy metric.
To sum it up, the positive aspect of nicely executed experiments is contrasted by low novelty of the method.  To be honest, I am not totally sure whether the contribution of the paper should be considered as a new method or as architectural optimizations of an existing one. This is corroborated by the experimental results on the first two datasets (tables 2 and 3): on 50 salads, where ref. [Lea et al. 2017]. seems currently to obtain state of the art performance, the improvement obtained by the proposed method allows it to get state of the art performance. On GTEA, where [Lea et al. 2017] does not currently deliver state of the art performance, the proposed method performs (slightly) better than [Lea et al. 2017] but does not obtain state of the art performance.
On the third dataset, JIGSAWS, reference [Lea et al. 2017]. has not been tested, which is peculiar given the closeness.