  Several text classification tasks such as sentiment analysis, news
categorization, multi-label classification and opinion classification are
challenging problems even for modern deep learning networks. Recently, Capsule
Networks (CapsNets) are proposed for image classification. It has been shown
that CapsNets have several advantages over Convolutional Neural Networks
(CNNs), while their validity in the domain of text has been less explored. In
this paper, we propose a novel hybrid architecture viz., BGCapsule, which is a
Capsule model preceded by an ensemble of Bidirectional Gated Recurrent Units
(BiGRU) for several text classification tasks. We employed an ensemble of
Bidirectional GRUs for feature extraction layer preceding the primary capsule
layer. The hybrid architecture, after performing basic pre-processing steps,
consists of five layers: an embedding layer based on GloVe, a BiGRU based
ensemble layer, a primary capsule layer, a flatten layer and fully connected
ReLU layer followed by a fully connected softmax layer. In order to evaluate
the effectiveness of BGCapsule, we conducted extensive experiments on five
benchmark datasets (ranging from 10,000 records to 700,000 records) including
Movie Review (MR Imdb 2005), AG News dataset, Dbpedia ontology dataset, Yelp
Review Full dataset and Yelp review polarity dataset. These benchmarks cover
several text classification tasks such as news categorization, sentiment
analysis, multiclass classification, multi-label classification and opinion
classification. We found that our proposed architecture (BGCapsule) achieves
better accuracy compared to the existing methods without the help of any
external linguistic knowledge such as positive sentiment keywords and negative
sentiment keywords. Further, BGCapsule converged faster compared to other
extant techniques.

Computation and Language