  In sponsored search, keyword recommendations help advertisers to achieve much
better performance within limited budget. Many works have been done to mine
numerous candidate keywords from search logs or landing pages. However, the
strategy to select from given candidates remains to be improved. The existing
relevance-based, popularity-based and regular combinatorial strategies fail to
take the internal or external competitions among keywords into consideration.
In this paper, we regard keyword recommendations as a combinatorial
optimization problem and solve it with a modified pointer network structure.
The model is trained on an actor-critic based deep reinforcement learning
framework. A pre-clustering method called Equal Size K-Means is proposed to
accelerate the training and testing procedure on the framework by reducing the
action space. The performance of framework is evaluated both in offline and
online environments, and remarkable improvements can be observed.

Information Retrieval;Machine Learning;Machine Learning