Reinforcement learning agents that operate in diverse and complex environments can benefit from the structured decomposition of their behavior. Often, this is addressed in the context of hierarchical reinforcement learning, where the aim is to decompose a policy into lower-level primitives or options, and a higher-level meta-policy that triggers the appropriate behaviors for a given situation. However, the meta-policy must still produce appropriate decisions in all states. In this work, we propose a policy design that decomposes into primitives, similarly to hierarchical reinforcement learning, but without a high-level meta-policy. Instead, each primitive can decide for themselves whether they wish to act in the current state. We use an information-theoretic mechanism for enabling this decentralized decision: each primitive chooses how much information it needs about the current state to make a decision and the primitive that requests the most information about the current state acts in the world. The primitives are regularized to use as little information as possible, which leads to natural competition and specialization. We experimentally demonstrate that this policy architecture improves over both flat and hierarchical policies in terms of generalization.
The paper draws upon the idea of information bottleneck to do task decomposition so as to learn policy primitives similar to hierarchical reinforcement learning that combine together in a competitive manner to specialize in different parts of the task's domain. These policy primitives don't need a higher-level meta-policy to stitch them together. Instead the decision is made in a decentralized manner balancing the cost of information acquisition with maximizing rewards.
The paper seems to build on the idea of decomposing the task primitives that specialize in different parts of the state space. A related recent paper [1] which used similar ideas, albeit with a central coordinator to do task composition is missing from related works.
There are other issues with the paper as well. 
- Decision making is not exactly "decentralized"? Computing $Z$ in $\alpha_k$, still requires the values from other primitives?
- Sec 5.1 seems tacked on. Motion Imitation is not RL. Experiment details about motion imitation unclear from description. 
- In Figure 5: Zero-shot generalization, it's unclear from the plot whether it actually generalized to solve the tasks A and B. Relative frequency of activation are meaningless without reporting the actual performance on these environments. Where do the 4 indices come from? From the plot above there were only 2 indices but plot below has 4.
- Continuous control tasks required pretraining.
Overall the experiments seem a little underwhelming. More details on transfer performance without pretraining would be quite helpful. I like the idea of a competitive ensemble figuring out a useful task decomposition and using an information bottleneck like approach makes sense.
[1] https://dl.acm.org/citation.cfm?id=3331671
*Edit*: Updated the score after the reading the revision and the authors' responses.