Autonomously discovering temporally extended actions, or skills, is a longstanding goal of hierarchical reinforcement learning. We propose a new algorithm that combines skill chaining with deep neural networks to autonomously discover skills in high-dimensional, continuous domains. The resulting algorithm, deep skill chaining, constructs skills with the property that executing one enables the agent to execute another. We demonstrate that deep skill chaining significantly outperforms both non-hierarchical agents and other state-of-the-art skill discovery techniques in challenging continuous control tasks.
Summary
The authors tackle the problem of skill discovery by skill chaining. In particular, the authors claim two key contributions over the state of the art in option discovery 1) learn initiation sets 2) do not need to specify the number of options and this is also learned. Skill discovery is formalized by skill chaining; wherein the skills are chained backward from a goal state, and in a way, such that termination of an option is the initiation of the option that precedes in its chain. 
Interesting and useful aspects of this work are goal-based learning of where options should initiate (although clarification on goals would be crucial), the discussion of the optimality of the discovered solutions, scales up various ideas already proposed in Konidaris & Barto (2009b). 
Skill discovery through skill chaining, in particular, has not been extensively explored in a deep RL setting and serves as a useful contribution.
Detailed comments:
The authors introduce DSC with an intuition based on a goal. However, it is never mentioned where this goal comes from. In sec 3, the authors describe an algorithm based on the presumed goal. It is not clear to me: the goal is at one instance defined as \beta_{o_i} := \mathcal{I}_{o_i - 1}, and then in the next few lines, it is said “goal state of the MDP or the initiation condition of a previously learned option”. Is it correct to say this is an algorithm (sec 3) for goal-based option discovery using DSC, where the goal is specified in the MDP? 
Before going into the details of the architecture, it would be useful for the reader to have a formalism or a clear algorithm where at least the definition of what a goal constitutes here is clearly stated. 
Intra-option policy: It is nice to see that the option’s internal policy is not driven by the task-specific reward and has its internal reward. In the sparse reward setting: how is the subgoal reward chosen? In dense reward: what kind of distance metric is used? Please provide details.
Policy over options: The foremost option constructions seems a bit weird: is there a single goal specified which helps determine the termination condition of the global option? What would happen if there are multiple goals in the MDP? 
Initiation set classifier: This is an interesting approach. Although, it raises a natural question: “	
We continue adding to our chain of options in this fashion until a learned initiation set classifier contains the start state of the MDP. “ What happens if this never happens, or is this process is guaranteed to converge?
Experiments: Initiation set visuals are nice and interpretable. Experiments are not very convincing: authors mostly demonstrate results on control tasks that are specific to navigation and could do a more rigorous analysis by considering different tasks such as visual domains, robot manipulation tasks.  In particular, authors should compare their method and discuss other skill chaining approaches such as [1,2,3].
[1] Shoeleh, Farzaneh, and Masoud Asadpour. "Graph-based skill acquisition and transfer learning for continuous reinforcement learning domains." Pattern Recognition Letters 87 (2017): 104-116.
[2] Metzen, Jan Hendrik, and Frank Kirchner. "Incremental learning of skill collections based on intrinsic motivation." Frontiers in neurorobotics 7 (2013): 11.
[3] Konidaris, George, et al. "Robot learning from demonstration by constructing skill trees." The International Journal of Robotics Research 31.3 (2012): 360-375.
My primary concern is the amount of engineering that is needed to get this to work. There are multiple steps which do not seem to be sequentially dependent on the success of the previous steps (eg: global option construction needs a goal, the next options are only discovered once the global actions has been constructed, there is an initiation period and internal components of an option are all learned through multiple algorithms DQN, SMDP Q learning, DDPG). 
Overall:		
+Break the assumption that all options are everywhere.
+Number of options per task is also learned and does not need to be specified a priori.
+Chains skills in a smart way - which could be very useful for lifelong learning, but the approach, as it is, is limited by the goal of the MDP (whatever that means in this context, state or a Reward). 
+Options are driven by internal rewards
-The heavy machinery used in the core algorithm seems to be inspired by Konandiais 2009b. It would be very useful to discuss, clarify and contrast what is novel and what is borrowed from Konandiais 2009b as is.