I make a summary of pros and cons of dynamic graph. List as follows:
pros:

easy to debug
flexible control flow,

example of dynamic layers 155
example of dynamic hidden units



cons:

slower

each forward pass defines a new computational graph.
each backward pass for a new graph needs new building and new derivative computing.
static graphs can optimize the graph up front while dynamic graph can not do it cite 155




some points I am not sure.


Does the dynamic graph need more GPU IO?


can we simulate all dynamic graph with static graph?


more reference
papers about DCG 
How to think with dynamic graphs

keep updating
welcome to add more features that I have missed. and more dynamic graph examples, use case