<p dir="auto">Is it feasible to extend tfcompile to support training?</p>
<p dir="auto">Our reinforcement learning pipelines sometimes spend significant time in Python side of session.run because networks are small, and need lots of session.run calls since each .run leads to an interaction with a simulator (ie, Mujoco, gym or universe)</p>
<p dir="auto">So you could have have 30k session.run calls where actual computation takes 200 usec, and Python session.run overhead is another 150 usec.</p>
<p dir="auto">This could be improved if the network was tfcompiled and we didn't have to use session.run, however, this would need tfcompile to be extended to support multiple sets of fetches for the compiled object. IE</p>
<ol dir="auto">
<li>network initialization, input: weights, output: None</li>
<li>network forward op: input: observation, output: action</li>
<li>network train op: input: observations, actions output: None</li>
<li>network variable read: input: None, output: weights</li>
</ol>
<h2 dir="auto">URL(s) with the issue:</h2>
<p dir="auto"><a href="https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/estimator/SessionRunHook" rel="nofollow">https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/estimator/SessionRunHook</a></p>
<h2 dir="auto">Description of issue (what needs changing):</h2>
<h3 dir="auto">Clear description</h3>
<p dir="auto">Yes clear description</p>
<h3 dir="auto">Correct links</h3>
<p dir="auto">The link to the source code correct.</p>
<h3 dir="auto">Parameters defined</h3>
<p dir="auto">All parameters defined and formatted correctly.</p>
<h3 dir="auto">Returns defined</h3>
<p dir="auto">Return values are defined.</p>
<h3 dir="auto">Raises listed and defined</h3>
<p dir="auto">No raises listed and defined.</p>
<h3 dir="auto">Usage example</h3>
<p dir="auto">No usage examples provided</p>
<h3 dir="auto">Request visuals, if applicable</h3>
<p dir="auto">No visuals included</p>