<p dir="auto">I have created a device that supports integer constant tensors (because I would like to be able to support the inputs to reshape etc...), but not the integer maths operators (because I'm not going to accelerate those initially).</p>
<p dir="auto">The gradient generation logic creates trees of integer maths.  The constants for this maths are placed on the device, while the maths nodes themselves are placed on the CPU (as expected).</p>
<p dir="auto">This isn't very sensible.</p>
<p dir="auto">I think it would be better that constants are not placed on a device, unless they are feeding downstream nodes that are also on the same device.</p>
<p dir="auto">There may be some classes of constant (ones with a specific memory placement, very large ones), which would end up on a different device to their downstream nodes, but I think that the default should be to keep them together.</p>
<p dir="auto">Are tf.TFRecord(num_parallel_reads) and interleave(num_parallel_calls) duplicated? what is the behavior if they are both set &gt; 1?  And the difference of shuffle before repeat and shuffle after repeat? I think the design of Dataset is too complicated, we need a more clear approach.</p>