Thanks for that suggestion! It looks to me that this is happening due to asynchronous execution of kernels on  the GPU. When I run the same program on a CPU, the variation in run times I talk about above vanishes. The trace the profiler shows also seems to suggest the same thing. Does that seem right to you? If yes, is there a way to synchronize kernel execution in tensorflow, say after every call to model() in my code above?