Hi, all. I’m trying to run my model through tfjs with webgl backend. But I do not know how to get the time cost of model prediction. I did call new Date.now() before and after model.predict() and got a result around 100ms, it’s good enough. But between predict and my actural output with tensor.array(), I got A UI block around 700ms. Dose that mean it did not finish prediction in 100ms and need the following 700ms?
As tf doc say:
When an operation is called, like tf.matMul(a, b), the resulting tf.Tensor is synchronously returned, however the computation of the matrix multiplication may not actually be ready yet. This means the tf.Tensor returned is just a handle to the computation. When you call x.data() or x.array() , the values will resolve when the computation has actually completed.
It sounds like general operations return synchronously before they actually done, so do model.predict()?
Another thing border me is that if model.predict acts like the other general operations, then why do that take 100ms to return a not done handle? 10ms predict with a handle return, and 700ms to run model before output with tensor.array() would be more reasonable.