An In-Depth Look at Google's Tensor Processing Unit Architecture
Interesting points I took from the paper[1]:* They actually started deploying them in 2015, they're probably already hard at work on a new version!* The TPU only operates on 8-bit integers (and 16-bit at half speed), whereas CPU/GPUs are 32-bit floating point. They point out in the discussion section that they did have an 8-bit CPU version of one of the benchmarks, and the TPU was ~3.5x faster.* Used via TensorFlow.* They don't really break out hardware vs hardware for each model, it seems like the TPU suffers a lot whenever there's a really large number of weights and layers that it must handle - but they don't break out the performance on each model individually, so it's hard to see whether the TPU offers an advantage over the GPU for arbitrary networks.[1] https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk...