title: Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUs
summary: This paper presents a SYCL implementation of Multi-Layer Perceptrons (MLPs),
  which targets and is optimized for the Intel Data Center GPU Max 1550. To
  increase the performance, our implementation minimizes the slow global memory
  accesses by maximizing the data reuse within the general register file and the
  shared local memory by fusing the operations in each layer of the MLP. We show
  with a simple roofline model that this results in a significant increase in the
  arithmetic intensity, leading to improved performance, especially for
  inference. We compare our approach to a similar CUDA implementation for MLPs
  and show that our implementation on the Intel Data Center GPU outperforms the
  CUDA implementation on Nvidia's H100 GPU by a factor up to 2.84 in inference
  and 1.75 in training. The paper also showcases the efficiency of our SYCL
  implementation in three significant areas: Image Compression, Neural Radiance
  Fields, and Physics-Informed Machine Learning. In all cases, our implementation
  outperforms the off-the-shelf Intel Extension for PyTorch (IPEX) implementation
  on the same Intel GPU by up to a factor of 30 and the CUDA PyTorch version on
  Nvidia's H100 GPU by up to a factor 19. The code can be found at
  https://github.com/intel/tiny-dpcpp-nn.
paper_authors._id: 6603878f527470e01645fe69
paper_authors.name: Kai Yuan
paper_authors._id: 6603878f527470e01645fe6a
paper_authors.name: Christoph Bauinger
paper_authors._id: 6603878f527470e01645fe6b
paper_authors.name: Xiangyi Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-27T09:32:01.042Z
paper_authors._id: 6475250c6d4dda6f7c6e9dba
paper_authors.avatarUrl: /avatars/ffccf38a96881a7db3dc19954bd55dbe.svg
paper_authors.fullname: Xiangyi Zhang
paper_authors.type: user
paper_authors.user: Shawnz516
paper_authors._id: 6603878f527470e01645fe6c
paper_authors.name: Pascal Baehr
paper_authors._id: 6603878f527470e01645fe6d
paper_authors.name: Matthias Kirchhart
paper_authors._id: 6603878f527470e01645fe6e
paper_authors.name: Darius Dabert
paper_authors._id: 6603878f527470e01645fe6f
paper_authors.name: Adrien Tousnakhoff
paper_authors._id: 6603878f527470e01645fe70
paper_authors.name: Pierre Boudier
paper_authors._id: 6603878f527470e01645fe71
paper_authors.name: Michael Paulitsch
paper_title: Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUs
paper_summary: This paper presents a SYCL implementation of Multi-Layer Perceptrons (MLPs),
  which targets and is optimized for the Intel Data Center GPU Max 1550. To
  increase the performance, our implementation minimizes the slow global memory
  accesses by maximizing the data reuse within the general register file and the
  shared local memory by fusing the operations in each layer of the MLP. We show
  with a simple roofline model that this results in a significant increase in the
  arithmetic intensity, leading to improved performance, especially for
  inference. We compare our approach to a similar CUDA implementation for MLPs
  and show that our implementation on the Intel Data Center GPU outperforms the
  CUDA implementation on Nvidia's H100 GPU by a factor up to 2.84 in inference
  and 1.75 in training. The paper also showcases the efficiency of our SYCL
  implementation in three significant areas: Image Compression, Neural Radiance
  Fields, and Physics-Informed Machine Learning. In all cases, our implementation
  outperforms the off-the-shelf Intel Extension for PyTorch (IPEX) implementation
  on the same Intel GPU by up to a factor of 30 and the CUDA PyTorch version on
  Nvidia's H100 GPU by up to a factor 19. The code can be found at
  https://github.com/intel/tiny-dpcpp-nn.
paper_ai_keywords: Multi-Layer Perceptrons (MLPs) Intel Data Center GPU Max 1550 global memory general register file shared local memory fusion roofline model arithmetic intensity inference CUDA Nvidia's H100 GPU Image Compression Neural Radiance Fields Physics-Informed Machine Learning Intel Extension for PyTorch (IPEX)