Have you look at 3D convolutional neural networks, there are multiple notebooks in this Kaggle competition that show how this can be done. There are also multiple discussion posts on the same

  
      

      kaggle.com
  

  
    

RSNA-MICCAI Brain Tumor Radiogenomic Classification 1

  Predict the status of a genetic biomarker important for brain cancer treatment