## PINNs for Medical Image Analysis: A Survey

Chayan Banerjee a , Kien Nguyen a, ∗ , Olivier Salvado b , Truyen Tran , Clinton Fookes c a

- a Queensland University of Technology (QUT), 2 George St, Brisbane QLD 4000, Australia
- b Commonwealth Scientific and Industrial Research Organisation (CSIRO), 1 Technology Court, Pullenvale QLD 4069, Australia
- c Deakin University, 75 Pigdons Rd, Waurn Ponds, VIC 3216, Australia

## Abstract

The incorporation of physical information in machine learning frameworks is transforming medical image analysis (MIA). By integrating fundamental knowledge and governing physical laws, these models achieve enhanced robustness and interpretability. In this work, we explore the utility of physics-informed approaches for MIA (PIMIA) tasks such as registration, generation, classification, and reconstruction. We present a systematic literature review of over 80 papers on physics-informed methods dedicated to MIA. We propose a unified taxonomy to investigate what physics knowledge and processes are modelled, how they are represented, and the strategies to incorporate them into MIA models. We delve deep into a wide range of image analysis tasks, from imaging, generation, prediction, inverse imaging (super-resolution and reconstruction), registration, and image analysis (segmentation and classification). For each task, we thoroughly examine and present in a tabular format the central physics-guided operation, the region of interest (with respect to human anatomy), the corresponding imaging modality, the dataset used for model training, the deep network architecture employed, and the primary physical process, equation, or principle utilized. Additionally, we also introduce a novel metric to compare the performance of PIMIA methods across di ff erent tasks and datasets. Based on this review, we summarize and distil our perspectives on the challenges, open research questions, and directions for future research. We highlight key open challenges in PIMIA, including selecting suitable physics priors and establishing a standardized benchmarking platform.

Keywords: Physics-informed, Physics-guided, PINN, Medical Image Analysis

## 1. Introduction

Recent advancements in MIA have achieved exceptional performance in tasks like image registration, image generation, feature extraction, image classification and image reconstruction Litjens et al. (2017). These tasks are essential for aiding in diagnosis, treatment planning, and facilitating therapeutic interventions. Yet, these achievements often rely on complex, data-intensive models lacking robustness, interpretability, and alignment with governing physical rules and commonsense reasoning van der Velden et al. (2022); Kaviani et al. (2022).

scarcity of quality data, all these issues become more prominent and limit the e ffi cacy of data-driven machine learning models.

Applying generic machine learning approaches to medical image analysis presents unique challenges arising from the scarcity of data, the expense of data collection, rigorous requirements for interpretability, precision standards, and considerable inter-patient variability. Purely data-driven models demand a large quantity of training data, exhibit slow convergence, and often require large model architectures with millions of parameters to train. In the MIA context due to the

∗ Corresponding author

Email addresses: c.banerjee@qut.edu.au (Chayan Banerjee),

k.nguyenthanh@qut.edu.au

(Kien Nguyen),

olivier.salvado@csiro.au (Olivier Salvado),

truyen.tran@deakin.edu.au (Truyen Tran), c.fookesb@qut.edu.au (Clinton Fookes)

Physics equations and mathematical models can provide simplified analogies for understanding physical processes inside human anatomy and physiological mechanisms of imaging to facilitate medical image analysis tasks, o ff ering valuable insights even though they do not precisely mirror the complexity of these systems. For example, modelling blood flow as a non-newtonian fluid Toghraie et al. (2020) or use of Navier-Stokes equations as the foundational physics model for understanding and simulating cardiovascular hemodynamics Li et al. (2021). Additionally, medical image acquisition can be formulated as di ff erential equations, boundary conditions and constraints. For example, dynamics of CT perfusion scanning de Vries et al. (2023), MRI image acquisition physics Borges et al. (2024) and MRI hardware constraints Peng et al. (2022). Physics-based methods, grounded in fundamental equations and domain knowledge, promise improved reliability and system safety by embodying the actual physical relationships at play Balageas et al. (2010); Hu et al. (2019), suggesting a need for a paradigm shift towards incorporating physical laws in MIA.

Recent studies highlight the advantages of incorporating physics principles with machine learning, establishing a dominant paradigm in the field. Physics-informed machine learning (PIML), which integrates mathematical physics into ma- chine learning models, enhances solution relevance and efficiency. As illustrated in Fig. 1, physics-informed models provide a balance between numerical models and purely datadriven models with regard to the amount of data and the amount of physics. PIML approaches accelerate neural network training, improve model generalization with fewer data, and manage complex applications while ensuring solutions adhere to physical laws Karniadakis et al. (2021); Hao et al. (2022). Incorporating physical principles into machine learning, as seen in PIML approaches, significantly boosts the robustness, accuracy, e ffi ciency, and functionality of computer vision models Hao et al. (2022); Meng et al. (2022a); Karniadakis et al. (2021). As MIA is progressively integrating machine learning and deep learning techniques to enhance its capabilities, it can exhibit the same benefits that PIML can bring.

Figure 1: Physics-informed machine models leverage physics information to enhance e ffi ciency and plausibility, bridging data-driven and numerical models, vital for accurate representations across medical domains.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-7dd6542a-7076-4d6e-9dfa-f81866bef2ab.jpg)

Besides the similarity with generic machine learning and computer vision tasks, MIA also presents distinct challenges due to the nature of its input data, which primarily consists of complex and noisy medical images of human anatomy. These challenges include limited and high-cost data, stringent interpretability requirements, high precision demands, and significant inter-patient variability. Medical images capture intricate phenomena occurring within the human body, such as blood flow, abnormalities like tumours, metal artifacts, and dynamic organ activity such as the beating heart and lung respiration. They are generated through specialized imaging technologies and often contain machine-specific information reflecting the specific processing steps taken by healthcare practitioners. E ectively interpreting medical images necessitates adff vanced domain-specific expertise in medical physics, biomedical engineering, radiology, and biomechanics. This multidisciplinary knowledge is essential to address the intricate complexities inherent in biological tissue characteristics and the sophisticated techniques used in medical imaging modalities, such as MRI, CT, and ultrasound. These unique characteristics distinguish MIA from the analysis of generic images. This distinction underlines the need for models and methods specifically designed for MIA tasks, and leverages the unique physics information available, leading to the exploration of the PIMIA field. The paper reviews state-of-the-art physicsinformed strategies in MIA, focusing on how physics knowledge is integrated into algorithms, what the physical processes are modelled as priors, and the specialized network architec- tures or augmentations employed to weave in physics insights.

(a) PIML research over the years ( 2018 - 2024 ( till June ) ), clearly showing an exponential trend (shown in dashed line).

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-67e935f4-9857-4b50-8be4-f97f6c1937cb.jpg)

(b) Research share of di ff erent PIMIA tasks, from the surveyed papers.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-57c2f20c-2836-4e51-9e1d-473de3c2b544.jpg)

Figure 2: Plots showing the latest research trend on the topic of PIMIA, (a) Published work over the years and (b) Research share of PIMIA tasks.

The recent literature on PIMIA has shown an exponentially increasing trend as evident through the plethora of publications over the last 7 years, see Fig. 2a. The accompanying pie chart presents the research share w.r.t. di ff erent PIMIA tasks. Clearly showing a heightened focus on physics incorporation in PIMIA tasks like inverse imaging especially image reconstruction and predictive modeling.

Our contributions in this paper are summarized as follows:

- · We propose a unified taxonomy to investigate what physics knowledge processes are modelled, how they are / represented, and the strategies to incorporate them into MIA models.
- · We delve into a wide range of image analysis tasks, from imaging, generation, prediction, inverse imaging (superresolution and reconstruction), registration, and image analysis (segmentation and classification)
- · For each task, we thoroughly examine and present in an easily referenceable tabular format the central physicsguided operation, the region of interest (with respect to human anatomy), the corresponding imaging modality, the dataset used for model training, the deep network architecture employed, and finally, the primary physical process, equation, or principle utilized.

- · Based on the review of tasks, we summarize our perspectives on the challenges, open research questions, and directions for future research.

We highlight key open challenges in PIMIA, including selecting suitable physics priors and establishing a standardized benchmarking platform. While physics may have been integrated into the training pipeline, particularly through data augmentation, tasks such as classification, segmentation, and super-resolution have yet to fully leverage physics priors. These tasks are ranked in order of least to moderately e ffi cient utilization of physics information.

## Focus of the current survey :

The field of physics-informed machine learning (PIML) is rapidly expanding, highlighted by surveys across various domains Hao et al. (2022) including cyber-physical systems Rai and Sahu (2020), hydrology Zhang et al. (2019), fluid mechanics Cai et al. (2022), climate modelling Kashinath et al. (2021) and reinforcement learning Banerjee et al. (2023b).

Several surveys have examined the integration of physics and knowledge in MIA, often highlighting specific areas. These surveys provide valuable insights, yet there remains an opportunity for a more comprehensive overview that captures broader trends and advancements in the field. Banerjee et al. (2023a) conducted a comprehensive survey on the incorporation of physics in computer vision tasks, albeit with a broader focus on computer vision in general, lacking detailed coverage of MIA. Conversely, Liu et al. (2021) concentrated on anatomy-aided deep learning (DL) specifically for medical image segmentation, presenting a thorough examination of anatomical information categories and representation methods. Hammernik et al. (2023) provided an in-depth survey of physics-driven DL techniques for MRI reconstruction, encompassing both linear and nonlinear forward models as well as classical and DL approaches. Xia et al. (2023) scrutinized physics- and model-based data-driven methods tailored for low-dose computed tomography (LDCT), emphasizing the integration of imaging physics principles into deep network architectures. In another work Maier et al. (2022), the author discusses 'known operator learning' in medical imaging.

Unlike existing surveys, our study o ff ers a comprehensive review of approaches that integrate physics-based information into MIA (i.e PIMIA). While other reviews either concentrate on non-physics-specific methods or focus on specific imaging modalities, our study explores a diverse range of physicsbased approaches and their applications across various imaging modalities and regions of interest. Additionally, research in the realm of PIML adapted for MIA is relatively recent, significantly rising over the past 3-4 years, despite the longstanding presence of knowledge-guided approaches in MIA.

Inclusion and Exclusion Criteria for Article Selection: Our selection criteria are guided by several key considerations to ensure the relevance and quality of the literature we review. First, we prioritize works that are closely aligned with the core topics of our survey, particularly those that incorporate physics-based approaches rather than methods guided by domain-specific knowledge and non-physics-specific informa- tion . Additionally, while we generally focus on publications from peer-reviewed journals and conferences, we have made exceptions to include some pre-prints, recognizing the importance of capturing novel advancements in the field. Lastly, our review is strictly confined to literature pertaining to medical image analysis, ensuring that all included works are directly relevant to this specialized area.

To ensure transparency, we have provide a detailed description of these criteria below. This includes the specific databases and journals searched, the keywords used, and the temporal scope of the included literature. By applying these criteria, we aim to provide a comprehensive and unbiased synthesis of the existing research, addressing the key themes and advancements in the field.

Data for this review was systematically collected from leading journals such as Medical Image Analysis , Physics in Medicine and Biology , IEEE Transactions on Medical Imaging , Medical Physics , Nature Reviews Physics , and Nature Machine Intelligence . Our search encompassed major databases including PubMed, IEEE Xplore, ScienceDirect, SpringerLink, Wiley Online Library, ACM Digital Library, ArXiv, Scopus, JSTOR, and Google Scholar.The keywords searched included physics informed medical image analysis physics informed imaging , physics informed generation , physics incorporated predictive modeling , physics informed inverse imaging , and physics informed image registration . The papers surveyed were specifically selected from the past 7 years due to the rapid advancements in physics informed approaches in medical imaging and analysis during this period. The time frame ensures that the review reflects the most current trends and breakthroughs in PIMIA, providing a relevant and up-to-date overview of the state-of-the-art research and practices.

The paper is structured as follows: Section 2 introduces Physics Informed Machine Learning (PIML), especially its benefits and categories to provide context for its application in MIA. Section 3 talks about the PIMIA approach and introduces a taxonomy for physics priors. Section 4 presents the state-of-the-art of the conventional works published in the realm of PIMIA. In Section 5 we provide a quantitative study and provide insights regarding actual benefits and best-fit scenarios of using PI in MIA. In Section 6 we discuss the existing challenges in incorporating physics priors into PIMIA tasks and highlight the research gaps and future research avenues. Finally in section 7 we conclude this article, where we highlight its crucial takeaways.

## 2. Physics-informed Machine Learning (PIML)

This section briefly introduces the physics-informed machine learning or PIML paradigm. It also discusses a taxonomy of biases typically used to categorize the point of incorporation of physics principles, data or equation w.r.t to a machine learning pipeline.

PIML endeavours to fuse mathematical physics models and observational data into the learning process to steer it towards physically consistent solutions in scenarios charac-

Table 1: List of Abbreviations used in this article

| Abbreviation   | Definition                                              |
|----------------|---------------------------------------------------------|
| DECT           | Dual-Energy Computed Tomography                         |
| GRE MRI        | Gradient Echo Magnetic Resonance Imaging                |
| CT             | Computed Tomography                                     |
| MRI            | Magnetic Resonance Imaging                              |
| PET            | Positron Emission Tomography                            |
| US             | Ultrasound                                              |
| DWI-MRI        | Di ff usion-Weighted Imaging Magnetic Resonance Imaging |
| 4D-Flow MRI    | Four-Dimensional Flow Magnetic Resonance Imaging        |
| CBCT           | Cone Beam Computed Tomography                           |
| MDCT           | Multi-Detector Computed Tomography                      |
| MRV            | Magnetic Resonance Venography                           |
| ECG            | Electrocardiography                                     |
| QSM            | Quantitative Susceptibility Mapping                     |
| DSA            | Digital Subtraction Angiography                         |
| ODT            | Optical Di ff raction Tomography                        |
| TRUS           | Transrectal Ultrasound                                  |
| CXR            | Chest X-Ray                                             |
| EEG            | Electroencephalography                                  |
| TCD US         | Transcranial Doppler ultrasound.                        |
| SWI            | Susceptibility weighted imaging                         |

terized by partial observation, uncertainty, and high dimensionality Kashinath et al. (2021); Hao et al. (2022); Cuomo et al. (2022). The inclusion of physics information, which embodies the fundamental principles of the modelled process, enriches ML models by conferring significant advantages Kashinath et al. (2021); Meng et al. (2022a):

- 1. Helps ensure that the ML model is both physically and scientifically coherent.
- 2. Enhances data e ffi ciency in model training, enabling training with fewer data.
- 3. Expedites the model training process, leading to faster convergence to an optimal solution.
- 4. Boosts the generalizability of trained models, enabling better predictions in unseen scenarios.
- 5. Enhances the transparency and interpretability of models, rendering them explainable and more trustworthy.

An important example of this approach of embedding physics information in the ML paradigm is the family of Physics-Informed Neural Networks (PINNs). PINNs are a class of deep learning algorithms that seamlessly integrate data and abstract mathematical operators, including partial di ff erential equations (PDEs). This integration allows for more interpretable ML methods that remain robust despite imperfect data, such as missing or noisy values, and can provide accurate and physically consistent predictions. By leveraging both data and physical models, PINNs are particularly suited for solving forward and inverse problems, discovering hidden physics, and tackling high-dimensional problems in various scientific and engineering applications.

We show an example of a typical PINN in Fig: 3 designed to approximate solutions to PDEs by leveraging neural networks. The PINN is defined by the neural network function NN( X t , ; θ ), where X = ( x y z , , ) and t are the spatial and temporal coordinates, respectively, and θ represents the trainable parameters. The network utilizes a nonlinear activation function σ .

Measurement data is specified as { ( xi , t i , ui ) , } where xi and t i are spatial and temporal coordinates, respectively, and ui are

Figure 3: The PINN algorithm from Karniadakis et al. (2021)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-4ed5bcfb-e038-4cf2-8251-97d2a14756de.jpg)

the observed values of U at these coordinates. Additionally, residual points { ( xj , t j ) } are defined, where the residuals of the PDE will be evaluated.

Mathematically, the total loss function L is formulated as:

$$L = \lambda _ { \text{data} } L _ { \text{data} } + \lambda _ { \text{PDE} } L _ { \text{PDE} }$$

Here, λ data and λ PDE are weighting factors that balance the contributions of the data loss and the PDE loss.

Training the neural network involves finding the optimal parameters θ ∗ that minimize the total loss function L :

$$\theta ^ { * } = \arg \min _ { a } L$$

θ

This optimization ensures that the neural network solution NN( X t , ; θ ) fits the observed data and adheres to the underlying physical laws described by the PDE. The algorithm runs iteratively until the loss is below a certain threshold ϵ (where ϵ is a positive number) or is terminated by the human operator.

Next we discuss the work by Wang et al. (2022) from fluiddynamics and show how PINNs are used to enhance velocity field measurements from sparse data. In this example by minimizing a loss function based on sparse data and Navier-Stokes PDE equations, the PINN (see Fig 4) reconstructs dense velocity fields and predicts pressure, showcasing significant improvements in resolution and accuracy for flow visualization and analysis. The PINN is defined by the neural network NN X t ( , ; θ ), the network approximates both the velocity field U = ( u v w , , ) and the pressure p .

The loss function L for training the PINN is given by:

$$L = L _ { \text{data} } + \alpha L _ { \text{eqns} },$$

where L data quantifies the deviation from experimental data and L eqns enforces the physical laws described by the NavierStokes equations. The parameter α adjusts the relative importance of the data fit and the PDE constraints. This approach aims to improve the resolution and accuracy of the reconstructed velocity fields and pressure distributions.

Previous literature e.g. Karniadakis et al. (2021) has identified three strategies for incorporating physics knowledge priors / into machine learning models: observational bias, learning bias, and inductive bias, these are discussed below,

Figure 4: Example of PINN application from Wang et al. (2022)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-eabeb033-86f2-47d0-bf59-433fb9e32e30.jpg)

## 2.1. Biases as taxonomy in PIML

Observational bias: Observational data are foundational inputs in machine learning, essential for capturing and reflecting underlying physical principles across diverse scales and modalities Karniadakis et al. (2021). This data is instrumental in imparting knowledge of fundamental physical laws and principles to ML models Lu et al. (2021); Kashefi et al. (2021); Li et al. (2020); Yang and Perdikaris (2019). Sources of observational data include sensor networks (environmental, seismic), remote sensing (satellite imagery, weather data), experimental measurements (laboratory, wind tunnels), and high-fidelity simulations (e.g., Computational Fluid Dynamics (CFD), Finite Element Analysis (FEA)). These diverse sources capture intricate phenomena across di ff erent spatial and temporal scales, providing rich insights into physical systems. Incorporating such data during the training phase enables Deep Neural Networks (DNNs) to e ff ectively model underlying physical processes. Variable fidelity observations ensure comprehensive coverage of physical behaviors, thereby enhancing the model's capacity for accurate interpolation and adherence to physical symmetries and conservation laws.

Learning bias: This approach enforces prior knowledge physics / information through soft penalty constraints. Methods in this category augment loss functions with additional terms based on the physics of the underlying process, such as momentum and conservation of mass. For instance, physics-informed neural networks (PINN) integrate information from both measurements and partial di ff erential equations (PDEs) by embedding the PDEs into the loss function of a neural network using automatic di ff erentiation Karniadakis et al. (2021). Some notable examples of soft penalty-based approaches include statistically constrained GAN Wu et al. (2020), PI auto-encoders Erichson et al. (2019), and encoding invariances by soft constraints in the loss function InvNet Shah et al. (2019).

Inductive biases: Prior knowledge can be incorporated through custom neural network-induced 'hard' constraints. For example, Hamiltonian NN Greydanus et al. (2019) encodes better inductive biases into NNs, drawing inspiration from Hamiltonian mechanics, and training models to respect exact conservation laws. Cranmer et al. introduced Lagrangian Neural Networks (LNNs) Cranmer et al. (2020), which can parameterize arbitrary Lagrangians using neural networks, and unlike most HNNs, LNNs can work where canonical momenta are unknown or di ffi cult to compute. Ad- ditionally, tensor basis networks (TNNs) Ling et al. (2016) incorporate tensor algebra into their operations and structure, allowing them to exploit the high-dimensional structure of tensor data more e ff ectively than traditional neural networks. Meng et al. (2022b) employs a Bayesian framework where functional priors are learned using a PI-GAN from data and physics. This is followed by using the Hamiltonian Monte Carlo (HMC) method to estimate the posterior PI-GAN's latent space.

MIA is progressively integrating machine learning (ML) and deep learning (DL) techniques to enhance its capabilities. PIML has demonstrated significant promise in augmenting the e ffi cacy of MIA Banerjee et al. (2023a); Hammernik et al. (2023); Xia et al. (2023). Subsequent sections will elucidate the application methodologies of PIML within the domain of MIA.

## 3. Physics-Informed Medical Image Analysis (PIMIA)

Physics-informed medical image analysis refers to a computational approach that incorporates principles from physics into the analysis of images, specific to medical discipline. PIMIA incorporates concepts and methodologies that are used in physics-informed machine learning (PIML), which is a broader term encompassing various domains, data types and modalities of physics information. In medical imaging, various physical processes govern how images are formed and how tissues interact with imaging modalities such as X-rays, MRI, CT scans, and ultrasound. The physics-informed analysis leverages this understanding to improve image reconstruction, segmentation, registration, and other tasks.

This approach typically involves the development of mathematical models that describe the physical phenomena underlying image formation, acquisition, device characteristics, and tissue properties. These models are then integrated into image analysis algorithms to enhance the accuracy, robustness, and interpretability of medical image processing tasks. Additionally, physics-informed methods can provide more reliable and clinically relevant results compared to purely data-driven approaches by constraining the solutions.

## 3.1. Physics incorporation in general ML and MIA:

In both general machine learning approaches and MIA, the incorporation of physics can enhance model performance and interpretability. However, there are notable di ff erences in how physics is integrated into these two domains. In general machine learning, physics incorporation often involve the development of physics-inspired models or the inclusion of physical constraints as regularization terms in optimization objectives. For example, in fluid dynamics simulations, machine learning models may incorporate principles of conservation laws and fluid dynamics equations to improve predictions.

On the other hand, in MIA, physics incorporation is more directly tied to the underlying imaging processes and tissue properties. For instance, in MRI reconstruction, physicsbased models describing the signal formation process, such as the Fourier transform and T1 T2 relaxation processes, are /

integrated into reconstruction algorithms to enhance image quality and reduce artifacts. Moreover, PIMIA often involves the calibration of imaging systems to account for physical effects like attenuation and scattering in X-ray or ultrasound imaging, ensuring more accurate quantitative measurements and diagnostic assessments. Therefore, while both domains benefit from physics incorporation, the emphasis and application di ff er, reflecting the specific requirements and challenges of each field.

## 3.2. Intuitive introduction to physics priors in MIA:

Physics information is essential in MIA, improving accuracy and reliability. This information e.g. MRI acquisition physics for data augmentation or signal evolution physics for motion artifact correction, ensures better performance of PIMIA tasks. Additionally, PINNs model fluid dynamics and hemodynamics, enhancing predictive modelling. By incorporating physics into tasks like MRI reconstruction and CT artifact reduction, medical analyses become more robust and precise.

Figure 5: The three-stage process begins with the collection of random points in k-space, which represents spatial frequency information. These points are gathered without hardware constraints. Subsequently, a TSP (Travelling Salesman Problem) solver is employed to arrange these points into a path that minimizes distance. This optimized trajectory undergoes refinement during training to accommodate machine / hardware constraints. This enhances MRI performance by improving data collection e ffi ciency and image quality while ensuring operational feasibility.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-02ec974a-cca9-4990-9d68-3002b447a006.jpg)

We now discuss an intuitive example. Weiss et al. (2019) (see Fig: 5) optimize MRI trajectory by integrating physics constraints and solving the Traveling Salesman Problem (TSP), for accelerated MRI. This method ensures trajectories adhere to hardware limitations including gradient coil capacities and slew rates. By tackling TSP, PILOT-TSP reorders trajectory points to minimize path length while respecting physical constraints, and refining trajectory designs. This process optimizes MRI data acquisition, improving image quality and e ffi ciency, particularly in single-shot scenarios. Through PILOT-TSP, the training pipeline e ff ectively combines physics principles with computational techniques to advance MRI trajectory design, addressing challenges in accelerated imaging.

In other examples Poirot et al. (2019) the authors employ DECT attenuation physics to facilitate material decomposition in brain imaging, using a custom ResNet-based architecture to harness the distinct attenuation properties at di ff erent energy levels in DECT scans. Similarly, Eichhorn et al. (2024) uses signal evolution physics to correct motion artifacts in GRE MRI of the brain, with a custom MLP-based architecture designed to account for patient movement. Lastly, Kissas et al. (2020) predict arterial flow in the cardiovascular system using 4D flow MRI data. These networks are trained to predict flow and pressure wave propagation, guided by conservation laws. Unlike traditional models, this method bypasses complex pre-processing and boundary conditions. Instead, it directly utilizes noisy clinical data, producing physically consistent predictions without the need for conventional simulators.

## 3.3. Physics prior categories with examples (Taxonomy I):

Depending on the source and form of the physics prior or information being incorporated in MIA tasks, we have categorised them into three major groups, which are elaborated with examples as follows.

## 3.3.1. Medical Imaging Physics:

This category presents such works that have typically incorporated physics information and parameters derived from imaging systems and processes into medical analysis tasks. Poirot et al. (2019) utilized DECT attenuation physics for material decomposition in brain imaging, while Eichhorn et al. (2024)) corrected motion artifacts in GRE MRI brain scans using signal evolution physics. Zhu et al. (2023) reduced metal artifacts in CT imaging with a beam hardening correction model.Borges et al. (2024) and Leung et al. (2020) applied MRI acquisition physics and PET modelling physics for data augmentation. Jiang et al. (2021) reconstructed heart image sequences with cardiac electrical activity and surface voltage data within a GCNN framework.

## 3.3.2. Process dynamics and Equations:

Here we present works that have primarily physics into medical analysis using Physics-Informed Neural Networks (PINNs) and similar methods, enhancing predictive modelling, imaging, and reconstruction. Works of Halder et al. (2023), Zapf et al. (2022), and Herrero Martin et al. (2022) employ PINNs to incorporate fluid dynamics, 4D PDEs, and electrophysiological models, respectively. Kissas et al. (2020) and Sarabian et al. (2022) use conservation laws and hemodynamic equations, while van Herten et al. (2022) and Buoso et al. (2021) leverage ODEs and cardiac mechanics constraints. Techniques in imaging and reconstruction, such as those in Kamali et al. (2023) and Zheng et al. (2024), utilize elasticity theory and Helmholtz equations. Moreover, studies like Zhang et al. (2023) and Zhang et al. (2022) predict hemodynamics and musculoskeletal forces using computational fluid dynamics and physical laws. These integrations demonstrate the e ffi cacy of physics-based constraints in improving accuracy and robustness in medical analyses.

## 3.3.3. Others:

Two key approaches have emerged for incorporating underlying physics into machine learning frameworks: Representation and Features, and Physical Information and Measurements. The former utilizes patterns, representations, or features to encode physics information, as exemplified by works integrating data confidence maps and periodic motion patterns into segmentation and registration tasks, respectively.

while the latter involves directly leveraging physical principles and measurements, which are derived from biological study sources and domain experts.

Representation and Features: These works use patterns, representations or features as a medium of incorporating underlying physics information into the learning process. (Peiris et al. (2023)) integrate data confidence maps into MRI CT / segmentation for pancreas and heart imaging. (Chen et al. (2023a)) utilize image data features for segmentation of skin and lung images in CXR and dermoscopy. (Fechter and Baltas (2020)) apply periodic motion patterns for deformable image registration in lung and heart CT MRI. (Liu et al. (2020)) use / k-space artifact patterns to augment MRI denoising for liver imaging.

Physical information and measurements: Altaheri et al. (2022) utilize EEG signals in Brain-Computer Interface technology, inherently involving the physics of electrical brain activity and signal processing. Frid-Adar et al. (2018) discuss the implicit physics underlying medical image generation, particularly in CT scans, governed by X-ray attenuation principles. Shi et al. (2020) tackles data scarcity in medical imaging by leveraging domain knowledge from radiologists to synthesize high-quality images, especially for thyroid nodules.

## 3.4. Medical Image Analysis (MIA) tasks (Taxonomy II)

Here we briefly introduce the di ff erent MIA tasks that we have used to organize the literature that we have reviewed.

Imaging encompasses the process of capturing visual data using various sensors and techniques, which serves as the foundational input for PIMIA tasks. This process involves the acquisition, processing, and analysis of visual information to extract meaningful patterns and features for subsequent computational analysis.

Image generation involves creating synthetic images using advanced algorithms, often leveraging deep learning frameworks such as Generative Adversarial Networks (GANs). This task is critical for producing realistic images from latent space representations, and facilitating applications in data augmentation, simulation, and creative design.

Inverse imaging focuses on reconstructing original images from indirect or incomplete measurements by solving illposed problems. It employs mathematical and computational methods to infer high-quality images from degraded or noisy data, often utilizing regularization techniques to ensure stable and accurate reconstructions.

Image super-resolution is the technique of enhancing the spatial resolution of an image by reconstructing highfrequency details from low-resolution inputs. This process involves upscaling the image using sophisticated algorithms that predict fine details and textures, thereby improving the visual quality and detail fidelity of the image.

Image reconstruction refers to the process of forming a visual image from raw sensor data by applying computational techniques. This task involves the use of mathematical models to convert data from sensor measurements into interpretable images, typically through iterative algorithms that refine the image quality based on physical principles.

Image registration is the process of aligning multiple images into a unified coordinate system. This involves geometrically transforming images to match a reference frame, ensuring spatial correspondence. It is essential for integrating data from di ff erent viewpoints, sensors, or time points, enabling coherent analysis and comparison.

Image segmentation and classification involve partitioning an image into distinct regions (segmentation) and assigning labels to these regions (classification). These processes utilize machine learning and pattern recognition techniques to identify and categorize objects and structures within the image, facilitating detailed image analysis.

Predictive modeling integrates physical laws and constraints into machine learning frameworks to enhance the accuracy and reliability of predictions. This approach combines data-driven methods with domain-specific physical knowledge, ensuring that model outputs adhere to known physical principles and exhibit realistic behaviour.

## 3.5. Typical pipeline

We introduce each stage of the MIA pipeline which serves as a typical backbone for MIA task and their connection with di ff erent types of biases:

Data acquisition: In this stage, visual data such as images, videos, or sensor data is input to the MIA algorithm. Physics incorporation here falls under observational bias, where direct or simulated physics data is used.

Pre-processing: Acquired visual data undergoes standardization to prepare it for MIA models. Physics-guided methods like super-resolution or image synthesis aid this stage, aligning with learning bias by enforcing physical constraints through regularization

Model design: This stage involves feature extraction and selecting customizing model architectures. / Physics is integrated through model design, enhancing feature extraction methods like using custom networks to extract transient features from images.

Model training: MIA models are trained by optimizing parameters to minimize losses. Physics incorporation often occurs through loss functions that enforce physical laws like conservation equations. This approach fits under learning bias, ensuring models adhere to physical constraints during training.

MIA head: This final component of the MIA pipeline encompasses modules that perform tasks such as prediction, segmentation, or reconstruction from visual data. These modules learn from the trained models to approximate functions or distributions by optimizing network parameters. The selection of loss functions directly impacts model e ffi ciency, with common choices including cross-entropy and pixel-wise losses. Customized loss functions are also used to tailor training towards specific data characteristics and desired outcomes, enhancing model performance in targeted ways.

Figure 6: This figure integrates the primary taxonomy (II) i.e. the MIA tasks, with the MIA pipeline backbone. It also introduces the bias-based scheme from PIML works and merges it with the pipeline. The illustration as a whole presents the typical locations of physics incorporation within the MIA pipeline concerning these biases.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-ce5d3a46-756c-4d17-ad24-c952f1e2e3e3.jpg)

Table 2: Categorization based on the type of physics prior used for physics incorporation in MIA tasks.

| Categories of Physics information priors   | Categories of Physics information priors                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|--------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| A.                                         | Medical Imaging Physics                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|                                            | Poirot et al. (2019), Eichhorn et al. (2024), Zhu et al. (2023), Halder et al. (2023), Kawahara et al. (2023),Zhang et al. (2024) Pan et al. (2023), Borges et al. (2024), Leung et al. (2020), Tirindelli et al. (2021), Fok et al. (2023),Borges et al. (2019), Zimmermann et al. (2024), Shen et al. (2022), Peng et al. (2022), Desai et al. (2021), Weiss et al. (2019), Jiang et al. (2021)                                                                                                                                                                                                   |
| B.                                         | Process dynamics and Equations                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
|                                            | Halder et al. (2023), Zapf et al. (2022), Herrero Martin et al. (2022), Kissas et al. (2020), Sarabian et al. (2022), van Herten et al. (2022), Buoso et al. (2021), Lopez et al. (2023), de Vries et al. (2023), Zhang et al. (2023), Zhang et al. (2022), He et al. (2023),Kamali et al. (2023) Kaandorp et al. (2021), Shone et al. (2023), Fathi et al. (2020), Sautory and Shadden (2024), Ragoza and Batmanghelich (2023), Zheng et al. (2024), Oh et al. (2020), Maul et al. (2024), Saba et al. (2022), Han et al. (2023), Hunt et al. (2023), Min et al. (2023),Ruiz Herrera et al. (2022) |
| C.                                         | Others                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|                                            | Peiris et al. (2023) Chen et al. (2023a) Fechter and Baltas (2020) Liu et al. (2020) Altaheri et al. (2022), Frid-Adar et al. (2018), Shi et al. (2020)                                                                                                                                                                                                                                                                                                                                                                                                                                             |

## 4. PIMIA Tasks: Detailed Survey

## 4.1. Imaging

Recent advancements in physics-guided machine learning models for imaging applications have demonstrated significant enhancements across various domains. Poirot et al.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-5a833a60-cd71-49c3-a193-cc9249beed7e.jpg)

lenges in T2* quantification from gradient echo MRI, enhancing image quality and reducing acquisition time for clinical applicability. Zhu et al. (2023), presents a physics-informed sinogram completion (PISC) method for metal artifact reduction (MAR) in computed tomography (CT) imaging. Leveraging physical models, it corrects sinograms based on beamhardening correction and normalized linear interpolation, effectively reducing metal artifacts while preserving structural details near metal implants.

Figure 7: The figure shows a physics-informed process of PHIMO Eichhorn et al. (2024). Preceded by undersampled reconstruction, the above process trains a multi-layer perceptron (MLP) utilizing a physics loss, to predict exclusion masks for motion-corrupted lines. PHIMO eliminates intra-scan motion events, enabling precise correction for severe motion artifacts in multiecho GRE MRI scans.

(2019) enhanced CT processing using a network based on the ResNet architecture that incorporates lookup virtual noncontrast (L-VNC) image data.

Other notable developments include Eichhorn et al. (2024), who introduced a motion correction method for quantitative MRI, leveraging physics-informed learning. It utilizes quantitative parameter estimation to exclude motion-corrupted kspace lines from reconstruction, particularly addressing chal-

The work by Kamali et al. (2023) utilizes PINNs to simultaneously reconstruct material parameters, including Young's modulus and Poisson's ratio, and stress distributions in linear elastic materials. It leverages the governing equations of linear elasticity, stress-strain relationships, and momentum balance equations. PINN integrates experimental measurements and finite element modelling to accurately estimate mechanical properties for clinical elasticity imaging applications. In Halder et al. (2023) the authors utilize real-time dynamic MRI and Physics-Informed Neural Networks (PINNs) to analyze esophageal bolus transport. It integrates detailed fluid dynamics and pressure-volume relationships, enhancing understanding of esophageal physiology. This approach o ff ers improved diagnostics for swallowing disorders and advances computational modelling in gastroenterology.

## 4.2. Image Generation

Within the realm of MIAtasks focused on generation, In Kawahara et al. (2023) the authors employs Generative Ad-

versarial Networks (GANs) to generate images with consideration of MR properties. Additionally, Pan et al. (2023) focuses on medical imaging for data augmentation purposes, leveraging custom models and datasets like ACDC MRI and BTCV. Addressing Medical MRI analysis, Borges et al. (2024) uses data augmentation techniques informed by UNet models and custom datasets, including ABIDE.

Figure 8: The figure shows the KACGAN architecture Shi et al. (2020). Here ultrasound images and their term descriptions are used as input. Term descriptions (t) are encoded into a 128-dimensional domain knowledge vector (K) through the term encoder ( φ ) compression. This domain knowledge (K) is combined with a 512-dimensional noise vector (z) to generate fake images ( I f ) using the generator network (G). The discriminator network (D) integrates this domain knowledge for better classification. Additionally, an image encoder ensures the similarity between domain knowledge and the ultrasound images. This process incorporates physics into data augmentation through domain knowledge and word embedding, enhancing the learning and generation of high-quality images.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-a5bf938b-6a80-49e0-b678-c58bd9404fda.jpg)

Leung et al. (2020) explores the integration of physics in PET imaging, focusing on the simulation of radioactive decay, photon interactions, detector responses, system blurring, and noise characterization. These simulations, crucial for realistic image generation, enhance the training and validation of neural networks (modified UNet) in medical imaging applications. In the work by Shi et al. (2020) the authors discuss a method employing an Auxiliary Classifier Generative Adversarial Network (ACGAN) to synthesize augmented medical images. By conditioning image synthesis on semantic labels, it enhances image diversity and realism for medical analysis. This approach leverages domain knowledge to generate synthetic data crucial for training machine learning models in medical imaging.

In their research Frid-Adar et al. (2018) presents a method using Generative Adversarial Networks (GANs) to generate synthetic medical images, aiding in data augmentation for CNN-based liver lesion classification. GAN-synthesized images improve classification accuracy, addressing challenges of limited annotated datasets in medical imaging, and potentially enhancing diagnostic support for radiologists. Tirindelli et al. (2021) introduces a novel approach to ultrasound (US) data augmentation by integrating physics-inspired transformations. It proposes deformations, reverberations, and Signal-to-Noise Ratio adjustments, tailored to the principles of US imaging. These techniques aim to generate realistic variations in US images, enhancing their suitability for deep learning-based medical applications. Zhang et al. (2024)

demonstrated a physics-guided di ff usion model (Phy-Di ff ) for enhancing di ff usion MRI (dMRI) synthesis. Phy-Di ff integrates dMRI physics into noise evolution and employs querybased mapping, alongside the XTRACT atlas for anatomical details. It aims to improve dMRI quality through principled noise management and anatomically accurate synthesis. In the work by Momeni et al. (2021), the authors generate synthetic microbleeds (sCMB) to enhance the training of neural networks for detecting cerebral microbleeds (CMB) in MRI scans. Using Gaussian modeling, it simulates diverse CMB characteristics, improving classifier performance without extensive ground truth. Physics-guided by MRI properties, sCMB mimics real lesions e ff ectively for robust training.

## 4.3. Inverse Imaging

Physics-informed inverse imaging techniques leverage deep learning models incorporating physical principles to enhance medical imaging. For instance, 4D Flow MRI super-resolution integrates fluid dynamics through PhysicsInformed Neural Networks (PINNs), while specialized architectures like SR UNet improve X-ray image fidelity. PINNs also aid in MRI reconstruction by embedding Navier-Stokes equations, optimizing k-space trajectories, and employing physical variables for accurate, high-resolution images. These methods, spanning various imaging modalities, ensure physically plausible outcomes, enhancing image quality and diagnostic capabilities in biomedical applications.

## 4.3.1. Image Superresolution

Figure 9: Data driven super-resolution and denoising of 4D-Flow MRI Fathi et al. (2020) . In the presented training pipeline flow velocities, pressure, and MRI image magnitude are modelled using a patient-specific deep neural network (DNN). The DNN is trained with 4D-Flow MRI images, incorporating fluid flow physics through regularization.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-4c297e62-60de-4beb-a98a-50184c846120.jpg)

In the super-resolution task for medical image Fathi et al. (2020) focused on 4D Flow MRI, aiming to improve spatial resolution and noise reduction through super-resolution techniques, leveraging physics-informed deep neural networks to incorporate principles of fluid dynamics and mass conservation. Another paper discusses enhancing the quality of X-ray images through super-resolution, employing a specialized SR UNet architecture (referred to as SRUNK), which likely incorporates modulation transfer function kernels to achieve higher image fidelity Fok et al. (2023).

Engaging in 4D Flow MRI, one study specifically targets super-resolution to enhance image quality, utilizing PhysicsInformed Neural Networks (PINN) that integrate Navier-

Stokes equations to inform the super-resolution process accurately Shone et al. (2023).

Another study focuses on biomedical imaging, particularly MRI, aiming to reduce scan times through super-resolution techniques by employing SRGAN, which incorporates physical variables to ensure that the generated high-resolution images are physically plausible and of high quality Chen et al. (2023b).

## 4.3.2. Image Reconstruction

Figure 10: The pipeline from Ragoza and Batmanghelich (2023) employ physics-informed neural networks (PINNs) for the inverse problem, encoding displacement field u ( x ) and elastic modulus µ ( x ) while adhering to a PDE. Using a dual-network strategy, one PINN fits wave images, minimizing 'wave loss' ( L wave), while the other minimizes a PDE residual, termed 'PDE loss' ( L PDE). An 'anatomical loss' function ( L anat ) refines predictions. Loss weights ( λ wave and λ PDE) fine-tune contributions during training for precise reconstruction.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-4643ab83-3112-4493-ae53-808be6008eaf.jpg)

In medical image reconstruction tasks, in the work by Burns and Liu (2023), the authors focuses on super-resolution (SR) image reconstruction in the field of microscopy, a key area of interest in medical imaging, where enhancing the resolution of biological images is very important for better analysis and diagnosis. Zimmermann et al. (2024) introduced PINQI, a novel physics-informed neural network for quantitative MRI, integrates signal and acquisition models with learned regularization. It employs unrolled optimization for accurate parameter mapping, significantly improving performance on highly undersampled T1-mapping tasks using synthetic and real-world data. Research by Liu et al. (2020) introduced a RARE framework, enhancing MRI reconstruction by using priors from artifact-removal CNNs trained on undersampled data. It outperforms traditional denoising approaches, demonstrating e ff ective 4D MRI reconstruction without fullysampled groundtruth, validated on both simulated and experimental data.

Peng et al. (2022) introduces a novel framework for accelerating Magnetic Resonance Imaging (MRI) by optimizing k-space sampling trajectories using a neural Ordinary Differential Equation (ODE). It frames k-space sampling as a dynamic system, incorporating MRI physics constraints, and uses a neural ODE to approximate trajectory dynamics, ensuring practical MRI acquisition. The work study by Oh et al. (2020) proposes an unsupervised deep learning method, leveraging physics-informed cycleGAN, to reconstruct quantitative susceptibility maps (QSM) from MRI phase images. By exploiting the known dipole kernel, it simplifies the architecture, providing faster and more stable training. This approach enhances QSM reconstruction accuracy, addressing the limitations of traditional supervised methods.

In thier work, Jiang et al. (2021) presents a novel approach to image reconstruction, integrating physics principles into a neural network framework. By leveraging electromagnetic theory, it models the relationship between cardiac electrical excitation and surface voltage signals. This physicsinformed method improves reconstruction accuracy, o ff ering insights into cardiac activity without requiring labelled data. Zheng et al. (2024) presents a novel approach, PIN-wEPT, for reconstructing electrical properties (EPs) in the human brain using magnetic resonance electrical properties tomography (MREPT). It integrates physics-informed neural networks with the Helmholtz equation, eliminating the need for specialized hardware configurations and providing accurate EPs distribution based on water content maps.

Ragoza and Batmanghelich (2023) demonstrates a novel approach for tissue elasticity reconstruction in Magnetic Resonance Elastography (MRE) using PINNs. Encoding physical principles through PINNs solves the inverse problem of tissue sti ff ness estimation without numerical di ff erentiation, facilitating more robust and accurate elasticity mapping. The research by Shen et al. (2022) introduces a novel approach, GIIR, for ultra-sparse 3D tomographic image reconstruction, integrating geometric priors of imaging systems. Utilizing a custom 3D-Net and geometric back-projection operator, it bridges 2D and 3D image domains, enhancing image reconstruction by leveraging physics-guided information, and promising advancements in biomedical imaging.

Maul et al. (2024) introduces a method for reconstructing time-resolved contrast agent concentrations in angiography. Leveraging physics principles of blood flow and contrast agent transport, it trains a neural network using data from computational fluid dynamics simulations. This aids in efficiently approximating artifact-free reconstructions, improving angiographic image quality. The work by Desai et al. (2021) proposed VORTEX, a method for accelerated MRI reconstruction using physics-driven data augmentation. Leveraging knowledge of MRI physics, it augments MRI data with realistic noise and motion, enhancing robustness to distribution drifts. Composing image-based and physics-driven augmentations enables improved data e ffi ciency and reconstruction quality in clinically relevant scenarios.

In Weiss et al. (2019), the authors introduced PILOT (Physics-Informed Learned Optimized Trajectories), a novel approach to MRI acquisition and reconstruction. PILOT jointly optimizes k-space trajectories and image quality, integrating physical constraints on gradient coils and slew rates. It uses deep learning to design e ffi cient acquisition schemes, enhancing MRI speed and quality. Saba et al. (2022) introduces a novel approach to optical di ff raction tomography (ODT) by employing physics-informed neural networks (PINNs). Named MaxwellNet, it reconstructs three-dimensional refractive index distributions of biological samples, integrating physical principles directly into the reconstruction process. This technique enhances accuracy and e ffi ciency while ensuring adherence to physics-based constraints.

## 4.4. Image Registration

Figure 11: In Hunt et al. (2023) the UNet-based model takes a reference and a moving image from cine MRI as inputs to generate a motion vector field (MVF), which aligns the images. This process accounts for complex patient motion during radiation therapy by capturing physical deformations in real time. By integrating these physics-informed transformations, the model enhances the accuracy and e ffi ciency of image registration crucial for precise tumour targeting and dose delivery.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-bdf2eda9-73a2-4ac5-b356-85c62becddfb.jpg)

Fechter and Baltas (2020) presents a novel approach for medical image registration, crucial for tasks like tracking organ motion. Utilizing deep learning and one-shot learning techniques, combined with physics-informed methodologies, it achieves accurate registration with minimal training data. The proposed algorithm demonstrates superior performance, especially for 4D datasets capturing periodic motion.

In their work Min et al. (2023) integrates medical image registration with linear elasticity equations, enabling precise alignment of images. By formulating the problem as a partial di ff erential equation (PDE) system, it optimizes image transformation while preserving physical properties. This approach enhances accuracy and reliability in medical imaging applications, facilitating more e ff ective diagnosis and treatment planning. Hunt et al. (2023) introduced a deep learning (DL) model for fast deformable image registration using 2D sagittal cine magnetic resonance imaging (MRI) acquired during radiation therapy. It leverages principles of MRI physics, respiratory motion, and real-time MRI-guided radiation therapy to develop a novel solution for real-time motion estimation in oncological treatments.

In their work He et al. (2023) explores optimizing surface mesh generation parameters for biomechanical-modelbased deformable image registration (BM-DIR) in liver and lung CT images. Incorporating physics principles, it refines boundary conditions for accurate image registration, vital for tasks like tumour tracking in radiation therapy. Physics-based modelling enhances precision in MIA and treatment planning. Han et al. (2023) introduces DNVF, a di ff eomorphic image registration method that incorporates physics-guided information using a neural velocity field represented by an MLP. This approach enables the precise alignment of medical images by modelling complex deformations, providing flexibility for optimization, and preserving desirable di ff eomorphic properties.

## 4.5. Image Segmentation &amp; Classification

Te work by Borges et al. (2019) combines multiparametric MRI-based simulations with physics-informed CNNs, and demonstrates robust segmentation performance across varied

Figure 12: Physics informed segmentation network from Borges et al. (2019), the pink box constitute the novel contribution. Here physics parameters are integrated into a 3D U-Net for MRI segmentation. A dedicated 'physics branch' with two fully connected layers incorporates N-dimensional vectors of physics parameters. These parameters, alongside their negative exponentiation, are concatenated into the network. The objective is to mitigate imaging parameter variability in MRI segmentations, aiming for consistency across acquisitions.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-44d7851b-143e-4b57-8e77-ea0f322ff8ed.jpg)

image acquisition conditions. The methodology involves simulating MRI sequences and integrating sequence parameters into the CNN architecture to improve segmentation consistency. Chen et al. (2023a) explores CoTrFuse, a novel network combining CNN and Transformer architectures for medical image segmentation. Leveraging attention mechanisms, it extracts both local and global information, improving segmentation accuracy. In Peiris et al. (2023) the authors present CoBioNet, a dual-view framework using adversarial learning for semi-supervised medical image segmentation. By leveraging labelled and unlabelled data, it outperforms existing methods, o ff ering robustness and scalability, and improving multi-view learning. Altaheri et al. (2022) focuses on enhancing braincomputer interface capabilities for motor imagery classification, utilizing a custom model to analyze electroencephalogram (EEG) input data.

## 4.6. Physics Informed Predictive Modeling

In the domain of MIAtasks focused on prediction, Kissas et al. (2020) engages in predicting arterial blood pressure within cardiovascular flow modelling, which holds significance for comprehending heart function and associated diseases. Additionally, Zapf et al. (2022) delves into estimating di ff usion coe ffi cients, an important endeavour contributing to our understanding of various medical and biological processes. Furthermore, Sarabian et al. (2022) addresses brain hemodynamic prediction, enriching our comprehension of cerebral blood flow and its implications in both health and disease. In the context of electrophysiological parameter estimation, Herrero Martin et al. (2022) is instrumental, particularly in cardiac health and diagnostics. The work by Herrero Martin et al. (2022) focuses on electrophysiological parameter estimation, a key area in cardiac health and diagnostics. Sarabian et al. (2022) addresses brain hemodynamic prediction, contributing to our understanding of cerebral blood flow and its implications in health and disease. Zapf et al. (2022) worked on estimating di ff usion coe ffi cients, which is crucial in understanding various medical and biological processes.

In their research Kaandorp et al. (2021) introduces IVIM-

Figure 13: FiberNet Ruiz Herrera et al. (2022) leverages physics-informed neural networks to translate multiple catheter recordings of cardiac electrical activation into a continuous estimation of cardiac fibre architecture and conduction velocity, crucial for predictive modelling based on the Eikonal equation. By solving an inverse problem, it deduces the conduction velocity tensor from sparse activation maps, enabling patient-specific models vital for personalized medicine.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-49858d99-03b2-4bb2-af37-d30ab32bbb99.jpg)

NET, a deep learning model for di ff usion-weighted magnetic resonance imaging (DWI). IVIM-NET integrates physicsbased constraints to predict IVIM parameters reflecting tissue microstructure and perfusion. By enforcing physical principles through a tailored loss function, it accurately estimates di ff usion and perfusion characteristics from DWI data, aiding in non-invasive tissue characterization.

Ruiz Herrera et al. (2022) proposes FiberNet, a PhysicsInformed Neural Network (PINN) method to solve the inverse problem of identifying fibres in the heart using electroanatomical maps. By simultaneously fitting neural networks to maps and predicting conduction velocity tensors, it extends previous methods, demonstrating feasibility in clinical settings and validating through numerical experiments. In their work Zhang et al. (2022) proposes a physics-informed deep learning framework for musculoskeletal modelling, integrating existing physics-based domain knowledge into data-driven models. It employs surface electromyography (sEMG) signals to predict muscle forces and joint kinematics, leveraging physics laws as soft constraints within a Convolutional Neural Network (CNN) architecture.

Zhang et al. (2023) proposes a novel framework that integrates computational fluid dynamics (CFD) with physicsinformed neural networks (PINNs) to predict hemodynamics in the cardiovascular system. By combining CFD simulations with deep learning, the framework enables real-time and noninvasive acquisition of hemodynamic parameters, facilitating diagnosis and treatment of cardiovascular diseases. The work by de Vries et al. (2023) introduces SPPINN, a novel approach for CT perfusion analysis. SPPINN integrates physics principles into neural networks, enhancing accuracy in estimating cerebral perfusion parameters despite high noise levels. Leveraging spatio-temporal physics-informed learning, it offers a promising method for robust perfusion analysis in medical imaging applications.

## 5. Quantitative study and insights

## 5.1. Calculation of performance improvement score

We introduced a summary metric to show the improvement induced using physics information in PIMIA tasks. We selected a pair of relatively better-performing works from each task and represented them using our 'improvement score' (see Fig. 14) for comparison.

A custom score was necessary due to the use of di ff erent metrics, losses, baselines, ROIs, datasets, and imaging modalities, it was di ffi cult to uniformly compare the performances even in a single task let alone across PIMIA tasks. A plethora of metrics, from RMSE to F1-score, assess image quality, segmentation accuracy, registration precision, and predictive modelling e ffi cacy. This diverse array spans from traditional measures like SSIM to indices like NMSE, SI and MSI, providing evaluation across various facets of PIMIA.

Following is the mathematical formulation of the improvement score ( S ) calculation, considering, P as the performance achieved after incorporating physics information, B as the performance achieved without incorporating physics information. Additionally, C is the improvement score for a particular case, in a certain PIMIA task category. Each C #case is calculated as the absolute percentage di ff erence between P and B , relative to B .

This is expressed as:

$$\mathcal { C } _ { \# \text{case} } = \left | \frac { ( \mathcal { P } - \mathcal { B } ) } { \mathcal { B } } \times 1 0 0 \right |$$

Then, the improvement score, denoted as S , is calculated as the average of C 1 and C 2 , where C 1 and C 2 represent the improvement scores for two di ff erent cases or published work in each PIMIA task category:

$$\mathcal { S } _ { M e a n } = \frac { ( \mathcal { C } _ { 1 } + \mathcal { C } _ { 2 } ) } { 2 }$$

This formulation captures the relative improvement achieved by incorporating physics information across two cases, providing a standardized metric for comparison. We also present the standard deviation of the above measure, which can be expressed as follows

$$\mathcal { S } _ { S t d. D e v. } = \sqrt { \frac { ( \mathcal { C } _ { 1 } - \mathcal { C } _ { 2 } ) ^ { 2 } } { 2 } }$$

The performance improvement score S Mean serves as a key indicator of the enhancement achieved through physicsinformed (PI) approaches. To simplify and encapsulate the essence of the PI-based improvement, we have applied a number of abstractions. When metrics or errors are reported as a mean and standard deviation in the source, only the mean value is used for comparison and score calculation. Furthermore, when performance data across multiple datasets or regions of interest (ROIs) is available, the baseline and improved scores with the greatest contrast are selected for analysis.

The performance data is based on the following works Poirot et al. (2019); Eichhorn et al. (2024) (imaging), Leung

Table 3: Organizing surveyed PIMIA papers into di ff erent MIA tasks, emphasizing unique aspects across categories: physics-guided operations, Region of Interest (ROI), imaging modality, training dataset, deep network architecture, and type of physics integration. Abbreviations used: Registr. - registration and Analy. - image analysis.(includes Segmentation and Classification), Super-res. - superesolution and Recon. - reconstruction.

| Reference                                       | Physics guided operation                              | ROI                | Modality                 | Training dataset                             | Deep Network architecture     | Type of Physics information (primary)                   |
|-------------------------------------------------|-------------------------------------------------------|--------------------|--------------------------|----------------------------------------------|-------------------------------|---------------------------------------------------------|
| Poirot et al. (2019)                            | Material decomposition                                | Brain              | DECT                     | Custom                                       | Custom (ResNet based)         | DECT attenuation physics                                |
| Eichhorn et al. (2024)                          | Motion artifact correction                            | Brain              | GRE MRI                  | Custom                                       | Custom (MLP based)            | Signal evolution physics                                |
| Zhu et al. (2023)                               | Metal artifact reduction                              | Diverse            | CT                       | Custom                                       | -                             | Beam hardening Correction Model                         |
| Imaging Kamali et al. (2023)                    | Tumor diagnosis                                       | Soft tissue        | Elastography             | Custom                                       | PINN                          | Linear elastic theory, physical                         |
| Halder et al. (2023)                            | Esophageal disorder diagnosis                         | Esophagus          | MRI                      | Custom                                       | PINN                          | measurement Fluid flow eqns., conservation laws         |
| Kawahara et al. (2023)                          | Synthesis of FLAIR, DWI images                        | Brain              | Synthetic                | Custom                                       | GAN                           | MRproperties                                            |
| Pan et al. (2023)                               | Image synthesis                                       | Abdomino-Thoracic  | X-Ray, MRI, CT           | ACDC MRI, BTCV etc.                          | Custom (MT-DDPM)              | Di ff usion process                                     |
| Borges et al. (2024)                            | Data augmentation                                     | Brain              | MRI                      | Custom, ABIDE                                | UNet                          | MRI acquisition physics                                 |
| Generation Leung et al. (2020)                  | Data augmentation                                     | Lung               | PET                      | Custom                                       | Custom (U-net                 | Physics of PET modelling                                |
| Tirindelli et al. (2021)                        | Data augmentation                                     | Spine              | US                       | Custom                                       | based) U-Net, DenseNet        | US waves based physics information                      |
| Shi et al. (2020)                               | Conditional image synthesis                           | Thyroid nodules    | US                       | Custom                                       | ACGAN                         | Domain knowledge (shape, Calcification etc.)            |
| Frid-Adar et al. (2018)                         | Liver lesion dataset synthesis                        | Liver              | CT                       | Custom                                       | Custom                        | Radiomic features of lesions                            |
| Zhang et al. (2024)                             | raw dMRI image synthesis                              | Brain              | di ff usion MRI          | HCP S1200                                    | HDiT                          | Di ff usion physics                                     |
| Momeni et al. (2021)                            | Training data augmentation                            | Brain              | SWI                      | AIBLEllis et al. (2009)                      | Custom                        | Physical properties of Cerebral Microbleeds             |
| Zapf et al. (2022)                              | Di ff usion coe ffi cient estimation                  | Brain              | MRI                      | Custom                                       | PINN                          | 4D PDE                                                  |
| Herrero Martin et al. (2022)                    | Electrophysiological                                  | Heart              | Optical mapping          | Simulated cardiac EP data                    | PINN                          | PDE, ODE, IC and BC                                     |
| Kissas et al. (2020)                            | Predicting arterial                                   | Artery             | 4D flow MRI              | Synthesized using DG solver                  | PINN                          | Conservation law constraints                            |
| Predictive Sarabian et al. (2022)               | Brain Hemodynamics                                    | Brain              | TCD US                   | Custom                                       | PINN                          | 1D ROM PDE, Constraints                                 |
| van Herten et al. (2022)                        | MP MRI quantification                                 | Heart              | DCE-MRI                  | Custom                                       | PINN                          | ODE residual loss                                       |
| Buoso et al. (2021)                             | Cardiac mechanics simulation                          | Heart              | MRI, CT                  | MMWHS                                        | PINN                          | NN projection layer, cost function                      |
| Lopez et al. (2023)                             | Cardiac strain estimation                             | Heart              | MRI                      | Custom SSFP-MRI                              | PINN                          | Near-incompressibility of cardiac tissue                |
| Modeling de Vries et al. (2023)                 | Estimate Cereberal perfusion parameters               | Brain              | CT perfusion             | ISLES 2018                                   | SPPINN                        | dynamics of CT perfusion                                |
| Zhang et al. (2023)                             | Hemodynamics prediction                               | Cardiovascular     | CTA                      | Custom                                       | PointNet (PINN)               | Comp. fluid dynamics, Navier-Stokes eqn.                |
| Zhang et al. (2022)                             | Muscle force and joint motion prediction              | Musculoskeletal    | Surface-electromyography | Custom                                       | CNN                           | Physics laws                                            |
| Ruiz Herrera et al. (2022)                      | Identification of fibers                              | Heart              | Elctroanatomical (Elm.)  | map FiberNet (custom PINN)                   | Custom                        | Physics info. extracted from Elm. maps                  |
| Kaandorp et al. (2021)                          | Estimate di ff usion-perfusion                        | Pancreas           | DWI-MRI                  | NCT01995240, NCT01989000                     | Custom IVIM-NET (PINN)        | Physical principles of DWI-MRI                          |
| Shone et al. (2023)                             | MRI Super-resolution                                  | Heart              | 4D-Flow MRI              | Synthetic (CFD)                              | PINN                          | NS eqn, and symmetry constraints                        |
| Fok et al. (2023)                               | CT super-resolution                                   | Hand, wrist, elbow | CT (CBCT, MDCT)          | Custom                                       | Custom (based on UNet)        | Modulation Transfer Function kernels in CT              |
| Inverse Fathi et al. (2020)                     | SR and denoising                                      | Vascular system    | 4D-Flow MRI              | Custom                                       | Custom                        | NS and conservation eqns., Fluid flow physics           |
| Sautory and Shadden (2024)                      | Flow denoising, SR                                    | Cardiovascular     | MRV                      | synthetic (CFD)                              | PINN                          | NS eqn., Newtonian fluid                                |
| Imaging Zimmermann et al. (2024)                | Quantitative MRI Reconstruction                       | Brain              | MRI                      | Custom                                       | Custom (PINQI)                | MRI signal and acquisition model                        |
| Liu et al. (2020)                               | Denoiser augmentation                                 | Liver              | MRI                      | Custom                                       | Simplified DnCNN denoiser     | k-space artifact pattern in 4D MRimages                 |
| Shen et al. (2022)                              | 3D tomographic reconstruction                         | Lung               | CT                       | LIDC, IDRI                                   | 3D-Net (custom)               | imaging geometry priors                                 |
| Ragoza and Batmanghelich (2023)                 | Tissue elasticity reconstruction                      | Liver              | MRE(MRI)                 | Custom                                       | PINN                          | PDE (Helmholtz equation)                                |
| Zheng et al. (2024)                             | Reconstruction of electrical properties               | Brain              | MRI                      | Custom                                       | PINwEPT(PINN)                 | PDE (Helmholtz equation)                                |
| Jiang et al. (2021)                             | Image-sequence reconstruction                         | Heart              | ECG                      | Custom                                       | GCNN                          | Cardiac electrical activity and surface voltage         |
| Peng et al. (2022)                              | Accelerated MRI                                       | Brain, Knee        | MRI                      | fastMRI                                      | PINN                          | MRI hardware constraints                                |
| (Super-res.and Oh et al. (2020)                 | QSM reconstruction                                    | Brain              | MRI                      | QSM 2016,19 challenges etc.                  | CycleQSM (custom PI-cycleGAN) | optimal transport theory                                |
| Maul et al. (2024)                              | contrast agent conc. reconstr.                        | Cerebrovascular    | DSA                      | Custom + AneuX                               | CNN                           | CFD simulations                                         |
| Recon.) Desai et al. (2021)                     | accelerated MRI reconstr.                             | Knee               | MRI                      | 3D FSE multi-coil Knee                       | 2D U-Net                      | MRI signal and acquisition models                       |
| Weiss et al. (2019)                             | Accelerated MRI                                       | Knee               | MRI                      | NYU fastMRI                                  | Custom (UNet based)           | MRI hardware based constraints                          |
| Saba et al. (2022)                              | reconstr. of 3D RID                                   | Biological cells   | ODT                      | Custom                                       | PINN                          | PDE (Maxwell's equation)                                |
| Han et al. (2023)                               | Di ff eomorphic image registration                    | Brain              | 3D MRI                   | Mindboggle                                   | Custom (FCN based)            | di ff eomorphic deformation of images                   |
| He et al. (2023)                                | Deformable image registration                         | Lung, Liver        | CT (4DCT)                | OASIS, Custom                                | Morfeus                       | biomechanical models                                    |
| Registr. Hunt et al. (2023)                     | Deformable image registration                         | Abdomino-Thoracic  | MRI                      | CineMRI(Custom)                              | VoxelMorph                    | MRI signal and acquisition models                       |
| Min et al. (2023)                               | Model prostrate motion                                | Prostate           | MRI, TRUS                | Custom (on prostrate cancer biopsy)          | PINN                          | Linear elasticity equations                             |
| Fechter and Baltas (2020)                       | Deformable image registration                         | Lung, Heart        | CT, MRI                  | DirLab, Popi, Sunnybrook                     | Custom (UNet based)           | Periodic motion pattern                                 |
| Altaheri et al. (2022)                          | Motor imagery classification                          | Brain              | EEG                      | BCI-2a dataset                               | Custom                        | EEG input data                                          |
| Analy. Borges et al. (2019) Chen et al. (2023a) | Brain MRI segmentation CXR / Dermoscopic Segmentation | Brain Skin, Lung   | MRI CXR, Dermascopy      | Custom, SABRE subsets ISIC-2017, COVID-QU-Ex | 3D U-Net Custom (CoTrFuse)    | Physics parameter as training input Image data features |
| Peiris et al. (2023)                            | MRI / CT Segmentation                                 | Pancreas, Heart    | MRI, CT                  | NIH Pancreas CT, LA MRI etc.                 | Custom (Co-BioNet)            | Data Confidence maps                                    |

Figure 14: Representative comparison of improvement in MIA task performance due to physics incorporation w.r.t. standard baselines. The performances are scored using custom performance measure i.e. Improvement score ( S Mean ) and the corresponding standard deviation ( S Std dev . . ).The plot indicates the highest performance with significant variability in Super-resolution tasks, and the least performance improvement is observed in Analysis tasks, with registration and reconstruction tasks showing minimal variability.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01026v1-figure-1c98273d-8f50-457f-82ad-e1d745487c85.jpg)

et al. (2020); Borges et al. (2024) (generation), van Herten et al. (2022); Kaandorp et al. (2021) (predictive-modeling), Shone et al. (2023); Fok et al. (2023) (Super-resolution), Peng et al. (2022); Desai et al. (2021)(reconstruction), He et al. (2023); Min et al. (2023) (registration) and Borges et al. (2019); Chen et al. (2023a)(Segmentation / Image analysis). Through these works we have considered the following er- rors and metrics, to measure the performance improvement: RMSE, MAE, DICE, NRMSE, SSIM and TRE.

## 5.2. Performance improvement due to physics incorporation

The comparative analysis of PIMIA tasks reveals significant variations in performance across di ff erent categories (see Fig: 14). The imaging task exhibits a notable 13.8% average improvement. In generation tasks, the score improves by 10.495%. Predictive modelling shows a substantial average improvement of 29.675%. The inverse imaging task, specifically in super-resolution, demonstrates the most significant gains with an average improvement of 44.27%. Conversely, tasks such as reconstruction and analysis display modest improvements, with average increases of 5.55% and 2.85%, respectively.

## 5.3. Discussion: PIML Bias based taxonomy and PIMIA

Observation bias approaches typically use multi-modal data and DNNs to capture underlying physical principles from training data, sourced from direct observations, representations, or physics data. This bias is evident in works incorporating physical information, measurements, and feature-type physics priors (see Section 3.3.3). For example, Peiris et al. (2023) use data confidence maps for MRI CT pancreas and / heart segmentation, and Chen et al. (2023a) leverage image data features for segmenting skin and lung images in CXR and dermoscopy. Fechter and Baltas (2020) utilize periodic

motion patterns for deformable image registration in lung and heart CT MRI, while Liu et al. (2020) enhance MRI denoising / for liver imaging using k-space artifact patterns. Additionally, direct applications of physical principles are seen in Altaheri et al. (2022) with EEG signals in Brain-Computer Interface technology reflecting electrical brain activity, Frid-Adar et al. (2018) discussing the physics of X-ray attenuation in CT scan image generation, and Shi et al. (2020) using radiologists' domain knowledge to synthesize high-quality images, addressing data scarcity in medical imaging.

A large number of the surveyed works have implemented physics priors through a Learning bias type approach, i.e. enforcing prior knowledge physics information through soft / penalty constraints. Most of the approaches have directly used a PINN or a PINN-inspired methodology to implement the physics-based constraints. For example Halder et al. (2023), Zapf et al. (2022), and Herrero Martin et al. (2022) employed PINNs to incorporate fluid dynamics, 4D PDEs, and electrophysiological models, respectively. Conservation laws and hemodynamic equations were utilized by Kissas et al. (2020) and Sarabian et al. (2022), while van Herten et al. (2022) and Buoso et al. (2021) leveraged ODEs and cardiac mechanics constraints.

Concerning Inductive bias, where physics prior is incorporated through custom neural network-induced 'hard' constraints. We could not find any individual work that has directly incorporated physics information through this approach.

## 5.4. When to choose PIMIA over typical data-driven MIA?

Incorporating physics information into data-driven models may become essential in certain specific scenarios. Typical data-driven approaches, while powerful in learning from diverse examples, require vast amounts of data, are timeconsuming to train, and often lack theoretical guarantees, potentially disregarding physical principles. In the context of MIA physics-informed DNN models / methods excel when data is limited, precision is important, and processing e ffi -ciency is essential.

They are particularly e ff ective in augmenting training and validation datasets by embedding known physical principles, such as MRI signal generation, which enables robust learning with limited data. This approach accelerates training, avoids local minima, and ensures solutions align with realworld physics, enhancing accuracy and reliability in medical imaging applications. These points are addressed in greater detail as follows:

## 1) Augmenting training and or validation dataset /

Obtaining large annotated datasets can be challenging in medical imaging due to privacy concerns, high costs, and the need for expert annotations. For example, acquiring labelled MRI scans for rare diseases is di ffi cult due to the low incidence of such cases. Physics-informed models can compensate by embedding known physical principles, such as the physics of MRI signal generation, into the learning process. This allows them to perform reliably even with smaller datasets, maximizing the utility of limited data and ensuring robust and accurate models without needing extensive labelled images.

## 2) Regularize learning of DNN models

## -E ffi cient training with smaller datasets

Physics-informed (PI) models enhance training e ffi -ciency by incorporating physical principles, which act as constraints. This integration reduces the search space for model parameters, leading to faster convergence and improved performance even with smaller datasets. Consequently, these models require less data while maintaining high accuracy and reliability. For example, in MRI reconstruction, PI models use known MRI signal physics as constraints, which enables e ffi cient learning from fewer scans. Thereby speeding up training and improving accuracy, requiring smaller datasets compared to purely data-driven models.

## -Helps avoid convergence to local minima.

PI models combine the computational e ffi ciency of physics-based algorithms with the adaptive learning capabilities of data-driven methods, thereby accelerating training and mitigating local minima issues. For example, in real-time MRI-guided surgery, physics-informed models o ff er rapid initial tissue structure approximations, reducing computational demands and ensuring timely, precision decisions crucial in clinical settings.

## -Making more physics-consistent solutions

Medical imaging techniques like MRI and CT scans are governed by complex physical principles which makes PI models particularly suited for these applications. For example, in cardiac MRI, understanding the blood flow dynamics and tissue properties is crucial for accurate diagnosis. Through incorporation of the physics of blood flow and tissue contrast, the models can better interpret the intricate details of cardiac images, leading to more accurate analyses. This allows for more e ff ective identification of conditions such as heart valve defects or myocardial infarction, which purely data-driven methods might miss.

## 3) Enables use of simpler models and architectures -Smaller Models for Speed

Physics incorporation in models leads to simpler model architectures, thereby reducing the number of parameters and computational complexity. Resulting in comparatively smaller models that can process data faster, making them ideal for real-time applications and environments with limited computational resources, enhancing e ffi ciency without sacrificing accuracy.

## -Faster Convergence

PI models integrate physical laws and constraints, guiding the optimization process and reducing the search space for solutions. This leads to faster convergence during training, as the model quickly aligns with physically plausible outcomes. As a result, training times are shortened, and computational resources are conserved.

- -Eradicate Impossible Solutions

By embedding physical principles directly into the model, physics-informed approaches eliminate impossible or non-physical solutions. This ensures that the model's predictions are consistent with real-world phenomena, enhancing reliability and trustworthiness in applications such as medical image analysis, where precision and realism are important.

## 6. Current challenges and future research direction

## 6.1. Challenges in Incorporating Physics into MIA

The integration of machine learning and physics priors in MIA has ushered in significant advancements in the field of computational MRI, accurate tomographic reconstruction and better predictive models to name a few. Despite the progress, several open questions and specific challenges remain, which need to be addressed to further refine and enhance these approaches. The following discussion outlines the key issues and challenges faced in the application of physics-informed methods in MIA:

- 1) Assessment of Overregularization Performance: The assessment of overregularization performance involves evaluating deep learning models that may su ff er from overfitting due to excessive constraints during training. In clinical settings, these models can generate visually realistic but potentially misleading images, complicating artifact identification. This challenge is exacerbated by the presence of artifacts, where constraints imposed by prior information (physics priors) may inadvertently generate or obscure clinically significant features. Addressing this issue requires robust methods to quantify how these constraints influence model performance, ensuring maintained diagnostic accuracy amidst variable data quality in clinical practice.
- 2) Model Generalization: It's essential for ML models in MIA to adapt across diverse imaging equipment, sites, and populations. For example, variations in image acquisition, in terms of image quality, temporal resolution, Field of View (FOV) and patient positioning / movement can significantly a ff ect model e ffi cacy. Integrating physics-based priors, which encompass anatomical knowledge, imaging technique and acquisition metadata, is pivotal in addressing these variations. The key consideration lies in whether to develop versatile models using these priors for broad applicability or specialized models optimized for specific imaging scenarios to enhance diagnostic precision.
- 3) Prospective and Retrospective acceleration with physics priors: Medical imaging often utilizes retrospectively accelerated acquisitions for faster image generation, but this may overlook certain signal acquisition e ff ects. Prospective acceleration, collecting data in real-time, offers more comprehensive information but presents challenges in transitioning from retrospective methods. Incorporating physics priors ensures that computational

models e ff ectively account for the nuances and complexities of the imaging process. The challenge lies in efficiently incorporating the complex physics of imaging systems e.g. MRI scanners in computational imaging methods.

## 4) Use of processed data:

Using processed images instead of raw data in MIA can sometimes lead to overly optimistic findings. This happens because processed images (e.g. DICOM) are altered representations of the original data (e.g. raw k-space data), potentially distorting the information in ways that may not accurately reflect real-world conditions. The challenge is in designing experiments that avoid such biases, ensuring that the findings derived from processed data align well with what would be observed in real-world scenarios, such as during patient scans. Adding physics information further complicates this alignment, requiring careful planning and refinement of experimental methods to maintain accuracy and reliability in MIA.

## 6.2. Future directions

Future directions include strengthening deep learning models across diverse imaging settings, refining uncertainty quantification in reconstruction, and integrating transformers with physics-based methods to optimize feature recognition and address data scarcity challenges in tomography.

- 1. Enhancing Model Robustness Across Imaging Settings: Strengthening deep learning models to withstand variability in imaging devices and settings through physicsbased priors for reliable medical image analysis. E.g. enabling models to handle variations in MRI scanners by integrating prior knowledge of magnetic field strengths and imaging protocols.
- 2. Improving Uncertainty Quantification in Reconstruction: Refining algorithms to accurately quantify uncertainty in image reconstruction under diverse sampling patterns, leveraging physics-based insights. E.g. enhancing algorithms to accurately assess uncertainty in CT scans with di ff erent slice thicknesses, using physicsbased insights on X-ray attenuation and reconstruction techniques.
- 3. Advancing DNN Integration with Physics-Based Methods: Combining transformers (an advanced ML model architecture) with physics-based approaches can direct attention to critical features, boosting performance. Exploring self-supervised learning with physics-based methods can address challenges in data-driven tomography, particularly with limited paired training data.

## 7. Conclusions

This paper introduces a state-of-the-art PIMIA paradigm that integrates data-driven methods with insights from physics and scientific principles. We present a unified taxonomy to classify PIMIA approaches based on the physics information,

their representation and incorporation in MIA models. Our review covers a wide range of tasks, including imaging, generation, prediction, inverse imaging (super-resolution and reconstruction), registration, and image analysis (segmentation and classification). A comprehensive summary of the discussed papers is provided in a tabular format in Table 3 to facilitate an understanding of how physics principles are integrated into MIA tasks.

The goal is to demystify the application of PIMIA methods across various MIA tasks, address current challenges, and encourage further research in this field.

## Acknowledgments

This research was partly supported by the Advance Queensland Industry Research Fellowship AQIRF024-2021RD4.

## References

Altaheri, H., Muhammad, G., Alsulaiman, M., 2022. Physics-informed attention temporal convolutional network for EEG-based motor imagery classification. IEEE Transactions on Industrial Informatics 19, 2249-2258.

Balageas, D., Fritzen, C.P., G¨emes, A., 2010. u Structural health monitoring. volume 90. John Wiley &amp; Sons.

- Banerjee, C., Nguyen, K., Fookes, C., Karniadakis, G., 2023a. Physicsinformed computer vision: A review and perspectives. arXiv preprint arXiv:2305.18035 .

Banerjee, C., Nguyen, K., Fookes, C., Raissi, M., 2023b. A survey on physics informed reinforcement learning: Review and open problems. arXiv preprint arXiv:2309.01909 .

Borges, P., Shaw, R., Varsavsky, T., Kl¨ aser, K., Thomas, D., Drobnjak, I., Ourselin, S., Cardoso, M.J., 2024. Acquisition-invariant brain MRI segmentation with informative uncertainties. Medical Image Analysis 92, 103058.

- Borges, P., Sudre, C., Varsavsky, T., Thomas, D., Drobnjak, I., Ourselin, S., Cardoso, M.J., 2019. Physics-informed brain MRI segmentation, in: International Workshop on Simulation and Synthesis in Medical Imaging, pp. 100-109.
- Buoso, S., Joyce, T., Kozerke, S., 2021. Personalising left-ventricular biophysical models of the heart using parametric physics-informed neural networks. Medical Image Analysis 71, 102066.

Burns, Z., Liu, Z., 2023. Untrained, physics-informed neural networks for structured illumination microscopy. Optics Express 31, 8714-8724.

- Cai, S., Mao, Z., Wang, Z., Yin, M., Karniadakis, G.E., 2022. Physicsinformed neural networks (pinns) for fluid mechanics: A review. Acta Mechanica Sinica , 1-12.
- Chen, Y., Wang, T., Tang, H., Zhao, L., Zhang, X., Tan, T., Gao, Q., Du, M., Tong, T., 2023a. CoTrFuse: a novel framework by fusing cnn and transformer for medical image segmentation. Physics in Medicine &amp; Biology 68, 175027.
- Chen, Z., Stapleton, M.C., Xie, Y., Li, D., Wu, Y.L., Christodoulou, A.G., 2023b. Physics-informed deep learning for t2-deblurred superresolution turbo spin echo MRI. Magnetic Resonance in Medicine 90, 2362-2374.
- Cranmer, M., Greydanus, S., Hoyer, S., Battaglia, P., Spergel, D., Ho, S., 2020. Lagrangian neural networks. arXiv preprint arXiv:2003.04630 .
- Cuomo, S., Di Cola, V.S., Giampaolo, F., Rozza, G., Raissi, M., Piccialli, F., 2022. Scientific machine learning through physics-informed neural networks: Where we are and what's next. arXiv preprint arXiv:2201.05624 .
- Desai, A.D., Gunel, B., Ozturkler, B.M., Beg, H., Vasanawala, S., Hargreaves, B.A., R´ e, C., Pauly, J.M., Chaudhari, A.S., 2021. V ortex: Physicsdriven data augmentations using consistency training for robust accelerated MRI reconstruction. arXiv preprint arXiv:2111.02549 .
- Eichhorn, H., Spieker, V., Hammernik, K., Saks, E., Weiss, K., Preibisch, C., Schnabel, J.A., 2024. Physics-informed deep learning for motioncorrected reconstruction of quantitative brain MRI. arXiv preprint arXiv:2403.08298 .
- Ellis, K.A., Bush, A.I., Darby, D., De Fazio, D., Foster, J., Hudson, P., Lautenschlager, N.T., Lenzo, N., Martins, R.N., Maru ff , P., et al., 2009. The australian imaging, biomarkers and lifestyle (aibl) study of aging: methodology and baseline characteristics of 1112 individuals recruited for a longitudinal study of alzheimer's disease. International psychogeriatrics 21, 672-687.
- Erichson, N.B., Muehlebach, M., Mahoney, M.W., 2019. Physics-informed autoencoders for lyapunov-stable fluid flow prediction. arXiv preprint arXiv:1905.10866 .
- Fathi, M.F., Perez-Raya, I., Baghaie, A., Berg, P., Janiga, G., Arzani, A., D'Souza, R.M., 2020. Super-resolution and denoising of 4d-flow MRI using physics-informed deep neural nets. Computer Methods and Programs in Biomedicine 197, 105729.
- Fechter, T., Baltas, D., 2020. One-shot learning for deformable medical image registration and periodic motion tracking. IEEE transactions on medical imaging 39, 2506-2517.
- Fok, W.Y.R., Fieselmann, A., Herbst, M., Ritschl, L., Kappler, S., Saalfeld, S., 2023. Deep learning in computed tomography super resolution using multi-modality data training. Medical Physics .
- Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., Greenspan, H., 2018. Gan-based synthetic medical image augmentation for increased cnn performance in liver lesion classification. Neurocomputing 321, 321331.
- Greydanus, S., Dzamba, M., Yosinski, J., 2019. Hamiltonian neural networks. Advances in neural information processing systems 32.
- Halder, S., Johnson, E.M., Yamasaki, J., Kahrilas, P.J., Markl, M., Pandolfino, J.E., Patankar, N.A., 2023. MRI-MECH: mechanics-informed MRI to estimate esophageal health. Frontiers in Physiology 14, 1195067.
- Hammernik, K., K¨ ustner, T., Yaman, B., Huang, Z., Rueckert, D., Knoll, F., Akc ¸akaya, M., 2023. Physics-driven deep learning for computational magnetic resonance imaging: Combining physics and machine learning for improved medical imaging. IEEE signal processing magazine 40, 98114.
- Han, K., Sun, S., Yan, X., You, C., Tang, H., Naushad, J., Ma, H., Kong, D., Xie, X., 2023. Di ff eomorphic image registration with neural velocity field, in: Proceedings of the IEEE CVF Winter Conference on Applications of / Computer Vision, pp. 1869-1879.
- Hao, Z., Liu, S., Zhang, Y., Ying, C., Feng, Y., Su, H., Zhu, J., 2022. Physicsinformed machine learning: A survey on problems, methods and applications. arXiv preprint arXiv:2211.08064 .
- He, Y., Anderson, B.M., Cazoulat, G., Rigaud, B., Almodovar-Abreu, L., Pollard-Larkin, J., Balter, P., Liao, Z., Mohan, R., Odisio, B., et al., 2023. Optimization of mesh generation for geometric accuracy, robustness, and e ffi ciency of biomechanical-model-based deformable image registration. Medical physics 50, 323-329.
- Herrero Martin, C., Oved, A., Chowdhury, R.A., Ullmann, E., Peters, N.S., Bharath, A.A., Varela, M., 2022. EP-PINNs: Cardiac electrophysiology characterisation using physics-informed neural networks. Frontiers in Cardiovascular Medicine 8, 2179.
- van Herten, R.L., Chiribiri, A., Breeuwer, M., Veta, M., Scannell, C.M., 2022. Physics-informed neural networks for myocardial perfusion MRI quantification. Medical Image Analysis 78, 102399.
- Hu, C., Youn, B.D., Wang, P., Hu, C., Youn, B.D., Wang, P., 2019. Case studies: prognostics and health management (phm). Engineering Design under Uncertainty and Health Prognostics , 303-342.
- Hunt, B., Gill, G.S., Alexander, D.A., Streeter, S.S., Gladstone, D.J., Russo, G.A., Zaki, B.I., Pogue, B.W., Zhang, R., 2023. Fast deformable image registration for real-time target tracking during radiation therapy using cine MRI and deep learning. International Journal of Radiation Oncology* Biology* Physics 115, 983-993.
- Jiang, X., Missel, R., Toloubidokhti, M., Li, Z., Gharbia, O., Sapp, J.L., Wang, L., 2021. Label-free physics-informed image sequence reconstruction with disentangled spatial-temporal modeling, in: Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part VI 24, Springer. pp. 361-371.
- Kaandorp, M.P., Barbieri, S., Klaassen, R., van Laarhoven, H.W., Crezee, H., While, P.T., Nederveen, A.J., Gurney-Champion, O.J., 2021. Improved unsupervised physics-informed deep learning for intravoxel incoherent motion modeling and evaluation in pancreatic cancer patients. Magnetic resonance in medicine 86, 2250-2265.

Kamali, A., Sarabian, M., Laksari, K., 2023. Elasticity imaging using

physics-informed neural networks: Spatial discovery of elastic modulus and poisson's ratio. Acta biomaterialia 155, 400-409.

Karniadakis, G.E., Kevrekidis, I.G., Lu, L., Perdikaris, P., Wang, S., Yang, L., 2021. Physics-informed machine learning. Nature Reviews Physics 3, 422-440.

Kashefi, A., Rempe, D., Guibas, L.J., 2021. A point-cloud deep learning framework for prediction of fluid flow fields on irregular geometries. Physics of Fluids 33, 027104.

Kashinath, K., Mustafa, M., Albert, A., Wu, J., Jiang, C., Esmaeilzadeh, S., Azizzadenesheli, K., Wang, R., Chattopadhyay, A., Singh, A., et al., 2021. Physics-informed machine learning: case studies for weather and climate modelling. Philosophical Transactions of the Royal Society A 379, 1-36. Kaviani, S., Han, K.J., Sohn, I., 2022. Adversarial attacks and defenses on ai in medical imaging informatics: A survey. Expert Systems with Applications 198, 116815.

Kawahara, D., Yoshimura, H., Matsuura, T., Saito, A., Nagata, Y., 2023. MRIimage synthesis for fluid-attenuated inversion recovery and di ff usionweighted images with deep learning. Physical and Engineering Sciences in Medicine 46, 313-323.

Kissas, G., Yang, Y., Hwuang, E., Witschey, W.R., Detre, J.A., Perdikaris, P., 2020. Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4d flow MRI data using physicsinformed neural networks. Computer Methods in Applied Mechanics and Engineering 358, 112623.

Leung, K.H., Marashdeh, W., Wray, R., Ashrafinia, S., Pomper, M.G., Rahmim, A., Jha, A.K., 2020. A physics-guided modular deep-learning based automated framework for tumor segmentation in pet. Physics in Medicine &amp;Biology 65, 245032.

- Li, G., Wang, H., Zhang, M., Tupin, S., Qiao, A., Liu, Y., Ohta, M., Anzai, H., 2021. Prediction of 3d cardiovascular hemodynamics before and after coronary artery bypass surgery via deep learning. Communications biology 4, 99.

Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., Anandkumar, A., 2020. Fourier neural operator for parametric partial di ff erential equations. arXiv preprint arXiv:2010.08895 .

Ling, J., Kurzawski, A., Templeton, J., 2016. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. Journal of Fluid Mechanics 807, 155-166.

Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A., van Ginneken, B., S ˜ ¡nchez, C.I., 2017. A survey A on deep learning in medical image analysis. Medical Image Analysis 42, 60-88.

Liu, J., Sun, Y., Eldeniz, C., Gan, W., An, H., Kamilov, U.S., 2020. Rare: Image reconstruction using deep priors learned without groundtruth. IEEE Journal of Selected Topics in Signal Processing 14, 1088-1099.

Liu, L., Wolterink, J.M., Brune, C., Veldhuis, R.N., 2021. Anatomy-aided deep learning for medical image segmentation: a review. Physics in Medicine &amp; Biology 66, 11TR01.

Lopez, P.A., Mella, H., Uribe, S., Hurtado, D.E., Costabal, F.S., 2023. Warppinn: Cine-mr image registration with physics-informed neural networks. Medical Image Analysis 89, 102925.

Lu, L., Jin, P., Pang, G., Zhang, Z., Karniadakis, G.E., 2021. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence 3, 218-229.

Maier, A., K¨stler, H., Heisig, M., Krauss, P., Yang, S.H., 2022. Known opero ator learning and hybrid machine learning in medical imaging-a review of the past, the present, and the future. Progress in Biomedical Engineering 4, 022002.

Maul, N., Birkhold, A., Wagner, F., Thies, M., Rohleder, M., Berg, P., Kowarschik, M., Maier, A., 2024. Physics-informed learning for timeresolved angiographic contrast agent concentration reconstruction. arXiv preprint arXiv:2403.01993 .

Meng, C., Seo, S., Cao, D., Griesemer, S., Liu, Y., 2022a. When physics meets machine learning: A survey of physics-informed machine learning. arXiv preprint arXiv:2203.16797 .

Meng, X., Yang, L., Mao, Z., del Aguila Ferrandis, J., Karniadakis, G.E., ´ 2022b. Learning functional priors and posteriors from data and physics. Journal of Computational Physics 457, 111073.

Min, Z., Baum, Z.M., Saeed, S.U., Emberton, M., Barratt, D.C., Taylor, Z.A., Hu, Y., 2023. Non-rigid medical image registration using physicsinformed neural networks, in: International Conference on Information Processing in Medical Imaging, Springer. pp. 601-613.

Momeni, S., Fazlollahi, A., Yates, P., Rowe, C., Gao, Y., Liew, A.W.C., Salvado, O., 2021. Synthetic microbleeds generation for classifier training without ground truth. Computer methods and programs in biomedicine 207, 106127.

- Oh, G., Bae, H., Ahn, H.S., Park, S.H., Ye, J.C., 2020. Cycleqsm: Unsupervised qsm deep learning using physics-informed cyclegan. arXiv preprint arXiv:2012.03842 .

Pan, S., Wang, T., Qiu, R.L., Axente, M., Chang, C.W., Peng, J., Patel, A.B., Shelton, J., Patel, S.A., Roper, J., et al., 2023. 2d medical image synthesis using transformer-based denoising di ff usion probabilistic model. Physics in Medicine &amp; Biology 68, 105004.

Peiris, H., Hayat, M., Chen, Z., Egan, G., Harandi, M., 2023. Uncertaintyguided dual-views for semi-supervised volumetric medical image segmentation. Nature Machine Intelligence 5, 724-738.

Peng, W., Feng, L., Zhao, G., Liu, F., 2022. Learning optimal k-space acquisition and reconstruction using physics-informed neural networks, in: Proceedings of the IEEE CVF Conference on Computer Vision and Pat-/ tern Recognition, pp. 20794-20803.

Poirot, M.G., Bergmans, R.H., Thomson, B.R., Jolink, F.C., Moum, S.J., Gonzalez, R.G., Lev, M.H., Tan, C.O., Gupta, R., 2019. Physics-informed deep learning for dual-energy computed tomography image processing. Nature Scientific reports 9, 17709.

Ragoza, M., Batmanghelich, K., 2023. Physics-informed neural networks for tissue elasticity reconstruction in magnetic resonance elastography, in: International Conference on Medical Image Computing and ComputerAssisted Intervention, Springer. pp. 333-343.

Rai, R., Sahu, C.K., 2020. Driven by data or derived through physics? a review of hybrid physics guided machine learning techniques with cyberphysical system (cps) focus. IEEE Access 8, 71050-71073.

Ruiz Herrera, C., Grandits, T., Plank, G., Perdikaris, P., Sahli Costabal, F., Pezzuto, S., 2022. Physics-informed neural networks to learn cardiac fiber orientation from multiple electroanatomical maps. Engineering with Computers 38, 3957-3973.

Saba, A., Gigli, C., Ayoub, A.B., Psaltis, D., 2022. Physics-informed neural networks for di ff raction tomography. Advanced Photonics 4, 066001.

- Sarabian, M., Babaee, H., Laksari, K., 2022. Physics-informed neural networks for brain hemodynamic predictions using medical imaging. IEEE Transactions on Medical Imaging 41, 2285-2303.

Sautory, T., Shadden, S.C., 2024. Unsupervised denoising and superresolution of vascular flow data by physics-informed machine learning. Journal of Biomechanical Engineering 146.

Shah, V., Joshi, A., Ghosal, S., Pokuri, B., Sarkar, S., Ganapathysubramanian, B., Hegde, C., 2019. Encoding invariances in deep generative models. arXiv preprint arXiv:1906.01626 .

Shen, L., Zhao, W., Capaldi, D., Pauly, J., Xing, L., 2022. A geometryinformed deep learning framework for ultra-sparse 3d tomographic image reconstruction. Computers in Biology and Medicine 148, 105710.

Shi, G., Wang, J., Qiang, Y., Yang, X., Zhao, J., Hao, R., Yang, W., Du, Q., Kazihise, N.G.F., 2020. Knowledge-guided synthetic medical image adversarial augmentation for ultrasonography thyroid nodule classification. Computer Methods and Programs in Biomedicine 196, 105611.

Shone, F., Ravikumar, N., Lassila, T., MacRaild, M., Wang, Y., Taylor, Z.A., Jimack, P., Dall'Armellina, E., Frangi, A.F., 2023. Deep physics-informed super-resolution of cardiac 4d-flow MRI, in: International Conference on Information Processing in Medical Imaging, Springer. pp. 511-522.

Tirindelli, M., Eilers, C., Simson, W., Paschali, M., Azampour, M.F., Navab, N., 2021. Rethinking ultrasound augmentation: A physicsinspired approach, in: Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part VIII 24, Springer. pp. 690-700.

Toghraie, D., Esfahani, N.N., Zarringhalam, M., Shirani, N., Rostami, S., 2020. Blood flow analysis inside di ff erent arteries using non-newtonian sisko model for application in biomedical engineering. Computer methods and programs in biomedicine 190, 105338.

- van der Velden, B.H., Kuijf, H.J., Gilhuijs, K.G., Viergever, M.A., 2022. Explainable artificial intelligence (xai) in deep learning-based medical image analysis. Medical Image Analysis 79, 102470.
- de Vries, L., van Herten, R.L., Hoving, J.W., Iˇ sgum, I., Emmer, B.J., Majoie, C.B., Marquering, H.A., Gavves, E., 2023. Spatio-temporal physicsinformed learning: A novel approach to ct perfusion analysis in acute ischemic stroke. Medical image analysis 90, 102971.

Wang, H., Liu, Y., Wang, S., 2022. Dense velocity reconstruction from particle image velocimetry particle tracking velocimetry using a physics-/ informed neural network. Physics of Fluids 34, 017116.

Weiss, T., Senouf, O., Vedula, S., Michailovich, O., Zibulevsky, M., Bronstein, A., 2019. Pilot: Physics-informed learned optimized trajectories for accelerated MRI. arXiv preprint arXiv:1909.05773 .

Wu, J.L., Kashinath, K., Albert, A., Chirila, D., Xiao, H., et al., 2020. Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems. Journal of Computational Physics 406, 109209.

Xia, W., Shan, H., Wang, G., Zhang, Y., 2023. Physics- model-based and / data-driven methods for low-dose computed tomography: A survey. IEEE signal processing magazine 40, 89-100.

Yang, Y., Perdikaris, P., 2019. Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems. Computational Mechanics 64, 417-434.

- Zapf, B., Haubner, J., Kuchta, M., Ringstad, G., Eide, P.K., Mardal, K.A., 2022. Investigating molecular transport in the human brain from MRI with physics-informed neural networks. Nature Scientific Reports 12, 15475.

Zhang, J., Yan, R., Perelli, A., Chen, X., Li, C., 2024. Phy-di ff : Physicsguided hourglass di ff usion model for di ff usion MRI synthesis. arXiv preprint arXiv:2406.03002 .

Zhang, J., Zhao, Y., Shone, F., Li, Z., Frangi, A.F., Xie, S.Q., Zhang, Z.Q., 2022. Physics-informed deep learning for musculoskeletal modeling: Predicting muscle forces and joint kinematics from surface emg. IEEE Transactions on Neural Systems and Rehabilitation Engineering 31, 484-493.

Zhang, X., Mao, B., Che, Y., Kang, J., Luo, M., Qiao, A., Liu, Y., Anzai, H., Ohta, M., Guo, Y., et al., 2023. Physics-informed neural networks (PINNs) for 4d hemodynamics prediction: An investigation of optimal framework based on vascular morphology. Computers in Biology and Medicine 164, 107287.

Zhang, X., Wu, J., Coutier-Delgosha, O., Xiao, H., 2019. Recent progress in augmenting turbulence models with physics-informed machine learning. Journal of Hydrodynamics 31, 1153-1158.

Zheng, M., Lou, F., Huang, Y., Pan, S., Zhang, X., 2024. Mr-based electrical property tomography using a physics-informed network at 3 and 7 t. NMR in Biomedicine , e5137.

Zhu, M., Zhu, Q., Song, Y., Guo, Y., Zeng, D., Bian, Z., Wang, Y., Ma, J., 2023. Physics-informed sinogram completion for metal artifact reduction in ct imaging. Physics in Medicine &amp; Biology 68, 065006.

Zimmermann, F.F., Kolbitsch, C., Schuenke, P., Kofler, A., 2024. Pinqi: an end-to-end physics-informed approach to learned quantitative MRI reconstruction. IEEE Transactions on Computational Imaging .
Chayan Banerjee Kien Nguyen Olivier Salvado Truyen Tran Clinton Fookes
PINNs for Medical Image Analysis: A Survey
The incorporation of physical information in machine learning frameworks is
transforming medical image analysis (MIA). By integrating fundamental knowledge
and governing physical laws, these models achieve enhanced robustness and
interpretability. In this work, we explore the utility of physics-informed
approaches for MIA (PIMIA) tasks such as registration, generation,
classification, and reconstruction. We present a systematic literature review
of over 80 papers on physics-informed methods dedicated to MIA. We propose a
unified taxonomy to investigate what physics knowledge and processes are
modelled, how they are represented, and the strategies to incorporate them into
MIA models. We delve deep into a wide range of image analysis tasks, from
imaging, generation, prediction, inverse imaging (super-resolution and
reconstruction), registration, and image analysis (segmentation and
classification). For each task, we thoroughly examine and present in a tabular
format the central physics-guided operation, the region of interest (with
respect to human anatomy), the corresponding imaging modality, the dataset used
for model training, the deep network architecture employed, and the primary
physical process, equation, or principle utilized. Additionally, we also
introduce a novel metric to compare the performance of PIMIA methods across
different tasks and datasets. Based on this review, we summarize and distil our
perspectives on the challenges, open research questions, and directions for
future research. We highlight key open challenges in PIMIA, including selecting
suitable physics priors and establishing a standardized benchmarking platform.