## Automated Sperm Morphology Analysis Based on Instance-Aware Part Segmentation

Wenyuan Chen 1† , Haocong Song 1† , Changsheng Dai 2 , Aojun Jiang 1 , Guanqiao Shan 1 , Hang Liu 1 , Yanlong Zhou 3 , Khaled Abdalla 4 , Shivani N Dhanani 4 , Katy Fatemeh Moosavi 4 , Shruti Pathak 4 , Clifford Librach , Zhuoran Zhang 5 , and Yu Sun 1 4

Abstract -Traditional sperm morphology analysis is based on tedious manual annotation. Automated morphology analysis of a high number of sperm requires accurate segmentation of each sperm part and quantitative morphology evaluation. Stateof-the-art instance-aware part segmentation networks follow a 'detect-then-segment" paradigm. However, due to sperm's slim shape, their segmentation suffers from large context loss and feature distortion due to bounding box cropping and resizing during ROI Align. Moreover, morphology measurement of sperm tail is demanding because of the long and curved shape and its uneven width. This paper presents automated techniques to measure sperm morphology parameters automatically and quantitatively. A novel attention-based instance-aware part segmentation network is designed to reconstruct lost contexts outside bounding boxes and to fix distorted features, by refining preliminary segmented masks through merging features extracted by feature pyramid network. An automated centerlinebased tail morphology measurement method is also proposed, in which an outlier filtering method and endpoint detection algorithm are designed to accurately reconstruct tail endpoints. Experimental results demonstrate that the proposed network outperformed the state-of-the-art top-down RP-R-CNN by 9 2% . AP p vol , and the proposed automated tail morphology measurement method achieved high measurement accuracies of 95 34% 96 39% 91 20% . , . , . for length, width and curvature, respectively.

## I. INTRODUCTION

Quantitative analysis of sperm morphology is vital for diagnosing male infertility [1]. Traditionally, sperm morphology analysis is performed manually by clinical staff who empirically follow the World Health Organization (WHO) guidelines [2] to analyze each sperm. Specifically, an experienced operator carefully observes zoomed-in sperm images to performs pixel-by-pixel annotation to label 5 contours for each part of the sperm (i.e., acrosome, vacuole, nucleus, midpiece and tail). Based on the annotated parts/contours, sperm morphology parameters are then manually calculated. Manual analysis of sperm morphology is a highly tedious process. As required by WHO, the morphology of at least 200 sperm should be analyzed within each semen sample,

1 Robotics Institute, University of Toronto, Canada. 2 School of Mechanical Engineering, Dalian University of Technology, Dalian, China. 3 School Of Artificial Intelligence, Henan University, Henan, China. 4 CReATe Fertility Center, Toronto, Canada. 5 School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China.

† The first two authors contribute equally to this work.

Corresponding authors: Changsheng Dai (daichangsheng@dlut.edu.cn); Zhuoran Zhang (zhangzhuoran@cuhk.edu.cn); Clifford Librach (drlbrach@createivf.com); Yu Sun (yu.sun@utoronto.ca).

(a) Input Image

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00112v1-figure-9e0e0fae-5f21-4071-aeeb-1d9a41175846.jpg)

(d) Centreline Fitting

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00112v1-figure-4ce638d7-9754-4334-a966-e5bb3bf0917d.jpg)

(b) Instance-aware Part Segmentation

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00112v1-figure-f00b1ea1-cfac-4a4a-8ff6-feb3b96f52b4.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00112v1-figure-ad325cff-7c67-453b-95ca-086b9feb0836.jpg)

(c) Ellipse and Rectangle Fitting

Fig. 1. (a) An image of stained sperm. Each sperm is to be segmented into five parts: acrosome, vacuole, nucleus, midpiece and tail. (b) Instanceaware part segmentation that not only distinguishes different sperm, but also segments parts for each sperm. (c) Ellipse and rectangle fitting for measuring sperm head and midpiece morphology parameters. (d) Centerline fitting for measuring sperm tail morphology parameters.

leading to manual annotation of &gt; , 1 000 contours per patient sample, which calls for techniques to automate the analysis task.

To achieve automated sperm morphology measurement, firstly, an instance-level sperm parsing is needed that not only correctly detects sperm but also accurately segments the parts for each sperm (see Fig. 1(a)&amp;(b)). Secondly, an automated morphology measurement method is needed for quantitatively calculating morphology parameters for each sperm part based on the segmented mask.

For the first step of instance-aware part segmentation (see Fig. 1(b)), state-of-the-art instance-aware part segmentation networks from the human parsing field employs a 'detectthen-segment" paradigm [3], [4], [5]. Firstly, various sperm are detected by bounding boxes. Then each sperm within a bounding box is cropped and resized to have a uniform shape through ROI Align [6]. Finally, the sperm is segmented into individual parts. This paradigm is a top-down method [5]. However, the top-down method is sensitive to detection errors since contexts outside a bounding box are cropped out (see Fig. 2(a)). Also, the resizing step in ROI Align leads to

Fig. 2. (a) Context loss due to bounding box cropping. (b) Feature distortion due to resizing in ROI Align. (c) Endpoints mislocated in Steger-based methods. (d) The normal of endpoints is mislocated due to the influence of gradient from the intersecting edge.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00112v1-figure-a4df3668-c74b-43a4-8db4-82321b682a55.jpg)

feature distortion and inaccurate contour segmentation (see Fig. 2(b)). Moreover, unlike traditional segmentation targets such as human, sperm is long and slim (high length-to-width ratio); thus, its segmentation suffers from higher context loss resulting from more detection errors and larger feature distortion because of greater aspect ratio resizing. Therefore, the top-down method produces larger segmentation errors for sperm analysis.

After segmentation, an automated sperm tail morphology measurement method is required. Sperm head parameters (e.g., length, width, and ellipticity) and midpiece parameters (e.g., length, width, and rotated angle) can be calculated through ellipse [7] and rectangle [8] fitting (see Fig. 1(c)). In comparison, morphology measurement of sperm tail is more challenging because the tail is long, curved, with uneven in width and cannot be easily fitted. Methods have been proposed for measuring curvilinear structures [9], [10], [11], [12], among which Steger-based methods [11], [12] are advantageous due to their sub-pixel accuracy. In Steger-based methods, points whose first derivative reaches zero along the normal are detected as centerline points. Then, width can be calculated according to distances between centerline points and edge points along the normal, and curvature can be calculated as the change of the normal of the centerline points (see Fig. 1(d)). However, endpoints detected by Steger-based methods tend to mislocate (see Fig. 2(c)) because endpoints' normal is affected by gradient from the intersecting edge and not perpendicular to the line direction (see Fig. 2(d)).

The problems tackled in this work include: 1) how to avoid context loss and feature distortion during the instanceaware part segmentation for sperm; 2) how to accurately reconstruct endpoints for measuring sperm tail morphology parameters. To address the first challenge, a novel attentionbased instance-aware part segmentation network is proposed to reconstruct lost contexts outside a bounding box and fix distorted features by refining masks generated following the 'detect-then-segment" paradigm. Specifically, the pre- liminary segmented masks are used to provide spatial cues for each located sperm, then merged with features extracted by a feature pyramid network (FPN) through the attention mechanism, finally refined by CNN to produce improved results. For the second challenge, an automated centerlinebased tail morphology measurement method is proposed, in which an outlier filtering method and endpoint detection algorithm are designed to accurately reconstruct endpoints.

Experimental results demonstrate that the proposed network achieved 57 2% AP . p vol (Average Precision based on part) on the collected dataset, outperforming the state-of-theart top-down RP-R-CNN by 9 20%; and the proposed tail . morphology measurement method substantially decreased measurement errors by 4 21% 3 02% 13 10% . , . , . for length, width and curvature, respectively compared with Stegerbased methods.

## II. RELATED WORKS

This section reviews instance-aware part segmentation works from the human parsing field, then discusses existing works on the measurement of sperm morphology parameters.

## A. Instance-Aware Part Segmentation

The instance-aware part segmentation network was originally proposed in the human parsing field, with the aim of segmenting various human parts (e.g., arm, leg and hair) and associating each segmented part with the corresponding instance. State-of-the-art methods can be categorized into bottom-up methods and top-down methods. The bottomup methods [13], [14], [15] first segment each part of the target as in semantic segmentation [16], [17], then group segmented parts into corresponding instances. For example, the part grouping network (PGN) [13] utilizes line information to separate adjacent instances. Gray-ML [14] and Graphonomy [15] employ graph neural networks to explore underlying label semantic relations. Although the bottomup methods are suitable for global semantic segmentation, their instance distinction ability is poor since they lack the bounding box locating procedure.

To accurately separate adjacent instances, the top-down methods first locate each instance within the image using bounding box detection, then segment parts for each located instance individually. Based on Mask R-CNN [18], the classical instance segmentation network, Parsing R-CNN [3] utilizes a geometric and context encoding module for improving part segmentation accuracy. RP-R-CNN [4] further boosts the parsing ability by introducing a global semantic network. AIparsing [5] utilizes FCOS [19], an anchor-free detector instead of the anchor-based detector such as in Mask R-CNN to improve detection performance. Although the top-down methods have strong instance distinction ability, they suffer from context loss and feature distortion due to bounding box cropping and resizing in ROI Align.

## B. Measurement of Sperm Morphology Parameters

Existing works mostly focused on sperm head and midpiece. In [7], [8], head morphology parameters and midpiece morphology parameters were calculated by ellipse and rectangle fitting. In [20], the head-midpiece angle was calculated as the angle between the major axis of head and the major axis of midpiece. However, morphology measurement of sperm tail is challenging and mostly ignored because sperm tail is long, curved, with uneven width and cannot be easily fitted by ellipse or rectangle. The methods proposed for measuring curvilinear structures can be categorized into pixel level methods and sub-pixel level methods. Pixel level methods include skeleton extraction methods [21] and direction template methods [22]. To improve measurement accuracy, sub-pixel level methods were developed, including grey centroid methods [9], [10] and Steger-based methods [11], [12]. The grey centroid methods first extract centroid points through vertically or horizontally line-by-line scanning, then link all centroid points to form the centerline. However, these methods perform poorly if the curvilinear structure is not directed either vertically nor horizontally [23]. In comparison, Steger-based methods calculate center points along the normal direction of centerlines, which are not affected by the direction of the curvilinear structure [24].

## III. METHODS

To achieve quantitative morphology measurement for sperm parts including head, midpiece and tail, two key techniques are developed in this work. A novel attention-based instance-aware part segmentation network is designed to accurately segment sperm parts; and an automated centerlinebased tail morphology measurement method is developed to quantitatively calculate tail morphology parameters based on the segmented masks.

A. Attention-Based Instance-Aware Part Segmentation Network

The overall structure of the proposed network is shown in Fig. 3. The input image is first fed into the convolutional backbone to extract features, then the extracted features are rescaled by the Feature Pyramid Network (FPN) [25] to acquire multi-scale features. Next, the preliminary segmentation module generates instance-level parsing masks following the top-down methods [3], [4], [5]. Firstly, sperm within the image are located by bounding boxes using candidate ROIs (region of interests) from the Region Proposal Network (RPN) [6] in the detection branch. Secondly, each sperm within a bounding box is cropped and resized by ROI Align to have a uniform shape, finally segmented into part masks separately. It is worth noting that segmentation is performed on P2 features generated by FPN because P2 features in Fig. 3 have the highest resolution and contain more details. However, similar to top-down methods, the parsing masks produced by the preliminary segmentation module suffer from context loss and feature distortion (see Fig. 2(a)&amp;(b)).

To solve this issue, an attention-based refinement module is proposed to refine preliminary segmented masks. In the proposed refinement module (see Fig. 3), parsing masks from the preliminary segmentation module are used to provide spatial cues of each located sperm instance, and features

Fig. 3. The structure of our proposed attention-based instance-aware part segmentation network. The convolutional backbone and FPN first extracts features from the input image then rescales extracted features to multiscale. Next, the preliminary segmentation module generates instance-level parsing masks as in top-down methods. Finally, the refinement module refines preliminary generated masks by merging features extracted by FPN through the attention mechanism. Besides, the edge information is utilized to better separate boundary between adjacent sperm.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00112v1-figure-5bad57aa-a88c-42dc-bbf4-c0c5a274d2ad.jpg)

extracted by FPN are used to compensate for lost details. Then parsing masks and extracted features are merged through the attention mechanism and later refined by CNN to produce improved instance-aware results. Moreover, edge information is incorporated to better separate the boundary between adjacent sperm.

Attention mechanism [26], [27] has proven to be an effective tool to selectively emphasize specific regions of an image. Therefore, the preliminary segmented masks can be used to indicate the location for each sperm during segmentation using the attention mechanism. Specifically, parsing masks { M i } n i = 1 (for n sperm) are first reshaped and re-projected back to their original position in the image using the bounding box information. Then, the re-projected masks { M i r } n i = 1 go through a 1 × 1 convolution and sigmoid activation to generate attention maps according to

$$A ^ { i } = S i g m o i d ( C o n v _ { 1 \times 1 } ( M _ { r } ^ { i } ) ) \text{ \quad \ \ } ( 1 )$$

where A i denotes the attention map encoding spatial information for the i -th sperm. As for extracted features from FPN, multi-scale features (i.e., P2, P3, P4, P5) are merged so that targets covered by small to large receptive fields are incorporated. Specifically, P3, P4, P5 are first upsampled to the same size as P2, then P2 and upsampled P3, P4, P5 are concatenated and go through a 1 × 1 convolution to produce fused features according to

$$F _ { f u s e } = C o n v _ { 1 \times 1 } ( C o n c a t [ F _ { P 2 }, F _ { P 3 }, F _ { P 4 }, F _ { P 5 } ] ) \quad ( 2 )$$

where Ffuse is the fused multi-scale features. Attention maps { A i } n i = 1 are then multiplied with fused features individually to produce instance-aware part features for each sperm. Finally, the produced instance-aware part features are refined by cascaded dilated convolution layers [28] with dilation rates of 1 and 2 to generate accurate parsing masks for each sperm by

$$\{ S _ { j } ^ { i } \} _ { j = 1 ; m } = C o n v _ { 1 \times 1 } ( D i l a t e d C o n v s ( A ^ { i } \odot F _ { f u s e } ) ) \quad ( 3 )$$

where { S i j } j = 1: m is the segmentation result for the i -th sperm with m parts, and the final output for n sperm are { S i j } i = 1: n j = 1: m . In addition, the edge-guided branch is added in parallel to the parsing branch to provide boundary information.

As for the loss, weighted cross entropy loss [29] and dice loss [30] are employed since sperm have a low area ratio compared with the background. The loss for the refinement module is

$$\mathcal { L } _ { r e f i n e m e n t } = \mathcal { L } _ { p a r s i n g } ^ { \text{WCE} } + \mathcal { L } _ { e d g e } ^ { \text{WCE} } + \lambda \mathcal { L } _ { p a r s i n g } ^ { \text{Dice} } + \lambda \mathcal { L } _ { e d g e } ^ { \text{Dice} } \ ( 4 ) \quad \text{center} \ p \$$

where L WCE parsing , L WCE edge , L Dice parsing , L Dice edge are weighted cross entropy loss and dice loss for the parsing branch and the edge branch respectively; and λ is a manually selected weight.

B. Automated Centerline-Based Tail Morphology Measurement

In Steger-based methods [11], [12], partial derivatives rx , ry , rxx , rxy , ryy of each pixel are first obtained by convolving the image with a discrete 2-D Gaussian partial derivative kernels. Then the normal direction n ( ) = [ t nx , ny ] T for each pixel is determined by calculating the eigenvalues and eigenvectors of the Hessian matrix H x y ( , ) = [ rxx rxy rxy ryy ] . According to [11], [12], the normal direction is perpendicular to the direction of curve-line, and centerline points are selected as points whose first derivative reaches zero (i.e., grey value reaches maximum) along normal (perpendicular to the line). Finally, points with minimum distance and minimum orientation change of normal are linked to form the centerline. However, endpoints detected by Steger-based methods tend to mislocate (see Fig. 2(c)) because their normal is affected by gradient from the intersecting edge and not perpendicular to the direction of line (see Fig. 2(d)).

To solve this issue, an outlier filtering method and an endpoint detection algorithm are proposed. The outlier filtering method is used to filter mislocated endpoints produced by Steger-based methods, and the endpoint detection algorithm is employed to reconstruct correct endpoints.

As shown in Fig. 4(a), the correctly detected point lies in the middle of two opposite edges while the mislocated point lies on the diagonal of two intersecting edges. Therefore, the normal lines that cross the correctly detected and mislocated point intersect with opposite edges and intersecting edges correspondingly. Suppose P 1, P 2 are correctly detected and mislocated point, respectively, N 1, N 2 are the corresponding normal, and E 11, E 12, E 21, E 22 are corresponding intersecting points with edges. Since normal N 1 is perpendicular to the direction of line [11], the gradient for E 11, E 12 (on the opposite edges) are close to parallel. On the other hand, since E 21, E 22 are on intersecting edges, the gradient for E 21, E 22 are not possible to be parallel. Thus, a constraint can be formed to discriminate correctly detected and mislocated points, i.e.,

Fig. 4. (a) Normal lines intersect with opposite edges ( E 11 , E 12) and intersecting edges ( E 21 , E 22) for correctly detected point P 1 and mislocated point P 2. Correspondingly, the gradient G 11 , G 12 for E 11 , E 12 are close to parallel, while gradient G 21 , G 22 for E 21 , E 22 are not. (b) The gradient for center point P can be decomposed to Gx ′ and Gy ′ along the line direction and perpendicular to the line direction. According to [12], Gy ′ = 0 (center point's gradient perpendicular to the line direction is zero); therefore, the center point's gradient direction is along the direction of line.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00112v1-figure-c867dfb4-5aa8-4317-ae06-524e73e0c2cf.jpg)

$$\begin{cases} \cos \alpha = \frac { r x _ { 1 } \times r x _ { 2 } + r y _ { 1 } \times r y _ { 2 } } { \sqrt { r x _ { 1 } ^ { 2 } + r y _ { 1 } ^ { 2 } } \times \sqrt { r x _ { 2 } ^ { 2 } + r y _ { 2 } ^ { 2 } } } \\ | \cos \alpha | \geq 0. 9 \end{cases}$$

where rx 1, ry 1 and rx 2, ry 2 are first partial derivatives for edge points E 1 and E 2, receptively.

After filtering mislocated points, the next step is to reconstruct missing endpoints. Without loss of generality, for any point P on the centerline, its gradient G can be decomposed into two directions Gx ′ and Gy ′ , along the direction of the line and the direction perpendicular to the line (see Fig. 4(b)). According to [11], [12], the first derivative (gradient) for center points along the normal direction (perpendicular to the line) is zero (i.e., Gy ′ = 0), which means G = Gx ′ and gradient direction for point P is along the direction of line. Thus, an endpoint detection algorithm is proposed by interpolating points along the direction of center point's gradient. As shown in Fig. 4(b), for a point PA ( cx , cy ) , if the angle of gradient is within [ 0 ◦ , 45 ◦ ] , then point PB ( cx + 1 , cy + ) 1 and PC ( cx + 1 , cy ) along gradient direction of PA are selected as candidate points. Then, the candidate point that minimizes w 1 × d + w 2 × β is selected as the next point, where d is distance between candidate point to gradient of current point, β is the angle difference between the gradient of candidate point and current point, and w 1 , w 2 are weights. Moreover, to avoid local minima and smooth the line, gradient for the selected next point is updated using the momentum technique [31] as

$$g _ { t + 1 } ^ { \prime } = \alpha \cdot g _ { t } + ( 1 - \alpha ) \cdot g _ { t + 1 } \text{ \quad \ \ } ( 6 )$$

where gt , gt + 1 is the gradient for current point and next point, α is the momentum hyper-parameter, and g t ′ + 1 is the updated gradient. Meanwhile, the normal for reconstructed endpoints is corrected as the direction perpendicular to gradient. This algorithm is continued until the interpolated point is out of the tail area.

Fig. 5. Qualitative comparisons of instance-aware part segmentation networks. PGN (bottom-up method) has low instance distinction and cannot separate intersecting sperm parts. RP-R-CNN (top-down method) has distorted features and contexts outside a bounding box are cropped out. In comparison, our proposed network achieves the best prediction by refining preliminary segmented masks to retrieve lost contexts outside the bounding box and fix distorted features.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00112v1-figure-63d74149-95dd-408f-846f-4aea4f09a6ae.jpg)

TABLE I QUANTITATIVE COMPARISONS OF INSTANCE-AWARE PART SEGMENTATION NETWORKS

| Methods           | Backbone   | mIOU   | AP p vol   | AP p 50   | PCP 50   |
|-------------------|------------|--------|------------|-----------|----------|
| Bottom-up:        |            |        |            |           |          |
| PGN [13]          | ResNet-101 | 57.5   | 25.9       | 33.9      | 30.0     |
| GPM [14]          | Xception   | 58.2   | -          | -         | -        |
| Deeplab V3+ [32]  | Xception   | 57.7   | -          | -         | -        |
| Top-down:         |            |        |            |           |          |
| Parsing R-CNN [3] | ResNet-50  | 54.7   | 47.7       | 56.7      | 51.4     |
| RP-R-CNN [4]      | ResNet-50  | 59.6   | 48.0       | 62.4      | 53.9     |
| AIParsing [5]     | ResNet-50  | 60.7   | 47.3       | 57.5      | 58.7     |
| Ours              | ResNet-50  | 69.1   | 57.0       | 76.1      | 68.9     |
| Ours              | ResNet-101 | 69.3   | 57.2       | 76.8      | 69.5     |

## IV. EXPERIMENTS AND RESULTS

## A. Datasets and Evaluation Metric

In this work, we collected a new dataset 1 for the sperm part segmentation task to distinguish unique sperm instances and also segment parts (i.e., acrosome, vacuole, nucleus, midpiece and tail) for each sperm. All images were manually labelled with each part of the sperm assigned both an instance ID and a part label. Each image has the resolution of 1280 × 1024 and includes 5-6 sperm averagely. The dataset contains 320 images in total, with 80% of them randomly selected as training samples and 20% randomly selected as testing samples.

To evaluate the performance of our proposed network, the standard mean intersection over union (mIoU) [33] was adopted for evaluating global semantic segmentation; and the Average Precision based on part (AP p ) [34] and Percentage of Correctly parsed semantic Parts (PCP) [34] were used to measure instance-level segmentation accuracy. Specifically, the AP p 50 , AP p vol and PCP50 values were reported, where the first and third metric has an IoU threshold of 0.5, and the secondly metric is the mean of a series of IoU thresholds ranging from 0.1 to 0.9, with the increment of 0.1.

## B. Comparisons with State-of-the-art Models

To evaluate the performance of sperm part segmentation, the proposed network was compared with several state-ofthe-art instance-aware part segmentation networks, including PGN [13], GPM [14], Deeplab V3+ [32] for bottom-up methods; and Parsing R-CNN [3], RP-R-CNN [4], AIParsing [5] for top-down methods, all using our collected dataset. Qualitative and quantitative comparison results are summarized in Fig. 5 and Table I, respectively. As can be seen, bottom-up methods gave low AP p vol , AP 50 p and PCP50 for instance-level segmentation (see Table I) and cannot separate intersecting sperm parts (see Fig. 5). This is because they have low instance distinction due to the lack of the bounding box locating procedure. On the other hand, topdown methods achieved significantly higher AP p vol , AP 50 p and PCP50 in instance-level segmentation compared with bottomup methods (e.g., 22 1%, 28 5% and 23 9% gains in AP . . . p vol , AP 50 p and PCP50 for RP-R-CNN vs. PGN). This is because top-down methods firstly locate each sperm within an image through detection and can better separate adjacent sperm.

In comparison, the proposed network further outperformed top-down methods by refining preliminarily generated masks to produce improved results. Specifically, our proposed network achieved 9 7%, 9 2%, 14 4% and 15 6% gain in terms . . . . of mIoU, AP p vol , AP 50 p and PCP50, respectively compared

TABLE II

## QUANTIFICATION OF MORPHOLOGY PARAMETERS OF SINGLE SPERM (AU:ARBITRARY UNIT)

|       | Head          | Head         | Head             | Acrosome area 2   | Nucleus area ( µ m 2 )   | Vacuole     | Vacuole        | Head-Midpiece angle ◦   | Midpiece      | Midpiece     | Midpiece         | Tail          | Tail         | Tail             |
|-------|---------------|--------------|------------------|-------------------|--------------------------|-------------|----------------|-------------------------|---------------|--------------|------------------|---------------|--------------|------------------|
| Sperm | length ( µ m) | width ( µ m) | ellipticity (AU) | ( µ m )           |                          | number (AU) | area ( µ m 2 ) | ( )                     | length ( µ m) | width ( µ m) | angle(max) ( ◦ ) | length ( µ m) | width ( µ m) | angle(max) ( ◦ ) |
| 1     | 4.67          | 2.69         | 1.74             | 3.54              | 6.79                     | 1           | 0.33           | 27.16                   | 3.68          | 0.58         | 5.27             | 31.38         | 0.65         | 20.91            |
| 2     | 5.05          | 2.57         | 1.96             | 2.89              | 7.28                     | 1           | 0.59           | 2.30                    | 4.27          | 0.60         | 7.61             | 27.58         | 0.59         | 18.83            |
| 3     | 5.08          | 2.67         | 1.90             | 4.37              | 6.48                     | 2           | 0.93           | 50.60                   | 2.17          | 0.43         | 37.27            | 20.85         | 0.58         | 15.83            |
| 4     | 4.47          | 2.61         | 1.71             | 3.57              | 6.04                     | 0           | NA             | 0.19                    | 3.55          | 0.59         | 8.84             | 41.95         | 0.60         | 23.97            |

with the state-of-the-art RP-R-CNN. Moreover, as shown in Fig. 5, the proposed network can more effectively retrieve lost contexts outside a bounding box and fix distorted features compared with RP-R-CNN thanks to the refining procedure. Therefore, the proposed network yielded segmentation results that were closer to the ground truth with more details and less distortion.

## C. Quantification of Morphology Parameters

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00112v1-figure-96e3bde0-6972-4977-848f-3d16f818fb67.jpg)

Sperm morphology measurement was performed on the segmented masks generated from the proposed network. Quantified values for each part are summarized in Table II, where Sperm 1,2,3,4 correspond to the sperm shown in the third row of Fig. 5. The head and midpiece parameters were calculated through ellipse and rectangle fitting [7], [20], while the tail parameters were calculated using the proposed automated centerline-based tail morphology measurement method (see demo in Video).

To evaluate the performance of tail morphology measurement, an additional experiment was conducted on 50 sperm using classical Steger-based methods and our proposed method. In this experiment, the length, width and curvature of sperm tail were used for quantitative comparison, with the benchmark data obtained by manually benchmarking zoomed-in masks in ImageJ with best care [7]. Qualitative and quantitative comparison results are summarized in Fig. 6(a)(b)(c). The errors of tail length, width and curvature measurements with the Steger-based method were 8 87% 6 63% 21 90%, respectively. These error were mostly . , . , . caused by mislocated endpoints and had significant impact on curvature measurement (see Fig. 6(a)). In comparison, our proposed method successfully reconstructed mislocated endpoints using the proposed outlier filtering method and endpoint detection algorithm (see Fig. 6(b)), and substantially decreased measurement errors by 4 21% 3 02% 13 10% for . , . , . tail length, width and curvature, respectively (see Fig. 6(c)). The errors for tail measurement with our proposed method were &lt; 10% for length, width and curvature, well meeting the requirement for determining normal/abnormal sperm [7].

## V. CONCLUSION

This paper presented automated techniques for quantitatively measuring morphology parameters for each part of a sperm. An attention-based instance-aware part segmentation network was proposed to accurately segment each part of the sperm by avoiding context loss and feature distortion. An automated centerline-based tail morphology measurement

Fig. 6. (a) &amp; (b) Qualitative comparison results for Steger-based methods and our proposed method. The proposed method can effectively fix mislocated endpoints. (c) Quantitative comparison of errors for measuring tail length, width and curvature with Steger-based methods and the proposed method. The experiment was conducted on 50 sperm with manual benchmarking.

method was designed to quantitatively calculate tail parameters based on the segmented masks. Experiments showed that the proposed network outperformed existing state-of-the-art networks, and the proposed automated centerline-based tail morphology measurement method was capable of accurately quantifying tail morphology parameters.

## REFERENCES

- [1] K. S. Murray et al. , 'The effect of the new 2010 world health organization criteria for semen analyses on male infertility,' Fertility and sterility , vol. 98, no. 6, pp. 1428-1431, 2012.
- [2] W. H. Organization, 'Who laboratory manual for the examination and processing of human semen,' Geneva, Switzerland: WHO Press , 2010.
- [3] L. Yang, , et al. , 'Parsing r-cnn for instance-level human analysis,' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 364-373.
- [4] L. Yang et al. , 'Renovating parsing r-cnn for accurate multiple human parsing,' in European Conference on Computer Vision . Springer, 2020, pp. 421-437.
- [5] S. Zhang et al. , 'Aiparsing: anchor-free instance-level human parsing,' IEEE Transactions on Image Processing , vol. 31, pp. 5599-5612, 2022.
- [6] S. Ren et al. , 'Faster r-cnn: Towards real-time object detection with region proposal networks,' Advances in neural information processing systems , vol. 28, 2015.
- [7] C. S. Dai et al. , 'Automated non-invasive measurement of sperm motility and morphology parameters,' in 2018 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2018, pp. 2682-2687.
- [8] C. Dai et al. , 'Staining-free, automated sperm analysis for in vitro fertilization lab use,' The Journal of Urology , vol. 208, no. 6, pp. 1303-1312, 2022.
- [9] R. Usamentiaga et al. , 'Fast and robust laser stripe extraction for 3d reconstruction in industrial environments,' Machine Vision and Applications , vol. 23, pp. 179-196, 2012.
- [10] Y. Li et al. , 'Sub-pixel extraction of laser stripe center using an improved gray-gravity method,' Sensors , vol. 17, no. 4, p. 814, 2017.
- [11] C. Steger, 'An unbiased detector of curvilinear structures,' IEEE Transactions on pattern analysis and machine intelligence , vol. 20, no. 2, pp. 113-125, 1998.
- [12] R. Yang et al. , 'Robust and accurate surface measurement using structured light,' IEEE Transactions on Instrumentation and Measurement , vol. 57, no. 6, pp. 1275-1280, 2008.
- [13] K. Gong et al. , 'Instance-level human parsing via part grouping network,' in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 770-785.
- [14] H. He et al. , 'Grapy-ml: Graph pyramid mutual learning for crossdataset human parsing,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 34, no. 07, 2020, pp. 10 949-10 956.
- [15] K. Gong et al. , 'Graphonomy: Universal human parsing via graph transfer learning,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 7450-7459.
- [16] Z. Min, F. J. Bianco, Q. Yang, R. Rodell, W. Yan, D. Barratt, and Y. Hu, 'Controlling false positive/negative rates for deep-learningbased prostate cancer detection on multiparametric mr images,' in Medical Image Understanding and Analysis: 25th Annual Conference, MIUA 2021, Oxford, United Kingdom, July 12-14, 2021, Proceedings 25 . Springer, 2021, pp. 56-70.
- [17] W. Yan, Q. Yang, T. Syer, Z. Min, S. Punwani, M. Emberton, D. Barratt, B. Chiu, and Y. Hu, 'The impact of using voxel-level segmentation metrics on evaluating multifocal prostate cancer localisation,' in International Workshop on Applications of Medical AI . Springer, 2022, pp. 128-138.
- [18] K. He et al. , 'Mask r-cnn,' in Proceedings of the IEEE international conference on computer vision , 2017, pp. 2961-2969.
- [19] Z. Tian et al. , 'Fcos: A simple and strong anchor-free object detector,' IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 44, no. 4, pp. 1922-1933, 2020.
- [20] Z. Zhang et al. , 'Quantitative selection of single human sperm with high dna integrity for intracytoplasmic sperm injection,' Fertility and Sterility , vol. 116, no. 5, pp. 1308-1318, 2021.
- [21] J.-H. Jang and K.-S. Hong, 'Detection of curvilinear structures and reconstruction of their regions in gray-scale images,' Pattern Recognition , vol. 35, no. 4, pp. 807-824, 2002.
- [22] L. Haijun et al. , 'A method for fast detecting the center of structured light stripe,' JOURNAL-HUAZHONG UNIVERSITY OF SCI-
- ENCE AND TECHNOLOGY NATURE SCIENCE CHINESE EDITION , vol. 31, no. 1, pp. 74-76, 2003.
- [23] L. He et al. , 'Robust laser stripe extraction for three-dimensional reconstruction based on a cross-structured light sensor,' Applied Optics , vol. 56, no. 4, pp. 823-832, 2017.
- [24] X. Xu et al. , 'Line structured light calibration method and centerline extraction: A review,' Results in Physics , vol. 19, p. 103637, 2020.
- [25] T.-Y. Lin et al. , 'Feature pyramid networks for object detection,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 2117-2125.
- [26] J. Hu et al. , 'Squeeze-and-excitation networks,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 7132-7141.
- [27] S. Woo et al. , 'Cbam: Convolutional block attention module,' in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 3-19.
- [28] F. Yu and V. Koltun, 'Multi-scale context aggregation by dilated convolutions,' arXiv preprint arXiv:1511.07122 , 2015.
- [29] O. Ronneberger et al. , 'U-net: Convolutional networks for biomedical image segmentation,' in Medical Image Computing and ComputerAssisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 . Springer, 2015, pp. 234-241.
- [30] F. Milletari et al. , 'V-net: Fully convolutional neural networks for volumetric medical image segmentation,' in 2016 fourth international conference on 3D vision (3DV) . Ieee, 2016, pp. 565-571.
- [31] I. Sutskever et al. , 'On the importance of initialization and momentum in deep learning,' in International conference on machine learning . PMLR, 2013, pp. 1139-1147.
- [32] L.-C. Chen et al. , 'Encoder-decoder with atrous separable convolution for semantic image segmentation,' in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 801-818.
- [33] J. Long et al. , 'Fully convolutional networks for semantic segmentation,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 3431-3440.
- [34] J. Zhao et al. , 'Understanding humans in crowded scenes: Deep nested adversarial learning and a new benchmark for multi-human parsing,' in Proceedings of the 26th ACM international conference on Multimedia , 2018, pp. 792-800.
Wenyuan Chen Haocong Song Changsheng Dai Aojun Jiang Guanqiao Shan Hang Liu Yanlong Zhou Khaled Abdalla Shivani N Dhanani Katy Fatemeh Moosavi Shruti Pathak Clifford Librach Zhuoran Zhang Yu Sun
Automated Sperm Morphology Analysis Based on Instance-Aware Part Segmentation
Traditional sperm morphology analysis is based on tedious manual annotation.
Automated morphology analysis of a high number of sperm requires accurate
segmentation of each sperm part and quantitative morphology evaluation.
State-of-the-art instance-aware part segmentation networks follow a
"detect-then-segment" paradigm. However, due to sperm's slim shape, their
segmentation suffers from large context loss and feature distortion due to
bounding box cropping and resizing during ROI Align. Moreover, morphology
measurement of sperm tail is demanding because of the long and curved shape and
its uneven width. This paper presents automated techniques to measure sperm
morphology parameters automatically and quantitatively. A novel attention-based
instance-aware part segmentation network is designed to reconstruct lost
contexts outside bounding boxes and to fix distorted features, by refining
preliminary segmented masks through merging features extracted by feature
pyramid network. An automated centerline-based tail morphology measurement
method is also proposed, in which an outlier filtering method and endpoint
detection algorithm are designed to accurately reconstruct tail endpoints.
Experimental results demonstrate that the proposed network outperformed the
state-of-the-art top-down RP-R-CNN by 9.2% [AP]_vol^p, and the proposed
automated tail morphology measurement method achieved high measurement
accuracies of 95.34%,96.39%,91.2% for length, width and curvature,
respectively.