A deep learning model for mass screening of COVID‐19

Abstract The objective of this research is to develop a convolutional neural network model ‘COVID‐Screen‐Net’ for multi‐class classification of chest X‐ray images into three classes viz. COVID‐19, bacterial pneumonia, and normal. The model performs the automatic feature extraction from X‐ray images and accurately identifies the features responsible for distinguishing the X‐ray images of different classes. It plots these features on the GradCam. The authors optimized the number of convolution and activation layers according to the size of the dataset. They also fine‐tuned the hyperparameters to minimize the computation time and to enhance the efficiency of the model. The performance of the model has been evaluated on the anonymous chest X‐ray images collected from hospitals and the dataset available on the web. The model attains an average accuracy of 97.71% and a maximum recall of 100%. The comparative analysis shows that the ‘COVID‐Screen‐Net’ outperforms the existing systems for screening of COVID‐19. The effectiveness of the model is validated by the radiology experts on the real‐time dataset. Therefore, it may prove a useful tool for quick and low‐cost mass screening of patients of COVID‐19. This tool may reduce the burden on health experts in the present situation of the Global Pandemic. The copyright of this tool is registered in the names of authors under the laws of Intellectual Property Rights in India with the registration number ‘SW‐13625/2020’.

The accuracy of 'RDT based on antigen detection' is dependent on the concentration of viruses in the sample collected from the respiratory tract. On the other hand, the accuracy of 'RDT based on host antibody' is dependent on the number of antibodies present in the blood of the patient. The antibody formation is highly decided by the age, nutrition, and prior health conditions of the person. 4 These tests are not reliable due to low accuracy and dependency on other factors such as age, nutrition, health history, etc. Therefore, the WHO does not recommend the above-stated tests for diagnostic purposes. But, the usage of these tests for research is encouraged. 5 WHO recommended molecular tests such as Polymerase Chain Reaction (PCR) as a confirmatory and laboratory test for COVID-19. 6 But, the test is restricted to the limited number of Government and Private Laboratories. Moreover, the effectiveness of this test is dependent on the collection, packaging, and shipment of the samples. Its sensitivity is reported as 60%-70%. 7 The low accuracy of the PCR test motivates researchers and health experts to introduce more ways of diagnosis. The radiological examination has been a reliable diagnosing technique for infectious diseases such as bacterial pneumonia, viral pneumonia, and tuberculosis, etc. for many decades. The authors in Reference 8 claimed that the abnormality in the lungs is captured in the CXR. The changes in the shape, size, textures, and reflection symmetry between the lungs are examined to diagnose the abnormality or disease.
A discussion with a senior radiology expert clarified that there are remarkable differences in chest X-ray images of non-infected, patients of COVID-19, and patients infected with bacterial pneumonia. He added that the typical bacterial pneumonia causes the opacities or pleural effusion to a confined region. Also, there is a regular pattern of opacities in X-ray images of a person infected with bacterial pneumonia. The rate of spreading of bacterial pneumonia is slow. Therefore, the X-ray images captured after every 2 or 3 days will show a minor variation in opacities. On the other hand, the irregular pattern of white patches or opacities is observed throughout the lungs in patients of COVID-19. The opacities are bilateral and pleural effusion is observed in all the lobes of the lungs. The opacities are more common near hila. Also, the rate of spreading is very fast. The causing agent of COVID-19 infects both the lungs completely within a short period of 2-3 days. The infection shows its large impact on both the lungs after 2-3 days. The authors in References 9-11 also claimed that the chest X-ray images of confirmed patients of COVID-19 show multi-lobar opacities in the lungs. These opacities are ground glass and mixed attenuation.
The above discussion shows that the X-ray images can be a useful modality for the screening of 'COVID-19' due to remarkable differences captured for infected and healthy lungs. But, manual reading of X-ray images is a time-consuming task. It is difficult for health experts to guarantee a quick response in the present situation of the global pandemic. Therefore, it is mandatory to search for an alternative method of screening. The Deep Learning (DL) techniques automated the process of feature extraction and feature selection. 12 There is a provision to finetune the features for obtaining the desired outcome. These techniques allow parallel computations and also learn about the variations in the datasets. Thus, DL techniques are effective in handling the huge and variegated datasets. 13 Convolutional Neural Networks (CNNs), a class of DL networks are useful in the automatic extraction of the relevant features from images and feature mapping. 13 Their fast computations, the ability of object detection, continuous learning, and precise classification are the motivations for the authors to employ CNN models in mass screening of COVID-19 from chest X-ray images. The basic architecture of the CNN model is shown in Figure 1.
The CNN architecture comprises of following layers to perform different operations.
i. Convolution Layer: This layer receives the input image as a matrix of digits. It includes another matrix "filter" or "kernel". The filter strides over an input image to extract features of the image without destructing the spatial relationship between the pixels. The convolution operation calculates the dot product of the input image and its filter. 14 The operation gives a feature map as shown in Figure 2C. Different filters generate different feature maps, so the convolution layer acts as a feature detector. Exemplified as the value of the first cell of the feature map is calculated as the dot product of the input image and filter striding over it as given below.
The input image and the filter considered for calculation of the value of the first cell of the feature map are shown with the yellow boundary in Figure 2A, and Figure 2B, respectively. The value obtained is shown within the yellow boundary in Figure 2C. The filter strides repeatedly over the input matrix to obtain all the values of the feature map. The striding of filter over the input matrix with a stride of '1' is shown in Figure 3.
ii. Pooling Layer: A pooling layer is a form of a nonlinear function. It is applied to a convoluted image to create a summary of the important features available in an image. The pooling layer reduces the number of parameters and computations on CNN. It reduces the resolution of a feature map but it does not lose its vital information. 14 In max pooling, the maximum value is picked from the feature map to determine the important features as shown in Figure 4. Selecting a subset of features finds fewer false patterns. Thus, this layer helps in controlling the problem of over-fitting. iii. Fully Connected Layer: This layer follows many convolutional layers and pooling layers. It represents a feature vector for an input image. It contains connections to all activations in a previous layer. This layer helps a network to learn the non-linear combinations of features for classification. 14 The applications of CNN in medical image analysis and classification attracted the researchers 11,[15][16][17][18][19] to provide technological assistants to medical practitioners for dealing with the global pandemic of COVID-19. The researchers in Reference 15 proposed the deep learning model "COVID-Net" for the diagnosis of COVID-19. The model reports the accuracy of 83.25% on X-ray images of patients of COVID-19 and bacterial pneumonia. The researchers in Reference 16 proposed the model "COVIDX-Net" for the classification of non-infected and patients of 'COVID-19'. It reports the highest accuracy of 90% that is an improvement of 6.75% over the model proposed in Reference 15.
The above-stated models face challenges such as overfitting, high computation complexity, and low accuracy for multi-class classification. To minimize these challenges, the authors in Reference 17 proposed a lightweight CNN-tailored shallow architecture for the detection of COVID-19 using CXRs. They experimented with 130 CXR of positive cases of COVID-19 and 51 CXRs of Non-COVID cases. Their model achieved the highest sensitivity and accuracy of 94.43% and 96.92%, respectively. The authors claimed that their model outperformed the DL tools, such as MobileNet and VGG-16. Also, their model is computationally efficient due to less number of parameters. A combination of the CNN model and SVM classifier proposed in Reference 18 reports the accuracy of 95.38% for the detection of COVID-19. The authors in Reference 19 claimed that applying ResNet-50 on X-ray images gives an accuracy of 98%. But they tested their model only on the dataset of 50 images. The researchers in Reference 11 proposed the CNN model "DarkCovidNet" for the diagnosis of COVID-19. They developed the model with 17 convolution layers and trained their model with publicly available datasets. 20,21 The model reported the highest accuracy of 98.08% for the binary classification and 87.02% for the multiclass classification. However, the proposed models reported good accuracy in classification. But, there is a scope for improving the accuracy of multiclass classification.
In this manuscript, the authors propose the convolutional neural network model "COVID-Screen-Net" for the screening of 'COVID-19' using the chest X-ray images. The design of the model is customized in such a way that it can accurately classify the X-ray images of COVID-19 even though trained with a small dataset. The model employs the GradCam for visualizing the most prominent features involved in the prediction of the infected regions of the lungs. This may prove useful to clinicians to validate the results of the model. The major contributions of this manuscript are as follows.
• To collect the anonymous dataset of chest X-rays from hospitals and different sources available online. • The cleaning and pre-processing of the dataset.
• The validation of the labeled dataset by the radiology experts.
• To utilize the potential of CNN in object detection and pattern matching for detection of COVID-19 in chest X-rays. • To develop a CNN model for mass screening of noninfected persons (normal), patients infected with COVID-19 and bacterial pneumonia into three classes viz. normal, COVID-19, and bacterial pneumonia. • To optimize the hyperparameters of the CNN model according to the training dataset. • To provide a tool for the mass screening of COVID- 19 with higher sensitivity and specificity than the tools and techniques proposed in the literature. • To present the visualization of the most prominent features involved in distinguishing the X-ray images into three classes viz. normal, COVID-19, and bacterial pneumonia.

| MATERIALS AND METHODS
In this section, the authors demonstrate the data set used, detailed architecture, and working of the proposed CNN model "COVID-Screen-Net".

| Dataset
The authors used the dataset of 3000 X-ray images, publicly available at. 20,21 The dataset contains 975 images of non-infected persons, 950 images of patients of 'COVID-19', and 950 images of patients of "Bacterial Pneumonia". Figure 5A-C shows the sample X-ray images of 'COVID-19', normal and bacterial pneumonia respectively. The authors divided the dataset into training, testing, and validation datasets. The testing set comprises 80% of the total X-ray images whereas the testing set contains 20% of the total dataset. The validation set comprises 10% of images randomly chosen from the training dataset. The authors also divided the dataset into 47 batches. Each batch contains 64 images. The model receives 38 batches in each epoch during the training and 19 batches during the testing. The authors conducted a set of experiments to select the appropriate batch size. This chosen batch size allows the model to gradually learn the features of the dataset. It minimizes the chances of making the model familiar with the whole dataset at the beginning of the training. Thus, it reduces the problem of generalization. 22 2.2 | The architecture of "COVID-Screen-Net" The model 'COVID-Screen-Net' is an effective application of 2-Dimensional (2-D) Convolutional Neural Networks (CNNs). It comprises five convolution layers. Each convolution layer is alternatively followed by a global max-pooling layer. The first convolution layer uses 32 filters and a kernel size of 3 × 3. It employs the ReLU activation function. 13 The next four convolution layers consist of 64 filters and a kernel size of 2 × 2. The feature map obtained on applying the five convolutions and max-pooling operations is given as an input to the flattened layer. The flattened layer is further connected to the fully connected dense layer which employs the "softmax" activation function. The overall architecture of the model is shown in Figure 6.

| Working of "COVID-Screen-Net"
The model 'COVID-Screen-Net' performs a sequence of operations for the screening of COVID-19, bacterial pneumonia, and normal as presented in Algorithm 1.
i. Pre-processing: The model receives X-ray images of dimensions 256 × 256 × 32 as an input. It applies the resize and augmentation techniques for the preprocessing of the input images. The authors applied the techniques for changing the contrast and brightness of X-ray images. Therefore, two images corresponding to an original image are obtained. These augmentation techniques are useful for increasing the size of the dataset.
ii. Feature extraction: The model uses pre-processed Xray images for feature extraction. It extracts the features and creates the feature maps using multiple convolutions and pooling operations. On a sample image of size 6 × 6, Figures 2 and 3, demonstrate the convolution and pooling operations respectively. iii. Visualization: The authors plotted the feature map obtained at the last activation layer. They used the GradCam to plot the feature map. The plotting at each convolution and activation layer gave an idea of how the model extracts all possible features in the initial layers and ignores the less prominent features in later layers. iv. Classification: The pre-processed images are given as input to the first convolution layer which contains a set of 32 filters, each of kernel size 3 × 3. This layer employs the ReLU activation function (f(x)). 13 It is a non-saturating activation function as given in Equation (1). It is applied to every pixel. It replaces negative values with zero. Thus, it eliminates negative values from a feature map. This function brings the non-linearity in the output of neurons of the model. Hence, it improves the ability to learn the complicated and complex forms of non-linear data. As claimed in Reference 13, ReLU is preferred over other activation functions such as sigmoid and tanh as it does not activate all neurons at the same time.
This solves the problem of vanishing gradients for The output of the first convolution layer is given to the first global max-pooling layer. This layer extracts the features with the maximum value, from each feature map. It also makes the model learn even when a small change in data is performed. Therefore, it is useful in improving the adaptability of the model. Now, four convolutions and pooling operations are applied alternatively using the convolution layers and global max-pooling layers. These operations reduce the dimensions of an input image and give an output of dimensions 8 × 8 × 64. The obtained output is given to the flattening layer which converts the multi-dimensional feature map to a 1-Dimensional (1-D) array of size 4096. The 1-D array is important to create one to one connection of the extracted features with the fully connected dense layer. Now, the dropout function is applied to reduce the number of neurons to 256. The change in the shape of output and trainable parameters of the "COVID-Screen-Net" are shown in Table 1. These 256 neurons establish the connections with the next fully connected dense layer. The fully connected dense layer employs the "softmax" activation function 13 for predicting the probability distribution of outputs into multiple classes namely "COVID-19", normal, and 'bacterial Pneumonia. Figure 7 demonstrates the black box working of the 'COVID-Screen-Net'.

| EXPERIMENTS
The authors used the memory and processing resources available at Google Colab 23 for conducting the experiments. Google Colab is an online training platform freely available for storage and processing. It provides the facility of processing the data on Tesla K80 GPU and RAM capacity of 12 GB. It is efficient in the continuous execution of a model for 12 hr without any halt. The authors used the dataset in References 20,21 for training the "COVID-Screen-Net". They fine-tuned the training parameters given in Section 3.1 to optimize the performance of the model.

| Training parameters
The proposed model employs the RMSProp optimizer 13 at the dense layer. The model uses the following training parameters.
i. rho: This is the exponentially weighted average over the square of the gradients. It is the decay factor for the learning rate of the model. In this manuscript, the authors pre-set the optimum value of 'rho' as 0.9. The value is determined by performing the set of experiments. Figure 8 demonstrates the impact of change in the value of 'rho', on the accuracy of the model. The increase in the value of 'rho' above 0.9 leads to a sharp decrease in the accuracy. On the contrary, the decrease in value of 'rho' below 0.9 shows a gradual increase or decrease in the accuracy. The variation in the accuracy, Area Under Curve(AUC), precision, recall, and F1 measure with the change in the value of 'rho' is shown in Table 2. ii. Decay rate: This is the learning rate of the model.
The model reaches close to the local minimum at the end of the training. In this research, the authors set the decay rate as '0'. The value is determined after conducting a set of experiments. Figure 9 Algorithm 1
1. Apply pre-processing: resize and augmentation on input X-ray images. 2. Pass the image into the Convolution layer to create the feature map.

Input the feature map to the Global Max
Pooling layer for extracting the feature with maximum value. 4. Repeat steps 2 and 3 for n = 1 to 5. 5. Apply flattening on the output obtained from the last global max-pooling layer to get the 1-D array of features. 6. Pass the 1-D array of features obtained in step 4 to the Dropout layer for reducing the number of neurons.
7. Establish the connections of neurons selected by the dropout layer to the fully connected dense layer. 8. Apply the "softmax" activation function. 9. Draw activation maps to highlight the features involved in actual decision making. 10. Obtain outputs into three classes: COVID-19, normal and bacterial pneumonia as outputs.  demonstrates the effect of the decay rate on accuracy. At the pre-set value '0', the model attains the optimum value of the decay rate as 0.0001. An increase in the decay rate sharply decreases the accuracy of the model. Table 3 shows the variation in the performance metrics namely accuracy, AUC, precision, recall, and F1 measure of the "COVID-Screen-Net" with the change in the value of decay rate. iii. Learning Rate: This hyper-parameter determines the rate of learning of the machine-learning model. It decides the number of steps the model requires to minimize the value of the loss function. Figure 10 demonstrates the effect of the learning rate on the accuracy of the "COVID-Screen-Net". In this research, the authors set the value of the learning rate as 0.0004 based on the set of experiments. The impact of the learning rate on the performance of the model is shown in Table 4. iv. Epsilon: This parameter prevents the erroneous values of gradients such as divided by zero. In this manuscript, the authors used the default value of epsilon as 1e-10.

| Results
The authors assigned class labels 0, 1, and 2 to the normal, bacterial pneumonia, and "COVID-19" respectively. Table 5 shows the confusion matrix of 570 images used in the testing dataset. The authors evaluated the performance of the "COVID-Screen-Net" using the following evaluation metrics. ii. Specificity (Sp): This is the percentage of the correctly classified instances to the normal class as defined in Equation (6). The "COVID-Screen-Net" achieves the specificity of 95.50% as calculated in Equation (7).

Sp = Number of correct predictions in normal class Total number of samples in normal class ð6Þ
iii. Prevalence (P): This is the degree of contribution of a class in the complete dataset. Its definition is given in Equation (8). The dataset used for testing the models "COVID-Screen-Net", "DarkCovid Net",  iv. Average Accuracy (Avg_Acc): This is the sum of products of prevalence with sensitivity and specificity as defined in Equation (12). The proposed model 'COVID-Screen-Net' reports the average accuracy of 99.71% on the test dataset as calculated in Equation (13). The accuracy of the 'COVID-Screen-Net' randomly varies when it is executed from the 0th to the 120th epoch. Figure 11 demonstrates the trends of its accuracy until the model achieves its highest accuracy. It is clear from Figure 11 that the accuracy sharply increases in executing the model from 0th to 10th epochs. It achieves the maximum accuracy of 97.71% at the 50th epoch. On further training, the model gives a random increase and decrease inaccuracy. The trends of increase and decrease in the accuracy give the information that the model gradually learns about the dataset and the training parameters.
The accuracy of the model is also dependent on the batch size. It achieves an accuracy of 94% when a batch size of 16 images is given as an input. On the contrary, the accuracy decreases and becomes 90.5% when the batch size is increased to 128 images. The maximum accuracy of 97.71% is achieved for the optimum batch size of 32 images. v. Categorical cross-entropy loss: It measures the degree of dissimilarity between the actual label distribution (y) and the predicted label distributionŷ ð Þ: The definition of the categorical cross-entropy loss (L cross − entropyŷ , y ð ÞÞ is given in Equation (14). Here, i denotes the epoch number.
On performing the set of experiments, the authors recorded the values of loss as shown in Figure 12. There is a random increase and decrease in the values of the loss function when the number of epochs increases from 1 to 50. It reveals that the model is continuously learning the training parameters using the values of loss obtained in previous epochs. Repetitive learning is important for training the model and improving its efficacy. The model achieves the highest accuracy at the lowest value of the loss. It is clear from Figure 12, that the "COVID-Screen-  Table 5, shows the number of test images predicted correctly in their labeled classes. The high values of AUC under ROC 0.97 for the class label '0' (normal), 0.99 for the class label "1" (bacterial pneumonia), and 0.98 (COVID-19) for the class label "2" prove that the model is effective in making the correct predictions for each class. Figure 13 demonstrates the average ROC for all three classes. vii. Precision: It is the ratio of the number of correct predictions in a particular class to that of the total number of correct predictions made in all the classes. The model "COVID-Screen-Net" achieves the maximum precision of 100% on the test dataset. The values of the precision of each class are shown in Figure 14.
The high values of precision prove that the model is effective in extracting the relevant instances of each class label from the total number of extracted instances. viii. Recall: It is the ratio of the number of correct predictions to a particular class to that of the total number of predictions made in that class. The model 'COVID-Screen-Net' gives the maximum recall of 100%. The values of recall for the class label '1' and '2' prove that the model extracts all the relevant instances from the given instances. Figure 14 shows the values of recall for each class. ix. F1 Score: This is the harmonic mean of precision and recall. The proposed model gives an average F1 score of 99.6%. The high values of the F1 score prove, its efficacy in classifying the test images correctly into their actual classes viz. normal, 'bacterial pneumonia', and 'COVID-19'.
The performance of the model is recorded for a different number of epochs while training and testing for each class. It is observed that the model achieved the best performance at the 50th epoch. Figure 15 demonstrates the variation in the performance of the model with a change in the number of epochs.

| Comparative analysis
The authors executed the "DarkCovidNet", "VGG-19", and "ResNet-50" on the dataset in References 20,21. They used the same dataset for training and testing of the proposed model "COVID-Screen-Net". The Confusion matrices of "DarkCovidNet", "VGG-19", and "ResNet-50" are shown in Tables 6, 7, and 8 respectively. The "DarkCovidNet" reports a sensitivity of 96.87% for COVID-19 and 81.54% for the bacterial pneumonia class. The model reports the specificity of 64.94% for the normal class. This model achieves the maximum average accuracy of 80.57% as calculated in Equation (15). The model "VGG-19" achieves the maximum sensitivity of 97.80% for the COVID-19, 96.50% for the bacterial pneumonia class. The model achieves the specificity of 67.95% in the normal class. This model reports the highest average accuracy of 86.88%, as calculated in Equation (16).
The model "ResNet-50" gave the maximum sensitivity of 100% for the COVID-19 and 98.5% for the bacterial pneumonia Class. The model achieves a specificity of 68.85%. This model reports the maximum average accuracy of 88.55%, as calculated in Equation (17 It is clear from the above discussion that the DarkCovidNet, VGG-19, and ResNet-50 are effective in classifying the samples of COVID-19 and bacterial pneumonia class. But, these prove ineffective in the correct classification of the samples of the normal class. The high number of miss classification of the samples of 'normal' class to "COVID-19" or 'bacterial pneumonia' classes degrade the reliability of these models. On the contrary, the proposed model "COVID-Screen-Net" correctly classifies the samples from each class. Thus, it achieves the highest average accuracy of 97.71% as calculated in Equation (12). Figure 16 demonstrates the comparison in the average accuracy of 'COVID-Screen-Net' with 'DarkCovidNet', 'VGG-19', and 'ResNet-50'. The figure clearly shows that the model 'COVID-Screen-Net' outperforms the other three models.
The authors also made a comparison in the AUC score of 'COVID-Screen-Net' with 'DarkCovidNet', 'VGG-19', and 'ResNet-50'. Figure 17 demonstrates the AUC score of these models. It is clear from Figure 17 that the 'COVID-Screen-Net' gives the maximum AUC score for the same dataset.
The execution time is an important factor to adopt the model for medical diagnosis. Therefore, the authors made a comparison in the execution time of the 'COVID-Screen-Net' with 'DarkCovidNet', 'VGG-19', and 'ResNet-50'. The comparison in the execution time is shown in Figure 18. This figure shows that the 'ResNet-50' takes the maximum time for computations. This is due to the more number of convolution and pooling layers in the model. The model 'VGG-19' takes lesser time than 'ResNet-50' but, it requires more time than 'DarkCovidNet' and 'COVID-Screen-Net'. The Model 'COVID-Screen-Net' takes the minimum time for execution. This is due to the use of an optimum number of convolution and pooling layers in the network. The authors optimized the number of layers to overcome the problem of overfitting and to minimize the computation time. The authors applied the augmentation techniques to increase the size of the dataset. For training the model, they took an equivalent number of images in each class. Therefore, the model undergoes unbiased training. It resolved the problem of class imbalance and improves the average accuracy of the model. test. He made this data publicly available at the Github repository 20 for developing and evaluating the machine learning or deep learning models for the detection of 'COVID-19'. Similarly, Paul Mooney collected and compiled the X-ray images from different sources for the confirmed cases the bacterial and viral pneumonia. He made these images available at the Kaggle repository. 21 But, the size of the dataset is small and is insufficient for the training of the deep learning models.
To provide a solution for the requirement of a huge training dataset, the authors in 24 discussed the importance of active learning in training for AI-driven tools. In active learning, the model continuously receives the input data, detects the anomaly in the dataset if any, and consistently learns from it. The authors also focused that the potential of AI-driven tools that can be utilized to diagnose the newly identified disease such as COVID-19 by using the multitudinal and multimodel datasets for training. This is a time-saving technique where the tools are trained using a dataset collected from one geographical region and the trained tools may prove useful in the diagnosis of a disease in a different region. For example, the COVID-19 reported in China for the first time, and later on its cases are identified in other countries of the world. So, the dataset collected from China at an early stage can be used to train the model. This trained model with the feature of continuous learning may be useful in the screening of COVID-19 across the world.
The researchers in Reference 11 proposed the deep learning model "DarkCovidNet" comprising 17convolution layers for the detection of COVID-19. They evaluated their model on the dataset comprising 127 Xray images of COVID-19 patients, 20 500 X-ray images of no findings, and 500 X-ray images of pneumonia. 21 They claimed an accuracy of 98.08% for the binary classification. But, the model reported an accuracy of 87.02% for the multi-class classification. On executing the model on the dataset used for training and testing the "COVID-Screen-Net", the model achieved the maximum average accuracy of 80.57%. This model employs a deep network of 17 layers for the small size of the dataset. Thus, it may lead to the problem of overfitting. Moreover, there is a huge difference in the size of datasets of COVID-19 and the other two classes viz. No-findings and Pneumonia. This may cause the problem of class imbalance.
The researchers in Reference 15 proposed the deep learning model "COVID-Net" for the detection of COVID-19. The model consists of 4-convolution layers. They trained and evaluated their model on X-ray images collected from References. 20,21 They used 68 X-ray images of confirmed patients of 'COVID-19', 1203 cases of healthy persons, 931 cases of bacterial pneumonia, and 660 cases of non-COVID-19 viral pneumonia. They reported an accuracy of 83.25% for the multi-class classification. Similar to the model proposed in, 11 this model may undergo biased training due to the huge variation in the dataset size of different classes.
The researchers in Reference 16 proposed the deep learning framework "COVID-NetX" based on the seven existing architectures. They compared the efficacy of seven deep learning models viz. VGG-19, DenseNet201, Inception, ResNetV2, Xception, ResNetV2, and MobileNetV2 for detection of COVID-19 using 50 X-ray images of chest. They claimed that the VGG-19 and Den-seNet201 reported the highest accuracy of 90% on training and testing with 25 X-ray images of COVID-19 confirmed cases and 25 X-ray images of non-infected persons in Reference 18. They recommended the use of VGG-19 and DenseNet201for detection of COVID-19 using X-ray images. The model 'VGG-19' reported the average accuracy of 86.88% on the same dataset 20,21 as used for testing the "COVID-Screen-Net".
The researchers in Reference 18 proposed the combination of the CNN model and SVM classifier. They reported an accuracy of 95.38% on training and testing their model using 25 X-ray images of non-infected persons and 25 X-ray images of "COVID-19" patients. 15 The authors in Reference 19 claimed that applying ResNet-50 on X-ray images reports an accuracy of 98% on binary classification into class labels namely "COVID-19" and non-infected. The evaluation of ResNet-50 on the same dataset 20,21 as used for testing the 'COVID-Screen-Net' reported the average accuracy of 88.55%.
The authors in Reference 25 proposed the Truncated Inception Net model for classification of COVID-19 positive from other non-COVID cases and/or healthy CXRs. The model achieved an accuracy of 99.96%. The authors compared the performance of this model with COVID-Net, 15 a combination of ResNet-50 and SVM, 16 ResNet-50, 18 and Inception Net V3. 19 Based on the comparison, they claimed that the Truncated Inception Net model outperformed these models.
The above discussion shows that the highest accuracy of multi-class classification into normal, COVID-19, and bacterial bneumonia is reported by the model proposed in Reference 19 is 88.55%. Employing deep networks for the small dataset and biased training are also limitations observed in this model. The deep learning model "COVID-Screen-Net" proposed in this manuscript performs the multi-class classification and effective in the automatic screening of 'COVID-19'. The model reports an average accuracy of 97.71% on the testing dataset of 570 X-ray images. To deal with the problem of biased training, the authors used an equal number of images for each class for training and testing the model. Moreover, the authors restricted the number of convolution layers to avoid the problem of overfitting on the small dataset. The model also presents the visualization of the features used by the model for automatic prediction of "COVID-19" and "Bacterial Pneumonia". The visualization in the GradCam may prove useful for clinicians in quick decision making. This model may prove an automatic and intelligent assistant for health experts. It can minimize the time and cost required for the screening of 'COVID-19'.

| VALIDATION
The model is sent for validation to the radiology experts of RG Stone Urology & Laparoscopy Hospital. Dr. RK Sharma, Chief Radiologist, and Head Radio Diagnosis Division evaluated the efficacy of the model on the different types of chest radiographs of the patients and shared their feedback. The authors implemented the suggestions and improved the efficacy and reliability of the model by training the model with different types of chest radiographs such as low contrast and high contrast for each class namely 'COVID-19', "bacterial pneumonia", and normal. They validated the sensitivity and reliability of the model by testing the model on a real-time dataset. The approval and recommendation are given as supplementary material.

| CONCLUSION
The improvement in the efficacy of the health services has become the prime objective worldwide due to the "Global Pandemic" caused by the 'COVID-19'. The cost-effective and timely diagnosis of the disease is important to reduce its outbreak. The deep learning model 'COVID-Screen-Net' can provide a technological solution for the screening of 'COVID-19' using the Xray images of the chest. The model distinguishes the Xray images of normal, 'bacterial pneumonia', and 'COVID-19'. It reports the average accuracy of 97.71% on the test dataset of 570 X-ray images publicly in References 20,21 X-ray imaging requires low cost and widely available. It also minimizes direct contact with the patient. Moreover, it avoids the risk of sample damage as observed in tests based on blood samples and swabs taken from the throat. The model can be employed at remote locations where the medical experts and lab facilities are not available. The radiographers can capture the X-rays in contactless mode. They can use the model to classify normal and abnormal cases. The abnormal cases can be recommended for further medical tests and counseling on a priority basis. Therefore, this model may prove a useful assistant for medical practitioners in the quick screening of 'COVID-19'. This model can also be customized for the screening of 'COVID-19' using CT scan images. Its applicability can be enhanced to classify the more number of diseases using chest radiographs.
A model is an effective tool for the classification of chest radiographs. But, its performance may degrade for the poor-quality radiographs. The performance of the model may below in case the chest radiographs do not show any infection.

ACKNOWLEDGMENTS
We would like to thank Dr. Avinash, Dr. Deepak Buddhi, and Dr. Ravi Kumar, experts in the department of radiology at RG hospital Delhi for their valuable inputs. Their remarks and suggestions helped us to improve the robustness, average accuracy, and reliability of the model.

DATA AVAILABILITY
The data that support the findings of this study are openly available in ["Kaggle" and "Github"] at https:// github.com/ieee8023/covid-chestxray-dataset, https:// github.com/ieee8023/covid-chestxray-dataset [21][22]. The anonymous data collected from hospitals will be made available at "Github" repository at the publication stage of the article. The collected data will be made available with the consent from the health experts.


System Identification with Student’s t -Process Dynamical Model

Knowing the exact location of a spacecraft is one of the most important tasks in the operation of a satellite. However, it is diﬀicult to accurately determine the position and velocity of a satellite far from the ground. In order to meet such requirements, various estimation methods have been developed. In this study, we propose a t-process dynamic estimation model, which is based on the t-distribution, and it is suggested that it is more robust than the Gaussian model. In the present study, we have successfully developed a particle filter-based t-process dynamic estimation model.

Introduction
Given a set of inputs and outputs we need to infer and identify the behaviour of system considered for the problem of information and controls. In the literature of the system identifications, various methodologies have been developed and proposed. One of the most famous methodology is the family of state space modelings, in particular, linear state space models have been applied to many problems of the system identifications because of its mathematical simplicity and tractability. In addition, the rigorous treatment of filtering scheme for the linear state space model has been developed by Kalman. Then the Kalman filter has been used in the area of motion control, robotics and data assimilation and so on.
Despite the usefulness of the linear state space modeling, nonlinear behaviour is often observed in real systems. To identify the behaviour of nonlinear systems, kernel method has been developed. In this framework, the domain of inputs is mapped from a finite dimensional vector space to a infinite dimensional Hilbert space, and then linear modeling is implemented on the mapped domain. Thus, the kernel method is considered to be a linear modeling on the infinite dimensional Hilbert space.
Adding to the Gaussian noise, the kernel method is extended to stochastic systems as the Gaussian process regression (GPR), which is classified into category of non-parametric Bayesian modeling. As with the linear state state model, the GPR incorporates state variables and then its filtering scheme is designed.
Because of the definition of the the GPR, it is restricted to be utilized to observed data under Gaussian noise. However, non-Gaussian fluctuations are also observed in the real world, such as fluid turbulence, financial time series, and cosmic lays. To identify the system under non-Gaussian noise, the Student's t-process regression (TPR) has been proposed in the literature of the non-parametric Bayesian modeling. As with the GPR, the TPR also consists of the kernel functions to map the finite dimensional input data onto infinite dimensional Hilbert space. In addition, a latent variable model of the TPR has been proposed in the area of mathematical finance recently, and state space modeling on the form of an ordinary differential equation has been considered.
In this study, we propose a Student's t-process dynamical model (TPDM) as a state space model of the TPR. Further, a simultaneous online estimation for state variables and parameters of the TPDM with the aid of particle filtering. Section 2 provides a brief introduction of the TPR, where the basic concept of the GPR is also explained. In Sec. 3, we propose the TPDM as a natural extension of the TPR with latent variables. Followed by this section, we test the TPDM for a problem of state estimation for satellite orbits in Sec. 4. Finally, section 5 is devoted to conclusions of this study and future perspectives.

Gaussian process regression
In the problem of system identifications, the relations between input x∈X ⊂R D and output y∈R are assumed to be described by where ϕ : X →R is an appropriate smooth function and ω is a Gaussian white noise with zero mean and variance Proceedings of the 52nd ISCIE International Symposium on Stochastic Systems Theory and Its Applications Osaka, Oct. 29-30, 2020 σ 2 0 . Suppose ϕ(·) be a random function with mean and covariance where m : X →R is a mean function and k : X ×X →R is a kernel function which is symmetric positive definite. The random function ϕ(·) is a Gaussian process if for a set of input {x 1 , where ϕ X , m X and K X,X are respective, function vector, mean vector and covariance matrix. They are defined by and Given pairs of inputs and outputs D = is implemented by the Gaussian process used as prior distribution: where y = [y 1 , y 2 , · · ·, y N ] T is an output vector and I is the N -dimensional identity matrix. The law of summation for Gaussian distributed random variables yields Eq. (8).
The prediction task of the GPR is implemented by its conditional probability density function. For an additional input data x * one can obtain predicted output y * by the identified system model. In the Bayesian sense, the predicted output y * is sampled from the conditional probability density function y * ∼p(y * |x * , X, y). (9) The conditional probability density function p(y * |x * , X, y) is also the Gaussian distribution as where the mean and variance are derived as It is confirmed that Eqs. (11) and (12) map the additional input to the predicted value.

Student's t-process regression
In what follows we assume the mean function to be zero for the sake of brevity. Suppose the regression model in Eq. (1) is parameterized by λ≡σ −2 0 and K X,X →λ −1 K X,X . The λ-parameterized GPR is given as y∼N (λ(K X,X + σ 2 0 I)|λ).
Note here that the notation of the mean functions is omitted in Eq. (13). The parameter λ is assumed to be a random variable being subjected to a one-parameter family of the gamma distribution defined by with ν being a positive real parameter called as degree of freedom. Marginalizing the conditional Gaussian distribution in Eq. (13) with respect to λ by the gamma distribution in Eq. (14), we obtain where T (I + K X,X ; ν) is the Student's t-distribution defined by In this way, the GRP leads to the Student's tdistribution with the kernel functions in Eq. (16), which is known as the TPR. Through the same procedure of the GPR, the prediction task of the TPR is implemented by its conditional probability density function. For an additional input x * , the conditional probability density function for the predicted output y * is given as where m * , k * , and ν * are derived as It is notably confirmed that k * and ν * include correction terms with respect to the size of given data set.

Student's t-process latent variable model
For the situation that only output data are observed, the Student's t-process latent variable model (TPLVM) has been proposed as a method of unsupervised learning in the sense of non-parametric Bayesian models. Without input data the matrix X in Eq. (16) is regarded as a parameter which should be estimated. To estimate the hyper parameters of the TPLVM including X, variational inference has been installed by the Kullback-Leiler (KL) divergence as where p(X) and q(X) are a prior and approximated distributions for X. Minimization for the KL divergence yields a plausible inference for q(X).

Student's t-process dynamical model
Incorporating dynamical latent variables into the TPLVM, we develop a Student's t-process dynamical model (TPDM). Suppose (x t , y t ) be a pair of state and observed variables at time t, the TPDM is defined by the following state space model: The dimensions of the state and observed variables are allowed to be different. As with other state space models, filtering, smoothing, and predicting schemes for the TPDM are able to be designed. In general the kernel functions contain its intrinsic parameters. For instance, the radial basis function, one of the most used kernel functions, is of the form: where α and l are positive real parameters. Within the degree of freedoms ν x and ν y , the parameters of the kernel functions is composed of parameter vector θ.
The specific value of θ can be estimated by the method of maximum likelihood within the set of observed data {y 1 , y 2 , · · ·, y T }, or other estimation methods.
It is pointed out that point estimations such as the method of maximum likelihood is likely to fall in overestimation. To overcome this problem, we employ particle filtering as an online Bayesian estimation. On the processes of the particle filtering we treat the parameter vector θ as a dynamical variable whose evolution is described by where a is a positive real parameter in the open interval (0, 1),θ t andV θt are ensemble mean and covariance of M particles, respectively. By this method, the state variables and parameters are estimated simultaneously.

Application for satellite orbit estimation
Here, we adopt the TPDM to the problem of satellite orbit estimation. In this problem state variables represent two-dimensional position and velocity elements [3]: In addition, the motion of a satellite is assumed to be a simple circular motion of a qualitative point and to be given by where G is the gravitational constant, m is the mass of the satellite, M is the mass of the earth, and r is the position vector of the satellite when the center of the earth is at the origin. In this setup, the true position of the satellite is not able to be observed, and the information available is the observed quantity Y , which is composed of the state vector and a white noise w: Representative simulated results are shown in Fig. 1. The green line shows the true position of the satellite, the gray line is the observed position information with the white noise, and the red line is the position of the satellite estimated from the t-process dynamic model from the observed information. The value on the vertical axis represents the x-coordinate of the twodimensional coordinates. There are some deviations, but generally the true values are captured.

Conclusions
In this study, we have developed the TPDM as an extension of the TPLVM for state space modeling. As an example of the TPDM, we estimated the true satellite orbit from the observed data consist of the position of the satellite and the observed noise. We will continue to work on increasing the dimension of the state variables and adapting the TPDM to more complicated models, and compare the model with other filtering methods to find out its advantageous points.


A dai-liao hybrid conjugate gradient method for unconstrained optimization

One of todays’ best-performing CG methods is Dai-Liao (DL) method which depends on non-negative parameter  and conjugacy conditions for its computation. Although numerous optimal selections for the parameter were suggested, the best choice of  remains a subject of consideration. The pure conjugacy condition adopts an exact line search for numerical experiments and convergence analysis. Though, a practical mathematical experiment implies using an inexact line search to find the step size. To avoid such drawbacks, Dai and Liao substituted the earlier conjugacy condition with an extended conjugacy condition. Therefore, this paper suggests a new hybrid CG that combines the strength of Liu and Storey and Conjugate Descent CG methods by retaining a choice of Dai-Liao parameterthat is optimal. The theoretical analysis indicated that the search direction of the new CG scheme is descent and satisfies sufficient descent condition when the iterates jam under strong Wolfe line search. The algorithm is shown to converge globally using standard assumptions. The numerical experimentation of the scheme demonstrated that the proposed method is robust and promising than some known methods applying the performance profile Dolan and Mor´e on 250 unrestricted problems.  Numerical assessment of the tested CG algorithms with sparse signal reconstruction and image restoration in compressive sensing problems, file restoration, image video coding and other applications. The result shows that these CG schemes are comparable and can be applied in different fields such as temperature, fire, seismic sensors, and humidity detectors in forests, using wireless sensor network techniques.

One of todays' best performing CG methods is Dai-Liao (DL) method which depends on non-negative parameter and conjugacy conditions for its computation. Although numerous optimal selections for the parameter were suggested, the best choice of remains a subject of consideration. The pure conjugacy condition adopts an exact line search for numerical experiments and convergence analysis. Though, a practical mathematical experiment implies using an inexact line search to find the step size. To avoid such drawbacks, Dai and Liao substituted the earlier conjugacy condition with an extended conjugacy condition. Therefore, this paper suggests a new hybrid CG that combines the strength of Liu and Storey and Conjugate Descent CG methods by retaining a choice of Dai-Liao parameter that is optimal. The theoretical analysis indicated that the search direction of the new CG scheme is descent and satisfies sufficient descent condition when the iterates jam under strong Wolfe line search. The algorithm shown to converge globally using standard assumptions, where the numerical experimentation of the scheme demonstrated that the proposed method is robust and promising than some known methods applying the performance profile presented by Dolan and Mor´e on 250 unrestricted problems. Numerical assessment of the tested CG algorithms with sparse signal reconstruction and image restoration in compressive sensing problems, file restoration, image video coding and other applications show that these CG schemes are comparable and can be apply in different fields such as temperature, fire, seismic sensors and humidity detectors in forest and so on using the wireless sensor network techniques.

Introduction
Conjugate Gradient (CG) method was initially suggested for solving linear system of equation. Subsequently, the solution of a linear system is comparable to minimizing a positive definite quadratic function, for this reason (Babaie-Kafaki & Ghanbari, 2014a), the method was later modified to solve unconstrained minimization problems (Rao, 2009). Therefore, the method constitutes an excellent choice for solving optimization problems by scientists, engineers and mathematicians (Babaie-Kafaki, 2011). The method is categorized by absent of matrix storage with powerful theoretical properties (Djordjevic, 2017). The problem as the form: → is a function that is twice continuously differentiable, and the CG scheme that iteratively solves the problem is given where > 0 is a step-size obtained by a suitable line search and here is a search direction (Gilbert & Nocedal, 1992). Generally, the distance to move along the search direction can be attained by solving onedimensional minimization called an exact line search such that the objective function is minimized to find , that is, However, for large scale problems usually, an exact line search is not possible so any value of that satisfies certain properties called Wolfe conditions is accepted (Nocedal & Wright, 2006): where 0 < < < 1 , and that is a path towards minimum needs to be descent (Babaie-Kafaki & Ghanbari, 2014c). Whereas other value of constitute up of (4) and known strong Wolfe condition is also accepted (Nocedal & Wright, 2006). Therefore, the direction towards minimum can be obtained by the formula where is a scalar CG (update) parameter which is determined by some inner products (Ding, et al., 2010). Mainly, the CG schemes vary by the selection of coefficient. Some well-known CG schemes can be divided into two (Babaie-Kafaki, et al,2010). The schemes in the first category may perform poorly theoretically but numerically behave well due to an important feature known as restart that helps them circumvent jamming automatically (Babaie-Kafaki & Ghanbari, 2014c). These CG parameters were initially suggested by Hestenes and Stiefel (1952), Polak, Ribie're and Polyak (1967), Liu and Storey (1991) with the following coefficients, respectively: where ‖. ‖ symbolizes Euclidean norm and define = +1 − and = +1 − (Andrei, 2008a).
The other category is prone to poor numerical performance as a result of jamming, but they have powerful theoretical properties (Andrei, 2008b). These schemes were earlier proposed by Fletcher and Revees (1964), Dai and Yuan (1991) and Fletcher (1987) with the following CG parameters respectively: The schemes in (9) vary with other selections in theory because their theoretical properties require Lipchitz assumption only but not including boundedness assumption (Hager, & Zhang, 2006). The FR method's poor practical performance is associated to taking tiny steps without meaningful progress to reach the minimum (Powell, 1984). Specifically, if a bad path is taken, then tiny steps from −1 and will be generated, the next path along step are likely to be poor except a restart along the gradient direction is made (Babaie-Kafaki, 2013). Babaie-Kafaki, et al. (2011) Pointed out that despite such deficiency, the FR method was proved to be theoretically powerful with exact line search on general functions; later this result was extended to an in-exact line search to improve the efficiency of the scheme (Hager & Zhang, 2006). In general, the performance of the methods in the first category is efficient but their convergence is uncertain (Hager & Zhang, 2006). The behavior of these schemes needs to be improved to avoid jamming (Djordjevic, 2017). Therefore, Researchers were interested in combining CG schemes of the two schemes (Babaie-Kafaki & Mahdavi-Amiri, 2013). Although, CD scheme is closely related to FR scheme with exact line search but, the restriction < 1 2 in FR is not required for CD to attain sufficient descent using strong Wolfe line condition. Moreover, CD scheme is theoretically powerful for the generalized Wolfe conditions with < 1 and = 0 restrictions (Hager & Zhang, 2006). Meanwhile, Djordjevic (2017) pointed out that no much research has been done on the choice of except for the work of Liu & Storey (1991) initially, but the analysis of PRP techniques should be applied to the LS method (Hager & Zhang, 2006). Since LS and PRP schemes are identical when an exact line search is used (Dai, 2001).
Babaie-Kafaki & Ghanbari (2014c) suggested two Hybrid Conjugate Gradient (HCG) methods where the CG coefficients were obtained from standard and modified secant equations respectively. The parameters are calculated as an affine combination of and . While Djordjevic (2017) suggested a hybrid parameter using Liu & Storey (1991) with Conjugate Descent CG parameters Convex Combination (LSCDCC) from conjugacy condition as an affine combination of and . To achieve global convergence for general function, HCG adopted ≥ 0 restriction, while the other coefficient is hypothetically superior for uniformly convex function. Subsequently, the selection of CG parameter received little attention by researchers except for the work of (Djordjevic, 2017;Salihu, et al, 2020) recently motivated this work. Given the above, a large number of hybrid conjugate gradients techniques were proposed (Andrei, 2008a) that modified different coefficients to maximize their strengths and minimize their weaknesses (Babaie-Kafaki, et al. 2010). Among them: (e.g. see (Yuan, 1991;Gilbert & Nocedal, 1992;Andrei,2008c;Dai & Yuan, 2001;. The excellent contributions of Andrei and Babaie-Kafaki on hybridization using convex combination and that of Djordjevic motivated us to extend their approaches to access and combine the strength of LS and CD CG update parameters.

Extended Conjugacy Condition of Dai and Liao CG Method (ECCDL)
In the earlier CG methods; conjugacy condition +1 = 0 that rests on the exact line search plays a significant part in the mathematical experiment and convergence analysis (Sun & Yuan, 2006). Though, a practical mathematical experiment implies using an inexact line search to find the step-size . Especially in a situation where +1 is not equal to zero, then it may take other form, to avoid such defect Dai & Liao (2001) substituted the pure conjugacy condition with extended conjugacy condition. Due to simpler structure and low memory requirements of Dai-Liao conjugate gradient methods; Yao, et al. (2019) proposed some three-term Dai-Liao CG algorithms that possess efficient conjugate gradient structures. Esmaeili, et al. (2018) suggested a new CG scheme to solve problems emerged from astronomical imaging, file restoration, image video coding and other applications. Similarly, Guo & Wan (2019) developed CG algorithm for sparse engineering signal problem. Numerical tests indicated that the algorithm is an alternative for recovering sparse signal problems and beats earlier methods. Recently, Liu, et al. (2020) transformed nonlinear unconstrained optimization problems as -tensor equation to solve reallife issues originating from engineering and economics. Numerical results revealed that the proposed CG scheme is efficient than some known methods. One of todays' best performing CG method is Dai & Liao (DL) method which depends on non-negative parameter t for its computation. Although numerous optimal selections for the parameter were suggested as in (Babaie-Kafaki, & Ghanbari 2014b;Babaie-Kafaki,2015;Babaie-Kafaki & Ghanbari, 2017;Waziri, et al, 2019;Salihu, et al, 2021), the best choice of t still remains subject of consideration (Babaie-Kafaki, & Ghanbari, 2015). Motivated by the above, in this section, using similar approach in (Andrei, 2008b;Andrei, 2009;Babaie-Kafaki, et al, 2010;Djordjevic, 2017), this research will combine the attractive features of CG update parameters proposed by Liu and Storey (1991) with CG parameter proposed by Fletcher (1987) conjugate descent using Extended Conjugacy Condition of Dai and Liao (2001) CG method called (ECCDL) as follows: From relations (8) and (9), we can write (10) Therefore, using vector on relations (7) and (10) we obtain Applying +1 = 0 on (13) will lead to the following hybridization parameter in (Djordjevic ,2017): Similarly, if we apply Dai-Liao extended conjugacy condition: on (13) and after some algebra another new hybridization parameter is propose as: The justification of the choice of the method in this work, an algorithm with update parameter that do not require the calculation of the Hessian matrix for solving large scale problems is preferred. For this reason, we assume that does not satisfies +1 = 0. Therefore, the beautiful structures of CG update parameters proposed by Liu and Storey (1991) with CG parameter suggested by Fletcher conjugate descent using Extended Conjugacy Condition of Dai and Liao (ECCDL) CG method is proposed in such a way that, if the modulating parameter = 0, then (16) reduces to the method in (Djordjevic, 2017). So, as for the optimal choice of the method we assume ≠ 0 in the term. Next the algorithm of the proposed method is presented as follows: Algorithm 1 (ECCDL).

Remark:
The update parameter computed by (16) may be outside interval [0, 1]. However, to have a proper convex combination in (10)-(11). The following rules are applied: if Therefore, under this selection for , the direction +1 in (12)-(13) is a proper convex combination of and .

Theoretical Analysis
To demonstrate the sufficient descent condition of the ECCDL method, we apply the following theorem.

Convergence Analysis
The following assumptions are required to establish the global convergence of ECCDL method: Boundedness Assumptions: Assumption 3.1. The set = { ∈ ∶ ( ) ≤ ( 0 )} is bounded from below where 0 is the starting point of CG method in (2) and (7). That is, there exist a positive constant such that

Lipschitz Assumptions: Assumption 3.2. In a neighborhood , the objective function f is continuously differentiable and its gradient ( ) is Lipchitz continuous on , that is, there exist a constant > 0 such that
for all , ∈ .

Lemma 3.3 Let Assumption 3.1 holds and is a descent direction and the step-size satisfies
then Proof: Using (26) and (29), it holds that −(1 − ) ≥ ( +1 − ) ≤ ‖ ‖ 2 . Subsequently, when > 1 and the search direction is < 0, it is not difficult to claim that (30) holds. Clearly, from relations (6) and (18), the step-size in the ECCDL algorithm fulfils (30). Consequently, according to (18) and since = 0 does not fulfil (6), we can easily conclude that ≠ 0 ∀ ≥ 0, which means ≠ 0 and as such there exist > 0 so that Generally, any CG method with strong Wolfe line search converges. However, only weak form of the (Zoutendijk, 1970) condition is required for general function (Dai & Liao, 2001). The theorem below establishes the useful theoretical property of ECCDL through the strong Wolfe line search.
The proof is using contradiction, that theorem (3.2) is not true.
Proof: Let ≠ 0 and assume that (32) does not hold. Then, we have a constant > 0, such that It follows from the first inequality that ∈ (0,1) and subsequently the second inequality holds from the Cauchy Schwartz inequality. But when ∉ (0,1), it is easy to get the above inequality according to selection in Step 5 of algorithm 1. Thus, from (7) and (34) we get and this indicates that Contrarily, from (18), (28) and (33), we can achieve Also, applying Lemma (3.1), we conclude that Obviously, this is a contradiction of (36) and hence (33) is not satisfied which implies that (32) is proved. ∎

Results and Discussion
In engineering, medical sciences, biological and other areas of science; digital image processing plays an important role. Therefore ;Ibrahim, et al. (2020) utilized Hybrid Liu and Storey and Fletcher and Revees (HLSFR) algorithm of Djordjevic (2019) in restoring one dimensional signal sparse problem using mean squared error (MSE) with the LS and FR CG algorithm to suggest a hybrid algorithm for unconstrained minimization problems and extend the result to convex monotone equations. Numerical assessments with some image restoration in compressive sensing CG algorithms show that the proposed scheme is efficient and promising than other schemes with smaller number of iterations, computing time and MSE on different noise sample problems. The HLSFR algorithm is the foundation of Ibrahim et al. (2020) work and is similar to Hybrid Hestenes and Stiefel and Fletcher and Revees (HHSFR) algorithm of Djordjevic (2018), LSCDCC of Djordjevic (2017) and ECCDL algorithms which can be applied to compressive sensing problems that has wide variety of applications as shown in Ibrahim et al. (2020) for example; the wireless sensor networks that are usually placed in field can be used in temperature, fire, seismic sensors and humidity detectors in forest, etc. Therefore, in this section, we present the performance of ECCDL and compare with that of LSCDCC of Djordjevic (2017) and HCG method of Babaie-Kafaki & Ghanbari (2014c). To implement the hybridize CG parameters, the codes were run on a computer with a processor and memory of 2.20 and 3.0 , respectively, using Matlab 8.3 ( 2014 ) on 250 unconstrained optimization problems. The test problems are the unconstrained problems obtained from (Andrei, 2008c;and Gould, et al, 2003). Since CG schemes are used to solve largescale unconstrained optimization, we choice 25 problems that are tested 10 times for: 100; 200; 500; 1,000; 2,000; 5,000; 10,000; 20,000; 50,000 and 100,000 with summary of the numerical results and list of test functions shown in table 1-2 respectively. All the algorithms were implemented using (4) and (6) with = 0.0001 and = 0.001, and the step length is computed with initial trail value = 1 and the modulating parameter = 0.5 .The same stopping criterion ‖ ‖ ∞ ≤ 10 −5 is used. All the test functions were minimizing from standard starting points.
Test function results are obtained by running a solver on set of problems and recording the number of iterations and the computing time. Interpretation of figures 1-2 show the performance of these methods using (Dolan & Mor´e, 2002) profile. The ( ) is the portion of problems with performance ration , thus, a solver with high values ( ) or at the top right of the figures are preferable. That is, for each method, we plot the percentage ( ) of the problems for the best time for each algorithm within a factor of ( ) versus time . The left side gives the percentage of the test problems of the method that is fastest. The right side gives the percentage of the test functions successfully solved by each method. The interpretation of figure 1 shows the probability of ECCDL method is the winner on a given problem is 61%. While LSCDCC and HCG methods win 39% and 15% percentages respectively, when the factor is chosen within the interval 0 < < 0.5. Clearly, ECCDL method has the most wins, because it has the highest probability of being closer to the optimal solution. However, if we extend our of interest to ≥ 0.5, ECCDL and HCG algorithms solved the test functions in a given time and reach 88% and 87% respectively, while LSCDCC method is 85% to. It is easy to see that the performance of ECCDL and HCG algorithms are computationally efficient than LSCDCC scheme.
Since the computing time is also affected by the computer atmosphere like operating system and busy status, we additionally compare the number of iterations of the algorithms. Figure 2 shows that the fraction of ECCDL method is the winner on a given problem is 82%. While LSCDCC and HCG methods win 80% and73% percentages respectively, when the factor is chosen within the interval 0 < < 0.5 Clearly, ECCDL method wins, since it has the highest possibility of being closer to the best solution. However, if we extend our of interest to ≥ 0.5, ECCDL and HCG algorithms solved the test functions in a given number of iterations and reach 88% and 87% respectively, while LSCDCC method is 85%, it is easy to see that the performance of ECCDL and HCG algorithms are computationally efficient than LSCDCC scheme.

Conclusion
One of todays' best performing CG method is DL method which depends on non-negative parameter for its computation. Although numerous optimal selections for the parameter were suggested, the best choice of remains a subject of consideration. In this paper, we have presented a new hybrid Dai-Liao conjugate algorithm in which the parameter is computed from and in such a way that if the modulating parameter = 0 then it reduces to the method that uses the pure conjugacy condition. Theoretical and numerical computations adopt inexact line search when compared with some known CG coefficients using strong Wolfe condition show the algorithm is robust, efficient and converge globally compared to LSCDCC and HCG methods on 250 unconstrained optimization problems. Numerical assessments of these CG algorithms show that the schemes are comparable with smaller iterations and computing time and can be applied to compressive sensing problems with a wide variety of applications, as shown in Ibrahim et al.(2020) method.


A New Rockfall Simulation Tool based on Unity 3D

Objectives: Rockfall occur frequently in the mountainous areas of Southwest China, causing huge casualties and property losses every year. A rapid simulation method of the collapse process is urgently needed. The commonly used collapse simulation software still has obvious shortcomings, such as low terrain accuracy, no consideration of the structural characteristics of rock mass, as well as block collision and fragmentation. The logic operation based on CPU still limits the calculation speed. In this study, a new method for rapid simulation of the three-dimensional motion process of collapse is proposed. Methods: The Unmanned aerial vehicle (UAV) aerial photography modelling combined with field investigation is used to obtain the slope surface model and determine the characteristic parameters of rockfall. The simulation software of a large-scale collapse movement process is developed by Unity 3D platform which integrates PhysX physics engine and CPU-GPU parallel computing capability. Results: The software can simulate the whole process of collapse, impact, fragmentation, and deposition. It can output the three-dimensional trajectory, velocity, energy and jumping height, which can be used for the engineering design. Finally, Xiejiayan rockfall in Nayong, Guizhou Province has been selected as an application case, to verify the feasibility and suitability in practice.

Introduction
Rockfall is a common type of geological disaster in mountainous areas, which has obvious concealment, sudden and catastrophic characteristics, and is easy to cause casualties and property losses [1,2] . With global warming, there were more and more collapse disasters in southwest China. For example, a largescale collapse of the mountain in Longchang Town, Kaili, Guizhou Province, on February 18, 2013, with a volume of 3×10 4 m 3 , caused 5 people dead and relocated 21 households with 79 persons [3] . On August 28, 2017, another mountain collapse occurred in Pusa Village, Zhangjiawan Town, Nayong County, Guizhou Province, with a volume of about 6×10 5 m 3 , which killed 35 people [4] . On June 24, 2017, a high-level landslide occurred in Xinmo Village, Diexi Town, Maoxian County, Sichuan Province, the Xinmo landslide with a volume of about 4.5×10 6 m 3 , caused 64 farmhouses to be buried, 83 people dead or missing, and blocked more than 1,000 meters of river channel [5] . It can be seen that rockfalls have a great role in mountainous constructions and lives.
The research on the mechanical mechanism and movement characteristics of the collapse movement process include field tests, model tests and numerical simulation methods [6][7][8][9][10] . Numerical simulation methods have the advantages of low cost, simplicity and portability. Then, it is commonly used for rockfall evaluation. Compared to the main software (Table 1), it can be seen these methods still have obvious shortcomings and did not consider the fragmentation process of collapsed bodies. E.g., Rockfall software is often used in the calculation of distance, bounce height and kinetic energy. But it needs to specify the terrain profile manually, which cannot truly reflect the rockfall motion trajectory [11] . Rockfall Analyst is a secondary development component of ArcGIS, which can realize the automated and semiautomated analysis of collapse motion trajectory simulation and risk mapping, but ignores rock fragmentation as well [12] . CRSP-3D is a collapse simulation program designated by the U.S. Federal Highway Administration related to engineering projects, but it does not consider the structural characteristics of the rock mass, and cannot set the location of the dangerous rock [13] . It has certain limitations in simulating large-scale collapse movement [14] . In addition, key parameters such as movement distance, jumping height, and impact force are required in the quantitative risk assessment of collapse. Therefore, further study on the rockfall movement simulation is to be important in scientific and social value. The Unity3D engine is a set of integrated development environments including graphics, sound, physics and other functions developed by the Danish company Unity Technologies. It can create interactive 3D scenes. The operating system is highly compatible. It is widely used in system simulation and virtual reality [15] . The engine integrates the PhysX physics engine, provides simultaneous detection of multiple object continuous collisions and supports GPU parallel computing functions [16] . Based on the theory of rigid body dynamics, it can simulate the behaviour of rigid body bouncing, rolling and sliding, solve the motion speed and energy, and obtain the motion trajectory. We think it might be a new method for solving the three-dimensional simulation of the large-scale collapse movement process. Chen Shushu (2018) has already used the Unity3D platform to develop the movement scene of the debris flow [17] . Chiu (2015) used Unity to create a landslide motion scene and uploaded it to the Web and mobile platforms for an application in science education on geological disasters [18] . However, the above applications have not yet introduced the physics engine technology. As a result, the simulation results lack physical support and the effect is poor.
Given this, according to a large number of collapse prototype cases investigation, this paper uses a physics engine to ensure the relationship between collapse dynamics theory and computer simulation. A rockfall simulation tool has been developed based on Unity3D to realize a physical process of fragmentation in the movement process. The Xiejiayan rockfall in Guizhou Province was selected as a case application to verify its reliability and applicability, which can provide a new simulation method and evaluation method for the prevention and mitigation of rockfall disasters.

Theory and Simulation Method of Rockfall Dynamics
The simulation of the rockfall movement process is mainly to deal with the interaction between rockfall and slope, which may involve multiple collisions, rebounds, rolling, sliding and impact fragmentation processes [19] . When falling rocks hit the slope surface, if the average slope angle of the collision point is less than 45 degrees, it is considered that the rock starts to roll, and if it is greater than 45 degrees, we are more likely to see a rebound. In the numerical simulation of the movement process, the rock block and the slope are regarded as rigid bodies. The energy loss in the collision between rigid bodies is usually processed by a contact function including friction and recovery coefficients [20] . The friction and recovery coefficients use the parameters of the physical material assigned to the object for effective control in the Unity3D engine [21] . Physical materials allow users to control the interaction between objects through the dynamic friction coefficient, static friction coefficient, and bouncing coefficient used by the physics engine PhysX. Depending on the state of motion before the collision and the geometry of the slope and falling rock objects, each collision may contain multiple contact points ( Figure 1). The linear velocity and angular velocity of the block that bounces off the slope surface after the collision depends on the product of the iterative solution of the velocities of all contact points in each collision under the constraints of contact and friction.  Fig.1 Conceptual diagram of the multiple collisions, and contact points generated during a single rock-slope interaction. v and ω variables represent incident (i) and rebound (r) translational and rotational velocities.
The linear velocity and angular velocity of each contact depend on the friction and recovery equations. During the collision, the coefficient of friction (μ) is used to determine the ratio of the normal force (N) between the simulated rockfall surface and the slope surface. The ratio is used to calculate the friction force when the block moves. The acting director of the friction force (Ff) is opposite to the tangential component of the input velocity vector. Solve the friction force in each contact and use it to calculate the linear velocity and angular velocity after the contact. Under the standard definition of friction angle, the friction coefficient (μ) is usually equal to the tangent of the friction angle (φ). In the Unity environment, the surface collision is broken down into two contact points, causing the applied friction to double ones [22] . Therefore, the effective friction angle is equal to the arctangent angle twice the unit friction coefficient (Eq. 1). The formula for the friction force applied by each contact point (Eq. 2) and the conversion between the friction coefficient and the friction angle are shown below. Unlike the coefficient of dynamic friction applied to the tangential component of velocity, the coefficient of restitution RV only works in the normal direction. This coefficient is equal to the ratio of the relative speed of the object before and after each collision (Eq. 3).
After analyzing the friction and recovery coefficient of the engine, to simulate the rockfall fragmentation in the movement process, a component called a "fixed joint" was used in the engine ( Figure 2) to connect the rock object. The two rock objects connected by a fixed joint are restricted by constraints and cannot move relative to each other in any direction, nor can they rotate. When the external force does not reach the threshold of the fracture force of the fixed joint, it will keep moving relatively. If the external force of the rock block exceeds the threshold, the fixed joint will be disconnected, and the two rock objects will no longer be constrained to separate from each other. It can simulate the fracture of the structural plane caused by the external force of the rock block exceeding the strength of the structural plane.

Three-dimensional simulation of the rockfall movement process
The three-dimensional rockfall movement process simulation software is composed of three modules: slope surface model build, dangerous rock mass fine model and rockfall movement process simulation. The technical route process is shown in Figure 3.

Slope surface model build
There are many ways to build a slope surface model, including reading the grey map to generate the terrain model and using the Sketch Up software to make the 3D model based on a terrain map. This paper proposes a simple and accurate method for building a slope surface model. First, Unmanned Aerial Vehicle is used to obtain digital images, perform uniform colour distortion correction, and import Context Capture for aerial triangulation. The new reconstruction project builds a triangulated mesh model in FBX format and a JPG format texture map, which is imported into Unity3D. Then, the coordinate should be adjusted and collision components should be added to complete the slope surface model.

Rockfall failure mass model build
The rockfall failure mass is the main body of rockfall movement simulation, and various parameters need to be set to the model beforehand. We developed a structural plane measurement program on the slope surface model, which was developed in C++ and OSG engine. Using the OSGB format model loading plug-in provided by OSG, the model can be loaded directly into the scene. In terms of user interaction, QTGUI is used as the bottom-level dependency library for GUI development, and the software interface is designed as shown in Figure 4. After the user loads the model and selects three points that are not collinear on a flat surface, the software can read the three-point space coordinates and use the three-point method [23] to calculate the dip direction and dip angle.  Fig.4 Interface of a structural plane measurement program

Rockfall movement simulation
Rockfall movement simulation is a core module of the software. It simulates the whole movement process of the rockfall failure mass from falling from instability to stopping at the bottom of the slope and obtaining its movement characteristic parameters. The specific steps are: use C# to write a script program to obtain the three-dimensional coordinates, speed, distance from the underlying surface and other motion characteristic parameters of the object, and add the script program to each dangerous rock block, output the data classification to a table file, and finally analyze and process the data to obtain information such as kinetic energy, the movement path of the block stone, and the ground elevation under the movement path of the block stone.

Geological setting
The Xiejiayan rockfall is located at a high mountainous landform with strong weathering and unloading. The elevation of the slope foot is about 1740m, the top is about 1884m. The elevation of the source area of rockfall failure mass is about 1840 meters, and the altitude difference is 140-150m. The rocks are limestone, milestone and mudstone, with a dip inclination is 189-193°, and the slope inclination is about 228°. The slope is a dip-bedding slope with a dip angle of 8-18°. The slope is steep with an angle of 70-85°, forming a cliff with few vegetations and lots of rockfall failure masses. The rockfall deposit is 15~25°, which is covered by low shrubs. The bottom of the slope is scattered with collapsed rocks before, and a passive protection net has been built to protect the residential houses.  Here, rockfalls often happen and to be scattered on the bottom of the slope surface. Based on field investigation and UAV detection, the deposition is mainly distributed close to shrubs, particular some large-size rocks. The overall view of the source area and deposition area of rockfall is shown in Figure  6. It can be seen, most of the rock volume on the slope is less than 1m³, some large-size rocks can be up to 5.0m 3 , the largest volume can be up to 24m 3 . To understand the mechanism of rockfall, the discontinuities in rock mass have been checked by a self-developed structural joints measurement program compared with a field survey. Three sets of structural joints (C, J1 and J2) in the source rockfall failure masses are obtained, (C: 191∠14°, J1: 239∠73°, J2: 96∠85°), which can be drawn by a polar stereographic projection of the upper hemisphere (Fig. 7). It can be seen, each set of the dominating discontinuity is ensured by measurement 100 times. The values of dip tendency and dip angle are almost the same with an average error of 3.7° and 4.5°, respectively. According to the combination of rock joints (C, J1 and J2), the boundary, scale and movement direction of potential rockfall can be determined. The spacing of rock joints and volume of rockfall failure masses is listed in Table 2. It can be seen, the volume of rockfall blocks is at the range of 0.5～ 4m³, and the volume of historical rockfall events at the bottom of the slope is concentrated at the range of 0.5～5m³. There is a strong correlation between them.

Model building
The model building includes two parts: the terrain model and the rockfall failure mass model (Fig.  8). The terrain model is obtained by UAV-based aerial photography, and the volume of rockfall failure masses can be obtained by measuring the terrain model. Each rockfall failure mass is formed by a group of rockfall block models arranged according to the characteristics of the structural plane. The rockfall block model includes block shape and size with parameters of density, fixed joint fracture force threshold, dynamic friction coefficient, static friction coefficient and bounce coefficient. The rockfall block model was set at the cube shape for the reason to be close to the shape of the rockfall deposits and the divided rock mass in the rockfall source area as well. As shown in Table 3, d1, d2, and d3 represent the size of the rockfall blocks, which is selected according to the structural plane spacing of rockfall failure mass and the size characteristics of the historical rockfall block mass. The density of limestone is selected as 2.26g/cm 3 based on the uniaxial tests and "Rock Mechanics Parameter Handbook" [24] . The fixed joint fracture force threshold is set as 1000N to be the rock mass shear strength. The dynamic friction coefficient is set to be 0.3, the static friction coefficient is 0.3, and the bounce coefficient is 0.6.

Rockfall movement simulation
Based on the developed rockfall movement simulation software, the Xiejiayan rockfall has been done. Taking the No. 75 rockfall block as an example (Fig. 8), it can be seen that the rockfall is to be unstable due to gravity, then it becomes a free-fall movement. While impacting on the slope surface for the first time, the force exceeds the shear strength then the rock fractured into three independent rock blocks. The No. 75 rockfall block bounced after it collided with the slope surface. Due to the steep slope, it collided with the slope surface four times and bounced to reach the gentle terrain at the bottom of the slope. The No. 75 rock block rolled a certain distance along the slope, and finally stopped under the action of friction, without reaching the passive net (Fig. 9).

Result
In terms of the proposed simulation method, the three-dimensional movement path of each rock can be obtained (Fig. 10). It can be seen, the simulated rockfall deposits are in good agreement with the results from field investigation. The rockfall movement simulation is confined by the physical principle, the kinetic energy of rockfall movement is transformed from the rock mass static energy. When rockfall collides the slope surface, the movement speed will be reduced under the effect of the coefficient of restitution, until the end of the movement. Usually, when the slope is steeper, the boulder has a larger bouncing. The kinetic energy often reaches the maximum when it reaches the middle of the slope, and the maximum bounce height as well (Fig. 11). The Xiejiayan rockfall frequent happens in recent decades of years. So far, the unstable rock mass in the source area still exists, which is prone to become rockfall events in the rainy season and earthquakes. The proposed rockfall simulation method might play a great role in rockfall prevention and mitigation. The potential movement route, the kinetic energy and the bounce height are very important during a support design or risk management for the residents.

Conclusions
This paper proposes a simulation method to obtain rockfall movement features based on the Unity3D engine. The Xiejiayan rockfall is selected as a case study to illustrate its suitability and feasibility to mitigate the rockfall losses. Three remarkable conclusions can be ensured as follows: (1)The terrain model and rockfall failure mass model can be determined according to the UAV-based photography, combined with a field survey. Unity3D and PhysX physics engine is used to develop the simulation software under CPU-GPU high-performance parallel calculation method.
(2)The presented software is used to simulate the movement process in Xiejiayan rockfall. Compared with traditional two-dimensional simulation, the three-dimensional route of the rockfall movement process is more reliable.
(3)The three-dimensional dynamic visual display of the rockfall movement process can be presented, which is much more realistic and intuitive. The simulation not only can help geological disaster management but also promote public risk perception.
For technical reasons, we haven't realized the function of dividing rock and soil with different properties and giving different surface parameters in the software. We will continue to study this problem in-depth, to improve the accuracy of simulation results.