Feature fusion using deep learning for smartphone based human activity recognition

Identification of human physical activities is an active research area since long due to its application in personalized health and fitness monitoring. The performance accuracy of human activity recognition (HAR) models mainly depend on the features which are extracted from domain knowledge. The features are the input of the classification algorithm to efficiently identify human physical activities. Manually extracted features (handcrafted) need expert domain knowledge. Thus these features have significant importance to identify different human activities. Recently deep learning methods are utilized to extract the features automatically from raw sensory data for HAR models. However, state-of-the-art HAR literature established that the importance of handcrafted features can’t be ignored as it is extracted from expert domain knowledge. Thus, in this paper we use the fusion of both the handcrafted features and automatically extracted features using deep learning (DL) for HAR model to enhance the performance of HAR. Extensive experimental results demonstrate that our proposed feature fusion based HAR model gives higher accuracy compared with state-of-the-art HAR literature for both the self collected and public dataset.

Introduction
HAR systems are gaining popularity in this digital age as they are used to identify various context-aware activities to get accurate medical facilities in the early stage of the disease and various ambient intelligence applications [16,41]. HAR systems can provide medical facilities using remote health monitoring. The collection of physical human activity signals plays a crucial role. Several sensors are used to accumulate the physical signals. Wearable sensors are commonly used to accumulate physical signals. However, it is problematic for the users to wear various body sensors. In addition, hardware cost is also associated with wearable sensors and sophisticated signal processing techniques are required for it [12]. Ambient sensors are also used to collect data for physical human activities. Ambient sensors can only be used in some specific area. Moreover, it suffers from privacy policy. Recent advancement of the smartphone, with many in-built sensors, has become a powerful tool to collect the physical human activity signals. Due to the noninvasive property and diversity of in-built sensors the smartphone is widely adopted for HAR. Continuous physical activity data can be collected using a smartphone as every person carries a smartphone with themselves. So, a massive amount of data can be collected using smartphone based in-built sensors. Accelerometer and gyroscope are two in-built sensors of smartphones popularly used in HAR [4,11,36].

Motivation
Smartphone based HAR models can be implemented using shallow Machine Learning (ML) or DL algorithms. In shallow ML algorithms, feature engineering is an important phase to extract the relevant features. Then the features are fed to any classification algorithm to identify various human physical activities efficiently and correctly [12,22]. As the raw smartphone sensory signals are not in the appropriate form, as they are highly fluctuating and oscillatory. Thus it is very difficult to identify the rudimentary patterns using these raw signals. So without feature engineering the classifiers are inadequate to give correct results. Moreover, without extracting the proper features the classifier fails to identify similar human physical activities such as walking and moving upstairs. Moreover, the manually extracted statistical features are used to overcome the problem regarding the position of the smartphone, while collecting the data. In HAR literature, most of the HAR solutions are position specific. That means, while collecting the data the position of the smartphone is fixed and the accuracy of the classifiers vary according to the position of the smartphone. The manually extracted features are not influenced by the position of the smartphone while collecting the data [2]. As a result, some standard statistical manually extracted features are gathered from raw smartphone sensory data. After extracting the handcrafted features, the shallow ML classifiers are used to recognize various human physical activities. Hence, shallow ML algorithms rely on handcrafted features [1,3,20]. The DL algorithms are one step ahead from shallow ML algorithms as DL algorithms automatically learn relevant features from raw sensor data without any human interference and also recognize human physical activities at the same time [18,40].
Both the shallow ML algorithms with handcrafted feature extraction and DL algorithms with automatically learned features have attained great success to implement smartphone based HAR models. Hence, it is so obvious to believe that the combination of manually extracted features with automatically learned features from any DL method will improve the potentiality of smartphone based HAR model [13]. Chen et al. [13], proposed feature fusion of handcrafted and automatic learning features using deep ''Long Short-Term Memory'' (LSTM) to enhance the accuracy of their proposed smartphone based HAR model. However, the authors did not consider the training time and testing time for the proposed model. The authors used two stacked LSTM which increase the training time and test time. In the HAR literature, enhancing the accuracy is not the only concern. We need an efficient and accurate HAR solution with admissible accuracy as well as the training and test time should be less. Moreover, the HAR solution should be simple enough so that it consumes minimum storage and energy. Convolutional Neural Network (CNN) is one of the popular simple DL algorithms in various application domains of smart health [14,23,31]. The aforementioned issues motivate us to use feature fusion of handcrafted and automatic learning features using CNN to escalate the performance of HAR models using smartphone sensors in terms of accuracy, training time and testing time. According to the best of our knowledge, the feature fusion using handcrafted features and automatically extracted features using CNN never used before in the HAR domain with acceptable accuracy, training time and testing time.
We summarize our contribution is as follows: -We propose an experimental feature fusion model which efficiently integrate both the handcrafted features and DL algorithm to improve the efficiency of HAR model using smartphone sensors. -We use both the self-collected and public standard dataset of smartphone based HAR for the evaluation of our proposed method. Our exhaustive experimental results reveal the proposed approach notably outperforms state-of-the-art approaches.

Related work
In this section, some relevant works for smartphone based HAR using various ML and DL algorithms are explored. In the literature of the last ten years, the majority of smartphone based HAR solutions using machine learning algorithms relied on feature extraction.

Shallow machine learning algorithms
Feature engineering and activity identification are two major phases of shallow ML algorithms. Since raw smartphone sensory data are noisy and ineffective to efficiently identify various human physical activities, it is necessary to extract relevant features with expert domain knowledge. Then the extracted features are fed to shallow machine learning algorithms to implement efficient smartphone based HAR models. The researchers in Ref. [5] investigated different human activities using four different classifiers. The authors used different locations to collect the data using the accelerometer sensor of smartphones and also extracted different features to feed to the classifiers. Decision tree (C4.5) classifier has given the highest accuracy of 95.2% among all the four classifiers. Antos et al. [6] investigated the identification of 3 different human activities using two different classifiers. The output of the SVM classifier is given as an input to the hidden Markov model to increase the accuracy. The data is collected using the smartphone's accelerometer sensor. Thirteen different features are used to do the experiment. The pant pocket location of accelerometer sensory data gets the highest accuracy of 95.2%. Bayat et al. [9], proposed a recognition system for human activities using accelerometer sensors built in smartphones. The fusion of five classifiers are used to classify the activities using self collected data and leading to an accuracy of 91.15%. In Ref. [37], the authors used K-nearest neighbor to identify six different human physical activities with 88.1% accuracy. Whereas, the authors in Ref. [35] achieved the classification accuracy of 96% using SVM classifier for seven different activities while keeping the mobile in the pant pocket. Unsupervised learning algorithm used in Ref. [26], to classify six human physical activities. In Ref. [20], SVM and k-nearest neighbor algorithms were used to classify five different human activities with the accuracy of 97.12%. An extreme learning machine was proposed in Ref. [12] to classify six different human activities with 98.88% accuracy. Similarly, in Ref. [8] and [36], random forest and multi-layer perceptron algorithms were used to classify human activities with 99.86% and 93% accuracy, respectively. The authors in Ref. [20], proposed smartphone based HAR system. In this work, the authors used time and frequency domain features. Moreover, the authors used two descriptors to extract feature sets from signals. SVM and KNN classifiers were used to recognize human physical activities. Recently, a hybrid method of filter and wrapper feature selection was proposed in Ref. [1] in smartphone based HAR systems. SFFS was used to extract various features with SVM. The SVM was used as a classifier in this. In a recent review article, [34], the authors compared various HAR systems with different perspectives such as position of smart phone, classification algorithms, human activities and accuracy. Almost, all the proposed HAR models using traditional ML are using handcrafted feature extraction to efficiently identify several human physical activities.

Deep learning algorithms
DL algorithms are able to learn the relevant features automatically. As a result, the performance of DL algorithms is remarkably high in case of smartphone based HAR models. CNN is gaining popularity in the HAR domain due to its hierarchical feature extraction capability. For example, Zeng et al. [42], proposed a CNN based HAR using mobile sensors. Three different publicly available datasets including ''Antitraker'' are used in this research work for experiment. The authors have taken the advantages of ''local dependency'' and ''scale invariance'' of CNN to achieve the accuracy of 88.19%, 76.83% and 96.88% using three different datasets respectively. In Ref. [24], the authors presented a smartphone based HAR model which can automatically learn features using ''Sparse Auto-Encoder (SAE)''. The tri-axial accelerometer, gyroscope and the magnitude of both are used as different channels. In this work, using statistical metrics the classification accuracy is achieved by 97.55%. Ronao et al. [29], proposed another HAR solution using CNN. In this work, the authors used a HAR smartphone dataset from ''UCI'' repository. Also, the authors used manual hyperparameter tuning to achieve 95.75% accuracy to identify human physical activities. In their other work [30], the authors presented a CNN which is used to learn effective features from raw smartphone sensor data. In this work, they used temporal ''fast Fourier transformation'' on the raw data with CNN to implement the HAR model. Bevilacqua et al. [10], proposed CNN based HAR to identify sixteen different lower limb activities using five different sensors including accelerometer and gyroscope. In another work, Jiang et al. [21], proposed a CNN based HAR solution using ''UCI'' public dataset with 97.5% accuracy.

Handcrafted features with DL algorithms
In Ref. [27], the authors used time and frequency domain features with CNN to implement a smartphone based HAR system and achieved accuracy of 96.41%. The authors in Ref. [32], used two directional features with bidirectional long short-term memory (BLSTM) for incremental learning in HAR and achieved approximately 93% of average accuracy. Almaslukh et al. [2] proposed a robust position independent HAR system using CNN with 88% accuracy. In this work, the data was collected using smartphones and smartwatches located in seven different positions of the body. Also, the authors used some handcrafted features to locate the position of the smartphone. However, the authors did not integrate the handcrafted features and automatically extracted features to enhance the performance of the proposed HAR model. In Ref. [13], the authors used fusion of handcrafted features with automatic learned features using deep LSTM, to enhance the performance of the proposed HAR model. In this work, the authors achieved the average accuracy of 96.44% and 98.67% using public dataset from UCI and their own collected dataset respectively.
According to the smartphone based HAR literature, proper feature fusion in between handcrafted features and automatic learning features is performed only in Ref. [13] using a deep LSTM algorithm. In real application, both the manually extracted features with expert domain knowledge and automatic extracted features using DL are used to implement efficient smartphone based HAR models. In this paper, our aim is to implement a smartphone based HAR model using both the features to escalate the performance of the proposed HAR model using CNN.

The proposed model
This research work consists of different phases to identify human activities with higher accuracy and lower time cost.

Data collection
In this work, the data is gathered using Samsung Galaxy On-Max android smartphone. We create one android application to collect sensor data of six different human physical activities such as sitting, standing, walking, lying, walking upstairs and walking downstairs. The android smartphone application uses tri-axial accelerometer and gyroscope sensors with frequency of 50 Hz to accumulate the data, keeping the device in the front pant pocket or in hand. The data is collected from 25 subjects including 15 females and 10 males aged about 15-45 years, height about 163-172 cm and weight about 52-65 kg. The subjects are totally healthy without any medical complications. All subjects are asked to perform six normal human physical activities. All the activities are performed for three minutes with the repetition of five times by each of the subjects. All the activities are performed in both the indoor and outdoor conditions. Figure 1 shows the experimental setup of data collection in our experiment. This dataset consists of 15562 instances and 146 features.

Data pre-processing
The raw data collected from the smartphone in-built sensors consists of noise. As the data is collected using mobile applications there may be possibilities of missing and redundant data as there is a difference in individual data collection. To remove the high frequency noise and the gravitational acceleration from the signal, a low-pass elliptic filter with 20 Hz cutoff frequency followed by a high-pass elliptic filter with 0.5 Hz cutoff frequency are applied respectively [15].
Each signal is divided in 5 s sliding window with an overlap of 2 s between two consecutive windows as of state-of-the-art literatures [12,20] show that 2-5 s sliding window with 20-50 Hz frequency is the ideal situation for the segmentation of the collected data.

Handcrafted features
Feature extraction is the key major step in conventional machine learning to build a good classifier. Smartphone sensors that collect raw data are noisy. Hence, using this data we are not able to recognize physical human activities efficiently. Various time and frequency domain statistical features are useful for HAR. Time domain features are useful to separate various static features such as standing and laying. Frequency domain features are useful to differentiate in between static and dynamic features. We have taken the extracted features similar as [4,7,12,28], which are the standard features used in various HAR literature shown in Table 1. To improve the training performance much more features have been extracted to describe each activity window.

Automatic feature learning
The great advantage of DL algorithms is the ability to learn relevant features automatically from raw sensory data. The smartphone sensor data is time-series data [25]. Research has shown that there are many major advantages over other strategies in the use of CNNs for time series classification [39]. CNN is a highly noise-resistant model. Using CNN we can extract highly relevant and deep features.
A typical architecture of CNN is shown in Fig. 2. The network architecture of our CNN consists of an input layer, convolution layer which extracts the features from raw sensory data, a pooling layer to reduce the size of the extracted features, fully connected layer to integrate all the extracted features and a softmax layer to make differentiation between activities.

CNN based HAR
This section demonstrates the feature extraction method used in CNN. Our proposed method is shown in Fig. 3. Triaxial accelerometer time-series data is represented with overlapping window sizes of length w. This data is given as input to CNN. There are three different types of layers in our CNN-based model. A weight matrix w l i;j is adjusted to learn the network. It is the input of h l i 's unit and output of the h lþ1 j unit. ith unit of layer l is denoted by u l i . x l i and y l i are the total input and output of u l i and h l i units respectively.

Convolutional input layer
Convolutional layer is the first layer of the CNN, extracts the important features and patterns from data. In order to form a generous representation of data, multiple feature maps are included in this layer. Multiple feature maps of accelerometer data are represented as x :;j i ; j ¼ 1; Á Á Á ; J. Suppose there is an N unit layer as input, followed by a convolutional layer. If we use kernel or filter size as M, the convolutional layer consists of N À M þ 1 units. The output of the convolutional layer is represented as : Here, the activation function is r, the j-th feature map is b j , the weight for the j-th feature map and m-th filter index is w j m .

Max-pooling Layer
The extracted features by convolutional layer are partitioned in different sets. In each set, max operation is applied to get the output values, given by Here, polling size and stride are R and S respectively. The convolutional and polling layer stacked one after another to extract discriminant features.

Training procedure
Initially, the forward propagation is performed in a convolutional layer with N number of nodes using Eq. 1. The size of the convolutional layer is N À M þ 1, where M is the filter size. The max-pooling layer takes the input from the output of the convolutional layer. The max-pooling is done using Eq. 2.
If the input layer consists of N nodes then the maxpooling layer consists of N r nodes, where r is converted to a single value window width using the max function. After that a fully connected layer is followed by the max-pooling layer. The forward propagation is represented as: The topmost layer is the softmax classifier. The feature vector extracted from the convolutional and pooling layers is given as an input to the softmax classifier. The feature vector The topmost pooling layer consists of I number of units. The softmax classifier is denoted as where c = activity class -M = index of the last layer -N c = total number of activity classes After the first iteration of forward propagation, error value is generated using loss function L. We update w, using gradient descent. In our case, we use ''cross entropy cost function'' as a loss function. The gradient is measured as follows for the fully connected layer: where y l i ¼ rðx l i Þ þ b l i is the non-linear mapping function. The node j of ðl þ 1Þ-th layer is x lþ1 j , x l i ¼ P j w lÀ1 j;i y lÀ1 j . The gradient in convolutional layer is calculated as: We stop the CNN training phase if it does not boost its output for five successive epochs on the validation sample.

Proposed feature fusion
Extracted features with expert domain knowledge and the features learned by DL algorithms both have significant impact in HAR model. To take the advantage of both the features, we propose a feature fusion concept to improve the identification accuracy of various human physical activities using smartphone in-built sensors. The proposed feature fusion model is outlined in Fig. 3. In this work, we have explored CNN for feature learning. The raw sensory data is given as input in 2D CNN for feature learning. At the end, the learned features are taken as input by the fully connected layer. Simultaneously, the manually extracted features as shown in Table 1, are taken as input by another fully connected layer to get more conceptual features. Finally, both the features in combination are given as input to the softmax layer for human physical activity classification.

Performance evaluation
In this section, we explain the overall performance of our proposed fusion method after exhaustive experiment.

Experimental setup
The description of our own dataset used in our experiment is illustrated in Sect. 3.1 in detail. To establish the veracity of the proposed feature fusion method, in our experiment, we use another public dataset which is taken from open source ''UCI Machine Learning Repository'' [3] for smartphone based HAR. This smartphone sensor based dataset consists of 30 subjects aged about 19 to 48 years, with 6 different human physical activities such as sitting, standing, walking, lying, walking upstairs and walking downstairs.
Here, a waist-mounted smartphone SamsungGalaxySII ð Þ with in-built sensors is used. Both the accelerometer and gyroscope sensors are used to collect the data. In this dataset total number of instances are 10299, number of features are 561. The public UCI dataset is already pre-processed using noise filters. Here, Butterworth low pass filter was used to separate the gravitational components from the body motion components. In this, a low pass filter with 0.3 HZ cutoff frequency was used, assuming the gravitational force only low frequency components. Then fixed-width sliding windows of 2.56 s with 50% overlapping was used.
In both the datasets, 70% of the total data are used for training the model and the rest 30% of the dataset are used for testing purposes. We use both the time domain and frequency domain features in our experiment as mentioned in Table 1. To validate the performance of the recommended model, we compare with two conventional ML methods such as SVM [3], Ensemble Learning Machine [33], and one DL of CNN [17]. Default parameters are used for all the benchmark schemes. In our proposed feature fusion, the CNN algorithm is used for feature learning. There are so many hyperparameters used by CNN algorithms. Using the trial and error method we have fixed the hyperparameter values such as batch size is of 1024, number of epochs varies from 100 to 200, Initial learning rate is 0.005, filter-size is set to 20, max-pooling size is set to 3, number of filter map 180, 60, and 30 respectively. The top two fully connected hidden layers have 1024 and 30 nodes respectively.

Results on our own collected dataset
The experimental results using self-collected dataset are tabulated in Table 2. Using our own dataset, automatic feature learning using CNN outperforms the conventional ML methods such as SVM and ELM with manually extracted features. This signifies that the automatic extracted features are more relevant to identify human physical activities using our own collected dataset. However, the proposed feature fusion method outperforms all the 3 benchmark schemes. Figure 4, shows the classification accuracy in terms of percentage using our own collected dataset.

Results on UCI public dataset
The experimental results using UCI public dataset are tabulated in Table 3. According to our experimental result, it is clear that using our feature fusion approach we are able to get higher accuracy compared to all the 3 benchmark schemes we have considered. Our proposed method outperforms the other benchmark schemes. Using UCI public dataset, the performances of conventional ML methods such as SVM and ELM outperforms the DL method, CNN. Both SVM and ELM have used handcrafted features. Whereas, CNN utilizes the automatic feature learning concept. Therefore, there is a significant impact of manually extracted features to recognize human physical activities. In our proposed feature fusion, we have fused both the features to enhance the performance of classifiers. Figure 5, shows the classification accuracy in terms of percentage using UCI public dataset.
The proposed feature fusion method outperforms using both the datasets. However, the UCI public dataset gives higher classification accuracy compared to our own collected dataset. The higher number of features can be a  reason to get higher accuracy using UCI dataset. There can be other reasons as well such as the position and orientation of the smartphone while collecting the data. So, using different datasets sometimes the conventional ML methods with manually extracted features give higher classification accuracies and sometimes DL method with automatic feature extraction gives higher classification accuracy. Therefore, we can conclude that both the manually and automatically extracted features are relevant to identify different human physical activities. As a result, the fusion of these features outperforms all other aforementioned methods using both the datasets which utilizes the advantages of both the features.

Compared with state-of-the-art smartphone based HAR literature
We compare our proposed feature fusion method with some state-of-the-art smartphone based HAR literature as mentioned in Table 4. Most of them have not mentioned the training and testing time of their proposed model. The most relevant research work [13], has not mentioned the training and testing time. However, the performance accuracy of our proposed fusion model is higher than the proposed model of Chen et al. [13] using two different datasets. Compared to the research work of Dhanraj et al. [14], the accuracy of our proposed method is much higher. However, the training and testing time are higher using our proposed method. Due to the feature fusion, it may be possible to take higher training and testing time but it is admissible. Otherwise, in terms of accuracy our proposed method using two different datasets out performs the stateof-the-art smartphone based HAR literature.

Application
The significant uses of HAR are in dynamic and helped living frameworks for smart homes, medical care observing applications, checking and reconnaissance frameworks for indoor and outside exercises, and tele-submersion applications. Moreover, studies of HAR are helpful to identify various lifestyle diseases such as obesity, diabetes. Moreover, it can also be used for rehabilitation purposes such as cardiac rehabilitation or any neurological rehabilitation.Therefore, the research community is always trying to enhance the HAR systems using various ML or DL    [38], the authors proposed a ''SpatioTemporal Human Activity Model (STHAM), for simulating SARS-CoV-2 transmission dynamics''. The movement of the people and their interaction with individuals in the society is the primary cause of the flow of ''SARS-COV-2'' [38].

Conclusion and future scope
In this work, we propose a feature fusion model using manually extracted features with expert domain knowledge and automatic learned features by CNN for smartphone based HAR. In this, we have used both the self collected dataset and UCI public dataset to assess the performance of the recommended feature fusion method. The experimental results show that the performance of the classifiers is not always the same. For instance, the performance of the shallow ML algorithms (SVM and ELM) with handcrafted features are better than CNN. Therefore, there is a significant impact of the handcrafted features in case of HAR. However, the performance of CNN is better than shallow ML algorithms (SVM and ELM) with handcrafted features using our own collected dataset. So, it is obvious to conclude that the feature fusion of handcrafted feature and automatic learned feature enhance the performance of the HAR model. In our experiment, we use CNN for automatic feature selection. As a result, the training time and testing time is highly admissible in our proposed fusion based HAR model. Finally, from the experimental results we conclude that our proposed approach outperforms the stateof-the-art smartphone based HAR literature for both the datasets in terms of accuracy, training time and testing time.
In future work one can consider different dynamic activities to identify with several other deep learning approaches. The position and orientation of the smartphone to escalate the performance of the smartphone based HAR system can be considered in future.


extfieldsofstudy:
Computer Science
Physics

s2fieldsofstudy:
Computer Science
Physics

text:
Exploring Airline Gate-Scheduling Optimization Using Quantum Computers

This paper investigates the application of quantum computing technology to airline gate-scheduling quadratic assignment problems (QAP). We explore the quantum computing hardware architecture and software environment required for porting classical versions of these type of problems to quantum computers. We discuss the variational quantum eigensolver and the inclusion of space-efficient graph coloring to the Quadratic Unconstrained Binary Optimization (QUBO). These enhanced quantum computing algorithms are tested with an 8 gate and 24 flight test case using both the IBM quantum computing simulator and a 27 qubit superconducting transmon IBM quantum computing hardware platform.

Introduction
One of the many complex problems facing the airline industry today is how to optimize large combinations of cargo and passenger traffic among the planes, gates, personnel and the air traffic flows originating or terminating at an airport. Finding solutions to this complex combinatorial optimization problem is important for airlines in order to be able to control costs and minimize inefficiencies (Stollenwerk et al. (2019)).

Article submitted to INFORMS Journal on Computing
A mathematical framework that can be used to compute optimal solutions for this type of problem is known as the quadratic assignment problem (QAP). This is a difficult combinational optimization problem of the NP-hard class (Sahni and Gonzalez (1976)).
A widely cited application of a QAP is the assignment of facilities to locations such that each facility is assigned to only one location. Given that facility versus location assignment costs are known, the problem is finding a minimum cost allocation of facilities to locations taking the costs as a sum of all distance and flow products. A constrained QAP is a natural framework for assigning flights to gates under a variety of constraints and objectives.
Generally, such QAP type problems of size n > 30 cannot be solved optimally on digital computers in reasonable time. For example, see (Loiola et al. (2007)), or (Gonçalves et al. (2017)). Improvements through approximations, heuristics and special structures (Kim et al. (2017)) can all be implemented in order to get reasonable solutions for some, but not all, purposes and time frames. The question remains if one can do better than reasonable or do more than some.
One expectation of a viable quantum computer is the capability to find an optimal solution to large quadratic assignment problems. Delta Air Lines has partnered with North Carolina State University in an effort to explore how quantum computing may be able to address such types of flight-gate scheduling optimizations. The goal will be to understand what are the potential capabilities for a quantum computer to be able to optimally solve large quadratic assignment problems that are so computationally prohibitive they could never be computed using classical digital computers.
In Section 2 we present a combinatorial optimization model (CO) model that is a natural QAP approach to gate scheduling. In Section 3 we describe the quantum computing hardware architecture and software environment used when porting this type of model to a quantum computer. In Section 4 we discuss the simulations and experimental results and analysis and comparison between the digital and quantum computing methods. Finally in Section 5 there is a discussion of conclusions that can be inferred from the work and a look toward future steps for improving this algorithm as quantum computing hardware platforms become more robust.

Digital Computing Implementation of Gate Scheduling Optimization Algorithms
The natural base formulation of the gate-scheduling problem is a QAP where the decision is which gate to assign to each plane. The objective is naturally quadratic with the major 3 component being the interaction of pairs of planes. For this paper we will use the passenger walking distance between gates given a predicted value of the number of passengers connecting between planes. To model the plotting problem at a fixed point in time the following variables need to be specified.
• P = Set of planes to park at gates • G = Set of gates at which to park planes • v ij = Expected number of passengers connecting between planes p i and p j • d kl = Walking distance between gates g k and g l The challenge is to minimize the objective function J with respect to the constraints that follow: Modifications to this initial generic formulation need to be made in order to arrive at a specific useful model. As an example, we would want to replace the constraint Eq.(3) with the following.
The constraints in Eq.
(2) and Eq.(4) mathematically state that a plane needs to be assigned a gate, but a gate does not need to be assigned a plane.
This set of constraints should be modified further. There is a long list of modifications that should be considered for a useful model. Here is a small sampling from this list.
• Gate Restrictions For various reasons, not every gate can accommodate every plane.
• Gate Closings Not all gates can be used all the time.
• Contentions Even if two particular planes can individually be assigned to two particular gates, proximity may preclude both assignments being made at the same time.
• Alleyway Congestion Gates using the same alleyway should avoid planes departing close in time.
• Adjacent Gates Gates close in proximity should avoid planes departing close in time.
Solving a realistic version of this gate plotting model for a large and busy airport would be impractical by normal methods. An interesting aspect of quantum computing is that the underlying QAP itself seems to be a more natural problem to solve. What one avoids in classical computing is instead what one uses in quantum computing. Exploring the option for utilizing a quantum computer to optimize these gate-scheduling problems may be a viable path forward in the future.

Quantum Computing Implementation of Gate Scheduling Optimization Algorithms
Calculating a complete solution of a plane-gate scheduling constrained QAP problem for a very large and busy airport is impractical with digital computing technologies. A quantum computer may have the potential to circumvent some of these difficulties.
Exploring the possibility of implementing a gate scheduling optimization algorithm for a quantum computing system requires a re-design of the digital version. This includes both developing a working understanding of both the quantum computing hardware architecture and software environment so that the digital version of this QAP algorithm can be successfully re-structured and migrated to a quantum computing hardware system.

Quantum Computing Hardware Architectures
Quantum computers are based on a fundamentally different premise for computation when compared to a digital computer. The digital computer operates on the mathematics of a base 2 system, whereas the fundamental premise of computation for a quantum computer is based on something called a qubit. Unlike a classical bit that only has two states (sometimes designated as "On" or "Off" or "0" or "1"), a quantum bit (qubit) can be represented by many different weighted, linear combinations of "0" and "1". The rules for quantum computation are based on the physics of quantum mechanics and concepts of superposition and quantum entanglement rather than the electrical engineering design of digital logic circuits. with that computation or of re-running a computation from the beginning in a deterministic way. Upon completion of the computation, the idea of an "answer" also has a very different interpretation. Whereas a digital computer will give a deterministic answer, a quantum computer will provide a histogram representing a spectrum of measurements with their relative probabilities.
Today there are several different types of quantum computing hardware architectures that have been developed. These include an analog-based platform based on Quantum Annealing, various superconducting transmon devices based on non-linear Josephson junction circuits, and trapped ion devices. In this paper, the IBM superconducting, transmon-based quantum computing hardware platforms were used for the computations. We selected the IBMQ Toronto, which is a 27-qubit quantum computing processor.
Noise is an additional consideration when running on a quantum computing hardware platform. Today's quantum computers are best described as Noisy Intermediate Scale Quantum (NISQ) machines. Such types of noise present in these machines today adversely impacts the stability of the quantum computations. Noise essentially destroys the coherence of the quantum state and limits the size of the application that can be run on these machines.
Mitigation of this noise and development of error correction methodologies that will extend the quantum computer's computational capabilities are both active area of research. Fig. 1 shows the general architecture and qubit connectivity of the machine and a graphical "heat map" showing the level of 1 qubit and 2 qubit errors for this processor.
A critical component in the overall planning for the quantum computation is choosing the qubits for the computations on a quantum computing hardware architecture with the best profile for low error rates on that platform. Quantum computing programs that run on today's NISQ platforms carefully place the quantum codes for that algorithm on the subset of qubits and connections with the best coherence and smallest error parameters in order to deliver a better execution of the program. Some additional properties of the Toronto processor and a brief discussion of error rates and their impact on quantum calculations are presented in the Appendix. The main goal of VQE is to find the desired eigenvectors and eigenvalues by optimizing a parameterized quantum program. The lowest eigenvalue is of primary interest in the VQE approach since it is the optimum solution of the problem, same as the lowest energy is the solution to a Hamiltonian in physical systems. VQE can be used to find the solution for Eq. (6), Because |ψ(θ) is the normalized solution for Eq. (6), it can be the minimum eigenvalue of Finding the minimum eigenvalue can be an NP-hard problem as the size of the Hamiltonian matrix increases. To address this problem, different quantum techniques have been developed (Moll et al. (2018)). Quantum phase estimation (QPE) is one of the main techniques that can be applied to a unitary quantum circuit derived from a problem to find its Article submitted to INFORMS Journal on Computing

Figure 2
The schematic diagram of the variational quantum eigensolver (VQE) algorithm minimum eigenvalue, that is the solution of the Eq. (6). However, practical resource constraints with today's NISQ hardware limit the implementation of this algorithm. Hence, it is not possible to find a complete solution to this problem using near-term noisy quantum computers.
Alternatively, researchers have been developing some hybrid classical-quantum algorithms Different classical optimizers can be used to solve the optimum values for θ. These techniques perform various forms of gradient descent methods to apply iterative optimization algorithms to solve the problem (Kandala et al. (2017), Romero et al. (2018)).
This paper considers two well-known classical optimizers.
• The constrained optimization by linear approximation (COBYLA) algorithm is a numerical optimization method that finds the min/max of the objective function without knowing the gradient value (Gomez and Hennart (1994)).
• The simultaneous perturbation stochastic approximation (SPSA) algorithm is used to optimize a system with multiple unknown parameters. This algorithm only uses the cost function measurements and is efficient in high-dimensional optimization problems. The gradient approximation using this technique requires only two measurements regardless of dimension of the problem which makes this algorithm to be a powerful technique to optimize a problem with multiple unknown parameters (Spall (1987)-Spall (1992).
The implementation and results for each of the optimizers will be presented in section 4.

Binary Optimization Problem and Quantum Hamiltonian
Combinatorial optimization (CO) is a major class of optimization problems that has various applications in industry and research. In particular, CO for Graph Coloring has been extensively used in different application domains such as transportation management, social media, telecommunication, and resource scheduling. Most of these applications are NP-hard problems that are attractive but challenging for researchers to solve.
Three approaches can be considered to tackle CO: Exact Algorithms, Approximation Algorithms, and Heuristic algorithms. (Halim and Ismail (2017), Festa (2014)) An exact algorithm, by definition, is one that is guaranteed to find a solution, if one exists.
Therefore, unless P = NP, an exact algorithm cannot have worst-case polynomial running time for an NP-hard problem. Thus, in practice, an exact algorithm can take too much time to reach an optimal solution, though it might find good solutions in reasonable time.
An approximation algorithm is an exact algorithm to an approximate version of the prob- close it comes to optimal, e.g., the solution found will have an objective value at least 4/5 of the true maximum of the original problem.
A heuristic algorithm, whether it works on the original problem or an approximate version, does not have a proven guarantee that it will find a solution in finite time for the problem on which it works. Heuristic algorithms are usually faster and often simpler than other algorithms and, so, represent another trade off between speed and solution quality.
Recently, Quadratic Unconstrained Binary Optimization (QUBO) has been introduced to cover a large variety of CO problems (Hao et al. (2014), McGeoch (2014), Kadowaki and Nishimori (1998), Bertels et al. (2020)). The QUBO technique can find an optimal solution for many types of NP-hard problems such as graph partitioning, graph coloring, scheduling management, and register allocation.
A QUBO problem can be formulated by Eq. (8) as described in (Glover and Kochenberger (2018)), where Q is a n × n matrix with real numbers, X is the state vector with element of {0, 1} n and P is a positive and large number. The goal of Eq. (8) is to find the minimum of the objective function which is the minimum eigenvalue of the matrix H. Eq. (8) can also be expressed by Eq. (9) as described in (Lucas (2014)), where W i are the linear weights Q ii for i = 1, 2, ..., n and W ij are the quadratic weights Q ij for i < j. Here x i is a binary variable that can take 0 or 1 and as a result x 2 i = x i . In order to transfer Eq. (9) into an Ising model, the following substitution can be used as where x i ∈ {0, 1} and Z i ∈ {−1, 1} for i = 0, ..., n. By applying the substitution the Hamiltonian matrix would be (Barahona et al. (1982), Istrail (2000)), This Hamiltonian matrix will be a form of tensor products of Pauli Z operators. Note that since the Z is a diagonal matrix, the Hamiltonian matrix should be diagonal so that it can be used in a VQE optimization process. In order to solve the k-coloring problem using the standard QUBO optimization technique, nk qubits are required. The required number of qubits is the main constraint of the standard QUBO optimization approach. Serra et al.
(2021) has recently presented a quantum annealing (QA) technique to find near optimal solution for the QUBO optimization problems by preparing the ground state of the Ising spin Hamiltonian. In this paper we have considered a different type of quantum computer which is a universal Gate-based superconducting quantum computer and has capability to find the optimum solution to the combinatorial optimization problem. In addition, in section 3.3 a new algorithm is introduced which solves the k-coloring optimization problem by applying n log(k) number of qubits which exponentially reduces the required number of qubits compared to the standard QUBO technique.

Space-Efficient Graph Coloring Quantum Computing algorithms
In section 3.2.1, we showed that the standard QUBO optimization algorithm requires nk number of qubits which increases exponentially with the problem size. Therefore, it is infeasable to run a gate scheduling problem with a reasonable size on existing NISQ devices.
In this section, a new space-efficient technique that maps a k-coloring problem to a lower number of qubits is explained. The number of colors is encoded to m bits (total 2 m colors) and the goal is to assign a color labeled by m bitstrings to each vertex, (Tabi et al. (2021)-Krawiec andZimborás (2020)). As mentioned in the standard k-coloring approach, two neighboring nodes should not have the same color, therefore in the space-efficient technique a cost function should evaluate to nonzero if the neighboring nodes have the same color code.
For a graph with adjacency matrix A, the graph coloring problem can be mapped to the ground state of the following Hamiltonian Eq. (12).
The ground state of the n log(k)-qubit Hamiltonian matrix is the solution to the graph coloring problem which corresponds to the proper bitstring assignment of the graph vertices.
If the number of colors is not a power of 2, i.e. 2 m−1 < k < 2 m , m number of bitstrings is considered to map the colors, however, the bitstrings in which m i=1 2 k−i b i < k are allowed for should be added to the Hamiltonian matrix to penalize the non-allowed bitstring which is (b 1 , b 2 ) = (1, 1). By applying Eq. (12) and considering the new penalty term, the corresponding Hamiltonian matrix will be, As seen in Eq. (13), the last term will be added since four colors has been considered for a problem that only requires three colors. This penalty term helps remove one of the binary coded colors from the list.

Simulation and Experimental Results
This section discusses the simulation and experimental results for both the standard QUBO optimization technique and the space-efficient coloring scheme. All quantum simulations and computations on both IBM's quantum simulator and quantum computing hardware platforms were accessed through the NC State University Quantum Computing Hub connection into the IBM Quantum Network.
The gate scheduling problem described in this paper was formulated and mapped as a graph coloring problem and tested on both a quantum simulator and an actual quantum computing hardware platform. We show both simulation and experimental results of applying both graph coloring implementations discussed in Sections 3.2.1 and 3.3 to find an optimal solution for an example gate allocation problem and compare the results by considering different criteria such as number of required qubits, run-time and circuit depth.
For our example test case, we consider a setup with eight gates and 24 flights. Fig. 3 shows the corresponding derived graph based on the flights arrival and departure time interval. The graph nodes correspond to the flights and an edge between two nodes shows the time overlap The corresponding derived graph based on the flights arrival and departure time interval.
between the two flights. The goal here is to colorize the graph using quantum embedding approaches to minimize the problem objective function. By considering Eq. (11) and Eq. (12) and converting the graph coloring problem to the corresponding Hamiltonian matrices, the problem was mapped onto the IBM Qiskit platform, (Qiskit (2021)) and used to solve for the ground state of the Hamiltonian matrices.
A Noise-Aware Compilation method from Qiskit was used for the circuit compilation and optimization. This method maps the circuit to match the hardware topology as discussed in Section 3.1 and also does light-weight optimizations by collapsing adjacent gates as appropriate. We notice a direct benefit from the efficient embedding in all aspects of the comparison.
As both circuits run the same ansatz architecture, we see that the efficient embedding proves advantageous, as it uses fewer qubits.
As discussed in the hardware architecture section (Section 3.1) efficient embedding for mapping the problem to qubits on the hardware platforms focuses on the minimization of the usage of SWAP operations. Quantum algorithms formulated with fewer 2-qubit interactions will require fewer SWAPS to implement them on quantum platform. A design change requiring fewer 2-qubit operations results in a shorter circuit (program) depth with a more efficient embedding and hence a shorter run-time. As discussed in Section 3.3, the efficient embedding scheme requires fewer qubits. This shorter run-time allows larger gate-scheduling problem sizes to be implemented on the quantum platform. Simulation results were obtained using IBM's Aer simulator. Experimental results were obtained by running the circuits on IBM's Toronto 27 qubit quantum computing hardware platform. Fig. 4 shows that the graph was successfully colored by applying the quantum space efficient embedding technique. As it can be seen the quantum machine is able to find the optimum solution for this problem.
We investigate different classical optimizers, considering different well-known methods such as COBYLA and SPSA to optimize the parameters of the quantum circuits. The results has been shown in Fig. 5   Due to the required number of qubits and the software/hardware limitation, the previous case study (24 flights and 8 gates) cannot be solved by the standard graph coloring technique using today's quantum technologies. In order to evaluate and compare the presented quantum graph coloring techniques in terms of the number of qubits, program depth and run-time, a case study of flight gate scheduling problem with five flights has been considered.
Both techniques were applied and the derived Hamiltonian matrices were run on the quantum machine. Table 1 shows a comparison between the two implementations described in sections 3.2.1 and 3.3 in terms of the number of qubits, run-time and circuit depth. As it can be seen for the standard embedding, the required number of qubits exponentially increases as the size of the problem increases. The efficient embedding technique will reduce the number of qubits exponentially and requires smaller program depth which results in lower run-time on NISQ quantum machines to solve an optimization problem. Fig. 6 shows the measurement probability of the possible solutions for this case study by running the corresponding Hamiltonian, derived by efficient embedding technique, on a noisy IBM quantum machine.
The SPSA optimizer was utilized to optimize the circuit parameters. As can be seen the correct solution has the highest probability among the other possibilities and IBM-Q machine was capable of solving this optimization problems however, there is still challenges to tackle the optimization problems with relevant practical dimension due to the hardware limitation (noise), the required number of qubits and program depth.

Figure 6
Hardware results with the measurement probabilities for a bank with eight flights and eight gates.

Conclusions and Next Steps
The experimental results reported here showed that smaller circuit size (program length) allows for a more efficient mapping to match the limitations of the hardware topology. As a result we were able to map larger problem sets to the actual quantum hardware platforms.
This is a critical advancement in the usage of variational quantum algorithms solving such optimization problems. Using quantum computers with the methodologies discussed here, it is possible to utilize variational algorithms such as VQE to gate allocation problems with large sizes in the near future.
This effort is forward-looking in the sense that actionable solutions to solve these complex gate-scheduling problems are currently beyond the capability of contemporary Noisy Intermediate-Scale Quantum (NISQ) technology. However, over the past several years there has been significant advancements both in designing and manufacturing quantum hardware processors with larger numbers of qubits and improved performance characteristics.
Advancements in error mitigation and error correction are also shaping the ability of application developers to formulae more complex problems that can be implemented on quantum computers.
The technology for building larger quantum computer hardware processors is developing rapidly and possibly at some point in the next 5 to 10 years the computational capability of a quantum computer will overtake the capabilities of largest digital computers that can be manufactured for solving these type of NP hard optimization problems. The results from this basic research focused on developing and applying state-of-the-art quantum algorithms and programming for constrained QAP problems on today's quantum computing hardware platforms will place Delta Air Lines in a "quantum ready" position to capitalize and exploit quantum computing technology in the future as an alternative to today's digital computational methods.

Author Contributions
HM, YO and MI wrote the python code and ran the simulations and quantum computations on Toronto. RS and JH organized the test gate-scheduling database and wrote the introduction and classical optimization sections of the paper. PD, HM and MI wrote the quantum optimization, results, conclusions and appendix sections. PD did the overall organization and integration of the material in the paper and PD and MS did the final editing of the document.

Appendix. Superconducting Transmon Quantum Computing Architecture
In this section, we briefly describe the architecture of the IBMQ Toronto, a 27-qubit IBM superconducting transmon quantum computing hardware platform used in our experiments. Fig. 1 shows the general architecture and qubit connectivity of the machine and the color coding map that indicates the expected error for each individual qubit and 2-qubit connection (CNOT error). A key aspect about IBM quantum hardware is that each qubit and qubit connection has different error rates. These rates, and other system parameters such as qubit coherence times are not static and change with time. IBM performs periodic calibrations on their quantum systems and the system properties update once this calibration sequence is complete. The properties play a critical role in quantum circuit execution, as they are usually utilized in compiling and optimizing the quantum program.
The IBM machines are accessed via a cloud portal where users can write quantum computing programs in python using jupyter notebooks. A jupyter notebook can then be submitted either to a quantum computing simulator or one of several different hardware backends onto the actual quantum computing hardware platforms. The daily measured and published error rates are important factors and considerations when loading programs (circuits) representing quantum algorithms onto a quantum computing hardware platform.
Programs loaded onto a quantum computer require that each computation in the program be executed on adjacent qubits. The actual physical layout of the qubits shows that there is limited connectivity between each qubit. For large programs it is not possible to place each computation on the qubits so that they are adjacent throughout the execution of the entire program. This limited connectivity of the qubits on the machine requires that the information for the calculation that is contained on distant qubits must to be moved until the two qubits are adjacent. This is called a SWAP operation. This has a major implication Article submitted to INFORMS Journal on Computing on a quantum program's length that can be executed on a quantum computer before the machine loses its quantum mechanical wavefunction coherence depth in time and stops functioning. A key goal of compiling a quantum program is to adequately map it to the subset of qubits and connections with the best coherence, error parameters and a connectivity that guarantees the minimum amount of extra computation (SWAP gates) so that it results in the best and longest execution of the program.


Adaptive Adjustment Object Detection Algorithm under Multiple Mechanisms Based on GAN

Target tracking is prone to problems such as target loss and identity jump when the target is occluded and the attitude is changed. In order to solve this phenomenon, this paper proposes the adaptive adjustment object detection algorithm under multiple mechanisms based on GAN. )is algorithm introduces a gradient penalty mechanism to the discriminator and uses the relative discriminator structure to reconstruct the discriminator, so as to improve the discriminatory ability of the discriminator. )en, through the feedback mechanism, the obtained data is fed back to the generator network in time, and the genetic mechanism is used to speed up the positioning of the key areas of the image. Experimental results show that compared with other existing algorithms, this algorithm can effectively locate and distinguish under different environments. And, the target still maintains a high resolution. When the target is occluded, it can effectively avoid the phenomenon of target loss and identity jump.

Introduction
Target detection is one of the important research directions in the field of computer vision. It integrates technologies such as positioning, recognition, and classification and is widely used in fields such as intelligent monitoring and human-computer interaction. In recent years, the target detection algorithm has made great breakthroughs. e representative target detection algorithms can be divided into two main categories. One is the R-CNN algorithm (R-CNN, Fast R-CNN, Faster R-CNN, etc.) [1,2]. e algorithm needs to generate the target first. Candidate frame (target location), and then, classify and regress the candidate frame. e other is one-stage algorithms such as Yolo and SSD [3][4][5], which use the convolutional neural network (CNN) to extract image features and establish detection models to classify and locate targets. After comparison, the second type of algorithm is faster, but the accuracy is lower. In practical applications, the target in the image often has problems such as being occluded and deformed, making the extracted target feature insignificant or lacking part of the feature information, leading to classification errors and tracking failures, and then prone to the problem of identity jump. erefore, we are comparing the advantages and disadvantages of existing algorithms; this paper adopts the generative confrontation network in the second type of algorithm. is model is a unique network structure that captures potential data distribution. e true and false confrontation error design based on game theory construction makes it possible to deal with different types of tasks [6,7].
Based on the idea of generative confrontation, this paper proposes a multimechanism adaptive target detection algorithm. First, through the organic combination of the punishment mechanism, feedback mechanism, and scoring mechanism, it effectively solves the problems of difficulty in training and model collapse of the generative confrontation network. en, use the relative discriminator structure to construct the discriminator to improve the probability of the discriminator in the process of fighting competition. e result is sent to the generator network through the feedback mechanism, and finally, the individual with the optimal value is obtained through the scoring mechanism, so as to quickly and effectively locate the key area of the image. After experiments, the experimental results are basically in line with expectations. e target detection algorithm can effectively locate and distinguish different objects and still maintain a high resolution for the occluded target, while maintaining a low number of missing alarms and false alarms.

Generative Adversarial Network (GAN)
e generative adversarial network (GAN) was proposed by Goodfellow. It uses two convolutional neural networks using game training to generate samples similar to the original picture [8][9][10].
By observing the structure of the generative confrontation network in Figure 1, GAN is mainly composed of two parts: generation network (G) and discriminant network (D). And, it uses a competing mechanism [11][12][13]; the purpose is to enable the generator to generate data similar to the real data distribution, which made the GAN achieve the effect of being fake. Aiming at the problem of data prediction, the generator is trained to learn the data distribution and predict the image distribution data. Generated data distribution and real data distribution are shown in Figure 2.
Because the difference between the generated data distribution P G and the real data distribution P data cannot be calculated, therefore, the discriminator uses data sampling to calculate the divergence of P G and P data . And, use these to discriminate the difference between the two data distributions, and the objective function of the discriminator is (1) Among them, E( * ) is mathematical expectation and D( * ) is the output result of the discriminator. e objective function is to maximize the discriminator, equivalent to the divergence generated data and real data. e optimal generator achieves the purpose of prediction by minimizing the divergence of the distribution between the real image data and the generated data. e optimal generator can be expressed as GAN can generate samples that are similar to the real data distribution, but it has problems such as difficulty in training, difficulty in network convergence, and mode collapse.

The Discriminator Network Introduces a Gradient Penalty Mechanism
In order to make the training of the model more stable and solve the problem of model collapse, this paper adds a gradient penalty to the discriminator objective function. After adding the gradient penalty, the objective function is Among them, λ gp is the weight parameter. P penalty is the probability distribution of the sample for calculating the gradient penalty. Sampling a point in P G and P data , respectively, the method is random sampling. ‖ * ‖ 2 is a twonorm number.
In the proposed data prediction algorithm, the loss function can reduce the difference in probability distribution between the predicted data and the real data so that the predicted data generated by the generator can more accurately fit the real data distribution. e loss function is defined as Adding a traditional loss function to the loss function of GAN can improve the accuracy of the data generated by the generator. e role of the generator is not only to generate data that can deceive the discriminator, but the generated data should be close enough to the real data. Since data prediction is a regression problem, adding the output result data of the discriminator to the loss function can punish the extreme error between the predicted value and the true value. erefore, the loss function after introducing the gradient penalty is where λ MSE is the weight parameter, and the loss function of rough research, it can be found that L MSE (G) is the loss function that is only extracted for the generator. Just considering the generator data to improve the accuracy of the algorithm is not enough. In GAN, the data generated by the discriminator corresponds to the generator has a guiding role. erefore, this paper adopts a feedback mechanism for the data output by the discriminator network based on the introduction of a gradient penalty mechanism in the generator. is improves the accuracy of the loss value L MSE (G).
e discriminator in this article is constructed with a relative discriminator structure to estimate the probability that the real image is more real than the fake image and then promote the discriminator to obtain better probability results in the process of fighting competition, thereby increasing the loss of the generator value L MSE (G) to strengthen the network parameter optimization of the generator. By collecting the real data and false data of the discriminator and estimating the corresponding sample expectations, the following function is obtained: Among them, E I real and E I fake are the sample expectations of real data and fake data, respectively. is paper uses the minimum loss function to calculate.
For the global discriminator, the ideal values of E I fake (D(I fake )) and E I fake (D(I real )) should be close to 0 and 1. e expected discrimination result is D(I real ) � 1 and D(I fake ) � 0. erefore, the regression targets of D RI (I real , I fake ) and D RI (I fake , I real ) are 1 and 0.
For the generator, its role is to confuse the authenticity judgment of the discriminator as much as possible, which is opposite to the return goal of the discriminator. e regression targets of D RI (I real , I fake ) and D RI (I fake , I real ) are 0 and 1. erefore, the adversarial function to obtain the global discriminator is defined as follows: In this paper, by setting the feature-preserving loss function, the confrontation function L MSE (G) of the global discriminator is input into the above loss function for feature error calculation; finally, the loss function is defined as follows: By observing Figures 3-5, it can be seen that, after the reconstruction of the generated network in this paper, the image is basically effectively repaired, and it has a good real consistency. Although the repaired part is somewhat unclear and may have subtle changes, but it basically does not affect the detection effect.

Generating Network Scoring Mechanism
However, for the targets in the video, they are basically in a state of movement and overlap. We need to quickly and effectively locate the key areas of the image [14,15]. erefore, based on the above formula, this paper proposes a scoring mechanism for the generative network in the generative confrontation network. Each iteration of the generator network is composed of three stages: mutation, evaluation, and selection so that the generator can quickly and accurately locate the key areas in the image.

Variation.
Mainly for the input random noise signal z, the gradient penalty loss function of this article is used in the new round of update process. e penalty loss function L(G, D) for the generating network gradient is calculated only for noise data, and finally, L MSE(G, D) is obtained and the network data is updated:  Figure 2: Generated data distribution P G and real data distribution P data .

Scientific Programming
Among them, λ gp is the weight parameter and P z is the distribution of noise data.

Evaluation.
is stage is the data evaluation stage. e network uses the above loss function and selects the best sample under the feedback of the discriminator, so as to learn the correct training sample distribution and improve the stability of model training. e evaluation mechanism is as follows.
is stage is quantified as an adaptability score, which mainly analyzes the quality of the samples generated by the network and the diversity of the generated samples. It evaluates the quality of the generated samples and the diversity of the generated samples; the adaptability score can be expressed as Among them, F q is the quality score of the generated sample and F d is the diversity score of the generated sample. It is used to measure whether the generator can fully disperse the generated samples, which can further avoid mode collapse. c is a super parameter that balances the proportion of the F q and F d .
Calculate the highest fitness score. And, compare it with the highest score on the network to get the global best score.

Selection.
Select the best individual according to the evaluation mechanism. Others with lower scores will gradually decay so that the generated network has a certain directionality, and the best value sample is selected. e structure of the article generation adversarial network is shown in Figure 6.

Simulation Experiment
is paper uses the ImageNet dataset and the MOT training dataset to test the algorithm. All experiments in this paper are carried out in the same experimental environment, and Scientific Programming the data obtained by the article algorithm are compared with traditional algorithms, YOLO-based target detection, Deep Sort, and literature algorithms [16] to verify the effectiveness of the algorithm.
Considering that there is a big difference between artificial occlusion and occlusion in the real world, this chapter directly simulates the actual dataset, and the result of target detection is shown in the figure below.
By observing the target detection effect map in Figure 7, we can see that the article algorithm can effectively locate and distinguish different objects and still maintain a high resolution for the occluded target, even if the characters in Figures 7(d) and 7(i) are occluded. More than half of the area can still accurately identify and mark the target.

Accuracy Test.
is paper uses the evaluation criteria in MOTChallenge to evaluate the performance of the proposed algorithm and compares the measured data with traditional target detection algorithms and literature algorithms. Among them, the MOTChallenge dataset has a comprehensive data type, which includes different viewing angles, camera movement methods, and pictures taken in different climates, which can effectively evaluate algorithms. Standards are shown in Table 1.
is article uses evaluation criteria: MOTA, MOTP, IDs, FP, and FN. By observing the identity jump problem of the algorithm in the target detection process. rough the test, we can obtain the data as shown in Table 2.
By observing the target tracking effect data comparison table in Table 2, we can see that the MOTA, IDs, and FP in this paper are better than other algorithms. In terms of MOTP data, the article algorithm is only 0.4% higher than the literature algorithm. It can be seen that the article algorithm can preserve better detection accuracy, while performing well in the number of false alarms (FP) and number of missed alarms (FN). It can effectively prove that the article algorithm can effectively reduce the problem of identity jumps.
By observing Figure 8, it can be seen that, under the same accuracy, the number of identity jumps of the algorithm in this paper is smaller than that of other algorithms. And, in terms of the highest accuracy, the article algorithm is about 2% higher than the literature algorithm. Observing Figure 9, one can clearly see that the algorithm in this paper is in the lower left corner of the image, and the number of missed alarms and false alarms are the lowest.

Conclusion
ere are problems such as target loss and identity jump in current target detection. In response to these phenomena, we propose a GAN-based multimechanism adaptive target detection algorithm. e article optimizes the construction model of GAN by introducing gradient penalty and feedback and scoring mechanism into the original GAN, which achieves an effective improvement in detection accuracy. Experimental results show that compared with other existing algorithms, the article algorithm has a more stable training model, which can effectively reduce the problem of target identity jump, while maintaining high detection accuracy. Although the method is effective, there are still areas that can be improved, such as the use of a more appropriate size convolution kernel to extract features, which will continue to be studied in future works.

Data Availability
e data used to support the findings of the study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper.