DOA Estimation in Non-Uniform Noise Based on Subspace Maximum Likelihood Using MPSO

: In general, the performance of a direction of arrival (DOA) estimator may decay under a non-uniform noise and low signal-to-noise ratio (SNR) environment. In this paper, a memetic particle swarm optimization (MPSO) algorithm combined with a noise variance estimator is proposed, in order to address this issue. The MPSO incorporates re-estimation of the noise variance and iterated local search algorithms into the particle swarm optimization (PSO) algorithm, resulting in higher e ﬃ ciency and a reduction in non-uniform noise e ﬀ ects under a low SNR. The MPSO procedure is as follows: PSO is initially utilized to evaluate the signal DOA using a subspace maximum-likelihood (SML) method. Next, the best position of the swarm to estimate the noise variance is determined and the iterated local search algorithm to reduce the non-uniform noise e ﬀ ect is built. The proposed method uses the SML criterion to rebuild the noise variance for the iterated local search algorithm, in order to reduce non-uniform noise e ﬀ ects. Simulation experiments conﬁrm that the DOA estimation methods are valid in a high SNR environment, but in a low SNR and non-uniform noise environment, the performance becomes poor because of the confusion between noise and signal sources. The proposed method incorporates the re-estimation of noise variance and an iterated local search algorithm in the PSO. This method is e ﬀ ectively improved by the ability to reduce estimation deviation in low SNR and non-uniform environments.

Introduction
Obtaining original signal-related information from signal sources containing interference is a very important issue [1,2]. The main sources of interference in the development of mobile communication technologies are low signal-to-noise ratio (SNR) and non-uniform noise. Array signal processing technologies have been applied to estimate the direction of arrival (DOA), using sensing elements arranged in different geometries to sample the wave field and collect spatial-related information to calculate the signal source DOA [3][4][5][6]. In wireless communications, low SNR and non-uniform noise are types of propagation phenomena, which can lead to misrecognition of the signal source and significant degradation of DOA estimation performance [7][8][9][10][11][12].
Among DOA estimation techniques, the maximum-likelihood (ML) [4,5,13] and multiple signal classification (MUSIC) [14] methods are the most representative. The ML algorithm assumes that the noise has a white Gaussian distribution and that the energy is uniform. The MUSIC method uses the autocorrelation matrix of the received signal to perform feature decomposition and decomposes its feature vector into signal and noise subspaces, utilizing the characteristics of orthogonality between the signal and noise to establish the DOA search criteria of the signal source. The performance of the ML and MUSIC methods is adversely affected by low SNR, however [12]. Therefore, Ji et al. [15] proposed the spatial MUSIC algorithm, while Zhang et al. [16] used the colony algorithm to solve procedure, in which the population is called a swarm. Each swarm consists of many particles and is updated based on the influence of individual experiences, the best past experience of each individual, and the overall best experience. The swarm characteristics of parallel multi-directional search are different from the general heuristic method. The advantage of PSO is that it is simple to solve and that it has the characteristics of parallel multi-directional search, which can quickly find the optimal method but is more likely to converge to a local optimum result and does not guarantee convergence to the global optimum, especially when the objective function has a high dimension or is a complex non-linear function [25,26].
To reduce the premature convergence of PSO and to obtain an adequate solution for DOA estimation under low SNR and non-uniform noise environments, this paper proposes the combination of the iterated local search algorithm and the PSO to construct a MPSO for solving the DOA under low SNR and non-uniform noise conditions. The proposed MPSO algorithm is simple and practicable, as it adopts a first-order Taylor series expansion of the target function using the SML criterion [9,27], in order to reduce the non-uniform noise effect, therefore increasing the capacity of PSO to find the best solution. The first-order Taylor series approximates the spatial search vector and cuts it down to a direct one-dimensional optimization [20]. Simulation results show that the proposed method has a considerably improved ability to decrease the estimation bias under non-uniform noise and a low SNR environment.
The remainder of the paper is structured as follows: Section 2 describes the SML DOA estimator. Section 3 presents the SML DOA estimator using MPSO. Section 4 presents numerical simulation results, illustrating the effect of the proposed method. The final section outlines our conclusions, referring to the proposed estimator.

SML DOA Estimator
Assume a P narrowband signal impinges on M (P < M) sensors in a uniform linear array (ULA) system. The tth measured snapshot of the received signal is written as [1]: where θ = θ 1 θ 2 · · · θ P T is the unknown DOA, the superscript T indicates transposition, N is the number of snapshots, n(t) = n 1 (t) n 2 (t) · · · n M (t) T is the sensor noise, T is the P × 1 vector of signal amplitudes, and A(θ) is the M × P composite steering matrix, expressed as where a(θ i ) is the steering vector, χ is the wavelength, and d is the sensor spacing between two neighboring sensors. In this paper, the sensor noise, n(t), is considered to be non-uniform and to be a zero mean Gaussian process, such that where E[·] is the expectation, the superscript H denotes the complex conjugate transpose, diag {.} is a diagonal matrix composed of the bracketed elements, and σ 2 i is the ith sensor's noise variance. In general, the sensor noise n(t) is uncorrelated with all signals. The array covariance corresponding to Equation (1) can be expressed as where is the covariance matrix of the signal amplitudes. The array covariance matrix can be estimated by the sample average,R: The ML estimator for non-uniform noise can be found using the weighted least-squares approach [10], using the normalized composite steering matrix and noise component. The maximum likelihood problem becomes a least-squares solution [10]: where σ 2 is the M × 1 vector of noise variance, |.| 2 denotes the l 2 norm, x(t) is the normalized receiving signal, x(t) = R −1/2 n x(t), and A(θ) is the normalized steering matrix, A(θ) = R −1/2 n A(θ). The A(θ)s(t) of Equation (6) is separable and, for fixed A(θ), s(t) can be obtained by using the pseudo-inverse [9]: Given Equation (7), substituting the ML estimator into Equation (6) results in where , I is the unit diagonal matrix, and tr{.} is the trace of the matrix. Equation (8) is a multi-objective minimization problem. In general, L(θ, σ 2 ) is a very highly non-linear function of θ and σ 2 , and the cost function is highly non-linear; it is impossible to represent the target function with any closed-form expression [16]. However, the ML estimator still has inferior performance when the noise is non-uniform and in a low SNR environment. It is well-known that the signal subspace projection method can weaken the noise effect of the received noisy data vector [9]. Applying eigende composition, the sample covariance matrix Equation (5) can be expressed aŝ where λ 1 ≥ λ 2 ≥ · · · ≥ λ M are the eigenvalues ofR and e m denotes the eigenvector associated with λ m for m = 1, 2, · · · , M. The column spans of E s = [e 1 , . . . , e P ] and E n = [e P+1 , . . . , e M ] are defined as the signal and noise subspaces, respectively. The covariance matrix can be expressed as the summation of two orthogonal components, E s E H s and E n E H n . Hence, this paper adopts the signal subspace projection E s E H s x(t), the projection of x(t) onto the columns of E s . Properly constructing the signal subspace projection-based approach to filter the non-uniform noise can effectively enhance the performance. The SML estimator using the ML estimator of Equation (8) can be expressed as: Selection of a signal subspace under a low SNR or non-uniform noise is very difficult when using eigendecomposition. As the received signal has low SNR or non-uniform noise, E s may contain a noise subspace and E n may contain a signal subspace. This paper adopts the reiterated procedure of a method to reduce such confusion. First, it is assumed that the noise variance is constant for all sensors (λ M ) and that the SML estimator in Equation (10) can be expressed as Next, using Equation (11) to obtain the DOA, the noise varianceR n can be estimated bŷ whereθ is the estimated DOA. This procedure implies that, for a fixedR n , the solution θ minimizes L(θ) in Equation (11), and vice versa. Once θ is obtained, a refined result forR n can be achieved using Equation (12). Hence,θ andR n can be estimated in an iterative manner. The steering matrix can be determined using a first-order Taylor series approximation expansion at the estimation DOAθ, expressed as [1]: where ηθ is a small value and A (θ) = d dθ A(θ) θ=θ . Substituting Equation (13) into Equation (11), the following expression can be obtained: which is a direct one-dimensional optimization problem. Then, it can easily be shown that the optimum ηθ is given by = 0. The value of |ηθ| has two characteristics [8]: first, if the value of |ηθ| is small, it can achieve a convergence result that may be local or global. However, if the value of |ηθ| is large, the results may remain far from convergence. Using these characteristics of |ηθ|, θ is updated by: where ε is the search precision value. In this paper, Equation (15) is repeated until |ηθ| < ε. The proposed procedure is as follows: 1.
Given the received signal x(t), computeR in Equation (5) and the eigendecomposition in Equation (9). 2.
Assume that the noise variance, which is constant for all sensors, is λ M . 3.
From the above, we propose a new method, which is a hybrid algorithm that combines PSO and the estimated DOA θ with an iterated local search algorithm for |ηθ|.

The Proposed Method
This paper proposed a hybrid algorithm that incorporates PSO with an iterated local search algorithm. The DOA with SML estimator criterion cannot be directly carried out under a non-uniform noise and low SNR environment, in which the closed form of the contained criteria are θ, σ 2 at the same time. Therefore, MPSO is introduced to solve the issue.
The MPSO includes the following two components: First, the PSO-based SML estimation method searches the entire space. Second, the local search using the property achieves a more accurate search around potential solutions of the first component. The design process of the MPSO is expressed below.

The PSO-Based SML Estimation
The PSO algorithm is an optimized search method based on a group that is easy to use, as the algorithm requires few parameters. The swarm of the PSO algorithm consists of many particles. Each individual particle represents a solution; it has its own position and velocity, the initial values of which are set randomly. Then, each particle obtains a value measure from a target function; the changing of the particle position is regulated by the value of the objective function. Our objective is to minimize the value of the fitness function. We use the fitness function L(θ) presented in Equation (11). There are three kinds of influences on the movement of particles. First, movement in the previous direction; second, movement towards the position of the individual particle's optimization situation; and, finally, movement towards the position of the global optimization situation of the overall swarm [21].
T be the current velocity, be the current position. During each iteration, the update to the velocity v i,j (t + 1) and position θ i (t + 1) of each particle is as follows: f or all i = 1, 2, · · · , S, j = 1, 2, · · · , P where v i,j (t) is the velocity of the jth dimension of the ith particle, κ is the inertia weight (this value is typically set as 0 ≤ κ < 1), c 1 and c 2 are set near 2.0 [27], ϕ 1,i (t) and ϕ 2,i (t) are uniformly distributed random numbers in the range [0,1] p i,j is the individual best position of the jth dimension of the ith Processes 2020, 8, 1429 6 of 12 particle, and g j is the global best position of the jth dimension. The individual best position of each particle is updated using where p i = p i,1 p i,2 · · · p i,P T . The overall best position can be found as follows: where g = g 1 g 2 · · · g P T .
The value of the velocity v i,j (t) can be limited to the range [−v max , v max ], in order to reduce the number of particles escaping the search space with an uncontrollable trajectory [22]. In this paper, we use v max = 0.1 × θ max = 18 • for the ULA system [8].

The MPSO Estimator
The MPSO proposed in this paper combines the application of a local search method into the PSO algorithm, in order to solve the problem that the SML estimator criterion cannot obtain a closed form solution. Using the PSO to estimateθ in Equation (12), the characters of |ηθ| in the local search method and the re-estimatedR n are combined. A small |ηθ| value will generate convergent results, while a large |ηθ| value requires a greater time to converge and may cause a value that changes the estimated deviation towards a more correct value. The pseudocode of the Algorithm 1 MPSO is as follows:
Step 4: Set p i ← θ i (t) and g = min{p i }.
Step 13: Go to Step 5 until the stopping criterion is satisfied.
Based on the above analysis, Figure 1 presents a flowchart of the proposed algorithm.

Simulation Result
Two examples were considered in order to illustrate the practicability of using the proposed algorithm for DOA estimation in a non-uniform noise and low SNR environment. Simulation results were used to compare the performance of the proposed algorithm with the MUSIC [14], minimum variance distortionless response (MVDR) [1], and power domain ML (Power-Domain) methods [10]. The non-uniform noise (using the worst noise power ratio; WNPR) and SNR were defined using:

Simulation Result
Two examples were considered in order to illustrate the practicability of using the proposed algorithm for DOA estimation in a non-uniform noise and low SNR environment. Simulation results were used to compare the performance of the proposed algorithm with the MUSIC [14], minimum variance distortionless response (MVDR) [1], and power domain ML (Power-Domain) methods [10]. The non-uniform noise (using the worst noise power ratio; WNPR) and SNR were defined using: where σ 2 max and σ 2 min are the maximum and minimum non-uniform noise variances, respectively. The simulated results were obtained by averaging 500 independent Monte Carlo (MC) runs. In the literature, a variety of statistical methods have been applied to compare performance. These include the mean absolute error (MAE) and the root-mean-squared error (RMSE), which are defined as whereθ j,p is the jth MC run for the θ p estimate value. The search grid capacity for the spectrum scan of MUSIC [14] was set as 0.001 • . The initial set of parameters of the proposed estimators were defined as where c 1 , c 2 are acceleration coefficients, χ is the inertial weight, S is the size of the swarm, T is the number of iterations, and ε is the search precision value.
The first example considered a four-element ULA with half-wavelength inter-element spacing, where the noise power was given as σ 2 = 5 10 0.1 6 t , WNPR = 50, and the source had the DOA θ 1 = 5 • . Figure 2 illustrates the RMSE values of the estimated DOA versus the number of snapshots under SNR = −15 dB. The proposed method achieved a faster convergence approach with 75 snapshots, while the other methods converged at about 150 snapshots. Figure 3 shows that the RMSE of the various estimators versus different SNRs under snapshots was 100. In Figure 3, we can see that all estimator RMSEs were near zero under high SNR, but the performance of other estimators decayed under low SNR conditions. Table 1 shows the RMSE and MAE under various SNRs. The bold text is the best estimated value under the same SNR. The proposed algorithm had RMSE and MAE values smaller than those of the other estimators, especially under low SNR environments.    The second example considered an eight-element ULA and two sources with DOAs θ =  Figure 4 indicates the RMSE of the various estimators versus different snapshots under SNR = 0 dB. Again, this figure indicates that the proposed estimator not only carried out faster convergence (at 50 snapshots) but also offered an improvement in DOA evaluation accuracy. Figure 5 indicates that the RMSE of the various estimators versus different SNRs under snapshots was 100. In Figure 5, we can see that the other estimators were not sensitive to various low SNRs (SNR < −5 dB), when comparing their performance with the proposed method. As hoped, the results indicate again that the DOA evaluation accuracy was improved by the proposed estimator. Table 2

Conclusions
DOA estimation cannot be directly carried out under a non-uniform noise and low SNR environment, in which the closed form of the contained criteria is   2 θ, σ . Generally, it is necessary to process the DOA ( θ ) and the noise variance ( 2 σ ). In this paper, a new re-iterated process was

Conclusions
DOA estimation cannot be directly carried out under a non-uniform noise and low SNR environment, in which the closed form of the contained criteria is θ, σ 2 . Generally, it is necessary to process the DOA (θ) and the noise variance (σ 2 ). In this paper, a new re-iterated process was introduced, in which the noise variance is fixed to estimate the DOA and, vice versa, the DOA is fixed to estimate the noise variance. After several iterations, the procedure converges to the nearest correct estimates of the DOA and the noise variance. The proposed solution combines the MPSO scheme, which uses the fixed noise variance to estimate the DOA through the PSO algorithm, using the best particle to estimate the noise variance. An MPSO that incorporates the re-estimation of noise variance and an iterated local search algorithm is applied in the PSO, resulting in an efficient reduction of the non-uniform noise effect under a low SNR. The iterated local search of the MPSO method exploits the characteristics of the first-order Taylor expansion |ηθ|. A small |ηθ| value can guarantee convergent results that may be local or global while, with a large |ηθ| value, convergence will take a longer time and may provide a value that updates the estimated deviation toward the correct value. Empirical evidence shows that the DOA estimation methods are valid in a high SNR environment, but in a low SNR and non-uniform noise environment, the performance becomes poor because the noise is confused by the source of the signal. The proposed method incorporates the re-estimation of noise variance and an iterated local search algorithm in the PSO. This method is effectively able to reduce estimation deviation and to achieve high accuracy and fast convergence in low SNR and non-uniform environments. Generally, a low SNR and non-uniform environment is caused by foul weather. This problem occurs in mountaineering, so this method provides valid DOA estimation in this environment, and can be used for positioning system issues.


Integrity verification of ADS-B navigation data source

The importance of data source integrity to ADS-B monitoring performance is analyzed in this paper, and the corresponding monitoring algorithm is given. At present, ADS-B monitoring data mainly comes from GNSS, and GPS, as the main component of GNSS at this stage, its public data is relatively easy to obtain. Therefore, GPS is selected as ADS-B data source, the integrity of the navigation data source is analyzed and the data of an airport is verified by simulation in this paper.

Introduction
ADS-B can provide more accurate and real-time aircraft position monitoring information than radar monitoring system. ADS-B horizontal position data comes from the navigation system, and the airborne navigation source directly determines the monitoring performance of ads-b. In order to characterize the quality characteristics of aircraft horizontal position, ADS-B system evaluates the accuracy and integrity of relevant position information. Ground equipment can monitor the accuracy and integrity of navigation data of each aerial device. When an aircraft gives an alarm on the accuracy or integrity of the navigation data source, the air traffic controller is required to increase the separation between the aircraft and others; if more aircrafts give alarm, the separation between all aircraft shall be increased. Therefore, the integrity of position information provides guarantee for the reliability of monitoring data and the safety of aircraft. As a new monitoring technology, ADS-B is still in the experimental stage in China, and the research on the integrity of navigation source is less. Therefore, it is necessary to study the ADS-B navigation data source and its integrity.

Integrity analysis
2.1Integrity requirements ADS-B navigation source availability is defined as the percentage of time that the system can be used for navigation in a certain period of time, which is based on providing reliable information to users. It is required to meet the requirements of accuracy and integrity at the same time. Integrity is judged by whether the protection level of user positioning error exceeds the standard. The integrity information is encoded by NIC (navigation integrity category) and SIL (source integrity level). In the actual monitoring process, the ground staff will judge the NIC and SIL of the received ADS-B message and analyze whether they meet the minimum monitoring requirements [1]. Rc<7.5m and VPL<11m SIL (survey integrity level) is determined by the navigation data source [2]. It is defined as the probability of missed alarm when the requirements of NIC are not met. The coding rules are shown in table 2.

2.2Integrity analysis based on RAIM algorithm
At present, there are three methods for GPS integrity monitoring: RAIM(Receiver Autonomous Integrity Monitoring ) , SBAS ( GPS/Space-Based Augmentation System ) and GBAS (GPS/Ground-Based Augmentation System). RAIM is a mature method in theory and practice. It is a kind of integrity monitoring technology which only uses the pseudo range measurement information of navigation satellite. RAIM algorithm is usually embedded in the receiver, which can make the receiver execute RAIM algorithm while receiving GPS measurement signal for positioning. The basic principle of algorithm is to detect the consistency between the measured values of each satellite, so redundant observation information is needed [3]. If the GPS receiver can receive N visible satellite signals at a certain time, i.e. generate N observations, then the GPS measurement equation [4] is:  y = Hx + (1) Among them, y is 1 N  dimensional observation vector and x is 4 1  position state vector to be measured, including the real three-dimensional position coordinates of the user receiver itself and the clock deviation of the receiver;  is The calculated pseudo range vectorŷ is compared with the actual one, and the pseudo range residual vector w is obtained. The mathematical derivation relationship is as follows: Matrix S is defined as: The sum of squares F of the components of pseudo range residuals is: When there is no fault, the degree of freedom of the normalized variability If there is a detection alarm, it is a false alarm. If FA P is the probability of false alarm, then: The VPL expression is as follows: After the above methods, the c R and VPL have been calculated, then the NIC can be determined.

Integrity simulation verification
The data of an airport in China is shown in Table 3. According to the data, simulation verification is carried out. Determine the position of the simulation point in the ECEF coordinate system, and convert the longitude and latitude points in the WGS-84 coordinate system of the airport point into the ECEF coordinate system [5][6](In the ECEF coordinate system used in GPS, the XOY plane coincides with the equatorial plane of the earth, where the X axis points to the longitude 0  direction and the Y axis points to the east longitude 90  direction).


Satellite position prediction, According to the United States website [7], the calendar parameters are updated every Saturday at 24:00, i.e. a new issue of the calendar is released. Reference provides the calculation method of GPS satellite position, and calculates the position coordinates of a satellite in the future t time. At present, there are 32 satellites in orbit in the GPS space segment, numbered from PRN01 to PRN32. The GPS almanac data of this period provides the orbit parameters of 30 satellites.
 Establishment of station-center coordinate system with current observation point. The three coordinate axes are perpendicular to each other and meet the right-hand rule. They point to the East, North and zenith directions respectively. In the station-center coordinate system, the position information of satellite is expressed by the elevation, azimuth and distance of satellite relative to aircraft.
 Without considering the terrain sheltering angle, the visible satellites are screened according to the sheltering angle of 5 °.
 The observation matrix H is established according to the selected visible satellites.  Calculate HDOP and c R as above.
The results of this simulation are shown in Figure 1.  Figure 1. c R Schematic diagram of change over time

Concluding remarks
Integrity is very important for satellite navigation, if the aircraft is sailing at high speed, it may deviate from the route in a short time if the integrity is unknown. Because GPS is the most commonly used navigation source at present, and if it is used as the main navigation system, the integrity function becomes more important. The results show that the integrity category of the airport meets the requirements.


IITP-AI-NLP-ML@ CL-SciSumm 2020, CL-LaySumm 2020, LongSumm 2020

The publication rate of scientific literature increases rapidly, which poses a challenge for researchers to keep themselves updated with new state-of-the-art. Scientific document summarization solves this problem by summarizing the essential fact and findings of the document. In the current paper, we present the participation of IITP-AI-NLP-ML team in three shared tasks, namely, CL-SciSumm 2020, LaySumm 2020, LongSumm 2020, which aims to generate medium, lay, and long summaries of the scientific articles, respectively. To solve CL-SciSumm 2020 and LongSumm 2020 tasks, three well-known clustering techniques are used, and then various sentence scoring functions, including textual entailment, are used to extract the sentences from each cluster for a summary generation. For LaySumm 2020, an encoder-decoder based deep learning model has been utilized. Performances of our developed systems are evaluated in terms of ROUGE measures on the associated datasets with the shared task.

Introduction
Massive amounts of scientific articles are published day by day (Cohan et al., 2015;Goharian, 2017, 2018), which impose a big challenge for researchers in various fields to keep themselves upto-date with the new developments. A bibliometric analyst's study shows that after nine years, the number of published articles will be doubled (Bornmann and Mutz, 2015). The scientific document summarization objective is to provide a summary of the reference paper. This summary should contain all the important facts. Therefore, it reduces the human effort to understand the document.
Challenges of each style of the summary are as follows: • Objective of CL-SciSumm is to generate a short summary of a paper which must contain all relevant facts and findings. To solve this problem, we have used the extractive summarization technique. We have used unsupervised techniques and explored different features for scientific summarization. These used features help in identifying important sentences of the article using different aspects.
• Objective of LongSumm is to generate a long summary of the scientific article that should be extractive and abstractive. The generated long summary must contain all important facts of the article. To solve the extractive long summarization problem, we have an unsupervised technique similar to CL-SciSumm. To solve the abstractive LongSumm problem, we have used the encoder-decoder based generative model.
• Objective of CL-LaySumm is to generate a lay summary that can be understood by a non-technical reader. The generated summary should not contain any technical words or jargon. We have solved this task using the abstractive summarization technique. Here, Fine-tuned BERT based encoder-decoder architecture is used to solve the problem.
The current paper addresses this issue by participating in the three shared tasks, namely, CL-SciSumm 2020, LaySumm 2020, LongSumm 2020 (Chandrasekaran et al., 2020). These tasks' goals are to generate medium, Lay (understandable for the non-technical audience), and long summaries of the scientific articles. We are using an extractive approach for CL-SciSumm, For LongSumm, an extractive followed by an abstractive approach is utilized, and for LaySumm, an abstractive approach is utilized. The detailed descriptions of these tasks are provided in the subsequent sections.

CL-SciSumm 2020
It is the sixth shared task on scientific document summarization. In the literature, two approaches have been used to solve this problem. The first one considers abstract as the summary of the paper, but the problem with the approach is it provides only the theme of the paper. The abstract may not convey all the important points of the summary (Yasunaga et al., 2019;Atanassova et al., 2016). Therefore, the second approach has been followed to solve the scientific summarization, which is citation-based summarization (Qazvinian et al., 2013). It utilizes a set of citations referencing to the original article (reference paper to be summarized). Citations are short descriptions that explain the reference paper all its contributions; this text can be termed as citation text or citance.

Dataset
The dataset contains a blind test set of 20 papers and corresponding citing papers. Each paper belongs to the computation linguistics and natural language processing domain. It can be found at https: //github.com/WING-NUS/scisumm-corpus/ tree/master/data/Test-Set-2018.
Training data is also provided. But, as our approach is purely unsupervised; therefore, we are making use of only test data.

Tasks Descriptions
• Task-1 (A) In this task, the objective is to identify the spans of text (cited text spans) in the reference paper (RP) for each citance given the RP and citing papers (CPs).
• Task-1 (B) is to classify each cited text span into facets that are predefined (Hypothesis, Aim, Method, Results, and Implication).
• Task-2 is to produce a summary of the reference paper by utilizing its citation. The generated summary length should be less than or equal to 250 words.

System Description
In this section, the steps followed in our proposed framework for solving different sub-tasks are elaborated.
Task 1 (A) To find out the cited text span in the reference paper for each citance, we have utilized the word mover's distance (Kusner et al., 2015).
For each citation sentence, WMD is used to identify the most similar sentences from the reference paper. WMD denotes the semantic similarity between sentences. Here we have selected the top five most similar sentences from the reference paper.
Task 1 (B) To classify each cited text span, we have calculated the similarity between the cited text span and all the five facets using word mover's distance (WMD). The cited text span is assigned that facet, which is closest in terms of WMD.
Task 2 To generate a structured summary, we have used the unsupervised technique, i.e., clustering, followed by the sentence extraction from each cluster based on various sentence-scoring functions. The sentence having a high score from each cluster is included in the summary until the desired length of the summary is reached. A series of steps followed are as follows: 1. Grouping of the sentences has been done using the traditional clustering techniques, namely, K-means (Lloyd, 1982), K-medoid (Kaufman et al., 1987), and DB-scan (Ester et al., 1996).
2. We have determined the document center/representative sentence (RS) of the reference paper. It is that sentence in the article which is most similar to the remaining sentences. We can also call it as an article's center. In other words, the sentence having the minimum average WMD with respect to other sentences is called the RS.
3. Clusters are ranked based on their distances from the representative sentence (RS). In other words, the cluster closest to the RS is assigned the highest rank.
4. After ranking the clusters, we have calculated the scores of the sentences within each cluster based on the following features and then selected the highest scored sentence from each cluster considering their rankings. Note that the selection of sentences from the ranked clusters (in a sequence) will continue until we get the desired length of the summary.
Position of the Sentence (F1): In the literature, it has been shown that important sentences are found in the title and lead sentences of a paragraph. It is expressed as follows m i = 1 n i where n i is the position of a sentence in the reference paper. The sentence is given the highest priority, which lies at the start of the document (Saini et al., 2019).
Similarity with the title (F2): In any document, the sentence, which is very much similar to the title of the document, can be an important sentence for the summary (Saini et al., 2019) as it represents the theme of the article. Here word mover's distance is used to find the similarity.
Length of the Sentence (F3): In the previous works, it is shown that longer sentences can be relevant for generating a summary for a document describing some news (Mendoza et al., 2014;Saini et al., 2019). The sentence is assigned the highest priority, which has the longest length.
Textual Entailment (F4): Textual entailment (Saini et al., 2020) has been used as an antiredundancy measure. In a good summary, sentences should not be related to each other to have more coverage. Here, initially, the cluster centers are included in the summary following the ranked order of the clusters. In the next step, those sentences are selected from the ranked clusters and included in the summary, which does not entail any sentence in summary.

Submitted Run
Details of the submitted systems are provided in Table 1. Here we have used five clusters for K-means and K-medoid, whereas DB-scan decides the number of clusters automatically. In Table 1 each run describes the features used for the selection of sentences within clusters to form the summary. Here, twelve different runs have been used for task-2.

Result
Results of task-1 (a), task-1 (b) and task-2 are shown in Table 2, Table 3 and Table 4 respectively.

CL-LaySumm 2020
The motivation of the CL-LaySumm Shared Task is to automatically produce Lay Summaries of technical (scientific research article) texts. A Lay Summary is defined as a textual summary easily understood by a non-technical audience. It is typically produced either by the authors or by a journalist or commentator. The corpus released in shared tasks covers three distinct domains: epilepsy, archeology, and materials engineering. In a lay summary, there should not be any technical jargon. It should reflect the overall scope, goal, and potential impact of a scientific paper. It is typically less than 150 words in length. The objective is to generate summaries that represent the content, understandable and interesting to a lay audience.

Dataset
The dataset has a training set of 572 articles having corresponding lay summaries. It contains a blind test set of 37 papers.

System Description
Neural network based approach formulates abstractive summarization problem as sequence to sequence problem, here encoder is used to read the token of source documents x = [x 1 , x 2 , ...... We have used standard encoder-decoder architecture for our lay summarization task. Here the encoder is pre-trained BERTSUM, and it is finetuned on CNN daily mail dataset, and the decoder is a six-layer transformer network (as shown in Fig  1 ). It should be noted that there is a difference between encoder and decoder as the encoder is pretrained while the decoder has to be trained. This can create an unstable process of fine-tuning, due to which encoder and decoder can have the problem of under-fitting and over-fitting. To resolve this problem, the different optimizers for encoder and decoder have been used.

Precision
Recall F1 score micro avg macro avg micro avg macro avg micro avg macro avg 0.0222 0.0221 0.1049 0.1058 0.0367 0.0365

Runs
Human  Table 4: Task-2 scores of different runs in terms of rouge scores. Here, R-2 and R-SU4 are denoting rouge-1, rouge-SU4 respectively. All the reported values are f1 scores. l rD =l rD .min(step 0.5 , step.warmup −1.5 D ) (2) where l r = 2e − 3 and warm-up = 20000 for the encoder whereas l rD = 0.1 and warm-up =10000 for decoder. Here the assumption is that the pretrained encoder must be trained with a lower learning rate and a lower learning rate smoothens the decay. This process helps the encoder in training with a better gradient when the decoder is in stable condition.
We have used a two-stage fine-tuning approach, first is fine-tuning for extractive summarization and then for abstractive summarization. It has been shown in the literature (Li et al., 2018) (Gehrmann et al., 2018) extractive object helps in obtaining a better abstractive summary.

Result
Our system has the following score (shown in Table  5).

LongSumm 2020
In all the previous works of scientific summarization Goharian, 2017, 2018), there is a summary length constraint of a maximum of 250 words. But in the current LongSumm shared task, the generated summary can be the length of between 100-1500 words.

Dataset
This dataset consists of a training set of 1705 papers associated with extractive summaries and 531 papers associated with abstractive summaries. It has a blind test set of 22 files. It can be found at https://github.com/guyfe/LongSumm.

System Description
We have used both extractive and abstractive approaches on the blind test set to generate a structured summary. We have used clustering followed by the sentence-scoring and extraction procedure within each cluster based on various features. The sentence having the highest score is included in the  Figure 1: Architecture for Lay Summarization task R-1 (f) R-2 (r) R-2 (f) R-2 (r) R-l (f) R-l (r) 0.3132 0.3705 0.0631 0.0746 0.1662 0.1973 Table 5: Obtained scores on blind set of CL-LaySumm, Here, R-1, R-2, and R-l are denoting rouge-1, rouge-2, and rouge-l, respectively, f and r representing f-1 score and recall, respectively. summary. Note that for LongSumm extractive summarization, we have used the same methodology as used for CL-SciSumm 2020. A deep learningbased encoder-decoder model is used for Long-Summ abstractive summarization.

Submitted Run:
Details of submitted systems are provided in Table  6. Here we have used five clusters for K-means and K-medoid, whereas DB-scan decides the number of clusters automatically. In Table 6 each run describes the features used for the selection of sentences within the cluster to form the summary. F1  run1  run2  run3  F2  run4  run5  run6  F3  run7  run8  run9  F4 run10 run11 run12 We have used deep learning-based technique as run13, which is as follows: run13: Proposed model has utilized encoderdecoder based deep learning model. Fine-tuned BERT model has been used for the generation of embedding. We have used same model as used for LaySumm.

Result
The results of all runs are shown in Table 7, here from run-1 are run-12 are the scores of long summarization using an extractive approach, whereas run-13 is the score of long summarization using the abstractive approach.
to extract the cited text span from the reference paper; for task 1 (B), the similarity-based measure has been used to identify the facet of each cited text span. Task 2 is based on clustering, followed by sentence extraction from each cluster based on their relevance/score. For LongSumm, we have utilized clustering and deep learning techniques and reported 13 different ways to generate a long summary. For LaySumm, we have proposed a deep learning-based encoder-decoder model that generates the lay summary utilizing the fine-tuned BERT language model's embedding.


Automated Fundoscopy for Glaucoma Detection and Classifiction

glaucoma is leading chronic eye dieses in the world that leads to vision lost. The main cause of Glaucoma is intrinsic deterioration of the optic nerve which leads high intraocular pressure of the eye. Manually detection of glaucoma is tedious and costly. In our work we are providing automated system for glaucoma detection which is based on fully connected conditional random filed (FC-CRF) model, it works on long and thin structure. Conditional random filed provide a platform for structure prediction. Taking benefit of current results, validating assumption and parameters of our system learned automatically with the help of structured output support vector machine. Our system trained both quantitatively and qualitatively on publically existing data sets: DRIVE, STARE, CHASEDB1 and HRF. Once we obtain segmentation results further classification is done by SVM and K-NN classifier results of our proposed system is analyzed with gold standard labeling provided each data sets in terms of TP,TN,FP and FN. importance of our proposed system is it works for enlarge structure which can provide a platform to other biomedical and biological applications.

INTRODUCTION
Retina is a part of central nervous system resides in peripheral location represents the neural portion of human eye. Detecting macular degeneration (ARMD) which may result total vision lost or blurred vision, diabetic Eye or diabetic Retinopathy(DR) occurs due to diabetic mellitus, and retinopathy of prematurity(ROP) occurs due to unwanted growth of retinal vessels, segmentation of blood vessels contains elementary role in diagnosis of eye syndromes. Segmentation is an important step in visualization of programs for primary detection of eye syndrome. The development of Automated detection system is beneficial it simply means the system combine all screening programs in a single system to provide accurate results. The idea behind developing automated system is ophthalmologist unable to detect exact results in a single screening hence it is not feasible to get results in time [1]. These systems usually perform morphological operations on retinal structure of blood vessels this immense information will help us for further examination and treatment [1]. For diabetic retinopathy early detection plays a vital role to avoid vision lost [2]. Segmentation process plays important part for the analysis of retinal vessels. Earlier this process perform manually by experts but it is tedious due to low contrast nature of fundus images ,uneven width of vessels, shape which affects analyst to get accurate results hence it is time consuming and ambiguous [3].These facts encourage us for building an automated fundoscopic system for blood vessel segmentation without manual efforts [1].Many research persons working in this field of automated detection and classification of vessel segmentation but still it is open for all to get accurate results. Generally for classification of presented approaches, there are number of classifiers, classification techniques contain two types, supervised and unsupervised. To learn a model Supervised techniques need trained samples. In this case these samples are nothing but picture cells and their features with analysis. We studied several classifiers in literature survey they are k-NN [4], Bayesian [5], SVM [6]- [7], NN [8]- [9], decision tree [10]- [11], Gaussian Mixture Model [3], Adaptive boosting [12]. A recently introduced B-COSFIRE trainable filter highlights the retinal vasculature [13]. Even if the method is unsupervised in term of training a classifier, it adopts parameters which are mainly based on trained data. In unsupervised learning method system doesn't require labeled classes and annotations; hence this technique had lower accuracy, unsupervised learning technique involved thresholding, vessel tracking [14] region growing [15]- [17], active contours [18]- [19] morphological operations performed this task [20], matched filter responses [21]- [22], wavelet transform [23]- [24]. This study purposes supervised methods. Conditional random fields (CRFs) are discriminative statistical modeling technique means it is has been used when class levels are dependant. Earlier segmentation using fully connected CRFs is applied to detect tumor of liver and brain through CAT and MRI scanning. [25]- [26].In our work we are working on thin retinal two dimensional structures. Local classification unable to classify while incorporates previous information regarding contours of the preferred patterns for learning stage. CRFs obtain this information using pairwise potential. SOSVM used for learning phase prior to learn local neighborhood based CRFs [27]- [28].Here we use fully connected CRFs for learning by skipping SOSVM, as its computational complexity it requires multiple calls during training algorithm on other side dense CRFs slow in nature [29].In this study, we carry Out our prior effort with more advanced research information and performance throughput. We also adapt the policy of estimation and set new parameters to improve performance of training phase.

Revised
Additionally, we validate our results, quantitatively and qualitatively on freely existing standard datasets (DRIVE, STARE, CHASEDB1 and HRF).Our system performs on multiple quality measures to obtain accurate results.

II. METHODOLOGY
In our proposed work we are providing automated fundoscopic system for glaucoma detection by discriminatively fully connected conditional random filed (FC-CRF) model. It can graphically plot retinal images, it means we used pixel as node having connection with an edge to its very closest neighbors. Using this technique we are able to get detailed information of neighbor pixels as well as interface between elongated pixels. Using this technique we obtain exact segmentation result but it is time consuming because it is computationally complex at the training phase. This problem is overcome by defining pairwise edge potential in the form of linear combination of Gaussian kernels [30], it means it uses approximate mean arena of the primary CRF, capable towards exact segmentations within fraction of second.

 FC-CRF Segmentation of retinal vessels
To understand this technique we are having certain mathematical formulations as we know here we are going to apply energy minimization on conditional random field. This process is carried out using features of unary and pair wise potentials. Simply unary potential encode local information of pixel and pair wise potential assign labels to two neighbor pixels in our case it define inference.

E(y) is energy function
Ψ u is unary energy.
Ψ p is pairwise energy. By adding equation (2) and (3) we get energy function. Where Wu yi weighted vector β yi biased function related with label y i.. After getting energy function we need pixels those differs form mean value of the group to obtain this we use Gaussian kernel which is mathematically depicted as follows.
Where k (m) is kernel f (m) is random feature ,W p (m) group of linear weight and μ(y i ,y j ) label compatible function.

A. Learning of Structured support SVM through conditional random field
It is a machine learning algorithm mostly used in binary classification.SVM works on linearly separable points (features), means it contain a hyperplane that separate the points (features) in the single category. In our case this condition doesn't satisfied because we are having multiple features, to overcome this we apply 1-slack formulation (slack variables are responsible for measuring distance between point to its marginal hyperplane.).Using this technique we learn vector W having different unary features with respect to bias and pairwise kernels. By using mathematical formulation we calculate weights W.
Where C can defined as regularization constant.

B. Classification
In this work we are having two types of classifiers SVM and k-NN for classification.  SVM In our work we can define support vector machine as a support vector classifier. Here kernels (similarity quantifier) extend the feature space. It is supervised learning technique classification and regression analysis is done by analyzed data. In our case we need non-linear classification to deal high dimensions for this purpose we tune the following parameters. a) Kernel parameter:-responsible for linear or non-linear separation. b) Regularization parameter:-Responsible for svm optimization in training phase by measuring the quantity of misclassifying avoided points. c) Gamma parameter:-defines the low (far), high (close) influence in training phase. d) Margin parameter:-line separation for high class points.

 k-NN
It is distribution free means depends on observations means here we don't know target variable. Related identified neighbors are defined by number K .we get closest training samples using K factor, when k=1 we call it nearest neighbor algorithm. a) Implementation of k-NN model Step1: data loading.
Step4: distance calculation between test and training data.
Step5: ascending order sorting of calculated distance values.
Step6: find recurrent class of k rows.
Step7: return the predicted class.

Fig.1.(a)input image (b)gray scale image (c)complemented Eigen image (d)complemented wavelet output (e)complemented local output (f)background normalize image (g)vessel segmentation result (h)svm
and k-nn classification result. In our system we select one image from CHASEDB1 database our system converts that image into grayscale image then after for image enhancement our system apply Eigen vector .after that system can take mean to remove noise from a image by subtracting background of enhanced image using filters in the last system apply wavelet transformation using canny edge transformation to get normalization of image finally our system obtain segmentation using FC-CRF and using SVM and K-NN features we get results if any injury or daises occurs we get bulged pattern of nerves. To better understand our experimental results we have to go through operational flow diagram of our system which is shown in figure 2. , S e : defines sensitivity S p : defines specificity P r :defines precision respectively Acc: defines accuracy. We can also use several other performance measurements like G-mean, F1-score and MCC (Mathew's correlation coefficient).   Fig.3.performance graph of classifiers

IV. CONCLUSION
This study proposes a blood vessel segmentation system to detect different eye daises our system performs segmentation and classification for segmentation we use FC-CRF model the advantage of FC-CRF is we get retinal vasculature accurately then unary potential .we use two classifiers K-NN and SVM for best results. The benefit of our system is it works under dense potential it also benefit other biomedical and biological applications.


Class Distribution Alignment for Adversarial Domain Adaptation

Most existing unsupervised domain adaptation methods mainly focused on aligning the marginal distributions of samples between the source and target domains. This setting does not sufficiently consider the class distribution information between the two domains, which could adversely affect the reduction of domain gap. To address this issue, we propose a novel approach called Conditional ADversarial Image Translation (CADIT) to explicitly align the class distributions given samples between the two domains. It integrates a discriminative structure-preserving loss and a joint adversarial generation loss. The former effectively prevents undesired label-flipping during the whole process of image translation, while the latter maintains the joint distribution alignment of images and labels. Furthermore, our approach enforces the classification consistence of target domain images before and after adaptation to aid the classifier training in both domains. Extensive experiments were conducted on multiple benchmark datasets including Digits, Faces, Scenes and Office31, showing that our approach achieved superior classification in the target domain when compared to the state-of-the-art methods. Also, both qualitative and quantitative results well supported our motivation that aligning the class distributions can indeed improve domain adaptation.

I. INTRODUCTION
The excellent performance of deep learning crucially depends on the availability of a large amount of labeled data. However, this requirement cannot be well met in many real scenarios, and the absence of labels in a target domain makes classification difficult. Leveraging labeled data from auxiliary source domains has been shown as a promising way to improve this situation. Nevertheless, the inevitable discrepancy of data distributions between the source and target domains poses new challenges. This has made unsupervised domain adaptation (UDA) an active research topic in artificial intelligence in the recent years.
Previous deep domain adaptation methods [1], [2], [3] learn transferable feature representation across different domains. It embeds domain adaptation modules into several high layers of a CNN network to narrow the distribution gap. Recently, inspired by the powerful generative adversarial network (GAN) [4], a few adversarial domain adaptation methods have been Considering both of them, our method will be able to better align the class distributions given samples (i.e., P (y|x) and Q(y|x)), and thus helps to improve classification in the target domain.
developed [5], [6], [7]. They leveraged an adversarial setting to learn a domain discriminator (to distinguish from which domain a latent feature representation was derived) and domaininvariant feature generator(s) (to confuse the discriminator). However, these methods usually transform features rather than images, which prevent them from producing interpretable and visualizable transformation that is often highly desirable in practical applications. This paper follows a special yet important setting in adversarial domain adaption: adversarial image translation between different domains, which leverages GAN to generate sourcelike or target-like images for domain adaptation. This approach can effectively deal with the changes in appearance or texture between domains, i.e., style transfer. Several methods along this line [8], [9], [10], [11], [12] have been proposed in the recent years, well demonstrating their efficacy. Usually, these methods take a fundamental assumption that when the images from different domains conform a same marginal distribution after domain adaptation, their joint distribution with labels can be expected to be aligned at the same time. Nevertheless, this assumption could be too strong to hold in many crossdomain scenarios. Once this happens, these methods will suffer from ONLY aligning the marginal distributions and deliver less satisfactory classification in the target domain. This case is illustrated in the top of the light yellow box in Fig. 1 to better present the potential issue. As shown, if only aligning the marginal distributions, some target samples could still be misclassified because the class-specific samples between the
To address the above issue, we explicitly propose to align the class probability distributions given samples between the source and target domains 1 , instead of merely aligning the marginal distributions. This is because the class distribution crucially determines the classification result of a target sample. Also, it was proved by the Theorem 1 in the literature [15] that the expected target error is upper-bounded by three terms: 1) source error, 2) domain divergence, and 3) the difference in labeling functions across domains. The difference of labeling function can be significantly affected by the difference of the class distributions between the two domains, thus we do our best to align the class distributions to reduce the target error. Note that according to the Bayes' rule, when both marginal distribution P (x) and joint distribution P (x, y) are aligned, the class distribution P (y|x) shall also be aligned. Therefore, we propose a novel Conditional ADversarial Image Translation approach (CADIT) to simultaneously align (1) the marginal distributions and (2) the joint distributions between the two domains (see Fig. 1). Only aligning one of them will not ensure the desired alignment of the class distributions. Table I lists the main difference of existing domain adaptation methods from our method. It is worth noting that a few methods [3], [6] have attempted to approximately align the joint distributions. However, due to the unavailability of target sample labels, effectively aligning the joint distributions still remains an open issue (see the Remark for detailed differences).
Specifically, to align the marginal distributions, we adopt the widely-used CycleGAN [16] as a base network of adversarial image translation. Nevertheless, we found that CycleGAN, as an unsupervised image translation method, does not necessarily maintain the class labels of samples during translation, which is however important in our method. To deal with this issue, we put forward a discriminative structure-preserving (DSP) loss to prevent the undesired flips of class labels. Also, to align the joint distributions, we develop a novel joint adversarial generation (JAG) loss in both domains to jointly distinguish the authenticity/fakeness of the samples and their labels in an adversarial way. Since the labels of target samples are not available, we obtain their pseudo-labels and utilize the entropy-based prediction confidence to handle the uncertainty of these pseudo-labels. The target samples with more reliable pseudo-labels are allowed to contribute more to the joint distribution alignment. Besides, we incorporate a classifier into each discriminator in both domains to simultaneously perform the image generation and classification. Considering that the two classifiers could be affected by the image generation and thus produce inconsistent classification predictions for a same target sample, we impose a classification consistence constraint (CLC) loss on their predictions between the two domains to address this issue. By integrating the above components, we produce a more efficient and more interpretable unsupervised domain adaptation approach. In sum, the main contributions of this paper include: • We propose a novel adversarial domain adaptation approach, CADIT, to explicitly strive to align the class distributions between the two domains. • To realize the proposed approach, we designed and effectively integrated three novel losses: DSP loss, JAG loss and CLC loss to impose the key requirements. • Both qualitative and quantitative experimental results on multiple benchmark datasets (i.e., Digits, Faces, Scenes and Office31) validate the effectiveness and advantage of the proposed CADIT with respect to the state-of-the-art methods.

II. RELATED WORK
Recently, unsupervised domain adaptation has aroused a great interest of researchers. Existing methods can be roughly classified into two categories as follows.
Deep Domain Adaptation Methods. This category of methods [2], [3], [17] employed deep networks to learn the domain-invariant representation, which is transferable between different domains. Long et al. [1] introduced multiple domain adaptation modules into the high layers of deep convolutional network to match the mean embeddings of distributions according to a maximum mean discrepancy criterion. Afterwards, they [3] proposed a joint maximum mean discrepancy criterion to align the distributions of multiple domain-specific fully connected layers. In [18], an unified deep domain adaptation framework with domain alignment layers was proposed to match the feature distributions between different domains.
Adversarial Domain Adaptation Methods. This category of methods applied GANs to reduce the domain gap. There are several methods [19], [5], [20] that developed the domain-invariant feature generators and a domain discriminator to distinguish their authenticity/fakeness. Tzeng et al. [5] proposed a generalized framework to combine adversarial learning, discriminative feature learning and untied weight sharing. CDAN [6] built a multilinear map of the high-level feature representation and classification prediction between the source and target domains to adapt the map between them. To learn domain-invariant and semantic representations, a graph convolutional adversarial network [21] was built to jointly perform the alignment of data structure, domain and class centroid.

Ct
Real/Fake pseudo-labels Source Labels

Generator from source-to-target
Generator from target-to-source Besides, several other methods [9], [13], [12] conducted adversarial image translation for domain adaptation by generating the source-like or target-like images from real images. Coupled GAN (CoGAN) [22] trained a coupled generative model to learn the joint data distributions of the two domains. UNIT [23] extended CoGAN to learn image translation by integrating variational autoencoder and GAN. ADGAN [9] simultaneously performed image translation and learned the shared feature embedding, by imposing the source and target distributions to be close in a latent space. SBADA [10] employed the structure of CycleGAN for image translation and proposed target self-labeling and class consistence between source domain images and their circle generation results.
Remark. Note that, our method belongs to adversarial image translation for domain adaptation that has a compelling visualization and interpretation of generation results. For the above existing methods (see Table I), these methods ADGAN [9], DupGAN [13], SBADA [10], ACAL [14] only aligned the marginal distributions of samples between the two domains and this, as we have argued, may not be sufficient for reducing domain gap. On the other hand, the two methods [3], [6] attempted to approximately align the joint distributions by aligning the high-level feature representation and the prediction between two domains. However, such an alignment could be affected by the incorrect predictions of target samples, which is effectively mitigated in our method by considering the prediction uncertainty. Also, these two methods did not consider the alignment of the marginal distributions and this will not help to pursue the class distribution alignment. They will be experimentally compared with our method (see Table  ??). Very recently, several works have began to concern the class-level alignment in domain adaption from different aspects [21], [7], [20], [24]. This supports the timeliness and significance of our work from another perspective. At the same time, we highlight that having key differences from our work, these methods are not designed for cross-domain image translation, and do not explicitly strive to align the class distributions as our method either.

III. OUR APPROACH
Formally, given a source domain (without known labels), x s and x t represent the images in the source and target domains, respectively, and y s represents the label of the source domain. In the target domain, the label y t is not available. The sample x s (or x t ) obeys the marginal distribution P (x s ) (or Q(x t )). Its joint distribution with labels and class distribution are denoted as P (x s , y s ) and P (y s |x s ) (or Q(x t , y t ) and Q(y t |x t )), respectively. Due to the discrepancy of the distributions between the two domains, the classifier trained on the source domain cannot be directly applied to the target domain. Thus, the task of UDA is to reduce the domain discrepancy to improve the generalization of the classifier on target samples. To achieve this, existing methods usually align either marginal or joint distributions by various strategies. Differently, we propose to align both of them in order to make the corresponding class distributions well matched, and this is implemented by conducting GAN-based image translation between the two domains.
Specifically, we design two GANs (G t , D t ) and (G s , D s ) to generate a target-like sample G t (x s ) from the source sample x s and a source-like sample G s (x t ) from the target sample x t . For simplified notation, G t (x s ) and G s (x t ) are abbreviated aŝ x st andx ts , respectively. Then, the image translation between the two domains is realized by: align to Q(x t ) (in the target domain), and Q(x ts ) (translated from the target domain) align to P (x s ) (in the source domain), as well as ensuringx st to be well associated with its own source label y s ; (2) (joint distribution alignment) making the joint distribution P (x st , y s ) (translated from the source domain) align to the joint distribution Q(x t , y t ) (in the target domain), and Q(x ts , y t ) (translated from the target domain) align to P (x s , y s ) (in the source domain).
By enforcing the two alignments in steps (1) and (2), we can expect the corresponding class distributions P (y s |x st ) and Q(y t |x t ) (or Q(y t |x ts ) and P (y s |x s )) to be better matched after image translation. In turn, this will make the classifier trained by the labeled samples (translated from the source domain) more applicable to classify the unlabeled target samples.
The framework diagram of our method is depicted in Fig. 2 with three following parts: (1) a discriminative cycle-consistent GAN with two generators and two discriminators to align the marginal distributions between the two domains after image translation, (2) a joint adversarial generation (JAG) loss in each discriminator to align the joint distributions of samples and their labels between the two domains, and (3) a discriminative structure-preserving (DSP) loss and a classification consistence (CLC) loss to prevent label flipping and enhance the classification performance. They are detailed as follows.

A. Discriminative Cycle-consistent GAN
To align the marginal distributions, we adopt widely-used CycleGAN [16] as a base network to realize our method, due to its success on image-to-image translation. CycleGAN is used to build the two generators G t and G s to learn two source-to-target and target-to-source mapping respectively: G t : x s → x t and G s : x t → x s , where a target-like imagex st (or a source-like imagex ts ) is generated to adapt to real images x t (or x s ). To reduce the feasible region of mapping functions, a cycle-consistent loss is used to make Meanwhile, against G s and G t , the two discriminators D s and D t are designed to distinguish the authenticity or fakeness of images in both the domains, respectively. Moreover, to simultaneously perform the image translation and classification, we additionally design two classifiers C s and C t based on their discriminators. Accordingly, the adversarial generation loss in the target domain can be written as: where L ce (·) represents the cross-entropy loss to train the classifier C t in the target domain by using the samplesx st translated from source-to-target. Likewise, in the source domain, the classifier C s is trained by the real source samples x s . Also, the adversarial generation loss in the source domain can be written as: Also, as in CycleGAN, the cycle-consistent loss is calculated as: DSP Loss. Since CycleGAN is originally designed for unsupervised image translation, it does not guarantee to maintain the label of images during image translation, which is however important for our alignment. As a result, an imagex st (translated from source-to-target) could be misaligned with its source domain label y s . To prevent the label flipping (ensuring the label consistence before and after image translation), we propose a discriminative structure-preserving (DSP) loss to make the generated images (from source-to-target) preserve their discriminative class structure in the source domain: where d(x i st ,x j st ) represents the distance between any two generated imagesx i st andx j st (from source-to-target), calculated by Euclidean distance of their features. W ij is the weight between them (1 for the same class, -1 for different classes), according to their source domain labels. In this way, the generated images are encouraged to be similar when they belong to a same class and dissimilar otherwise. Thus, a generated image is subject to other generated images during image translation, so its label is difficult to be flipped, which has also been validated in the experiments (see Fig. 5 and Tables VII-VIII).
CLC Loss. As shown above, during domain adaptation, the two classifiers C t and C s are used to predict the labels of target sample x t and its generated samplex ts (from target-to-source), respectively. Since the two classifiers could be affected by the image translation and thus produce inconsistent classification predictions, we impose a classification consistence constraint (CLC) loss on their predictions to aid the classifier training as: Obviously, the CLC loss is also useful to prevent the label flipping during image transaltion. Note that since the predictions of source samples in the two classifiers are constrained by their labels in Eqs.(1)(2), their consistence constraint is safely omitted.

B. Joint Adversarial Generation
To align the joint distributions, based on CycleGAN, we develop two novel joint discriminators J t and J s in the two domains to jointly distinguish the authenticity/fakeness of samples and labels.
JAG Loss. Note that the true label y t of target sample x t is not available during training, so we adopt the classification prediction C t (x t ) as its pseudo-label. However, the pseudolabels may be incorrect that could affect the alignment of the joint distributions. To alleviate the impact of false pseudolabeled samples, we expect to set higher confidence weight for more reliable pseudo-labels that are encouraged to contribute more to the joint distribution alignment. Inspired by entropy minimization to handle the uncertainty, we design an entropybased confidence weight γ(x t ) = 1 − H(x t )/ log C for x t and its pseudo-label, where H(x t ) is the entropy of its prediction probabilities on all classes, and C is the number of classes.
In the joint discriminator J t of the target domain, we expect to align the joint distributions of real and generated images (from source-to-target) associated with their respective labels, and thus design a joint adversarial generation (JAG) loss in the target domain as: where the paired samples and labels (x st , y s ) or (x t , C t (x t )) are simultaneously fed into the joint discriminator J t to distinguish the authenticity/fakeness. The lower the weight γ(x t ), the less the target sample x t works. On the contrary, those target samples with higher weight are more useful to align the joint distributions. Likewise, in the joint discriminator J s of the source domain, we also introduce the entropy-based confidence weight γ(x ts ) to balance its possibly-incorrect pseudo-label. Thus, the JAG loss in the source domain can be written as: L JGANs (G s , J s , C s ) = E (xs,ys)∼P (xs,ys) [log J s (x s , y s )] + E (xts,yt)∼Q(xts,yt) [γ(x ts ) log(1 − J s (x ts , C s (x ts )))].

C. Main Objective
In summary, to effectively align the class distributions for domain adaptation, we integrate the aforementioned seven terms in Eqs. (1)-(7) into a whole objective function as below: where parameters λ 1 , λ 2 , λ 3 are the weight of cycle-consistent term (for image translation), CLC loss term (for target samples translated from target-to-source), and DSP loss term (for source samples translated from source-to-target), respectively. For the values of λ 1 , λ 2 , λ 3 in the experiments, we empirically set them as (5, 0.1, 10 -4 ) for Digits, (10, 0.5, 10 -3 ) for Faces and Scenes, and (10, 10 -3 , 10 -4 ) for Office31 respectively. Also, we found that our method is robust against the variation of three parameters.
Note that in Eq. (8), two generators G s , G t and two classifiers C s , C t are solved by minimizing the above objective, while four discriminators D s , J s , D t and J t are solved by maximizing the above objective. As the traditional GAN, the generators and discriminators are trained in an adversarial manner, and they are optimized alternately. In the training phase, the labeled source samples and unlabeled target samples are used to train our model. In the testing phase, we adopt the learned classifier C t to predict the labels of the unlabeled target samples.
Network Architecture. We implement our method CADIT using PyTorch on a single NVIDIA GeForce GTX 2080TI GPU. For the generators G s and G t , we adopt two different network structures on all the datasets due to their different image sizes. For Digits, we construct them by designing a simple two-layer convolutions and two-layer deconvolutions. For the other datasets, we apply the same architecture of the generators as that in CycleGAN, which contains two stride-2 convolutions, six residual blocks, and two fractionally-strided convolutions with stride 1/2. For the discriminators, we adopt a same network structure in all the datasets, which includes a common part and three different branches, as shown in Fig. 2. The common part is implemented by two stride-2 convolutions and a fully-connection layer with the LeakyReLU activation, and all the three branches are a specific fully-connected layer. Also, for the adversarial generation loss function in D s , J s , D t , J t , we adopt mean squared error (MSE) loss for training.
Network Training. We use the Adam solver [25] with the related gradient weights of 0.5 and 0.999 to train our model. The base learning rate is initially set as 0.002, and then decreases from 0.002 to 0.0002 as the number of iterations increases. The batch size is set as 64 for Digits, 8 for Faces and Scenes, and 6 for Office31, respectively. To reduce the training time on Faces and Scenes, we first train the CycleGAN network (by setting λ 1 as 10) to initialize our generators, and then alternately optimize our generators and discriminators.

IV. EXPERIMENTS
We evaluate the performance of our method on unsupervised domain adaptation for image classification. The experiments are conducted on three domain adaptation datasets: (1) Digits, (2) Faces, and (3) Scenes. For all the experiments, we follow the standard unsupervised protocol: using the entire labeled samples in the source domain and unlabeled samples in the target domain. It is worth noting that the samples between the two domains are not paired. Also, the labels of samples in the target domain are not available during the training phase.

A. Digits: MNIST↔USPS
For Digits, MNIST and USPS, two widely-used handwritten digit image datasets are viewed as two different domains. Following the same protocol in [26], [27], we randomly sample 2,000 images in MNIST and 1,800 images in USPS to construct the source/target-domain dataset, and uniformly rescale all images to the size of 16×16. We test two domain adaptation settings: (1) MNIST→USPS (abbreviated as M→U), and (2) USPS→MNIST (abbreviated as U→M). In each setting, we apply the learned classifier C t for digital image recognition (0-9) in the target domain.  We compare our method with fifteen representative UDA baselines: four traditional domain adaptation methods (JDA [26], TJM [27], JGSA [28], and TPN [17]), three deep domain adaptation methods (DANN [29], DRCN [30], and JAN [3]), four adversarial domain adaptation methods (ADDA [5], CDAN [6], RAAN [31] and MCD [32]), and four related adversarial image translation methods (CoGAN [22], ADGAN [9], CDRD [33] and SBADA [10]). Table II reports the average accuracy results,in which JAN, CDAN and SBADA were experimented by the publicly released codes, and the other twelve methods were reported by the literature. For fair comparison, the baselines applied the same protocol of data as our method (using randomly sampled 2000/1800 images instead of the entire training sets). As observed, our method shows the best performance in both settings and the average. This verifies the effectiveness of our method.
Also, we show the t-SNE results of M→U for different adversarial image translation methods in Fig. 3. Specifically, we compare the real and generated images for CycleGAN, SBADA and CADIT, and compare the generated images from both domains for ADGAN. Due to their small size and grayscale intensity, we simply project their intensity values to 2D embedding space as same as SBADA [10]. It is observed that the samples between the two domains are not aligned before image translation (see Fig. 3(a)), while the samples after image translation via our method are best aligned and clustered than the other methods (see Fig. 3(b)-(e)). Although the results of ADGAN show a good cluster structure, they are not well aligned between the two domains. This indicates that the class distributions between the two domains are well aligned in our method. In addition, we show the visualization results of typical examples in image translation between MNIST and USPS in Fig. 4. As observed, the generated images can well capture the style of digital images in the current domain.

B. Digits: SVHN↔MNIST
SVHN [34] was obtained by cropping house numbers in Google Street View images, and contains over 600k color images with a large variety of shapes and textures. Following the protocol [5], we use the entire training data of SVHN and MNIST as two domains, and test two settings: (1)  MNIST→SVHN (abbreviated as M→S), which presents a more challenging case of domain adaptation due to larger domain gap than M↔U.
We compare our method with nine representative UDA baselines: five deep/adversarial domain adaptation methods (DANN [29], DRCN [30], ADDA [5], JAN [3], and CDAN [6]) and four latest and related adversarial image translation methods (ADGAN [9], SBADA [10], CyCADA [11], and ACAL [14]). For fair comparison, the baselines applied the same protocol of data with our method. We evaluate the performance of JAN and CDAN by their publicly released codes and meanwhile report the best results of the other seven methods in the literature in Table III, compared with our method. As observed, the M→S and average result of our method outperform the rest baselines. In addition, we show the visualization results of typical examples in image translation from SVHN to MNIST by using three different methods (SBADA, ACAL and CADIT) in Fig. 5. As observed, during image translation via SBADA, the labels of some digits are changed, e.g., '4' is often translated to '6', '1' is often translated to '9'. For ACAL, all the images of '2' are translated to '7'-like images, which make them difficult to distinguish. Also, all the images of '4' are not well generated. Compared with SBADA and ACAL, our CADIT can effectively prevent label flipping by using the proposed DSP loss, and most of the generated digits seem more clear and better generated.

C. Faces: Photo→Sketch
For Faces, we use facial photo and sketch images as the source and target domains, respectively. Inspired by [33], we construct a facial photo/sketch dataset by using the CelebA dataset [35] that contains more than 200K celebrity photos an- notated with forty facial attributes. Specifically, we randomly select half of the CelebA dataset as the source domain, and then convert the other half into sketch images by using the method in [36] for the target domain. Since the number of attributes is large, we choose three common and representative attributes: "glasses", "smiling" and "male" (similar with that in [33]) to perform three individual binary classification tasks.
These tasks are to recognize if a target sketch image has the corresponding attribute, respectively, following the standard UDA setting. We compare our method with eight latest and most-related methods, JAN [3], CDAN [6], ADGAN [9], DupGAN [13], SBADA [10], DWT [18], ACAL [14] and JDDA [7]. For fair comparison, we test these methods by utilizing their released source codes and learning the best parameters. The average classification results of five independent runs on facial images are recorded in Table IV. As observed, CADIT obtains the highest accuracy in all the compared methods for the three classification tasks, which shows that our method with jointly aligning the marginal and joint distributions is very effective for classifying target samples again. Also, the visualization results of typical examples in facial image translation from photo to sketch are illustrated in Fig. 6. The generated sketchlike images are clear and realistic, which shows that our method has a promising visualization performance during adaptation besides its superior classification performance.

D. Scenes: Photo→Painting
For Scenes, we use scene photo and painting images as the source and target domains, respectively. We collect 1,838 scene photo images from the publicly accessible dataset 2 , and then randomly select half of the photo collection as the source domain. Afterwards, we construct painting images by applying the method in [36] on the rest half of the photo collection for the target domain. For performance evaluation, these scene images are manually labeled with three interest attributes: "night", "winter", and "mountain". Then, we test three binary classification tasks with UDA setting: using labeled source photo images to recognize target painting images with the corresponding attribute, respectively. We compare CADIT with the eight related methods (same as Faces) on Scenes. The average classification results of five independent runs are recorded in Table V. It is observed that CADIT achieves the highest accuracy against the other methods for all the three classification tasks, which validates the efficacy of our method again. Also, Fig. 7 (3) DSLR, which are abbreviated as A, W and D, respectively. Amazon images is downloaded from online merchants, Webcam images are with low-resolution by a web camera, and DSLR images are with high-resolution by a digital SLR camera. We only focus on four challenging unsupervised domain adaption tasks among the three domains: A→W, W→A, A→D and D→A. To alleviate the effect of the illgenerated images on classification, we train a classifier on the feature map learned by the generator, as in ADGAN [9]. For good performance, we initialize the generators G s , G t by using the pre-trained ResNet-50 model trained on Imagenet.
We compare our method with seven latest and most-related methods, DANN [29], ADDA [5], JAN [3], CDAN [6], ADGAN [9], and JDDA [7]. For fair comparison, we apply the standard protocol for unsupervised domain adaptation same as the baselines, and use all labeled source data and all unlabeled target data for experiments. We evaluate the averaged classification results of our method on three independent runs and report the best results of the baselines in the literature in Table  VI. As observed, our method performs best in the two tasks (W→A and W→D), and is comparable with CDAN in the averaged results. their nine different combinations on Digits, Faces and Scenes in Tables VII and VIII, respectively. We observe that the more terms are added, the better the performance is. Also, our method (Row 9) shows the best performance among all the methods.
(1) Specifically, the first three rows show that bi-directional cycle-consistent GAN outperforms the single source-totarget or target-to-source ones; (2) Rows 4-6 show that the addition of the CLC loss and JAG loss brings a large performance improvement, respectively; (3) Rows 7-9 show that using an entropy-based confidence weight (γ ≤ 1) to measure the reliability of predicting target samples is useful to align the joint distributions, and the DSP loss also presents a significant effect to enhance the performance of our method; (4) In addition, comparing all these methods, we notice that introducing L JGAN t/s (comparing Rows 3 and 5) into our method achieves the most significant performance improvement in all terms, which implies that applying JAG loss for the joint distribution alignment is most effective to reduce the classification deviation caused by domain gap. In sum, the results of Tables VII and VIII show that all the proposed loss terms are useful, and they collectively form an effective solution of UDA.
2) Distribution Discrepancy:  [15], [29], the proxy A-distance is adopted in our experiments, and the lower the A-distance, the less the domain discrepancy. The A-distance can be calculated as d A = 2(1 − 2 ), where is the generalization error of the domain classifier [29]. Fig. 8 depicts the A-distance of all the five methods. As observed, d A in our CADIT is lower than those in other four methods for all the tasks, which again indicates that our method can effectively reduce the domain discrepancy.

V. CONCLUSION
We present a novel method to simultaneously align the joint and marginal distributions between domains for adversarial image translation in order to better match the class distributions that is important for classification. Specifically, we develop the JAG loss, DSP loss and CLC loss on top of the structure of CycleGAN. Extensive experiments validate that our method achieves the outstanding performance in Faces and Scenes and the competitive results in Digits and Office31, compared with the state-of-the-arts. Also, the ablation study