Learning to Deceive in Multi-Agent Hidden Role Games

Deception is prevalent in human social settings. However, studies into the effect of deception on reinforcement learning algorithms have been limited to simplistic settings, restricting their applicability to complex real-world problems. This paper addresses this by introducing a new mixed competitive-cooperative multi-agent reinforcement learning (MARL) environment inspired by popular role-based deception games such as Werewolf, Avalon, and Among Us. The environment's unique challenge lies in the necessity to cooperate with other agents despite not knowing if they are friend or foe. Furthermore, we introduce a model of deception, which we call Bayesian belief manipulation (BBM) and demonstrate its effectiveness at deceiving other agents in this environment while also increasing the deceiving agent's performance.

Introduction
Human tendency to trust in the honesty of others is essential to effective cooperation and coordination [25]. At the same time, trust opens up the risk of being exploited by a deceptive party. Deception is prevalent in many real-life settings, especially in those that are adversarial in nature. Examples include cyber-security attacks [1], warfare [8], games such as poker [5], and even every-day economic interactions [16]. Outside of simplistic signaling games (e.g. [23,7]), use, and defence against, deceptive strategies has received relatively little research attention.
The popularity of social deception games, such as Werewolf, Avalon, and Among Us, reveals both a human fascination with deception, and the challenges that it creates. These games require players to cooperate with their unknown teammates to achieve their goals, all while trying to hide their identities. Therefore, social deception games could provide an interesting setting for research into deception. To explore deception beyond simple signalling games, we introduce a new hidden role mixed competitive-cooperative multi-agent reinforcement learning (MARL) environment Rescue the General (RTG). Our environment features three teams: red and blue, who have conflicting goals, and a neutral green team. To perform well, agents must learn subtle trade-offs between acting according to their team's objectives and revealing too much information about their roles.
To this end, we trained two sets of agents to play RTG, one in which the agent pursues the rewards of the game as per normal, and another where the agents are provided an intrinsic reward [31] to incentivise deceptive actions. We compare the behaviour of the two groups and analyse the their performance when pitted against each other. 1 The main contributions of this work are: We designed RTG with the following objectives: the game should be fast to simulate but complex to solve; game observations should be both human and machine-readable; code for the environment should be open source and easy to modify; the game should be mixed competitive/cooperative (i.e., not zero-sum); hidden roles should be a core game mechanic; 4 and good strategies should require non-trivial (temporally extended) deception.

Teams and Objectives
The game consists of three teams: red, green, and blue. The blue team knows the location of a general and must perform a rescue by dragging them to the edge of the map, but requires multiple soldiers to do so. Green are 'lumberjacks' who receive points for harvesting wood from the trees scattered around the map, and whose interests are orthogonal to the other teams. The red team does not know the general's location and must find and kill the general. Similar to the StarCraft Multi-Agent Challenge [30], each soldier is controlled independently, and has a limited vision (a range of six tiles for red, and five for green and blue). The complication is that no soldier knows any other's identity', including their own team members. Communication is limited to the soldiers' actions, namely movement by one tile north, south, east or west and the ability to shoot in one of the four cardinal directions. All soldiers have an ID colour allowing soldiers to track previous behaviour.
Player teams are randomised at the beginning of each game. Their locations are randomily initialised such that they are near to each other, but always more than 2-tiles away from the general.

Observational Space
Egocentric observations are provided to each agent in RGB format, as shown in Figure 1. Status indicator lights give the agent information on their x-location, y-location, health, turns until they can shoot, turns until a game timeout, and team colour. A marker indicating the direction of the general is also given to blue players. Rewards depend on the scenario and are detailed in Section 3.4. Figure 1: The global observation (left) and an agent's local observation (right). Soldiers are painted with their unique ID color on the outside, and their team color on the inside. Local observations omit the team color for all non-red players. Each map tile is 3x3 and encoded in RGB colour.

Scenarios
The RTG environment provides six scenarios of differing levels of challenge. These scenarios are given in Table 1. Additional custom scenarios can also be created through configuration options (for details see Appendix A).
In the primary scenario, Rescue, the blue team must rescue a general without allowing a lone red player to find and shoot the general. The Wolf scenario features a single red player who must kill all three green players before being discovered. R2G2 demonstrates orthogonal objectives, with two green players harvesting trees while two red players attempt to find and kill the general. Red2, Green2 and Blue2 are simple test environments where each team can learn their objective unobstructed.

Reward Structure
The rewards are structured such that a victory for red or blue always results in a score of ten, and are outlined in Table 2. R b refers to the total reward for the blue team this game, and is used to make sure blue always receives a maximum of ten points for a win. Green is also able to score ten points if they harvest all the trees on the map, but unlike red and blue, they will not terminate the game by achieving their objective. Due to the complexity of rescuing the general (two players must work together to move the general tile-by-tile) we provide small rewards to blue for completing partial objectives.

Bayesian Belief Manipulation Model
Our deception model is based on belief manipulation [10]. Each agent keeps track of what it believes other agents believe that an action would imply about their role. We refer to this as 'inverse' prediction. That is, predicting what others would predict about ourselves. Using these inverse predictions and an assumption of Bayesian belief updates, a reward is generated to incentivise agents to take actions that would mislead other agents about their true role. We call this Bayesian belief manipulation (BBM).

Theory of Mind
In order to deceive another player, we need to model our belief about their belief of how our actions would be interpreted. Take, for example, the situation in Figure 2. If red believes that blue knows that red can see the general, this action will be considered a very 'red' move. However, if red believes that blue is unaware of the general, this action can, perhaps, be taken without revealing red's role. Thus, agents must model, not only the beliefs about others but also others' beliefs about themselves.

Assumptions
In our model the deceiving agents operate under the following assumptions: 1. All non-deceiving agents are Bayesian in their approach to updating belief about roles. 5 2. The policy of each role is known by all players. 6 Figure 2: Should red believe they are giving away their role by shooting east? This depends on whether red knows that blue knows that red is shooting the general (depicted with a cross). Blue's limited vision is shaded, and both soldier's roles are visualised for context. If blue knows that red knows where the general is, blue will interpret this as a very 'red' move.
3. Agents are limited to second-order theory of mind.
The second assumption amounts to knowing the type of the policy, that is, knowing the set of potential policies, but not knowing which one the agent is acting out. Because the set of policies could, in theory, be very large, this assumption is not as limiting as it sounds. The third assumption follows the findings of [9], who conclude that while agents with first-order and second-order theory of mind (ToM) outperform opponents with shallower ToM, deeper levels of recursion show limited benefit.

Notation
We use the following notation: • A i refers to the i th agent, and A z i refers to the event that the i th agent has role z. • h t i is A i 's history of local observations up to and including timestep t. Where t is omitted it can be assumed to be the current timestep.
• Z is the set of all roles, and π z the policy of role z.
We also write P i to mean the probability that the A i assigns to an event, and P i,j to be A i 's estimation of the the probability that A j would assign to an event. For example P 1,2 (A r 1 ) is A 1 's estimate of how much A 2 believes that A 1 is on the red team.

The Model
Given the assumptions in Section 4.2, we consider how a Bayesian agent j would update their belief about agent i's role given that they observed A i taking action a in context h i . We denote A i 's true role with * .
Because A j does not know h i , they must estimate it as h j,i . A i in turn does not know what h j,i is and so must estimate this too as h i,j,i . Therefore A i can estimate A j 's new belief about their role, if they were to take action a as Given that the probability of taking an action given history and role is, by definition, the known policies for each role, A i can estimate the Bayes factor with which A j would update their belief about A i 's true role as As we wish to encourage deceptive agents to minimise this belief, we must reward agents for actions that generate ρ < 1 and disincentivise actions that would generate ρ > 1. Therefore, we provide agents with a − log(ρ) reward bonus, which fulfils this criterion, and whose interpretation is an estimate of the negative log of the Bayes factor in a Bayesian update for the other agent.
Implementing this bonus requires each agent i to model both h i,j,i for each other player j, as well as model P i,j (A z i ) for each z ∈ Z. The predicted history for each agent must then be run through each known policy to produce action distributions estimations for each role.

Direct Policy Prediction
The requirement that each agent must model all other other agents' complete histories can be removed by predicting the current policies for each agent directly. This avoids predicting irrelevant details about the other agents' observations, is more computationally efficient, and relaxes assumption (2) about needing to know the policies ahead of time. 7 We do this by modelling This direct policy prediction method amounts to each agent predicting what agent thinks it would do in its current circumstance, if it was each of the known roles.

Deception Bonus
Deceptive agents are given a bonus to their rewards based on their estimate of ρ in equation 3. Similar to [20], we filter out all but a 10% random sample of non-visible agents as their contribution adds a lot of noise. 8 We also exclude dead players from receiving or giving deception bonuses. The ρ estimates for all remaining players are summed and added as an intrinsic reward as where r t i is the reward at timestep t for player i, and α i is the magnitude of the deception bonus reward for A i . Therefore, agents are incentivised to take actions that would mislead other agents about their true role. This formulation has several advantages. First, agents are not incentivised to deceive players who already know their role, as in this case the Bayesian update would be very small. Second, agents are not incentivised to pretend to be another role when they can get away with it. If no other player is around, or if they believe that this action could not be interpreted negatively (as in Figure 2), they will not be penalised. It is important to note, that in a complete information game all agents agree on all histories, and this situation would not occur.

Training and Evaluation
To evaluate the effect of BBM as an intrinsic motivation, we trained two groups of agents on the Rescue scenario. Each group consisted of four agents, trained with different random seeds. Agents in all groups trained with deception modules; however, the first group had α = 0 for all agents, while the second group set α = 0.5 9 for red team players, and α = 0 for all other players. We trained the agents under the centralised learning, decentralised execution paradigm [13] detailed further in Section 5.2 7 Each player's current policy is needed as targets during training, so effectively this limitation is still in place. However, it is now a training detail rather than inherent to the model, and could potentially be removed by inferring policy from observed actions. 8 The argument that they did not observe the action and could therefore not update their belief is not valid here, as they could potentially see the consequences of the action in the future (a dead body for example). Our model does not handle this case and is left for future work. 9 α = 0.5 was chosen as initial tests showed this gave agents roughly one quarter of their reward from BBM. We kept the deception bonus reward small so as not to cause agents to loose sight of their primary objective, which was to win the game. Here π r , π g , π b are the current policy estimations for each player if they were red, green and blue. V ext and V int are the value estimations for the extrinsic and intrinsic returns respectively. s t i is the local observation at timestep t for the ith player, and n is the number of players in the game. The output from the deception module is to generate deception bonus' only, and is not required once training is complete.

Agent Architecture
Our BBM agent is composed of two distinct modules: the policy module and the deception module (see figure 3). The policy module was trained using local observations only, while the deception module (discarded after training) required target information from other agents.
Both modules feature the same encoder architecture, which is loosely based on DQN [28], but modified to use 3x3 kernels to the match the 3x3 tile size and lower resolution of the RTG environment. To address partial observability a recurrent layer, in the form of Long Short Term Memory (LSTM) [14], was added to the encoder, with residual connections from the convolution embedding to the output (see Appendix B for more detail on the encoder architecture.).

Training Details
The policy module takes an agent's local observation as input, encodes it via the encoder, and then outputs a policy and extrinsic/intrinsic value estimations for each of the three roles. During training, gradients are only backpropagated through the output head matching the role of the given agent. 10 The policy module was trained using Proximal Policy Optimisation (PPO) [32] by running 4 epochs over batches of size 3,072 segments, each containing 16 sequential observations, using back propagation through time.
We also experimented with using separate models for each policy, but found little difference in the results, at the cost of longer training times (as did [2]). A global value estimate based on the observations of all players was also considered, but we did not find significant differences in the estimates and so opted for local value estimation instead.
To make sure at least some games were winnable for each side we modified the game rules during training so that in half of the games, a player from a randomly selected team was eliminated before the game started. We found failing to do this sometimes caused one team to dominate the other, prohibiting progress.
The policy and deception modules were trained simultaneously on the same data. They share an identical encoder structure but used separate weights. Because of this, the deception module can be removed once training has finished, allowing for decentralized execution. The policy module produces the policy and value estimate based on local observations only. The deception module outputs predictions for player roles and player actions, as well as their inverse predictions (that is, what it predicts that other agents are predicting about itself). See Figure 3 for details.
We trained each of the eight runs for 300-million interactions (which equates to 50-million game steps) over 5-days on four RTX2080 TIs using PyTorch 1.7.1 [29]. Checkpoints every 10-million interactions were saved, allowing for evaluation of the agents' performance at various points during training. Hyperparameters for the model, given in Table 3, were selected by running a grid search on the Red2 scenario. For more details refer to Appendix C.

Losses
Action predictions were trained by minimising the Kullback-Leibler (KL) [22] divergence between the predicted distribution and ground-truth targets taken from the agents during rollout generation. We trained the inverse action predictions in the same way, using the other agents' predictions during rollout generation as targets.
We trained the role predictions by minimising the negative likelihood loss, using players' true roles as targets. We trained inverse role predictions by minimising the KL divergence between the agent's prediction of other agents belief about its role, and their actual prediction about the agent's role.

Intrinsic Rewards
Raw deception bonuses were calculated, clipped to the range (-20,20) and logged for each agent. Bonuses were then zeroed for all non-red players, normalised such that the intrinsic returns had unit variance, multiplied by φ(t) where φ(t) = min(1.0, t/10 × 10 6 ) and t is the timestep (in terms of interactions) during training then scaled by α i . This creates a warm-up period where agents can learn the game with without an auxiliary objective. Similar to [6] our model outputs intrinsic and extrinsic value estimations separately, then combines them when calculating advantages.

Evaluation
Evaluating agents within a multi-agent scenario poses some unique challenges. Unlike games where agents compete against a static environment such as Atari, self-play means that as the agent gets stronger, so does its opponent. Thus, an agent's skill progress might not be visible from plots of the scores alone.
To address this we evaluated each run by playing 16 games, every ten-epochs, 11 against opponents from each of the eight runs (128 games in total). The green team was played using the same agent as the adversary. Performance across the four runs was averaged.
We also recorded raw deception bonuses, taken before normalisation or scaling, for all players, even those who did not receive them. This allowed for monitoring of deceptive behaviour on non-deceptively incentivised players. Accuracy of role predictions was taken from data generated during training.

Effect of Deception on Agents' Behaviour
In this section we present the effect of deception on the agents in terms of their ability to predict roles and their performance in the game, and discuss the tendency towards honest behaviour when no deception incentive is provided.

Role Prediction
Identifying other agents' role is a difficult task for blue players in the Rescue scenario. At the end of training, blue assigns an average probability to players' true role of 0.617 ± 0.08(95% CI) when trained against deceptive adversaries, as compared to 0.707 ± 0.07(95% CI) when trained against non-deceptive adversaries (see Figure 4). 12 Red's role predictions were very high at 0.995 ± 0.001(95% CI), which is expected as they are able to see the roles of all visible players. A blue player who knows their role, and assigns remaining probability based on naive population counts would on average score 0.659. Therefore, the deceptive agents red caused the blue team to predict roles more poorly than if they had no in-game observations at all. Figure 4: Probability assigned to true role the blue team. Values shown are the log-averages taken during training and represent the average probability a blue player assigned to each player's true role. As seen by the decrease in role prediction accuracy, the blue team struggles to identify players' roles when faced with deceptively motivated adversaries. The black line indicates a naive, count-based, estimation of player roles.

Performance of Deceptive Agents
Despite the effectiveness on role deception, BBM has only a minor impact on the agents' score in our experiment. As seen in Figure 5 (a), red players trained with deception see an increase in performance at the end of training from −8.71 ± 0.040(95% CI) without deception to −7.21 ± 0.58(95% CI) with deception. This result, while statistically significant, is only seen at the end of training, and does not represent a large difference in performance. It does, however, suggest further experimentation into the effect of a larger deception bonus. When facing deceptive agents blue is slower to learn a winning strategy, but eventually converges to the same outcome (see Figure 5 (b)).

Deceptive Tendencies in Unincentivised Agents
We look now at agents' behaviour when not given explicit deception incentives. Blue learns to act honestly when faced with either deceptive or honestly adversaries. This is expected in the Rescue scenario as blue must identify teammates in order to coordinate with them. An unexpected result is that green, who initially acts honestly, learns an increasingly deceptive strategy, despite no explicit incentive to do so. We suspect that as red develops a strategy that involves pretending to be green, green must respond by pretending not to be green to survive blue's hostility.
The red players, when incentivised to do so, learn to obtain a high degree of deception bonus (see Figure 6 (a)). When no explicit incentive is provided red players do not naturally learn a deceptive strategy. This suggests that either deception is of less value for the red team in the Rescue scenario than expected, or that a deceptive, but effective, strategy is difficult to learn for red.

Conclusions and Future Work
Our work presents a new, open-source environment, RTG for development and testing of deception in a complex hidden role game, as well as a novel a model of deception that makes use of second-degree ToM to manipulate other agents belief about it's own role. Our empirical results show BBM to be very effective at manipulating other agents belief about their role while at the same time improving their performance in the game. We also found that deceptive behaviour, when not incentivised, is not learned naturally by the competitive red or blue teams in the Rescue scenario, but surprisingly is learned by the neutral green team.
These results suggest several questions for future research into the effect of deception in a MARL setting. Can the relatively modest performance improvement in performance be extended by increasing the magnitude of the deception bonus, or would this cause agents to act deceptively at the cost of their primary objective? What would the impact of an honesty bonus be on the blue team in the Rescue scenario, would this encourage better cooperation? How would incorporating social aspects, namely explicit communication, impact deceptive behaviour? And how could our model, which assumes only one deceptive team, be extended to account for two or more deceptive teams.

A Scenario Configuration Options
We designed Rescue the General (RTG) to be highly customisable through configuration options. We used only a subset of the configuration settings in the supplied scenarios, with the full set outlined in Table 6. We also designed a voting system where players could initiate votes to remove players. However, we found that agents never developed an effective strategy for this system, and so did not use it.

B Encoder Architecture
In our model, both the policy module and deception module share the same encoder architecture, but with separate parameters. The encoder design is a modification of the network often used in Deep Q-Networks (DQN), but with smaller 3x3 kernels to match our game's tile-size. The encoder takes a single local observation scaled to [0..1], s t , as input, as well as the LSTM cell states (h t−1 , c t−1 ) from the previous timestep. The input is fed through the convolutional layers, flattened and projected into n dimensions, where n are the number LSTM units, then fed through the LSTM layer. See Figure 7 for details. We found that adding a residual connections skipping the LSTM layer improved training performance, and did not negatively affect the agents' ability to remember past events. 13 The policy module used 512 LSTM units, whereas the deception module used 1024.  Figure 7: Encoder Architecture. Input s t is fed into a convolutional neural network, a linear layer, and finally, an LSTM layer.

C Hyperparameter Search
We performed a hyperparameter search for the policy module using a randomized grid-search. Our search used 128 runs over ten epochs on the Red2 scenario. 14 We used the average score taken from the last 100 episodes during training, where score := R × 0.99 t 13 We verified this by observing that red players were able to retain information about the roles of previously seen players 14 We did not use the Rescue scenario as it is more difficult to evaluate performance, and runs would require significantly more training time.
with R being the red teams score at the end of the episode and t being the episode length. Discounting score in this way prefers hyperparameter settings that can more quickly find and kill the general and distinguishes between runs that consistently score the optimal ten points.
The out features setting controls the number of units in the final linear layer. When residual mode was active out features were set to the same value as LSTM units. The off mode disabled the LSTM layer, on passed through the LSTM layer as per normal, concatenate concatenated the LSTM output with the linear layer, and residual added residual connections.
Hyperparameter values were selected during the search by sampling uniformly-random from the values in Table 4. We then plotted each hyperparameter by selecting the best five runs for each value setting and graphing their min, mean and max. The purpose of our hyperparameter search was not to find optimal settings but to find reliable settings that would allow us to assess the difference between deceptively incentivised agents, and the control agents. Therefore, in cases where hyperparameters had similar performance, we preferred values used in prior work and those with better computational efficiency.
A separate hyperparameter search was used to find hyperparameters for the deception module, using 64 runs over 25 epochs on R2G2 with hidden roles. This search was performed using an older version of the algorithm where agents predicted other agents observations rather than action distributions and used the negative log of the mean-squared prediction error as the score. We found these settings to work well on the updated algorithm and did not modify them. Details are given in Table 5.
We enabled window slicing (max window size) on the deception module. During deception module training, shorter windows were extracted from the longer rollout segments. This allowed the deception module to use a shorter backpropagation-through time length than the policy module, which we found to be effective. We also found that the deception module benefited from much smaller mini-batch sizes (256) than the policy module (2048). Due to differences in scale between the role prediction loss and the action/observation prediction loss, we multiplied action/observation prediction losses by loss scale.


extfieldsofstudy:
Computer Science
Engineering

s2fieldsofstudy:
Computer Science
Medicine

text:
Multiple EffNet/ResNet Architectures for Melanoma Classification

Melanoma is the most malignant skin tumor and usually cancerates from normal moles, which is difficult to distinguish benign from malignant in the early stage. Therefore, many machine learning methods are trying to make auxiliary prediction. However, these methods attach more attention to the image data of suspected tumor, and focus on improving the accuracy of image classification, but ignore the significance of patient-level contextual information for disease diagnosis in actual clinical diagnosis. To make more use of patient information and improve the accuracy of diagnosis, we propose a new melanoma classification model based on EffNet and Resnet. Our model not only uses images within the same patient but also consider patient-level contextual information for better cancer prediction. The experimental results demonstrated that the proposed model achieved 0.981 ACC. Furthermore, we note that the overall ROC value of the model is 0.976 which is better than the previous state-of-the-art approaches.

Introduction
Skin cancer is the most widespread cancer diagnosed in the world. Moreover, the mortality rate of skin cancer is relatively high compared to other cancers. Specifically, melanoma is responsible for 75% of skin cancer deaths, despite being the least common skin cancer. Melanoma, usually called "malignant melanoma", is a malignant tumor that results from changes in melanocytes. Although melanoma is a typical skin disease, it can also occur in mucous membranes and internal organs. In recent years, the morbidity and mortality of melanoma have been increasing year by year, and it currently has no suitable treatment methods other than early surgical resection. Therefore, the early diagnosis of melanoma is particularly important. Hence it is necessary to get to know at the earliest whether the symptoms of the patient correspond to cancer or not.
There has been a lot of work published in the domain of skin cancer classification using deep learning and computer vision techniques. The works used a lot of different approaches including classification only, segmentation and detection, image processing using different types of filters etc. For example, Almansour et al. [1] developed an algorithm for the classification of melanoma by using k-means clustering, and Support Vector Machine (SVM). Abbas et al. [2], and Capdehourat et al. [3] separately used AdaBoost MC to classify skin lesions. Also, Isasi et al. [4] developed an algorithm for the diagnosis of melanoma. Past research on machine learning methods and their applications in pattern recognition systems were restricted to the transformation of a raw input into a representation of important handmade feature vectors which then, could be fed to a classifier for the classification and detection of patterns. However, in the past recent years with the exponential rise in computation power and a large amount of data, deep learning techniques have become popular among researchers. For instance, Esteva et al. [5] made a breakthrough on skin cancer classification by a pre-trained GoogleNet Inception v3 CNN model to classify 129,450 clinical skin cancer images including 3,374 dermatoscopic images. In addition, Yu et al. [6] developed a convolutional neural network with over 50 layers on ISBI 2016 challenge dataset for the classification of malignant melanoma. However, dermatologists evaluate every one of a patient's moles to identify outlier lesions that are most likely to be melanoma. Existing deep learning methods have not adequately considered this clinical frame of reference. Hence, we use ResNet and EffNet for better skin cancer classification.
As for the advantages of choosing ResNet model and EffNet model，ResNet can solve the problem that the effect of the deep network is decreased. The ResNet network can converge faster, and the residual structure can solve the problem of network degradation and gradient disappearance and explosion. EffNet is an open-source library that uses a new compound model scaling method and leverages recent progress in AutoML to improve neural network scaling techniques. By comparing and analyzing the experimental results, it can be found that the overall ROC value of the model is 0.976. This paper aims at realizing a deep learning approach for a multi-class classification by applying state-of-the-art pre-trained ResNet and EffNet on public skin cancer databases and FNN on patient-level contextual information which can yield higher diagnostic accuracy. Better detection of melanoma can positively impact millions of people.
The main contributions of this work can be summarized as follows: 1. The model makes a better prediction on skin cancer classification by comparing the results of ResNet and EffNet.
2. The model uses images within the same patient and patient-level contextual information for further providing assist on cancer prediction.

Model Formulation
The overall framework of the proposed model is shown in Fig. 1. The text and image data have been preprocessed and divided into the training set and the test set. The transfer learning of ResNet and EffNet is done before model training. For the image part, the image information is trained through ResNet50 and EffNet, and the model which has the best ROC has been obtained through comparison. Furthermore, FNN is used to process the text data. It should be noted that we also use the K-fold method to process non-overlapping data sets, and finally get the prediction model. The model can give the final prediction result of "Yes" or "No" for the input image and text information.

Transfer Learning
Transfer learning in computer vision is the process of using a pre-trained model on a new dataset of images. This is useful when the data set is not particularly larger. ML models thrive when there is a lot of data available to train on, but in many scenarios, obtaining large datasets is quite difficult. The pre-trained model is trained on a greater dataset, and the weights are preserved. The trained layers of the model are used as the basis for our new model. We use the ResNet model and EffNet model to predict whether or not a lesion is a malignant or benign tumor.

The Structure of ResNet50
Residual Networks (ResNet) [7] are deep convolutional networks where the basic idea is to skip blocks of convolutional layers by using shortcut connections. The basic blocks named "bottleneck" blocks follow two simple design rules: (i) for the same output feature map size, the layers have the same number of filters; (ii) the number of filters is doubled if the feature map size is halved. ResNet with 50 layer network architecture that is used for classification is shown in Fig. 2. The portrayal strategy of arranged deep neural frameworks resembles that of a multilayer energy process. The image is used ad data to transmit layer by layer until the point when the moment that the characterizes result.
At the primary layer, ResNet50 utilizes 7x7 convolutions with stride 2 to downsample the contribution by the request of 2, like the pooling layer. At that point, it's trailed by three characters obstructs before down-sampling again by 2. The ownsampling layer is additionally a convolution layer, yet without the personality association. It proceeds with like that for a few layers profound. The last layer is normal pooling which makes 1000 element maps for ImageNet information, and nourished into softmax layer straightforwardly; thus it's completely convolutional and at the end, we obtain the classification for the image that belongs to which class.
The activation function which is used in convolutional layers is rectified linear unit (Relu) which is a non-linear activation function.
( ) max( , 0) f y y = (1) At every readiness organize solitary center points are either dropped out of the net with probability p; therefore a lessened framework is left. Drawing nearer and dynamic edges to a dropped-out center point are moreover emptied. The dropout esteem is set to 0.5 that implies those hubs which have esteem under 0.5 are precluded from the preparation arrangement. At long last, the softmax work is utilized for the last expectation of the yield class.
The softmax function finds out the probabilities of every target class over all possible target classes. Later the processed probabilities are helpful for choosing the target class for the given sources of info. It ranges from 0 to 1 and the total of the extensive number of probabilities are proportional to one.

The Structure of EfficientNet
Convolutional neural networks are usually developed under a fixed resource budget. If more resources are available, they will be expanded to obtain better accuracy. For example, the network depth, network width, and input image resolution can be increased. However, it is very difficult to manually adjust the depth, width, and resolution to enlarge or reduce. When the amount of calculation is limited, it is difficult to determine which one to enlarge or reduce. In other words, the combined space is too large. Manpower cannot be exhausted. Based on the above background, this paper proposes a new model scaling method, which uses a simple and efficient composite coefficient to enlarge the network from the three dimensions of depth, width, and resolution, instead of arbitrarily scaling the dimensions of the network like traditional methods. Based on neural structure search technology, an optimal set of parameters (composite coefficients) can be obtained. As can be seen from the figure below, EfficientNet [8] is not only much faster than other networks, but also has higher accuracy.

Fig. 3 Multiple EffNet model
By enlarging the basic model of EfficientNet, a series of EfficientNet models are obtained. This series of models defeated all previous convolutional neural network models in terms of efficiency and accuracy. In particular, EfficientNet-B7 obtained the top-1 accuracy rate of 84.4% and the top-5 accuracy rate of 97.1% on the ImageNet data set. And compared with other models with the highest accuracy at the time, the size was reduced by 8.4 times and the efficiency was increased by 6.1 times. And through transfer learning, EfficientNet reached the most advanced level at the time on multiple well-known data sets.
In this case, we choose the basic network model EfficientNet-B0 in the EfficientNet series. When the model is trained on the ImageNet dataset, it contains a total of 5,330,564 parameters, of which 5,288,548 parameters are required for gradient descent training. The parameters that do not need to be trained are 42016 in the mean and variance in the Batch Normalization layer. The core structure of the network is the mobile inverted bottleneck convolution (MBConv) module, which also introduces the attention thought of the compression and excitation network (Squeeze-and-Excitation Network，SENet). The highest accuracy rate at the time was achieved on the ImageNet data set.
The mobile flip bottleneck convolution is also obtained through the neural network architecture search. The module structure is similar to the depthwise separable convolution. The mobile flip bottleneck convolution first performs a 1*1 point-by-point convolution on the input and based on the expansion ratio (expand ratio). Change the output channel dimension (for example, when the expansion ratio is 3, the channel dimension will be increased by 3 times. But if the expansion ratio is 1, then directly omit the 1*1 point-by-point convolution and subsequent batch normalization and activation functions). Then proceed to the depthwise convolution of k*k. If compression and excitation operations are to be introduced, the operations will be performed after deep convolution. Then restore the original channel dimension at the end of 1*1 point-by-point convolution. Finally, the drop connects and skip connection of the input are performed. This approach is derived from the paper "Deep networks with stochastic depth", which gives the model a random depth and shortens the model training needs Time, improve the model performance (note that in EfficientNet, only when the same mobile flip bottleneck convolution recurs, will the connection deactivation and the input skip connection be performed, and the deep convolution step Long becomes 1), connection inactivation is an operation similar to random dropout, and an identity jump is added at the beginning and end of the module. Note that after each convolution operation in this module, batch normalization is performed, and the activation function uses the Swish activation function. The network is divided into 9 stages in total. The first stage is an ordinary convolutional layer with a convolution kernel size of 3x3 and a step size of 2 (including BN and activation function Swish). Stage2~Stage8 are all repeatedly stacking the MBConv structure (last The Layers in a column indicates how many times the Stage repeats the MBConv structure), and Stage9 consists of a common 1x1 convolutional layer (including BN and activation function Swish), an average pooling layer and a fully connected layer. Each MBConv in the table will be followed by a number 1 or 6, where 1 or 6 is the magnification factor n, that is, the first 1x1 convolutional layer in MBConv will expand the channels of the input feature matrix by n times, where k3x3 or k5x5 Represents the size of the convolution kernel used by Depthwise Conv in MBConv. Channels represent the Channels that output the feature matrix after passing through the Stage.

Combination of FNN and CNN
We use the ResNet50 model mentioned above to process the images in the dataset and use the FNN model to process csv data in the dataset.
And connecting the results processed by the ResNet50 model and the FNN model for classification shown in Fig. 4.

Fig. 4 Combination of FNN and ResNet
In addition, we also use the EffNet model mentioned above to process the images in the dataset and use the FNN model to process csv data in the dataset.
And also connecting the results processed by the EffNet model and the FNN model shown in Fig. 5 for classification.

Fig. 5 Combination of FNN and EffNet
In the ResNet model and EffNet model, we use the BatchNorm1d function to perform batch standardization operations on small batches of input. And fully connected layer plus dropout layer to prevent the model from overfitting and improve model generalization ability.

GroupKFold
The same group will not appear in two different folds (the number of distinct groups has to be at least equal to the number of folds).
We're using patient_id for our grouping column: there are multiple patients with multiple images taken, so we need to be careful with that. The realization idea of GroupKFold is shown in Fig. 6.

Experiments
All experiments are conducted on a computer with 2.60GHz Intel i7-9750H processor and 16GB RAM under Windows 10 operating system. The program codes of data preprocessing and graphs modeling are written by Python 3.7.

Datasets
The datasets are taken from the Society for Imaging Informatics in Medicine (SIIM), as the leading healthcare organization for informatics in medical imaging, the SIIM 's mission is to advance medical imaging informatics through education, research, and innovation in a multi-disciplinary community. SIIM is joined by the International Skin Imaging Collaboration (ISIC), an international effort to improve melanoma diagnosis. The ISIC Archive contains the largest publicly available collection of quality-controlled dermoscopic images of skin lesions.
The details of the datasets utilized in the experiments are illustrated as follows.

Fig. 7 Data Distribution
The images are provided in DICOM format. This can be accessed using commonly available libraries like pydicom, and contains both image and metadata. It is a commonly used medical imaging data format.
Images are also provided in JPEG and TFRecord format (in the jpeg and tfrecords directories, respectively). Images in TFRecord format have been resized to a uniform 1024x1024. Metadata is also provided outside of the DICOM format, in CSV files.  This is a very important topic in this classification problem, as the 2 classes we are dealing with are highly imbalanced, with 98% of the data being benign and only 2% of the data being malignant. This is also the kind of problem where you don't want to have False Negatives. It's worse to tell a patient they don't have cancer when they actually do, than to tell they do have it and they actually don't. So, having balanced classes is crucial. This experiment takes an approach to solve this problem: oversampling or undersampling. The oversampling is, of the minority class, increasing the number of images through augmentations. In the experiment, we want to eliminate the influence of illumination, size, contrast and saturation on recognition. We use a library that goes hand in hand with PyTorch and it's easily used to augment data. 4. Further, we randomly convert the image to grayscale with a probability of p (default 0.1). As you may have noticed, all images are for light skin colors, so no preprocessing in this area is required. However, hair removal might be a good augmentation that will help the model perform better. We have realized the hair removal method with the help of CV2. The following is the effect achieved.  It seems like people have a lot of issues with the torso, and after that the extremities of the body (upper/lower). We have about 100-200 cases of cancer in the mouth or genitalia (the areas with the lowest rate of cancerous growth) and the palms and soles also are safe (probably because they are not exposed to any external sources in the dayto-day life of the person).
One could indeed fathom that the torso is frequently exposed either during the occasional workout or the occasional swim, or in some cases, the occasional extreme adventure. It also could be that the torso was exposed to UV light (which is the cause of melanoma) in highly populated regions (where pollution allows the sun's UV rays to come in). In addition to the picture, other attributes of the patient are also very important, such as age, location, and gender; so, we should transform all categorical features numerically. After doing this, the data preprocessing is all completed.

Confusion matrix
A confusion matrix contains information about actual and predicted classification done by a classification system. The performance of such systems is commonly evaluated using the data in the matrix. The confusion matrix in the melanoma classification problem is shown in Fig. 14.   Fig. 14 Confusion matrix in a melanoma classification problem • a is the number of people who did not have cancer and were also predicted to not have cancer • b is the number of people who did not have cancer but the prediction says they do • c is the number of people who have cancer but the predictions say they don't • d is the number of people who have cancer and were also predicted to have cancer.
A few standard terms are defined in the matrix: accuracy, true positive rate, false positive rate, true negative rate, false negative rate and precision. True positive rate is the proportion of positive cases that are properly identified and can be calculated using the equation: The false positive rate is the proportion of negative cases that were incorrectly classified as positive, and calculated with the equation: b False positive rate ab The true negative rate was defined as the proportion of negatives cases which are classified correctly, and is calculated using the equation.
a True negative rate ab The dales negative rate is the proportion of positive cases that were incorrectly classified as negative, and are calculated using the equation: Fig. 14 shows that there are four possible outputs which represent the elements of a 2x2 confusion matrix or a contingency table. If the sample is benign and it is classified as benign, it is counted as a true positive (TP); if it is classified as malignant, it is considered as a false negative (FN) or Type II error. If the sample is malignant and it is classified as benign, it is considered as true negative (TN); if it is classified as benign, it is counted as false positive (FP), false alarm or Type I error. As we will present in the next sections, the confusion matrix is used to compare the classification performance of different models.

The area under the ROC curve (AUC)
AUC (area under the curve) is the area under the ROC curve. The calculation formula of AUC area is shown in Eq. (7).
( ) We need to use the two variables defined above to build ROC diagram: False positive rate (FPR), True positive rate (TPR). If you take FPR as the x-axis and TPR as the y-axis, you can get a coordinate system shown in Fig. 15. The dashed line in Fig. 15 is our hypothetical prediction result. The ROC curve drawn is generally above the dotted line, because the result is generally better than the average. The bold curve in the figure is the ROC curve, and the area under the ROC curve is the AUC.
We can use the following explanation to understand the quality of the prediction results. Point (0,1), that is, FPR=0, TPR=1. According to the formula mentioned above, if FPR=0, then FP=0, which means there are no false positives. If TPR=1, then FN=0, which means there is no false counterexample. This is the perfect situation; all predictions are correct. The benign is predicted to be benign, and the malignant is predicted to be malignant, and the classification is 100% correct. This reflects the significance of FPR and TPR, so we hope that the smaller the FPR, the better, and the larger the TPR, the better. Point (1,0), that is, FPR=1, TPR=0. This point forms a contrast with the point (0,1), just the opposite. So, this is the worst-case scenario, because all predictions are wrong. Point (0,0), that is, FPR=0, TPR=0. Then FP and TP=0. So, the significance of this point is that all samples are predicted to be malignant. In other words, no matter what sample is given to me, it is predicted to be malignant. Point (1,1), that is, FPR=1, TPR=1. This point is in contrast with the point (0,0). The meaning of this point is to predict all samples as benign.
After understanding these four points, we can know that the closer a point is to the upper left corner, the better the prediction effect of the model. The larger the value of AUC, the higher the prediction accuracy of the classification model. If it can reach the upper left corner, which is the point (0,1), that is a perfect result.

Experiment 1: Comparison between ResNet and EffNetB6
We apply ResNet and EffNetB6(fine-tune version) to the training set and validation set respectively, and compare the convergence speed, accuracy and recall rate of the two CNN models. Fig.16 and Fig.17 show the training curve of ResNet and EffNetB6. In the training set, by comparing the curve of the loss function, it can be observed that EffNetB6 starts faster and converges faster; in the same number of iterations, the accuracy of EffNetB6 is higher, and it can reach 0.72 in the first iteration; in the recall rate of benign samples, the performance of EffNetB6 is slightly better than ResNet, but in the recall rate of malignant samples, there is a significant difference between EffNetB6 and ResNet, the performance of EffNetB6 is obviously better. But at the same time, we also notice that although EffNetB6 has the characteristics of faster convergence, higher accuracy and recall in the training set, EffNetB6 is far less stable than ResNet in the validation set, which will affect the reliability of EffNetB6 in practical application to a certain extent. For this problem, we make the following assumptions: (1) this is due to the overfitting problem in the validation set; (2) because we choose to freeze the model parameters (fine tune) and generalize them directly to our application. Therefore, because we find that the performance of the model is not ideal, based on assumptions, we realize that this is because the weight of the image network is not suitable for this kind of image classification. Because of the better performance of EffNetB6 in the above experiments, we choose EffNetB6 as the follow-up experimental object. We set the EffNetB6 layer as trainable to train the whole model for better performance. From the experiment without opening the frozen layer, we observed that EffNetB6 is better and faster than ResNet. After opening the frozen layer in EffNetB6, the training effect is obviously better, and the performance on the verification set is more stable, which confirms the previous assumptions. Even in the recall rate of malignant samples, EffNetB6 performs better in the validation set. Although the convergence rate is much slower after the frozen layer is opened, the possible reason is that the training features are more accurate after the frozen layer is opened. Fig. 19 confusion matrix of final model (all layers of EffNetB6 are opened and dropout is added to prevent overfitting) From the confusion matrix (shown in Fig. 19), the proportion of false positive samples was 0.5%, and the proportion of false negative samples was 2.3%. The TPR was 0.830, FPR was 0.00579, and ROC was 0.976. The model performed well and had good accuracy and reliability.

Experiment 2: Comparison between CNN only and CNN+FNN
Traditional skin cancer classification only uses convolutional neural network analysis of photos. However, other patient information is also particularly important for cancer diagnoses, such as age, gender, and tumor location. Therefore, we used a combination of CNN and FNN, and we compared the performance of using CNN alone to analyze photos and using a combination of CNN and FNN to analyze photos and discrete attributes, respectively.
It can be concluded from the performance comparison graph that the recall rate of cancer judgments has been improved after adding an FNN for analyzing discrete attributes. It is reflected in the training set and the validation set.

Conclusion
This paper proposed a melanoma classification model based on EffNet and ResNet to achieve better use of patient information and improve the diagnostic accuracy. Our model not only uses images of suspected tumors for training, but also takes patientlevel context information into account. Meanwhile, this paper compares the advantages of EffNet and ResNet convolutional neural networks in terms of the ROC value of the entire train data. The experimental results have shown that the model is superior to the existing methods in accuracy.
In the future, this work will focus on optimizing datasets and improving models. For example, more clinical patient information and the priority of these characteristics can be considered in the model. The past medical history and living habits of patients should be taken into account in the model. Furthermore, ResNet and EffNet can be combined for training in a large ensemble model to obtain a better performance in comparison with the current work.


A GAN-BPNN-Based Surface Roughness Measurement Method for Robotic Grinding

Existing machine vision-based roughness measurement methods cannot accurately measure the roughness of free-form surfaces (with large curvature variations). To overcome this problem, this paper proposes a roughness measurement method based on a generative adversarial network (GAN) and a BP neural network. Firstly, this method takes images and curvature of free-form surfaces as training samples. Then, GAN is trained for roughness measurement through each game between generator and discriminant network by using real samples and pseudosamples (from generator). Finally, the BP neural network maps the image discriminant value of GAN and radius of curvature into roughness value (Ra). Our proposed method automatically learns the features in the image by GAN, omitting the independent feature extraction step, and improves the measurement accuracy by BP neural network. The experiments show that the accuracy of the proposed roughness measurement method can measure free-form surfaces with a minimum roughness of 0.2 μm, and measurement results have a margin of 10%.

Introduction
Grinding is a widespread but essential process in manufacturing products. It is widely used in aerospace, automobile manufacturing, rail transit and other industries [1]. Traditionally, grinding is done by manual operation or multi-axis CNC machine tools. The former is time-consuming and labor-intensive, while the latter is limited by operating space. Recently, robotic belt grinding has become an alternative due to its low cost, efficiency and large operating space [2]. For small workpiece, the robot generally uses appropriate contact force to ensure the stability of the workpiece [3]. The robot holds the workpiece to complete the grinding operation and uses its dexterity to move to the best inspection position on the surface of the workpiece [4]. Multiple sampling points are planned according to the curvature of the workpiece surface, and the robot moves to each sampling point in turn to complete the surface image sampling.
As an important indicator of the quality of machined surfaces, surface roughness has a significant impact on product life and reliability [5]. The traditional probe-based measurement method [6][7][8][9] uses a probe that slides over the surface of the workpiece for measurement. This method has been proven to be highly reliable and highly accurate. However, it has several limitations, such as long measurement times, high environmental requirements and complex steps.
With the development of optical technology, a variety of noncontact measurement methods has been developed to address the limitations of the contact measurement method. For example, optical microscopy, confocal laser scanning microscopy and white light interferometry are used to measure surface roughness, but these methods are mainly used in laboratories, because their high cost and complicated operation are not suitable for real industrial production. Currently, the most popular noncontact measurement method feature extraction step. Finally, the BP neural network represents the correlation between the GAN discriminant value and the roughness (Ra).

Generative Adversarial Network
A GAN consists of two parts: the generative and discriminative networks, which use an adversarial game between generators and discriminators to achieve self-supervised learning. The main difference from the traditional network model is that the data training process contains both consistent and adversarial data. The generators and discriminators each optimize in different directions and form a competitive relationship with each other, but they also depend on each other to form a unified whole. In the adversarial training mode, the generator no longer learns directly from the training dataset, but learns iteratively in an indirect way through the optimization directions output by the discriminator, generating pseudosamples to mix the spurious with the genuine. A GAN computes faster and has greater expansion flexibility than a traditional network. In this paper, we propose using a GAN to discriminate the roughness of the surface of workpieces using the structure shown in Figure 1.
network play with each other to automatically learn the features in the image the feature extraction step. Finally, the BP neural network represents the corr tween the GAN discriminant value and the roughness (Ra).

Generative Adversarial Network
A GAN consists of two parts: the generative and discriminative networks, an adversarial game between generators and discriminators to achieve self-s learning. The main difference from the traditional network model is that the da process contains both consistent and adversarial data. The generators and disc each optimize in different directions and form a competitive relationship with but they also depend on each other to form a unified whole. In the adversari mode, the generator no longer learns directly from the training dataset, but le tively in an indirect way through the optimization directions output by the disc generating pseudosamples to mix the spurious with the genuine. A GAN comp and has greater expansion flexibility than a traditional network. In this paper, w using a GAN to discriminate the roughness of the surface of workpieces using ture shown in Figure 1.

Roughness
Ra Discriminator Pseudo images In other words, the loss function of GAN is as follows: The disadvantages of GAN: (1) GAN training is unstable, and the training degree of generator and dis should be carefully balanced.
(2) When the generator learns that some features of the real data successfu the discriminator, it will not update. As a result, the generated samples lack div a collapse mode occurs in GAN.
(3) When the overlap between the real and generated distributions is neg JS divergence between the real and generated distributions is minimized to log2. When the discriminator is optimal, the loss of the minimized generator is to log2, and G is no longer updated, resulting in the disappearance of the gen dient. p z (z) represented a prior on input noise variables, G(z) was generator of GAN, and D(x) represented the probability that x comes from real images. GAN's goal is training D(x) to maximize the probability of assigning the correct label to both training real samples and pseudosamples from G(z), at the same time training G(z) to minimize log(1 − D(G(z))). In other words, the loss function of GAN is as follows: The disadvantages of GAN: (1) GAN training is unstable, and the training degree of generator and discriminator should be carefully balanced.
(2) When the generator learns that some features of the real data successfully deceive the discriminator, it will not update. As a result, the generated samples lack diversity, and a collapse mode occurs in GAN.
(3) When the overlap between the real and generated distributions is negligible, the JS divergence between the real and generated distributions is minimized to a constant log2. When the discriminator is optimal, the loss of the minimized generator is also closer to log2, and G is no longer updated, resulting in the disappearance of the generator gradient.
In order to overcome the above shortcomings, the generator and discriminator network structures of the original GAN network are redesigned. In this paper, a deep convolutional neural network is used as the network structure of the generator and discriminator: The structure of the generative network is shown in Figure 2, and the corresponding discriminator network structure is shown in Figure 3.
Machines 2022, 10, x FOR PEER REVIEW 4 of convolutional neural network is used as the network structure of the generator and di criminator: The structure of the generative network is shown in Figure 2, and the correspondin discriminator network structure is shown in Figure 3.  From Equation (1), it can be seen that the objective of the discriminator is to obta the maximum value of ( ) , V G D . Then, it is necessary to maximize ( ) D x , which mea maximizing the probability that real samples and generated samples will be assigned th correct label. At the same time, the generator which means minimizing the difference between the generated samples and the true sam ples. Therefore, the value function of GAN can be decomposed into two optimizatio problems: (1) Fix the generator G , train the discriminator D , so that it can maximize the corre determination of whether the sample is from the real sample or from the sample generate by G . The objective function of D is, Finding the derivative of Equation (2) with respect to ( ) D x , such that its derivativ 0 gives, (3), the optimal discriminator is obtained as follows:  convolutional neural network is used as the network structure of the generator and dis criminator: The structure of the generative network is shown in Figure 2, and the correspondin discriminator network structure is shown in Figure 3.  From Equation (1), it can be seen that the objective of the discriminator is to obtai the maximum value of ( ) , V G D . Then, it is necessary to maximize ( ) D x , which mean maximizing the probability that real samples and generated samples will be assigned th correct label. At the same time, the generator which means minimizing the difference between the generated samples and the true sam ples. Therefore, the value function of GAN can be decomposed into two optimizatio problems: (1) Fix the generator G , train the discriminator D , so that it can maximize the correc determination of whether the sample is from the real sample or from the sample generate by G . The objective function of D is, Finding the derivative of Equation (2) with respect to ( ) D x , such that its derivativ 0 gives, (3 Rewriting Equation (3), the optimal discriminator is obtained as follows:  From Equation (1), it can be seen that the objective of the discriminator is to obtain the maximum value of V(G, D). Then, it is necessary to maximize D(x), which means maximizing the probability that real samples and generated samples will be assigned the correct label. At the same time, the generator G is trained to minimize log(1 − D(G(z))), which means minimizing the difference between the generated samples and the true samples.
Therefore, the value function of GAN can be decomposed into two optimization problems: (1) Fix the generator G, train the discriminator D, so that it can maximize the correct determination of whether the sample is from the real sample or from the sample generated by G. The objective function of D is, Finding the derivative of Equation (2) with respect to D(x), such that its derivative 0 gives, Rewriting Equation (3), the optimal discriminator is obtained as follows: (2) Fix the discriminator D and train the generator G, such that L = max minimizes the difference between the generated samples and the real samples. The objective function of G is given by, Substituting Equation (4) into Equations (2) and (5), we obtain, Machines 2022, 10, 1026

of 17
Introducing two important similarity metrics KL divergence (Kullback-Leibler divergence) and JS divergence (Jensen-Shannon divergence), Equation (6) is rewritten as, From Equation (7), it can be seen that fixing the discriminator, the goal of training the generator is to have a JS dispersion of 0 between P g and P data , i.e., P g = P data . At this point, the discriminator D * (G) = 1 2 . Thus, the equilibrium between generator and discriminator is realized by reciprocally training the GAN.

Measurement of Surface Roughness of Workpieces
In this method, a neural network is used to discriminate the surface roughness of a grinded workpiece. A small number of data images are used to train a generative adversarial network (GAN) to generate the required large dataset for training the discriminative network, which outputs the surface roughness measurements of the workpiece.

Image Acquisition and Data Preprocessing Methods Subsection
The radius of the curvature and roughness of the workpiece surface directly determine the refractive index of light, a factor which affects the imaging effect of the workpiece surface.
Therefore, 72 images of workpieces with different curvature radii and roughness were collected in this study. The 12 images with the roughest and smoothest surfaces were selected as the original training dataset for the GAN generator, with 6 smooth and 6 rough images, as shown in Figure 4. However, the amount of data in the original dataset of 12 images was too small to be used directly for training the GAN discriminator, which would lead to severe overfitting of the discriminator. Therefore, the original images were first rotated and panned to generate a sufficient number of derived images (32,000 images were generated in the experiment). (2) and (5), we obtain,

Substituting Equation (4) into Equations
Introducing two important similarity metrics KL divergence (Kullback-Leibler divergence) and JS divergence (Jensen-Shannon divergence), Equation (6) From Equation (7), it can be seen that fixing the discriminator, the goal of training the generator is to have a JS dispersion of 0 between g P and data P , i.e., DG= . Thus, the equilibrium between generator and discriminator is realized by reciprocally training the GAN.

Measurement of Surface Roughness of Workpieces
In this method, a neural network is used to discriminate the surface roughness of a grinded workpiece. A small number of data images are used to train a generative adversarial network (GAN) to generate the required large dataset for training the discriminative network, which outputs the surface roughness measurements of the workpiece.

Image Acquisition and Data Preprocessing Methods Subsection
The radius of the curvature and roughness of the workpiece surface directly determine the refractive index of light, a factor which affects the imaging effect of the workpiece surface.
Therefore, 72 images of workpieces with different curvature radii and roughness were collected in this study. The 12 images with the roughest and smoothest surfaces were selected as the original training dataset for the GAN generator, with 6 smooth and 6 rough images, as shown in Figure 4. However, the amount of data in the original dataset of 12 images was too small to be used directly for training the GAN discriminator, which would lead to severe overfitting of the discriminator. Therefore, the original images were first rotated and panned to generate a sufficient number of derived images (32,000 images were generated in the experiment).

GAN-Based Surface Roughness Discrimination Method
The generative network attempts to create images that the discriminant network cannot distinguish from real images during the training process, while the discriminant network is equivalent to a binary classification network to distinguish between the real training images and the images created by the generator. In this way, the model parameters are continuously optimized by the adversarial training between the two networks. Over multiple iterations, the images created by the generative network become closer to the real images, and the discriminant network increasingly understands the meaning of the roughness represented by the images, and finally gives the roughness value.
The process of training the GAN is shown in Figure 5.

GAN-Based Surface Roughness Discrimination Method
The generative network attempts to create images that the discriminant network cannot distinguish from real images during the training process, while the discriminant network is equivalent to a binary classification network to distinguish between the real training images and the images created by the generator. In this way, the model parameters are continuously optimized by the adversarial training between the two networks. Over multiple iterations, the images created by the generative network become closer to the real images, and the discriminant network increasingly understands the meaning of the roughness represented by the images, and finally gives the roughness value.
The process of training the GAN is shown in Figure 5.
Step 1: The original datasets (surface roughness images) are classified in a binary fashion and labeled as "smooth" and "rough" according to a certain threshold.
Step 2: Real surface roughness images marked by the classification in step 1 are rotated and panned to expand datasets.
Step 3: The GAN is trained using the "rough" category data in the dataset in Step 2.
Step 4: The GAN is trained using the "smooth" category data in the dataset in Step 2.
Step 5: The image data is generated from the generation network trained in steps 3 and 4 to complete the data set expansion.
Step 6: The discriminative network is retrained for the expanded smooth and rough datasets generated by the generative network in step 5 to obtain the final roughness recognition network model.  Step 1: The original datasets (surface roughness images) are classified in fashion and labeled as "smooth" and "rough" according to a certain threshold.
Step 2: Real surface roughness images marked by the classification in step tated and panned to expand datasets.
Step 3: The GAN is trained using the "rough" category data in the dataset in Step 4: The GAN is trained using the "smooth" category data in the dataset i Step 5: The image data is generated from the generation network trained in and 4 to complete the data set expansion.
Step 6: The discriminative network is retrained for the expanded smooth an datasets generated by the generative network in step 5 to obtain the final roughnes nition network model.

Training Results
To teach the GAN the meaning of "rough," real smooth images are added data to train the discriminator along with the output of the generator, as shown i 6a. Figure 6b shows examples of the surface roughness of workpieces generated GAN.

Training Results
To teach the GAN the meaning of "rough," real smooth images are added as fake data to train the discriminator along with the output of the generator, as shown in Figure 6a. Figure 6b shows examples of the surface roughness of workpieces generated by the GAN. Step 1: The original datasets (surface roughness images) are classified in a binary fashion and labeled as "smooth" and "rough" according to a certain threshold.
Step 2: Real surface roughness images marked by the classification in step 1 are rotated and panned to expand datasets.
Step 3: The GAN is trained using the "rough" category data in the dataset in Step 2.
Step 4: The GAN is trained using the "smooth" category data in the dataset in Step 2.
Step 5: The image data is generated from the generation network trained in steps 3 and 4 to complete the data set expansion.
Step 6: The discriminative network is retrained for the expanded smooth and rough datasets generated by the generative network in step 5 to obtain the final roughness recognition network model.

Training Results
To teach the GAN the meaning of "rough," real smooth images are added as fake data to train the discriminator along with the output of the generator, as shown in Figure  6a. Figure 6b shows examples of the surface roughness of workpieces generated by the GAN.  The GAN is trained using the expanded datasets, and then the GAN is able to generate a series of smooth and rough images as the training set. The problem is treated as a binary classification problem using the sigmoid function as the output activation function and the binary cross-entropy as the loss function for training, with the rough images labeled '1' and the smooth images labeled '0'. In this method, the same network structure of the classifier with the discriminator of the GAN is used, as shown in Table 1. In the method described above, a dataset with only roughness Ra of 0.2, 0.4, 3.2, 6.3 and radius of curvature R of 50, 1000, and infinity are used as the training set. However, the dataset in the actual test contains more cases of roughness and radius of curvatures. A qualified classifier should be able to accurately distinguish the cases which are not encountered in the training set. Some of the actual test results are shown in Table 2, which visually reflects the classifier's scoring of images with different roughness levels. The smaller score means the smoother surface. The test results show that the method can accurately determine the untrained intermediate roughness (0.8 and 1.6 are the untrained cases), which indicates that the network can understand the concept of "roughness."

Discriminating Method of Workpiece Surface Roughness
Roughness (Ra) is a small-distance (usually less than 1 mm) peak-valley that forms from a microgeometric shape error of the surface of the part. As shown above, the real surface image can obtain a discriminant value, nonlinearly related to roughness Ra through the trained GAN. In order to establish a mapping relationship between the discrimination value of GAN and surface roughness Ra, this paper proposes a surface roughness Ra discrimination method based on the BP neural network, which is shown in Figure 7.

Discriminating Method of Workpiece Surface Roughness
Roughness (Ra) is a small-distance (usually less than 1 mm) peak-valley that forms from a microgeometric shape error of the surface of the part. As shown above, the rea surface image can obtain a discriminant value, nonlinearly related to roughness Ra through the trained GAN. In order to establish a mapping relationship between the discrimination value of GAN and surface roughness Ra, this paper proposes a surface rough ness Ra discrimination method based on the BP neural network, which is shown in Figure  7. The input layer of the BP neural network for image discriminant value and radius of curvature is denoted as x i (i = 1, 2), and the output surface roughness is denoted as z. The number of neurons in the hidden layer is s, denoted as y i (1, 2, · · · , s). The bias value of the hidden layer is γ, and the bias of the output layer is θ. w denotes the connection weights from the input layer to the hidden layer, v denotes the connection weights from the hidden layer to the output layer, while f y and f z are the activation functions of the hidden and output layers, respectively. The forward propagation process of the BP neural network is expressed as: Equation (8) completes the mapping from x i (i = 1, 2) to z. α denotes the target value of the output, and the error of the BP neural network is: The error back-propagation process of the BP neural network is implemented by minimizing the objective function by Equation (9). The weights and biases of each layer are adjusted for each propagation as follows: From Equation (8), it can be seen that the data transfer process between the layers is transformed and connected by the activation function. Meanwhile, in the error backpropagation process in the BP neural network in Equation (10), the derivatives of the activation function adjust the connection weights of each layer to reduce the error to the desired range. Considering the requirement that the activation function be continuously differentiable, the sigmoid function is chosen as the output layer activation function, and the tanh function is chosen as the output layer activation function. To ensure accuracy of roughness estimation, this study determines the number of neurons in the hidden layer according to the Hecht-Nelson method [29] because the hidden layer affects the stability of the network. The number of input layer neurons is n, while the number of hidden layer neurons is 2n + 1. That is, the number of hidden layer neurons in the network is 5. Networks with different numbers of neurons have also been tested, as shown in Table 3. After determining that there should be one hidden layer for the BP neural network model, the number of neurons in the hidden layer must be determined. For each neural network model in Table 3, the average error between the actual and predicted values were recorded in 10 training sessions. It is clear from Table 3 that as the number of neurons in the hidden layer increases, the accuracy does not necessarily follow. When the number of hidden layer neurons increases from 3 to 5, the average error decreases from 12.42% to 6.24%. However, as the number of hidden layer neurons continues to increase to 7, the average error increases from 6.24% to 14.63%. The 2-5-1 neural network structure has the smallest error, and so this network structure is used in the real-time surface roughness model in this study.

Experiments
In this study, a robot is used to hold the workpiece while grinding a complex, curved item. The robotic grinding platform is built as shown in Figure 8, consisting of a belt grinder, a lightweight robot with seven degrees of freedom and a camera. Surface roughness is a key feature of the surface texture of a workpiece because it has a significant impact on the service life and reliability of a mechanical product. This paper proposes a vision-based roughness measurement method for comprehensively considering the cost, usability, and efficiency. This method consists of three stages: image acquisition, self-supervised learning and result discrimination. A narrow-angle industrial camera is used to build a vision-based surface roughness measurement system, as shown in Figure 9. Surface roughness is a key feature of the surface texture of a workpiece because it has a significant impact on the service life and reliability of a mechanical product. This paper proposes a vision-based roughness measurement method for comprehensively considering the cost, usability, and efficiency. This method consists of three stages: image acquisition, self-supervised learning and result discrimination. A narrow-angle industrial camera is used to build a vision-based surface roughness measurement system, as shown in Figure 9. Surface roughness is a key feature of the surface texture of a workpiece because it has a significant impact on the service life and reliability of a mechanical product. This paper proposes a vision-based roughness measurement method for comprehensively considering the cost, usability, and efficiency. This method consists of three stages: image acquisition, self-supervised learning and result discrimination. A narrow-angle industrial camera is used to build a vision-based surface roughness measurement system, as shown in Figure 9. The narrow-angle Balser ace 2 industrial camera meets the needs of this method for surface roughness measurement of curved workpieces. Its detailed parameters are shown in Table 4. Table 4. Parameters of Balser ace 2 narrow angle camera.

Resolution ratio
2.3 MP Figure 9. Surface roughness measurement system based on vision.
The narrow-angle Balser ace 2 industrial camera meets the needs of this method for surface roughness measurement of curved workpieces. Its detailed parameters are shown in Table 4. Balser ACE 2 samples images of standard grinding surface roughness contrasts as shown in Figure 10a, and images of standard milled surface roughness contrasts as shown in Figure 10b. There is no significant difference between images Ra 0.2 and Ra 0.1. The camera can distinguish surfaces with surface roughness Ra greater than 0.2. The image samples in Figure 10 are added to the training set of the proposed GAN to enrich the samples and increase the measurement accuracy of the model. Balser ACE 2 samples images of standard grinding surface roughness contrasts as shown in Figure 10a, and images of standard milled surface roughness contrasts as shown in Figure 10b. There is no significant difference between images Ra 0.2 and Ra 0.1. The camera can distinguish surfaces with surface roughness Ra greater than 0.2. The image samples in Figure 10 are added to the training set of the proposed GAN to enrich the samples and increase the measurement accuracy of the model. The blade-grinding process is divided into three processing stages according to the requirements of the grinding process: rough grinding, semi-finishing grinding and finishing grinding. The rough grinding stage removes a large amount of the material using the grinding parameters belt line speed 5 m/s, feed speed 3 mm/s, belt mesh 120 and contact force 15 N. The semi-finishing stage removes a small amount of material with the grinding parameters of belt line speed 5 m/s, feed speed 3 mm/s, belt mesh 320 and contact force 10 N. The finishing stage is mainly for finishing the surface of the workpiece with the grinding parameters belt line speed 5 m/s, feed speed 3 mm/s, belt mesh 320, and contact force 5 N. After each grinding stage, the surface roughness is measured and analyzed.
In this study, an industrial camera was used to take pictures of the workpiece surface under a specific light environment to measure the surface roughness conditions, as shown in Figure 11. In order to compare the measurement accuracy of different learning algorithms, ANN and CNN are added to train the surface roughness measurement model in this paper. The GAN-BPNN, ANN, CNN were used to judge the smoothness of the polished processing surface in Figure 11, as shown in Table 5. The proposed GAN-BPNN has The blade-grinding process is divided into three processing stages according to the requirements of the grinding process: rough grinding, semi-finishing grinding and finishing grinding. The rough grinding stage removes a large amount of the material using the grinding parameters belt line speed 5 m/s, feed speed 3 mm/s, belt mesh 120 and contact force 15 N. The semi-finishing stage removes a small amount of material with the grinding parameters of belt line speed 5 m/s, feed speed 3 mm/s, belt mesh 320 and contact force 10 N. The finishing stage is mainly for finishing the surface of the workpiece with the grinding parameters belt line speed 5 m/s, feed speed 3 mm/s, belt mesh 320, and contact force 5 N. After each grinding stage, the surface roughness is measured and analyzed.
In this study, an industrial camera was used to take pictures of the workpiece surface under a specific light environment to measure the surface roughness conditions, as shown in Figure 11. In order to compare the measurement accuracy of different learning algorithms, ANN and CNN are added to train the surface roughness measurement model in this paper. The GAN-BPNN, ANN, CNN were used to judge the smoothness of the polished processing surface in Figure 11, as shown in Table 5. The proposed GAN-BPNN has a margin of error of 10%. However, the prediction error of ANN and CNN were more than 10%. a margin of error of 10%. However, the prediction error of ANN and CNN were more than 10%. Figure 11. Roughness measurement based on machine vision. In order to measure blade surface roughness more accurately, this paper designs a sampling point distribution method based on constant chord height. The method realizes a dense distribution of sampling points in the region with a large curvature variation and a sparse distribution in the region with a small curvature variation, as shown in Figure 12.
(2) The baseline AE was obtained by connecting the end points of the curve.
(3) Determine whether there exists a point on the curve AE whose distance to the datum line is greater than  . If there is a point on AE , whose distance from the datum line AE is the farthest point is    In order to measure blade surface roughness more accurately, this paper designs a sampling point distribution method based on constant chord height. The method realizes a dense distribution of sampling points in the region with a large curvature variation and a sparse distribution in the region with a small curvature variation, as shown in Figure 12.  (1) Set a chord height ε.
(2) The baseline AE was obtained by connecting the end points of the curve.
(3) Determine whether there exists a point on the curve AE whose distance to the datum line is greater than ε. If there is a point on AE, whose distance from the datum line AE is the farthest point is n 0 , then delete the original datum line AE.
(4) Set n 0 to B and connect the AB and BE as the new datum line.
(5) Determine whether there exists a point on the curve AB whose distance to the datum line is greater than ε, and if there is a point on AB whose distance from the datum line AB is the farthest point is n 1 . Similarly, point n 2 on curve AB is obtained. Rewrite A, n 1 , B, n 2 , E as A, B, C, D, E. Repeat the above process until all points on the curve to the datum line position are less than the threshold ε. In this paper, five sampling points (A, . . . , E) marked in the figure were taken as examples to compare the roughness changes of blades before and after grinding, as shown in Figure 13. The robot planed the motion trajectory according to the position and curvature of the sampling points (20 mm × 20 mm area centered on this point), collecting images of the five sampling points in turn at the best angle, as shown in Figure 14. Repeat the above process until all points on the curve to the datum line posi less than the threshold  . In this paper, five sampling points (A,…, E) marked in th were taken as examples to compare the roughness changes of blades before an grinding, as shown in Figure 13. The robot planed the motion trajectory accordin position and curvature of the sampling points (20 mm × 20 mm area centered point), collecting images of the five sampling points in turn at the best angle, as sh Figure 14. The image data of the sampling points before and after polishing can obtain responding predicted value by the roughness discrimination model of the GAN + ral network, as shown in Table 6. It can be seen that the relative error between t dicted roughness and the real roughness (measured by SJ-210) has a margin of 10%, which meets the needs of industrial detection. The image data of the sampling points before and after polishing can obtain the corresponding predicted value by the roughness discrimination model of the GAN + BP neural network, as shown in Table 6. It can be seen that the relative error between the predicted roughness and the real roughness (measured by SJ-210) has a margin of error of 10%, which meets the needs of industrial detection.

Conclusions
In this paper, we propose a roughness evaluation method combining generative an adversarial network (GAN) and a BP neural network, which avoids the influence of surface image differences with different curvatures on the accuracy of roughness

Conclusions
In this paper, we propose a roughness evaluation method combining generative an adversarial network (GAN) and a BP neural network, which avoids the influence of surface image differences with different curvatures on the accuracy of roughness measurements. The method automatically learns features in the image by generators playing with discriminators, eliminating the independent feature extraction step. This does not merely shorten the prediction time but reduces the complexity of model training as well. Experiments show that the proposed method can measure a free-form surface with a minimum roughness of 0.2 µm, and measurement results have a margin of error of 10%. Since the proposed method does not require the operator to have the field knowledge of feature recognition, the method is easier to apply in a factory. However, the proposed roughness evaluation method using GAN takes a long time for model training due to its convolution process and depth structure characteristics. To overcome this problem, the adaptive model can be considered in future research to automatically adjust the hyperparameters.
Author Contributions: G.Z. was responsible for methodology and manuscript writing. C.L. was responsible for software. K.M. was responsible for experiments. F.N. was responsible for reviewing papers and providing funding support. H.L. provided resources. All authors have read and agreed to the published version of the manuscript.
Funding: This work is supported by Self-Planned Task (NO.SKLRS202204B) of State Key Laboratory of Robotics and System (HIT).