A new image encryption algorithm based on the OF-LSTMS and chaotic sequences

In this paper, a novel image encryption algorithm based on the Once Forward Long Short Term Memory Structure (OF-LSTMS) and the Two-Dimensional Coupled Map Lattice (2DCML) fractional-order chaotic system is proposed. The original image is divided into several image blocks, each of which is input into the OF-LSTMS as a pixel sub-sequence. According to the chaotic sequences generated by the 2DCML fractional-order chaotic system, the parameters of the input gate, output gate and memory unit of the OF-LSTMS are initialized, and the pixel positions are changed at the same time of changing the pixel values, achieving the synchronization of permutation and diffusion operations, which greatly improves the efficiency of image encryption and reduces the time consumption. In addition the 2DCML fractional-order chaotic system has better chaotic ergodicity and the values of chaotic sequences are larger than the traditional chaotic system. Therefore, it is very suitable to image encryption. Many simulation results show that the proposed scheme has higher security and efficiency comparing with previous schemes.

www.nature.com/scientificreports/ the pixel position while changing the pixel value, that is, permutation and diffusion are carried out at the same time, so the time consumption is less and the proposed encryption algorithm is efficient. The rest of this paper is organized as follows. In "The application of OF-LSTMS in the proposed algorithm" and "The 2DCML fractional-order chaotic system" sections the OF-LSTMS and the 2DCML fractional-order chaotic system are presented respectively. The proposed image encryption and decryption scheme are described in "The proposed image encryption and decryption algorithm" section. Simulation results and performance analyses are reported in "Performance analyses" section.

The application of OF-LSTMS in the proposed algorithm
LSTM structure is a chain structure with recurrent neural network module. It adopts matrix multiplication calculation method and gradient update parameter training method. LSTM structure consists of four parts: memory unit, forgetting gate, input gate and output gate 29,[46][47][48][49][50] . Image encryption algorithm requires reversibility and does not need to learn features of the sequence, so it is necessary to modify the structure of LSTM. Based on the structure of LSTM, we propose the Once Forward Long Short Term Memory Structure (OF-LSTMS), which uses XOR operation instead of traditional matrix operation. The OF-LSTMS can realize that the encrypted image be obtained after only one forward propagation after original image entering the algorithm. The OF-LSTMS is as shown in Fig. 1.
The operation used by the Neural Network (NN) structure of the traditional LSTM is matrix multiplication (MM), but it is not applicable in the image encryption task. MM cannot make the sum items (after being activated by MOD function) included in the output neurons be accurately restored to the input neurons during the decryption process, so the operation used by the NN structure of OF-LSTMS is XOR. The blue box NN in Fig. 1 (the part of The OF-LSTMS) shows the formal representation of NN structure and Fig. 2 shows the details of the XOR operation used by the NN structure of OF-LSTMS. As shown in Fig. 2, there are four original pixels, each of which contains 8-bit binary values. These pixels are divided into four groups of binary values, and XOR operation is performed with the same four sets of binary values (the values are from chaotic sequences). This operation process is called segmentation as shown in Fig. 2 (The part on the left which marked by green). In addition, the four groups of binary values in each rectangle in Fig. 2 come from the binary values of four different original pixels, which are finally calculated and then converted to decimal values. This operation process is called recombination as shown in Fig. 2 (The part on the right which marked by yellow).
The length of binary value used for recombination is 8 in order to ensure that no information is lost in the process of converting binary value to decimal value. Therefore, the number of neurons in the process of  where bitDic(X t ) is a multivalued function that the binaries of the input X t divide into 4 parts and each part has 2 bits. NN(X t , CS X t ) is a mathematical formula for the NN structure of the OF-LSTMS that contains 4 neurons. CS X t (from the chaotic sequences) is the parameters of the corresponding inputs at the current moment. bitCon(x bit 1 , x bit 2 , x bit 3 , x bit 4 ) is a function that combines 4 binaries of 2 bits into 8 bits and converts them to decimals. X NN t is the outputs of the NN structure of the OF-LSTMS. The function of the forgetting gate in the traditional LSTM is to determine which information inherited from the cell body at the previous moment to discard, which is determined by the input sequence at the current moment and the output at the previous moment. But it is not applicable in the image encryption task. The essence   www.nature.com/scientificreports/ of the image encryption task is to scramble the image, and the less original image information contains in the encrypted image, the better the encryption effect. Therefore, the OF-LSTMS proposed in this paper designs the forgetting gate to the increasing gate, as shown in blue box A in Fig. 3. The cell body information at the current moment is obtained by accumulating with Eq. (2) the inputs' parameters at the previous moment and the information inherited from the cell body at the previous moment chaotically.
where C t−1 is the information inherited from the cell body at the previous moment. CS X t−1 is the inputs' parameter at the previous moment. C temp t is the information at the current moment after the increasing gate. The function of the input gate in the traditional LSTM is to determine which information is added to the information at the current moment after the increasing gate, and it uses the multiplication operation. But the input gate in the OF-LSTMS uses add operation, which is due to the limitations in the image encryption. The input gate in the OF-LSTMS is shown in blue box I in Fig. 3 and can be represented by where C temp t is the information to be added to the information of the cell body at the current moment after the input gate.
The operation used in the OF-LSTMS is different from the operation used in traditional LSTM, and it uses XOR. As shown in blue box C in Fig. 3, it can be represented by where C t is the information of the memory unit in the cell body that will be used to encrypt the input sequence at the current moment.
The function of the output gate in the OF-LSTMS is to use the information in the cell body and additional chaotic sequences to encrypt the input sequence. It is shown in blue box O in Fig. 3, and can be represented by  . In this algorithm, the original image sequence enters into the OF-LSTMS in order to operate with different chaotic sequences. The memory information C t and the encrypted pixel value O t can be obtained at the same time. According to the memory information C t , the new position of the encrypted pixel can be obtained, that is, the encrypted pixel position and pixel value are determined at the same time. The synchronization of permutation and diffusion can improve the efficiency of encryption.
The 2DCML fractional-order chaotic system The proposed chaotic system. Comparison with the traditional logistic map, the fractional-order logistic map contains larger key space and more parameters. Zhang et al. 52 exhibited the features of the fractionalorder chaotic system in dynamical behaviors. The following iteration equation is obtained 52 : The parameters α, μ, r and the initial values x 0 of the fractional-order logistic system can be designed the secret keys.
Based on the fractional-order logistic map, the proposed system coupled by the neighborhood links of the 2DCML system 53 as follows: where i , j are the lattices ( 1 ≤ i, j ≤ L ) , ε is the coupling parameter ( 0 ≤ ε ≤ 1 ), n is the time index (n = 1,2,3, …) and f (x) is the fractional-order logistic map with the iteration equation is obtained as Eq. (6).
The new features of the proposed chaotic system in dynamical behaviors. To qualify the new features of the proposed system in dynamical behaviors mathematically, the bifurcation diagrams, the Lyapunov exponents, the space-amplitude diagrams and the patterns diagrams are widely analyzed theoretically in this section.
In the proposed system, the bifurcation diagrams are shown in Fig. 4c. In Fig. 4a the parameter μ in the traditional logistic map is (3.57, 4). In Fig. 4b the parameter μ in the fractional-order logistic differential map breaks the range of µ ∈ (3.57, 4) and the numerical range of chaotic sequence x n also breaks the range of (0,1). While an important phenomenon in Fig. 4c is that the proposed system not only retains the advantages of the www.nature.com/scientificreports/ fractional-order logistic map, but also its periodic windows are reduced substantially compared with the Fig. 4a,b and the gaps between bifurcation points vary closer. Due to the neighborhood coupling leading the instability of the possible periods of orbits, the times of period doubling bifurcations is misled and unobvious. Therefore, both the parameter range and the values of chaotic sequences of the proposed system are larger than the traditional logistic mapping and the fractional-order chaotic logistic system. In our encryption algorithm, the proposed chaotic system is selected to generate chaotic sequences, and the parameters are selected as secret keys. Any system holding chaotic behavior presented at least one positive Lyapunov exponent. In Fig. 5, comparing with the Lyapunov exponents of the fractional-order chaotic logistic system and the traditional logistic maps system, the positive interval of Lyapunov exponents of the proposed chaotic system is far greater than that of the former two systems. Therefore, the proposed chaotic system has strong chaotic characteristics and can generate better chaotic sequences. It is more suitable for encryption algorithm.
Because the fractional-order logistic map contains larger key space and more parameters comparing with the traditional logistic map, the proposed system contains more universality of chaos in space than the 2DCML system with the same parameter ε, which is shown in the space-amplitude plots as Fig. 6. In addition, the snapshot patterns shown in Fig. 7 indicate that the proposed system presents more complex chaotic resolutions than the 2DCML system. For example, Fig. 7f indicates that the same parameter ε which lead the proposed system www.nature.com/scientificreports/ in fully developed turbulence pattern, can only lead the 2DCML system in defect turbulence pattern which is shown in Fig. 7e.

The proposed image encryption and decryption algorithm
Without loss of generality, the images are employed to present the encryption scheme for simplicity. The corresponding encryption algorithm and decryption algorithm can be presented as follows.
Step 1. Generate the key sequence K and the initial values α ′ , µ ′ , x ′ , r ′ and ε ′ of the proposed system. The proposed algorithm utilizes a 160-bit secret key K, which is generated by the hash algorithm MD2. For source images, even if only one bit is changed, its hash value will change completely. By dividing the 160-bit secret key into 16-bit blocks (K i ), and the new initial values can be obtained by the following formulas:  www.nature.com/scientificreports/ where α , µ , x 0 , r and ε are the initial given values.
Step 2. The chaotic sequences are iterated according to Eq. (7), and the chaotic sequences used in the OF-LSTMS are determined.
The original image is transformed into a sequence, and each 4 pixels value is regarded as a subsequence.
The subsequences are encrypted in the OF-LSTMS according to "The application of OF-LSTMS in the proposed algorithm" section. Assume the size of the encrypted image is 512 × 512 and the image is divided into 65,536 pixel sequences with 4 pixels as a group. Then the shape of input data into the OF-LSTMS is [1,65,536,4].
Finally, the ciphered image is obtained. The encryption process is shown as Fig. 8.

Decryption algorithm.
The decryption process is contrary to the encryption process. Using the secret keys provided by the sender, the receivers decrypt the cipher image according to the contrary operations of the encryption algorithm. The decryption process is shown as Fig. 9.

Performance analyses
In order to evaluate the security of the proposed encryption algorithm, we undertake a series of statistical analysis on the encryption and decryption results, and show the analysis results in detail in this section.
Key space. Only if the key space is large enough, it can resist violent attacks. The secret keys include decimal parameters α, μ, r, ε and the initial value x 0 . The total key space is 10 80 if the accuracy of the computer is 10 16 . In the proposed encryption algorithm, because the total key space is more than 2 425 that the key space can satisfy the security requirements.
Key sensitivity. The characteristic of chaotic system is that the small change of initial value will lead to completely different chaotic sequences. We modify one of the parameter values, while the others remain unchanged. Simulation and analysis show that small changes in the key will lead to significant changes in the output, so the algorithm is very sensitive to the key. Figure 12 shows the results of the corresponding μ, α, r, x 0 and ε tests, respectively.

Histogram analysis.
Histogram analysis is an important image analysis method, which can reflect the frequency distribution of pixel values in the image. Figures 13 and 14 show that encrypted images have completely different histograms against the original images. It shows that the encrypted image has no relationship with the original image. Therefore, the proposed image encryption algorithm can resist histogram analysis attacks.
Differential attack. To evaluate the encryption algorithm's ability to resist differential attacks, we employ the unified average changing intensity (UACI) and the number of pixels change rate (NPCR) which are defined by  Table 1, which shows that the proposed encryption algorithm is very sensitive to a pixel change in the original image. Mean Squared Error (MSE) and Peak Signal to Noise Ratio (PSNR) can be used to test the efficiency of encryption and decryption 40,41 . We calculate the PSNR and the MSE of the original images and the encrypted images. It can be seen from Table 2 that the PSNR value is lower, while the MSE value is higher, which indicates that the image encryption process is more efficient. Similarly, we also calculate the PSNR and the MSE of the original images and the decrypted images. The results in Table 2 show that the PSNR value is higher and the MSE value is lower, indicating that the image decryption of algorithm is efficient too. www.nature.com/scientificreports/ Correlation analysis. An efficient encryption scheme should reduce the correlation between adjacent pixels in the ciphered image significantly. In order to test the image correlation, we randomly select 3000 pairs of adjacent pixels from the image to calculate the correlation coefficients of adjacent pixels in the vertical, horizontal and diagonal directions using the following equation: where x and y represent two adjacent pixels and S is the total number of adjacent pixels (x, y). E(x) is the expectation of x and D(x) is the variance of x, respectively. The pixels distribution of the plain images and the cipher images in three directions are shown in Figs. 15,16,17,18,19,20. From figures it shows that the points in the encrypted images are randomly distributed, and the correlation of the images are greatly reduced. Meanwhile, Table 3 lists the correlation coefficients of the encrypted images which values are almost close to 0.

Information entropy.
Information entropy is the most important criterion to evaluate the efficiency of an image encryption algorithm. We calculate the information entropy of the cipher images and the results are listed in Table 4. The entropy of encrypted images are close to 8, which proves the proposed scheme is sufficient to withstand entropy-based attacks.
Comparison with existing algorithms. In recent years, some researchers have combined chaotic maps with optimization methods, DNA coding, S-box and mathematical transformation to propose secure and effective image encryption schemes 1,4,25,40,41 . Some encryption algorithms have improved the efficiency of encryption by optimizing the permutation and diffusion process 16,18 . In this paper, a new efficient image encryption algorithm is proposed by combining the OF-LSTMS with the 2DCML fractional-order chaotic system. By using the OF-LSTMS, the pixel position is changed while the pixel value is changed, which realizes the synchronization of permutation and diffusion. At the same time, the 2DCML fractional-order chaotic system has better chaotic ergodicity than traditional chaotic system. Compared with some recent literature, it is clear that the proposed encryption algorithm is better in performance, as shown in Table 5.

Robustness analysis.
A feasible encryption algorithm needs the ability of anti-interference, and robustness is an important indicator 2,7,26,44 . We add Salt-and-Pepper noise with different intensities to the ciphertext images, and we also make different degrees of data loss to the ciphertext image at different locations. The decryption effects are shown in Fig. 21. The quantitative results of resisting noise and occlusion attacks are listed in  www.nature.com/scientificreports/      In addition, the 2DCML fractional-order chaotic system contains good features as larger key space, better chaotic sequences which it is more suitable for encryption algorithm. The parameters used in the input gate, the output gate and memory unit of the OF-LSTMS are initialized according to the proposed chaotic system generated, which are different from the traditional LSTM initialization method. The proposed encryption algorithm greatly reduces the time consumption and improves the efficiency of image encryption. The extensive simulated experimental results such as key sensitivity, correlation, NPCR, UACI, information entropy and robustness analysis prove that proposed algorithm is high security and efficiency for image encryption applications.     www.nature.com/scientificreports/


Optimization of Convolutional Neural Networks Architectures Using PSO for Sign Language Recognition

: This paper presents an approach to design convolutional neural network architectures, using the particle swarm optimization algorithm. The adjustment of the hyper-parameters and ﬁnding the optimal network architecture of convolutional neural networks represents an important challenge. Network performance and achieving efﬁcient learning models for a particular problem depends on setting hyper-parameter values and this implies exploring a huge and complex search space. The use of heuristic-based searches supports these types of problems; therefore, the main contribution of this research work is to apply the PSO algorithm to ﬁnd the optimal parameters of the convolutional neural networks which include the number of convolutional layers, the ﬁlter size used in the convolutional process, the number of convolutional ﬁlters, and the batch size. This work describes two optimization approaches; the ﬁrst, the parameters obtained by PSO are kept under the same conditions in each convolutional layer, and the objective function evaluated by PSO is given by the classiﬁcation rate; in the second, the PSO generates different parameters per layer, and the objective function is composed of the recognition rate in conjunction with the Akaike information criterion, the latter helps to ﬁnd the best network performance but with the minimum parameters. The optimized architectures are implemented in three study cases of sign language databases, in which are included the Mexican Sign Language alphabet, the American Sign Language MNIST, and the American Sign Language alphabet. According to the results, the proposed methodologies achieved favorable results with a recognition rate higher than 99%, showing competitive results compared to other state-of-the-art approaches.

Introduction
Deep neural networks have demonstrated their capacity to solve classification problems using a hierarchical model, millions of parameters, and learning with big databases. Convolutional neural networks (CNN) are a special class of deep neural networks that consist of several convolutions, pooling, and fully connected layers; this has proven to be a robust method for image or video processing, classification, and pattern recognition. In recent years CNN has attracted attention for achieving superior results in various applications in the computer vision domain, such as medicine, aerospace, natural language processing and robotics [1,2].
CNN are widely used in the field of industry, however, when designing CNN architectures, we face some challenges which include the high computational costs for information processing and finding the optimal CNN parameters (architecture) for each problem [3]. CNN architectures are made up of numerous parameters and, depending on their configuration, can generate a variety of classification results when applied to solve the same tasks; the setting of the hyper-parameter values is usually based on a random search, performing several tests or adjusting manually and this represents a complex search process. To solve this challenge, various researchers have proposed the implementation of evolutionary computation approaches to automatically design the optimal CNN architectures and to increase its performance [4,5]. In Sun et al. [6,7], an evolutionary approach is implemented to automatically obtain CNN architectures, achieving good results against the state-of-the-art architectures. In Ma et al. [8], the authors present an analysis of different methodologies based on evolutionary computing techniques to optimize CNN architectures, these were tested on benchmark data sets and achieved competitive results. Baldominos et al. [9] implement an approach to automatically design CNN architectures, using genetic algorithms (GA) in conjunction with grammatical evolution.
In other research works, the PSO algorithm is used to optimize CNN architectures, obtaining favorable results in the solution of different applications. In Sun et al. [18], Singh et al. [19] and Wang et al. [20], PSO is applied to automatically design CNN architectures; these approaches are tested on known benchmark datasets, and the results obtained are competitive against the state-of-the-art architectures. Besides this, PSO has been implemented in other fields of machine learning, including the optimization of different types of artificial neural network architectures, given favorable solutions for a plethora of problems [21,22]. In [23], PSO optimizes models of modular neural networks and is applied to obtain the blood pressure trend. In [24], a hybrid ANN-PSO method is applied to model the electricity price forecasting for the Indian energy exchange. As well as in [25], the PSO variants are applied to generate optimal modular neural network architectures obtaining competitive results for human recognition. In [26], the PSO algorithm is used to optimize deep neural network architectures and is tested in image classification tasks. In [27] a new paradigm of hybrid classification based on PSO is presented, which is applied for the prediction of medical diagnoses and prognoses. Furthermore, in [28] the artificial bee colony (ABC) [29] and PSO are used to optimize multilayer perceptron neural networks (MLP); the approach is applied to estimate the heating load and cooling load of energy efficient buildings; and the authors report that PSO outperforms ABC, improving the MLP performance. In the listed works, we can note the advantages that PSO offers in the optimization process, increasing performance in different tasks.
In research related to CNN approaches applied to the recognition of sign language, we find the work presented in [30] where a CNN model with stochastic pooling is implemented in the recognition of the Chinese sign language spelling, achieving a rate of 89.32 ± 1.07% recognition. In [31] a CNN method for Arabic sign language (ArSL) recognition was applied, where the authors report a value of 90.02% precision. In [32] a 3D-CNN approach is applied to sign language recognition for extensive vocabulary, images are captured through a Kinect, the authors report effectiveness of 88.7%.
The contribution of this research work focuses on implementing a hybrid methodology, where the PSO algorithm is applied to find the optimal design of parameters for CNN architectures. This work presents two optimization approaches; in both, the parameters considered are the number of convolution layers, the filter size used in each convolutional layer, the convolution filters number and, the batch size. In the first approach, the consistency of the parameters between each layer is maintained in the same conditions and the objective function is given by the recognition rate. In the second approach, the aim is to find more random searches in the architectures that the PSO produces; in this case, the values for each convolution layer are completely different, and the objective function is given by the highest recognition rate, and the lowest Akaike information criterion (AIC); the latter helps to obtain more robust performance of the network with the minimum parameters as the AIC allows penalizing the number of parameters used in each training. The optimized architectures are tested with three sign language databases, including the Mexican Sign Language (MSL) alphabet, the American Sign Language (ASL) alphabet [33], and the American Sign Language MNIST (ASL MNIST) [34]. This research aims to impulse the investigation in the soft computing area for the development of tools to help the deaf community for a more inclusive society [35].
The structure of this work is organized as follows. Section 2 presents the general theory about convolutional neural networks. Section 3 introduces PSO theory, including definitions, functionality, and the main equations. Section 4 details the methodology for developing the two PSO-CNN optimization approaches. Section 5 describes an analysis of the experimental results achieved after the optimized architectures are implemented for the three databases. Additionally, Section 5 presents a statistical test to compare the two optimization proposals, and we also show a comparative analysis against other CNN approaches focused on sign language recognition. Finally, Section 6 gives important conclusions and future works.

Convolutional Neural Networks
Biologically inspired computational models are capable of far outperforming previous forms of common artificial intelligence of machine learning. One of the most impressive forms of ANN (artificial neural network) architecture is that of CNN, which is mainly implemented to solve difficult image-based pattern recognition tasks.
CNNs are a specialized type of ANN with supervised learning, which process their layers by emulating the visual cortex of the human eye. This procedure allows the recognition of characteristic patterns in the input data, which makes it possible to identify objects through a set of hidden layers, which have a hierarchy and are specialized. The first layers are capable of detecting curves and lines and to the extent that you work with deeper layers, it is possible to achieve the recognition of more complex shapes, such as a silhouette or peoples' faces.
These types of networks are designed to operate specifically with image processing. The design of its architecture emulates the behavior of the visual cortex of the brain when processing and recognizing images [36]. Its main function is to locate and learn the information characteristic patterns, such as curves, lines, color tones, etc., through the application of convolution layers, which facilitate the process of identification and classification of objects [37,38].
The basic CNN architecture is presented in Figure 1, which consists of five layers: the input, convolution, non-linearity (ReLu), pooling, and classification layer [39,40], these are described in the following subsections. CNNs are widely implemented in applications that need the use of artificial vision techniques. Although the results that have been obtained are very promising, the reality is that they incur high computational costs; therefore it is essential to implement techniques that allow your performance to be increased. For this reason, an optimization of the CNN parameters is presented to improve the recognition percentage and reduce computational cost. In Figure 2, we can appreciate some parameters that can be optimized in each CNN layer [41].

Input Layer
It is the first layer of a CNN, here the images or videos are entered that are going to be processed by the neural network to extract their characteristics. All information is stored in two-dimensional matrices. To increase the effectiveness of the algorithms and reduce the computational cost, it is recommended to carry out a previous preprocessing of the images to be trained, such as segmentation, normalization of pixel values, extraction of characteristics of the objects or the background to keep the most relevant information, working them in grayscale, etc.

Convolution Layer
One of the most distinctive processes of this type of network is convolutions. It consists of taking a group of pixels from the input image and making a dot product with a kernel to produce numerous images that are the feature maps; these maps are distinct and depend on the type and size of the convolution filter implemented in the image.
Among the important characteristics that it gives to the kernel, is to detect lines, edges, focus, blur, curves, colors, among others. This is achieved by performing the convolution between the image and the kernel, multiplying the filter values pixel by pixel with those of the image, by traveling the filter from left to right; this representation can be appreciated in Figure 3 [42], where * stands the convolution operation.

Non-Linearity Layer
The activation function in the convolutional layer has the same proposal that the activation used in any neural network, commonly a non-linearity function is used to normalize the images. There exist different activation functions; one of the most used in this type of models is the rectified linear unit (ReLU) function which brings back a value of zero if it receives a value less than zero as input, nevertheless for any value greater than zero the same parameter comes back [41,42].

Pooling Layer
The pooling task is used to reduce the dimensionality of the network, in other words, allows the reduction of the number of parameters, which shortens training time and combats over-fitting [41]. Among the most used types of grouping, we can mention the following: (1) mean, select the arithmetic mean of the values, (2) max pooling, select the pixel with the largest value in the feature map and (3) sum, take the sum of all the elements present in the feature map.
The pooling operation is usually done using a 2 × 2 filter, assuming that we have a 4 × 4 future map (obtained after the convolution layer), and this operation is carried out; first, the future map is divided into 4 segments with the size of the filer (2 × 2), second, in each segment an pixel value is selected according to pooling type (mean, max, sum). An example is illustrated in Figure 4.

Classifier Layer
This layer appears in the CNN architecture after total convolutional and pooling layers; this is a fully connected layer that interprets the feature representations obtained by the previous layers and performs the high-level reasoning function. It has a similar principle to the conventional multilayer perceptron neural system, and in this layer, the CNN recognizes and classifies the images that are part of the output. In a multiclass classification problem, this fully connected layer has the same number of outputs as the classes defined in the study case to be solved. The Softmax function has become one of the most popular options for the classification task, due to its effectiveness [42].

Particle Swarm Optimization
It is a stochastic algorithm established on the intelligence of the swarm and inspired by the way birds forage for food; each bird is represented using particles which "move" in a multidimensional search space and "adjust" based on the experience of neighbors and your own.
The possible solution to the problem is depicted by the particle, which can be considered as "an individual element in a flock" [43]. PSO uses local and global information to find the best solution using a fitness function and the speeds at which the particles are moving.
PSO is very prone to premature convergence and falls into local optimum, so since its introduction in 1995 by Kennedy and Eberhart [44], various optimization variants have been proposed [45][46][47][48].

Algorithm 1. The PSO algorithm
Initialize the parameter of the problem (a random population). while (completion criteria are not met) begin For each particle i do begin Update the position p i using (1). Update the velocity x i using (2). Evaluate the fitness value of the particle If is necessary using (3)(4) Update pbest i (t) and gbest i (t). end end Algorithm 1 describes the process carried out by the PSO. This algorithm is defined by the equations that allow updating of the velocity with Equation (2) and the position with Equation (1).
In Equation (1), p i (t) is the position of particle i in a time t, within the search space. By adding a velocity x i (t) it is possible to change the position of the particle [45].
In Equation (2), x represents the velocity and i the particle. The parameters c 1 and c 2 define the cognitive and social factors, respectively. The random values in the interval [0,1] are depicted by r 1 and r 2 , ω is an inertia weight and the best position of the particle (pbest i ) is determined by y i and the best global position (gbest ) byŷ.
The swarm is assumed to consist of n particles, so an objective function f is implemented to perform the computation of particle fitness with a maximization task. The personal and global best values are updated using Equations (3) and (4), respectively, at a time t [48].
According to Equations (1) and (2), the movements of the particle in the search space are illustrated in Figure 5.
The red and yellow circles represent the movement that a particle makes when the parameters c 1 and c 2 are updated. When c 1 > c 2 , the particle moves in the direction of the yellow circle. When this condition is met, it means that the swarm performs the exploration process, so they "fly" in the search space to find the area that allows it to find the global optimum.
This movement allows the particles to perform long displacements, thus covering the whole search space. In the case of c 2 > c 1 then, the particle motion will be towards the red circle. It is here that the exploitation process takes place; it consists of the swarm "flying" in the best area of the search space, making small motions, which allow an intensive search [49].

Convolutional Neural Network Architecture Optimized by PSO
This Section presents two optimization approaches where the PSO algorithm is applied to optimize the parameters of CNN architectures, these approaches are denoted as PSO-CNN-I and PSO-CNN-II. The first objective is to select the most relevant parameters that have influence to obtain good performance of CNN and then implement the PSO algorithm to find these optimal parameters.
The parameters to be optimized were selected after evaluating the performance of a CNN with an experimental study, where the parameters were changed manually. As mentioned above, different CNN parameter values produce a variety of results for the same task; for this reason, the aim is to find the optimal architectures. The parameters listed below were chosen to be optimized in this work.

•
The number of convolutional layers; • The filter size or filter dimension used in each convolutional; • The number of filters to extract the future maps (the convolution filter number); • The batch size number: this value represents the number of images that are entered into CNN in each training block.
The general methodology of the proposal is presented in Figure 6, as the "training and optimization" block is the most important part of the whole process, where the CNN is initialized to integrate the parameter optimization by applying the PSO algorithm. In this process, the PSO is initialized according to the parameter given for the execution (the parameters are explained below) and this generates the particles. Each particle is a possible solution and its position has the parameter to be optimized, so each solution represents a complete CNN training. The training process is an iterative cycle that ends when all the particles generated by the PSO are evaluated for each generation. The computational cost is higher and, it depends on the database size, the size of particles, the number of iterations of the PSO and, the number of particles in each iteration. That is to say, if the PSO is executed with 10 particles and 10 iterations, the CNN training process is executed 100 times. The steps to optimize the CNN by the PSO algorithm are illustrated in Figure 7 and explained as follows.

1.
Input database to train the CNN. This step consists of selecting the database to be processed and classified for the CNN (ASL alphabet, ASL MINIST and MSL alphabet). Is important to mention that all the elements of each database need to keep a similar structure or characteristics. In other words, images with the same scale and color gamma (grayscale, RGB, CMYK); additionally, with the same dimensions of pixels and a similar format of file (JPGE, PNG, TIFF, BMP, etc.). 2.
Generate the particle population for the PSO algorithm. The PSO parameters are set to include the number of iterations, the number of particles, inertial weight, cognitive constant (W1), and social constant (W2); the parameters used in the experimentation are presented in Table 8. This step involves the design of the particles; the structures of these are presented in Tables 1 and 3 according to the two optimization architecture proposals in this paper.

3.
Initialize the CNN architecture, with the parameter obtained by the PSO (convolution layers number, the filter size, number of convolution filters, and the batch size) the CNN is initialized and in conjunction with the additional parameter specified in Table  8, the CNN is ready to train the input database. 4.
CNN training and validation. The CNN reads and processes the input databases taking the images for training, validation, and testing; this step produces a recognition rate and the AIC value. These values return to the PSO as part of the objective function.

5.
Evaluate the objective function. The PSO algorithm evaluates the objective function to determine the best value. As in this research, we are considering two approaches, in the first, the objective function is only the recognition rate (Equation (5)) and in the second, the objective function consists of the recognition rate and the AIC value (Equation (6)).

6.
Update PSO parameters. At each iteration, each particle updates its velocity and position depending on its own best-known position (Pbest) in the search-space and the best-known position in the whole swarm (Gbest). 7.
The process is repeated, evaluating all the particles until the stop criteria are found (in this case, it is the number of iterations). 8.
Finally, the optimal solution is selected. In this process, the particle represented by Gbest is the optimal one for the CNN model.

PSO-CNN Optimization Process (PSO-CNN-I)
This first approach, which we are going to identify as PSO-CNN-I, consists of implementing a particle with four positions, one position for each parameter to be optimized ( Figure 8). Table 1 presents the detail of the particle composition where the position x 1 corresponds to the number of layers with a search space from 1 to n, that is to say, that method can produce architectures with a minimum of one layer and maximum n, for the purposes of this work, we are using n = 3. The x 2 position represents the number of convolution filters used to extract the characteristics, with a search space of 32 to 18 filters. Position x 3 is the filter size; the search space is from 1 to 4 where this values represents a position, the value reached is mapped with the values of Table 2 to obtain the filter size (i.e., if the particle generates a value of 1 this represents a filter size of [3 × 3], to get a value of 2 the filter size will be [5 × 5] and so on, respectively, for each value. The last position represents the batch size (x 4 ), this is initialized considering the search space ranges from 32 to 256. In this optimization process, the consistency of the parameters between the layers is maintained in the same conditions, that is, if after the PSO execution it generates a particle with 3 convolutional layers (x 1 ), 50 filters (x 2 ), a filter dimension of 3 × 3 (x 3 ) and batch size of 50 (x 4 ). The same values of filter numbers (x 2 ) and filter size (x 3 ) will apply to the three convolution layers of the CNN.   [9,9] In this process, the objective function defined by Equation (5) is given by the recognition rate (precision) that the CNN returns after it is trained with the parameters generated by the PSO.
Objective f unction = Recognition Rate,

PSO-CNN Optimization Process (PSO-CNN-II)
In this second proposal, identifying as PSO-CNN-II, the particle structure consists of eight positions whose structure is presented in Figure 9, where each position represents the parameter to be optimized. The difference from the previous approach (PSO-CNN-I in Section 4.1) is finding more random searches in the architectures that the PSO produces; because in this case, the values for each convolution layer are completely different. Table 3 presents the detail of the particle composition, the description of each position, and the search space used. As we can see in Table 3, the positions x 3 , x 5 and x 7 represent an index with an integer value between 1 to 4, and depending on the value taken by the PSO, a mapping is made with values presented in Table 2.  Batch size in the training [32,256] According to the values to optimize in this new approach, the x 1 position is used to control the number of convolution layers and the activation of the positions x 2 to x 7 . If PSO generates a particle with a value of one for x 1 , only the position x 2 and x 3 will be activated to generate the number of filters of the convolutional layer 1 and the filter size to use in this layer. In other words, if PSO produces a particle with a value of three in the x 1 position, the positions from x 2 to x 7 will be activated to generate the number of filters to use in the convolutional layer 1 (x 2 ), the filter size of layer 1 (x 3 ), the number of filters of layer 2 (x 4 ), the filter size for layer 2 (x 5 ), the number of filters of layer 3 (x 6 ), and the filter size for layer 3 (x 7 ), respectively; these values are completely different from each other, therefore this methodology helps to produce more heterogeneous CNN architectures.
Another difference between this proposal and the previous one (PSO-CNN-I) is that the objective function changes, for this we are using the recognition rate together with the Akaike information criteria (AIC). The AIC penalizes the architectures according to the number of parameters used; that is to say, the model is penalized when it needs more parameters. The objective function is considered the highest recognition rate and the lowest AIC. The AIC is defined in Equation (6).
According to our problem, in Equation (6) k is the number of parameters of the model (number of layers and filter number) and L is the maximum value of the recognition that the CNN can reach; in this case, the value is 100. Figure 10 illustrates an example of a particle generated by PSO. Based on the structure of Figure 10, we have a three-layer convolutional architecture, where the first layer consists of 100 convolution filters with a filter size of 3 × 3. The second layer has 85 convolution filters with a filter size of 5 × 5 and the third convolution layer has 50 convolutional filters with a filter size of 7 × 7. Finally, the batch size is 32. The CNN is training with these values, and the recognition rate is calculated, additionally the AIC is obtained based on the parameters of the positions x 1 , x 2 , x 4 and x 6 which represents the number of convolution layers and the number of filters for each convolutional layer. After applying Equation (6), this architecture produces the AIC defined in Equation (7).
Assuming that there are two architectures with the same recognition rate but with different AICs (Table 4), the model will take the architecture with the lowest AIC, as this would help penalize the parameters that are needed to train the network and thus produce optimized and simpler architectures.

Experiments and Results
This section describes the three databases implemented in the case studies (ASL alphabet, ASL MNIST, and MSL alphabet), the static parameters used to set the PSO algorithm and the CNN process, the experimental results obtained in the two optimization approaches that were performed (PSO-CNN-I and PSO-CNN-II), as well as the comparison analysis against other approaches.

Sign Language Databases Used in the Study Cases
The characteristics of the sign databases are described below.

American Sign Language (ASL Alphabet)
The ASL alphabet consists of 87,000 images in color format, with a dimension of 200 × 200 pixels. This database contains 29 classes, these are labeled in a range of 0 to 28, with a one-to-one assignment for each letter of the American alphabet A-Z (0 to 25 for the alphabet; that is, 0 = A and 25 = Z) the other three classes correspond to the space symbols, delete, and null (26 to 28; i.e., 26 = space, 27 = delete and, 28 = null). Table 5 presents the general description of the ASL alphabet database and Figure 11 illustrates a sample of the images.  Figure 11. A sample of the ASL alphabet database.

American Sign Language (ASL MNIST)
ASL MNIST consists of a collection of 34,627 grayscale images with a dimension of 28 × 28 pixels. This database has 24 labeled classes in a range from 0 to 25 with assignment for each letter of the alphabet A-Z (the class 9 = J and 25 = Z, were excluded due to gestural movements). Table 6 presents a description of this database and Figure 12 illustrates a sample of the sign images.

Mexican Sign Language (MSL Alphabet)
The MSL alphabet database was obtained from a group of 18 people, including deaf students and sign language translation teachers. Students are part of an inclusive group in a high school in Mexico. This database consists of 21 classes with the alphabet of the MSL without movement as illustrated in Figure 13. Ten images were captured for each letter, achieving a total of 3780 grayscale images with a dimension of 32 by 32. Table 7 displays a general overview of the MSL alphabet database.

Parameters Used in the Experimentation
In the CNN parameter settings, some static parameters were used, including the learning function, the activation function in the classifying layer, the non-linearity activation function, and the epoch number. The fixed parameters considered in the PSO configuration are the number of particles, the iterations number, the inertial weight, and the social and cognitive constants. The static parameters used for PSO and CNN are presented in Table 8. The dynamic parameters optimized by PSO are the number of convolutional layers, the size of the filters used in each convolutional layer, the number of convolutional filters, and the batch size (Tables 1 and 3).

Optimization Results Obtained by the PSO-CNN-I Approach
This section presents the simulation results produced after the CNN architecture is optimized considering the approach described in Section 4.1. The experimentation consists of 30 executions carried out on the three databases; the aim is to obtain the optimal CNN architecture, that is, minimum parameters necessary to maximize the recognition rate.
The first experiment was applied in the ASL alphabet (Table 5), using a distribution of 80% of the total images for training and 20% for testing. Table 9 shows the values achieved after 30 executions, where the higher recognition rate was a value of 99.87% and the mean was 99.58%. Based on the results, we can see that the optimal architecture achieved by the PSO was as follows: three convolutional layers, 128 filters per layer with a filter size of 7 × 7, and the batch size with a value of 256.
In another test, the PSO-CNN-I approach was applied to the ASL MNIST database; Table 10 presents the results achieved by the CNN where the best recognition rate was a value of 98.82% and the mean of 99.53%. According to this analysis, the optimal architecture for this study case is two convolutional layers, with 117 convolutional filters in both layers with a filter size of 7 × 7 and the batch size with a value of 129.  Table 11 presents the experimental results obtained when the approach is applied in the MSL alphabet database. As we can see in Table 11, the best accuracy reached by the CNN was 99.37% with a mean of 99.10%. In this case, the optimal architecture is as follows: one convolutional layer with 122 convolutional filters, filter size of 3 × 3, and batch size of 128.

Optimization Results Obtained by the PSO-CNN-II Approach
The results presented in this section consist of 30 executions of the PSO-CNN-II approach applied in the ASL alphabet, ASL MNIST, and MSL alphabet databases; the objective is to maximize the recognition rate and minimize the value of AIC.
The experimental results obtained from the ASL alphabet database after applying the PSO-CNN-II optimization approach (Section 4.2) are presented in Table 12. In this test, the database was distributed so that 70% of the data were kept for the training phase and 30% of the data for testing. Table 12 shows the best recognition rate with a value of 99.23% and a mean of 98.69%. The best architecture found by the PSO for the CNN had the following structure: three convolutional layers where the first layer had 84 convolutional filters and 3 × 3 size filters; the second layer with 128 convolutional filters with the size of 9 × 9 and the third layer with 128 convolutional filters and 7 × 7 size filters. In this approach, the objective function is composed of the recognition rate and the AIC value; that is, the best recognition rate is evaluated first and then the AIC value, if it was the case that CNN achieved two or more architectures with the same recognition rate, the process takes the architecture with the minimum AIC, with the goal of achieving an optimal architecture with the fewest parameters and the highest recognition rate. In Table 13, we present the results where the PSO-CNN-II was implemented in the ASL MNIST database. In this test, the best recognition rate was 99.80%, an AIC value of 506.79 and, a mean of 99.48%. The optimal parameters found by the PSO were the following: two-layer CNN architecture, the first layer had 128 filters of convolution and a filter size of 5 × 5; the second layer had 128 convolutional filters with a filter size of 9 × 9, and the batch size was 128. In another experiment, the optimization approach was applied to the MSL alphabet database after 30 simulations. The results obtained are presented in Table 14, where the best recognition rate was 99.45% with an AIC of 248.79. The general mean for this study case was a value of 98.91%. In this optimization, one-layer CNN architecture was achieved, with 128 convolutional filters, 3 × 3 filter sizes, and 154 batch sizes. Table 15 presents a summary of the results obtained after the two approaches were applied to the three databases. We can see that good results were achieved in all the cases; we can analyze that for the ASL alphabet and the ASL MNIST, the PSO-CNN-I optimization approach was better with mean values of 99.58% and 99.53%, respectively. For the MSL alphabet database, the PSO-CNN-II optimization method achieved a better recognition rate with a mean value of 98.91%. Although, if the results were analyzed with respect to the AIC value, for the ASL MNIST and the MSL alphabet, the PSO-CNN-I reached the lowest values with AIC of 462.79 and 236.80, respectively, and for ASL alphabet, the PSO-CNN-II achieved a better AIC value. A low AIC value means that the CNN architecture required fewer parameters, so it is important to determine what is most relevant to any problem, the CNN accuracy, or to configure the CNN architectures with minimal parameters that can be implemented in real-time systems. To confirm if significant evidence exists between the architectures and to identify which is better, the Wilcoxon signed-rank test was applied [50]; this is a non-parametric test that is recommended to be applied when the numerical data are not normally distributed, as is the case with the experimental results of metaheuristic algorithms. The Wilcoxon test was performed to compare the PSO-CNN-I and PSO-CNN-II optimization processes, considering the results presented in Tables 9-14. The general description of the values used to execute the Wilcoxon test is presented in Table 16 and described below:

•
The objective is to reject the hypothesis null (H 0 ) and support the alternative hypothesis (H 1 ).

Description Hypothesis
Null hypothesis PSO-CNN-I architecture (µ 1 ) = PSO-CNN-II architecture (µ 2 ) H 0 : Alternative hypothesis PSO-CNN-I architecture (µ 1 ) > PSO-CNN-II architecture (µ 2 ), The first Wilcoxon test was applied for the ASL alphabet results (Tables 9 and 12). Table 17 shows the R + , R − , and the p-value (the p-values have been computed by using SPSS), where R + represents the sum of ranks for the problems in which the first algorithm outperformed the second, and R− the sum of ranks for the opposite. The results obtained indicate an R + of 455, an R − of 10 and the p-value of <0.001. Because the p-value is less than the alpha value of α = 0.05, then we support the alternative hypothesis with a 95% level of evidence, and we can affirm that the PSO-CNN-I architecture is better than the PSO-CNN-II.   Table 18 presents the results after the Wilcoxon test was applied for the ASL MNIST results (Tables 10 and 13). This test obtains the values R + = 245.5, R − = 189.5, and the p-value = 0.545. Since the p-value is greater than the alpha value of α = 0.05, the null hypothesis is accepted with a 95% level of evidence; therefore, we can affirm that evidence does not exist to determine that the PSO-CNN-I architecture is better than the PSO-CNN-II.  Finally, Table 19 presents the Wilcoxon test for the results of the MSL alphabet (Tables 11 and 14). The results obtained indicate the values R + = 291, R − = 115, and the p-value = 0.045. We can see that the p-value is less than the alpha value of α = 0.05; therefore, we support the alternative hypothesis with a level of evidence of 95%, and we can affirm that the PSO-CNN-I architecture is better than the PSO-CNN-II.

State-of-the-Art Analysis Comparison
To obtain more evidence about the performance of the optimization approaches presented in this paper, we make a comparative analysis (Table 20) against the state-ofart research, where CNN models are implemented in Alphabet Sign Language database recognition. The results presented in Table 20 represent the best recognition rate values reported by the authors, the detail of which is explained as follows: Zhao et al. [51] reports an accuracy of 89.32%, the CNN architecture has two convolutional layers, two pooling layers, the batch size is 150, and 80 iterations. The authors generated their own ASL database, this was captured in five people covering 24 letters of the alphabet, and each person's letters had about 528 photos. Table 20. State-of-the-Art Comparison.

Reference
Recognition Rate (%) Dataset Y. Zhao and L.Wang [40] 89.32 ASL own D.Rathi [41] 95.03 ASL MNIST L.Y.Bin y Y.Huann [42] 95.00 ASL own R. Dionisio [39] 97 Rathi [52] presents an optimization of the transfer learning model (based on CNN) and it was applied to the ASL MNIST database, using 27,455 images of 24 letters of the ASL alphabet. The data split was as follows, 80% of the data was for training, 10% for testing, and 10% of data for validation purposes with a training batch size of 100. The best recognition rate evidenced by the author was a value of 95.03%.
In Bin et al. [53], an architecture of four convolutional layers and two pooling layers was presented. The database was generated by the researchers themselves, taking characteristics of the ASL MINIST and consisting of 4800 images; the best accuracy reported by the authors was 95.00%.
Dionisio et al. [54], reported a recognition rate of 97.64% for the ASL MNIST with a six-layer convolutional architecture, three pooling layers, a filter size of 3 × 3, and a batch size of 128. The database was divided using 10% of data for phase testing, 10% for phase validation, and 80% for training.
Finally, we present the recognition rate achieved by our two approaches PSO-CNN-I and PSO-CNN-II with the best values of recognition rates of 99.98% and 99.80%, respectively. In the PSO-CNN-II, the best architecture obtained for the ASL MINIST was of two layers with 117 filters per layer with a size filter of 7 × 7 and batch size of 129. For the PSO-CNN-I, it was of two layers with 117 filters per layer with a size filter of 7 × 7 and batch size of 129.
As one can observe in Table 20, the highest performance was obtained by the proposed model (PSO-CNN-I) with a value of 99.98%, achieving an advantage over the rest of the approaches.

Conclusions and Future Work
In summary, in this paper, we present two approaches to optimize CNN architectures by implementing the PSO algorithm, these being applied to sign language recognition. The main contribution was to find some CNN hyper-parameters; in the proposals the number of convolutional layers, the size of the filter used in each convolutional layer, the number of convolutional filters, and the batch size were included. According to the experimentation and the results obtained in the two PSO-CNN optimization methodologies, we can conclude that the recognition rate increased in all case studies carried out, providing a robust performance with the minimum parameters. Overall, the recognition rates achieved by the three databases were as follows: for the ASL MNIST database, the best value was 99.98% and an average of 99.53% with the PSO-CNN-I approach. For the ASL alphabet database, the best accuracy was 99.87% and an average of 99.58% with PSO-CNN-I, and for the MSL alphabet, the best value was 99.45% and an average of 98.91% after applying the PSO-CNN-II approach. After a comparative analysis against other state-of-the-art works focused on sign language recognition (ASL and MSL), we can confirm that the optimization approaches of this work present competitive results.
This research focused on optimizing the number of convolutional layers, the filter size used in each convolutional layer, the number of convolutional filters, and the batch size. The results provide evidence of the importance of applying optimization algorithms to find the optimal parameters of convolutional neural network architectures.
As future work, the PSO algorithm could be applied to optimize other CNN hyperparameters, implement another version of the PSO algorithm or explore different evolutionary computational techniques, to produce more robust CNN architectures that will be implemented in different sign language datasets used in other countries. In the experimental test, the images were introduced as static images, but we are considering working with input images in real-time or capturing them through video. On the other hand, our idea is to be able to implement the use of this proposal in the development of assisted communication tools and to contribute to human−computer iteration applications that can be of support to the deaf community.


Gabriel Pereira TOWARDS REFUSING AS A CRITICAL TECHNICAL PRACTICE: STRUGGLING WITH HEGEMONIC COMPUTER VISION

Computer Vision (CV) algorithms are overwhelmingly presented as efficient, impartial, and desirable further developments of datafication and automation. In reality, hegemonic CV is a particular way of seeing that operates under the goal of identifying and naming, classifying and quantifying, and generally organizing the visual world to support surveillance, be it military or commercial. This paradigm of Computer Vision forms a ‘common sense’ that is difficult to break from, and thus requires radical forms of antagonism. The goal of this article is to sketch how refusing CV can be part of a counter-hegemonic practice – be it the refusal to work or other, more creative, responses. The article begins by defining hegemonic CV, the ‘common sense’ that frames machine seeing as neutral and impartial, while ignoring its wide application for surveillance. Then, it discusses the emergent notion of refusal, and why critical technical practice can be a useful framework for questioning hegemonic sociotechnical systems. Finally, several potential paths for refusing hegemonic CV are outlined by engaging with different layers of the systems’ 'stack.'

The computer scientist Joseph Redmon, creator of the widely-used Computer Vision library YOLO, announced in 2020 he would no longer be developing the algorithm he created. The reason for this is made clear in a paper he co-wrote on the new features of YOLO: But maybe a better question is: "What are we going to do with these detectors [Computer Vision algorithms] (Redmon and Farhadi 4) This humorous -yet critical -paragraph is crowned by a footnote, which states: "The Research and Google." Redmon's refusal to continue his work with Computer Vision (CV) -algorithms of image analysis and recognition -builds upon his realization of the contribution he makes, as a computer scientist, to algorithms of oppression (see Noble; Ochigame) and their nefarious impacts in society (e.g. consumer surveillance, drone attacks, tracking of migrants by governments). Rather than trying to reform refuse, to deny his labor as part of developing such technology. [1] Redmon's refusal throws a wrench in the system, breaking from the hegemonic presentation of CV (and AI) as a way of making everything better, faster, and more innovative. The techno-utopian and techno-solutionist discourse on CV pushed by Silicon Valley companies and other tech/ government entities presents these techautomation. In reality, CV operates under the goal of identifying and naming, classifying and quantifying, and generally organizing the visual world to support surveillance, be it military or commercial. This hegemonic paradigm of Computer Vision forms a "combreak from, and thus requires radical forms of antagonism, refusal, and resistance.
My goal in this article is to sketch a scenario of refusing as a reaction to hegemonic CV, in order to help readers engage in their own practices of antagonism -be it the refusal to work as shown by Redmon or other, more creative, responses. I particularly engage with the notion of refusing because, as seen in Redmon's example, it shifts the discussion away from a reform of technical character and towards a more radical counterhegemonic practice. Towards this goal, I as hegemonic CV, the 'common sense' that frames machine seeing as neutral and impartial, while ignoring its wide application for surveillance. Afterwards, I discuss the notion of refusal, and why I think critical technical practice serves as a useful framework for questioning hegemonic sociotechnical systems. Finally, I outline several paths for refusing hegemonic CV by engaging with different layers of its stack. These potential resistance acts, as I show, can take shape in varied forms -artistic projects, activist initiatives, and community organizing can all offer counterhegemonic pathways for CV.
Hegemonic CV: The limited ways of seeing of surveillance, advertisement, and the military Who makes Computer Vision algorithms? How are they being trained to see? What is made visible through these algorithmic ways of seeing, and what is otherwise ignored? The answer to these and many other questions points to the often-ignored sociotechnical complexity involved in the widespread adoption of CV. As they get implemented in smart cameras or automated cars, these algorithms carry within them not the capacity to 'see,' but that of making judgements over what in the visual world should be seen and how.
CV's algorithmic power emerges from how, through their affordances and -Matteo Pasquinelli and Vladan Joler compare algorithms to lenses: "Instruments of measurement and perception always come with inbuilt aberrations. In the same way that the lenses of microscopes and telescopes are never perfectly curvilinear and smooth, the logical lenses of machine learning embody faults and biases" (2). This is much similar to Amoore's framing of algorithms as "aperture instruments," thus suggesting that it is through "the processes of feature extraction, reduction, and condensation" that "algorithms generate what is of interest in the data environment" (16). These analogies to other instruments of perception are useful because they help understand contemporary CV as one possible lens, with many other possibilities. Algorithmic models are always imperfect and biased towards somethingas Amoore puts it, they're "always already partial accounts" (20).
The particular lens of the hegemonic Computer Vision today is made of many ideological decisions over how the visual world should be understood and processed, including the ontology and epistemology that should be used in this process (see Azar et al.). What I call hegemonic CV is the dominant paradigm of automated ways of seeing, directly connected to surveillance, both military and commercial (e.g. automated military drones and biased proctoring systems). This paradigm is not directly stated or enforced, operating through consent and culture rather than force. Much as described by Gramsci, and later extended by Laclau and Mouffe, hegemonic ideological formations are produced and negotiated as the outcome of constant struggles for power, emerging from a wider cultural/social history. What's crucial is that they get sedimented as a "common sense" (Gramsci),[2] an "accumulation of taken-for-granted 'knowledge'" (Crehan 43). These collections of beliefs and ideas are "not a single unique conception, identical in time and space," (Gramsci 343) but fragmentary and contradictory, "a product of history and a part of the historical process" (327).
to imagine alternative lenses to see the world: as hegemonic CV further entrenches itself in our lives, our human ways of seeing and wider societal processes are changed (see Cox).
CV is just one of the many data capitalist/colonialist algorithmic operations through which value is extracted from appropriating people's data, be it their face, their pictures, or other visual material (Couldry and Mejias). The main imperatives of 'smart technologies' such as hegemonic CV is extracting more data from all sources possible, while also "creating systems that monitor, manage, and manipulate the world and people" (Sadowski 9). In Sarah Myers West's words, this operation is marked by a change in power relations: "Access to data, and the ability to transform raw data into useful information, is asymmetrical, and power lies in the institutions with the technical and economic resources to render it intelligible" (37). Hegemonic CV is based on these structural conditions and built upon their limitations, thus operating in order to meaningfully limit who gets to see and who is seen -making itself into a crucial site of centralized and unequal surveillance and data extraction.
Hegemonic CV intentionally focuses tion from its unequal power structures and the many problems involved in its limited perception of the visual world. Hegemonic CV presents itself as objective, hiding its deep commitments to military and surveillant ways of seeing formed by Western, white, and capitalist frames (Silva; Silva et al.; Buolamwini and Gebru; Mirzoeff; Pereira and Moreschi). As scholars have demonstrated time and again, supporting these structural inequalities are, among other issues, exploitative labor practices (Tubaro et al.; Irani "The Cultural Work of Microwork") and problematic data sets (Thylstrup; Harvey and LaPlace; Prabhu and Birhane; Crawford and Paglen). Furthermore, there are many intended and unintended, known and unknown, consequences of Computer Vision, which are mostly ignored in exchange for Wilken).
Despite the many ways CV could have been formed, my argument is that a 'common sense' has emerged that frames CV in a particular way that's not just, equitable, and algorithmic imaginaries perform a "discursive closure," cutting off alternatives that seek to work in a different way (3). Understanding the existence of a hegemonic CV allows to better think about possible oppositions, resistances, and alternatives to its outsized hegemony.

As described by Raymond Williams,
It can be persuasively argued that all or nearly all initiatives and contributions, even when they take on manifestly alternative or oppositional forms, are in practice tied to the hegemonic: that the dominant culture, so to say, at once produces and limits its own forms of counter-culture. (114) The conceptualization of hegemony enables thinking of our practice as part of wider struggles for re-constituting these systems. The notion of refusing departs from understanding that counterhegemonic struggles are responses constructed in the interstices of hegemonic forms. That is, even though we may try to re-imagine CV, we're still located in relation to this dominant system.

Why a Critical Technical Practice, and the case for refusing as a verb
Redmon's refusal to continue working in Computer Vision uses his privileged position to throw a wrench in the gears of hegemonic CV, helping to both expose these harmful technologies and delay their development. The book "Breaking Things at Work" by the scholar Gavin Mueller presents a long vision of how workers, not unlike Redmon, have for long resisted the expansion of automation, seeing these "new machines as weapons wielded against them in their struggles for a better life" (e-book). Mueller suggests how Luddism -multiple forms of collective resistance to uncontrolled technological development -can form a decelerationist political project to challenge the continuous development and deployment of technologies. Such a Luddite approach sees technology as a site of struggle in which antagonism is necessary to challenge hegemonic goals and assumptions -subverting technological control to regain power for workers. Mueller's analysis part of a generative politics -not only breaking machines and sabotage, but also forms of struggle such as that over policy and legislation. It means not only to say no to the development of new technical systems, but also to actively envision how we can center other values and paradigms.
All through industry, activism, policy, and research there are increased calls that center rejection (saying "no") as a strategy to combat the problems caused by algorithm development and deployment in society. Just to cite a few recent calls: the "Feminist Data Manifest-No" proposes 10 points of refusal for harmful data regimes; Seeta Peña Gangadharan discusses people's unwillingness "to accept data-driven systems in the terms and conditions that government or private actors present to us" (3); Sarah Hamid, in a remarkable interview, argues for abolishing "carceral technologies," and organizing "against the logics, relationships, and systems that scaffold their existence"; and Chelsea Barabas suggests tech designers should "turn down requests and opportunities to build technologies that are likely to produce harm." All of these pleas are located in a wider global environment in which tech workers have been putting pressure on companies' unethical developments, as shown in the case of Google workers' refusal to build Project Maven and the Tech Won't Build It activist group.
The goal of refusing is compelling, and these initiatives make visible how it is both of antagonizing hegemonic technological systems. There are certainly many ways such disposition can operate, and I'll focus here on just one of them: the concept of "critical technical practice" (referred here as CTP). In his seminal text conceptualizing the term, "Toward a Critical Technical Practice: Lessons Learned in Trying to Reform AI," Phil Agre recognizes "computing has been constituted as a kind of imperialism; it aims to reinvent virtually every other site of practice in its own image" (131). Agre's CTP emerges from his own personal story as a computer scientist that only after some time began to realize the political, social, and cultural constitution of technology, therefore suggesting how practice should avoid the separation between computer science and critical Practitioners would work interdisciplinarily, "one foot planted in the craft work of design work of critique," in order to create alternatechnical, cultural, and interpersonal object" (Harwood 32).
The problem I see in the conceptualization of CTP brought by Agre is that, as it was created over 20 years ago, it is somewhat detached from the current historical morefusing major revolutions and believing that change can be brought through individual practitioners and their "intrapersonal" critical consciousness. The notion of CTP has been further adapted through the past two decades, as scholars/practitioners have spun it into a design (e.g. Phoebe Sengers, Matt Ratto, Garnet Hertz, Alan Blackwell, Shaowen and Jeffrey Bardzell) or artistic practice (e.g. YoHa and Matthew Fuller, Winnie Soon).
conceptualization for thinking of technological practice as a mode of critique -making computation as a way of understanding and re-thinking computation itself.
Much as these developments have been important, I suggest CTP now needs to be further updated with contemporary notions of collective resistance, as well as more radical forms of antagonism and resistance. Agre himself acknowledged that critical research that worked from inside the system -including his own -could be quickly appropriated by the industry or the military. This further suggests how the goal of reform (or critiquing from the inside) can end up being innocuous, and reinforces the need for practices that defy institutionalization and that more strongly reject the 'common sense'.
As hegemonic CV has become particularly entrenched, my argument is to use CTP to engage with radical contemporary proposals for refusing. Rather than simply building more stuff, or building better, this means centering considerations about what not to build, and of more strongly thinking about building as a way of creating alternatives. The concept of "refusing" allows us to abandon and instead supports a multifaceted, activist disposition through different approaches, including arts, activism, community organizing, and research. Refusing as a practice is not just for those who have coding or technical skills. Much to the contrary, it needs to engage with the stack of technological systems from their labor practices to their philosophical underpinnings.
A practice that centers 'refusing' doesn't necessarily mean not using AI or algorithms, but breaking from its hegemonic paradigms to imagine how they could be different if they could center marginalized perspectives. Refusing, importantly, is to go against the tech/AI hype (Vinsel) and instead show how and when these technologies may not work as supposed. Moreover, and paraphrasing Agre in Your Face Is Not a Bar Code, this means understanding that surveillant technologies will "work well enough to be dangerous, and poorly enough to be dangerous as well." Finally, 'refusing' as a lens for a critical technical practice is not a negative, but a generative proposition -therefore always a verb indicating a continuous struggle. It's not just saying "no," but understanding how technology is located within a wider array of historical, social, and cultural conditions. It, drawing for the work of data feminism, needs to "name and challenge sexism and other forces of oppression, as well as… seek to create more just, equitable, and livable futures" (D'Ignazio and Klein 6). As the artist and researcher Caroline Sinders proposes, our practice needs to be "productive, as well as provocative," operating as "band aids": "not meant to create an end to all other potential solutions, but serve to offer rather, equity and violence created by society and are poetic witnesses of those gaps." Refusing needs to be an interdisciplinary practice that aims to affect the world, moving beyond the practitioner and attempting to disrupt hegemonic structures of power and make change. Refusing both breaks the system in operation and creates alternative systems.

Speculations on what refusing can mean across the 'stack' of hegemonic CV
I will now focus on the particular case of Computer Vision, to highlight potential directions refusing as a critical technical practice may take when engaging with such technologies. CV, as any other algorithmic technology, is tremendously complex, formed of many different interlocking social, cultural, economic, and legal aspects. Any attempt at considering such wide scale systems is necessarily partial, focusing only on some parts of the whole. One attempt at looking at the 'stack' of CV has been suggested by Azar, Cox, and Impett, a "vertical cartography" comprised of six different levels (10). While their original goal was organizing critical to use these layers to envision sites of refusing interventions by artists, researchers, and activists. For this, I both point to important still remain to be intervened on. I hope, with this, to offer more questions than answers and suggest critical pathways for a counterhegemonic practice: (  Redmon, discussed in the introduction, is just one of many attempts by tech workers labor, a practice in which activists play a major role (Mueller). However, how can practitioners also involve other workers, that are often not given much power in the system, the possibility of antagonizing the development of CV? (see e.g. the work of xtine burrough with Amazon Mechanical Turkers).
(2) Computational level (which problems are being solved: e.g. 'object detection') Refusing CV's hegemonic ideology of surveillance and tracking, could other 'problems' be solved? Hegemonic CV focuses mostly on detecting and oriented parameters. One possibility is the use of CV from a disobedient gaze, "surveilling the most powerful, as opposed to those marginalized" (Pereira 154; see also Barabas et al.). An example of this is VFRAME, by Adam Harvey and Jules LaPlace, which seeks to create AI/CV tools for human rights uses, including a "Munition Detector" which could locate illegal munitions and support the work of activists. What if, instead of the obsession for solving problems, CV focused on of these systems both as a sign of their limitations and as a way to think otherwise" (Pereira 156)? This form of refusal suggests the embrace of error, indeterminacy, opacity, and situatedness instead of the solution of bias and partiality (Amoore). This means using algorithmic ways of seeing as ways for exploring alternative visibilities and to create new connections that couldn't be otherwise (see Pereira and Moreschi for an example of using CV to look at artworks through the lens of error).
(3) Data level (who labels, which images are chosen, who takes the photographs) How can we refuse problematic data sets and their troubled histories? Vinay Prabhu and Abeba Birhane even name computer vision a pyrrhic win due to the "problematic practices and consequences of large scale vision datasets" (1). Data sets often rely on the extraction of images without agreement from users (see Harvey & LaPlace; Thylstrup), use precarious labor of Amazon Mechanical Turkers (Irani, "Justice for 'Data Janitors'"), as well as orgazine seeing through labels that encode racist, classist, and sexist histories (Hanna et al.; Crawford and Paglen; Smits and Wevers Considering refusal as part of this wider stack serves to argue that counterhegemonic responses can happen across different areas, even in places that so far remain little acknowledged. It shows how refusal can have many different facets and intensities beyond the work of computer scientists like Redmon. It importantly highlights the radical work being done by activists, artists, community organizers, researchers, etc. While useful, this stack is just one of many possible cartographies of action. The Stop LAPD Spying activist group, for example, suggests a framework for mapping surveillance divided on different layers: "Ideological, Institutional, Operational, and Community." Such a way of mapping illuminates how practice also needs to consider how communities should be involved in the creation and use of CV. Which communities should be centered that haven't yet, and which are being marginalized by CV use? This essay only hopes to serve as a starting point, so I leave it to you, the reader, to consider what other 'counter-cartographies' of refusing are possible, and how they can support your practice.

Conclusion: Performing alternatives, bearing witness to limitations
When the computer scientist Joseph of Computer Vision, due to his perception of the harms and injustices these technologies were causing, he threw a wrench in the system. Hegemonic Computer Vision advances these algorithms as natural developments, neutral and impartial operations, even though they're mostly used for supporting multiple forms of surveillance. In this article, I've demonstrated how refusing is a powerful counterhegemonic stance to this 'common sense,' especially through personal and collective antagonism to uncontrolled technological development. Bridging this stance with critical technical practice's focus on developing nical can serve to perform alternatives and pave the way to radical reimaginations -or at least create some 'band-aids' that bear witness to how much work we still need to do.
We needn't only reform CV, but to depart from an active refusal of its ideology and organizations, completely breaking it apart in pieces before building something new. Our tions, but to refuse the unsettling 'common sense' of technological progress through action. Much like the Luddites, refusing to work and sabotaging, but also building new ways of seeing and generating new collective engagements with the visual world.