extfieldsofstudy:
Computer Science
Medicine

text:
Analysis of the EEG Rhythms Based on the Empirical Mode Decomposition During Motor Imagery When Using a Lower-Limb Exoskeleton. A Case Study

The use of brain-machine interfaces in combination with robotic exoskeletons is usually based on the analysis of the changes in power that some brain rhythms experience during a motion event. However, this variation in power is frequently obtained through frequency filtering and power estimation using the Fourier analysis. This paper explores the decomposition of the brain rhythms based on the Empirical Mode Decomposition, as an alternative for the analysis of electroencephalographic (EEG) signals, due to its adaptive capability to the local oscillations of the data, showcasing it as a viable tool for future BMI algorithms based on motor related events.

INTRODUCTION
A brain-machine interface (BMI) (Rao, 2013) is a system that allows controlling a device through the analysis of the electric biosignals that can be acquired from the brain, with the help of scalp electrodes. Electroencephalographic (EEG) signals are thus acquired in a non-invasive way, for its posterior processing and interpretation by the algorithms associated with the BMI. EEG activity is usually categorized by different rhythms which are associated with certain brain activities and specific frequency bands: activity below 4 Hz is related to Delta band [sleep waves (Amzica and Steriade, 1998) and motion related cortical potentials (Shibasaki and Hallett, 2006)]; the Theta band is usually assessed between 4 and 7 Hz and has been related, in the literature, to response repression (Kirmizi-Alsan et al., 2006); the Alpha band, which oscillates between 8 and 15 Hz, varies when the eyes are closed and it has been used with Beta band (16-31 Hz) for decoding the motor activity (Pfurtscheller et al., 2006); finally, the Gamma band (32-100 Hz) has been used in movement and attentive focus (Rao, 2013;Costa et al., 2016).
BMIs have emerged as a promising tool to rehabilitate or assist people when they are commanding robotic exoskeletons (He et al., 2018). In the case of the control of robotic exoskeletons by means of a BMI, the most common paradigms are based on: • Movement Related Cortical Potential (MRCP) are timefrequency changes of the signal associated with movements or decision making (Shibasaki and Hallett, 2006). However, their low potential and frequency makes them difficult to detect in a single trial due to the necessity to mitigate artifacts or other mental processes, averaging several events. • Event related Des/Synchronization (ERD/ERS) is based on the fact that during motor intention there is a previous decrement of the power in alpha and beta bands during the 2 s before the voluntary movement, followed by an increase of the power (Pfurtscheller and Neuper, 1994;Pfurtscheller et al., 1997Pfurtscheller et al., , 2006. For the clear identification of the ERD/ERS, it is also necessary tp average several trials. This paradigm has also been used during the mental task of motor imagery (MI), as it can produce a similar effect to the real movement (Jeon et al., 2011;Del Castillo et al., 2018). • Steady-State Visual Evoked Potentials (SSVEPs) are related to the response offered by the brain due to an external stimulus. Its nature is not directly related to the movement or MI, but this response can be used to control an exoskeleton with a BMI (Kwak et al., 2015;Zhang et al., 2015;Gui et al., 2017). Although the potential is not related to a movement action, it can be used to train the user to make a connection between the visual stimulus and the desired robotic action.
In these paradigms, the extraction of features is usually accomplished by the filtering of the frequency bands, associated with the rhythms related to the brain activity being analyzed. Although the features are generally extracted using traditional signal processing filters, there are other alternatives in the literature based on time vs. frequency analysis techniques, such as the wavelet transform (Xu and Song, 2008;Yang et al., 2010;Kant et al., 2019) and the Stockwell transform (Ortiz et al., 2017). However, they have not been used in conjunction with an exoskeleton (He et al., 2018). In the present study, a different approach will be studied using a decomposition algorithm, the Empirical Mode Decomposition (EMD). In this study, the EMD algorithm will be used to detect the variations of the EEG signals during MI tasks in comparison to the relaxed state, when a user is commanding a lower-limb exoskeleton. Due to the oscillatory nature of EEG and the lack of stationary behavior of EEG rhythms, EMD is an interesting algorithm for to use for the analysis of these signals. EMD has been used to detect epilepsy (Martis et al., 2012;Li et al., 2013) or even as an EOG removal technique (Looney et al., 2008), however, even as its utility for EEG analysis has been demonstrated (Rutkowski et al., 2010), its application for developing a BMI to command an exoskeleton has not been explored yet. The results will be compared to those obtained by a Butterworth filter. In addition, the activation of the exoskeleton will be analyzed in the time domain, thanks to the properties of the Hilbert transform.

Subjects
Three able-bodied adults voluntarily participated in the study. All the information was given to the subjects before the experiment FIGURE 1 | Image of a subject executing one of the experimental trials. and they agreed by signing an informed consent form. All procedures were approved by the Institutional Review Board of the University of Houston (USA).

Equipment
Two non-invasive bundles of 32 wet electrodes were used for the EEG acquisition with the help of an actiCap (Brain Products GmbH, Germany) unit. The distribution followed the 10 − 10 international system, reallocating four scalp electrodes around the eyes to assess the ocular artifacts in bipolar configuration (TP10-TP9 for Vertical-EOG and PO10-PO9 for Horizontal-EOG). Reference and ground were placed in the lower lobes of each ear. Data were wirelessly transmitted by a Move transmitter and amplified by two BrainAmp amplifiers (Brain Products GmbH, Germany). The lower-limb exoskeleton used was the REX (Rex Bionics, New Zealand). Activation commands were sent to the exoskeleton wirelessly, while the status of the exoskeleton was received by the computer through wire serial port communication. The control of the exoskeleton was done by custom firmware developed in Matlab. Figure 1 shows a subject commanding the REX exoskeleton during one of the trials. Although REX can be commanded without external help, two individuals were placed by the sides of the exoskeleton to avoid any potential risk of losing balance.

Protocol
The objective of the paper was to compare the assessment of the EEG variations observed in different rhythms associated with motor imagery tasks when a subject is using a lower-limb exoskeleton. With this aim in mind, the protocol was designed as detailed below.
The whole protocol of a experimental session lasted around 2 h and involved the following parts: (1) adaptation of the exoskeleton to the subject, (2) placing and gelling the 64 electrodes, achieving an impedance value below 20k for each electrode, (3) execution of several runs to make sure the subject is comfortable with the REX movement, and finally (4) the fulfillment of the 10 experimental trials. Figure 2 shows the details of an experimental trial, the different events, and the status of the exoskeleton.
Each trial started with 15 s of free mental task to help the subject concentrate in the experiment and to allow enough time to pass for the ocular artifact removal algorithm convergence (Kilicarslan et al., 2016). This period of time was not considered for the analysis, as it was not associated with any mental task. After that, an acoustic signal warned the subject to relax and to try imagine a rest state of no movement. This provided the reference state (bottom blue points) to the MI event (red points) that occurs after the rest event. After 10 s of rest, a new acoustic cue indicated to the subject to focus on the mental task of the movement of their legs (MI start). At the same time, a command was issued to move the exoskeleton. However, as REX has a variable time for activation, the start of the movement lagged for up to 2 s (start of first step point). After the first step was completed (around 5 s after the cue signal) the exoskeleton kept walking at a steady pace (cycling walk) recording the EEG signal for MI assessment (red top). After ∼20 s of MI, another cue signal indicated to the subject to start making a reverse count (black top line). This part serves as a control event to be sure that the MI features were not caused by motion artifacts. After another 20 s, a final acoustic cue indicated the final rest event. This final event was not used in this study as it had no associated MI event.

Empirical Mode Decomposition
EMD was developed by Norden E. Huang as a decomposition method to improve the extraction of the instantaneous frequency and amplitude of non-stationary signals (Huang et al., 1998). The algorithm decomposes the original signals in several modes, called intrinsic mode functions (IMF), to provide a better timefrequency representation using the Hilbert transform. Each IMF contains the local oscillation information of the signal, extracted from higher to lower frequencies.
Given a discrete signal s(t), its EMD can be obtained following the next steps: 1. Assign the signal to a temporal signal x(t) = s(t). 2. Find the local extrema of x(t). 3. Find the maximum envelope e + (t) of x(t) by passing a natural cubic spline through the local maxima. Similarly, find the minimum envelope, e − (t), with the local minima. 4. Compute an approximation to the local average: m(t) = (e + (t) + e − (t))/2. 5. Find the proto-mode function z i (t) = x(t) − m(t). 6. Check whether z i (t) is an IMF. An IMF is a wave that satisfies two conditions: the number of extrema and the number of zero crossings may differ by no more than one; and its local average is zero. The threshold used to set this last condition is critical, as a high value can lead to overtraining and a low value can leave components unextracted. Moreover, to avoid the extraction of accidental IMFs, the conditions must be accomplished in at least two or three consecutive iterations. 7. If z i (t) is not an IMF, repeat the loop on z i (t). If z i (t) is an IMF then set IMF n (t) = z i (t) and begin the process again considering x(t) = x(t) − IMF n (t), being n = 1 : M. 8. The process stops when the desired limit of IMFs has been achieved (n = M), or a maximum number of sifting operations have been done i = p, without the IMF conditions fulfilled. In this case, the residual is considered as res(t) = s(t) − n=1 n=M IMF n Nevertheless, the algorithm is not exempt of drawbacks, such as the difficulty to separate near tones or border effects (Rilling and Flandrin, 2008). Border effects and the time of computation can be a drawback for the EMD employment in real time EEG signals. As processing windows are usually short during online processing, a great part of the data is susceptible to being spoiled by the border effects of a bad envelope calculation during the sifting process. As this research performs an offline analysis of the data, it was exempt of this issue. Border effects were outside the processing windows, as the whole trial was used as input data to the EMD algorithm and not partially processed in epochs. However, its application in real time would require the use of overlapped processing epochs, neglecting the border content. Nevertheless, this effect was studied for the EEG signals of this research. For 1 s epochs overlapped by 0.5 s, the time of processing was around 0.3 s per epoch on average and 40% of the window was spoiled. As the time of computation depends on the hardware, and the border effect depends on the sampling frequency, it is difficult to recommend an epoch length. However, an epoch should be as long and with as short a shifting as the hardware allows for computational times.

Pre-processing
The data acquisition was done at a 1 kHz sampling frequency. However, in order to obtain a consistent set of IMFs, the data was resampled to 200 Hz. The number of IMFs and the frequency tone associated with them depends on the sampling frequency, so a 1 kHz sampling frequency would not provide relevant tones over the 100 Hz. Resampling to 200 Hz provide tones below 100 Hz. In addition, the ocular artifact removal algorithm parameters are optimized to work at 100-200 Hz sampling frequencies.
The H ∞ algorithm employs the information of the pairs of electrodes TP10-TP9 and PO10-PO9 (Kilicarslan et al., 2016). The optimized parameters used were the following: γ = 1.15, q = 10 −10 and p 0 = 0.5. Finally, a 60Hz notch filter was applied to mitigate the network power supply component.

IMF Extraction
Once the previous filters were applied, the EMD, as explained in section 2.2, was used to obtain the different modes of the nine electrodes located at the motor cortex: FC1, FCz, FC2, C1, Cz, C2, CP1, CPz, and CP2. Figure 3 shows an example of decomposition of one of the trials for the electrode Cz. Border effects can be seen at the tails of the signal, especially clear in the case of the end of IMF5. In this study, only the first five IMFs were considered (M = 5), as they were enough to obtain the EEG rhythms to be analyzed as instantaneous frequency indicators.
Instantaneous frequency (f (t)) and amplitude (A(t)) were computed using the Hilbert transform (Huang et al., 1998) of each IMF. Both can be obtained based on the derivative of the instantaneous phase and the module of the analytical function. Given a signal s(t) = IMF n (t), its analytical complex function z(t) can be obtained by applying the Hilbert transform (H()) as: obtaining the instantaneous frequency as a function of the derivative of the instantaneous phase: (2) Figure 4 shows the instantaneous frequency for the five IMFs considered. As it can be seen that IMF 1 oscillates in the Gamma band (40-60 Hz), IMF2 in the Beta band (20-30 Hz), IMF3 in the Alpha band (5-15 Hz), IMF4 in the Theta band (2-7 Hz), and IMF5 in the Delta band (<4 Hz). This associates each of the IMFs with a significant EEG rhythm. It is necessary to clarify that the order of the IMF and its relationship with each rhythm can differ depending on the sampling frequency and the possible noise content of the signal. This is one of the reasons why the signals were resampled to 200 Hz in order to have a decomposition similar to the EEG rhythms.

Event Assessment
Two different analyses were carried out. The first analyzed the behavior of the different EEG rhythms during the MI event.
In this analysis, the changes of power in the different IMFs were calculated taking the previous rest state as a reference. In order to avoid taking into account evoked potentials, the events considered only the stationary parts of the events. This means that only the power of the IMFs, when the exoskeleton was stopped or during stable walking, was considered (see Figure 2 bottom blue for rest and top red for MI). For a determined start event, the variation of power was assessed for each IMF (i) and electrode (ch) based on the power of the signal during the MI periods: and the rest periods: as the percentage of variation of the power taking the rest period as reference: As t = t 2 − t 1 can be different for rest and MI periods, power was divided by it.
In the second analysis, the rhythms were evaluated by its instantaneous variation in time. The instantaneous power of each IMF was computed based on its analytical function as: 3. RESULTS

Motor Imagery Analysis (Power Variation)
A statistic analysis was carried out to detect the significant differences for the P(%) between the subjects and the electrodes using SPSS (Andy, 2013). Data included the P(%) of the five extracted IMFs for each of the three subjects and the nine 16.9 ± 2.4 9.5 ± 1.3 −2.5 ± 1.6 −9.1 ± 4.7 −5.7 ± 5.7
A Shaphiro-Wilks test of normality (S-W) was applied to check for the normality assumption (Ghasemi and Zahediasl, 2012). Averaging the electrodes, S1 and S2 followed a normal distribution for all the IMFs (S1: D (10)  Using the individual electrode P(%) output for each IMF as a dependent variable, splitting the data for each subject and electrode, the S-W test results indicated that S1 followed a normal distribution for all the IMFs and electrodes (D(10) = Analyzing P(%) of the IMFs as dependent variables for each subject without splitting the data by electrode, the S-W test indicated that S1 followed a normal distribution for all the IMFs with only a small deviation from normality in the Q-Q plot for IMF1 [D(90) = 0.919, p < 0.05] and a small asymmetry in the histogram toward the high tail. S-W test for S2 indicated a deviation from the normal distribution for IMF1-4 (D(90) = [0.947 − 0.972], p < 0.05) with only a small skewness (S-shape) in the Q-Q plots. S3 distribution was not normal with a clear double peak shape for all the IMFs in the histograms (D(90) = [0.560 − 0.734], p < 0.05) and a clear kurtosis deviation in the Q-Q plots, which indicated different performance in some of the trials. As normality could not be assured for all the subjects and electrodes, data were FIGURE 6 | Instantaneous power of IMF3 after the acoustic cue for MI (black line) for trial 1 of subject S1. The value is compared with the instantaneous power of the signal filtered by a Butterworth filter (5-15 Hz). The peak response is higher using the IMF. REX start of movement is represented by a red spot.
analyzed using non-parametric tests to compare the subject and electrode dependency.
For the averaged electrodes, using the Kruskal-Wallis (K-W) test, the IMF output was significantly affected by the subject (H(2) = [9. 757, 11.027, 10.934, 17.559, 8.565], p < 0.05, appearing in the data for each IMF). The Mann-Whitney test was used to analyze the differences between the subjects. A Bonferroni correction was applied and so all effects are reported at a 0.0167 level of significance. S1 results were significantly different in comparison to S2 for 17,12,0,15] Analyzing each subject individually, the K-W test indicated that there were no differences between the electrodes for the five IMFs of all the subjects: S1 (H(8) = [3.721, 3.279, 10.252, 6.226, 7.732], p > 0.05), S2 (H(8) = [3.174, 2.183, 2.153, 6.357, 3.910], p > 0.05), and S3 (H(8) = [1. 496, 0.501, 6.790, 11.344, 4.984], p > 0.05). This means that the electrodes behaved in a similar way for a given IMF and subject. This matches the boxplot representation. The left part of Figure 5 shows the boxplots of the power variation for the average of the nine electrodes during the 10 trials accomplished by a subject, while the right part shows the same data for the Cz electrode. The associated descriptive statistics to the boxplots can be seen in Table 1 for all the electrodes and its average, and particularized for the Cz electrode in Table 2.
As can be seen, if the electrodes were averaged through the 10 trials, there was a similar trend for the three subjects, with a clear increment of the power, especially for high frequency IMFs associated with gamma (IMF1) and beta bands (IMF2). However, IMFs3-5 (alpha, theta, and delta bands) showed a more erratic behavior. Focusing our attention on one of the most representative electrodes, Cz (see right part of Figure 5), its boxplot follows the same trend than the average. However, the deviation, as can be seen in the boxplot and Table 2, is higher, which indicates the difficulty to see this pattern in a single trial without averaging the electrodes.
The comparison of the averaged P(%) calculated using EMD and the components extracted from the signal by a second order Butterworth filter, show that the results were similar ( Table 3). The table shows, using bold text, the methods with a higher increment for P(%) in the first two IMFs.

Gait Intention Analysis (Instantaneous Power)
In order to study the transient when a subject focuses on the motion intention, an additional study was carried out. The ERD/ERS phenomenon indicates that for certain subjects, the motor intention involves a desynchronization of the mu band (8-12 Hz) followed by a posterior synchronization (Pfurtscheller et al., 2006). Although the phenomenon can be observed in a single trial, it usually requires averaging several trials. Moreover, the effect is also more difficult to notice in foot imagery than in hands. In this study, EMD is proposed as a technique to evaluate the desynchronization and posterior synchronization of the signal during the motion intention. As literature focuses its attention on the mu band, the IMF under study was IMF3, as its instantaneous frequency oscillated around 10 Hz (see Figure 4 IMF3).
With this purpose in mind, the instantaneous power of IMF3 was assessed as indicated by equation 6. Figure 6 shows the instantaneous power during the 5 s after the acoustic cue was issued (t = 0) for one of the trials (black line). A red spot indicates the real start of the REX movement, which indicates that the previous time to the red spot is a MI period with no movement, exempt for this reason of any possible motion artifacts. The instantaneous power was compared to the one obtained by only using a second order Butterworth bandpass filter to the signal between 8 and 15 Hz (blue line). Upon inspection of the figure, it can be seen that there are several peaks during the motor imagery period previous to the activation of the exoskeleton. For several electrodes there is a first peak around 1 s before the movement and another one just before the movement. As can be seen, the peak that appears just before the start of the movement is especially clear, using the IMF power (black line), compared to that of the filtered signal (blue line). This indicates that there is an increment of power around 1 s after the acoustic cue, followed by a decrement and a posterior increment just before the real movement starts, which could be related to the ERD/ERS phenomenon. However, the peak magnitudes are variable, which makes it difficult to quantify the phenomenon in a single trial. Table 4 shows a comparison of both methods for the time at which the maximum peak of the instantaneous power is computed in the five s after the acoustic cue for the MI event is issued. The mean time shows the averaged time for the 10 trials accomplished by each subject per electrode. A negative time indicates that the peak achieved was average for a channel before the REX activation. According to K-W test, the subjects showed no significant differences when using EMD [H(2) = 5.42, p > 0.05], while they did for the frequency filter band [H(2) = 27.13, p < 0.001]. As Table 4 shows, a negative time was achieved for the average of the 10 trials in all the subjects for the electrodes C1, CPz, Cz, and FC2 when using the EMD method. The filtered signal showed a higher dispersion for the times, detecting the intention later for the majority of the electrodes. Furthermore, Figure 6 shows that there are also several peaks for certain electrodes after the exoskeleton started moving. To check the evolution and magnitude of these peaks and its possible relationship with the statistical results of section 3.1, the instantaneous power is represented for the full trial in Figure 7. This was limited to IMF1 and IMF2, as these IMFs were the ones that showed a clearer increment of power during MI in Table 2. To see the behavior in a clearer way, the instantaneous power was averaged with a moving mean of 1 s. Figure 7 shows this averaged value for the MI (red), rest (blue), and the control period of the reverse count (green) events. Upon inspection of Figure 7, it is evident that the MI periods show a clear increment in rest periods. In addition, they also show an increment in reverse count operations, which indicates that the increment is not related to any possible motion or exoskeleton artifact, since during the reverse count periods the exoskeleton was moving in the same way than during the MI events. In fact, reverse count periods show a similar power to rest periods.

DISCUSSION
By analyzing the results, it has been demonstrated that EMD can be a useful tool in characterizing the EEG rhythms during MI events associated with a BMI operating an exoskeleton.
Regarding motor imagery, an increment of power was noticed, not only in average for the MI vs. rest periods, but also in comparison to an alternative mental task during movement (reverse count) in the case of the high frequency IMFs. The power for the IMFs associated with these IMFs showed a different pattern only for the MI events, while showing a similar one for the rest periods. This is consistent with the literature on gamma power increment during motion (Seeber et al., 2014). This behavior could be explained by the attention focus to gait during these periods (Costa-García et al., 2019), which could be useful for the future development of a BMI based on high frequency rhythms assessing the attention to gait. On the other hand, lower frequency IMFs showed a more inconsistent behavior during MI tasks. This can be explained by the nature of EMD, as low frequency IMFs are more susceptible to over-training problems. In addition, in the case of a real-time application, shorter windows of computation could make it more difficult to obtain a consistent result using these IMFs due to possible border effects. This advise against the use of high order IMFs (low frequency ones), at least for BMI control, even as the residual captures the low frequency oscillations of the EEG channel in a precise way (see Figure 3). Furthermore, the statistical analysis revealed that the results were dependant on the subject. However, given a determined subject and IMF, they have a similar behavior for the nine electrodes considered. Nevertheless, the high deviation of the P(%) per individual electrode makes its use in a single trial difficult, without averaging the nine electrodes through the trials.
Regarding motor intention, the analysis revealed important facts. As Figure 6 shows, motor intention analyzed by the IMF associated with alpha band followed a pattern similar to the one explained by the ERD/ERS (Pfurtscheller et al., 2006;Jeon et al., 2011) in a single trial. The effect was more noticeable than when using a frequency filter. However, it requires a processing window of at least 4 s, which could compromise the BMI response in real time, as processing would delay the command and increase the computational times. In addition, the magnitude of the peaks was variable, which makes computing them in a significant stable base, or to estimate a statistical trend to be used in a BMI algorithm, difficult, even as statistically the maximum peak is usually achieved during the first 5 s before the actual start of movement.
The results obtained for the P(%) were similar to the ones obtained for the filtered signal, but clearer for the motor intention despite its higher computational load. Nevertheless, EMD could be an interesting alternative for the assessment of ERD/ERS, but would require further investigation and an experimental protocol specifically designed for its study. Future research will explore the viability of using this paradigm for the detection  The maximum peak is computed for each electrode within the 5 s after the MI acoustic cue is issued. Zero reference is marked by the instant the exoskeleton starts moving. A negative time indicates that the detection is accomplished before the exoskeleton starts moving. The table shows the averaged time for the ten trials of each subject and its 95% confidence interval. Electrodes with negative times are in bold and methods with negative times for all the subjects appear between brackets. Frontiers in Neurorobotics | www.frontiersin.org of motor intention through the analysis of moving epochs and the usability of high frequency features, extracted with EMD or other time-frequency algorithms, for a BMI that commands an exoskeleton in real time.

DATA AVAILABILITY STATEMENT
The datasets presented in this article are not readily available because of University of Houston IRB restrictions.
Requests to access the datasets should be directed to JC-V (jlcontreras-vidal@uh.edu).

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by Institutional Review Board of the University of Houston (USA). The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individuals for the publication of any potentially identifiable images or data included in this article.

AUTHOR CONTRIBUTIONS
MO and JA: conceptualization. MO: methodology, validation, formal analysis, investigation, writing-original draft preparation, and visualization. MO and EI: software and data curation. JC-V: resources. MO, EI, JC-V, and JA: writing-review and editing. JC-V and JA: supervision and project administration. MO, JC-V, and JA: funding acquisition. All authors contributed to the article and approved the submitted version.

FUNDING
This research was funded by the Spanish Ministry of Education and Professional Formation through grant CAS18/00048 José Castillejo; by the Spanish Ministry of Science and Innovation, the Spanish State Agency of Research, and the European Union through the European Regional Development Fund in the framework of the project Walk-Controlling lower-limb exoskeletons by means of brain-machine interfaces to assist people with walking disabilities (RTI2018-096677-B-I00); and by the Consellería de Innovación, Universidades, Ciencia y Sociedad Digital (Generalitat Valenciana) and the European Social Fund in the framework of the project Desarrollo de nuevas interfaces cerebro-máquina para la rehabilitación de miembro inferior (GV/2019/009).


extfieldsofstudy:
Biology
Computer Science

text:
Using Relion software within Scipion framework

Scipion is a modular image processing framework integrating several software packages under a unified interface while taking care of file formats and conversions. Here new developments and capabilities of the Scipion plugin for the Relion software are presented and illustrated with the image processing pipeline of published data. The user interfaces of Scipion and Relion are compared and the key differences highlighted, allowing this manuscript to be used as a guide for both new and experienced users of these software. Different streaming image processing options are also discussed demonstrating the flexibility of the Scipion framework. Synopsis An overview of the Scipion plugin for the Relion software is presented and various capabilities of image processing within Scipion framework are discussed.

Introduction
The cryogenic electron microscopy (cryo-EM) field continues to grow rapidly due to constant improvements in both instrumentation and software algorithms (Danev et al., 2019). With more high-resolution structures being solved every year (https://www.ebi.ac.uk/pdbe/emdb/statistics_main.html/) by single-particle analysis (SPA) and subtomogram averaging methods, automation of existing image processing pipelines becomes more relevant. Moreover, the introduction of user-friendly graphical interfaces (GUI) plays an important role in attracting more scientists to the field. Scipion (de la Rosa-Trevín et al., 2016) is an integrative software suite that aims to reach a larger scientific audience by providing a convenient modular platform for building custom automatable pipelines. The platform can be adapted to suit specific image processing needs of structural biologists, electron microscopy facilities or software developers. The Scipion framework includes multiple modules (plugins) each providing a set of Python wrappers around a particular cryo-EM software. Scipion collects these modules within a unified GUI allowing for seamless transition between different packages and file formats. Here we describe in detail the Scipion plugin for the Relion software (Zivanov et al., 2019) and illustrate the advantages of the image processing using the Scipion platform.

Plugin overview
With the high popularity of Relion within the cryo-EM community (https://www.ebi.ac.uk/pdbe/emdb/statistics_main.html/) the plugin for Relion is one of the most widely used by Scipion users (http://scipion.i2pc.es/report_protocols/protocolTable/). Recently the plugin has undergone several important changes. First, the code has been migrated to Python 3 since Python 2.7 was deprecated in January 2020. While being entirely transparent to users, this transition has brought many improvements for software developers and allowed us to keep the code base up-to-date. Second, a new STAR file parser emtable (de la Rosa-Trevin and Sharov, 2020) was developed to simplify and speed up metadata conversion between Scipion and Relion, replacing the functions of the plugin for XMIPP (de la Rosa-Trevín et al., 2013), previously used for this task. emtable is available as a small self-contained Python module and can be used to manipulate STAR files independently from Scipion.
Considerable effort was put into compatibility between different Relion versions. Starting with Relion version 3.1, information about optic groups has been added as a second data table to STAR files. Optic groups were designed to keep all information related to the data acquisition (pixel size, voltage etc.) separate from the particle metadata and to allow users to combine different datasets. Despite the fact that we no longer provide support for Relion 3.0 or older versions, users are still able to import older STAR files into Scipion and then continue with Relion 3.1. Most plugin protocols are also likely to work with Relion 3.0. We have created a special protocol called "assign optics groups" to account for the differences between the Scipion EM data model and new optics groups data table. Users can assign the parameters of one or more (using an extra STAR file) optics groups to a set of images at essentially any step of the image processing pipeline. This allows more flexibility compared to the Relion GUI where users have to specify optics groups at the import stage.
At the moment the plugin provides wrappers for most of the programs available from Relion (version 3.1) with the exception of protocols for helical image processing (He and Scheres, 2017) and subtomogram averaging (Bharat et al., 2015) which are still in development.

User interface design
The original idea behind a Scipion wrapper for Relion programs was to make the user interface similar to the original Relion GUI so that Relion users starting to use Scipion could easily orient themselves through many available options. Below we describe the interface of the plugin for Relion and compare it to the Relion GUI ( fig. 1).

Figure 1.
Comparison of the 3D auto-refine protocol GUI in Relion (a) and Scipion (b). Advanced protocol parameters in Scipion are highlighted with gray background.
Every protocol window in Scipion has a similar layout composed of two parts. The top panel includes computation-related parameters such as submission to a cluster queue, waiting list (the current protocol will not start until the listed protocols have finished) and parallelization options (GPU IDs, number of MPI processes and/or threads) if the protocol supports them. Furthermore, many protocols define two levels of user expertise: normal and advanced (see "Expert Level" in fig. 1). Selecting the advanced level displays additional parameters that are less common or reserved for special cases. Many plugin protocols also offer a string-type variable called "Extra parameters" or "Additional arguments". Such strings are reserved for Relion command-line options and arguments that are not available through the standard GUI and can be utilized by advanced Relion users.
The bottom panel of the protocol window includes task-specific parameters as found in the Relion GUI, organised in similarly named tabs. Some of the parameters have an interactive wizard attached ( fig. 2), meaning that one can quickly visualize how changing certain values might affect the results. Most wizards are used to select a particle mask size or to choose image filtering levels, however there are also special cases like choosing a detector MTF file or creating defocus groups. Despite many similarities with the Relion GUI, the plugin for Relion software has several differences due to a conceptually different design of the Scipion framework. One example is the "continue" mode for classification/refinement protocols. While Relion allows continuation of a finished job, in Scipion every run is unique and the user must create a new protocol which should reference the previous one. In this way the meaning of the number of iterations becomes different between Relion and Scipion: for Relion it's a total number of iterations, but for Scipion it's a number of extra "continue" iterations.
Another example is subset selection, which is done in Relion with a separate job type. In Scipion subsets can be created naturally from any set of items in an interactive manner. Scipion uses XMIPP viewer to display sets of images and associated metadata ( fig.  3a). These sets can be displayed as an image gallery or as a table, where each column's values can be sorted or plotted using built-in tools. This means any STAR file can be visualised with Scipion without using a text editor. Moreover, Scipion includes a set of protocols that can perform various operations on image sets such as joining, splitting or intersection. When displaying single items like a micrograph or a particle, Scipion offers a range of ImageJ (Schneider et al., 2012) image processing tools that can be used, for example, to calculate a fast Fourier transform (FFT) or to interactively create a 2D mask from an image ( fig. 3b). The plugin for Relion also includes custom-designed viewers that not only let users visualize output sets of items from a given protocol, but also create various plots for FSC, SSNR, B-factors, angular distribution and other metrics to quickly follow job progress and iteration results. Protocols for refinement and classification provide the possibility to track changes between iterations, such as orientations, offsets or number of images per class. Other examples include frame motion plots for movie alignment protocols and defocus variation plots for CTF refinement protocols. Combined with the possibilities of creating custom plots from STAR files-derived metadata and displaying volumes in ChimeraX (Pettersen et al., 2020), the plugin for Relion offers a rich interactive interface for both novice and expert users.

On-the-fly processing
On-the-fly processing (or streaming) is often used for more efficient data collection by providing quick feedback on data quality. It can also considerably shorten the data processing time by overlapping it with microscope acquisition. Scipion provides various options for running workflows in the streaming mode some of which are discussed below.
Firstly, most import protocols have a "Streaming" tab where users can specify a file timeout and a global timeout. The former is used to recognize when the input file size stops growing and is ready to be imported, while the latter is used to stop streaming once no new data is being acquired. Secondly, for several plugins that can run on GPUs Scipion provides a possibility for parallel multi-GPU execution. Such examples are motioncor2 (Zheng et al., 2017) for movie alignment and GCTF (Zhang, 2016) for CTF estimation. By specifying a number of threads equal to the number of GPUs plus one, a user can submit streaming jobs to several GPUs simultaneously. For instance, setting the number of threads to 3 and the GPU IDs to "0, 1" will lead to thread 1 being a master, while thread 2 will run on GPU 0 and thread 3 on GPU 1. Each worker thread will process a single movie in streaming giving a speed boost in processing. More complex combinations are also possible but depend on the underlying protocol.
Multi-GPU execution, however, is different for Relion protocols where a user usually has to specify both MPI processes and threads. We have left the syntax for GPU IDs the same as in the original Relion GUI to avoid confusion. Users can refer to the Relion website documentation for more details.
The protocols for CTF estimation, particle extraction and auto-picking in most Scipion plugins also have a "Streaming" tab where a user can provide a batch size. By default (batch size 1), all input protocol items will be processed one by one which is common for on-the-fly processing pipelines. However, users can specify a larger number of items that will be then processed "in batches", which sometimes can minimise computational load and speed up processing. Setting this value to 0 is used for non-streaming cases when all input items are processed together.
Altogether, multiple plugins in Scipion support various streaming scenarios for most image preprocessing steps up to 2D classification. Such protocols can be found in the protocol search window of the Scipion project (available by pressing Ctrl+F). Below we show an example of streaming processing on real data.

Interaction with other software
Besides the ability to import data from various cryo-EM packages provided by Scipion itself, the plugin for Relion includes several protocols that can export results into a 5 self-contained folder, allowing users to continue image processing outside Scipion. Export at the level of coordinates, CTFs (micrographs with CTF information) and particles is possible. We are working towards running all export protocols in streaming so that results could be immediately accessed by other software packages or by Relion itself (outside Scipion).
From the beginning the Scipion framework has always been focused on providing a transparent combination of different software packages. As a result, Relion users are free to incorporate various programs for movie alignment, CTF estimation and particle picking into a standard Relion pipeline. Global movie alignment parameters produced by Motioncor2 (Zheng et al., 2017) or cisTEM -unblur (Grant et al., 2018) can be used as input for Relion's bayesian polishing. Moreover, the plugin includes several protocols which do not have their own GUI in the RELION interface. Such examples are centering of class averages, movie compression, gain estimation for movie compression, volume symmetrization and several others.

Image processing
To demonstrate the capabilities of the plugin for Relion we have chosen the EMPIAR-10389 dataset (Righetto et al., 2020). The deposited dataset consists of two sets of movies: one acquired with beam-image shift (multi-shot) and one without (single-shot). The published workflow leads to a 2Å resolution map and comprises a wide range of protocols including the basic steps from pre-processing to 3D refinement along with more complex image processing tasks such as changing the dataset binning, particle re-centering, joining of different datasets, particle polishing and CTF refinement. We think this workflow provides a good example to illustrate Scipion's capabilities for image processing. Even though based predominantly on Relion, this example shows the advantages of performing these operations within the Scipion framework, rather than using the Relion GUI. To reduce computational costs we have simplified the pipeline by removing several intermediate steps, however the general workflow outline remains the same ( fig. 4). To validate the results reproducibility between Relion and Scipion, we have first run this workflow in the Relion 3.1 GUI (outside Scipion) to replicate the steps authors used in their publication to the best of our knowledge. Both procedures are described below.

Processing in Relion outside Scipion
The raw movies were downloaded from EMPIAR (Iudin et al., 2016), split into 4 optic groups based on the movie filename (3 multi-shot groups and 1 single-shot group) and imported separately into the Relion project. Movie STAR files for multi-shot groups were joined together and then both multi and single-shot datasets were processed in the following manner. Movies were motion-corrected with Relion motioncor (no frame grouping, using 5x5 tiles) and amplitude power spectra sums were produced for further CTF estimation with CTFFIND 4.1.14 (Rohou and Grigorieff, 2015). Micrographs with estimated CTF fit lower than 4Å were excluded from further processing. The single-shot dataset was first processed separately. 7 Initially, the LoG-picker algorithm was used to pick particles, which were then extracted using a 512 pixel box size and binned 8 times to a pixel size of 5.11Å in order to speed-up the downstream processing. Around 80,000 binned particles were classified into 80 2D classes with the "Ignore CTF until first peak" option enabled. Particles from the best 2D classes showing relevant high-resolution features were regrouped (using the "Subset selection" job) and subjected to the stochastic gradient descent algorithm implemented in Relion to generate an ab initio 3D map (applying tetrahedral symmetry). The resulting map was used as a reference for the masked 3D refinement that reached the Nyquist limit for the binned data. The particles were then re-extracted with a binning of 4 times using re-centered coordinates by applying shifts from the refinement, and the 3D refinement was repeated. Reextraction and refinement cycles were repeated until the unbinned box size was used, leading to a 2.84Å reconstruction from ~ 45,000 particles. In comparison with the publication, we did not perform further refinements, 3D classification or Bayesian polishing for this dataset.
Afterwards we used a reference-based Relion particle picker with the 3D reference map from the last 3D refinement with the data binned twice. We picked approx. 97.000 and 129.000 particles from the single and multi-shot datasets, respectively. The particles were separately extracted, joined and 2D classified (again binned 8 times with a 64 pixel box size to speed-up calculations). Then the particles were extracted again (binned twice) and compared with their counterparts (binned 8 times). Here we had to modify the STAR files manually so they would point to the best particle subset after the 2D classification but with a different binning level. This particle subset was submitted for a 3D refinement that led to a 2.66Å structure from a total of ~ 118.000 particles. Next, we performed a masked 3D classification (without imposing symmetry) without alignment to further clean the data. Finally, all particles from the best looking and largest 3D class were re-extracted unbinned and refined to a 2.6Å map. We continued the processing with three sequential rounds of CTF refinement to correct for magnification anisotropy, to refine the defocus per particle and astigmatism per micrograph and to correct for high-order aberrations, respectively. This was followed by Bayesian particle polishing using the parameters from the published manuscript. A final round of 3D refinement and post-processing led to a 2.08Å map. This is above the 1.98A reported in the publication. We believe the difference to be caused by the fact that we did not iterate 3D refinements after each CTF refinement step which could have improved the angular accuracy.

Processing in Scipion
After completing the workflow in the Relion GUI, we have switched to the Scipion plugin for Relion. Here all protocols parameters were set consistent with the Relion values used above. Similarly, we have first focused on the single-shot dataset. Gain references and defect files were downloaded from EMPIAR in advance and the preprocessing was performed in streaming mode. As movies were being downloaded the following steps were executed for each movie: import, Relion's motion correction, CTF estimation with ctffind4 (via the plugin for cisTEM), sorting micrographs by CTF fit (via "ctf consensus" protocol of the plugin for XMIPP), Relion's LoG picking and particle extraction.
Scipion offers several options for the workflow's execution: a) importing a predefined workflow file in JSON format into a Scipion project and then running it (done via Scipion GUI or through the command line (https://scipion-em.github.io/docs/docs/facilities/facilitiesworkflows.html)); b) scheduling protocols to run one after another using the waiting list option; c) creating a workflow interactively: as soon as a single output item of one protocol appears, it can be connected to another protocol as input. Additionally, it is also possible to process items in batches in streaming mode as described earlier. Here we used a combined approach and set up Scipion to run import and motion correction on a one-by-one item basis while CTF estimation, picking and extraction were run in batches on every 20 micrographs. Further processing of the single-shot dataset up to the final 3D refinement with unbinned data was done using the exact same job parameters as for Relion. Rescaling of 3D reference volumes was done with the plugin for XMIPP. Particle re-extraction was performed in a different way compared to Relion. Currently, the particle extraction protocol in the plugin for Relion does not support input particles, so a user must run an "extract coordinates" protocol, which produces a new set of coordinates taking into account particle shifts, in our case from the 3D refinement. This is followed by a normal particle extraction step. Final 3D refinement for the single-shot dataset reached a 2.87Å resolution.
For the multi-shot dataset the preprocessing was done in a standard non-streaming way as joining item sets currently is not possible on-the-fly in Scipion. Similarly to the Relion workflow described above, particles from both single and multi-shot datasets were picked using a 3D reference volume from the previous 3D refinement with the data binned twice. The 8 times binned particles from both datasets were extracted, joined and submitted to 2D classification. Then twice binned particles were re-extracted, joined and cross-referenced with their more binned counterparts using the "pwem -subset" protocol with intersection option. Here no STAR file editing was required to compare particles with different binnings nor to select the best subset after classification. Scipion is able to compare any two item sets as long as they belong to the same "type", in our case, particles. Afterwards, 3D classification without alignment, followed by re-extraction of unbinned particles from the best class and subsequent 3D refinement was performed resulting in a 2.60Å structure. Before starting CTF refinements we used "relion -assign optics" protocol providing a simple STAR file with two columns, movie name and optics group number, as input to tell Relion that we have 4 different beam tilt classes in our particle set. When using Relion GUI, this step is done at the import stage or by manually editing the STAR files. In Scipion we provide a semi-automated solution where optic groups can be assigned at any step of the workflow. Once all three CTF refinement rounds were completed, we joined the single and multi-shot movies after motion correction for and assigned optic groups again (now to the movies), since particle polishing in Relion also requires movies as input. The final 3D refinement with polished particles reached 2.14Å resolution. We find the final resolution difference between the Relion workflows inside or outside Scipion negligible, which is expected as Scipion wrapper essentially executes the exact same Relion commands provided that input parameters are the same. Minor variations can be attributed to variations in user-based class or particle selections, different 3D refinement starting seeds or small variations in the shifts application during particle re-extraction.

Conclusions
Here we describe the functionality of the Scipion plugin for Relion software with the focus on single particle image processing for cryo-EM. The plugin provides a set of Python wrappers for the Relion GUI and command-line programs while at the same time offering all capabilities of Scipion's image and metadata handling via a user-friendly graphical interface. More documentation on the Scipion software can be found at https://scipion-em.github.io/docs/. The plugin code is freely available on Github (https://github.com/scipion-em/scipion-em-relion) and distributed under the GNU general public license (GPL3). For user questions and feedback, please use Github Issues or the Scipion mailing list at https://sourceforge.net/projects/scipion/lists/scipion-users. 9


s2fieldsofstudy:
Computer Science
Psychology

text:
Identification of Hand Movements from Electromyographic Signals Using Machine Learning

Electromyographic (EMG) signals provide information about a person's muscle activity. For hand movements, in particular, the execution of each gesture involves the activation of different combinations of the forearm muscles, which generate distinct electrical patterns. Conversely, the analysis of these muscle activation patterns, represented by EMG signals, allows recognizing which gesture is being performed. In this study, we aimed to implement an automatic identification system of hand or wrist gestures based on supervised Machine Learning (ML) techniques. We trained different computational models and determined which of these showed the best capacity to identify six hand or wrist gestures and generalize between different subjects. We used an open access database containing recordings of EMG signals from 36 subjects. Among the results obtained, we highlight the performance of the Random Forest model, with an accuracy of 95.39%, and the performance of a convolutional neural network with an accuracy of 94.77%.

Introduction
EMG signals are generated by the electrical activity of skeletal muscle fibers and can be captured through two ways. First, invasive methods involve the insertion of a needle directly into the muscle; second, non-invasive methods rely on surface electrodes that are placed on the skin in areas close to the target muscle. In particular, surface EMG is the most suited for establishing human-machine interfaces of daily use [1]. These kind of signals present subtle variations between subjects, associated with anatomical differences and motor coordination [2]. The variations are random and highly non-stationary [3]; therefore, it is necessary to use different techniques of feature extraction and selection for time series, which are also useful for developing optimal models of artificial intelligence (AI) [4].
These signals are used for several purposes by health professionals, including disease diagnosis and rehabilitation. Recently, with the rise of automatic learning techniques, these signals are used in systems for the classification of movements [5][6][7], the recognition of motor unit action potential (MUAP) [8], and the diagnosis of neuromuscular diseases [9]. These tasks are achieved by measuring and studying the features that can be extracted from EMG signals. Feature extraction can be done through different techniques, such as autoregressive models [10], signal entropy measurements [11], and statistical measurements of amplitude in the time and frequency domains [4]. In particular, the Wavelet transform, which provides information in the frequency domain at both high and low frequencies, is highly used [12,13].
The feature extraction stage is followed by the implementation of classification models that can learn from the extracted features. Regarding computational models, AI is considered the main domain, which includes ML and Deep Learning (DL). AI refers to the process of displaying human intelligence features in a machine. ML is used to achieve this task, while DL is a set of models and algorithms used to implement ML [14]. In general, ML and DL technologies are powerful for extracting features and finding relationships between data; therefore, these approaches are suitable for tasks in which they can take advantage of human experience [15][16][17]. Machine and Deep Learning algorithms applied to electrophysiological signals constitute a rapidly growing field of research and allow researchers to train computer systems as experts that can be used later on to support the decision-making process.
In ML, the most commonly used models are Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Ensemble classifiers. A SVM was used in [18] to classify human gait phases based on bilateral EMG recordings. The KNN model was used in [8] to indicate how well different types of features could be used to decompose an EMG signal into its component motor unit potential trains (MUPTs). As for Ensemble classifiers, these include the ensemble of SVM used in [19] to detect neuromuscular disorders using statistical measurements from each sub-band of the discrete wavelet transform (DWT) as features; Random Forest (RF) used in [9] to classify disorders as neuropathies or myopathies based on EMG signals using features extracted from DWT coefficients and power spectral density; and Rotation Forest used in [20] for the diagnosis of neuromuscular disorders. Finally, in DL, different architectures are reported, including fully connected artificial neural networks (ANN) and convolutional neural networks (CNN). In particular, [21] used an architecture that includes a convolution layer with 32 filters, a ReLu activation layer, a MaxPooling layer, a fully connected layer, and, lastly, a softmax output layer with six units.
This article proposes a novel approach to EMG signal processing, in which not all the information of the time series is considered. Here, we use only the raw data for hand gestures (i.e. the muscular activation values provided by each channel) to identify the patterns associated with six gestures, regardless of the subject performing the action. We compared different ML algorithms to improve the precision in the identification and classification of hand gestures from the EMG signals. More important, we aimed to generalize among different subjects, which is still an issue regarding this classification task. The classification accuracies obtained using this approach are over 90% for several models, which demonstrates its feasability.
This article is organized as follows: section 2 presents the materials and methods, including the database and models used in the experimental process, which is explained in detail in section 3. Section 4 contains the experimental results and, lastly, sections 5 and 6 correspond to the discussion and conclusions, respectively, derived from the research.

Database
The information used to develop this research was retrieved from a free database called EMG data for gestures Data Set available on the website of the UCI Machine Learning Repository. This database contains surface EMG (sEMG) signal recordings in a time interval where six different static hand gestures are executed, providing a total of 72 recordings from 36 patients. The static hand gestures referred to include resting hand, grasped hand, wrist flexion, wrist extension, ulnar deviation, and radial deviation. These gestures were performed for three seconds, each with a three-second interval between gestures. The information was collected using a MYO thalmic bracelet; this device has eight channels equally spaced around the patient's forearm. Accordingly, each recording consists of an eight-channel time series, in which each segment is properly labeled with numbers zero to six, where zero corresponds to the intervals between gestures and numbers one to six refer to the gestures mentioned above. Table 1 relates the gestures to the classes or labels.

Label
Gesture 0 Intervals between gestures 1 Resting hand 2 Grasped hand 3 Wrist flexion 4 Wrist extension 5 Ulnar deviation 6 Radial deviation Table 1. Labels of the gestures contained in the database This database was consolidated by [2] during their research on the factors limiting human-machine interfaces via sEMG.

Conventional models
The ML models implemented in this experiment were the following: K-Nearest Neighbors (KNN), Logistic Regression (LR), Gaussian Naive Bayes (GNB), Multilayer Perceptron (MLP) using one hidden layer, Random Forest (RF), and Decision Tree (DT).

K-Nearest Neighbors (KNN)
is one of the most recognized ML algorithms for solving classification tasks. This model is quite simple and consists of storing the labeled training data. To classify an unlabeled object, a similarity metric between the object and the labeled objects is computed to identify the k closest elements to the unlabeled object. Then, the labels of these closest elements are used to perform the classification. If k=1, then, the label of the unlabeled object is determined by the label of the closest element of the training set; otherwise, if k> 2, the label is determined by vote [22].
Logistic Regression (LR) is a linear classification model that computes the weighted sum of the imput features and uses a Sigmoid activation function to compute class probabilites and make a prediction [22].
Gaussian Naive Bayes (GNB) is based on a probabilistic model derived from Bayes' theorem. This classification model assumes that all features are independent from each other and determines the probability that an instance belongs to a class based on the feature probabilities. In the Naive Bayes classifier, the features are assumed to be normally distributed (Gaussian distribution) [23].
Multilayer Perceptron (MLP) model has multiple stages of processing and can be considered a generalization of linear models. Essentially, this model uses multiple linear units that compute the weighted sum of the input features and are affected by an activation function. In MLP, this process is repeated multiple times to compute an intermediate processing step called the hidden layer, which is then used to generate the final result [22].
Random Forests (RF) are in the category of ensemble learning methods, which combine multiple ML models to create a more robust one. A RF is basically a collection of Decision Trees, where each tree is different from the others. The differences between the trees can be accomplished in two ways; by randomizing the selection of the training data to build the tree or randomizing the selection of the features used in each question [22]. The overall prediction is made using the predictions of the ensemble by majority vote.
Decision Trees (DT) are widely known ML algorithms used for classification or regression tasks. The basis of this model is that it learns a hierarchy of yes/no (if/else) questions that allow making a prediction [22].

Deep Learning models
The concept of DL comes from artificial neural network research. Unlike the traditional proposals, modern DL has achieved training stability and the ability to generalize, even on big Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 28 February 2020 doi:10.20944/preprints202002.0443.v1 data. Nowadays, DL is the technique of choice to achieve the highest predictive accuracy since DL algorithms perform quite well in a variety of problems. DL architectures are computational models involving hierarchical feature extraction, which is typically accomplished using multiple levels of nonlinearity [14]. Particularly, artificial neural networks occupy a prominent position among the many supervised learning algorithms currently used in ML [24].

Fully connected artificial neural network (ANN) architecture
A fully connected artifical neural network is characterized by having all units between layers connected; therefore, the activation values computed by a unit are passed to every unit in the following layer. In general, the activation values of a layer are given by equation 1.
W represents the weights of the layer, x corresponds to the input data of the layer, which can also be the activation values of the previous layer , and b is the bias. The sign · is used to indicate the matrix multiplication operation and g() represents the activation function used, usually nonlinear.
The model implemented here corresponds to a network with three hidden layers and one output layer. The output layer has six units corresponding to the number of movements that we aim to identify. The activation function for the output layer was softmax and the loss function was categorical cross-entropy. The diagram of the ANN architecture is shown in figure 1. The main feature of these networks is that they use the convolution operation to extract features from the data. Each convolutional layer has several kernels, or filters, that are slid over the data while computing the number of coefficients that can be used as features. Given its nature, the output dimension of a convolutional layer depends on the size and amount of filters used. The convolution operation for one layer is presented in equation 2.
f i represents the i-th filter of the layer, x corresponds to the input data of the layer, which can also be the activation values of the previous layer, and b is the bias. The sign * is used to indicate the convolution operation and g() represents the activation function used.
The model implemented in this study corresponds to a convolutional neural network with two 1D convolutional layers, one ReLu activation layer, three fully connected layers, and one output layer. The output layer has six units corresponding to the number of movements that we aim to identify. The activation function for the output layer was softmax and the loss function was categorical cross-entropy. The diagram for the CNN architecture is presented in figure 2.

Tools
The complete experimental procedure was implemented in Python v. 3.7.3 programming language. We used Pandas library v. 0.24.2 for database import and preprocessing. The ML models mentioned in section 2.2, GridSearchCV, Principal Component Analysis (PCA), train-test split, and the scaling function were implemented using the Scikit-learn library v. 0.21.2 [25]. The artificial neural network architectures (section 2.3) and the corresponding training and validation processes were implemented using the Keras library v. 2.2.4 [26].
The resources used in this research are available in a public GitHub repository containing the EMG recordings files from the database, as well as a Jupyter notebook with the implemented source code. The repository is available here.

Importing the data
The data from the database is presented as a plain text file (.txt). We used the read_csv function from Pandas library to import these files.

Preprocessing
We removed the data that was labeled with the number zero since it corresponds to the resting intervals between gesture execution and does not contain relevant information for the solution of the problem. Thus, only the data captured during the execution of the six gestures of interest were kept. We did not apply any feature extraction procedure (i.e. raw data) because we intended to use the values of the eight channels provided by the device as discriminating information between gestures.

Dataset creation
For dataset creation, we guaranteed that the patients were correctly distributed into the training, validation, and test sets. Accordingly, each patient was present in only one set to avoid bias induced by considering data from the same patient for training and testing. With this in mind, the training set was formed by concatenating the data from 30 patients; the validation set comprised three patients, and test set included the remaining three patients. The patients conforming each of the sets were randomly selected. A second group of sets was created by scaling the first one; in this way, all the features equally contributed to the training process. The sets described above were used during the hyperparameter adjustment process and to compute the test set accuracy metric. Finally, to conserve the correct patient distribution during the cross-validation process, each of the nine k-folds was manually generated by concatenating the data from four random patients.

Hyperparameter adjustment
Following dataset creation, we adjusted the parameters inherent to each classification method assessed. Table 2 summarizes the parameters that were adjusted in the ML models, as well as the ranges used for each one. For the parameters not listed in this table, we used the default values defined by the library.

Model
Parameters and range KNN n_neighbors in the range of 1 to 6, with step 1.  Given the number of variable parameters for ANN and CNN architectures, we decided to adjust these one or two parameters at a time using the GridSearchCV method implemented in the Scikit-learn library. This method selects the best combination of parameters using the result of a cross-validation process. The parameters were adjusted in the following order: batch_size and epochs, optimization algorithm, network weight initialization, activation function, and, finally, neurons in hidden layers. Table 3 presents the adjusted parameters for the ANN and their corresponding ranges, and Table 4 shows the parameters set for the CNN.

Model training and hyperparameter optimization
For parameter optimization, we used an approach similar to a grid search. Given a set of parameters and value ranges, the model was trained and tested on the validation set using every possible combination. For traditional ML models, this process was implemented manually using for cycles, whereas for artificial neural networks, we used the GridSearchCV function from Scikit-learn library due to the high number of parameters.

Evaluation of the model using the best parameters
After performing hyperparameter optimization, the best set of parameters for each model was used to generate the definitive evaluation metric. For this, we used cross-validation to measure model performance. This method uses the complete feature matrix and divides the dataset into k-folds parts. In this case, we used k-folds = 9 and each part contained data from four randomly selected patients. The data was iterated over these partitions, using eight for training and one for validation until they all passed through both states. This procedure was used to check the stability of the results. In addition to cross-validation, we calculated model accuracy on the test set to estimate the model's ability to generalize. The test set had not been used in any previous stage of the experiment. Finally, as class measurements, we used precision, recall, and F1-score.

Results
Parameter optimization of the ML models was visualized by the graphs shown in figure 4, which display the model performance behavior during parameter adjustment. According to these graphs, we selected the parameter values and dataset (Normal/Scaling) for each of the ML models, which are listed in table 5.

Model
Best parameter value  For the ANN architecture, the results of the parameter adjustment using GridSearchCV are presented in table 6. Using these values, we trained the ANN and generated graphs of performance accuracy and loss in each training epoch to observe the model behavior and determine the performance  Figure 5 contains the model accuracy and loss curves during training. For the CNN architecture, the results of the parameter adjustment using GridSearchCV are shown in table 7. The number of epochs was manually adjusted according to the model training curves; as a result, the most appropriate value found was Epochs = 100. Figure 6 shows the accuracy and loss curves during training for CNN using the parameters listed in table 7.

Parameter
Optimal value batch_size 50,000 epochs 150 optimization algorithm Nadam(lr = 0.01) network weight initialization normal activation function tanh neurons in hidden layers 1,050   The two methods mentioned in section 3 for measuring model performance, namely cross-validation and test set accuracy, were implemented using the optimized parameters for each model. The results are shown in table 8, where STD stands for standard deviation.

Model
Cross-validation Accuracy (%) ± STD (%) Summarizing the results from table 8, the LR, GNB, and MLP models displayed the lowest performance. On the other hand, the best results belong to the KNN, RF, DT, and DL models; the best three results are highlighted in yellow. For these models with the best performance, we also calculated accuracy, recall, and F1-score per class. We found that the resting hand gesture shows the best classification according to these measures. The results are found in table 9.  Table 9. Performance by class

Discussion
In this study, we explored the performance of different ML and DL techniques in the task of classifying six hand gestures from electromyographic signals without using any feature extraction method for time series. We demonstrate the feasibility of using the information provided by each of the channels as discriminant features between the gestures. Thus, it is possible to use each record as an instance of the features matrix and obtain a more significant amount of labeled information for model training. Also, this approach allows avoiding computational costs and reduces the time required to implement the different feature extraction methods. For instance when training using Google Colaboratory platform in a GPU environment, the KNN model takes less than a minute for training and testing; RF takes approximately seven minutes, and both DL models take nearly 20 minutes. These computational time metrics are not provided by other authors, which makes it difficult to determine if our approach is faster. A novel finding was that SVMs, or at least the Python implementation by the Scikit-Learn library, require a high amount of computational resources when training over a high-dimensional feature matrix, such as the one generated here. We used ensemble classifiers (Bagging) to train up to ten models in parallel with a fraction of the dataset; however, the computational time and results were not encouraging enough to spend additional time attempting to run this model.
The best results described here are comparable to literature reports on similar problems and on the same classification task. Concerning the best results from previous studies, we highlight those by [20], who obtained 99.2% accuracy in MUAP classification using the statistical properties of each sub-band of the discrete Wavelet transform as features. On the other hand, [21] report a 98.12% accuracy for the identification of seven hand gestures using four time-domain features and stacked sparse autoencoders as a classifier. Working on a similar classification task, [27] achieved a 96.38% accuracy when using RF model to identify three hand gestures in a subject-indenpendent fashion with data from 10 subjects, the best model on their investigation matches the model that achieved the highest accuracy in our investigation.
The contribution of this study is to demonstrate the feasibility of identifying, by means of traditional ML and DL models, different hand gestures from raw EMG signals recorded from the surface of the subject's forearm, as well as to generalize among 36 subjects and still achieve high classification accuracy. An important detail that we want to mention, since we did not see the researchers suggest this topic, is the correct way to distribute the patients in the training, validation and test sets, this implies that the information of a patient is limited to only one of the sets; this way the results are reliable avoiding the bias induced by having information of the same patient in two or even all three sets. This is an important point when designing and implementing ML models since they can memorize the information in the training set, and if we have the same information in the validation or test set, we can obtain biased performance measures.

Conclusions
Our results confirm the feasibility of using Machine Learning and Deep Learning techniques to identify muscle activation patterns, specifically, hand movements. Further, we present a different approach to this classification problem using raw data provided by the MYO thalmic bracelet device to train models with high performance levels, regardless of the subject to which the signals belong. This allows overcoming the different anatomical and biological factors between subjects, which translate into subtle differences in EMG signals. The best performances reported here were obtained by implementing Random Forest with 95.391% precision, Convolutional Neural Network with 94.77%, Artificial Neural Network with 93.73%, Decision Tree with 93.29%, and K-nearest neighbor with 93.22%. As future work, we intend to optimize and adapt our trained models for real-time classification tasks; for instance, in the control of movements of a prosthesis.


The flatness levels of flexible road based on roadroid software in Perintis Kemerdekaan Street Makassar

This study aims to evaluate working road pavement conditions, the road pavement conditions value based on the International Roughness Index (IRI) method which is used as a basis for knowing the type of road maintenance handling type. For taking IRI, a vehicle that has good performance is used, while IRI value recording uses Roadroid software that has been installed on an Android-based mobile. The data results from this study indicate that the Perintis Kemerdekaan Road produces IRI values of 2.607 and 2.408 for each direction. From this value, it is converted in road stability level which shows that the road is in a steady category so that in general this segment for handling requires regular maintenance periodically to achieve maintaining the service level.

Introduction
Road infrastructure that is burdened by high and repeated traffic volumes will cause a decline in road quality. As an indicator, it can be seen from the road surface conditions, both structural and functional conditions that have been damaged. Road damage is caused partly by excessive overloaded traffic, heat/air temperature, water and rain, as well as the initial quality of bad road products. Therefore, in addition to being appropriately planned, the road must be adequately maintained so that it can service traffic growth over the road plan. Routine and periodic road maintenance needs to be done to maintain road safety and comfort for users and maintain durability until the road age planning. Driver comfort is affected by road surface unevenness level so that it is necessary to check road conditions periodically. Surveys of pavement conditions need to be regularly conducted both structurally and non-structurally to determine the level of road service available. Non-structural (functional) examinations are aimed at checking roughness and slip resistance. The test is intended to define the road surface that is unevenness level value. This measurement is used in maintenance planning or road improvement programs.
One parameter of road conditions that can be used is the IRI (International Roughness Index). The IRI is a road surface flatness which is stated by the number of vertical changes in the road surface for each unit of road length (mm/km). To find out the flatness road surface level can be measured using various methods recommended by Bina Marga and AASHTO [1].
In its implementation, measuring road roughness requires a high cost. Considering the limited allocation of survey funds to public works agencies in the region, as an alternative, an Android-based Road (Roadroid) level measurement application can be used. Applications are a practical choice that has advantages regarding relatively low prices, producing accurate data, and easy to operate. Roadroid is a program developed in Sweden by Lars Forsflof with the prototype that appeared in 2002 and was

Pavement Damage
As a result of repetitive loading, the burden and other factors will decrease the quality of service, which will eventually be damaged. Damage to a road construction can occur due to [2], a) Damage due to poor implementation of initial work, as a result of design errors, inadequate supervision and poor material quality; b) Damage due to usage and time such as surface wear, traffic abrasion, installation of utilities, marking on road surfaces, connection brittle etc.; c) Damage due to particular causes such as accidents, holes and landslides. Road damage can be divided into types of damage, namely structural and functional damage. The second description of the damage is [3], a) Structural Damage is damage to the road structure, in part or in whole, which causes road pavement to no longer be able to support traffic loads. For this reason, it is necessary to strengthen pavement structure by overlaying or repairing existing pavement layer; b) Functional Damage is road surface damage which can disrupt road function. This damage can be related or not with structural damage. In functional damage, road pavement is still able to withstand the workload but does not provide the level of comfort and safety as desired. For this reason, the pavement surface layer must be treated so that the surface returns well. According to the Road Maintenance Manual No: 03/MN/B/1983 issued by the Directorate General of Highways, road damage can be distinguished [4], a) Cracking, namely hair cracking, cracking of alligator cracks, edge cracks, cracks of shoulder joints and edge joint cracks, lane joint cracks, widening cracks, reflection cracks, shrinkage cracks, slippage cracks; b) Distortion, namely grooves (ruts), curly (corrugation), shoving, collapsed (grade depressions), upheaval; c) Disintegration, namely potholes, ravelling, stripping; d) Wearing (polished aggregate); e) Bleeding/flushing; f) Decrease in utility plantations.

Surface Flatness
The level of road flatness (International Roughness Index, IRI) is one of the factors/service functions (functional performance) of a road pavement that is very influential on the driver's comfort (riding quality). The quality of the existing and planned roads must be following applicable standards and provisions. The main condition of a good road is strong, flat, waterproof, durable and economical throughout the planned life. It is necessary to periodically monitor and evaluate so that the correct construction repair method can be determined. Measuring the road surface flatness level has not been widely conducted in Indonesia given the constraints of limited equipment so that the flatness requirements in monitoring and evaluation of existing road construction cannot be carried out properly according to national standards in the road sector. To find out the road surface flatness level can be measured using various methods that have been recommended by Bina Marga and AASHTO. The road surface is flatness measurement methods, commonly known are NAASRA methods (SNI 03-34261994). Another technique that can be used for measurement and analysis of flatness of pavement is Straight Edge Rolling, Slope Profilometer (AASTHO Road Test), CHLOE Profilometer, and Roughometer [5].

Road Conditions
Road conditions are something that needs to be considered in determining road maintenance programs. According to the Department of Public Works of the Directorate General of Highways (1992), road conditions can be classified as follows; a) Roads with good conditions are roads with really flat pavement surfaces, no waves and no surface damage; b) Roads with moderate conditions are roads with flat pavement surface, starting with waves but no surface damage; c) The road with a mildly damaged state is a road with a pavement surface that has begun to wave, starting with surface damage and patching (less than 20% of the road are being reviewed); d) Roads with severely damaged conditions are roads with pavement surfaces that have a lot of damage such as wavy, crocodile cracks, and large enough peeling (20-60% of the road being reviewed) accompanied by damage to the foundation layer such as collapsing.
Maintenance is all types of work needed to maintain and repair the road to remain in good condition or work related to both, thus preventing the decline or deterioration of quality at the rate of rapid change 3 that occurs as soon as construction is conducted. Assessment of road conditions in this study only uses the IRI (International Roughness Index) method.

Definition of Road Stability
Road Stability is a condition that needs to be considered to see a condition of road pavement construction. The definition of each term road stability is as follows [6], a) Steady Road Construction is a road with construction conditions in a stable corridor which for handling only requires maintenance activities. The solid road of construction is set according to the Minimum Service Standards, the road is in good and medium condition, where the limit of the IRI <8 m/km is used in this study; b) Unstable roads are roads with conditions outside the stable corridor which for minimum handling are periodic and maximum maintenance of road improvements with the aim of increasing construction structure value. The concept of road stability level used by the Directorate General of Highways is based on data available from the collection system owned, the parameters used are; a) Road flatness parameters or International Roughness Index (IRI); b) Road width parameters and Volume/Capacity Ratio (VCR); c) Parameter of road width and Daily Traffic Volume (LHR).

International Roughness Index (IRI)
The International Roughness Index is a parameter used to determine road surfaces unevenness level. Roughness parameters are presented on a scale that illustrates the road pavement surface unevenness felt by the rider as a function of longitudinal and transverse sections of the road surface. Besides these factors, Roughness is also affected by operational parameters of the vehicle, which includes suspension wheels, vehicle shape, vehicle flatness and speed.
In general, road roughness can be defined as the deviation of road surface measured from a flat plane, plus other parameters that can affect the following; vehicle dynamic movement, travel quality, construction dynamic load and drainage of water on the surface. The International Roughness Index (IRI) is used to measure road surface roughness, that is measured at each location is assumed to represent all the physical in that location. Road surface roughness is the name given to the unevenness extending on the road surface. It is measured by the effect scale of the surface of the vehicle moving on it. The scale that is widely used in developing countries such as Indonesia is the International Roughness Index [7].
This flatness road level (IRI) is one of the service functions (functional performance) of a road pavement that is very influential in comfort (riding quality). One technical indicator for assessing road surface performance is the IRI (International Roughness Index) value, which is a measure of size that describes surface unevenness value indicated as the cumulative length of the surface rise and fall per unit length. The road surface flatness is considered as a result of the overall road pavement conditions. If it is relatively flat, then the road is considered good starting from the lower to top level of pavement and vice versa [8]. The IRI value is expressed in meters of up and down per kilometre road length (m/km). If the IRI value is 10 m/km, it means that the number of amplitude (up and down) of the road surface is 10 m in each km of road length. The higher the IRI value, the worse the pavement surface is. The IRI is a roughness measurement standard that refers to the Response-Type Road Roughness Measurement System (RTRRMS).
The road is the surface flatness measurement method known in general is the NAASRA method (SNI 03-3426-1994). Another method that can be used for measurement and analysis of pavement flatness is Straight Edge Rolling, Slope Profilometer/AASHTO Road Test, CHLOE Profilometer, and Rouge meter [5]. According to Saleh, et al. (2008) the determination of minimum road conditions is medium, in Figure 1 it appears the IRI level between 4.0 m/km to 8 m/km depending on the road function. If the IRI shows below 4.0, it means that the road is still in a routine maintenance phase, while if the IRI is between 4.1 and 8.0 which is categorised as being in moderate condition, it means that the road needs periodic maintenance (overlay). While if IRI ranges from 8 to 12, it means that the road needs to be considered for improvement. Meanwhile, if IRI > 12 means the road cannot be maintained, so the step that must be done is reconstruction [9].
The Directorate General of Highways uses the parameters of the International Roughness Index (IRI) in determining the condition of road construction, which is divided into four groups. As shown in  Figure 1. Relationship between condition, age and type of road handling.

Roadroid Applications
Roadroid is a network data collection and road conditions application that is easily accessible at any time and use relatively inexpensive equipment with accurate results, especially for the Roadroid emphasizing more on IRI surveys road surface flatness size/conditions expressed by the number of road surface vertical changes for each unit of road length (mm/km). The IRI is a hardness parameter that is calculated from the cumulative number of rising and surface fall of the longitudinal profile divided by the surface distance/length. Generally, the main problem in the area to conduct road flatness surveys is the limited allocation of survey funds to public works services. The use of the Roadraid application can be a practical choice that has advantages concerning its relatively low price, produces accurate data, and is easy to operate. Roadroid is a program developed in Sweden by Lars Forsflof with the prototype that appeared in 2002 and was developed to date.

Methodology
The purpose of this methodology is to explain the procedures for obtaining necessary data both primary data and other data needed, which will then be used in the processing and analysing data to obtain results that are in line with the expected objectives, namely assessing the pavement conditions for measuring inequality road pavement surface.

Location of Study
The location of the study was conducted on the national road segment in the Province of South Sulawesi, namely on the Perintis Kemerdekaan Road section of Makassar City. This road is a Primary Arterial Road which extends from south to north with a length of ± 15 km as seen in figure 2.

Roadroid Application Settings
The main thing that needs to be prepared for the Roadroid operation is the media device in the form of an Android Mobile that has a higher specification than the Android Operating system at least version 4.4. Each cellphone only gets one username and password to access the website, based on the mobile IMEI number. Besides, a car holder is needed to put the cellphone on the car dashboard. Cars that are used are still in the best performance (usually the age of the vehicle is a maximum of 5 years). Car performance dramatically affects the accuracy of IRI data obtained. The worse the car's performance, the higher the shock/vibration in the vehicle, causing the IRI data to be read larger than it should be. Other equipment needed is a list of road maps to be surveyed and highlighter to mark the roads that have been surveyed so that they are not surveyed more than once. The next stage is the Calibration devices. Calibration steps are by placing the cellphone on the dashboard in a stable position. Refer to Figure 3; (1) click the Roadroid application icon, (2) click ok, (3) click adjustment fitting, then the value X, Y, Z will appear in yellow. Wait until all the values of X, Y, Z are green after that click OK. The Roadroid application in cellphone needs adjusting the terrain conditions, and vehicle type of vehicle will be used in the survey. Some monitored parameters are; personal e-mail, cellphone type, vehicle type, CIRI sensitivity, length of each CIRI segment, automatic photo taking, the minimum speed of vehicle speed, sleep police button, and mobile orientation, etc. To retrieve data from a road section, the survey starts from the base section and taking data by clicking (Start/stop sampling), then giving the name of the road with an easy and integrated code with other roads. After that, let Roadroid work to end segment. When it reaches the end segment, the data collection is stopped by clicking (Start/Stop Sampling) again. Conditions and documentation of the road segment data have been recorded on devices, and surveyors can start surveys for other sections and so on. When there is an internet signal, this data should be uploaded to the internet immediately given the limited cell phone memory. This ICCEE 2019 IOP Conf. Series: Earth and Environmental Science 419 (2020) 012008 IOP Publishing doi:10.1088/1755-1315/419/1/012008 6 survey data will then be stored on the Internet and can be checked by going to the address roadroid.com. Moreover, log in with the user and password devices for data retrieval. Data obtained from this survey are; KML File and Shape File that can be generated in map applications such as Google Earth, Google Map etc. Data in the form of files (* .txt) generated for each segment of the road with a segment at least every 20m, 50m, etc. The data contains; (i) time and date of survey; (ii) segment name; (iii) GPS position (iv) KM/distance; (v) speed; (vi) changes in road vertical alignment; (vii) e0 IRI; and (viii) CIRI. Based on the experience and results of the analysis, it is recommended that e) IRI is used as a reference [10].

Results and discussions
The value of road unevenness reviewed is affected by the amount of Roadroid value obtained during the survey, where the greater Roadroid value generated, the higher of road unevenness value (IRI) which will result in more extended and more severely damaged conditions. The factor causes the Roadroid value to increase is the quantity and location of the widespread type of asphalt damage, wherein the absence of serious handling the level of road quality will decrease dramatically due to daily traffic which burdened the national road, which tends to be passed by heavy vehicles, considering Perintis Kemerdekaan road is one of the national highways in Makassar and is a connecting road between provincial capitals that functions in fulfilling the need for goods and services for each region.  Based on Figure 4 (a) and 4 (b) it is known that the IRI value of Perintis Kemerdekaan road toward Makassar-Maros has lowest IRI value is 1.52 in STA 1 + 300, and the highest IRI value is 11.56 in STA 4 + 400. The average IRI was obtained at 2.607. While on the same section but in the opposite direction, namely in the direction of Maros-Makassar, it is known that the highest IRI value occurs in STA 7 + 600 with a value of 7.39 and the lowest value occurs in STA 7 + 800 with an IRI value of 1.60. The  Meanwhile, Figure 5 shows that the average surface flatness conditions of the Perintis Kemerdekaan road are in good condition. It can be seen from the percentage of road alignment, namely the direction of Makassar-Maros road grade conditions in the good category reaches 90.65% or equivalent to 9.7 km while in the direction of Maros-Makassar the good category reaches 92.52% or equal to 9.9 km. For the road surface conditions in the medium category in the direction of Makassar-Maros, only 8.41% is higher than in the opposite direction, which is 7.48%. Complete results related to road conditions and road length can be seen in the following Table 2.

Assessment of Road Stability
In road stability aspect on Perintis Kemerdekaan road segment is included in the stable category with stability value, reaching 99.07% for Makassar-Maros direction, even for the direction of Maros-Makassar reaching 100%. It means that road works as expected in serving the movement of vehicles both from Makassar City and to Makassar City direction. The description of road stability on Perintis Kemerdekaan road section as shown in Figure 6 and Table 3.

Required Road Maintenance
The purpose of road maintenance is to maintain a steady road condition following the level of service and its ability when the road is completed and operated until planning age is determined. Starting from this steady-state, road maintenance needs to be conducted routinely and continuously, especially in road construction type that uses flexible pavement systems. Routine road maintenance is carried out continuously throughout the year and carried out as soon as possible when the damage has not yet spread. Maintenance and repairs are carried out at the mild and local damage stage. It is done in connection with relatively low repair costs, and how to repair them is relatively easy/light. Periodic road maintenance is carried out regularly by also rejuvenating pavement materials and other materials. Besides that, re-levelling of the road surface was carried out. Both routine maintenance and periodic maintenance are not intended to improve the necessary capabilities.
Before deciding on actions related to road maintenance, and assessment of road conditions is needed. One assessment of road conditions is to visually assess road pavement conditions obtained by conducting a field survey using the IRI method obtained by survey using a car and using the Roadroid application. In determining the road handling needs, it is closely related to the IRI value generated from a survey. From the results of surveys and references from the Directorate General of Highways, as illustrated in Table 1, the model of handling needs for the Perintis Kemerdekaan road as shown in table 4. Based on Table 4, it can be seen that using the IRI method and using the Roadroid software gives an average IRI value of 2.607 for the Makassar-Maros road section and for the Maros-Makassar road section the resulting IRI value is 2.408. The results illustrate the level of stability of the road that is included in the stable category. Thus on this road only requires routine maintenance which can be done periodically.

Conclusion
Based on the results of the data from this study, the following conclusions can be taken; that the Perintis Kemerdekaan road produces an IRI value of 2.607 and 2.408 for each direction. From this value, it is converted into the level of road stability, including the road in the stable category, so that in general this segment for handling needs only requires regular maintenance regularly to achieve maintaining the service level. We want to thank especially to. Abrorhanif Paisal and M.Syuaib who have helped the data collection process and Roadroid data input