## Mobility-Aware Federated Self-supervised Learning in Vehicular Network

Xueying Gu , 1 Qiong Wu 1,2* , Pingyi Fan , 3 Qiang Fan 4

1 School of Internet of Things Engineering, Jiangnan University, Wuxi, 214122, China. 2 State Key Laboratory of Integrated Services Networks, Xidian University, Xi'an , 710071, China. 3 Department of Electronic Engineering, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, 100084, China. 4 Qualcomm, San Jose CA, 95110, USA.

*Corresponding author(s). E-mail(s): qiongwu@jiangnan.edu.cn; Contributing authors: xueyinggu@stu.jiangnan.edu.cn; fpy@tsinghua.edu.cn; qf9898@gmail.com;

## Abstract

Federated Learning (FL) is an advanced distributed machine learning approach, that protects the privacy of each vehicle by allowing the model to be trained on multiple devices simultaneously without the need to upload all data to a road side unit (RSU). This enables FL to handle scenarios with sensitive or widely distributed data. However, in these fields, it is well known that the labeling costs can be a significant expense, and models relying on labels are not suitable for these rapidly evolving fields especially in vehicular networks, or mobile internet of things (MIoT), where new data emerges constantly. To handle this issue, the selfsupervised learning paves the way for training without labels. Additionally, for vehicles with high velocity, owing to blurred images, simple aggregation not only impacts the accuracy of the aggregated model but also reduces the convergence speed of FL. This paper proposes a FL algorithm based on image blur level to aggregation, called FLSimCo, which does not require labels and serves as a pre-training stage for self-supervised learning in the vehicular environment. Simulation results demonstrate that the proposed algorithm exhibits fast and stable convergence.

Keywords: Federated learning, Self-supervised learning, Vehicle, Mobility

## 1 Introduction

The development of Internet of Things (IoT) makes many practical applications available to people, such as automatic navigation, weather forecast and self-driving systems, which not only reduces the probability of accidents, but also improves the happiness of life [1-3]. Training a good model can make the practical application more robust, which requires a lot of data. Moving vehicles can constantly collect new data by their devices, most of which can be captured in the form of images via camera mounted on the roof of the vehicle. Compared with the traditional method of collecting image data with onboard radar, this way greatly reduces the cost and technical requirements, making the shift of image collection paradigm to vision-based measurement (VBM) has broad promising [4-6].

After the acquisition is completed, the vehicle process the image data to realize image recognition and classification. It provides the necessary information for selfdriving and driver assistance systems [7-9], and helps drivers perceive and understand their surroundings. Especially in nighttime or bad weather conditions, image recognition and classification models can be critical valuable in identifying road signs, other vehicles, and obstacles, providing specific object classification, improving driver visibility, and improving driving safety. Therefore, image recognition and classification in self-driving plays a key role in reducing the risk of traffic accidents and improving road safety.

As mentioned earlier, training a robust model requires a lot of data. However, the data stored locally on each vehicle may be private and sensitive, and drivers are reluctant to share it with each other in order to protect their privacy. If the vehicles do not share information, they can only train the local model using their own data [10, 11]. Due to the single driving environment, the image categories will be skewed, so that the trained model does not have good transference. Compared to single vehicle training, FL can solve this issue by aggregating different vehicles' local trained models with local data. This approach can take a large data set into account in the training process without the need for vehicles to share local data, resulting in superior performance and enhanced generalization [12-14, 21].

However, this approach still faces the following two issues: low image quality and missing or incorrect labels. The first one is due to the motion blur caused by the movement of vehicles, and the other is because of the high cost and incorrectness of labeling.

In a vehicular network, a high velocity usually leads to insufficient exposure time for the camera sensor, causing motion blur. In this paper, we consider the motion blur caused by different vehicle velocities in the training process, and adjust corresponding weights of the model parameters for FL model aggregation.

Self-supervised learning [16-19] can abandon labels for pre-training, and thus mitigates the impact of incorrect labels on the model and removes the cost of the labeling process, which makes it suitable for IoT, where new data is constantly appearing.

To the best of our knowledge, few research works have taken into account both the privacy protection of vehicles and the blurring of images in real scenes, as well as the cost of labels during the training of models.

In this paper, we build a framework, called FLSimco 1 , to train the vehicle side local model by self-supervised learning at dual temperatures. Only upload the local models of the vehicles, after the completion of local training, and then the RSU through the FL aggregated the received local models, both to protect the privacy of the vehicle, and training a generation model.

The remaining of this paper is organized as follows. In Section 2, we will review related works. The Section 3 details the system model we designed. Section 4 will showcase and discuss the experimental results obtained in the simulation environment. Finally, we will summarize the research findings and present conclusions in the Section 5.

## 2 Review of related works

In recent years, the emergence of deep neural networks, particularly convolutional neural network (CNN), has facilitated significant advances in computer vision benchmarks. Self-supervised learning is a special form of unsupervised learning, and its main feature is that it uses information inherent in the data itself to help machine learning models get a better representation, thereby improving performance across a variety of tasks[20-22].

Until now, research on unsupervised learning has focused on mining shared features between pre-trained tasks and downstream tasks [23]. Wu et al. [24] introduced noise contrast estimation (NCE) loss as an objective function to distinguish between different instances. Each image was treated as a positive sample and the others as negative samples, effectively treating each image as a category, constituting an instance-level classification task. SimCLR [25, 26] has maximized the sample characteristics of similarity to study representations. It abandoned the memory library [27] and generated more negative samples by increasing the batch size. The core idea of these methods is to encourage models to place similar data representations(positive sample pairs) together in the embedding space, while separating dissimilar representations(negative sample pairs) to learn feature extraction. Since these methods do not consider the fact that the distance between samples of the same category should be smaller than that of different categories' samples, but treat all pairs equally. In this way, a large number of negative samples are required. It will pose significant challenges to storage and computing power in the vehicle environment, increasing the requirements of hardware configuration levels.

MoCo [28, 29], as a self-supervised learning method for CNN [30], has made an important contribution by introducing queue and momentum updating techniques to create a large and consistent dictionary conducive to contrast learning. It replaced the original memory library with a queue as an additional data structure to store these negative samples. Momentum encoder k is used to replace the original loss constraint term. The key advantage of the MoCo family is the use of the queue as negative samples and the momentum update of the queue, which greatly reduces the need for vehicle storage capacity and computing power in local training. [31] used MoCo for the pre-training model weights on ImageNet and achieves excellent results in downstream

1 The source code has been released at: https://github.com/qiongwu86/FLSimCo

Fig. 1 : Inter-anchor sample and intra-anchor samples

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00256v1-figure-03f81544-edb7-45c6-9378-796ef01c69f9.jpg)

fine-tuning tasks. [32] employed MoCo for unsupervised learning with large amounts of unlabeled data on remote servers. The local vehicles downloaded the trained model and used it as the initialization model.

These methods are characterized by the presence of a large default datasets that can be used for unsupervised learning, and the recognition accuracy increases with the expansion of the dictionary size[33-35]. Simultaneously, as the dictionary size increases, the recognition accuracy gradually improves. However, it is important to note that, larger dictionaries also requires larger memory size, increasing the storage cost and the computing capability of hardness. SimCo [16] attributed the need for a large dictionary in MoCo to hardness awareness between anchors, and the author believed that the consistency between positive and negative keys was more crucial than that between negative keys. Therefore, it utilized dual temperatures to differentiate inter-anchor samples and intra-anchor samples, as shown in Fig. 1, which eliminates the need for a dictionary of negative samples.

Due to the imbalance of data categories, the data distribution faces the challenge of non-independent identically distributed (non-IID) characteristics [36-39], requiring more training rounds and interaction frequency to improve accuracy. However, increased interaction means require more computing and network resources [40, 41]. To protect the privacy of local data and its distribution, FedCo proposed to employ FL to aggregate local models. Each vehicle passed its own k value to the RSU for forming a bigger new queue. However, a new queue consisting of the k values of different vehicles violates MoCo's inherent requirement for consistency in negative key pairs. When the vehicle uploaded the trained model and the corresponding k value, it can, to some extent, reconstruct the original input. As such, it does not ensure privacy protection and defeats the original purpose of using FL. [42] used federated self-supervised learning for a single-event classification, adding a binary classifier for each new event by adhering to a one-to-many paradigm. However, these FL algorithms do not take into account the effect of image blurring. As a result, they cannot effectively simulate real-world scenarios.

In this paper, we will train the local model on the vehicles side by self-supervised learning with dual temperatures. After the training is completed, only the local models of vehicles are uploaded, and then the received local model is aggregated in RSU by

Fig. 2 : System scenario

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00256v1-figure-55e91af3-ae62-4f34-8e0d-c08f818a1c8a.jpg)

FL, which not only protects the privacy of vehicles, but also produces a model with better transform through distributed training. At the same time, considering that the images collected in the actual vehicle environment may be blur, the blur level is used as the weight during aggregation, which makes the aggregated model more reliable and stable.

## 3 System model

## 3.1 System design

As shown in Fig. 2, let's consider a scenario where an RSU is deployed at the intersection and there are several vehicles driving in coverage of the RSU. Each vehicle driving straightly when cross the intersection. We firstly build a truncated Gaussian distribution model as the mobility model for each vehicle. At the same time, each vehicle is equipped with a large mount of images, called local data, captured by camera mounted in the roof before entering the coverage of the RSU. It is worth noting that vehicles coming from different directions may capture different categories of images from each other.

In each FL round, each vehicle downloads the global model from the RSU as it enters the coverage of the RSU. Then each vehicle adopts the global model for the pre-training of the self-supervised learning to classify each category of captured local image data. During this process, the vehicle randomly selects a specified number of images from its local data [39, 44, 45]. When the local training is complete, the vehicle uploads its local model to the RSU. Then, the RSU aggregates the local models from different vehicles to obtain a new global model. After that, FL moves on to the next round until the maximum round R max is reached. Next, we will describe the models for each round in the system.

## 3.2 Mobility model

Similar with [46], we adopt the following mobility model to reflect the real vehicle mobility. We consider the velocities of different vehicles are independent identically distributed (IID). The velocity of each vehicle follows the truncated Gaussian distribution. Let N r be the number of vehicles traveling within the coverage area of RSU in the round r , v n r be the velocity of vehicle n r ∈ [1 , N r ], and v min and v max be the minimum and maximum velocity of vehicles, respectively, namely velocity v n r ∈ [ v min , v max ], and the probability density function of v n r is expressed as [47]

$$f ( v _ { n _ { r } } ) = \begin{cases} \frac { e ^ { - \frac { 1 } { 2 \sigma ^ { 2 } } ( v _ { n _ { r } } - \mu ) ^ { 2 } } } { \sqrt { 2 \pi \sigma ^ { 2 } } [ e r f ( \frac { v _ { \max } - \mu } { \sigma \sqrt { 2 } } ) - e r f ( \frac { v _ { \min } - \mu } { \sigma \sqrt { 2 } } ) ] } } \\ v _ { \min } \leq v _ { n _ { r } } \leq v _ { \max }, \\ 0 & o t h e r w i s e. \end{cases} \\ \varepsilon \quad \dots \ \sim \ \cdot \quad \varepsilon \quad \cdot \quad \varepsilon \quad \cdot \quad \varepsilon \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdots \quad \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cd. \quad \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad\cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad\cd. \quad \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot$$

where erf ( µ, σ 2 ) is the Gaussian error function of velocity v n r with mean µ and variance σ 2 .

## 3.3 FLSimCo algorithm

Self-supervised learning requires a large amount of unlabeled data. In the vehicle scene, it is easy for the vehicles to collect enough image data during driving. However, due to the limitations of the environment, storage and camera perspective, the categories of image data of each vehicle are limited, which may not meet the IID [48, 49] requirement. To be specific, when the vehicle uses the image data stored by itself for local training, a model shewed to its own image categories is obtained, namely a model with poor generation. After the training is completed, the vehicle uploads the parameters of local model, and the RSU aggregates the received local model parameters from different vehicles, making the aggregated model more general and effectively solving the data non-IID problem.

We establish a novel algorithm called FLSimCo, which means simplified MoCo with no momentum in FL. Typically, the specific FLSimCo process is as follows:

Step 1, initialization : The RSU initializes a global model θ 0 for an encoder f 0 with random initial parameters.

Step 2, local training : For round r , N r vehicles take part in the FL, where each vehicle n r downloads the model θ r of encoder f r from the RSU.

The blur level L n r of local image data in the vehicle n r can be represented as [50, 51]:

$$L _ { n _ { r } } = \frac { H s } { Q } v _ { n _ { r } }.$$

In this context, v n r is the velocity of vehicle n r . Hs Q is the parameter of the camera, in which H represents the exposure time interval, s is the focal length, and Q represents pixel units.

For each image x i ∈ { x , 1 x , 2 . . . x i , . . . x M } , after applying two different data augmentation methods π 1 ( ) and · π 2 ( ), and passing through the encoder · f r , we obtain

anchor sample q i n r , positive sample k i n r and negative samples k j n r :

$$q _ { n _ { r } } ^ { i } = f ^ { r } \left [ \pi _ { 1 } \left ( x _ { i } \right ) \right ] = R e s N e t \left [ \pi _ { 1 } \left ( x _ { i } \right ) \right ] \left | _ { r }, \ i \in \left [ 1, M \right ],$$

$$k _ { n _ { r } } ^ { i } = f ^ { r } \left [ \pi _ { 2 } \left ( x _ { i } \right ) \right ] = R e s N e t \left [ \pi _ { 2 } \left ( x _ { i } \right ) \right ] \left | _ { r }, \ i \in \left [ 1, M \right ], \quad \ \ ( 4 )$$

̸

$$\begin{array} { c } k _ { n _ { r } } ^ { j } = f ^ { r } \left ( x _ { j } \right ) = R e s N e t \left ( x _ { j } \right ) | _ { r }, & j \in [ 1, M ] \ a n d \ j \neq i. \end{array}$$

According to [16], the loss can be transformed into

$$\mathcal { L } _ { q _ { n _ { r } } ^ { D T } } ^ { D T } = & - s g \left [ \frac { \left ( W _ { \beta } \right ) _ { n _ { r } } ^ { i } } { \left ( W _ { \alpha } \right ) _ { n _ { r } } ^ { i } } \right ] \times \\ & \log \frac { \exp \left ( \frac { q _ { n _ { r } } ^ { i } \cdot k _ { n _ { r } } ^ { i } } { \tau _ { \alpha } } \right ) } { \exp \left ( \frac { q _ { n _ { r } } ^ { i } \cdot k _ { n _ { r } } ^ { i } } { \tau _ { \alpha } } \right ) + \sum _ { j = 1 } ^ { K } \exp \left ( \frac { q _ { n _ { r } } ^ { i } \cdot k _ { n _ { r } } ^ { j } } { \tau _ { \alpha } } \right ) },$$

and

$$( 6 )$$

$$\left ( W _ { \beta } \right ) _ { n _ { r } } ^ { i } = 1 - \frac { \exp \left ( \frac { q _ { n _ { r } } ^ { i } \cdot k _ { n _ { r } } ^ { i } } { \tau _ { \beta } } \right ) } { \exp \left ( \frac { q _ { n _ { r } } ^ { i } \cdot k _ { n _ { r } } ^ { i } } { \tau _ { \beta } } \right ) + \sum _ { j = 1 } ^ { K } \exp \left ( \frac { q _ { n _ { r } } ^ { i } \cdot k _ { n _ { r } } ^ { j } } { \tau _ { \beta } } \right ) },$$

$$\left ( W _ { \alpha } \right ) _ { n _ { r } } ^ { i } = 1 - \frac { \theta ^ { \mu } \, ^ { \prime } \, ^ { \prime } \, ^ { \prime } \, \cos \left ( \frac { q _ { n _ { r } } ^ { i } \cdot k _ { n _ { r } } ^ { i } } { \tau _ { \alpha } } \right ) } \, \left ( \frac { q _ { n _ { r } } ^ { i } \cdot k _ { n _ { r } } ^ { i } } { \exp \left ( \frac { q _ { n _ { r } } ^ { i } \cdot k _ { n _ { r } } ^ { i } } { \tau _ { \alpha } } \right ) + \sum _ { j = 1 } ^ { K } \exp \left ( \frac { q _ { n _ { r } } ^ { i } \cdot k _ { n _ { r } } ^ { i } } { \tau _ { \alpha } } \right ) },$$

where L DT q i nr is the dual-temperature (DT) loss of the i -th image of the vehicle's anchor sample in the round r . K represents the queue length and sg [ ] · indicates the stop gradient. The denominator in the above equations consists of one positive sample and K negative samples. It is noting that ' ' in Eq. (6) - Eq. (8) means dot product. · τ α and τ β are different temperature hyper-parameters [52], and controls the shape of the samples distribution. Based on the different requirements for dictionary size by inter-anchor and intra-anchor, and the temperature's ability to control the feature distribution, different temperatures will be used to control the distance between different samples, thus eliminating MoCo's dependency on a large dictionary. This will remove the inter-anchor's reliance on a large dictionary.

The objective function can be defined as minimizing the loss function [53-55]

$$\hat { \theta } _ { n _ { r } } = \underset { \theta _ { n _ { r } } } { \arg \min } \frac { 1 } { M } \sum _ { i = 1 } ^ { M } \mathcal { L } _ { q _ { n _ { r } } ^ { i } } ^ { D T } \left ( \theta _ { n _ { r } }, \ q _ { n _ { r } } ^ { i }, \ k _ { n _ { r } } ^ { i }, \ k _ { n _ { r } } ^ { j } \right ),$$

where θ n r represents the parameter of local model of vehicle n r in round r . Each vehicle performs the local training to approach ˆ θ n r according to the following algorithm:

$$\theta _ { n _ { r } } \leftarrow \, \theta _ { n _ { r } } - \eta ^ { r } \nabla \mathcal { L } ^ { D T } \left ( \theta _ { n _ { r } }, \, q ^ { i } _ { n _ { r } }, \, k ^ { i } _ { n _ { r } }, \, k ^ { j } _ { n _ { r } } \right ), & & ( 1 0 )$$

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00256v1-figure-27c9c8c5-27e4-4d61-8b06-c51302b8c028.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00256v1-figure-fb9e59f3-a489-4ac2-8add-4d5704e501a4.jpg)

Cifar10\_Labels

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00256v1-figure-b49afd55-700a-4c89-8b24-94e3866e0a83.jpg)

(c) Non-IID with

α

= 1

Fig. 3 : Data Category Distribution Plot

where η r represents the learning rate for the round r .

Step 3, Upload model : N r vehicles upload the parameter { θ , 1 θ 2 . . . θ n r . . . θ N r } of local models after their local training finished, along with the velocity { v , v 1 2 , . . . v n r . . . v N r } of each vehicle.

Step 4, Aggregation and Update : After receiving the trained models from N r vehicles, the RSU employs a weighted federated algorithm to aggregate the parameters of N r models based on the blur level L n r . The expression for the aggregated model is

$$\theta ^ { r + 1 } = \sum _ { n _ { r } = 1 } ^ { N _ { r } } \left [ \frac { \left ( \sum _ { n _ { r } = 1 } ^ { N _ { r } } L _ { n _ { r } } - L _ { n _ { r } } \right ) \theta _ { n _ { r } } } { \sum _ { n _ { r } = 1 } ^ { N _ { r } } \left ( \sum _ { n _ { r } = 1 } ^ { N _ { r } } L _ { n _ { r } } - L _ { n _ { r } } \right ) } \right ].$$

Repeat the above step 2 to step 4 until reaching max round R max .

## 4 Results

In this section, we will introduce the setup of experiments, show the results, and give a brief explanation.

## 4.1 Experimental setup

We adopt an improved ResNet-18 with a fixed dimension of 128-D as the backbone model and employ SGD (Stochastic Gradient Descent) as the optimizer. In addition, inspired by the concept of cosine annealing, we gradually reduce the learning rate at different stages of training to improve the training efficiency of the model [56-58].

Datasets : This paper uses the datasets CIFAR-10 [59, 60] with 50,000 unlabeled images as the training datasets, which is distributed in 10 different categories, i.e., each category consists of 5,000 images.

- (1) IID : Samples that follow IID are independent of each other and share the same distribution. The appearance of each sample is not affected by the other samples. We uniformly assigned 10 categories from 50,000 images in CIFAR-10 to 95 vehicles, ensuring that each vehicle have at least 520 images available for training. As can be seen from the Fig. 3a, the categories on each vehicle are evenly distributed.
- (2) Non-IID : IID provides theoretical convenience, but in a practical scenario, there is very little data that meets the IID requirement. Therefore, it is importance to use data under non-IID conditions to transfer the model to the real scenario. As shown

Fig. 4 : FLSimCo VS other methods

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00256v1-figure-0c9130b1-4ab2-4e17-a699-5836316c2920.jpg)

in Fig. 3b and 3c, the Dirichlet distribution parameter α is 0.1 and 1, respectively [5]. Clearly, the smaller the α , the larger the gap between data categories. We set the Dirichlet distribution parameter α to 0.1 to simulate the Non-IID data in the vehicular scene, in order to simulate the uneven distribution of the image categories collected by each vehicle due to the limited viewing perspective and environmental constraints. In order to ensure that there is enough data for local training, for CIFAR-10 we ensure that there are at least 520 images per vehicle.

## 4.2 Results

According to FedCo [61], in the round r of the training process, we set each vehicle uploads all stored k -values (with a batch size set to 512 in the experiment) to the RSU (global queue set to 4096) to update the global queue.

As shown in Fig. 4, we consider the FedCo simulation result as baseline1. In addition, our proposed method FLSimCo is represented by a red line in the same diagram. It can be observed that FLSimCo outperforms FedCo given the same number of rounds. From FedCo's perspective, the update queues with k values from different vehicles has compromised the Negative-Negative consistency requirement in MoCo, resulting in a less accurate approach. Meanwhile, FedCo enables the vehicle's own k value to be uploaded to the RSU, which also goes against FL's purpose of protecting user privacy. In addition, we also conducted experiments on Non-IID data sets, and the results show that the training performance of Non-IID data sets is slightly lower than that of IID data sets, but still better than FedCo algorithm.

For Fig. 5, we assume that some images will be blurred when the vehicle velocity exceeds 100 km/h and conduct three comparative experiments.To better demonstrate its performance, we also introduce two baseline algorithm. FedAvg [62] is employed as baseline1, which averaging the model parameters accordingly. Baseline2 indicates that RSU will discard the model trained by the vehicle velocity exceeding 100 km/h , that is, discard the local model trained with the blurred images, and then use the FedAvg aggregation model parameters.

Fig. 5 : Aggregated with different weights

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00256v1-figure-c65d3a64-d0b0-47d4-9091-204ae1cdc41b.jpg)

From the results of baseline1 and baseline2, it is obvious that it is not beneficial to drop the model trained with motion blurred images(baseline2). Because it drops some models during the aggregation process, baseline2 has the lowest number of models used in the global training. This suggests that the number of models involved in aggregation is more critical than the quality of the models. Finally, our proposed aggregation method assigns smaller weights to models trained by faster vehicles. It can be seen that the proposed algorithm can effectively reduce the fluctuation of the loss function, while facilitating the loss function and converge to a smaller value faster. This demonstrated that proposed approach can reduce the effect of image blur and also increase the global model training speed.

## 5 Conclusion

In this paper, we propose a FLSimCo algorithm. First, on the basis of self-supervised learning, the advantages of FL are used. The main goal is to solve the problem of data category skew and to train a model with well generation without sharing private data. By integrating self-supervised learning and FL, the algorithm can improve the model's adaptability to data heterogeneity to achieve a more robust pre-training model. More importantly, considering the data blur caused by vehicle movement, the blur level is used as the weight of federated aggregation, which makes the algorithm more suitable for real scenarios.

In summary, the FLSimCo algorithm ensures the protection of vehicle privacy, and requires no labeled image data, while addressing the aggregation challenges in vehicular environments characterized by significant data shew and image blurring due to vehicle movement.

## Declarations

Competing interests : The author declare that there is no competing interest.

Open Access : This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article?s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licen ses/by/4.0/.

## References

- [1] Wu Q, Wang S, Ge H, Fan P, Fan Q, Letaief K (2023) Delay-sensitive Task Offloading in Vehicular Fog Computing-Assisted Platoons. IEEE Transactions on Network and Service Management.https://doi.org/10.1109/TNSM.2023.3322881.
- [2] Wu Q, Shi S, Wan Z, Fan Q, Fan P, Zhang C (2023) Towards V2I Age-aware Fairness Access: A DQN Based Intelligent Vehicular Node Training and Test Method. Chinese Journal of Electronics. pp. 1230-1244.
- [3] Long D, Wu Q, Fan Q, Fan P, Li Z, Fan J (2023) A Power Allocation Scheme for MIMO-NOMA and D2D Vehicular Edge Computing Based on Decentralized DRL. Sensors.
- [4] Mi C, Huang Y, Fu C, Zhang Z, Postolache O (2021) Vision-Based Measurement: Actualities and Developing Trends in Automated Container Terminals. IEEE Instrumentation and Measurement Magazine. pp. 65-76, https://doi.org/10.1109/MIM.2021.9448257.
- [5] Zhang Z, Ma S, Yang Z, Xiong Z, Kang J, Wu Y, Zhang K, Niyato D (2023) Robust Semi-supervised Federated Learning for Images Automatic Recognition in Internet of Drones. IEEE internet of things journal. pp. 5733-5745.
- [6] Ye D, Yu R, Pan M, Han Z (2020) Federated learning in vehicular edge computing: A selective model aggregation approach. IEEE Access. pp. 23920-23935, https://doi.org/10.1109/ACCESS.2020.2968399.
- [7] Anagnostopoulos C, Gkillas A, Piperigkos N, Lalos S (2023) Federated Deep Feature Extraction-based SLAM for Autonomous Vehicles. 2023 24th International Conference on Digital Signal Processing (DSP). https://doi.org/10.1109/DSP58604.2023.10167897.
- [8] Wu Q, Zhao Y, Fan Q, Fan P, Wang J, and Zhang C (2023) Mobility-Aware Cooperative Caching in Vehicular Edge Computing Based on Asynchronous Federated and Deep Reinforcement Learning, IEEE Journal of Selected Topics in

|      | Signal Processing, vol. 17, no. 1, pp. 66-81.                                                                                                                                                                                                                                                                     |
|------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [9]  | Wu Q, Shi S, Wan Z, Fan Q, Fan P, and Zhang C (2023) Towards V2I Age- aware Fairness Access: A DQN Based Intelligent Vehicular Node Training and Test Method, Chinese Journal of Electronics, vol. 32, no. 6, pp. 1230-1244.                                                                                      |
| [10] | Wu Q, Wang X, Fan Q, Fan P, Zhang C, Li Z (2023) High Stable and Accu- rate Vehicle Selection Scheme based on Federated Edge Learning in Vehicular Networks. China Communications. pp. 1-17.                                                                                                                      |
| [11] | Wu Q, Xia S, Fan Q, Li Z (2019) Performance Analysis of IEEE 802.11p for Continuous Backoff Freezing in IoV. Electronics.                                                                                                                                                                                         |
| [12] | Yan R, Qu L, Wei Q, Huang S, Shen L, Rubin D, Xing L, Zhou Y (2023) Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging. IEEE Transactions on Medical Imaging. https://doi.org/10.1109/TMI.2022.3233574.                                                        |
| [13] | Wang K, Yu F, Wang L, Li J, Zhao N, Guan Q, Li B, and Wu Q (2019) Interfer- ence Alignment with Adaptive Power Allocation in Full-Duplex-Enabled Small Cell Networks, IEEE Transactions on Vehicular Technology, vol. 68, No. 3.                                                                                  |
| [14] | Wu Q, Xia S, Fan P, Fan Q, and Li Z (2018) Velocity-Adaptive V2I Fair Access Scheme Based on IEEE 802.11 DCF for Platooning Vehicles, Sensors, vol. 18, no. 12, Art no. 4198.                                                                                                                                     |
| [15] | Fan J, Yin S, Wu Q, and Gao F (2010) Study on Refined Deployment of Wireless Mesh Sensor Network, in Proc. of IEEE International Conference on Wireless Communications, Networking and Mobile Computing (WICOM?10), Chengdu, China, pp. 370-375.                                                                  |
| [16] | Zhang C, Zhang K, Pham T, Niu A, Qiao Z, Yoo C, Kweon I (2022) Dual temperature helps contrastive learning without many negative sam- ples: towards understanding and simplifying MoCo. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR52688.2022.01404. |
| [17] | WuQ, Wang S, Ge H, Fan P, Fan Q, and Letaief K. B. (2024) Delay-sensitive Task Offloading in Vehicular Fog Computing-Assisted Platoons, IEEE Transactions on Network and Service Management, vol. 21, no. 2, pp. 2012-2026.                                                                                       |
| [18] | Wu Q, Liu H, Zhang C, Fan Q, Li Z, and Wang K (2019) Trajectory Protection Schemes Based on a Gravity Mobility Model in IoT, Electronics, vol. 8, no. 148.                                                                                                                                                        |
| [19] | Wu Q, Nie S, Fan P, Liu H, Fan Q, and Li Z (2018) A Swarming Approach to Optimize the One-Hop Delay in Smart Driving Inter-Platoon Communications, Sensors, vol. 18, no. 10, Art. no. 3307.                                                                                                                       |

| [20]   | Wu Q, and Zheng J (2015) Performance Modeling and Analysis of the ADHOC MAC Protocol for VANETs, in Proc. of IEEE International Conference on Communication (ICC?15), London, UK, pp. 3646-3652.                                                |
|--------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [21]   | Fan J, Wu Q, and Hao J (2010) Optimal Deployment of Wireless Mesh Sen- sor Networks based on Delaunay Triangulations, in Proc. of IEEE International Conference on Information, Networking and Automation (ICINA?10), Kunming, China, pp. 1-5.  |
| [22]   | Wu Q, Wang X, Fan Q, Fan P, Zhang C, and Li Z (2023) High Stable and Accu- rate Vehicle Selection Scheme based on Federated Edge Learning in Vehicular Networks, China Communications, vol. 20, no. 3, pp. 1-17.                                |
| [23]   | Doersch C, Gupta A, Efros A (2016) Unsupervised visual representation learning by context prediction. 2015 IEEE International Conference on Computer Vision (ICCV). https://doi.org/10.1109/ICCV.2015.167.                                      |
| [24]   | Wu Z, Xiong Y, Yu S, Lin D. Unsupervised Feature Learning via Non-Parametric Instance Discrimination. https://arxiv.org/pdf/1805.01978.pdf.                                                                                                     |
| [25]   | Chen T, Kornblith S, Norouzi M, Hinton G (2020) A Simple Framework for Con- trastive Learning of Visual Representations. https://arxiv.org/abs/2002.05709.                                                                                      |
| [26]   | Kong S (2023) Self-supervised Image Classification Using Convo- lutional Neural Network. 2023 IEEE 3rd International Conference on Power, Electronics and Computer Applications (ICPECA). https://doi.org/10.1109/ICPECA56706.2023.10075949.    |
| [27]   | Hjelm R, Fedorov A, Laavoie-Marchildon S, Grewal K, Bachman P, Trischler A, Bengio Y. Learning deep representations by mutual information estimation and maximization. https://arxiv.org/pdf/1808.06670.pdf.                                    |
| [28]   | He K, Fan H, Wu Y, Xie S, Girshick R (2020) Momentum con- trast for unsupervised visual representation learning. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR42600.2020.00975.      |
| [29]   | Chen X, Fan H, Girshick R, He K (2020) Improved Baselines with Momentum Contrastive Learning. https://arxiv.org/pdf/2003.04297.pdf.                                                                                                             |
| [30]   | Zhao J, Xiong X, Zhang Q, Wang D (2023) Extended Multi-Component Gated Recurrent Graph Convolutional Network for Traffic Flow Pre- diction. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/TITS.2023.3322745. |

| [31]   | Cai T, Gan H, Peng B, Huang Q, Zou Z (2022) Real-Time Classification of Disaster Images from Social Media with a Self-Supervised Learning Frame- work. IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium. https://doi.org/10.1109/IGARSS46834.2022.9883129.                                                                                                                                  |
|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [32]   | Zhao J, Li R, Wang H, Xu Z (2021) HotFed: Hot Start through Self-Supervised Learning in Federated Learning. 2021 IEEE 23rd Int Conf on High Performance Computing and Communications; 7th Int Conf on Data Science and Systems; 19th Int Conf on Smart City; 7th Int Conf on Dependability in Sensor, Cloud and Big Data Systems and Application. https://doi.org/10.1109/HPCC-DSS- SMARTCITY-DEPENDSYS53884.2021.00046. |
| [33]   | Wu Q, and Zheng J (2016) Performance Modeling and Analysis of the ADHOC MAC Protocol for Vehicular Networks, Wireless Networks, vol. 22, no. 3, pp. 799-812.                                                                                                                                                                                                                                                             |
| [34]   | Wu Q, and Zheng J (2014) Performance Modeling of the IEEE 802.11p EDCA Mechanism for VANET, in Proc. of IEEE Global Communications Conference (Globecom?14), Austin, USA, pp.57-63.                                                                                                                                                                                                                                      |
| [35]   | Long D, Wu Q, Fan Q, Fan P, Li Z, and Fan J (2023) A Power Allocation Scheme for MIMO-NOMA and D2D Vehicular Edge Computing Based on Decentralized DRL, Sensors, vol. 23, no. 7, Art. no. 3449.                                                                                                                                                                                                                          |
| [36]   | Zhao J, Li Q, Ma X, Yu FR. Computation Offloading for Edge Intelligence in Two-Tier Heterogeneous Networks. IEEE Transactions on Network Science and Engineering. https://doi.org/10.1109/TNSE.2023.3332949.                                                                                                                                                                                                             |
| [37]   | WuQ, and Zheng J (2014) Performance Modeling of IEEE 802.11 DCFBased Fair Channel Access for Vehicular-to-Roadside Communication in a Non-Saturated State, in Proc. of IEEE International Conference on Communication (ICC?14), Syndey, Australia, pp. 2575-2580.                                                                                                                                                        |
| [38]   | Wu Q, Xia S, Fan Q, and Li Z (2019) Performance Analysis of IEEE 802.11p for Continuous Backoff Freezing in IoV, Electronics, vol. 8, no. 12, Art. no. 1404.                                                                                                                                                                                                                                                             |
| [39]   | Wu Q, Wang W, Fan P, Fan Q, Wang J, and Letaief K. B. (2024) URLLC- Awared Resource Allocation for Heterogeneous Vehicular Edge Computing, IEEE Transactions on Vehicular Technology, doi: 10.1109/TVT.2024.3370196.                                                                                                                                                                                                     |
| [40]   | Zhao J, Quan H, Xia M, Wang D (2023) Adaptive Resource Alloca- tion for Mobile Edge Computing in Internet of Vehicles: A Deep Rein- forcement Learning Approach.IEEE Transactions on Vehicular Technology. https://doi.org/10.1109/TVT.2023.3335663.                                                                                                                                                                     |

- [41] Wu Q, Wang W, Fan P, Fan Q, Wang J, Letaief K (2024) URLLC-Awared Resource Allocation for Heterogeneous Vehicular Edge Computing. IEEE Trans.

Veh. Technol. https://doi.org/10.1109/TVT.2024.3370196.

| [42]   | Feng M, Kao C, Tang Q, Sun M, Rozgic V, Matsoukas S, Wang C (2022) Fed- erated self-supervised learning for acoustic event classification. ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). https://doi.org/10.1109/ICASSP43922.2022.9747472.   |
|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [43]   | Wu Q, Wang W, Fan P, Fan Q, Zhu H, and Letaief K. B. (2024) Cooperative Edge Caching Based on Elastic Federated and Multi-Agent Deep Reinforcement Learn- ing in Next-Generation Networks, IEEE Transactions on Network and Service Management, doi: 10.1109/TNSM.2024.3403842.                      |
| [44]   | Song S, Zhang Z, Wu Q, Fan P, and Fan Q (2024) Joint Optimization of Age of Information and Energy Consumption in NR-V2X System Based on Deep Reinforcement Learning, Sensors, vol. 24, no. 13, Art. no. 3448.                                                                                       |
| [45]   | Wan S, Lu J, Fan P, Shao Y, Peng C, and Letaief K. B. (2021) Convergence Analysis and System Design for Federated Learning Over Wireless Networks, IEEE Journal on Selected Areas in Communications, vol. 39, issue. 12. pp. 3622- 3639, doi: 10.1109/JSAC.2021.3118351.                             |
| [46]   | Wu Q, Zhao Y, Fan Q, Fan P, Wang J, Zhang C (2023) Mobility-Aware Cooper- ative Caching in Vehicular Edge Computing Based on Asynchronous Federated and Deep Reinforcement Learning. IEEE journal of selected topics in signal processing. pp. 66-81.                                                |
| [47]   | Yu Z, Hu J, Min G, Zhao Z, Miao W, Hossain MS (2021) Mobility-Aware Proac- tive Edge Caching for Connected Values using Federated Learning. IEEE Trans. Intell. Transp. Syst. pp. 5341-5351. https://doi.org/10.1109/TITS.2020.3017474.                                                              |
| [48]   | Hsieh K, Phanishayee A, Mutlu O, Gibbons P (2020) The Non-IID Data Quagmire of Decentralized Machine Learning. https://arxiv.org/abs/1910.00189.                                                                                                                                                     |
| [49]   | Zhao Y, Li M, Lai L, Suda N, Civin D, Chandra V (2022) Federated Learning with Non-IID Data. https://arxiv.org/abs/1806.00582.                                                                                                                                                                       |
| [50]   | Shirmohammadi S, Ferrero A. Camera as the instrument: the rising trend of vision based measurement. IEEE Instrumentation & Measurement Magazine. https://doi.org/10.1109/MIM.2014.6825388.                                                                                                           |
| [51]   | Corts-Osorio JA, Gmez-Mendoza JB, Riao-Rojas JC (2018) Velocity estimation from a single linear motion blurred image using discrete cosine transform. IEEE Transactions on Instrumentation and Measurement. pp. 4038-4050.                                                                           |
| [52]   | Hou K, Lv X, Zhang W (2020) An adaptive fusion panoramic image mosaic algorithm based on circular LBP feature and HSV color system. 2020 IEEE International Conference on Information Technology,Big Data and Artificial                                                                             |

|      | Intelligence (ICIBA). https://doi.org/10.1109/ICIBA50161.2020.9277348.                                                                                                                                                                                                                                                                                                                                                                                    |
|------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [53] | Xiong K, Fan P, Xu Z, Yang H, and Letaief K. B. (2014) Optimal Coop- erative Beamforming Design for MIMO Decode-and-Forward Relay Channels, IEEE Transactions on Signal Processing, vol. 62, issue. 6, pp. 1476 ? 1489, doi: 10.1109/TSP.2014.2298380.                                                                                                                                                                                                    |
| [54] | Chen X, Lu J, Fan P, and Letaief K. B. (2017) Massive MIMO Beamforming With Transmit Diversity for High Mobility Wireless Communications, IEEE Access, vol. 5, pp. 23032 ? 23045, doi: 10.1109/ACCESS.2017.2766157.                                                                                                                                                                                                                                       |
| [55] | Liu J, Xiong K, Ng D, Fan P, Zhong Z, and Letaief K. B. (2020) Max-Min Energy Balance in Wireless-Powered Hierarchical Fog-Cloud Computing Networks, IEEE Transactions on Wireless Communications, vol. 19, issue.11, pp. 7064 ? 7080, doi: 10.1109/TWC.2020.3007805.                                                                                                                                                                                     |
| [56] | Fan P, Feng C, Wang Y, and Ge N (2002) Investigation of the time-offset-based QoS support with optical burst switching in WDMnetworks, 2002 IEEE Interna- tional Conference on Communications. Conference Proceedings. ICC 2002, doi: 10.1109/ICC.2002.997330.                                                                                                                                                                                            |
| [57] | Jiang R, Xiong K, Fan P, Zhang Y, and Zhong Z (2019) Power Minimization in SWIPT Networks With Coexisting Power-Splitting and Time-Switching Users Under Nonlinear EH Model, IEEE Internet of Things Journal, vol. 6, issue. 5, pp. 8853-8869, doi: 10.1109/JIOT.2019.2923977.                                                                                                                                                                            |
| [58] | Guo Y, Xiong K, Lu Y, Wang D, Fan P, and Letaief, K. B. (2021) Achievable Information Rate in Hybrid VLC-RF Networks With Lighting Energy Harvesting, IEEE Transactions on Communications, vol. 69, issue. 10, pp. 6852-6864, doi: 10.1109/TCOMM.2021.3098030.                                                                                                                                                                                            |
| [59] | Krizhevsky A (2009) Learning Multiple Layers of Features from Tiny Images. https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf                                                                                                                                                                                                                                                                                                                 |
| [60] | Zheng H, Xiong K, Fan P, Zhong Z, and Letaief K. B. (2021) Age of Information- Based Wireless Powered Communication Networks With Selfish Charging Nodes, IEEE Journal on Selected Areas in Communications, vol. 39, issue. 5, pp. 1393 ? 1411, doi: 10.1109/JSAC.2021.3065038.                                                                                                                                                                           |
| [61] | Wei S, Cao G, Dai C, Dai S, Guo B (2023) FedCo: self-supervised Learning in Federated Learning with Momentum Contrast. 2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys). https://doi.org/10.1109/HPCC-DSS-SmartCity-DependSys57074.2022.00192. |

- [62] Mcmahan H, Moore E, Ramage D, Hampson S, Arcas B (2017) CommunicationEfficient Learning of Deep Networks from Decentralized Data. Proceedings of the 20 th International Conference on Artificial Intelligence and Statistics (AISTATS). https://arxiv.org/abs/1602.05629.
Xueying Gu Qiong Wu Pingyi Fan Qiang Fan
Mobility-Aware Federated Self-supervised Learning in Vehicular Network
Federated Learning (FL) is an advanced distributed machine learning approach,
that protects the privacy of each vehicle by allowing the model to be trained
on multiple devices simultaneously without the need to upload all data to a
road side unit (RSU). This enables FL to handle scenarios with sensitive or
widely distributed data. However, in these fields, it is well known that the
labeling costs can be a significant expense, and models relying on labels are
not suitable for these rapidly evolving fields especially in vehicular
networks, or mobile internet of things (MIoT), where new data emerges
constantly. To handle this issue, the self-supervised learning paves the way
for training without labels. Additionally, for vehicles with high velocity,
owing to blurred images, simple aggregation not only impacts the accuracy of
the aggregated model but also reduces the convergence speed of FL. This paper
proposes a FL algorithm based on image blur level to aggregation, called
FLSimCo, which does not require labels and serves as a pre-training stage for
self-supervised learning in the vehicular environment. Simulation results
demonstrate that the proposed algorithm exhibits fast and stable convergence.