Attack-Resistant Federated Learning with Residual-based Reweighting
Federated learning has a variety of applications in multiple domains by utilizing private training data stored on different devices. However, the aggregation process in federated learning is highly vulnerable to adversarial attacks so that the global model may behave abnormally under attacks. To tackle this challenge, we present a novel aggregation algorithm with residual-based reweighting to defend federated learning. Our aggregation algorithm combines repeated median regression with the reweighting scheme in iteratively reweighted least squares. Our experiments show that our aggression algorithm outperforms other alternative algorithms in the presence of label-flipping, backdoor, and Gaussian noise attacks. We also provide theoretical guarantees for our aggregation algorithm.
This paper presents an approach to robust federated learning that uses robust regression to weigh all the model parameter coefficients in order to achieve the robustness. Specifically, for each coefficient in the model, a repeated median estimator is used to compute a linear regression fit, and the residual of each individual model*s coefficient is normalized and used to compute a confidence score for that coefficient. Coefficients which have too large a confidence have their confidence reset to zero, to avoid the influence of outliers. The local model*s coefficients are now aggregated using a weighted average with weights given by these confidence scores. They compare their algorithm experimentally to reasonable baselines of model aggregation algorithms: the FedAvg algorithm, 3 recent robust FL algorithm, and an approach based on a standard robust regression estimator, using experiments on 4 different datasets and 4 different neural net architectures. They test the robustness of these algorithms to label flipping (MNIST, CIFAR-10), backdoor attacks, and multiplicative gaussian noise corruption of the model coefficients. Overall the paper presents an interesting and novel approach to robustness in FL, using a robust regression estimator to aggregate the model coefficients. The motivation of the algorithmic design is for the most part clear, but the rationale behind the particular choice of the parameter confidence score is unclear, and should be clarified. The theory in support of the method seems reasonablish, but key definitions and steps in the proof are not explained in detail, referring instead of an earlier paper. In particular, it is not clear how the smoothness of the loss function and subexponentiality of its derivatives enter into the analysis of the method, nor do these parameters enter into the final error bounds. Also, how is mu defined in the error bound: what does it mean that it is the expected global model --- does this mean this would be the model if all the participating workers were non-corrupted, honest, and had iid data? This theory is hard to parse: more effort should be spent in clarifying the assumptions and definitions and showing how the claimed result follows. The specific result referenced from earlier work should be stated unambiguously as a proposition so the reader sees how it applies where it is used. The experimental results show that the algorithm performs slightly better than the considered baselines in most situations considered, but the important question of the impact of hyperparameter selection for the method (e.g. the clipping at which the weights of *outlier* parameters are set to zero) and the competing methods (e.g. the clipping in the trimmed mean estimator) is not addressed-- the authors indicate that the method is robust to some choices and fixes them in the appendix. This makes it difficult to tell whether the method performs better due to careful or lucky hyperparameter selection. Although the method is interesting and novel, and seems principled, the theoretical claims are unclear, and the experimental evaluation is not sufficiently informative about the impact of hyperparameter selection to draw conclusions about the effectiveness of this method of model aggregation as opposed to the baselines considered. In particular because of the latter issue, I*m leaning towards reject, but would be willing to change my score if this were addressed.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Attack-Resistant Federated Learning with Residual-based Reweighting
Federated learning has a variety of applications in multiple domains by utilizing private training data stored on different devices. However, the aggregation process in federated learning is highly vulnerable to adversarial attacks so that the global model may behave abnormally under attacks. To tackle this challenge, we present a novel aggregation algorithm with residual-based reweighting to defend federated learning. Our aggregation algorithm combines repeated median regression with the reweighting scheme in iteratively reweighted least squares. Our experiments show that our aggression algorithm outperforms other alternative algorithms in the presence of label-flipping, backdoor, and Gaussian noise attacks. We also provide theoretical guarantees for our aggregation algorithm.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper presents an approach to robust federated learning that uses robust regression to weigh all the model parameter coefficients in order to achieve the robustness. Specifically, for each coefficient in the model, a repeated median estimator is used to compute a linear regression fit, and the residual of each individual model*s coefficient is normalized and used to compute a confidence score for that coefficient. Coefficients which have too large a confidence have their confidence reset to zero, to avoid the influence of outliers. The local model*s coefficients are now aggregated using a weighted average with weights given by these confidence scores. They compare their algorithm experimentally to reasonable baselines of model aggregation algorithms: the FedAvg algorithm, 3 recent robust FL algorithm, and an approach based on a standard robust regression estimator, using experiments on 4 different datasets and 4 different neural net architectures. They test the robustness of these algorithms to label flipping (MNIST, CIFAR-10), backdoor attacks, and multiplicative gaussian noise corruption of the model coefficients. Overall the paper presents an interesting and novel approach to robustness in FL, using a robust regression estimator to aggregate the model coefficients. The motivation of the algorithmic design is for the most part clear, but the rationale behind the particular choice of the parameter confidence score is unclear, and should be clarified. The theory in support of the method seems reasonablish, but key definitions and steps in the proof are not explained in detail, referring instead of an earlier paper. In particular, it is not clear how the smoothness of the loss function and subexponentiality of its derivatives enter into the analysis of the method, nor do these parameters enter into the final error bounds. Also, how is mu defined in the error bound: what does it mean that it is the expected global model --- does this mean this would be the model if all the participating workers were non-corrupted, honest, and had iid data? This theory is hard to parse: more effort should be spent in clarifying the assumptions and definitions and showing how the claimed result follows. The specific result referenced from earlier work should be stated unambiguously as a proposition so the reader sees how it applies where it is used. The experimental results show that the algorithm performs slightly better than the considered baselines in most situations considered, but the important question of the impact of hyperparameter selection for the method (e.g. the clipping at which the weights of *outlier* parameters are set to zero) and the competing methods (e.g. the clipping in the trimmed mean estimator) is not addressed-- the authors indicate that the method is robust to some choices and fixes them in the appendix. This makes it difficult to tell whether the method performs better due to careful or lucky hyperparameter selection. Although the method is interesting and novel, and seems principled, the theoretical claims are unclear, and the experimental evaluation is not sufficiently informative about the impact of hyperparameter selection to draw conclusions about the effectiveness of this method of model aggregation as opposed to the baselines considered. In particular because of the latter issue, I*m leaning towards reject, but would be willing to change my score if this were addressed.
assistant
This paper presents an approach to robust federated learning that uses robust regression to weigh all the model parameter coefficients in order to achieve the robustness. Specifically, for each coefficient in the model, a repeated median estimator is used to compute a linear regression fit, and the residual of each individual model*s coefficient is normalized and used to compute a confidence score for that coefficient. Coefficients which have too large a confidence have their confidence reset to zero, to avoid the influence of outliers. The local model*s coefficients are now aggregated using a weighted average with weights given by these confidence scores. They compare their algorithm experimentally to reasonable baselines of model aggregation algorithms: the FedAvg algorithm, 3 recent robust FL algorithm, and an approach based on a standard robust regression estimator, using experiments on 4 different datasets and 4 different neural net architectures. They test the robustness of these algorithms to label flipping (MNIST, CIFAR-10), backdoor attacks, and multiplicative gaussian noise corruption of the model coefficients. Overall the paper presents an interesting and novel approach to robustness in FL, using a robust regression estimator to aggregate the model coefficients. The motivation of the algorithmic design is for the most part clear, but the rationale behind the particular choice of the parameter confidence score is unclear, and should be clarified. The theory in support of the method seems reasonablish, but key definitions and steps in the proof are not explained in detail, referring instead of an earlier paper. In particular, it is not clear how the smoothness of the loss function and subexponentiality of its derivatives enter into the analysis of the method, nor do these parameters enter into the final error bounds. Also, how is mu defined in the error bound: what does it mean that it is the expected global model --- does this mean this would be the model if all the participating workers were non-corrupted, honest, and had iid data? This theory is hard to parse: more effort should be spent in clarifying the assumptions and definitions and showing how the claimed result follows. The specific result referenced from earlier work should be stated unambiguously as a proposition so the reader sees how it applies where it is used. The experimental results show that the algorithm performs slightly better than the considered baselines in most situations considered, but the important question of the impact of hyperparameter selection for the method (e.g. the clipping at which the weights of *outlier* parameters are set to zero) and the competing methods (e.g. the clipping in the trimmed mean estimator) is not addressed-- the authors indicate that the method is robust to some choices and fixes them in the appendix. This makes it difficult to tell whether the method performs better due to careful or lucky hyperparameter selection. Although the method is interesting and novel, and seems principled, the theoretical claims are unclear, and the experimental evaluation is not sufficiently informative about the impact of hyperparameter selection to draw conclusions about the effectiveness of this method of model aggregation as opposed to the baselines considered. In particular because of the latter issue, I*m leaning towards reject, but would be willing to change my score if this were addressed.