Currently the only techniques for sharing governance of a deep learning model are homomorphic encryption and secure multiparty computation. Unfortunately neither of these techniques is applicable to the training of large neural networks due to their large computational and communication overheads. As a scalable technique for shared model governance, we propose splitting deep learning model between multiple parties. This paper empirically investigates the security guarantee of this technique, which is introduced as the problem of model completion: Given the entire training data set or an environment simulator, and a subset of the parameters of a trained deep learning model, how much training is required to recover the model's original performance? We define a metric for evaluating the hardness of the model completion problem and study it empirically in both supervised learning on ImageNet and reinforcement learning on Atari and DeepMind Lab. Our experiments show that (1) the model completion problem is harder in reinforcement learning than in supervised learning because of the unavailability of the trained agent's trajectories, and (2) its hardness depends not primarily on the number of parameters of the missing part, but more so on their type and location. Our results suggest that model splitting might be a feasible technique for shared model governance in some settings where training is very expensive.
This paper proposes the interesting idea of analyzing how difficult
it is to re-initialize and re-train layers in neural networks.
They study these techniques in the context of ImageNet classification
and reinforcement learning in the Atari and DeepMind lab domains.
While these are interesting ideas and domains to study, I have
concerns with the positioning and execution of the paper.
[Positioning, execution and motivation]
On the positioning of the paper, a significant part of the introduction
and related work section is spent arguing that this approach can be used
for shared model governance in contexts where homomorphic encryption
or secure multi-party computation would instead be used.
Comparing the approaches studied in this paper to these
sophisticated cryptographically-motivated techniques seems
like too much of a stretch, as the methods serve very different
purposes and in most cases cannot even be directly compared.
The first and second paragraph discuss the vision of distributing
the training of models between multiple parties.
I agree that this is a useful area to study and direction
for the community to go, but as the introduction of this paper
states, this is the most interesting when the parties have
control over logically separate components of the modeling pipeline
and also when joint training of the components is being done,
potentially on disjoint and private datasets.
The empirical results of this paper do none of this,
as they only look at the case when a single layer is being
replaced.
Furthermore the motivation and positioning of the paper is
not carried through in the empirical setup, where they
investigate approaches that do training over all
of the parameters of the model, breaking the assumption
that the parties should be independent and should
not share information.
[Metrics for measuring model completeness]
Section 3.1 defines the metric of completion hardness that is
used throughout the rest of the paper. The metric looks at the
number of iterations that re-training the model takes to
reach the same performance as the original model.
It's not clear why this is an important metric and I am
not convinced it is the right one to use as it:
1) does not give a notion of how nicely the missing portion
was recovered, just that the accuracy reached the
same accuracy as the original network, and
2) methods with a very long per-iteration runtime such as
second-order and sampling-based methods could be used to
reach a good performance in a small number of iterations,
making these methods appear to be very "good" at
completing models. I don't think it is nice that this
metric relies on the same optimizer being used for the
original model and the completed model.
I think it's more interesting to study *how much* data is
required to recover missing portions of the model instead
of how many iterations are needed to recover the same performance.
The supervised learning experiments appear to be done
using the entire dataset while the RL experiments do
present a setting where the data is not the same.
[Empirical results]
I am also surprised by the empirical finding in Section 5.1
that T1 outperforms T2, since it seems like only optimizing
the parameters of the missing layer would be the best
approach. I think that if a similarity metric was used
instead, T2 would be significantly better at finding the
layer that is the most similar to the layer that was removed.
Some smaller comments:
1. In Section 3.1, the definition of C_T does not use T explicitly
   inside of it.
2. In the last paragraph of Section 3.1 and first paragraph of
   Section 3.2, N should be defined as an iteration that
   reaches the best loss.
3. The description of T3 does not say what method is used to
   optimize the over-parameterized layer, is it T1 or T2?
4. Why does T4 use T1 instead of T2?
5. In the experimental setup, why is T2 applied with a different
   learning rate schedule than the original training procedure?
6. Why is T2 not shown in the AlexNet results for Figure 2?
7. The dissimilar axes between the plots in Figure 2 and
   Figure 3 make them difficult to compare and interpret.
8. It's surprising that in Figure 3, the hardness of \alpha=1.0
   for T2 is 1.0 for everything.