Deep Neural Networks (DNNs) are increasingly deployed in cloud servers and autonomous agents due to their superior performance. The deployed DNN is either leveraged in a white-box setting (model internals are publicly known) or a black-box setting (only model outputs are known) depending on the application. A practical concern in the rush to adopt DNNs is protecting the models against Intellectual Property (IP) infringement. We propose BlackMarks, the first endto-end multi-bit watermarking framework that is applicable in the black-box scenario. BlackMarks takes the pre-trained unmarked model and the owner's binary signature as inputs. The output is the corresponding marked model with specific keys that can be later used to trigger the embedded watermark. To do so, BlackMarks first designs a model-dependent encoding scheme that maps all possible classes in the task to bit '0' and bit '1'. Given the owner's watermark signature (a binary string), a set of key image and label pairs is designed using targeted adversarial attacks. The watermark (WM) is then encoded in the distribution of output activations of the DNN by fine-tuning the model with a WM-specific regularized loss. To extract the WM, BlackMarks queries the model with the WM key images and decodes the owner's signature from the corresponding predictions using the designed encoding scheme. We perform a comprehensive evaluation of BlackMarks' performance on MNIST, CIFAR-10, ImageNet datasets and corroborate its effectiveness and robustness. BlackMarks preserves the functionality of the original DNN and incurs negligible WM embedding overhead as low as 2.054%.
summary:
The paper proposes an approach for model watermarking (i.e., watermarking a trained neural neteowrk). The watermark is a bit string, which is embedded in the model as part of a fine-tuning procedure, and can be decoded from the network from the model's specific predictions for a specific set of inputs (called keys) chosen during the fine-tuning step. The process generates a watermark when we can be confident that a model that didn't go through the exact same fine-tuning procedure gives significantly different predictions on the set of keys. The application scenario is when a company A wants to deploy a model for which A has IP ownership, and A wants to assess whether a competitor is (illegaly) re-using A's model. The approach presented in the paper works in the black-box setting, meaning that whether a model posesses the watermark can be assessed only by querying the model (i.e., without access to the internals of the model).
The overall approach closely follows Merrer et al. (2017), but extends this previous work to multi-bit watermarking. The similarity with Merrer et al. is that keys are generated with a procesdure to generate adversarial examples, watermarking is performed by specifically training the network to give the source label (i.e., the label of the image from which the adversarial example has been generated). The differences with Merrer et al. lie in the fact that each key encoded a specific bit (0 or 1), and the multi-bit watermark is encoded in the predictions for all keys (in case of a multi-class classifier, the labels are first partitionned into two clusters to map each class to either 0 or 1). In contrast, Merrer et al. focused on "zero-bit" watermarking, meaning that all keys together are only used to perform a test of whether the model has been watermarked (not encode the watermark). Another noticeable difference with Merrer et al. is in step 4 of the algorithm, in which several unmarked models are generated to select better key images.
comments:
While overall the approach makes sense and most of the design decisions seem appropriate, many questions are only partly addressed. My main concerns are:
1- the watermarks are encoded in adversarial examples for which the trained model gives the "true" label (i.e., the watermark is embedded in adversarial examples on which the model is robust). The evaluation does not address the concerns of false alarms on models trained to be robust to adversarial examples. Previous work (e.g., Merrer et al.) study at least the effect of fine-tuning with adversarial examples..
2- A watermark of length K is encoded in K images, and the test for watermarking is "The owner can prove the authorship of the model if the BER is zero.". This leaves little room to model manipulation. For instance, the competitor could randomize its predictions once in a while (typically output a random label for one out of K inputs), with very small decrease in accuracy and yet would have a non-negligible probability of having a non-zero BER.
other comments:
1- overhead section: in step 4 of the algorithm, there is a mention of "construct T unmarked models": why aren't they considered in the overhead? This seems to be an extremely significant part of the cost (the overall cost seems to be more T times the cost of building a single unmarked model rather than a few percent)
2- step 2 page 4: "The intuition here is that we want to filter out the highly transferable WM keys": I must have misunderstood something here. Why are highly transferable adversarial examples a problem? That would be the opposite: if we want the key to generate few false alarms (i.e., we do not want to claim ownership of a non-watermarked model), then we need the adversarial examples to "transfer" (i.e., be adversarial for non-watermarked models), since the watermarked model predicts the source class for the key. Merrer et al. (2017) on the contrary claim " As such adversaries seem to generalize across models [...] , this frontier tweaking should resist model manipulation and yield only few false positives (wrong identification of non marked model).", which means that transferability of adversarial examples is a fundamental assumption underlying the approach.
3- under Eq. 1: "Note that without the additional regularization loss (LWM), this retraining procedure resembles ‘adversarial training’ (Kurakin et al., 2016).": I do not understand that sentence. Without L_{WM}, the loss is the usual classification loss (L_0), and has nothing to do with adversarial training.
4- more generally, the contribution of the paper is on multi-bit watermarking, but there is no clear application scenario/experiment  where the multi-bit is more useful than the zero-bit watermarking.