We study the problem of learning permutation invariant representations that can capture "flexible" notions of containment. We formalize this problem via a measure theoretic definition of multisets, and obtain a theoretically-motivated learning model. We propose training this model on a novel task: predicting the size of the symmetric difference (or intersection) between pairs of multisets. We demonstrate that our model not only performs very well on predicting containment relations (and more effectively predict the sizes of symmetric differences than DeepSetsbased approaches with unconstrained object representations), but that it also learns meaningful representations.
This paper presents a framework for learning representations of multisets.  The approach is built on top of the DeepSets (Zaheer et al., 2017) model, which already is capable of learning representations of multisets, with a new objective of predicting the size of symmetric differences between two multisets, plus a small change in the implementation of the DeepSets model by restricting the per-element module to output a point in the probability simplex.
I liked the background on fuzzy sets and the development of fuzzy multisets and different operations on these multisets.  The definitions and arguments are quite clear and helpful.  One small suggestion for page 4 is that I can understand why the formulation is the only sensible choice for multisets with desired properties, but a claim like this deserves a proper proof.
Model-wise the paper made two contributions for learning representations for multisets as mentioned above: (1) proposed the symmetric difference prediction as a task for learning representations, the argument for this task is that predicting symmetric difference implicitly encourages the model to learn about containment; (2) a slight change in the DeepSets model architecture where the outer rho function is identity and the inner phi function has to output a point in the simplex.
I found these technical contributions to be a bit small.  In addition to this, the paper only presents results on MNIST in a toyish setting, this makes me feel the paper may be more suited for publication in a workshop (idea is interesting, small scale experiments to illustrate the insights, but not complete enough to be published at a conference).
Regarding contribution (1), I can see why predicting symmetric difference makes sense as argued in the paper, but I’m not convinced that this is better than other alternatives.  In order to show that this is a reasonable approach for learning representations, some results that compare this with other possible learning objectives would be necessary.  But I don’t see any such results in this paper.
Regarding contribution (2), I feel the restriction of the phi function to output points in simplex is not very well motivated and confusing in the first read.  Again I can understand why we may want to do this but don’t see why we need to do this.  I’m also concerned that such an architecture may only be good for the task of predicting symmetric difference as it is customized for this task.  Figure 3 shows that an unrestricted model seems to learn better representations despite a worse symmetric difference prediction error, which again confirms the concern.
Another thing about the experiment setup: the second baseline, labeled “DeepSets” in Table 1 actually changed two things compared to the proposed approach: (1) changing the psi function and (2) also changed the symmetric difference function.  It would be good to isolate the contribution of the two.
Overall I feel this paper is not yet ready to be published at ICLR.