Probabilistic Rule Realization and Selection
Abstraction and realization are bilateral processes that are key in deriving intelligence and creativity. In many domains, the two processes are approached through rules: high-level principles that reveal invariances within similar yet diverse examples. Under a probabilistic setting for discrete input spaces, we focus on the rule realization problem which generates input sample distributions that follow the given rules. More ambitiously, we go beyond a mechanical realization that takes whatever is given, but instead ask for proactively selecting reasonable rules to realize. This goal is demanding in practice, since the initial rule set may not always be consistent and thus intelligent compromises are needed. We formulate both rule realization and selection as two strongly connected components within a single and symmetric bi-convex problem, and derive an efficient algorithm that works at large scale. Taking music compositional rules as the main example throughout the paper, we demonstrate our model's efficiency in not only music realization (composition) but also music interpretation and understanding (analysis).
The paper tackles rules realisation and selection, given a specific set of given rules that derives from an abstraction process, in the specific case of music compositional rules. The novelty of the paper resides in a new bi-convex optimisation model for rules selection and rule realisation. The challenge posed by rules realisation is due to the high dimensionality of the sample space. The challenge posed by rule selection appears in the case when all rules cannot be simultaneously realised, or not in their entirety. In this case one has to choose which rules to respect and in which amount of components. The strengths of the paper are:
- the goal and challenge of the model are very clearly stated,
- the paper is extremely well written with all notations defined and intuitive explanations of the theoretical results. I especially liked the interpretation of lines 285 - 296. 
- experimental results support the claim of the proposed innovations, as the model (i) behaves as expected when varying the hyper parameters (ii) reduces computation time.
Questions / comments :
- line 118 : can the author explain why the solution is expected to be "unique" on p thanks to group penalty ? If there is no group penalty (\lambda_p = 0) I would expect the solution to be unique, which is also what the authors state in line 131 "in setting \lambda_p sufficiently small, we secure a unique solution for the rule realisation part". The two statements seem unconsistent.
- line 148 : Theorem 1, claim (2) : am I right to understand that all rules are selected because for all rules the L2 norm converges to a non-zero value ?
- Group De-overlap : am I right to understand that this is not an additional dimensionality reduction but only an interpretation of the grouping presented in Section (3), while screening is an additional step to further reduce dimensionality ?
- line 304 : what do the authors mean by "failures in group selection" ? Does it mean that no rule is entirely selected ?
- line 305 : am I right to understand that a simple lasso on the number of rules is impossible because it makes no sense to weight the rules as the selection process is a 0/1 process ?
- I am not familiar with the literature in automatic music composition, and I understand that there is no "ground-truth" solution as creativity can be a desired behaviour. I like the fact that the author mention the gain in computational efficiency.  If that exists / is possible in the time of rebuttal I would like to see a baseline model to compare the behaviour in terms of selection / realisation of the proposed model.
- Can the author think of any other possible domain to apply their method ?