Meta-learning methods learn the meta-knowledge among various training tasks 1 and aim to promote the learning of new tasks under the task similarity assumption.
2 However, such meta-knowledge is often represented as a fixed distribution, which 3 is too restrictive to capture various specific task information. In this work, we 4 present a localized meta-learning framework based on the PAC-Bayes theory. In 5 particular, we propose an LCC-based prior predictor that allows the meta learner to 6 adaptively generate local meta-knowledge for specific tasks. We further develop
In this work, the authors introduce PAC-Bayesian generalization bounds for Meta-Learning. In their framework, they have a hyper-prior distribution, a class of hyper-posteriors and an algorithm A that takes a sample set Si from task Di and a prior P and returns a posterior Q. 
Pros:
-- Overall, the main text of the paper is finely written.
-- The motivation is well articulated.
-- The relationship with local coordinate coding seems like an interesting direction.
-- The experiments seem sensible. 
The novelty of the bound in Thm. 1:
-- It seems that several aspects of the high-level derivation methodology of the bound are not new. Instead of applying McAllester’s bound twice (as done by Galanti et al. 2016, Amit and Meir 2018), the authors employ Catoni’s bound (a different variant of PAC-Bayes) twice. In addition, the authors apply the standard Gaussian randomization which leads to L2 regularization terms as the capacity terms -- also well known in the literature (see for example Hazan et al. 2013, etc’). 
-- I would be happy if the authors point out which parts of their derivations are novel, for instance, the application of local coordinate coding, etc'.
I think the authors' claim that their bound (Thm. 1) is tighter than previously existing bounds is a bit questionable:
-- There are already PAC-Bayesian bounds with the proposed orders of magnitudes with the exact same coefficients by Catoni 2007 and Pascal Germain et al. 2009. In fact, the authors employ Catoni’s bound within the appendix. In my opinion, it should be addressed in the main text. 
-- In addition, their bound has two coefficients that are being ignored in their analysis of its magnitude: c/(1-exp(-c)) (here, c stands for c1 or c2) for the error term and 1/(1-exp(-c)) for the capacity term and an additional constant that depends on n,mi and \delta. In the previous bounds, the coefficients are 1 for the error term and 1 for the capacity terms. I think the paper would greatly benefit from a direct comparison between the two.
For instance, as a direct comparison between the two bounds, I would select c, such that, 1/(1-exp(-c)) is close to 1. However, in this case, the coefficient 1/(1-exp(-c)) of the capacity term is huge and makes the overall capacity term very large, which is not favorable. Therefore, it seems to me that the proposed bound is favorable only when the training error is very small (realizable case) and c can be selected to be large. However, when the training error is assumed to be small, it is well known that the gap between the generalization risk and the empirical risk is of order O(capacity/m) (see Thm. 6.8 (bullet 3) in S. Shalev-Schwarz and S. Ben-David 2014). Again, this is also a property of the original bound by Catoni.
Finally, the authors mention that the parameters c1 and c2 control the tradeoff between the empirical error and the capacity terms. I think this is a bit inaccurate, since, c1 and c2 are selected a-priori to the estimation of the error term. Therefore, should be independent of the error term. 
-- The presented bound has an additional constant “E_i[const(n,mi,delta)]”. 
It is unclear to me what is the magnitude of this quantity. I think it might improve the paper if the authors explicitly analyze this term.
From the proof of Thm. 2 (Eq. 36) it seems to be of order >= ( O_{\alpha,\beta}(\gamma,C) + (1/m_{ik})^{1/2} )^2. 
What is the definition of m_{ik} (couldn’t find it, maybe it was defined somewhere -- I think it should have been recalled)? I’m assuming it is mi or something similar. 
Second, I’m not sure how to estimate the size of O_{\alpha,\beta}(\gamma,C). From Lem. 1 it depends on some arbitrarily selected quantity \epsilon > 0 and |C| is a function of \epsilon, so I’m not sure how to measure it. 
------------------------------------
Soundness: 
There are a few things that I'd be happy if clarified regarding Thms. 1 and  2. 
-- I’m not sure what is w^Q_i (couldn’t find its definition anywhere). I guess w^Q_i is the center of Q_i = A(Si,P), where P ~ \mathcal{Q}. How can w^Q_i be free within the inequality without being bound to an expectation over P? Especially since you take an expectation over P within the training error. I guess the correct formulation of the bound is one that has  E_{P ~ \mathcal{Q}}[||w^Q_i - w^{\mathcal{Q}}||^2] instead of ||w^Q_i - w^{\mathcal{Q}}||^2. 
Maybe the issue is somewhere in between Eq 39 and 40, where the authors take an expectation over Pi (which is not clearly defined as well), but not over Qi?
-- In Eq. 27 last equality: we have a term of the form: E_v ||w^Q - w^P||^2. Where is v in the distance squared?
-- I’m not sure what is the motivation of applying Eq. 28, the bound should be tighter with the LHS term instead of the RHS.
-- In the top half of page 15, P stands for a prior independent of \mathcal{Q}, while in the main text, P~\mathcal{Q}. Therefore, the relationships between different arguments are unclear to me.   
-- Finally, I think the paper could be improved if the authors would organize the appendix.
Experiments:
In your experiments you compare the derived algorithm to other PAC-Bayesian baselines. Can you show that your algorithm outperforms other existing baselines in the literature of Meta-Learning (in terms of generalization, etc')?
Typos:
-- “Catino” ---> “Catoni”.
Finally, I think the authors should address the public comment by Tomer Galanti. It seems that Thm. 9 in Galanti et al. (2016) introduces a PAC-Bayesian theorem for Meta-Learning (they call it transfer learning), similar in its nature to Thm. 1 in the current paper. In Thm. 9 they have a hyperprior, denoted by P, learn a hyper posterior Q, for selecting posteriors qi for many different tasks di. In their framework, for each task di, their method returns a posterior qi that minimizes the bound for a specific task. This distribution, qi, is a function of a prior B selected by Q (P in your notation) and the i’th task’s dataset si (Si in your notation). Therefore, instead of denoting it by A(Si,P) as done by the authors, they simply call it a minimizer (but it is a function of the same argument that the authors address). Overall, it seems to me that Galanti et al. had a very similar setting, with different notations and focus on a specific algorithm A. 
I think that overall, the paper has interesting insights and the relationship with local coordinate coding seems like an interesting direction. However, I think the paper should not be published in its current form.