content_image/440324.jpg content_image/440318.jpg content_image/440330.jpg content_image/440329.jpg content_image/440328.jpg content_image/440327.jpg content_image/440322.jpg content_image/440331.jpg content_image/440332.jpg content_image/440326.jpg content_image/440325.jpg content_image/440333.jpg content_image/440320.jpg content_image/440319.jpg content_image/440321.jpg content_image/440317.jpg
the random variables are Gaussian approximations to actual random variables in the problems. To be specific, first observe that 1 √ 

Combining (66) with the definition of z$_{i}$$_{→}$$_{j}$ in (9) and the fact that z = Φ x , we have 

where 

The induction hypothesis (42c) and Lemma 6(a) then show that 

where the convergence is in distribution. 

With these definitions, we first consider the convergence of µ q i ← $_{j}$( t ) . Using (19b), (20b) and (66), we have that 

<img src='content_image/440324.jpg'>

By Lemma 6(a), given x$_{j}$ and s$_{j}$ , all the terms in (70) are independent. 

Also, using (69), we have the limit 

where the convergence in (a) is in distribution; (b) follows from the assumption that D$_{3}$ ( · ) is uniformly bounded and (c) follows from the fact that (68) shows that | u$_{ℓ}$ | 3 = O ( d − 3 / $^{2}$) → 0 . We can apply the Modified Law of Large Numbers (Lemma 2) to the sum (70) to obtain the limit 

<img src='content_image/440318.jpg'>

where the limit in (a) is in distribution and follows from (70) and (71); (b) follows from (30c) and the Modified Law of Large Numbers; (c) follows from the definition of E$_{out}$ ( · ) in (34) and the fact that the expectation over ( y, z ) in (34) 

<img src='content_image/440330.jpg'>

<img src='content_image/440329.jpg'>

<img src='content_image/440328.jpg'>

<img src='content_image/440327.jpg'>

<img src='content_image/440322.jpg'>

is identical to ( y G ℓ (0) , z G ℓ (0)) ; and (d) follows from the SE equation (37a). 

We next turn to the distribution of ̂ q$_{i}$$_{←}$$_{j}$ ( t ) . Using (19), the update (20a) can be simplified to 

<img src='content_image/440331.jpg'>

So, using (69) and (72), 

<img src='content_image/440332.jpg'>

where here and below we use the shorthand lim$_{d,n}$ for lim$_{d}$ lim$_{n}$ . Now define 

<img src='content_image/440326.jpg'>

so we can rewrite (74) as 

<img src='content_image/440325.jpg'>

where the last step follows from (74) and (58). 

Now, applying Lemma 7 to e$_{ℓ}$ in (75), 

<img src='content_image/440333.jpg'>

<img src='content_image/440320.jpg'>

From the definition of u$_{ℓ}$ in (68), the O ( | u$_{ℓ}$ | $^{2}$) terms are O (1 /d ) and thus can be ignored. Applying the Modified Central Limit Theorem (Lemma 3) to the sum in (76) along with (30c) we obtain 

<img src='content_image/440319.jpg'>

Also since x$_{j}$ ∼ p$_{X}$ ( x$_{j}$ ) and s$_{j}$ ∼ p$_{S}$ ( s$_{j}$ ) , (72) and (77) together show that for any iteration t , 

where the convergence is in distribution, x ∼ p$_{X}$ ( x ) and s ∼ p$_{S}$ ( s ) . Applying (21) to (78) shows (42a). Therefore, we have 

<img src='content_image/440321.jpg'>

<img src='content_image/440317.jpg'>

