text: Survey
  
  * Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project
  
  Document related concepts
  
  Fundamental theorem of algebra wikipedia, lookup
  
  Transcript
  ```1
  Chapter 2
  Some Basic Large Sample Theory
  1. Modes of Convergence
  Convergence in distribution, →d
  Convergence in probability, →p
  Convergence almost surely, →a.s.
  Convergence in r−th mean, →r
  2. Classical Limit Theorems
  Weak and strong laws of large numbers
  Classical (Lindeberg) CLT
  Liapounov CLT
  Lindeberg-Feller CLT
  Cramér-Wold device; Mann-Wald theorem; Slutsky’s theorem
  Delta-method
  3. Replacing →d by →a.s.
  4. Empirical Measures and Empirical Processes
  The empirical distribution function; the uniform empirical process
  The Brownian bridge process
  Finite-dimensional convergence
  A hint of Donsker’s Mann-Wald type theorem
  General empirical measures and processes
  5. The Partial Sum Process and Brownian motion
  Brownian motion
  Existence of Brownian motion and Brownian bridge as continuous processes
  6. Sample Quantiles
  2
  Chapter 2
  Some Basic Large Sample Theory
  1
  Modes of Convergence
  Consider a probability space (Ω, A, P ). For our ﬁrst three deﬁnitions we suppose that X, Xn , n ≥ 1
  are all random variables deﬁned on this one probability space.
  Deﬁnition 1.1 We say that Xn converges a.s. to X, denoted by Xn →a.s. X, if
  (1)
  Xn (ω) → X(ω)
  for all ω ∈ A where P (Ac ) = 0 ,
  or, equivalently, if, for every > 0
  (2)
  P ( sup |Xm − X| > ) → 0
  m≥n
  as n → ∞ .
  Deﬁnition 1.2 We say that Xn converges in probability to X and write Xn →p X if for every
  >0
  (3)
  P (|Xn − X| > ) → 0
  as n → ∞ .
  Deﬁnition 1.3 Let 0 < r < ∞. We say that Xn converges in r−th mean to X, denoted by
  Xn →r X, if
  (4)
  E|Xn − X|r → 0
  as n → ∞ for functions Xn , X ∈ Lr (P ) .
  Deﬁnition 1.4 We say that Xn converges in distribution to X, denoted by Xn →d X, or Fn → F ,
  or L(Xn ) → L(X) with L referring to the the “law” or “distribution”, if the distribution functions
  Fn and F of Xn and X satisfy
  (5)
  Fn (x) → F (x)
  as n → ∞
  for each continuity point x of F .
  Note that Fn ≡ 1[1/n,∞) →d 1[0,∞) ≡ F even though Fn (0) = 0 does not converge to 1 = F (0).
  The statement →d will carry with it the implication that F corresponds to a (proper) probability
  measure P .
  Deﬁnition 1.5 A sequence of random variables {Xn } is uniformly integrable if
  (6)
  lim lim sup E |Xn |1[|Xn |≥λ] = 0 .
  λ→∞ n→∞
  3
  4
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  Theorem 1.1 (Convergence implications).
  A. If Xn →a.s. X, then Xn →p X.
  B. If Xn →p X, then Xn →a.s. X for some subsequence {n }.
  C. If Xn →r X, then Xn →p X.
  D. If Xn →p X and |Xn |r is uniformly integrable, then Xn →r X.
  If Xn →p X and lim supn→ E|Xn |r ≤ E|X|r , then Xn →r X.
  E. If Xn →r X then Xn →r X for all 0 < r ≤ r.
  F. If Xn →p X, then Xn →d X.
  G. Xn →p X if and only if every subsequence {n } contains a further subsequence {n } for which
  Xn →a.s. X.
  Theorem 1.2 (Vitali’s theorem). Suppose that Xn ∈ Lr (P ) where 0 < r < ∞ and Xn →p X.
  Then the following are equivalent:
  A. {|Xn |r } are uniformly integrable.
  B. Xn →r X.
  C. E|Xn |r → E|X|r .
  Before proving the theorems we need a short review of some facts about convex functions and
  some inequalities. We ﬁrst brieﬂy review convexity. A real valued function f is convex if
  (7)
  f (αx + (1 − α)y) ≤ αf (x) + (1 − α)f (y)
  for all x, y and all 0 ≤ α ≤ 1. This holds if and only if
  1
  1
  f
  (8)
  (x + y) ≤ [f (x) + f (y)]
  2
  2
  for all x, y provided f is continuous and bounded. Note that (8) holds if and only if
  (9)
  1
  f (s) ≤ [f (s − r) + f (s + r)]
  2
  for all r, s .
  Also
  (10)
  f (x) ≥ 0
  for all x implies f is convex .
  We call f strictly convex if strict inequality inequality holds in any of the above. If f is convex,
  then there exists a linear function l such that f (x) ≥ l(x) with equality at a prespeciﬁed x0 in the
  interior of the domain of f ; this is called the supporting hyperplane theorem.
  Deﬁnition 1.6 Assuming the following expectations (integrals) exist,
  (11)
  µ ≡ E(X) = the mean of X .
  (12)
  σ 2 ≡ V ar[X] ≡ E(X − µ)2 = the variance of X .
  (13)
  E(X k ) = k-th moment of X for k ≥ 1 an integer .
  (14)
  E|X|r = r-th absolute moment of X for r ≥ 0 .
  (15)
  E(X − µ)k = k- th central moment of X .
  (16)
  Cov[X, Y ] ≡ E[(X − µX )(Y − µY )] = the covariance of X and Y .
  1.
  MODES OF CONVERGENCE
  5
  Proposition 1.1 If E|X|r < ∞, then E|X|r and E(X k ) are ﬁnite for all r ≤ r and integers
  k ≤ r.
  Proof.
  Now |x|r ≤ 1 + |x|r ; and integrability is equivalent to absolute integrability.
  2
  Proposition 1.2 V ar(X) ≡ σ 2 < ∞ if and only if E(X 2 ) < ∞. In this case σ 2 = E(X 2 ) − µ2 .
  Proof.
  Suppose that σ 2 < ∞. Then σ 2 + µ2 = E(X − µ)2 + E(2µX − µ2 ) = E(X 2 ). Suppose
  that E(X 2 ) < ∞. Then E(X 2 ) − µ2 = E(X 2 ) − E(2µX − µ2 ) = E(X − µ)2 = V ar[X]. 2
  Proposition 1.3 (cr −inequality). E|X + Y |r ≤ cr E|X|r + cr E|Y |r where cr = 1 for 0 < r ≤ 1
  and cr = 2r−1 for r ≥ 1.
  Proof.
  Case 1: r ≥ 1. Then |x|r is a convex function of x; take second derivatives. Thus
  |(x + y)/2|r ≤ [|x|r + |y|r ]/2; and now take expectations.
  Case 2: 0 < r ≤ 1: Now |x|r is concave and ↑ for x ≥ 0; examine derivatives. Thus
  x+y
  y
  r
  r
  r−1
  |x + y| − |x| =
  rt dt =
  r(x + s)r−1 ds
  0
  xy
  ≤
  rsr−1 ds = |y|r ,
  0
  and now take expectations.
  2
  Proposition 1.4 (Hölder inequality). E|XY | ≤ E 1/r |X|r E 1/s |Y |s ≡ Xr Y s for r > 1 where
  1/r + 1/s = 1 deﬁnes s. When the expectations are ﬁnite we have equality if and only if there
  exists A and B not both 0 such that A|X|r = B|Y |s a.e.
  Proof.
  Now
  The result is trivial if E|X|r = 0 or ∞. Likewise for E|Y |s . So suppose that E|X|r > 0.
  |ab| ≤
  |a|r
  |b|s
  +
  ,
  r
  s
  as in the ﬁgure .
  Now let a = |X|/Xr and b = |Y |/Y s ; and take expectations. Equality holds if and only if
  |Y |/Y s = (|X|/Xr )1/(1−s) a.e.; if and only if
  |Y |s
  =
  E|Y |s
  |X|
  Xr
  s
  s−1
  =
  |X|r
  E|X|r
  a.e.
  if and only if there exist A, B =
  0 such that A|X|r = B|Y |s . This also gives the next inequality as
  an immediate consequence. 2
  Proposition 1.5 (Cauchy-Schwarz inequality). (E|XY |)2 ≤ E(X 2 )E(Y 2 ) with equality if and
  only if there exists A, B not both 0 such that A|X| = B|Y | a.e.
  6
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  Remark 1.1 Thus for non-degenerate random variables (i.e. non-zero variance) with ﬁnite variance we have
  (17)
  −1 ≤ ρ ≤ 1
  where
  (18)
  ρ ≡ Corr[X, Y ] ≡ Cov[X, Y ]
  V ar[X]V ar[Y ]
  is called the correlation of X and Y . Note that ρ = 1 if and only if X − µX = A(Y − µY ) for some
  A > 0 and ρ = −1 if and only if X − µX = A(Y − µY ) for some A < 0. Thus ρ measures linear
  dependence, not dependence.
  Proposition 1.6 log E|X|r is convex in r for r ≥ 0. It is linear if and only if |X| = c a.e. for some
  c.
  Proof.
  Let 0 ≤ r ≤ s. Apply the Cauchy-Schwarz inequality to |X|(s−r)/2 and |X|(s+r)/2 and
  take logs to get
  1
  log E|X|s ≤
  log E|X|s−r + log E|X|s+r .
  2
  2
  Proposition 1.7 (Liapunov inequality). Let X be a random variable. Then E 1/r |X|r is ↑ in r for
  r ≥ 0.
  Proof.
  The slope of the chord of y = log E|X|r is ↑ in r by proposition 1.6. That is,
  r
  (1/r) log E|X| is ↑ in r. We used P (Ω) = 1 < ∞ to show that E|X|r < ∞ if E|X|r < ∞ for
  r ≤ r in proposition 1.1. 2
  ≥ µr−t
  Exercise 1.1 Let µr ≡ E|X|r . For r ≥ s ≥ t ≥ 0 we have µrs−t µr−s
  s .
  t
  Proposition 1.8 (Minkowski’s inequality). For r ≥ 1 we have E 1/r |X +Y |r ≤ E 1/r |X|r +E 1/r |Y |r .
  Proof.
  (a)
  It is trivial for r = 1. Suppose that r > 1. Then for any measure
  E|X + Y |r ≤ E|X||X + Y |r−1 + E|Y ||X + Y |r−1
  ≤ {Xr + Y r } |X + Y |r−1 s
  = {Xr + Y r } E
  1/s
  |X + Y |
  = {Xr + Y r } E
  1/s
  |X + Y |r .
  by Hölder’s inequality
  (r−1)s
  If E|X + Y |r = 0, it is trivial. If not, we divide to get the result.
  2
  Proposition 1.9 (Basic inequality). Let g ≥ 0 be an even function which is ↑ on [0, ∞). Then for
  all random variables X and for all > 0
  (19)
  P (|X| ≥ ) ≤
  Eg(X)
  .
  g()
  1.
  MODES OF CONVERGENCE
  Proof.
  7
  Now
  Eg(X) = E{g(X)1[|X|≥] } + E{g(X)1[|X|<] }
  (a)
  ≥ E{g(X)1[|X|≥] } ≥ g()E{1[|X|≥] }
  = g()P (|X| ≥ )
  as claimed.
  2
  The next two inequalities are immediate corollaries of the basic inequality.
  Proposition 1.10 (Markov’s inequality).
  (20)
  P (|X| ≥ ) ≤
  E|X|r
  r
  for all > 0 .
  Proposition 1.11 (Chebychev’s inequality).
  (21)
  P (|X − µ| ≥ ) ≤
  V ar[X]
  2
  for all > 0 .
  Proposition 1.12 (Jensen’s inequality). If g is convex on (a, b) where −∞ ≤ a < b ≤ ∞ and if
  P (X ∈ (a, b)) = 1 and E(X) is ﬁnite (and hence a < E(X) < b), then
  (22)
  g(EX) ≤ Eg(X) .
  If g is strictly convex, then equality holds in (22) if and only if X = E(X) with probability 1.
  Proof.
  Let l be a supporting hyperplane to g at EX. Then
  Eg(X) ≥ El(X)
  = l(EX)
  since l is linear and P (Ω) = 1
  = g(EX) .
  Now g(X) − l(X) ≥ 0. Thus Eg(X) = El(X) if and only if g(X) = l(X) almost surely, if and only
  if X = EX almost surely. 2
  Exercise
  1.2 For any function h ∈ L2 (0, 1), deﬁne a new function T h on (0, 1) by T h(u) =
  u
  u−1 0 h(s)ds for 0 < u ≤ 1. Note that T is an averaging operator. use the Cauchy-Schwarz
  inequality to show that
  1
  0
  1
  {T h(u)}2 du ≤ 4
  h2 (u)du .
  0
  Thus T : L2 (0, 1) → L2 (0, 1) is a bounded linear operator with T ≤ 2. [Hint: write T h(u) =
  u
  u−1 0 h(s)sα s−α ds for some α.]
  Exercise 1.3 Suppose that X ∼ Binomial(n, p). Use the basic inequality proposition 1.9 with
  g(x) = exp(rx), r > 0, to show that for ≥ 1
  X/n
  (23) P
  ≥ ≤ exp(−nph())
  p
  8
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  where h() = (log() − 1) + 1. From this, show that for λ > 0 we have
  2 √
  X
  λ
  λ
  √
  (24) P
  ,
  n
  − p ≥ λ ≤ exp − ψ
  n
  2p
  p n
  where ψ(x) ≡ 2h(1 + x)/x2 is monotone decreasing on [0, ∞) with ψ(0) = 0 and ψ(x) ∼ 2 log(x)/x
  as x → ∞.
  Proof.
  (Proof of theorem 1.1). A follows easily since, for any ﬁxed > 0
  P (|Xn − X| ≥ ) ≤ P (∪m≥n [|Xm − X| ≥ ]) → 0 .
  To prove B, ﬁrst note that Xn → X implies that for every k ≥ 1 there exists an interger nk such
  that P (|Xnk − X| > 1/2k ) < 2−k ; we can
  assume nk ↑ in k; if not, take nk ≡ nk + k. Let Am ≡
  −k
  −k = 2−m+1 . On Ac = ∩
  −k
  ∪k≥m [|Xnk − X| > 2 ] so that P (Am ) ≤ ∞
  k≥m [|Xnk − X| ≤ 2 |],
  m
  k=m 2
  c
  |Xnk − X| ≤ 2−k for all k ≥ m; i.e. on Acm , Xnk (ω) → X(ω). Thus Xnk → X on A ≡ ∪∞
  m=1 Am ,
  c
  ∞
  −m+1
  and P (A ) = P (∩m=1 Am ) = limm P (Am ) ≤ limm 2
  = 0.
  Markov’s inequality gives C via P (|Xn − X| ≥ ) ≤ E|Xn − X|r /r → 0. Hölder’s inequality
  with 1/(r/r ) + 1/q = 1 gives E via
  E|Xn − X|r
  (a)
  ≤ {E|Xn − X|r (r/r ) }r /r {E1q }1/q
  = {E|Xn − X|r }r /r → 0 ;
  or, alternatively, use Liapunov’s inequality.
  Vitali’s theorem 1.2 gives D.
  Consider F. Let Xn ∼ Fn and X ∼ F . Now
  Fn (t) = P (Xn ≤ t) ≤ P (X ≤ t + ) + P (|Xn − X| ≥ )
  ≤ F (t + ) + for all n ≥ some N .
  Also
  Fn (t) = P (Xn ≤ t) ≥ P (X ≤ t − and |Xn − X| ≤ ) ≡ P (AB)
  ≥ P (A) − P (B c ) = F (t − ) − P (|Xn − X| > )
  ≥ F (t − ) − for n ≥ some N .
  Thus
  (b)
  F (t − ) − ≤ lim inf Fn (t) ≤ lim sup Fn (t) ≤ F (t + ) + .
  If t is a continuity point of F , then letting → 0 in (b) gives Fn (t) → F (t). Thus Xn →d X.
  Half of G follows from B since any Xn →p X. We turn to the other half. Assume that Xn →p X
  fails. Then there exists 0 > 0 for which δ0 ≡ lim sup P (|Xn − X| ≥ 0 ) > 0. Thus there exists
  a subsequence {n } for which P (|Xn − X| ≥ 0 ) → δ0 > 0. Thus neither Xn nor any further
  subsequence Xn can →a.s. X. This is a contradiction. Thus Xn →p X. 2
  Proof of Vitali’s theorem, theorem 1.2: postponed.
  Some Metrics on Probability Distributions
  Suppose that P and Q are two probability measures on some measurable space (or sample
  space) (X, A). Let P denote the collection of all probability distributions on (X, A).
  1.
  MODES OF CONVERGENCE
  9
  Deﬁnition 1.7 The total variation metric dT V on P is deﬁned by
  (25)
  dT V (P, Q) = sup |P (A) − Q(A)| .
  A∈A
  Deﬁnition 1.8 The Hellinger metric H on P is deﬁned by
  1
  √
  √
  2
  (26) H (P, Q) =
  | p − q|2 dµ
  2
  where p, q are densities with respect to any common dominating measure µ of P and Q (the choice
  µ = P + Q always works).
  Proposition 1.13 For P, Q ∈ P, let p and q denote densities with respect to any common dominating measure µ (µ = P + Q always works). Then
  1
  (27)
  |p − q|dµ .
  sup |P (A) − Q(A)| =
  2
  A∈A
  In other words,
  (28)
  Proof.
  1
  dT V (P, Q) =
  2
  |p − q|dµ .
  Let r ≡ p − q. Note that 0 = rdµ = r+ dµ − r− dµ, so that r+ dµ = r− dµ, and
  +
  −
  |p − q|dµ = r dµ + r dµ = 2 r+ dµ .
  Let B ≡ [p − q ≥ 0] = [r ≥ 0]. Then for any set A,
  pdµ −
  qdµ| = | (p − q)dµ|
  |P (A) − Q(A)| = |
  A
  A
  A
  = |
  (p − q)dµ +
  (p − q)dµ|
  A∩B c
  A∩B
  1
  r+ dµ ≤ r+ dµ =
  (a)
  |p − q|dµ .
  ≤
  2
  A
  On the other hand
  (b)
  (p − q)dµ| =
  |P (B) − Q(B)| = |
  B
  1
  r dµ =
  2
  +
  |p − q|dµ .
  The claimed equality follows immediately from (a) and (b).
  2
  Proposition 1.14 (Scheﬀé’s theorem). Suppose that {Pn }n≥1 , and P are probability distributions on a measurable space (X, A) with corresponding densities {pn }n≥1 , and p with respect to a
  dominating measure µ, and suppose that pn → p almost everywhere with respect to µ. Then
  (29)
  dT V (Pn , P ) → 0 .
  10
  Proof.
  (a)
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  From the proof of proposition 1.13 it follows that
  dT V (Pn , P ) = dT V (P, Pn ) = rn+ dµ
  where rn+ = (p − pn )+ satisﬁes rn+ →a.e. 0 and rn+ ≤ p for all n with
  follows from (a) and the dominated convergence theorem. 2
  pdµ = 1 < ∞. The conclusion
  Exercise 1.4 Show that the Hellinger distance H(P, Q) does not depend on the choice of a dominating measure µ.
  Exercise 1.5 Show that
  √
  2
  (30) H (P, Q) = 1 −
  pq dµ ≡ 1 − ρ(P, Q)
  where the Hellinger aﬃnity ρ(P, Q) satisﬁes ρ(P, Q) ≤ 1 with equality if and only if P = Q.
  Exercise 1.6 Show that
  (31) dT V (P, Q) = 1 − p ∧ qdµ ≡ 1 − η(P, Q)
  where the total variation aﬃnity η(P, Q) satisﬁes η(P, Q) ≤ 1 with equality if and only if P = Q.
  The Hellinger and total variation metrics are diﬀerent, but they metrize the same topoplgy on
  P, as follows from the inequalities in the following proposition.
  Proposition 1.15 (Inequalities relating Hellinger and total variation metrics).
  √
  (32) H 2 (P, Q) ≤ dT V (P, Q) ≤ H 2 (P, Q){1 + ρ(P, Q)}1/2 ≤ 2H(P, Q) .
  Exercise 1.7 Show that (32) holds.
  2.
  2
  CLASSICAL LIMIT THEOREMS
  11
  Classical Limit Theorems
  We now state some of the classical limit theorems of probability theory which are of frequent use
  in statistics.
  Proposition 2.1 (WLLN). If X, X1 , . . . , Xn , . . . are i.i.d. with mean µ (so E|X| < ∞ and µ =
  E(X), then X n →p µ.
  Proposition 2.2 (SLLN). If X1 , . . . , Xn , . . . are i.i.d with mean µ (so E|X| < ∞ and µ = E(X)),
  then X n →a.s. µ.
  Proposition 2.3 (CLT). If X1 , . . . , Xn are i.i.d. with mean µ and variance σ 2 (so E|X|2 < ∞),
  √
  then n(X n − µ) →d N (0, σ 2 ).
  Proposition 2.4 (Multivariate CLT). If X1 , . . . , Xn are i.i.d. random vectors in Rd with mean
  µ = E(X) and covariance matrix Σ = E(X − µ)(X − µ) (so E(X X) = EX2 < ∞), then
  √
  n(X − µ) →d Nd (0, Σ).
  Proposition 2.5 (Liapunov CLT). Let Xn1 , . . . , Xnn be row independentrandom variables
  n with
  n
  2
  3
  2
  2
  µni =
  E(Xni ), σni ≡ V ar(Xni ), and
  nγni ≡ E|Xni − µni | < ∞. Let µn ≡ 1 µni , σn = i=1 σni ,
  n
  3
  γn ≡ 1 γni . If γn /σn → 0, then i=1 (Xni − µni )/σn →d N (0, 1).
  row
  with 0 means and ﬁnite
  Proposition 2.6 (Lindeberg-Feller CLT).
  n Let Xni be
  nindependent
  2 ≡ V ar(X ). Let S ≡
  2 =
  2 . Then both S /σ → N (0, 1)
  variances σni
  X
  and
  σ
  σ
  ni
  n
  ni
  n
  n
  d
  n
  i=1
  i=1 ni
  2 /σ 2 : 1 ≤ i ≤ n} → 0 if and only if the Lindeberg condition
  and max{σni
  n
  (1)
  n
  1 E{|Xni |2 1[|Xni |≥σn ] } → 0
  σn2
  for all > 0
  i=1
  holds.
  Proposition 2.7 (The Cramér-Wold device). Random vectors Xn in Rd satisfy Xn →d X if and
  only if a Xn →d a X in R for all a ∈ Rd .,
  Proposition 2.8 (Continuous mapping or Mann - Wald theorem). Suppose that g : Rd → R is
  continuous a.s. PX . Then:
  A. If Xn →a.s. X then g(Xn ) →a.s. g(X).
  B. If Xn →p X then g(Xn ) →p g(X).
  C. If Xn →d X then g(Xn ) →d g(X).
  Proposition 2.9 (Slutsky’s theorem). Suppose that An →p a, Bn →p b, where a, b are constants,
  and Xn →d X. Then An Xn + Bn →d aX + b.
  Proposition 2.10 (g -theorem or the delta-method). Suppose that Zn ≡ an (Xn − b) →d Z in Rm
  where an → ∞, and suppose that g : Rm → Rk has a derivative g at b; here g is a k × m matrix.
  Then
  (2)
  an (g(Xn ) − g(b)) →d g (b)Z .
  12
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  Deﬁnition 2.1 A sequence of random variable is said to be bounded in probability, and we write
  Xn = Op (1), if
  (3)
  lim lim sup P (|Xn | ≥ M ) = 0 .
  M →∞ n→∞
  If Yn →p 0, then we write Yn = op (1). For any sequence of non-negative real numbers an we write
  Xn = Op (an ) if Xn /an = Op (1), and we write Yn = op (an ) if Yn /an = op (1).
  Proposition 2.11 If Xn →d X, then Xn = Op (1).
  Exercise 2.1 Prove proposition 2.11.
  Exercise 2.2 (a) Show that if Xn = Op (1) and Yn = op (1), then Xn Yn = op (1).
  (b) Show that if Xn = Op (an ) and Yn = Op (bn ) then Xn + Yn = Op (cn ) where cn = max{an , bn }.
  (c) Show that if Xn = Op (an ) and Yn = Op (bn ), then Xn Yn = Op (an bn ).
  Proposition 2.12 (Polya - Cantelli lemma). If Fn →d F and F is continuous, then Fn − F ∞ ≡
  sup−∞<x<∞ |Fn (x) − F (x)| → 0.
  Exercise 2.3 Suppose that ξ1 , . . . , ξn are i.i.d. Uniform(0, 1).
  (a) Show that nξn:1 = nξ(1) →d Exponential(1).
  (b) What is the joint limiting distribution of (nξn:1 , nξn:2 )?
  (c) Can you extend the result of (b) to (ξn:1 , . . . , ξn:k ) for a ﬁxed k ≥ 1?
  (d) How would you extend (c) to the situation with kn → ∞ as n → ∞?
  Exercise 2.4 Suppose that X1 , . . . , Xn are independent Exponential(λ) random variables with
  distribution function Fλ (x) = 1 − exp(−λx) for x ≥ 0.
  (a) We expect Xn:n to be on the order of bn ≡ Fλ−1 (1 − 1/n). Compute this explicitly.
  (b) Find a sequence of constants an so that an (Xn:n − bn ) →d “something” and ﬁnd “something”.
  SKOROKHOD’S THEOREM: REPLACING →D BY →A.S.
  3.
  13
  Skorokhod’s Theorem: Replacing →d by →a.s.
  3
  Our goal in this section is to show how we can convert convergence in distribution into the stronger
  mode of almost sure convergence. This often simpliﬁes proofs and makes them more intuitive.
  Deﬁnition 3.1 For any distribution function F deﬁne F −1 by F −1 (t) ≡ inf{x : F (x) ≥ t} for
  0 < t < 1.
  Proposition 3.1 F −1 is left - continuous.
  Proof.
  To show that F −1 is left - continuous, let 0 < t < 1, and set z ≡ F −1 (t). Then F (z) ≥ t
  by the right continuity of F . If F is discontinuous at z, F −1 (t − ) = z for all small > 0, and
  hence left continuity holds. If F is continuous at z, then assume F −1 is discontinuous from the left
  at t. Then for all > 0, F −1 (t − ) < z − δ for some δ > 0, and hence F (z − δ) ≥ t − for all > 0.
  hence F (z − δ) ≥ t, which implies F −1 (t) ≤ z − δ, a contradiction. 2
  Proposition 3.2 If X has continuous distribution function F , then F (X) ∼ Uniform(0, 1). For
  any distribution function F and any t ∈ (0, 1),
  P (F (X) ≤ t) ≤ t
  with equality if and only if t is in the range of F . Equivalently, F (F −1 (t)) ≡ F ◦ F −1 (t) ≥ t
  for all 0 < t < 1 with equality if and only if t is in the range of F . Also, F −1 ◦ F (x) ≤ x for
  all −∞ < x < ∞ with strict inequality if and only if F (x − ) = F (x) for some > 0. Thus
  P (F −1 ◦ F (X) = X) = 0 where X ∼ F .
  Exercise 3.1 Prove proposition 3.2.
  Theorem 3.1 (The inverse transformation). Let ξ ∼ Uniform(0, 1) and let X = F −1 (ξ). Then for
  all real x,
  [X ≤ x] = [ξ ≤ F (x)] .
  (1)
  Thus X has distribution function F .
  Proof.
  Now ξ ≤ F (x) implies X = F −1 (ξ) ≤ x by the deﬁnition 3.1 of F −1 . If X = F −1 (ξ) ≤ x,
  then F (x + ) ≥ ξ for all > 0, so that right continuity of F implies F (x) ≥ ξ. Thus the claimed
  event identity holds. 2
  Proposition 3.3 (Elementary Skorokhod theorem). Suppose that Xn →d X0 . Then there exist
  random variables Xn∗ , n ≥ 0, all deﬁned on the common probability space ([0, 1], B[0, 1], λ) for which
  d
  Xn∗ = Xn for every n ≥ 0 and Xn∗ →a.s. X0∗ .
  Proof.
  (a)
  Let Fn denote the distribution function of Xn and let
  Xn∗ ≡ Fn−1 (ξ)
  for all n ≥ 0
  14
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  d
  where ξ ∼ Uniform(0, 1). Then Xn∗ = Xn for all n ≥ 0 by theorem 3.1. It remains only to show
  that Xn∗ →a.s. X0∗ .
  Let t ∈ (0, 1) be such that there is at most one value z having F (z) = t. (Thus t corresponds to
  a continuity point of F −1 .) Let z = F −1 (t). Then F (x) < t for z < z. Thus Fn (x) < t for n ≥ Nx
  provided x < z is a continuity point of F . Thus Fn−1 (t) ≥ x provided x < z is a continuity point
  of F . Thus lim inf Fn−1 (t) ≥ x provided x < z is a continuity point of F . Thus lim inf Fn−1 (t) ≥ z
  since there are continuity points x that ↑ z.
  We also have F (x) > t for z < x. Thus Fn (x) > t, and hence Fn−1 (t) ≤ x for n ≥ some Nx
  provided x > z is a continuity point of F . Thus lim supn Fn−1 (t) ≤ x provided x > z is a continuity
  point of F . Thus lim supn Fn−1 (t) ≤ z since there are continuity points x that ↓ z.
  Thus Fn−1 (t) → F −1 (t) for all but a countable number of t’s. Since any such set has Lebesgue
  measure zero, it follows that Xn∗ = Fn−1 (ξ) →a.s. F −1 (ξ) = X0∗ . 2
  Proposition 3.4 (Continuous mapping or Mann-Wald theorem). Suppose that g : R → R is
  continuous a.s. PX . Then:
  A. If Xn →a.s. X0 , then g(Xn ) →a.s. g(X0 ).
  B. If Xn →p X0 , then g(Xn ) →p g(X0 ).
  C. If Xn →d X0 , then g(Xn ) →d g(X0 ).
  Proof.
  A. Let N1 be the null set such that Xn (ω) → X0 (ω) for all ω ∈ N1c , and let N2 be
  the null set such that g is continuous at X0 (ω) for all ω ∈ N2c . Then for ω ∈ N1c ∩ N2c we have
  g(Xn (ω)) → g(X0 (ω)). But P (N1 ∪ N2 ) ≤ P (N1 ) + P (N2 ) = 0 + 0 = 0, and the convergence
  asserted in A holds.
  B. By theorem 1.1 part G, Xn →p X0 if and only if for every subsequence {Xn } there is a further
  subsequence {Xn } ⊂ {Xn } such that Xn →a.s. X0 . We will apply this to Yn = g(Xn ). Let Yn =
  g(Xn ) be an arbitrary subsequence of {Yn }. By part G of theorem 1.1 there exists a subsequence
  {Xn } of {Xn } such that Xn →a.s. X0 . By A we conclude that Yn = g(Xn ) →a.s. g(X0 ) = Y0 .
  But by part G of theorem 1.1 (in the converse direction) it follows that Yn = g(Xn ) →p g(X0 ) = Y0 .
  C. Replace Xn , X0 by Xn∗ , X0∗ of the Skorokhod theorem, proposition 3.3. Thus
  (a)
  g(Xn ) = g(Xn∗ ) →a.s. g(X0∗ ) = g(X0 ) .
  d
  d
  Since →a.s. implies →p which in turn implies →d , (a) implies that g(Xn ) →d g(X0 ).
  2
  Remark 3.1 Proposition 3.4 remains true for random vectors in Rk and, still more generally,
  for convergence in law (weak convergence) of random elements in a separable metric space. See
  Billingsley (1986), Probability and Measure, page 399, for the ﬁrst, and Billingsley (1971), Weak
  Convergence of Measures: Applications in Probability, theorem 3.3, page 7, for the second. The
  original paper is Skorokhod (1956) where the separable metric space case was treated immediately.
  Proposition 3.5 (Helly Bray theorem). If Xn →d X0 and g is bounded and continuous (a.s. PX ),
  then Eg(Xn ) → Eg(X0 ).
  Proof.
  For the random variables Xn∗ of proposition 3.3, it follows from A of proposition 3.4
  ∗
  that g(Xn ) →a.s. g(X0∗ ). Thus by equality in distribution guaranteed by the construction of proposition 3.3 and the dominated convergence theorem,
  Eg(Xn ) = Eg(Xn∗ ) → Eg(X0∗ ) = Eg(X0 ) .
  3.
  SKOROKHOD’S THEOREM: REPLACING →D BY →A.S.
  15
  2
  Remark 3.2 If Eg(Xn ) → Eg(X) for all bounded continuous functions g, then Xn →d X0 .
  (Proof: box in the indicator function 1(−∞,x] by the bounded continuous functions g+ , g− deﬁned
  by connecting (x, 1) to (x + , 0) linearly and (x − , 1) to (x, 0) linearly, respectively.) This gives a
  way of deﬁning →d more generally:
  Deﬁnition 3.2 Suppose that Xn , n ≥ 0 take values in the complete separable metric space (M, d).
  Then we say that Xn converges in law or distribution to X0 , and we write Xn →d X0 or Xn ⇒ X0 ,
  if
  Eg(Xn ) → Eg(X0 )
  for all g ∈ Cb (M );
  here Cb (B) denotes the collection of all bounded continuous functions from M to R.
  Proposition 3.6 If Xn →d X0 , then E|X0 | ≤ lim inf n→∞ E|Xn |.
  Proof.
  For the random variables Xn∗ of proposition 3.3, Xn∗ →a.s. X0∗ . It follows from the the
  equality in distribution of proposition 3.3 and Fatou’s lemma
  E|X0 | = E|X0∗ | = E(lim inf |Xn∗ |) ≤ lim inf E|Xn∗ | = lim inf E|Xn | .
  n
  n
  n
  2
  Corollary 1 If Xn →d X0 , then V ar(X0 ) ≤ lim inf n V ar(Xn ).
  d
  Exercise 3.2 Prove corollary 1. Hint: Note that with Xn = Xn for all n ≥ 0 with Xn independent
  of Xn , we have V ar(Xn ) = (1/2)E(Xn − Xn )2 .
  Proposition 3.7 (Slutsky’s theorem). If An →p a, Bn →p b, and Zn →d Z, then An Zn + Bn →d
  aZ + b.
  Proof.
  Now An →p a, Bn →p b, and Zn →d Z where a, b are constants, implies that
  (Zn , An , Bn ) →d (Z, a, b) in R3 . Hence by the R3 version of Skorokhod’s theorem, there exists a
  d
  d
  sequence (Zn∗ , A∗n , Bn∗ ) = (Zn , An , Bn ) such that (Zn∗ , A∗n , Bn∗ ) → (Z ∗ , a, b) = (Z, a, b). Hence
  (a)
  An Zn + Bn = A∗n Zn∗ + Bn∗ →a.s. aZ ∗ + b = aZ + b .
  d
  d
  Since →a.s. implies →p which in turn implies →d , (a) yields the desired conclusion.
  2
  16
  4
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  Empirical Measures and Empirical Processes
  We ﬁrst introduce the empirical distribution function Gn and empirical process Un of i.i.d. Uniform(0, 1)
  random variables. Suppose that ξ1 , . . . , ξn , . . . are i.i.d. Uniform(0, 1). Their empirical distribution
  function is
  1
  1[0,t] (ξi )
  n
  n
  (1)
  Gn (t) =
  for 0 ≤ t ≤ 1
  i=1
  =
  =
  #{ξi ≤ t, i = 1, . . . , n}
  n
  k
  for ξn:k ≤ t < ξn:k+1 , k = 0, . . . , n
  n
  where 0 ≡ ξn:0 ≤ ξn:1 ≤ · · · ≤ ξn:n ≤ ξn:n+1 ≡ 1 are the order statistics. The uniform empirical
  process is deﬁned by
  √
  (2)
  for 0 ≤ t ≤ 1 .
  Un (t) ≡ n(Gn (t) − t)
  The inverse function G−1
  n of Gn is the uniform quantile function. Thus
  (3)
  G−1
  n (t) = ξn:i
  for (i − 1)/n < t ≤ i/n, i = 1, . . . , n .
  The uniform quantile process Vn is deﬁned by
  √
  (4)
  for 0 ≤ t ≤ 1 .
  Vn (t) ≡ n(G−1
  n (t) − t)
  Note that
  (5)
  nGn (t) ∼ Binomial(n, t)
  for 0 ≤ t ≤ 1 ,
  so that
  (6)
  Un (t)
  has mean 0
  and variance t(1 − t)
  for 0 ≤ t ≤ 1 .
  In fact
  (7)
  Cov[1[0,s] (ξi ), 1[0,t] (ξi )] = s ∧ t − st
  for 0 ≤ s, t ≤ 1 .
  Moreover, applying the multivariate CLT to (1[0,s] (ξi ), 1[0,t] (ξi )), it is clear that
  0
  s(1 − s) s ∧ t − st
  (Un (s), Un (t)) →d N2
  (8)
  ,
  as n → ∞ ,
  0
  s ∧ t − st t(1 − t)
  for 0 ≤ s, t ≤ 1.
  We deﬁne {U(t) : 0 ≤ t ≤ 1} to be a Brownian bridge process if it is a Gaussian process on
  indexed by t ∈ [0, 1] having
  (9)
  EU(t) = 0
  and
  Cov[U(s), U(t)] = s ∧ t − st
  for all 0 ≤ s, t ≤ 1. The process U exists and has sample functions U(·, ω) which are continuous
  for a.e. ω as we will show below. Of course the bivariate result (8) immediately extends to all the
  ﬁnite-dimensional marginal distributions of Un : for any k ≥ 1 and any t1 , . . . , tk ∈ [0, 1],
  (10)
  (Un (t1 ), . . . , Un (tk )) →d (U(t1 ), . . . , U(tk )) ∼ Nk (0, (tj ∧ tj − tj tj )) .
  4.
  EMPIRICAL MEASURES AND EMPIRICAL PROCESSES
  17
  Thus we have convergence of all the ﬁnite-dimensional distributions of Un to those of a Brownian
  bridge process U, and we write
  (11)
  Un →f.d. U
  as n → ∞ .
  We would like to be able to conclude from (11) that g(Un ) → g(U) as n → ∞ for various continuous
  functionals such as g(x) = sup0≤t≤1 |x(t)| for x ∈ D[0, 1], the space of all right-continuous functions
  on [0, 1] with left-limits. The conclusion (11) is not strong enough to imply this, but (10) can be
  strengthened to a result that does. The Mann-Wald theorem suggests g(Un ) →d g(U) should be
  true for “continuous” functionals g, and this raises the question of what metric should be used to
  deﬁne continuous.
  The Empirical Process on R
  Let X1 , . . . , Xn , . . . be i.i.d. F with order statistics Xn:1 ≤ · · · ≤ Xn:n . Their empirical distribution function Fn is deﬁned by
  1
  1(−∞,x] (Xi )
  n
  n
  (12)
  Fn (x) =
  for − ∞ < x < ∞ .
  i=1
  √
  The empirical process is deﬁned to be n(Fn − F ). It will be very useful to suppose that random
  variables Xi∗ , i = 1, . . . , n, are deﬁned by
  (13)
  Xi∗ = F −1 (ξi )
  i = 1, . . . , n
  for the ξi ’s of (1) .
  theorem 3.1 shows that these Xi∗ ’s are indeed i.i.d. F . Recall from theorem 3.1 that also
  (14)
  1[Xi∗ ≤x] = 1[ξi ≤F (x)]
  on (−∞, ∞)
  a.s.
  for these particular Xi∗ ’s. Thus for these Xi∗ ’s we have
  (15)
  F∗n = Gn (F )
  on (−∞, ∞) a.s.
  and
  (16)
  √
  n(F∗n − F ) = Un (F )
  on (−∞, ∞) a.s..
  Note from (10) and (16) that
  √
  (17)
  n(Fn − F ) →f.d. U(F )
  as n → ∞ .
  Theorem 4.1 (Glivenko - Cantelli). Let I denote the identity function on [0, 1], I(t) = t, for
  0 ≤ t ≤ 1. Then
  (18)
  Gn − I∞ ≡ sup |Gn (t) − t| →a.s. 0
  0≤t≤1
  and
  (19)
  Fn − F ∞ ≡
  as n → ∞.
  sup
  −∞<x<∞
  |Fn (x) − F (x)| →a.s. 0
  18
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  Proof.
  Since
  Fn − F = F∗n − F ∞ = Gn (F ) − F ∞ ≤ Gn − I∞ ,
  d
  (a)
  where the equality in distribution holds jointly in n and with equality if F is continuous, it suﬃces
  to prove the ﬁrst part.
  Fix a large integer M . Then
  Gn − I∞
  =
  =
  ≤
  ≤
  max
  sup
  1≤j≤M (j−1)/M ≤t≤j/M
  max {
  sup
  |Gn (t) − t|
  1≤j≤M (j−1)/M ≤t≤j/M
  (Gn (t) − t) ∨
  sup
  (j−1)/M ≤t≤j/M
  (t − Gn (t))}
  max {(Gn (j/M ) − (j − 1)/M ) ∨ (j/M − Gn ((j − 1)/M ))}
  1≤j≤M
  max {Gn (j/M ) − j/M ) ∨ ((j − 1)/M − Gn ((j − 1)/M ))} + 1/M
  1≤j≤M
  →a.s. 0 + 1/M
  since Gn (j/M ) →a.s. j/M , j = 1, . . . , M . But M was arbitrary; hence Gn − I∞ →a.s. 0.
  2
  The next natural step is to show that
  (20)
  and
  (21)
  Un ⇒ U
  √
  as n → ∞
  d
  n(Fn − F ) =
  √
  in (D[0, 1], · ∞ )
  n(F∗n − F ) = Un (F ) ⇒ U(F )
  as n → ∞
  in (D(−∞, ∞), · ∞ ) .
  This is essentially what was proved by Donsker (1952). However, it turned out later that
  there are measurability diﬃculties here: (D[0, 1], · ∞ ) is an inseparable Banach space, and even
  though U takes values in the separable Banach space (C[0, 1], · ∞ ), in this case the unfortunate
  consequence is that Un is not a measurable element of (D[0, 1], ·∞ ); see Billingsley (1968), Chapter
  xx. Roughly, the Borel sigma-ﬁeld is too big. Hence an attractive alternative formulation is one
  that works around this diﬃculty essentially by carrying out an explicit Skorokhod construction
  d
  of uniform empirical processes U∗n = Un deﬁned on a common probability space with a Brownian
  bridge process U∗ and satisfying
  (22)
  U∗n − U∗ ∞ = sup |U∗n (t) − U∗ (t)| →a.s. 0 .
  0≤t≤1
  This is the content of the following theorem:
  Theorem 4.2 There exists a (sequence of) Brownian bridge processes U∗n corresponding to a
  triangular array of row independent Uniform(0, 1) random variables ξn1 , . . . , ξnn , n ≥ 1, and a
  Brownian bridge process U∗ all deﬁned on a common probability space (Ω, A, P ), such (22) holds.
  Thus it follows that
  √
  (23) n(F∗n − F ) − U∗ (F )∞ →a.s. 0
  as n → ∞.
  The convergence in (22) was strengthened in the papers of Komlós, Major, and Tusnády (1975),
  (1978) as follows: the construction can be carried out so that the convergence in (22) holds with
  the rate n−1/2 log n: there is a construction of U∗n and U∗ so that
  (24)
  log n
  U∗n − U∗ ∞ ≤ C √
  n
  a.s. ,
  4.
  EMPIRICAL MEASURES AND EMPIRICAL PROCESSES
  19
  for some absolute constant C. Moreover, there is a construction of the sequence(s) {U∗n }n≥1 and
  U∗ = U∗n on a common probability space so that the joint in n distributions are correct and
  (25)
  U∗n − U∗ ∞ ≤ C
  (log n)2
  √
  n
  a.s. .
  In any case, these results have the following corollary:
  Corollary 1 (Donsker’s theorem). If g : D[0, 1] → R is · ∞ −continuous, then g(Un ) → g(U).
  Here are some examples of this:
  Example 4.1 (Kolmogorov’s (two-sided) statistic). If F is continuous, then
  (26)
  √
  d
  n(Fn − F )∞ = Un (F )∞ = Un ∞ →d U ∞ .
  It is known, via reﬂection methods (see Shorack and Wellner (1986), pages 33-42) that
  (27)
  P (U∞ > λ) = 2
  ∞
  (−1)k+1 exp(−2k 2 λ2 )
  for λ > 0 .
  k=1
  Example 4.2 (Kolmogorov’s one-sided statistic). If F is continuous, then
  (28)
  √
  n(Fn − F )+ ∞ ≡
  √
  sup
  −∞<x<∞
  d
  +
  +
  n(Fn (x) − F (x)) = U+
  n (F )∞ = Un ∞ →d U ∞ .
  It is known (see Shorack and Wellner (1986), pages 37 and 142) that
  (29)
  P (U+ ∞ > λ) = exp(−2λ2 )
  for λ > 0 .
  Example 4.3 (Birnbaum’s statistic). If F is continuous,
  (30)
  ∞
  √
  −∞
  d
  n(Fn (x) − F (x))dF (x) =
  ∞
  −∞
  Un (F )dF =
  0
  1
  Un (t)dt →d
  1
  U(t)dt .
  0
  1
  Now 0 U(t)dt is a linear combination of normal random variables, and hence it has a normal distribution. It has expectation 0 by Fubini’s theorem since E(U(t)) = 0 for each ﬁxed t. Furthermore,
  again by Fubini’s theorem,
  E
  0
  Hence
  1
  0
  1
  2
  U(t)dt
  1
  1
  = E
  U(s)ds
  U(t)dt
  0
  0
  1 1
  =
  E{U(s)U(t)}dsdt
  0
  0
  1 1
  1
  (s ∧ t − st) dsdt = .
  =
  12
  0
  0
  U(t)dt ∼ N (0, 1/12).
  20
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  Example 4.4 (Cramér - von Mises statistic). If F is continuous,
  ∞
  1
  1
  ∞
  √
  d
  2
  2
  2
  { n(Fn (x) − F (x)) dF (x) =
  {Un (F )} dF =
  {Un (t)} dt →d
  {U(t)}2 dt .
  −∞
  −∞
  0
  0
  In this case it is known that
  1
  ∞
  1
  d
  (31)
  {U(t)}2 dt =
  Z2 ,
  2π2 j
  j
  0
  j=1
  where the Zj ’s are i.i.d. N (0, 1), and this distribution has been tabled; see Shorack and Wellner
  (1986), page 148.
  Example 4.5 (Anderson - Darling statistic). If F is continuous,
  ∞ √
  ∞
  1
  1
  { n(Fn − F )}2
  {Un (F )}2
  {Un (t)}2
  {U(t)}2
  d
  dF =
  dF =
  dt →d
  dt .
  F (1 − F )
  −∞
  ∞ F (1 − F )
  0 t(1 − t)
  0 t(1 − t)
  It is known in this case that
  1
  ∞
  {U(t)}2
  1
  d
  (32)
  dt =
  Z2
  j(j + 1) j
  0 t(1 − t)
  j=1
  where the Zj ’s are i.i.d. N (0, 1). This distribution has also been tabled; see Shorack and Wellner
  (1986), page 148.
  General Empirical Measures and Processes
  Now suppose that X1 , X2 , . . . , Xn , . . . are i.i.d. P on the measurable space (S, S). We let Pn
  denote the empirical measure of the ﬁrst n of the Xi ’s:
  1
  δXi ;
  n
  n
  (33)
  Pn ≡
  i=1
  here δx denotes the measure with mass 1 at x ∈ S: δx (B) = 1B (x) for B ∈ S. Thus for a set B ∈ S,
  1
  1
  1B (Xi ) = #{i ≤ n : Xi ∈ B} .
  n
  n
  n
  (34)
  Pn (B) =
  i=1
  Note that when S = R so that the Xi ’s are real-valued, and B = (−∞, x] for x ∈ R, then
  (35)
  Pn (B) = Pn ((−∞, x]) = Fn (x) ,
  the empirical distribution function of the Xi ’s at x.
  The empirical process Gn is deﬁned by
  √
  (36) Gn ≡ n(Pn − P ) .
  The question is how to “index” Pn and Gn as stochastic processes.
  Some history: For the case S = Rd , the empirical distribution function {Fn (x), x ∈ Rd }, is the
  case obtained by choosing the class of sets to be the lower-left orthants
  C = Od ≡ {(−∞, x] : x ∈ Rd } ,
  4.
  EMPIRICAL MEASURES AND EMPIRICAL PROCESSES
  21
  and this direction was pursued in some detail through the 1950’s and early 1960’s. However, with a
  little thought it becomes clear that many other classes of sets will be of interest in Rd . For example
  why not consider the empirical process indexed by the class of all rectangles
  Rd ≡ {A = [a1 , b1 ] × · · · × [ad , bd ] : aj , bj ∈ R, j = 1, . . . , d}
  or the class of all closed balls
  Bd ≡ {B(x, r) : x ∈ Rd , r > 0}
  where B(x, r) = {y ∈ Rd : |y − x| ≤ r}; or the class of all half-spaces
  Hd ≡ {H(u, t) : u ∈ S d−1 , t ∈ R}
  where H(u, t) ≡ {y ∈ Rd : y, u ≤ t} and S d−1 ≡ {u ∈ Rd : |u| = 1} denotes the unit sphere in
  Rd ; or the class of all convex sets in Rd
  Cd ≡ {C ⊂ Rd : C is convex} ?
  All of these cases correspond to the empirical process indexed by some class of indicator functions
  {1C : C ∈ C}
  for the appropriate choice of C. Thus we can consider the empirical measure and the empirical
  process as functions on a class of sets C which map sets C ∈ C to the real-valued random variables
  √
  and
  Gn (C) = n(Pn (C) − P (C)) .
  Pn (C)
  Note that for any class of sets C we have
  sup Pn (C) ≤ 1 < ∞
  C∈C
  and
  √
  √
  sup | n(Pn (C) − P (C))| ≤ n < ∞ ,
  C∈C
  so we can regard both Pn and Gn as elements of the space l∞ (C) ≡ {x : C → R| supC∈C |x(C)| < ∞}.
  More generally still, we can think of indexing the empirical process by a class F of functions
  f : S → R. For example, when S = Rd a natural class which might easily arise in applications is
  the class of functions
  F = {ft (x) : t ∈ Rd }
  where ft (x) = |x − t|. This is already an interesting class of functions when d = 1.
  For any ﬁxed measureable function f : S → we will use the notation
  P (f ) =
  f dP,
  Pn (f ) =
  1
  f (Xi ) .
  n
  n
  f dPn =
  i=1
  From the strong law of large numbers it follows that for any ﬁxed function f with E|f (X)| < ∞
  (37)
  Pn (f ) →a.s. P (f ) = Ef (X1 ) .
  By the central limit theorem (CLT) it follows that for any ﬁxed function f with E|f (X)|2 < ∞
  √
  (38) Gn (f ) = n(Pn (f ) − P (f )) →d G(f ) ; ∼ N (0, V ar(f (X1 ))
  22
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  here G denotes a P −Brownian bridge process: i.e. a mean zero Gaussian process with covariance
  function
  (39)
  Cov[G(f ), G(g)] = P (f g) − P (f )P (g) .
  The question of interest is: for what classes C of subsets of S, C ⊂ A or classes of functions F, can
  we make these convergences hold uniformly in C ∈ C, or uniformly in f ∈ F? These are the kinds
  of questions with which modern empirical process theory is concerned and can answer.
  To state some typical results from this theory, we ﬁrst need several deﬁnitions. If d is a metric
  on a set F, then we deﬁne the covering numbers of F with respect to d as follows:
  (40)
  N (, F, d) ≡ inf{k : there exist f1 , . . . , fk ∈ F such that F ⊂ ∪kj=1 B(fj , )} ;
  here B(f, ) ≡ {g ∈ F : d(g, f ) ≤ }. Another useful notion is that of a bracket: if l ≤ u are two
  real-valued functions deﬁned on S, then the bracket [u, l] is deﬁned by
  (41)
  [u, l] ≡ {f : l(s) ≤ f (s) ≤ u(s) for all s ∈ S} .
  We say that a bracket [u, l] is an −bracket for the metric d if d(u, l) ≤ . Then the bracketing
  covering number N[] (, F, d) for a set of functions F is
  (42)
  N[] (, F, d) ≡ inf{k : there exist −brackets [l1 , u1 ], . . . , [lk , uk ] such that F ⊂ ∪kj=1 [lj , uj ]} .
  One more bit of notation is needed before stating our theorems: an envelope function F for a
  class of functions F is any function satisfying
  (43)
  |f (x)| ≤ F (x)
  for all x ∈ S, and all f ∈ F .
  Usually we will take F to be the minimal measurable majorant
  ∗
  (44)
  F (x) ≡
  sup |f (x)|
  f ∈F
  ,
  where here the ∗ stands for “smallest measurable function above” the quantity in parentheses
  (which need not be measurable since it is, in general, a supremum over an uncountable collection).
  [Note that this F is not a distribution function!]
  Now we can state two generalizations of the Glivenko-Cantelli theorems.
  Theorem 4.3 Suppose that F is a class of functions with ﬁnite L1 (P )−bracketing numbers:
  N[] (, F, L1 (P )) < ∞ for every > 0. Then
  (45)
  Pn − P F ≡ sup |Pn (f ) − P (f )| →a.s. 0 .
  f ∈F
  Theorem 4.4 Suppose that F is a class of functions with:
  A. An integrable envelope function F : P (F ) < ∞.
  B. The truncated classes FM ≡ {f 1[F ≤M ] : f ∈ F} satisfy
  (46)
  n−1 log N (, FM , L1 (Pn )) →a.s. 0
  for every > 0 and 0 < M < ∞. Then, if F is also “suitably measurable”,
  (47)
  Pn − P F ≡ sup |Pn (f ) − P (f )| →a.s. 0 .
  f ∈F
  4.
  EMPIRICAL MEASURES AND EMPIRICAL PROCESSES
  23
  Note that the key hypothesis (46) of Theorem 4.4 is clearly satisﬁed if
  (48)
  sup N (, FM , L1 (Q)) < ∞
  Q
  for every > 0 and M > 0; here the supremum is over ﬁnitely discrete measures Q.
  Finally, here are two generalizations of the Donsker theorem.
  Theorem 4.5 (Ossiander’s uniform CLT). Suppose that F is a class of functions with L2 (P )
  bracketing numbers N[] (, F, L2 (P )) satisfying
  ∞
  (49)
  log N[] (, F, L2 (P )) d < ∞ .
  0
  Then
  (50)
  Gn =
  √
  n(Pn − P ) ⇒ G
  in l∞ (F)
  as n → ∞ .
  Theorem 4.6 (Pollard’s uniform CLT). Suppose that F is a class of functions satisfying:
  A. The envelope function F of F is square integrable: P (F 2 ) < ∞.
  B. The uniform covering numbers supQ log N (F Q,2 , F, L2 (Q)) satisfy
  ∞
  (51)
  sup log N (F Q,2 , F, L2 (Q)) d < ∞ .
  0
  Q
  Then
  (52)
  Gn =
  √
  n(Pn − P ) ⇒ G
  in l∞ (F)
  as n → ∞ .
  For proofs of Theorems 45 - 4.6, see van der Vaart and Wellner (1996). Treatments of empirical
  process theory are also given by Dudley (1999) and Van de Geer (1999).
  24
  5
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  The Partial Sum Process and Brownian Motion
  We deﬁne {S(t) : 0 ≤ t ≤ 1} to be Brownian motion if S is a Gaussian process indexed by t ∈ [0, 1]
  having
  (1)
  E(S(t)) = 0
  Cov[S(s), S(t)] = s ∧ t
  and
  for all 0 ≤ s, t ≤ 1. These ﬁnite-dimensional distributions are “consistent”, and hence a theorem
  of Kolmogorov shows that the process S exists. Note that (1) and normality imply that
  (2)
  S
  has stationary independent increments .
  Exercise 5.1 Suppose that U is a Brownian bridge and Z ∼ N (0, 1) is is independent of U. Let
  (3)
  S(t) ≡ U(t) + tZ
  for 0 ≤ t ≤ 1
  is a Brownian motion.
  Exercise 5.2 Suppose that S is a Brownian motion. Show that
  (4)
  U(t) ≡ S(t) − tS(1)
  for 0 ≤ t ≤ 1
  is a Brownian bridge.
  Now suppose that X1 , . . . , Xn are i.i.d. random variables with mean 0 and and variance 1, and
  set X0 ≡ 0. We deﬁne the partial sum process Sn by
  1 Xi
  Sn (t) ≡ Sn (k/n) = √
  n
  k
  (5)
  k
  k+1
  ≤t<
  ,
  n
  n
  for
  i=1
  for 0 ≤ k < ∞. Note that
  1 Cov[Xi , Xi ]
  n
  j
  Cov[Sn (j/n), Sn (k/n)]
  =
  k
  i=1 i =1
  =
  1
  n
  j∧k
  V ar[Xi ] =
  i=1
  → s∧t
  j∧k
  n
  if j/n → s and k/n → t .
  Also,
  1 Sn (t) = √
  Xi =
  n
  [nt]
  (6)
  i=1
  [nt]
  √
  [nt] 1 Xi →d tN (0, 1) ∼ N (0, t) .
  n
  [nt]
  i=1
  for 0 ≤ t ≤ 1 by the CLT and Slutsky’s theorem. This suggests that
  (7)
  Sn →f.d. S
  as n → ∞ .
  This will be veriﬁed in exercise 5.3. Much more is true: Sn ⇒ S as processes in D[0, 1], and hence
  g(Sn ) →d g(S) for continuous functionals g.
  5.
  THE PARTIAL SUM PROCESS AND BROWNIAN MOTION
  25
  Exercise 5.3 Show that (7) holds: i.e. for any ﬁxed t1 , . . . , tk ∈ [0, 1]k ,
  (Sn (t1 ), . . . , Sn (tk )) →d (S(t1 ), . . . , S(tk )) .
  Existence of Brownian motion and Brownian bridge as continuous processes on C[0, 1]
  The aim of this subsection to convince you that both Brownian motion and Brownian bridge
  exist as continuous Gaussian processes on [0, 1], and that we can then extend the deﬁnition of
  Brownian motion to [0, ∞).
  Deﬁnition 5.1 Brownian motion (or standard Brownian motion, or a Wiener process) S is a
  Gaussian process with continuous sample functions and:
  (i) S(0) = 0;
  (ii) E(S(t)) = 0, 0 ≤ t ≤ 1;
  (iii) E{S(s)S(t)} = s ∧ t, 0 ≤ s, t ≤ 1.
  Deﬁnition 5.2 A Brownian bridge process U is a Gaussian process with continuous sample functions and:
  (i) U(0) = U(1) = 0;
  (ii) E(U(t)) = 0, 0 ≤ t ≤ 1;
  (iii) E{U(s)U(t)} = s ∧ t − st, 0 ≤ s, t ≤ 1.
  Theorem 5.1 Brownian motion S and Brownian bridge U exist.
  Proof.
  (a)
  We ﬁrst construct a
  
   t
  1−t
  h00 (t) ≡ h(t) ≡
  
  0
  Brownian bridge process U. Let
  0 ≤ t ≤ 1/2 ,
  1/2 ≤ t ≤ 1 ,
  elsewhere .
  For n ≥ 1 let
  (b)
  hnj (t) ≡ 2−n/2 h(2n t − j),
  j = 0, . . . , 2n − 1 .
  For example, h10 (t) = 2−1/2 h(2t), h11 (t) = 2−1/2 h(2t − 1), while
  h20 (t) = 2−1 h(4t),
  h21 (t) = 2−1 h(4t − 1) ,
  h22 (t) = 2−1 h(4t − 2),
  h23 (t) = 2−1 h(4t − 3) .
  Note that |hnj (t)| ≤ 2−n/2 2−1 .
  The functions {hnj : j = 0, . . . , 2n − 1, n ≥ 0} are called the Schauder functions; they
  are integrals of the orthonormal (with respect to Lebesgue measure on [0, 1]) family of functions
  {gnj : j = 0, . . . , 2n − 1, n ≥ 0} called the Haar functions deﬁned by
  g00 (t) ≡ g(t) ≡ 21[0,1/2] (t) − 1,
  gnj (t) ≡ 2n/2 g00 (2n t − j) ,
  Thus
  (c)
  0
  1
  2
  gnj
  (t)dt = 1,
  j = 0, . . . , 2n − 1, n ≥ 1 .
  1
  gnj (t)gn j (t)dt = 0
  0
  if
  n = n , or j = j ,
  26
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  and
  (d)
  t
  hnj (t) =
  0 ≤ t ≤ 1.
  gnj (s)ds,
  0
  −1
  Furthermore, the family {gnj }2j=0,
  n≥0 ∪ {g(·/2)} is complete: any f ∈ L2 (0, 1) has an expansion in
  terms of the g’s. In fact the Haar basis is the simplest wavelet basis of L2 (0, 1), and is the starting
  point for further developments in the area of wavelets.
  n −1
  Now let {Znj }2j=0,
  n≥0 be independent identically distributed N (0, 1) random variables; if we
  wanted, we could construct all these random variables on the probability space ([0, 1], B[0,1] , λ).
  Deﬁne
  n
  Vn (t, ω) =
  n −1
  2
  Um (t, ω) =
  j=0
  m
  Znj (ω)hnj (t) ,
  Vn (t, ω) .
  n=0
  For m > k
  (e)
  m
  |Um (t, ω) − Uk (t, ω)| = |
  Vn (t, ω)| ≤
  n=k+1
  m
  |Vn (t, ω)|
  n=k+1
  where
  (f)
  |Vn (t, ω)| ≤
  n −1
  2
  |Znj (ω)||hnj (t)| ≤ 2−(n/2+1)
  j=0
  max
  0≤j≤2n −1
  |Znj (ω)|
  since the hnj , j = 0, . . . , 2n − 1 are = 0 on disjoint t intervals.
  Now P (Znj > z) = 1 − Φ(z) ≤ z −1 φ(z) for z > 0 (by “Mill’s ratio”) so that
  (g)
  √
  √
  √
  2
  P (|Znj | ≥ 2 n) = 2P ((Znj ≥ 2 n) ≤ √ (2 n)−1 e−2n .
  2π
  Hence
  (h)
  P
  maxn
  0≤j≤2 −1
  √
  |Znj | ≥ 2 n
  √
  2n
  ≤ 2n P (|Z00 | ≥ 2 n) ≤ √ n−1/2 e−2n ;
  2π
  √
  since this is a term of a convergent series, by the Borel-Cantelli lemma max0≤j≤2n −1 |Znj | ≥ 2 n
  occurs inﬁnitely often with probability zero; i.e. except on a null set, for all ω there is an N = N (ω)
  √
  such that max0≤j≤2n −1 |Xnj (ω)| < 2 n for all n > N (ω). Hence
  (i)
  sup |Um (t) − Uk (t)| ≤
  0≤t≤1
  m
  2−n/2 n1/2 ↓ 0
  n=k+1
  for all k, m ≥ N ≥ N (ω). Thus Um (t, ω) converges uniformly as m → ∞ with probability one to
  the (necessarily continuous) function
  (j)
  U(t, ω) ≡
  ∞
  n=0
  Vn (t, ω) .
  5.
  THE PARTIAL SUM PROCESS AND BROWNIAN MOTION
  27
  Deﬁne U ≡ 0 on the exceptional set. Then U is continuous for all ω.
  Now {U(t) : 0 ≤ t ≤ 1} is clearly a Gaussian process since it is the sum of Gaussian processes.
  We now show that U is in fact a Brownian bridge: by formal calculation (it remains only to justify
  the interchange of summation and expectation),
  ∞
  ∞
  Vn (s)
  Vm (t)
  E{U(s)U(t)} = E
  n=0
  =
  ∞
  E{Vn (s)Vn (t)}
  n=0
  =
  ∞
  m=0
  E
  n=0
  
  n −1
  2
  
  ∞ 2
  −1 n=0 j=0
  n
  =
  n=0 j=0
  1
  gnj dλ
  0
  n −1
  2
  k=0
  s
  gnj dλ
  0
  ∞ 2
  −1 s
  Znj
  j=0
  n
  =
  Znk
  
  
  t
  gnk dλ
  0
  
  t
  gnk dλ
  0
  1
  1
  1[0,s] gnj dλ
  0
  1[0,t] gnk dλ + st − st
  0
  1[0,s] (u)1[0,t] (u)du − st
  =
  0
  = s ∧ t − st
  where the next to last equality follows from Parseval’s identity. Thus U is Brownian bridge.
  Now let Z be one additional N (0, 1) random variable independent of all the others used in the
  construction, and deﬁne
  (k)
  S(t) ≡ U(t) + tZ =
  ∞
  Vn (t) + tZ .
  n=0
  Then S is also Gaussian with 0 mean and
  Cov[S(s), S(t)] = Cov[U(s) + sZ, U(t) + tZ]
  = Cov[U(s), U(t)] + stV ar(Z)
  = s ∧ t − st + st = s ∧ t .
  Thus S is Brownian motion. Since U has continuous sample paths, so does S.
  2
  Exercise 5.4 Graph the ﬁrst few gnj ’s and hnj ’s.
  Exercise 5.5 Justify the interchange of expectation and summation used in the proof. [Hint: use
  the Tonelli part of Fubini’s theorem.]
  Exercise 5.6 Let U be a Brownian bridge process. For 0 ≤ t < ∞ deﬁne a process B by
  t
  (8)
  .
  B(t) ≡ (1 + t)U
  1+t
  Show that B is a Brownian motion process on [0, ∞).
  28
  6
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  Quantiles and Quantile Processes
  Let X1 , . . . , Xn be i.i.d. real-valued random variables with distribution function F , and let X(1) ≤
  X(2) ≤ . . . ≤ X(n) denote the order statistics. For t ∈ (0, 1), let
  (1)
  F−1
  n (t) ≡ inf{Fn (x) ≥ t}
  so that
  (2)
  F−1
  n (t) = X(i)
  i−1
  i
  < t ≤ , i = 1, . . . , n .
  n
  n
  for
  Let ξ1 , . . . , ξn be i.i.d. Uniform(0, 1) random variables, and let 0 ≤ ξ(1) ≤ . . . ≤ ξ(n) ≤ 1 denote
  their order statistics. Thus, with G−1
  n (t) ≡ inf{x : Gn (x) ≥ t},
  (3)
  G−1
  n (t) = ξ(i)
  for
  i−1
  i
  < t ≤ , i = 1, . . . , n .
  n
  n
  Now
  (4)
  (X1∗ , . . . , Xn∗ ) ≡ (F −1 (ξ1 ), . . . , F −1 (ξn )) = (X1 , . . . , Xn ),
  d
  so
  (5)
  ∗
  ∗
  (X(1)
  , . . . , X(n)
  ) ≡ (F −1 (ξ(1) ), . . . , F −1 (ξ(n) )) = (X(1) , . . . , X(n) ),
  d
  Hence it follows that
  (6)
  −1
  (G−1
  F−1
  n (·) = F
  n ( · )) ,
  d
  −1
  and to study F−1
  n it suﬃces to study Gn .
  Proposition 6.1 The sequence of uniform quantile functions G−1
  n satisfy
  (7)
  −1
  G−1
  n − I∞ ≡ sup |Gn (t) − t| = Gn − I∞ →a.s. 0 ,
  0≤t≤1
  and hence, if F −1 is continuous on [a, b] ⊂ [0, 1], then
  (8)
  −1 b
  −1
  a ≡ sup |F−1
  (t)| →a.s. 0 .
  F−1
  n −F
  n (t) − F
  a≤t≤b
  Proof.
  (a)
  Note that Gn −1 − I∞ = Gn − I∞ by inspection of the graphs. Thus
  −1 b
  −1
  a = F −1 (G−1
  (I)ba →a.s. 0
  F−1
  n −F
  n )−F
  d
  since F −1 is uniformly continuous on [a, b] and G−1
  n − I∞ →a.s. 0.
  Deﬁnition 6.1 The uniform quantile process Vn is deﬁned by
  √
  Vn ≡ n(G−1
  (9)
  n − I) .
  The general quantile process is deﬁned by
  √
  −1 d √
  −1
  (10)
  n(F−1
  ) = n(F −1 (G−1
  ).
  n −F
  n )−F
  2
  6.
  QUANTILES AND QUANTILE PROCESSES
  29
  Theorem 6.1 The uniform quantile process can be written as
  (11)
  Vn = −Un (G−1
  n )+
  √
  n(Gn ◦ G−1
  n − I) ,
  d
  and hence for the specially constructed Un of theorem 4.2 it follows that, with V ≡ −U = U,
  (12)
  Vn − V∞ →a.s. 0
  n → ∞.
  Proof.
  First we prove the identity (11):
  √
  Vn =
  n(G−1
  n − I)
  √
  √
  −1
  −1
  n(G−1
  =
  n − Gn (Gn )) + n(Gn (Gn ) − I)
  √
  −1
  = −Un (G−1
  n ) + n(Gn ◦ Gn − I) .
  as
  Now G−1
  n − I∞ →a.s. 0 by proposition 6.1, and
  (a)
  −1
  Gn ◦ G−1
  n − I∞ = sup |Gn (Gn (t)) − t| =
  0≤t≤1
  1
  .
  n
  Hence
  Vn − V∞
  ≤
  Un (G−1
  n ) − U∞
  ≤
  1
  −1
  −1
  Un (G−1
  n ) − U(Gn )∞ + U(Gn ) − U∞ + √
  n
  1
  Un − U∞ + U(G−1
  n ) − U∞ + √
  n
  0 + 0 + 0 = 0.
  ≤
  →a.s.
  since U is a continuous (and hence uniformly continuous) function on [0, 1].
  2
  Theorem 6.2 Let Q = F −1 , and suppose that Q is diﬀerentiable at 0 < t1 < · · · < tk < 1. Then
  (13)
   √
  
  
  
  
  
  
  
  
  
   →d
  
  
  
  
  
  
  
  −1 (t ))
  n(F−1
  k
  n (t1 ) − F
  ·
  ·
  ·
  √
  −1
  n(Fn (tk ) − F −1 (tk ))
  Q (t1 )V(t1 )
  ·
  ·
  ·
  Q (tk )V(tk )
  
  
  
   ∼ Nk (0, Σ)
  
  
  where
  (14)
  Σ ≡ (σij ) = Q (ti )Q (tj )(ti ∧ tj − ti tj ) .
  Moreover, if Q is nonzero and continuous on [a, b] ⊂ [0, 1], then for any [c, d] ⊂ [a, b]
  (15)
  √
  −1
  ) − Q Vdc →a.s. 0
  n(F −1 (G−1
  n )−F
  Note that Q (t) = 1/f (F −1 (t)).
  as n → ∞ .
  30
  Proof.
  CHAPTER 2. SOME BASIC LARGE SAMPLE THEORY
  √
  Suppose that k = 1 and let t1 = t. Then
  √
  −1
  n(F−1
  (t)) = n(Q(G−1
  n (t) − F
  n (t)) − Q(t))
  −1
  Q(Gn (t)) − Q(t) √
  n(G−1
  =
  n (t) − t)
  G−1
  n (t) − t
  Q(G−1
  d
  n (t)) − Q(t)
  Vn (t)
  =
  for the special Vn process
  G−1
  n (t) − t
  →a.s. Q (t)V(t) ∼ N (0, (Q (t))2 t(1 − t)) .
  Similarly
   √
  (a)
  
  
  
  
  
  2
  −1 (t ))
  n(F−1
  k
  n (t1 ) − F
  ·
  ·
  ·
  √
  −1 (t ))
  n(F−1
  (t
  )
  k −F
  k
  n
  
  
  
  
   d √ 
   = n
  
  
  
  
  Q(G−1
  n (t1 )) − Q(t1 )
  ·
  ·
  ·
  Q(G−1
  (t
  1 )) − Q(t1 )
  n
  
  
  
  
   →a.s.
  
  
  
  
  
  
  
  Q (t1 )V(t1 )
  ·
  ·
  ·
  Q (tk )V(tk )
  
  
  
  .
  
  
  Theorem 6.3 (Bahadur representation of quantile processes). The uniform quantile process can
  be written as
  (16)
  Vn = −Un + op (1)
  where the o(1) term is uniform in 0 ≤ t ≤ 1; i.e.
  (17)
  Vn + Un ∞ →p 0 .
  In fact
  (18)
  lim sup
  n→∞
  n1/4 Vn + Un ∞
  1
  =√
  2
  bn (log n)
  a.s.
  √
  where bn ≡ 2 log log n. Moreover, if Q (t) exists, then
  √
  √
  −1
  (19)
  n(F−1
  (t)) = − Q (t) n(Fn (F −1 (t)) − t) + op (1) .
  n (t) − F
  Corollary 1 (Asymptotic normality of the t−th quantile). Suppose that Q = F −1 is diﬀerentiable
  at t ∈ (0, 1). Then
  √
  t(1 − t)
  −1
  n(F−1
  (t)
  −
  F
  (t))
  →
  Q
  (t)N
  (0,
  t(1
  −
  t))
  =
  N
  0,
  (20)
  .
  d
  n
  f 2 (F −1 (t))
  Corollary 2 (Asymptotic normality of a linear combination of order statistics). Suppose that J
  is bounded and continuous a.e. F −1 , and suppose that E(X 2 ) < ∞. Let
  1
  n
  1
  i
  X(i) ,
  (21) Tn ≡
  J
  µ=
  J(u)F −1 (u)du .
  n
  n+1
  0
  i=1
  Then
  (22)
  √
  n(Tn − µ) →d
  1
  JVdF −1 ∼ N (0, σ 2 (J, F ))
  0
  where
  (23)
  2
  1 1
  σ (J, F ) =
  0
  0
  J(s)J(t)(s ∧ t − st)dF −1 (s)dF −1 (t) .
  ```
  Related documents
metadata: {"found_math": false, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0}