FOREWORD xi
PREFACE xiii

Patrick Billingsley 1925–2011 xv

Chapter1 PROBABILITY 1

1. BOREL’S NORMAL NUMBER THEOREM, 1

The Unit Interval

The Weak Law of Large Numbers

The Strong Law of Large Numbers

Strong Law Versus Weak

Length

The Measure Theory of Diophantine Approximation*

2. PROBABILITY MEASURES, 18

Spaces

Assigning Probabilities

Classes of Sets

Probability Measures

Lebesgue Measure on the Unit Interval

Sequence Space*

Constructing s-Fields*

3. EXISTENCE AND EXTENSION, 39

Construction of the Extension

Uniqueness and the p–? Theorem

Monotone Classes

Lebesgue Measure on the Unit Interval

Completeness

Nonmeasurable Sets

Two Impossibility Theorems*

4. DENUMERABLE PROBABILITIES, 53

General Formulas

Limit Sets

Independent Events

Subfields

The Borel-Cantelli Lemmas

The Zero-One Law

5. SIMPLE RANDOM VARIABLES, 72

Definition

Convergence of Random Variables

Independence

Existence of Independent Sequences

Expected Value

Inequalities

6. THE LAW OF LARGE NUMBERS, 90

The Strong Law

The Weak Law

Bernstein's Theorem

A Refinement of the Second Borel-Cantelli Lemma

7. GAMBLING SYSTEMS, 98

Gambler's Ruin

Selection Systems

Gambling Policies

Bold Play*

Timid Play*

8. MARKOV CHAINS, 117

Definitions

Higher-Order Transitions

An Existence Theorem

Transience and Persistence

Another Criterion for Persistence

Stationary Distributions

Exponential Convergence*

Optimal Stopping*

9. LARGE DEVIATIONS AND THE LAW OF THE ITERATED LOGARITHM, 154

Moment Generating Functions

Large Deviations

Chernoff's Theorem*

The Law of the Iterated Logarithm

Chapter2 MEASURE 167

10. GENERAL MEASURES, 167

Classes of Sets

Conventions Involving 8

Measures

Uniqueness

11. OUTER MEASURE, 174

Outer Measure

Extension

An Approximation Theorem

12. MEASURES IN EUCLIDEAN SPACE, 181

Lebesgue Measure

Regularity

Specifying Measures on the Line

Specifying Measures in Rk

Strange Euclidean Sets*

13. MEASURABLE FUNCTIONS AND MAPPINGS, 192

Measurable Mappings

Mappings into Rk

Limits and Measurability

Transformations of Measures

14. DISTRIBUTION FUNCTIONS, 198

Distribution Functions

Exponential Distributions

Weak Convergence

Convergence of Types*

Extremal Distributions*

Chapter3 INTEGRATION 211

15. THE INTEGRAL, 211

Definition

Nonnegative Functions

Uniqueness

16. PROPERTIES OF THE INTEGRAL, 218

Equalities and Inequalities

Integration to the Limit

Integration over Sets

Densities

Change of Variable

Uniform Integrability

Complex Functions

17. THE INTEGRAL WITH RESPECT TO LEBESGUE MEASURE, 234

The Lebesgue Integral on the Line

The Riemann Integral

The Fundamental Theorem of Calculus

Change of Variable

The Lebesgue Integral in Rk

Stieltjes Integrals

18. PRODUCT MEASURE AND FUBINI’S THEOREM, 245

Product Spaces

Product Measure

Fubini's Theorem

Integration by Parts

Products of Higher Order

19. THE Lp SPACES*, 256

Definitions

Completeness and Separability

Conjugate Spaces

Weak Compactness

Some Decision Theory

The Space L2

An Estimation Problem

Chapter4 RANDOM VARIABLES AND EXPECTED VALUES 271

20. RANDOM VARIABLES AND DISTRIBUTIONS, 271

Random Variables and Vectors

Subfields

Distributions

Multidimensional Distributions

Independence

Sequences of Random Variables

Convolution

Convergence in Probability

The Glivenko-Cantelli Theorem*

21. EXPECTED VALUES, 291

Expected Value as Integral

Expected Values and Limits

Expected Values and Distributions

Moments

Inequalities

Joint Integrals

Independence and Expected Value

Moment Generating Functions

22. SUMS OF INDEPENDENT RANDOM VARIABLES, 300

The Strong Law of Large Numbers

The Weak Law and Moment Generating Functions

Kolmogorov's Zero-One Law

Maximal Inequalities

Convergence of Random Series

Random Taylor Series*

23. THE POISSON PROCESS, 316

Characterization of the Exponential Distribution

The Poisson Process

The Poisson Approximation

Other Characterizations of the Poisson Process

Stochastic Processes

24. THE ERGODIC THEOREM*, 330

Measure-Preserving Transformations

Ergodicity

Ergodicity of Rotations

Proof of the Ergodic Theorem

The Continued-Fraction Transformation

Diophantine Approximation

Chapter5 CONVERGENCE OF DISTRIBUTIONS 349

25. WEAK CONVERGENCE, 349

Definitions

Uniform Distribution Modulo 1*

Convergence in Distribution

Convergence in Probability

Fundamental Theorems

Helly's Theorem

Integration to the Limit

26. CHARACTERISTIC FUNCTIONS, 365

Definition

Moments and Derivatives

Independence

Inversion and the Uniqueness Theorem

The Continuity Theorem

Fourier Series*

27. THE CENTRAL LIMIT THEOREM, 380

Identically Distributed Summands

The Lindeberg and Lyapounov Theorems

Dependent Variables*

28. INFINITELY DIVISIBLE DISTRIBUTIONS*, 394

Vague Convergence

The Possible Limits

Characterizing the Limit

29. LIMIT THEOREMS IN Rk, 402

The Basic Theorems

Characteristic Functions

Normal Distributions in Rk

The Central Limit Theorem

30. THE METHOD OF MOMENTS*, 412

The Moment Problem

Moment Generating Functions

Central Limit Theorem by Moments

Application to Sampling Theory

Application to Number Theory

Chapter6 DERIVATIVES AND CONDITIONAL PROBABILITY 425

31. DERIVATIVES ON THE LINE*, 425

The Fundamental Theorem of Calculus

Derivatives of Integrals

Singular Functions

Integrals of Derivatives

Functions of Bounded Variation

32. THE RADON–NIKODYM THEOREM, 446

Additive Set Functions

The Hahn Decomposition

Absolute Continuity and Singularity

The Main Theorem

33. CONDITIONAL PROBABILITY, 454

The Discrete Case

The General Case

Properties of Conditional Probability

Difficulties and Curiosities

Conditional Probability Distributions

34. CONDITIONAL EXPECTATION, 472

Definition

Properties of Conditional Expectation

Conditional Distributions and Expectations

Sufficient Subfields*

Minimum-Variance Estimation*

35. MARTINGALES, 487

Definition

Submartingales

Gambling

Functions of Martingales

Stopping Times

Inequalities

Convergence Theorems

Applications: Derivatives

Likelihood Ratios

Reversed Martingales

Applications: de Finetti's Theorem

Bayes Estimation

A Central Limit Theorem*

Chapter7 STOCHASTIC PROCESSES 513

36. KOLMOGOROV'S EXISTENCE THEOREM, 513

Stochastic Processes

Finite-Dimensional Distributions

Product Spaces

Kolmogorov's Existence Theorem

The Inadequacy of RT

A Return to Ergodic Theory

The Hewitt–Savage Theorem*

37. BROWNIAN MOTION, 530

Definition

Continuity of Paths

Measurable Processes

Irregularity of Brownian Motion Paths

The Strong Markov Property

The Reflection Principle

Skorohod Embedding

Invariance*

38. NONDENUMERABLE PROBABILITIES, 558

Introduction

Definitions

Existence Theorems

Consequences of Separability*

APPENDIX 571

NOTES ON THE PROBLEMS 587

BIBLIOGRAPHY 617

INDEX 619