A First Look at Rigorous Probability Theory

by Jeffrey S. Rosenthal


This graduate-level probability textbook was originally published by World Scientific Publishing Co. in 2000 (second printing 2003; third printing 2005, fourth printing 2006), with a second edition in 2006 (second printing 2007). It may be ordered for U.S. $33 (cheap!) directly from the publisher, or from e.g. amazon.ca or amazon.com or amazon.co.uk or indigo.ca.

Below are some reviews and the preface and second-edition preface and table of contents. See also the errata in postscript / PDF (or the first edition errata in postscript / PDF).

NEW: There is now a free, public on-line solutions manual to all even-numbered exercises, by M. Soltanifar, L. Li, and myself.

(See also Evans and Rosenthal's introductory-level probability and statistics book, and an unexpected spoof video.)


SOME REVIEWS

FROM Publisher's Blurb:

This textbook is an introduction to probability theory using measure theory. It is designed for graduate students in a variety of fields (mathematics, statistics, economics, management, finance, computer science, and engineering) who require a working knowledge of probability theory that is mathematically precise, but without excessive technicalities. The text provides complete proofs of all the essential introductory results. Nevertheless, the treatment is focused and accessible, with the measure theory and mathematical details presented in terms of intuitive probabilistic concepts, rather than as separate, imposing subjects. The text strikes an appropriate balance, rigorously developing probability theory while avoiding unnecessary detail.

FROM Math Reviews:

2001h:60001 60-01
Rosenthal, Jeffrey S.(3-TRNT)
A first look at rigorous probability theory.
World Scientific Publishing Co., Inc., River Edge, NJ, 2000. xiv+177 pp. $24.00. ISBN 981-02-4322-7

This book is an introduction to probability theory using measure theory. It provides mathematically complete proofs of all the essential introductory results of probability and measure theory.

The book is divided into fifteen sections and two appendices. The first six sections contain the essential core of measure-theoretic probability theory: sigma-algebras; construction of probability measures; random variables; expected values; inequalities and laws of large numbers; and distributions of random variables. The following two sections introduce dynamic aspects of probability models: stochastic processes are introduced using gambling games as the motivating example and discrete Markov chains are discussed in some detail. The following section complements the results with measure-theoretic flavor by discussing and proving results such as the dominated convergence theorem and Fubini's theorem. Sections 10 to 14 contain a collection of further topics including weak convergence, characteristic functions (together with a proof of the central limit theorem), decomposition of probability laws, conditional probability and expectation and martingales. The final section then provides an appetizer for further topics in the subject of stochastic processes and applications. It contains material on Markov chains on general state spaces, diffusions and stochastic integrals, and the Black-Scholes formula. The appendices provide mathematical background and a guide to further reading.

The book is certainly well placed to establish itself as a core reading in measure-theoretic probability. However, a more complete and advanced book, such as [P. Billingsley, Probability and measure, Third edition, Wiley, New York, 1995; MR 95k:60001], might be needed as a complementary source for graduate students in mathematics and statistics. Furthermore, although the text contains a variety of excellent exercises, students from economics, computer science, engineering, etc., might find the addition of more applied examples and exercises beneficial.

I found this little book delightful reading and a worthwhile addition to the existing literature.

Reviewed by Rüdiger Kiesel

FROM Math Reviews (re Second Edition):

This is a fine textbook on probability theory based on measure theory. The parts of measure theory that are needed are developed within the book and a teacher of measure theory could find them quite useful. The construction of the Lebesgue measure (extension theorem) is unusual and interesting.

The reader will get basic ideas on most fundamental topics in probability theory in a detailed (as far as the proofs are concerned), mathematically rigorous and very readable way. [...] The author presents a very good selection on a mere 219 pages. [...]

Chapter 15 presents a nice heuristic introduction to Markov chains with general state space, continuous time Markov processes, Brownian motion, diffusions, and stochastic integrals.

Reviewed by Dalibor Volny

FROM amazon.com customer reviews:

(5 stars) Excellent primer to use as supplement or for review.
March 15, 2002
Reviewer: from California

This is a marvelous primer on measure-theoretic probability. I came across it a couple of years after taking a course based on Chung's famous text ("A Course in Prob. Theory") and found it to be an excellent book for review and remediation--that is, it helped me get a better overview of the material I had already learned and it helped me learn topics such as, say, uniform integrability, that didn't sink in too well the first time around.

According to the preface, the author prepared most of the book as supplemental class notes for the benefit of his students in a course whose main text was, if I recall correctly, Billingsley's excellent "Probability and Measure". The students were so enthusiastic about the usefulness of Professor Rosenthal's supplemental info that they insisted he publish it, despite his objection that the book wasn't original enough to warrant entry into an already crowded field. Well, the students made the right call: Rosenthal's clear and concise text will, I think, help almost any student learn measure-theoretic probability more efficiently. I'd also recommend it to folks who need a concise review of measure-theoretic probability.


(5 stars) Best Probability book ever!
July 10, 2006
Reviewer: Thomas R. Fielden (Portland, OR USA)

As a graduate student in mathematics I appreciate the rigorous and no nonsense treatment of the subject. I'm am using this text to study for my Ph.D. qualifying exam in statistics. It's explaining statistics in a language I understand.


(5 stars) A gem.
July 17, 2007
Reviewer: M. Henri De Feraudy (France)

This is my bedside book at the present time. It's compact, written with immense respect for the reader and even covers some financial applications.

It's recalling the measure theory I learned as an undergraduate with the right style.

So much better than some of the "Probabilty from dummies" I have put away.

When I finish the book I hope to move on to some of the heavier books with a clear idea of where I am going.


(5 stars) A delightful read and a great introduction.
June 12, 2009
Reviewer: A customer

I took this book from the library during a course in measure-theoretic probability, and how lucky I was to come across it!

A very well structured book, the choice of material (for an introduction) is excellent. As the title suggests, the book is rather rigorous (most results are with proofs, which helps understand the theory better), and at the same time the author does a good job at motivating the introduction of the mathematical concepts required to understand (rigorous) probability.

The best part is that, for any mathematician, this book will also be a lot of fun to read!

I would like to sincerely congratulate the author for making something this really good.


FROM Willmott Forums:

A great and very compact overview. I find it great as a guide to the terminology and as a road-map, with links to standard texts. I think it is used for the Univ. of Toronto's Fin. Eng. program (Author is at U of T.).


FROM amazon.com Graduate Probability Books Listmania:

A very very good book but short. Can not imagine the unbelievable potential for this book if the author writes a full version! If you can afford it buy it else you GOT TO check out of your library.


PREFACE TO THE FIRST EDITION

This text grew out of my lecturing the Graduate Probability sequence STA 2111F / 2211S at the University of Toronto over a period of several years. During this time, it became clear to me that there are a large number of graduate students from a variety of departments (mathematics, statistics, economics, management, finance, computer science, engineering, etc.) who require a working knowledge of rigorous probability, but whose mathematical background may be insufficient to dive straight into advanced texts on the subject.

This text is intended to answer that need. It provides an introduction to rigorous (i.e., mathematically precise) probability theory using measure theory. At the same time, I have tried to make it brief and to the point, and as accessible as possible. In particular, probabilistic language and perspective are used throughout, with necessary measure theory introduced only as needed.

I have tried to strike an appropriate balance between rigorously covering the subject, and avoiding unnecessary detail. The text provides mathematically complete proofs of all of the essential introductory results of probability theory and measure theory. However, more advanced and specialised areas are ignored entirely or only briefly hinted at. For example, the text includes a complete proof of the classical Central Limit Theorem, including the necessary Continuity Theorem for characteristic functions. However, the Lindeberg Central Limit Theorem and Martingale Central Limit Theorem are only briefly sketched and are not proved. Similarly, all necessary facts from measure theory are proved before they are used. However, more abstract and advanced measure theory results are not included. Furthermore, the measure theory is almost always discussed purely in terms of probability, as opposed to being treated as a separate subject which must be mastered before probability theory can be studied.

I hesitated to bring these notes to publication. There are many other books available which treat probability theory with measure theory, and some of them are excellent. For a partial list see Subsection B.3 on page 169. (Indeed, the book by Billingsley was the textbook from which I taught before I started writing these notes. While much has changed since then, the knowledgeable reader will still notice Billingsley's influence in the treatment of many topics herein. The Billingsley book remains one of the best sources for a complete, advanced, and technically precise treatment of probability theory with measure theory.) In terms of content, therefore, the current text adds very little indeed to what has already been written. It was only the reaction of certain students, who found the subject easier to learn from my notes than from longer, more advanced, and more all-inclusive books, that convinced me to go ahead and publish. The reader is urged to consult other books for further study and additional detail.

There are also many books available (see Subsection B.2) which treat probability theory at the undergraduate, less rigorous level, without the use of general measure theory. Such texts provide intuitive notions of probabilities, random variables, etc., but without mathematical precision. In this text it will generally be assumed, for purposes of intuition, that the student has at least a passing familiarity with probability theory at this level. Indeed, Section 1 of the text attempts to link such intuition with the mathematical precision to come. However, mathematically speaking we will not require many results from undergraduate-level probability theory.

Structure. The first six sections of this book could be considered to form a "core" of essential material. After learning them, the student will have a precise mathematical understanding of probabilities and sigma-algebras; random variables, distributions, and expected values; and inequalities and laws of large numbers. Sections 7 and 8 then diverge into the theory of gambling games and Markov chain theory. Section 9 provides a bridge to the more advanced topics of Sections 10 through 14, including weak convergence, characteristic functions, the Central Limit Theorem, Lebesgue Decomposition, conditioning, and martingales.

The final section, Section 15, provides a wide-ranging and somewhat less rigorous introduction to the subject of general stochastic processes. It leads up to diffusions, Ito's Lemma, and finally a brief look at the famous Black-Sholes equation from mathematical finance. It is hoped that this final section will inspire readers to learn more about various aspects of stochastic processes.

Appendix A contains basic facts from elementary mathematics. This appendix can be used for review and to gauge the book's level. In addition, the text makes frequent reference to Appendix A, especially in the earlier sections, to ease the transition to the required mathematical level for the subject. It is hoped that readers can use familiar topics from Appendix A as a springboard to less familiar topics in the text.

Finally, Appendix B lists a variety of references, for background and for further reading.

Exercises. The text contains a number of exercises. Those very closely related to textual material are inserted at the appropriate place. Additional exercises are found at the end of each section, in a separate subsection. I have tried to make the exercises thought provoking without being too difficult. Hints are provided where appropriate. Rather than always asking for computations or proofs, the exercises sometimes ask for explanations and/or examples, to hopefully clarify the subject matter in the student's mind.

Prerequisites. As a prerequisite to reading this text, the student should have a solid background in basic undergraduate-level real analysis (not including measure theory). In particular, the mathematical background summarised in Appendix A should be very familiar. If it is not, then books such as those in Subsection B.1 should be studied first. It is also helpful, but not essential, to have seen some undergraduate-level probability theory at the level of the books in Subsection B.2.

Further reading. For further reading beyond this text, the reader should examine the similar but more advanced books of Subsection B.3. To learn additional topics, the reader should consult the books on pure measure theory of Subsection B.4, and/or the advanced books on stochastic processes of Subsection B.5, and/or the books on mathematical finance of Subsection B.6. I would be content to learn only that this text has inspired students to look at more advanced treatments of the subject.

Acknowledgements. I would like to thank several colleagues for encouraging me in this direction, in particular Mike Evans, Andrey Feuerverger, Keith Knight, Omiros Papaspiliopoulos, Jeremy Quastel, Nancy Reid, and Gareth Roberts. Most importantly, I would like to thank the many students who have studied these topics with me; their questions, insights, and difficulties have been my main source of inspiration.

Jeffrey S. Rosenthal
Toronto, Canada, 2000
http://probability.ca/jeff/
Contact me

Second Printing (2003). For the second printing, a number of minor errors have been corrected. Thanks to Tom Baird, Meng Du, Avery Fullerton, Longhai Li, Hadas Moshonov, Nataliya Portman, and Idan Regev for helping to find them.

Third Printing (2005). A few more minor errors were corrected, with thanks to Samuel Hikspoors, Bin Li, Mahdi Lotfinezhad, Ben Reason, Jay Sheldon, and Zemei Yang.


PREFACE TO THE SECOND EDITION

I am pleased to have the opportunity to publish a second edition of this book. The book's basic structure and content are unchanged; in particular, the emphasis on establishing probability theory's rigorous mathematical foundations, while minimising technicalities as much as possible, remains paramount. However, having taught from this book for several years, I have made considerable revisions and improvements. For example: I thank Ying Oi Chiew and Lai Fun Kwong of World Scientific for facilitating this edition, and thank Richard Dudley, Eung Jun Lee, Neal Madras, Peter Rosenthal, Hermann Thorisson, and Balint Virag for helpful comments. Also, I again thank the many students who have studied and discussed these topics with me over many years.
Jeffrey S. Rosenthal
Toronto, Canada, 2006
http://probability.ca/jeff/
Contact me

Second Printing (2007). A few very minor corrections were made, with thanks to Joe Blitzstein and Emil Zeuthen.


TABLE OF CONTENTS (of Second Edition):

Preface to the First Edition     vii
Preface to the Second Edition     xi
1.  The need for measure theory     1
1.1.  Various kinds of random variables     1
1.2.  The uniform distribution and non-measurable sets     2
1.3.  Exercises     4
1.4.  Section summary     5
2.  Probability triples     7
2.1.  Basic definition     7
2.2.  Constructing probability triples     8
2.3.  The Extension Theorem     10
2.4.  Constructing the Uniform$[0,1]$ distribution     15
2.5.  Extensions of the Extension Theorem     18
2.6.  Coin tossing and other measures     21
2.7.  Exercises     23
2.8.  Section summary     27
3.  Further probabilistic foundations     29
3.1.  Random variables     29
3.2.  Independence     31
3.3.  Continuity of probabilities     33
3.4.  Limit events     34
3.5.  Tail fields     36
3.6.  Exercises     38
3.7.  Section summary     41
4.  Expected values     43
4.1.  Simple random variables     43
4.2.  General non-negative random variables     45
4.3.  Arbitrary random variables     49
4.4.  The integration connection     50
4.5.  Exercises     52
4.6.  Section summary     55
5.  Inequalities and convergence     57
5.1.  Various inequalities     57
5.2.  Convergence of random variables     58
5.3.  Laws of large numbers     60
5.4.  Eliminating the moment conditions     61
5.5.  Exercises     65
5.6.  Section summary     66
6.  Distributions of random variables     67
6.1.  Change of variable theorem     67
6.2.  Examples of distributions     69
6.3.  Exercises     71
6.4.  Section summary     72
7.  Stochastic processes and gambling games     73
7.1.  A first existence theorem     73
7.2.  Gambling and gambler's ruin     75
7.3.  Gambling policies     77
7.4.  Exercises     80
7.5.  Section summary     81
8.  Discrete Markov chains     83
8.1.  A Markov chain existence theorem     85
8.2.  Transience, recurrence, and irreducibility     86
8.3.  Stationary distributions and convergence     89
8.4.  Existence of stationary distributions     94
8.5.  Exercises     98
8.6.  Section summary     101
9.  More probability theorems     103
9.1.  Limit theorems     103
9.2.  Differentiation of expectation     106
9.3.  Moment generating functions and large deviations     107
9.4.  Fubini's Theorem and convolution     110
9.5.  Exercises     113
9.6.  Section summary     115
10.  Weak convergence     117
10.1.  Equivalences of weak convergence     117
10.2.  Connections to other convergence     119
10.3.  Exercises     121
10.4.  Section summary     122
11.  Characteristic functions     125
11.1.  The continuity theorem     126
11.2.  The Central Limit Theorem     133
11.3.  Generalisations of the Central Limit Theorem     135
11.4.  Method of moments     137
11.5.  Exercises     139
11.6.  Section summary     142
12.  Decomposition of probability laws     143
12.1.  Lebesgue and Hahn decompositions     143
12.2.  Decomposition with general measures     147
12.3.  Exercises     148
12.4.  Section summary     149
13.  Conditional probability and expectation     151
13.1.  Conditioning on a random variable     151
13.2.  Conditioning on a sub-sigma-algebra     155
13.3.  Conditional variance     157
13.4.  Exercises     158
13.5.  Section summary     160
14.  Martingales     161
14.1.  Stopping times     162
14.2.  Martingale convergence     168
14.3.  Maximal inequality     171
14.4.  Exercises     173
14.5.  Section summary     176
15.  General stochastic processes     177
15.1.  Kolmogorov Existence Theorem     177
15.2.  Markov chains on general state spaces     179
15.3.  Continuous-time Markov processes     182
15.4.  Brownian motion as a limit     186
15.5.  Existence of Brownian motion     188
15.6.  Diffusions and stochastic integrals     190
15.7.  Ito's Lemma     193
15.8.  The Black-Scholes equation     194
15.9.  Section summary     197
A.  Mathematical Background     199
A.1.  Sets and functions     199
A.2.  Countable sets     200
A.3.  Epsilons and Limits     202
A.4.  Infimums and supremums     204
A.5.  Equivalence relations     207
B.  Bibliography     209
B.1.  Background in real analysis     209
B.2.  Undergraduate-level probability     209
B.3.  Graduate-level probability     210
B.4.  Pure measure theory     210
B.5.  Stochastic processes     210
B.6.  Mathematical finance     211
Index.  213