W.K. Hastings, Statistician and Developer of the Metropolis-Hastings Algorithm

(Written by Jeffrey S. Rosenthal, March 2005; comments and suggestions welcome.)

The Metropolis-Hastings algorithm (or, Hastings-Metropolis algorithm) is the most common Markov chain Monte Carlo (MCMC) method. It is extremely widely used in applied statistics (and in statistical physics and computer science and more), to sample from complicated, high-dimensional probability distributions. A primary source for this algorithm is the paper:
W.K. Hastings (1970), Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97-109.
This paper has been cited well over two thousand times -- a huge number. However, despite this paper's importance, very little information about W.K. Hastings himself is publicly available. The following brief biography is intended to (partially) answer that need.

W. Keith Hastings was born on July 21, 1930, in Toronto, Ontario, Canada. He received his B.A. in Applied Mathematics from the University of Toronto in 1953, and then worked from 1955-59 as a "Consultant in Computer Applications" for the Toronto company H.S. Gellman & Co. Hastings recalls:

Harvey Gellman was a good mentor and encouraged me to pursue my ideas. Some of the projects involved simulations and this was my first contact with statistics and generation of samples from probability distributions.
Overlapping somewhat with this, Hastings received his M.A. in 1958, and his Ph.D. in 1962, both from the University of Toronto's Department of Mathematics (which included Statistics at that time). His Ph.D. thesis title was "Invariant Fiducial Distributions". His Ph.D. supervisor was initially Don Fraser (who mentioned Hastings' thesis results in a January 10, 1962 letter to R.A. Fisher), and later Geoffrey Watson (while Fraser visited Stanford in 1961-62). After completing his Ph.D., Hastings worked briefly at the University of Canterbury in New Zealand (1962-64), and at Bell Labs in New Jersey (1964-66). Hastings writes:
I was never comfortable working on statistical inference for my thesis. My investigations led to too many dead ends and the work seemed to involve more mathematical considerations than statistical ones. When Geoff took over as my supervisor I briefly considered changing topics, but ended up sticking with my original topic and completed my thesis. In New Zealand, I continued this work for a while but eventually gave it up, the final blow coming when I learned that Fiducial Probability was declared 'dead' in a session during a statistics conference held in Ottawa. Bell Labs provided a welcome and effective antidote to all this as I gradually turned towards the computational aspects of statistics. In effect, I was then returning to my professional roots.
From 1966 to 1971, Hastings was an Associate Professor in the Department of Mathematics at the University of Toronto. During this period, he wrote the famous paper listed above (which generalised the work of N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller (1953), "Equations of state calculations by fast computing machines", J. Chem. Phys. 21, 1087-1091). Hastings explains:
When I returned to the University of Toronto, after my time at Bell Labs, I focused on Monte Carlo methods and at first on methods of sampling from probability distributions with no particular area of application in mind. [University of Toronto Chemistry professor] John Valleau and his associates consulted me concerning their work. They were using Metropolis's method to estimate the mean energy of a system of particles in a defined potential field. With 6 coordinates per particle, a system of just 100 particles involved a dimension of 600. When I learned how easy it was to generate samples from high dimensional distributions using Markov chains, I realised how important this was for Statistics, and I devoted all my time to this method and its variants which resulted in the 1970 paper.
While at the University of Toronto, Hastings also supervised his one Ph.D. student, Peter Peskun (now at York University), whose 1970 dissertation "The Choice Of Transition Matrix In Monte Carlo Sampling Methods Using Markov Chains" developed the Peskun ordering on Markov chain kernels. Peskun recalls:
Dr. Hastings was down to earth and very good natured. I can still picture him tugging at his waist band as he chuckled over some comment he had just made. It was a pleasure having Dr. Hastings as my Ph.D. supervisor. He never meddled in what I was trying to do but was always happy to hear and listen to any new results that I came up with. He did make one important suggestion to me which was to express my initial heuristic results in matrix form. This was of tremendous help in proving, in particular, Peskun orderings.
In 1971, Hastings joined the Department of Mathematics at the University of Victoria (in British Columbia, on the west coast of Canada) as an Associate Professor, and was granted tenure there in 1974. He taught at Victoria for 21 years, usually teaching six one-semester courses per year. He did not supervise any more Ph.D. students, but he did supervise two M.Sc. students, and serve on the committees of four Ph.D. and two M.Sc. students. He held NSERC research grants from 1969 to 1980.

Hastings' C.V. lists only two other refereed research papers besides the famous (1970) one:

W.K. Hastings (1972), Test Data for Statistical Algorithms: Least Squares and ANOVA. J. Amer. Statist. Assoc. 67, 874-879.

W.K. Hastings (1974), Variance Reduction and Non-normality. Biometrika 61, 143-149.

It also lists one non-refereed publication ("Death and Taxes", American Studies in Papyrology 10, joint with A.E. Samuel, A.K. Bowman, and R.S. Bagnall), and a couple of memoranda for Bell Labs (including the suggestive-sounding "An Overview of Statistical Computing Software", 1966).

Hastings retired from the University of Victoria in 1992. As of this writing, he still lives in Victoria.

Sources: Professor Hastings' C.V. (hardcopy); direct e-mail correspondence with Professors Fraser, Hastings, and Peskun.