=========================================================================
ANALYSIS OF INSIDER ONTARIO LOTTERY WINS
[For the CBC Fifth Estate episode of October 25, 2006.]
by Jeffrey S. Rosenthal
**UPDATE** (February 2014): An article about this analysis
and the many subsequent developments, including links to
numerous newspaper reports, is now available at:
http://probability.ca/lotteryscandal/
(Dr. Rosenthal is a professor in the Department of Statistics at the
University of Toronto, and the author of "Struck by Lightning: The Curious
World of Probabilities". He maintains the web page probability.ca.)
** SUMMARY OF THE REPORT **
We consider the major ($50,000 plus) Ontario lottery wins, over the
period 1999-2006. Our main findings are as follows.
* Owners/employees of stores selling lottery tickets make up no more
than one part in 247 [or, using an "alternative" baseline count as
described below, one part in 148] of the adult Ontario population, and
represent no more than one part in 165 [alternatively one part in 100]
of the total amount spent on Ontario lottery tickets.
* This means we would "expect" them to win no more than about 35
[alternatively 57] of the 5,713 major Ontario lottery prizes awarded
in 1999-2006. In fact, they won about 200 of these prizes, or about
5.7 [alt. 3.5] times as many as expected, or about 165 [alt. 143] more
than expected. This suggests that some other lottery customers missed
out on major lottery prizes that they had won.
* The probability that this would happen by pure luck is less than one
chance in a trillion trillion trillion trillion, which is absolutely
impossible. Hence, store owners/employees won significantly more major
lottery prizes than they possibly could by pure luck. This conclusion
remains valid even under very conservative assumptions about how many
store owners/employees there really are, and how much they really spend
on lottery tickets.
* For the "Instant" lottery games alone, the situation is even more
extreme. We would expect the store owners/employees to win about 4
[alt. 7] major Instant prizes. In fact, they won about 60 of them --
about 15 [alt. 9] times as many as expected. The probability of this
happening by pure luck alone is, again, absolutely impossible.
* Focusing solely on store owners (as opposed to owners and employees
together) again leads to similar results, with about 83 observed major
wins when we would expect less than 26, an excess of about 57. It is
again absolutely inconceivable that they arose by pure chance alone.
** DETAILS OF THE ANALYSIS **
NUMBER OF RELEVANT INSIDERS:
The number of lottery retail locations is 10,300 (averaged by year).
The number of "insider" employees per retail location is unknown.
However, a random survey of 200 locations conducted by Fifth Estate
indicates that the average number is about 3.2. The survey figures have a
standard deviation (variability) of 1.65, leading to a Monte Carlo error
(sampling uncertainty) of about 1.65 / sqrt(200) = 0.12. This means
that, even given the survey uncertainties, we can be fairly confident
that there is an average of less than 3.2 + 0.12 = 3.32, or rounding up,
about 3.5 employees per retail location or 36,050 employees total.
Some have questioned this figure due to different types of stores being
involved (convenience, grocery, drug, etc.). Such confusion could be
avoided if we knew which insider wins correspond to which store type,
but unfortunately OLG has not yet provided that information.
As an "alternative" count, we note that an OLG employee (Alex Campbell,
Director of Business Optimisation) testified under oath at a trial that
the number of relevant insiders is "50,000 or 60,000", though it is not
clear how this figure was obtained. So, to allow for this possibility,
we also consider the "alternative" that the number of relevant store
employees/owners is 60,000 (the upper figure from the trial testimony).
AMOUNT THOSE RELEVANT INSIDERS SPEND ON LOTTERY TICKETS:
In the Fifth Estate survey, 136 of 200 owners/employees (68%) said they
played the lottery. Of those 136, five declined to specify an amount
spent on lotteries. Of the remaining 131, the average claimed amount
spent per year on lottery tickets is equal to $476.31, with a standard
deviation of $602.50, leading to a Monte Carlo error of $602.50 /
sqrt(131) = $52.64.
This means that we can be fairly confident that the average amount
spent by the 131 owners/employees is less than $476.31 + $52.64, or
$528.95, or rounding up, $550 per year. (It is possible that those
five owners/employees who refused to answer spend more than this, but
that would only have a small effect on the results.) The average annual
amount spent by all 195 owners/employees (omitting the five who refused
to answer) is then $550 * (131/195) = $370.
Combining this with the above figure of 36,050 total store
owners/employees leads to the conclusion that the total amount spent
annually on lottery tickets by lottery store owners/employees is less
than 36,050 * $370 = $13,338,500.
Or, using the "alternative" count of 60,000 owners/employees, we then
compute that the total amount spent annually on lottery tickets by
lottery store owners/employees is less than $370 * 60,000 = $22,200,000.
As an aside, we note that annual lottery sales in Ontario for this period
(averaged by year) are about $2.22 Billion. Given an adult Ontario
population (averaged by year) of about 8,900,000, this works out to
$249.44 per adult per year. Thus, on average, store employees spend at
most $370 / $249.44 = 1.5 times as much as the average adult on lotteries.
EXPECTED NUMBER OF INSIDER WINS:
The above analysis says that, using our survey figures, the fraction
of annual lottery spending that is done by lottery store employees is
probably less than $13,338,500 / $2.22 Billion, or one part in 166.
Now, there were 5,713 redemptions of major ($50,000 plus) prizes in
this 1999-2005 period. Hence, on average, we would expect the number
of such prizes won by store employees to be less than 5,713 / 166,
i.e. less than 35.
Or, using the alternative count of 60,000 owners/employees, the fraction
of annual lottery spending that is done by lottery store employees
would then be less than $22,200,000 / $2.22 Billion, or one part in 100.
Hence, we would expect the number of such prizes won by store employees
to be less than 5,713 / 100, or about 57.1.
ACTUAL NUMBER OF INSIDER WINS:
In point of fact, there were 214 major prizes claimed by "insiders"
during this period. (This is likely an undercount, as doubtless some
of the insiders e.g. denied that they were insiders, or got friends to
claim the prizes for them.)
For 160 of these 214 prizes, we have more detailed information. In two
of those cases the claimant status is known. Of the other 158, 148 are
listed as store owners or employees. So, extrapolating that figure to the
unknown claims, we conclude that the number of major wins by lottery store
owners/employees during this time period is about 214 * (148/158) = 200.
P-VALUE, THE PROBABILITY THAT THIS HAPPENED PURELY BY CHANCE:
The question then becomes, what is the "p-value", i.e. the probability
that the store owners and employees would simply get lucky and win 200
times instead of the expected 35 [alt. 57.1] times?
To compute this, we note that the number of wins should approximately
follow a "Poisson distribution" (introduced by the French mathematician
Siméon Denis Poisson in 1837). So the question is, what is the
probability that a draw from a Poisson distribution with mean 35
[alt. 57.1] would be at least as large as 200?
I wrote a computer program "poisson.c" (in the C programming language)
to compute this probability (and also confirmed all such calculations
using the Mathematica software package). The conclusion is that for
the figure of 35 expected wins, this p-value is equal to six
chances in a 1 followed by 82 zeroes, which is less than one chance in
a billion trillion trillion trillion trillion trillion trillion --
absolutely inconceivable.
Using the "alternative" count of 57.1 expected wins, the p-value becomes
6 x 10^(-49), or less than one chance in a trillion trillion trillion
trillion -- still absolutely inconceivable.
INSTANT GAMES ALONE:
Next, we consider the Instant lottery games on their own, to test whether
they alone are also unbalanced in favour of insider wins.
The actual number of major insider Instant wins in this time period
was 53+12=65. Thus, as above, the number of major wins by store
owners/employees may be estimated as 65 * (148/158) = 61.
I do not have clear figures for the total amount spent by Ontarians and
by store owners/employees on Instant games specifically, so I cannot
compare them directly. Instead, I assume that owners/employees spend
the same factor compared to the average adult as for overall lottery
spending, i.e. no more than 1.5 times as much.
The total number of major Instant wins in this time period was 577+97=674.
Thus, using our figure of 36,050 total owners/employees gives an expected
number of wins of 674 * 36,050 * 1.5 / 8,900,000 = 4.10 (about 1/15 as
much as actually observed). Or, using the "alternative" count of 60,000
total owners/employees, the expected number becomes 674 * 60,000 * 1.5 /
8,900,000 = 6.82 (about 1/9 as much as actually observed).
The p-value, i.e. the probability of observing 61 or more Instant
wins given an expected number of 4.10, is then less than one chance
in a one followed by 48 zeroes, or one chance in a trillion trillion
trillion trillion. Or, using the alternative expected value of 6.82,
the p-value becomes 1.8 x 10^(-36), or about one chance in a trillion
trillion trillion. Either way, this is still an unimaginably small
figure, i.e. it is quite impossible that the employees would have gotten
this lucky just by chance.
NON-INSTANT GAMES ALONE:
If we consider just the NON-Instant lotteries, then the number of
insider wins for those is about 200 - 61 = 139. Meanwhile, the expected
such number is no more than (5,713 - 674) / 166 = 30.4, or using the
alternative count, (5,713 - 674) / 100 = 50.4.
Using these figures, the corresponding p-value then becomes 1.1 x
10^(-46), or about one chance in ten billion trillion trillion trillion.
Or, using the alternative count, 9.1 x 10^(-25), or about one chance in
a trillion trillion. Either way, it is unimaginably unlikely that even
the non-Instant insider wins can be explained by pure luck alone.
ANALYSIS OF THE STORE OWNERS ALONE:
We now concentrate on just the store owners (as opposed to store
employees). Of the 158 insider wins for which the insider's status is
known, 61 are listed as store owners. As above, we extrapolate that
figure to the unknown 54 claims, leading to a conclusion of about 214
* (61/158) = 82.62, or about 83, insider major wins specifically by
store owners.
We know there are about 10,300 lottery ticket sales locations. Probably
most of these locations (e.g. convenience stores) have exactly one
individual owner. And although some special cases (e.g. Loblaws grocery
stores) may have multiple owners, we have been informed that no Loblaws
owner has won a major lottery prize, so this fact may be irrelevant.
In any case, to be very conservative in our assumptions, we assume an
average of 2 owners per ticket outlet, for a total of 20,600 owners.
The next question is store owners' annual lottery ticket spending.
Our survey of 200 convenience store owners/employees happened to include
75 owners. Of these, 61 reported buying lottery tickets to some extent
(while the other 14 said they did not). Of those 61, three declined to
specify how much they spend on tickets. The remaining 58 each specified
an amount. The average of the 58 self-reported yearly lottery ticket
spending figures was $516.53, with a standard deviation (variability)
of $580.96, leading to a Monte Carlo error (sampling uncertainty) of
about 580.96 / sqrt(58) = $76.28.
Hence, being conservative, we can say that the average annual lottery
ticket expenditures of owners who buy tickets is probably less than
$516.53 + $76.28 = $592.81. Thus, the overall average annual lottery
ticket expenditures of owners (whether they buy tickets or not)
is probably less than $592.81 * (61/75) = $482.15, or about $482.
(This is about 482/370 = 1.3 times as much as the average of all owners
and employees, and about 482/249.44 = 1.9 time as much as the average
of all Ontario adults.)
Putting this together, the total annual amount spent on lottery tickets
by store owners is probably less than $482 * 20,600 = $9,929,200.
The fraction of annual lottery spending that is done by lottery store
owners is then less than $9,929,200 / $2.22 Billion, or one part in 223.6.
Hence, we would expect the number of such prizes won by store employees
to be less than 5,713 / 223.6, i.e. less than 26.
Given an expected number of wins of 26, the probability of observing 83
or more wins is then given by 5.2 x 10^(-19), or less than one chance in
a trillion trillion. This is, again, absolutely inconceivable. That is,
even analysing just the store owners alone, and even assuming a generous
average figure of two owners per lottery outlet, it is still absolutely
inconceivable that the number of insider store owner major lottery wins
arose by pure chance alone.
COMPARISON TO OLG EMPLOYEES:
For comparative purposes, we also consider the employees of the Ontario
Lottery Gaming Corporation (OLG) itself.
There are about 7,150 such employees (averaged by year).
So, the number of major lottery prizes we would expect them to win is
about 5,713 * (7,150 / 8,900,000) = 4.6.
In fact, they are confirmed as winning 4 such prizes (plus there are
two claimants of "unknown" status) out of the 158 for which records
are available. Extrapolating this to the remaining prizes means the
OLG employees won about 214 * (4/158) = 5.42, i.e. about 5 prizes.
This figure is perfectly consistent with their expected number, 4.6.
It is not surprising or suspicious at all. That is, for the OLG
employees, the number of major prizes won is perfectly consistent with
what we would expect from the laws of chance, and there is absolutely
no evidence of anything being wrong.
THE PROBABILITY THAT AT LEAST ONE FIFTH ESTATE VIEWER MISSED OUT:
The average number of Fifth Estate viewers is, apparently, about 500,000.
With 500,000 viewers, the probability that any given Canadian adult is
watching is about 500,000 / 25,297,520, or about 1/50.
So, assuming that 165 people (as above) have missed out on major prizes
during this time period, the probability that all 165 are NOT watching is:
(49/50)^165 = 0.03566995.
So, the probability that at least one of them IS watching is
1 - 0.03566995 = 0.96433,
or over 96%.
Or, using the "alternative" result that about 143 people have missed out
on major prizes during this time period, the probability that at least
one of them is watching is:
1 - (49/50)^143 = 0.9443675
or over 94%.
EFFECTS OF EVEN MORE STORE EMPLOYEES:
A last-minute e-mail from the OLG claimed there are really 140,217
relevant insiders -- much more than the 36,050 that we estimated
ourselves, or the 50,000 to 60,000 claimed in the OLG trial testimony.
We believe the 140,217 figure to be significantly inflated, however we
now analyse it as well.
Using this latest OLG count, we obtain a figure for total amount spent
annually on lottery tickets by lottery store owners/employees of about
140,217 * $370 = $51,880,290. The fraction of annual lottery spending
that is done by lottery store owners/employees would then be less than
$51,880,290 / $2.22 Billion, or one part in 42.8. Hence, we would expect
the number of such prizes won by store employees to be less than 5,713 /
42.8, i.e. no more than 133.5.
Given an expected number of wins of 133.5, the probability of observing
200 or more wins is then given by 4.9 x 10^(-8), or less than one
chance in 20 million, still EXTREMELY unlikely. That is, even the
implausible-seeming figure of 140,217 store owners/employees STILL does
not explain the large number of insider major lottery wins.
For the Instant category alone, using the inflated population figure of
140,217 insiders, the expected number of major insider Instant wins would
be 674 * 140,217 * 1.5 / 8,900,000 = 15.93. Given an expected number
of wins of 15.93, the probability of observing 61 or more Instant wins
is then given by 6.9 x 10^(-18), or about seven chances in a billion
billion, again absolutely inconceivable. So, the implausible-seeming
figure still does not explain the large number of insider major Instant
lottery wins, either.
Furthermore, of course this last-minute OLG employee count does not in
any way affect the analysis (above) of the store owners alone, or in
any way explain the excess number of store owner major lottery wins.
MORE ABOUT THAT FIGURE OF 140,217:
As noted above, the figure of 140,217 lottery store owners/employees
seems unjustified given the available data. Furthermore, even using this
figure still does not explain the large number of insider wins. However,
the question remains as to whether the 140,217 figure is at all plausible.
One way to assess this question is as follows. If there were in
fact 140,217 store owners/employees, then with at most 20,600 owners,
this leaves at least 129,400 employees. By the above analyses, the
amount spent annually on lottery tickets by owners would be no more than
$9,929,200 (and even that figure is very generous, assuming an average of
two owners per location), while the amount spent by owners and employees
together would be about $55,500,000. So, we would expect the fraction
of major owner/employee wins which are in fact specifically owner wins
to be no more than 9,929,200 / 51,880,290 = 0.1913867, i.e. about 19%.
In fact, the fraction of major wins which were specifically from owners
is about 61 / 200 = 0.305, i.e. about 31%, or about 1.6 times as large
as expected. What is the probability of this arising purely by chance?
Well, out of the 200 insider wins, the probability distribution for the
number of insider wins specifically by owners should be Binomial(200,
0.1913867). So, the probability of there being at least 61 such wins,
by pure chance alone, can be computed using the "R" computer software,
with the command:
pbinom(61, 200, 0.1913867, lower.tail=FALSE, log.p=FALSE)
which works out to 4.1 x 10^(-5), or about one chance in 24,000.
While not "absolutely inconceivable", this is still extremely unlikely.
We conclude that even with very generous assumptions, the figure of
140,217 lottery store owners/employees is not consistent with the fraction
of major lottery wins specifically by owners as opposed to employees.
This appears to cast further significant doubt on the claimed figure
of 140,217.
REMOVING THE FORMER EMPLOYEE ("TURNOVER") COUNTS:
Another issue concerning that last-minute OLG figure of 140,217 is that
it includes not just current but also FORMER employees (or, "turnover" of
employees) of lottery stores, even though former employees are apparently
not counted as insiders when compiling insider win statistics and thus
should not be counted when computing expected values either.
If we remove the former employee category from the last-minute OLG counts,
then the number of owners/employees remaining is equal to:
4877*4 + 1937*5 + 1196*8 + 229*5 + 123*10 + 214*10 + 18*10 + 397*6 +
998*25 + 173*6 + 18*6 + 731*40 = 101,174,
i.e. 101,174 owners/employees. Using this count, the total amount
spent annually on lottery tickets by lottery store owners/employees
would be no more than 101,174 * $370 = $37,434,380. The fraction of
annual lottery spending that is done by lottery store owners/employees
would then be less than $37,434,380 / $2.22 Billion, or one part in 59.3.
Hence, we would expect the number of such prizes won by store employees
to be less than 5,713 / 42.8, i.e. no more than 96.33.
Given an expected number of wins of 96.33, the probability of observing
200 or more wins is then given by 2.0 x 10^(-20), or one chance in 50
billion billion, again absolutely inconceivable.
For the Instant games alone, the count of 101,174 owners/employees gives
an expected number of major insider Instant wins of 674 * 101,174 * 1.5 /
8,900,000 = 11.49. Given this, the probability of observing 61 or more
Instant wins is then given by 1.2 x 10^(-24), or about one chance in a
trillion trillion, again absolutely inconceivable.
POSSIBLE UNDERREPORTING OF LOTTERY EXPENDITURES:
One concern with some of the above analysis is that it is based partially
on store owners/employees self-reporting how much they spend on lottery
tickets. It seems likely that those selling lottery tickets would have
an accurate sense of how much they themselves spend on them, and would
not feel any "shame" in admitting the precise figure, so that their
self-reporting would be fairly accurate. However, the possibility of
underreporting still remains.
To investigate this, Fifth Estate conducted a small additional survey,
this time of the general population through random phone calls.
Since we know (see above) that the average Ontario adult spends $249.44
per year, this provides a check of how much the general population tends
to underreport their lottery spending.
The small survey indicates that the general population only reports
spending about $141.03 on lottery tickets, for an underreporting factor
of $249.44 / $141.03 = 1.77. Now, both logic and certain anecdotal
evidence (obtained while conducting the survey) suggests that lottery
owners/employees probably do not underreport as much as the general
population. In other words, just because the general population
underreports their lottery spending does not necessarily imply that
lottery owners/employees -- a rather different group -- do as well.
However, we now consider the effect if they do.
Such underreporting would mean that the amount spent by owners/employees
on lottery tickets would actually be about 1.77 times as much as the
figure used in the above analysis. How does this affect the results?
The expected number of total insider wins would now be 1.77 times as much
as before, or less than 1.77 * 35 = 61.95 {alt. 1.77 * 57.1 = 101.1].
The probability of observing at least 200 such wins would then become
5.8 x 10^(-44) [alt. 2.8 x 10^(-18)], either way still absolutely
inconceivable.
For Instant wins, the expected number of insider Instant wins would now be
less than 1.77 * 4.10 = 7.26 [alt. 1.77 * 6.82 = 12.07]. The probability
of observing at least 61 such wins would then become 5.2 x 10^(-35)
[alt. 1.4 x 10^(-23)], either way again absolutely inconceivable.
Finally, we consider combining this underreporting factor of 1.77 with the
OLG's last-minute, apparently inflated count of 101,174 owners/employees
(i.e., with the former employees omitted). That count would lead to
an expected number of total insider wins of 1.77 * 96.33 = 170.50.
The probability of observing at least 200 such wins would then become
0.0148341, or just over one percent. This is no longer "astronomically
small", but it is still very unlikely, and much less than the standard
statistically significant cutoff of five percent. So, even combining
the general population's underreporting factor with the OLG's last-minute
count still does not properly explain the large number of major insider
lottery wins.
For Instant wins, the expected number of insider Instant wins would
now be less than 1.77 * 11.49 = 20.34. The probability of observing at
least 61 such wins would then become 2.8 x 10^(-13) another absolutely
inconceivable event. So, even combining the general population's
underreporting factor with the OLG's last-minute count still does come
at all close to explaining the large number of major insider Instant
lottery wins.
=========================================================================