This was posted on ABC couple years ago. Makes good reading.
The Gambler`s Fallacy
The gambler's fallacy, also known as the Monte Carlo fallacy or the fallacy of the maturity of chances, is the belief that if deviations from expected behaviour are observed in repeated independent trials of some random process then these deviations are likely to be evened out by opposite deviations in the future. For example, if a fair coin is tossed repeatedly and tails comes up a larger number of times than is expected, a gambler may incorrectly believe that this means that heads is more likely in future tosses.Such an expectation could be mistakenly referred to as being "due". This is an informal fallacy.
The gambler's fallacy implicitly involves an assertion of negative correlation between trials of the random process and therefore involves a denial of the exchangeability of outcomes of the random process.
The inverse gambler's fallacy is the belief that an unlikely outcome of a random process (such as rolling double sixes on a pair of dice) implies that the process is likely to have occurred many times before reaching that outcome.
An example: coin-tossing
The gambler's fallacy can be illustrated by considering the repeated toss of a fair coin. With a fair coin, the outcomes in different tosses are statistically independent and the probability of getting heads on a single toss is exactly 1 / 2 (one in two). It follows that the probability of getting two heads in two tosses is 1 / 4 (one in four) and the probability of getting three heads in three tosses is 1 / 8 (one in eight). In general, if we let Ai be the event that toss i of a fair coin comes up heads, then we have,
Prleft(bigcap_{i=1}^n A_iright)=prod_{i=1}^n Pr(A_i)={1over2^n}.
Now suppose that we have just tossed four heads in a row, so that if the next coin toss were also to come up heads, it would complete a run of five successive heads. Since the probability of a run of five successive heads is only 1 / 32 (one in thirty-two), a believer in the gambler's fallacy might believe that this next flip is less likely to be heads than to be tails. However, this is not correct, and is a manifestation of the gambler's fallacy. The probability that the next toss is a head is in fact,
Prleft(A_5|A_1 cap A_2 cap A_3 cap A_4 right)=Prleft(A_5right)=1/2.
While a run of five heads is only 1 in 32 (0.03125), it is 1 in 32 before the coin is first tossed. After the first four tosses the results are no longer unknown, so they do not count. Reasoning that it is more likely that the next toss will be a tail than a head due to the past tosses—that a run of luck in the past somehow influences the odds in the future—is the fallacy.
Explaining why the probability is 1/2 for a fair coin
We can see from the above that, if one flips a fair coin 21 times, then the probability of 21 heads is 1 in 2,097,152. However, the probability of flipping a head after having already flipped 20 heads in a row is simply 1 in 2. This is an example of Bayes' theorem.
This can also be seen without knowing that 20 heads have occurred for certain (without applying of Bayes' theorem). Consider the following two probabilities, assuming a fair coin:
probability of 20 heads, then 1 tail = 0.520 * 0.5 = 0.521
probability of 20 heads, then 1 head = 0.520 * 0.5 = 0.521
The probability of getting 20 heads then 1 tail, and the probability of getting 20 heads then another head are both 1 in 2,097,152. Therefore, it is equally likely to flip 21 heads as it is to flip 20 heads and then 1 tail when flipping a fair coin 21 times. Furthermore, these two probabilities are as equally likely as any other 21-flip combinations that can be obtained (there are 2,097,152 total); all 21-flip combinations will have probabilities equal to 0.521, or 1 in 2,097,152. From these observations, there is no reason to assume at any point that a "change of luck" is warranted based on prior trials (flips), because every outcome observed will always have been equally as likely as the other outcomes that were not observed for that particular trial, given a fair coin. Therefore, just as Bayes' theorem shows, the result of each trial comes down to the base probability of the fair coin: 50%.
Other examples
There is another way to emphasize the fallacy. As already mentioned, the fallacy is built on the notion that previous failures indicate an increased probability of success on subsequent attempts. This is, in fact, the inverse of what actually happens, even on a fair chance of a successful event, given a set number of iterations. Assume you have a fair 16-sided die, and a "win" is defined as rolling a 1. Assume a player is given 16 rolls to obtain at least one win (1 - p(rolling no ones)). The low winning odds are just to make the change in probability more noticeable. The probability of having at least one win in the 16 rolls is:
1-(15/16)^16 = 64.4%
However, assume now that the first roll was not a win (93.8% chance of that, 15/16). The player now only has 15 rolls left and, according to the fallacy, should have a higher chance of obtaining the "win" since one "loss" has occurred. His chances of having at least one win are now:
1-(15/16)^15 = 62.2%
Simply by losing one toss the player's probability of winning dropped by 2%. By the time this reaches 5 losses (11 rolls left), his probability of winning on one of the remaining rolls will have dropped to ~50%. The player's odds for at least one win in those 16 rolls has not increased given a series of losses; his odds have decreased because he has fewer iterations left to pull a "win." In other words, the previous losses in no way contribute to the odds of the remaining attempts, but there are fewer remaining attempts to gain a win, which results in a lower probability of obtaining it.
The player becomes more likely to lose in a set number of iterations as he fails to win, and eventually his probability of winning will again equal the probability of winning a single toss, when only one toss is left: 6.2% in this instance.
Some lottery players will choose the same numbers every time, or intentionally change their numbers, but both are equally likely to win any individual lottery draw. Copying the numbers that won the previous lottery draw gives an equal probability, although a rational gambler might attempt to predict other players' choices and then deliberately avoid these numbers. Low numbers (below 31 and especially below 12) are popular because people play birthdays as their "lucky numbers"; hence a win in which these numbers are overrepresented is more likely to result in a shared payout.
A joke told among mathematicians demonstrates the nature of the fallacy. When flying on an aircraft, a man decides always to bring a bomb with him. "The chances of an aircraft having a bomb on it are very small," he reasons, "and certainly the chances of having two are almost none!"
A similar example is in the book The World According to Garp when the hero Garp decides to buy a house a moment after a small plane crashes into it, reasoning that the chances of another plane hitting the house have just dropped to zero.
Non-examples of the fallacy
There are many scenarios where the gambler's fallacy might superficially seem to apply but does not. When the probability of different events is not independent, the probability of future events can change based on the outcome of past events (see statistical permutation). Formally, the system is said to have memory. An example of this is cards drawn without replacement. For example, once a jack is removed from the deck, the next draw is less likely to be a jack and more likely to be of another rank. Thus, the odds for drawing a jack, assuming that it was the first card drawn and that there are no jokers, have decreased from 4/52 (7.69%) to 3/51 (5.88%), while the odds for each other rank have increased from 4/52 (7.69%) to 4/51 (7.84%). This is how counting cards really works, when playing the game of blackjack.
The outcome of future events can be affected if external factors are allowed to change the probability of the events (e.g. changes in the rules of a game affecting a sports team's performance levels). Additionally, an inexperienced player's success may decrease after opposing teams discover his or her weaknesses and exploit them. The player must then attempt to compensate and randomize his strategy. See Game Theory.
Many riddles trick the reader into believing that they are an example of Gambler's Fallacy, such as the Monty Hall problem.
Non-example: unknown probability of event
When the probability of repeated events are not known outcomes may not be equally probable. In the case of coin tossing, as a run of heads gets longer and longer, the likelihood that the coin is biased towards heads increases. If one flips a coin 21 times in a row and obtains 21 heads, one might rationally conclude a high probability of bias towards heads, and hence conclude that future flips of this coin are also highly likely to be heads. In fact, Bayesian inference can be used to show that when the long-run proportion of different outcomes are unknown but exchangeable (meaning that the random process from which they are generated may be biased but is equally likely to be biased in any direction) previous observations demonstrate the likely direction of the bias, such that the outcome which has occurred the most in the observed data is the most likely to occur again.
This is a different situation than the Gambler's Fallacy entirely because this exercise addresses a different question, even though all the parameters of the set may be the same. This exercise is geared to find the posterior probability that the coin being tossed is unfair (biased towards heads) given that we have observed multiple heads. The previous exercises are designed to estimate the posterior probability that a coin will be heads, given that previous tosses were tails and assuming that the coin is in fact fair, as stated in the assumptions of the section. The difference, using Bayes' Theorem on the current (1.) and previous (2.) examples:
1. p(unfair | heads^x) or the probability that a coin is unfair given you see x many heads
2. p(heads | fair, !heads^x) or the probability that you will get a heads given you already got x tails
The equation for 2. will always reduce to:
p(heads | fair,!heads^x) = p(heads)/p(!heads^x) * p(!heads^x | heads)
Now, since p(!heads^x) and p(heads) are mutually independent:
p(!heads^x | heads) = p(heads)*p(!heads^x)/p(heads)
Therefore, the equation cancels out everything except the term p(heads), leaving only:
p(heads | fair,!heads^x) = p(heads)
or the probability of getting a heads, given a fair coin that previously rolled x tails is equal to the probability of getting a heads. This equation is also readily adapted to disprove the Inverse Gambler's Fallacy using the same proof.
Performing the same operations on equation 1. yields: p(unfair | !heads^x) = p(unfair)/p(!heads^x) * p(!heads^x | unfair)
However, equation 1 does not simplify in the same manner because of the definition of p(!heads^x|unfair). Namely, because unfairness is defined by the presence of biased streaks. Therefore, the components !heads^x and unfairness are not mutually independent events. They share information; knowing one of the two tells you something about the other. Therefore, observing streaks of bias lend evidence to the notion that the coin itself is unfair. The accuracy of this prediction depends on the sample size.
This shows that the presented problem is not the same as Gambler's Fallacy (previous failures indicate increased probability of winning given a fair coin) or Inverse Gambler's Fallacy (a rare win occurring implies previous occurrences given a fair coin). This exercise is an estimation of the probability that the coin itself is indeed fair given a string of observances.
Psychology behind the fallacy
Amos Tversky and Daniel Kahneman proposed that the gambler's fallacy is a cognitive bias produced by a psychological heuristic called the representativeness heuristic . According to this view, "after observing a long run of red on the roulette wheel, for example, most people erroneously believe that black will result in a more representative sequence than the occurrence of an additional red," so people expect that a short run of random outcomes should share properties of a longer run, specifically in that deviations from average should balance out. When people are asked to make up a random-looking sequence of coin tosses, they tend to make sequences where the proportion of heads to tails stays close to 0.5 in any short segment more so than would be predicted by chance ; Kahneman and Tversky interpret this to mean that people believe short sequences of random events should be representative of longer ones .
The representativeness heuristic is also cited behind the related phenomenon of the clustering illusion, according to which people see streaks of random events as being non-random when such streaks are actually much more likely to occur in small samples than people expect
