On the Survival of Rats in the Slush Pile | Page 6

well proportioned, et cetera. In order to determine which of these rats are the strongest, we select a random sample, one that is truly representative of the rat population as a whole. We then put the sample group into a large vat and subject the rats to increasingly high levels of radiation.
As the levels of radiation increase, many of the rats will die. By the end of the experiment (unless you take it too far and kill them all) you will be left with a small number of survivors.
Taleb uses this hypothetical experiment, and its results, to illustrate a number of errors in thinking.
Flaws in the methodology
First, we need to think about the experimental procedure itself. Alert readers will already have noticed that the methodology of the experiment, as described for the purposes of this essay, is flawed.
The intention is to select the 'strongest' rats. But while the experiment will certainly reduce the numbers of rats, there is no guarantee that the survivors will be the strongest.
The surviving rats would only be the 'strongest' in the limited sense that they were the ones best able to withstand increasing doses of radiation. They might not be the strongest in terms of ability to survive without water, or ability to climb fences. The ability to withstand radiation might or might not be a useful characteristic in the real world.
Second, at least some of the survivor rats may have survived by pure chance. At the moment when the next blast of radiation was administered, a 'weak' rat may have been shielded from radiation by a 'strong' rat. Furthermore, there might be some variations in the way in which the radiation was distributed around the vat: in some spots (perhaps towards the rim) the rats might absorb less than in other spots.
In short, the design of the procedure leaves much to be desired; and this, we shall see, is the case with some procedures in publishing.
Survivorship bias
We have already noted the phenomenon which is known in statistics as survivorship bias; and history suggests that it is all too easy to fall prey to this lax way of thinking.
Survivorship bias involves mistaking what you see for what is really there. The tendency is for human beings to see only the survivors of some set of circumstances, and to ignore those who, for one reason or another, disappeared or dropped out as events proceeded. We often find ourselves earnestly discussing the traits in a cohort of survivors when, in truth, those traits are no different from those in a much larger population; if you consider the circumstances carefully it may be apparent that the survivors emerged as a result of sheer randomness, rather than through the possession of some special qualities.
It may be, if clear thinking is applied to any set of events, that those who dropped out, voluntarily, or were eliminated, perhaps as a result of chance, have at least as much to teach us about what is important and relevant as those who survived.
Nietzsche's error
Nietzsche is responsible for the aphorism 'What does not kill me makes me stronger.' If repeated, in a suitably solemn tone of voice, in front of a group of people who are aware that Nietzsche is a Big Name in Philosophy, this dictum may well induce nods of agreement. And you may sometimes hear people say, after a young person has had some kind of setback, 'Well, he will be all the better for the experience.' Once in a while it might even be true.
In general, however, Nietzsche's aphorism is nonsense. On the physical level, a car crash which brings you close to the point of death may leave you paralysed for life. So, although you are not actually dead, you are certainly not stronger than before. And in an emotional context, a bereavement which causes you seriously to contemplate suicide may, even if you do not succumb to the temptation, leave you lonely and depressed.
So it is with our rats. The rats which survived our experiment are by no means necessarily stronger. In reality, there is a good chance that they will be weaker. Radiation is not often good for you.
Taleb quotes a newspaper article about the Russian Mafia, which referred to the new generation of gangsters as being 'hardened by their Gulag experiences'. But, if any modern gangsters have indeed survived the Gulag, they are hardly likely to have been 'hardened'; the camps were not famous for providing fitnesstraining courses.
Despite these readily apparent flaws in Nietzsche's aphorism, there are circumstances in which peoplebehave as if it were true. We oftenassume that survivors of some intense selection process are stronger than those who were eliminated. We assume that survivors are necessarily the best of the cohort; whereas in reality the procedure may have been flawed and they may simply