Fooled by Randomness读后感摘抄

《Fooled by Randomness》是一本由Nassim Nicholas Taleb著作，Random House Trade Paperbacks出版的Paperback图书，本书定价：USD 18.00，页数：316，特精心收集的读后感，希望对大家能有帮助。

《Fooled by Randomness》读后感(一)：Ignored Common Sense --- Fat Tail

在這本書剛出來的時候讀過，是在某處上課時一個 fund of fund 的manager 推薦的。其實就是講 FAT TAIL 現象的。 但其中描述的確實都是被大家忽視的常識性現象。

後來見到了作者本人上課，一個看起來很不高興的人。他後來又寫了 BLACK SWAN，也是講fat tail 現象的，不過我很沒讀過。有時候很奇怪，爲什麽這么簡單的事情，他可以拿來寫這么厚的暢銷書。還四處演講，變的很炙手可熱。。。大概歐洲的知識分子都有這個本事吧。不同的是，聽說他確實拿這個賺了很多錢，在去年的市場大反轉中。

《Fooled by Randomness》读后感(二)：生活就是巧合

小概率事件，破坏性太强，要当心偏度(Skewness)

首先，封面字体设计很有意思，支离破碎但没有阅读障碍。

听说中文阅读是也一样的，即使颠倒语序也不么怎影响理解。

2005年出版的二手书

作者首先挑明，自己的座右铭就是爱惹太看重自己和自己知识的那些人。因为我们人类天生就是被设计成自我愚弄的，不然也不会有各种心理学上的偏差和偏误。交易会使人更加努力思考，所以作者自认无比幸运，有95%的空闲时间可以思考写作研究，他显得很享受。

以下是一些作者观点的摘录。多是主观断言和类似警句(证不证明也无所谓吧)。

01. 当今新闻类似瘟疫，世界变得复杂而我们的脑子被驯化得简化去看待问题。(噪音确实很多)

02. 事后诸葛亮看待过去总是那么确定，因为仅仅一个结果被观察到了。(盒子里的猫吗？)

03. 去除掉运气成分，每个人最终会回归到他的长期特性。以前运气好的被拉下，曾经运气有点背的最终会提升。(回归均值，貌似人和股票都适用)

04. 有的投资者，假如一直盯着标的价格分分钟都在看，那作者要赚的就是他们手里的钱。作者也尽量避免获取到过量信息，情愿读读诗歌，而不去看新闻，以至于他的员工认为他住在另外一个星球。

05. 理性和科学不用运用到生活的方方面面，只要用到威胁伤害我们生存的地方就可以了。

06. 成功交易者的脑海中只该去做别人不做的事情(作者教不了过多细节)。失败的交易者是，亏钱就不舒服，直到亏到比想象(不在计划中)中要多而最后毁灭掉信心。

07. 固有观念对交易者，科学家，甚至每一个人来说，都不是好事情。

08. 交易者或者投资者还需要理清一个概念，就是概率和收益期望的区别。作者本身青睐小亏大赚，不频繁地下注黑天鹅。他对市场不感兴趣，不做预测。他也越发坚信多数的计量经济学是无用的(至少对他)，金融工程也是伪科学含量很高。(不懂)

09. 极端经验主义，具有强力竞争性，以及缺乏逻辑推理能力是个死亡组合。

10. 他非常推崇波普尔理论，说Popper对科学的观点是，别把它看的太当回事了，类比到交易者也是。以及Popper提出的开放社会就是没有永恒真理的存在。(也不了解，听朋友说他觉得Popper做哲学研究不行)作者最后说Popper本人不够波普尔主义。

11. 用过去经验下赌注没问题(不然也没有别的什么方法？)，但不能只用过去数据的经验来确保错误的代价是可控的。

12. 猴子打字出荷马史诗的故事(直觉认为并非十分恰当)，过去的成绩预测未来的成绩没有多少相关性。一只黑天鹅可以推翻一个理论，也就是不可证真，只能证伪。

13. 我们被训练成只会关注利用眼前信息，而忽略看不见的那部分，就比如幸存者偏差。(这个实在是太普遍了)

14. 做市商不搞大的，而依靠客户数多，统计上占优势。他们就是这个行业里的牙医，钻一颗牙，拿一份钱。

15. 对人心智的研究一直还在进行中，思考带来的问题就是，它会给人以错觉和幻象，而且太耗体力了。AI先驱天才通才司马贺提出有限理性，人脑的短处就是边界线明显，不可能穷尽信息和计算去获取所谓最优解(差不多差不多就行了？应该是最终结局)。

16. 举出《快思慢想》(Thinking, Fast and Slow, 2011年出版，晚于此书)作者卡尼曼和特沃斯基研究的例子，两种思考模式。快速、直觉且情绪化vs较慢、较具计划性且更仰赖逻辑。(没人能完全对抗情绪)

16. 人们容易高估自己掌握的知识同时低估自己犯错的可能性。作者认为每个人都需要学点统计，特别需要了解置信区间这个概念。(自己也几乎完全忘记了，只记得贝叶斯是非常好用的统计方法)明星公司LTCM有诺奖得主创办人加持，最后却是破产的结局。

17. 不带一丝尴尬的修改自己先前的观点，是交易者的最佳特质。每天让自己的思维都从白纸一张开始。

书中道理多还是反复的讲述，引述另一个交易者的话语来概况。

三句话总结了。但是，黑客帝国里Morpheus却说。。。

Morpheus

Nassim Nicholas Taleb好像讲过，书要看第二遍时才会变得更好。假如有空再去看看，里面依然好多不懂的单词和长句，还有引申阅读。

如作者在维根斯坦的尺子这一小结里说，对一本书的评价不论好与坏，更多的是在展现读者而不是作者。以上是自己能看到的一些东西。NNT他说自己喜欢写书不喜欢写论文，非常认同，一本书，或者一个笔记，让自己满意了就好，没压力的东西。

《Fooled by Randomness》读后感(三)：Lady Fortuna has no mercy.

其實是n年前讀的2005年這一版，借用2008增訂版的讀書筆記，填上這個坑。推薦還未開始讀的各位直接看2008增訂版。

https://book.douban.com/subject/3303293/

Lady Fortuna has no mercy. #書# 2001《Fooled by Randomness》7/10 作者:Nassim Nicholas Taleb 出版社: Random House 副标题: The Hidden Role of Chance in Life and in the Markets 出版年: 2008-10 页数: 368 強迫症為了把幾年前的書筆記補上，再次拜讀，不過改讀2008年的增訂版。作者把一些理論引用和學者的名字補全，讀起來“好像”可信度又增加了一點。 因為是第二次讀，基本概念都知道，翻書很快，更像是完成一個任務，而不是思考讀到的內容。 黑天鵝掃一掃翅膀，之前的努力可能會徒勞無功，Lady Fortuna has no mercy，但是The only article Lady Fortuna has no control over is your behavior。Heroes are heroes because they are heroic in behavior, not because they won or lost. 大雨滂沱中笑對人生，我們能做的，僅此而已。 以下讀書筆記更多是摘錄原文和簡短總結章節方便自我記憶。 目錄 本書分成三篇。第一篇省思梭倫（Solon）的警語，他對幾件稀有事件所下的斷語，成為我終生的座右銘。我們會談到發生和未發生的歷史。第二篇談我在充滿隨機性的事業生涯中，碰到以及因此受害的許多概率偏差，這些隨機問題到現在還在愚弄我。第三篇總結指出要泯滅我們的本性或許不容易，我們需要的是一些小技巧，而不是冠冕堂皇的大道理。同樣，前人的謀略對我們大有助益。 This book is the synthesis of, on one hand, the no-nonsense practitioner of uncertainty whospen this professional life trying to resist being fooled by randomness and trick the emotions associated with probabilistic outcomes and, on the other, the aesthetically obsessed, literature-loving human being willing to be fooled by any form of nonsense that is polished, refined, original, and tasteful. It is as if there were two planets: the one in which we actually live and the one, considerably more deterministic, on which people are convinced we live. It is as simple as that: Past events will always look less random than they were (it is called the hindsight bias). (in the real world one has to guess the problem more than the solution). In this book, considering that alternative outcomes could have taken place, that the world could have been different, is the core of probabilistic thinking. Furthermore, the kind of luck in finance is of the kind that nobody understands but most operators think they understand, which provides us a magnification of the biases. ONE: IF YOU’RE SO RICH, WHY AREN’T YOU SO SMART？ An illustration of the effect of randomness on social pecking order and jealousy, through two characters of opposite attitudes. On the concealed rare event. How things in modern life may change rather rapidly, except, perhaps, in dentistry. TWO: A BIZARRE ACCOUNTING METHOD On alternative histories, a probabilistic view of the world, intellectual fraud, and the randomness wisdom of a Frenchman with steady bathing habits. How journalists are bred to not understand random series of events. Beware borrowed wisdom: How almost all great ideas concerning random outcomes are against conventional sapience. On the difference between correctness and intelligibility. THREE: A MATHEMATICAL MEDITATION ON HISTORY On Monte Carlo simulation as a metaphor for understanding a sequence of random historical events. On randomness and artificial history. Age is beauty, almost always, and the new and the young are generally toxic. Send your history professor to an introductory class on sampling theory. FOUR: RANDOMNESS, NONSENSE, AND THE SCIENTIFIC INTELLECTUAL On extending the Monte Carlo generator to produce artificial thinking and compare it with rigorous nonrandom constructs. The science wars enter the business world. Why the aesthete in me loves to be fooled by randomness. FIVE: SURVIVAL OF THE LEAST FIT—CAN EVOLUTION BE FOOLED BY RANDOMNESS？ A case study on two rare events. On rare events and evolution. How “Darwinism” and evolution are concepts that are misundderstood in the nonbiological world. Life is not continuous. How evolution will be fooled by randomness. A prolegomenon for the problem of induction. SIX: SKEWNESS AND ASYMMETRY We introduce the concept of skewness: Why the terms “bull” and “bear” have limited meaning outside of zoology. A vicious child wrecks the structure of randomness. An introduction to the problem of epistemic opacity. The penultimate step before the problem of induction. SEVEN: THE PROBLEM OF INDUCTION On the chromodynamics of swans. Taking Solon’s warning into some philosophical territory. How Victor Niederhoffer taught me empiricism; I added deduction. Why it is not scientific to take science seriously. Soros promotes Popper. That bookstore on Eighteenth Street and Fifth Avenue. Pascal’s wager. EIGHT: TOO MANY MILLIONAIRES NEXT DOOR Three illustrations of the survivorship bias. Why very few people should live on Park Avenue. The millionaire next door has very flimsy clothes. An overcrowding of experts. NINE: IT IS EASIER TO BUY AND SELL THAN FRY AN EGG Some technical extensions of the survivorship bias. On the distribution of “coincidences” in life. It is preferable to be lucky than competent (but you can be caught). The birthday paradox. More charlatans (and more journalists). How the researcher with work ethics can find just about anything in data. On dogs not barking. TEN: LOSER TAKES ALL—ON THE NONLINEARITIES OF LIFE The nonlinear viciousness of life. Moving to Bel Air and acquiring the vices of the rich and famous. Why Microsoft’s Bill Gates may not be the best in his business (but please do not inform him of such a fact). Depriving donkeys of food. ELEVEN: RANDOMNESS AND OUR MIND: WE ARE PROBABILITY BLIND On the difficulty of thinking of your vacation as a linear combination of Paris and the Bahamas. Nero Tulip may never ski in the Alps again. Do not ask bureaucrats too many questions. A Brain Made in Brooklyn. We need Napoleon. Scientists bowing to the King of Sweden. A little more on journalistic pollution. Why you may be dead by now. TWELVE: GAMBLERS’ TICKS AND PIGEONS IN A BOX On gamblers’ ticks crowding up my life. Why bad taxi-cab English can help you make money. How I am the fool of all fools, except that I am aware of it. Dealing with my genetic unfitness. No boxes of chocolate under my trading desk. THIRTEEN: CARNEADES COMES TO ROME: ON PROBABILITY AND SKEPTICISM Cato the censor sends Carneades packing. Monsieur de Norpois does not remember his old opinions. Beware the scientist. Marrying ideas. The same Robert Merton putting the author on the map. Science evolves from funeral to funeral. FOURTEEN: BACCHUS ABANDONS ANTONY Montherlant’s death. Stoicism is not the stiff upper lip, but the illusion of victory of man against randomness. It is so easy to be heroic. Randomness and personal elegance. This book is about luck disguised and perceived as nonluck (that is, skills) and, more generally, randomness disguised and perceived as non-randomness (that is, determinism). It manifests itself in the shape of the lucky fool, defined as a person who benefited from a disproportionate share of luck but attributes his success to some other, generally very precise, reason. Symbolism is the child of our inability and unwillingness to accept randomness; we give meaning to all manner of shapes; we detect human figures in inkblots. Disturbingly, science has only recently been able to handle randomness (the growth in available information has been exceeded only by the expansion of noise). Probability theory is a young arrival in mathematics; probability applied to practice is almost nonexistent as a discipline. In addition we seem to have evidence that what is called “courage” comes from an underestimation of the share of randomness in things rather than the more noble ability to stick one’s neck out for a given belief. economic life presents the best (and most entertaining) laboratory for the understanding of these differences. For it is the area of human undertaking where the confusion is greatest and its effects the most pernicious. For instance, we often have the mistaken impression that a strategy is an excellent strategy, or an entrepreneur a person endowed with “vision,” or a trader a talented trader, only to realize that 99.9% of their past performance is attributable to chance, and chance alone. it is our inability to think critically—we may enjoy presenting conjectures as truth. It is our nature. Our mind is not equipped with the adequate machinery to handle probabilities; such infirmity even strikes the expert, sometimes just the expert. We are faulty and there is no need to bother trying to correct our flaws. We are so defective and so mismatched to our environment that we can just work around these flaws. As much as you believe in the “keep-it-simple-stupid” it is the simplification that is dangerous. The book is composed of three parts. The first is an introspection into Solon’s warning, as his outburst on rare events became my lifelong motto. In it we meditate on visible and invisible histories and the elusive property of rare events (black swans). The second presents a collection of probability biases I encountered (and suffered from) in my career in randomness—ones that continue to fool me. The third illustrates my personal jousting with my biology and concludes the book with a presentation of a few practical (wax in my ears) and philosophical (stoicism) aids. Before the “enlightenment” and the age of rationality, there was in the culture a collection of tricks to deal with our fallibility and reversals of fortunes. The elders can still help us with some of their ruses. Part I • SOLON’SWARNING Skewness, Asymmetry, Induction 如果失敗的代價過於沉重、難以承受，那麼這件事成功的概率有多高根本無關緊要。 Part I is concerned with the degree to which a situation may yet, in the course of time, suffer change. which came with the help of luck could be taken away by luck (and often rapidly and unexpectedly at that). things that come with little help from luck are more resistant to randomness. Solon also had the intuition of a problem that has obsessed science for the past three centuries. It is called the problem of induction. I call it in this book the black swan or the rare event. Solon even understood another linked problem, which I call the skewness issue; it does not matter how frequently something succeeds if failure is too costly to bear. One • IF YOU’RE SO RICH,WHY AREN’T YOU SO SMART？ 要做短暫的輸家還是長期的贏家？後見之明來看當然很容易選擇，但深陷其中，謹慎的交易員被好運的交易員壓在地上摩擦，要扛住每天起床的自我懷疑，咬牙堅持自己投資理念和人生哲學，這除了自我的修養，何嘗不也是一種“能不能撐到證明自己正確那一刻”的運氣？ Nero’s temperament is such that he does not mind losing small change. “I love taking small losses,”he says. “I just need my winners to be large.” In no circumstances does he want to be exposed to those rare events, like panics and sudden crashes, that wipe a trader out in a flash. To the contrary, he wants to benefit from them. Trading forces someone to think hard; those who merely work hard generally lose their focus and intellectual energy. In addition, they end up drowning in randomness; work ethics, Nero believes, draw people to focus on noise rather than the signal Intellectual contempt does not control personal envy. This high-yield market resembles a nap on a railway track. One afternoon, the surprise train would run you over. You make money every month for a long time, then lose a multiple of your cumulative performance in a few hours. Scientists found out that serotonin, a neurotransmitter, seems to command a large share of our human behavior. It sets a positive feedback, the virtuous cycle, but, owing to an external kick from randomness, can start a reverse motion and cause a vicious cycle. It has been shown that monkeys injected with serotonin will rise in the pecking order, which in turn causes an increase of the serotonin level in their blood—until the virtuous cycle breaks and starts a vicious one (during the vicious cycle failure will cause one to slide in the pecking order, causing a behavior that will bring about further drops in the pecking order). Likewise, an increase in personal performance (regardless of whether it is caused deterministically or by the agency of Lady Fortuna) induces a rise of serotonin in the subject, itself causing an increase of what is commonly called “leadership” ability. One is “on a roll.” Some imperceptible changes in deportment, like an ability to express oneself with serenity and confidence, make the subject look credible—as if he truly deserved the shekels. Randomness will be ruled out as a possible factor in the performance, until it rears its head once again and delivers the kick that will induce the downward spiral. Almost no one can conceal his emotions. Behavioral scientists believe that one of the main reasons why people become leaders is not from what skills they seem to possess, but rather from what extremely superficial impression they make on others through hardly perceptible physical signals—what we call today “charisma,” for example. The biology of the phenomenon is now well studied under the subject heading “social emotions.” such physical manifestations of one’s performance in life, just like an animal’s dominant condition, can be used for signaling: It makes the winners seem easily visible, which is efficient in mate selection. Two • A BIZARRE ACCOUNTING METHOD “另類歷史”因為邏輯和直覺背道而馳，我們經常看不到，甚至看到卻忽略。特別是“結果導向”的人，無法思考或接受並未發生的“另類歷史”。我們最喜歡的以成敗論英雄，是局限在隨機發生的單一歷史裡面。假如加入“另類歷史”綜合評論，答案就不是那麼黑白分明了！ one cannot judge a performance in any given field (war, politics, medicine, investments) by the results, but by the costs of the alternative (i.e., if history played out in a different way). Such substitute courses of events are called alternative histories. Clearly, the quality of a decision cannot be solely judged based on its outcome, but such a point seems to be voiced only by people who fail (those who succeed attribute their success to the quality of their decision). (certainty is something that is likely to take place across the highest number of different alternative histories; uncertainty concerns events that should take place in the lowest number of them). In philosophy, there has been considerable work on the subject starting with Leibniz’ idea of possible worlds. For Leibniz, God’s mind included an infinity of possible worlds, of which he selected just one. In physics, there is the many-world interpretation in quantum mechanics (associated with the works of Hugh Everett in 1957) which considers that the universe branches out treelike at each juncture; what we are living now is only one of these many worlds. Finally, in economics: Economists studied (perhaps unwittingly) some of the Leibnizian ideas with the possible “states of nature” pioneered by Kenneth Arrow and Gerard Debreu. This analytical approach to the study of economic uncertainty is called the “state space” method—it happens to be the cornerstone of neoclassical economic theory and mathematical finance. A simplified version is called “scenario analysis,” the series of “what-ifs” used in, After a few dozen tries, one forgets about the existence of a bullet, under a numbing false sense of security. The point is dubbed in this book the black swan problem, which we cover in Chapter 7, as it is linked to the problem of induction, a problem that has kept a few thinkers awake at night. It is also related to a problem called denigration of history, as gamblers, investors, and decision-makers feel that the sorts of things that happen to others would not necessarily happen to them. Very rarely is the generator visible to the naked eye. One is thus capable of unwittingly playing Russian roulette—and calling it by some alternative “low risk” name. We see the wealth being generated, never the processor, a matter that makes people lose sight of their risks, and never consider the losers. The game seems terribly easy and we play along carelessly. Finally, there is an ingratitude factor in warning people about something abstract (by definition anything that did not happen is abstract). The degree of resistance to randomness in one’s life is an abstract idea, part of its logic counterintuitive, and, to confuse matters, its realizations nonobservable. I thus view people distributed across two polar categories: On one extreme, those who never accept the notion of randomness; on the other, those who are tortured by it. Realism can be punishing. Probabilistic skepticism is worse. It is difficult to go about life wearing probabilistic glasses, as one starts seeing fools of randomness all around, in a variety of situations—obdurate in their perceptional illusion. Heroes won and lost battles in a manner that was totally independent of their own valor; their fate depended upon totally external forces, generally the explicit agency of the scheming gods (not devoid of nepotism). Heroes are heroes because they are heroic in behavior, not because they won or lost. Such tendency to make and unmake prophets based on the fate of the roulette wheel is symptomatic of our ingrained inability to cope with the complex structure of randomness prevailing in the modern world. Mixing forecast and prophecy is symptomatic of randomness-foolishness Clearly, this idea of alternative history does not make intuitive sense, which is where the fun begins. For starters, we are not wired in a way to understand probability, researchers of the brain believe that mathematical truths make little sense to our mind, particularly when it comes to the examination of random outcomes. Most results in probability are entirely counterintuitive; people do not like to insure against something abstract; the risk that merits their attention is always something vivid. It is a fact that our brain tends to go for superficial clues when it comes to risk and probability, these clues being largely determined by what emotions they elicit or the ease with which they come to mind. In addition to such problems with the perception of risk, it is also a scientific fact, and a shocking one, that both risk detection and risk avoidance are not mediated in the “thinking” part of the brain but largely in the emotional one (the “risk as feelings” theory). The consequences are not trivial: It means that rational thinking has little, very little, to do with risk avoidance. Much of what rational thinking seems to do is rationalize one’s actions by fitting some logic to them. Beware the confusion between correctness and intelligibility. Part of conventional wisdom favors things that can be explained rather instantly and “in a nutshell”—in many circles it is considered law. Einstein’s remark that common sense is nothing but a collection of misconceptions acquired by age eighteen. What sounds intelligent in a conversation or a meeting, or, particularly, in the media, is suspicious. Three • A MATHEMATICAL MEDITATION ON HISTORY 蒙特卡羅仿真法，人為虛擬另類歷史。“在歷史之下求和”（summingunderhistories）——即是同時參考未實現的歷史，觀察各種結果的分佈情況——才能夠更好地理解和認清現實這個“唯一”歷史的真面目。 歷史不像其他“硬科學”，可以用實驗來驗證。但是蒙特卡羅提供了一個驗證可能性。概率數學說的遍曆性（ergodicity）說明在某些情況下，眾多非常長的樣本路徑（samplepath）最後看起來會彼此相似。而一條非常、非常長的樣本路徑的性質，類似於許多較短路徑平均值的蒙特·卡羅性質。 資訊爆炸帶來的噪音不僅僅淹沒了有效信號，也在不斷毒害所有人情緒，有百害而無一利。系統化地篩選新觀念、咨詢或方法，化繁為簡，才是王道。 Monte Carlo methods, in brief, consist of creating artificial history using the following concepts. First, consider the sample path. The invisible histories have a scientific name, alternative sample paths, a name borrowed from the field of mathematics of probability called stochastic processes. The notion of path, as opposed to outcome, indicates that it is not a mere MBA-style scenario analysis, but the examination of a sequence of scenarios along the course of time. The word sample stresses that one sees only one realization among a collection of possible ones. Now, a sample path can be either deterministic or random, which brings the next distinction. A random sample path, also called a random run, is the mathematical name for such a succession of virtual historical events, starting at a given date and ending at another, except that they are subjected to some varying level of uncertainty. However, the word random should not be mistaken for equiprobable (i.e., having the same probability). Some outcomes will give a higher probability than others. Stochastic processes refer to the dynamics of events unfolding with the course of time. Stochastic is a fancy Greek name for random. This branch of probability concerns itself with the study of the evolution of successive random events—one could call it the mathematics of history. The key about a process is that it has time in it. The glamorous reference to Monte Carlo indicates the metaphor of simulating the random events in the manner of a virtual casino. One sets conditions believed to resemble the ones that prevail in reality, and launches a collection of simulations around possible events. Monte Carlo simulation methods were pioneered in martial physics in the Los Alamos laboratory during the A-bomb preparation. They became popular in financial mathematics in the 1980s, particularly in the theories of the random walk of asset prices. Indeed, probability is an introspective field of inquiry, as it affects more than one science, particularly the mother of all sciences: that of knowledge. It is impossible to assess the quality of the knowledge we are gathering without allowing a share of randomness in the manner it is obtained and cleaning the argument from the chance coincidence that could have seeped into its construction. In science, probability and information are treated in exactly the same manner. I reckon that I outgrew the desire to generate random runs every time I want to explore an idea—but by dint of playing with a Monte Carlo engine for years I can no longer visualize a realized outcome without reference to the nonrealized ones. I call that “summing under histories,” borrowing the expression from the colorful physicist Richard Feynman who applied such methods to examine the dynamics of subatomic particles. Learning from history does not come naturally to us humans, a fact that is so visible in the endless repetitions of identically configured booms and busts in modern markets. it is not natural for us to learn from history. We have enough clues to believe that our human endowment does not favor transfers of experience in a cultural way but through selection of those who bear some favorable traits. In some respects we do not learn from our own history. Several branches of research have been examining our inability to learn from our own reactions to past events: For example, people fail to learn that their emotional reactions to past experiences (positive or negative) were short-lived—yet they continuously retain the bias of thinking that the purchase of an object will bring long-lasting, possibly permanent, happiness or that a setback will cause severe and prolonged distress (when in the past similar setbacks did not affect them for very long and the joy of the purchase was short-lived). Experts call one manifestation of such denigration of history historical determinism. In a nutshell we think that we would know when history is made; we believe that people who, say, witnessed the stock market crash of 1929 knew then that they lived an acute historical event and that, should these events repeat themselves, they too would know about such facts. Life for us is made to resemble an adventure movie, as we know ahead of time that something big is about to happen. It is hard to imagine that people who witnessed history did not know at the time how important the moment was. Somehow all respect we may have for history does not translate well into our treatment of the present. When you look at the past, the past will always be deterministic, since only one single observation took place. Our mind will interpret most events not with the preceding ones in mind, but the following ones. Imagine taking a test knowing the answer. While we know that history flows forward, it is difficult to realize that we envision it backward. Our minds are not quite designed to understand how the world works, but, rather, to get out of trouble rapidly and have progeny. If they were made for us to understand things, then we would have a machine in it that would run the past history as in a VCR, with a correct chronology, and it would slow us down so much that we would have trouble operating. Psychologists call this overestimation of what one knew at the time of the event due to subsequent information the hindsight bias, the “I knew it all along” effect. A mistake is not something to be determined after the fact, but in the light of the information until that point. A more vicious effect of such hindsight bias is that those who are very good at predicting the past will think of themselves as good at predicting the future, and feel confident about their ability to do so. Unlike many “hard” sciences, history cannot lend itself to experimentation. But somehow, overall, history is potent enough to deliver, on time, in the medium to long run, most of the possible scenarios, and to eventually bury the bad guy. Bad trades catch up with you, it is frequently said in the markets. Mathematicians of probability give that a fancy name: ergodicity. It means, roughly, that (under certain conditions) very long sample paths would end up resembling each other. The properties of a very, very long sample path would be similar to the Monte Carlo properties of an average of shorter ones. Mathematically, progress means that some new information is better than past information, not that the average of new information will supplant past information, which means that it is optimal for someone, when in doubt, to systematically reject the new idea, information, or method. Clearly and shockingly, always. The problem with information is not that it is diverting and generally useless, but that it is toxic. his 1981 paper may be the first mathematically formulated introspection on the manner in which society in general handles information. Shiller made his mark with his 1981 paper on the volatility of markets, where he determined that if a stock price is the estimated value of “something” (say the discounted cash flows from a corporation), then market prices are way too volatile in relation to tangible manifestations of that “something” (he used dividends as proxy). Prices swing more than the fundamentals they are supposed to reflect, they visibly overreact by being too high at times (when their price overshoots the good news or when they go up without any marked reason) or too low at others. The volatility differential between prices and information meant that something about “rational expectation” did not work. (Prices did not rationally reflect the long-term value of securities and were overshooting in either direction.) Markets had to be wrong. Shiller then pronounced markets to be not as efficient as established by financial theory (efficient markets meant, in a nutshell, that prices should adapt to all available information in such a way as to be totally unpredictable to us humans and prevent people from deriving profits). This conclusion set off calls by the religious orders of high finance for the destruction of the infidel who committed such apostasy. it will be the oldest, simply because older people have been exposed longer to the rare event and can be, convincingly, more resistant to it. The modern Greek poet C. P. Cavafy wrote a piece in 1915 after Philostratus’ adage “For the gods perceive things in the future, ordinary people things in the present, but the wise perceive things about to happen.” In their intense meditation the hidden sound of things approaching reaches them and they listen reverently while in the street outside the people hear nothing at all. Finally, this explains why people who look too closely at randomness burn out, their emotions drained by the series of pangs they experience. Regardless of what people claim, a negative pang is not offset by a positive one (some psychologists estimate the negative effect for an average loss to be up to 2.5 the magnitude of a positive one); it will lead to an emotional deficit. My problem is that I am not rational and I am extremely prone to drown in randomness and to incur emotional torture. I am aware of my need to ruminate on park benches and in cafés away from information, but I can only do so if I am somewhat deprived of it. My sole advantage in life is that I know some of my weaknesses, mostly that I am incapable of taming my emotions facing news and incapable of seeing a performance with a clear head. Silence is far better. Four • RANDOMNESS, NONSENSE, AND THE SCIENTIFIC INTELLECTUAL 文學和科學最大區別是文學語言的模糊不清和想象空間，所以蒙特卡洛仿真法可以產生一部文學作品，但卻無法生成一篇科學作品。隨機的混沌給予個人對美學隨意理解的自由度，這是生活的有趣點，無需理性。但面對市場和受隨機性影響的事物時，必須要務實和理智！ In their view, literary thinking could conceal plenty of well-sounding non-sense. They wanted to strip thinking from rhetoric (except in literature and poetry where it properly belonged). The way they introduced rigor into intellectual life is by declaring that a statement could fall only into two categories: deductive, like “2 +2 =4,” i.e., incontrovertibly flowing from a precisely defined axiomatic framework (here the rules of arithmetic), or inductive, i.e., verifiable in some manner (experience, statistics, etc.), like “it rains in Spain” or “New Yorkers are generally rude.” Anything else was plain unadulterated hogwash (music could be a far better replacement to metaphysics). Needless to say that inductive statements may turn out to be difficult, even impossible, to verify, as we will see with the black swan problem—and empiricism can be worse than any other form of hogwash when it gives someone confidence You can sometimes replicate something that can be mistaken for a literary discourse with a Monte Carlo generator but it is not possible randomly to construct a scientific one. Rhetoric can be constructed randomly, but not genuine scientific knowledge. This is the application of Turing’s test of artificial intelligence, except in reverse. Some aesthetic forms appeal to something in our biology, whether or not they originate in random associations or plain hallucination. Something in our human genes is deeply moved by the fuzziness and ambiguity of language; language is potent in bringing pleasure and solace. Testing its intellectual validity by translating it into simple logical arguments would rob it of a varying degree of its potency, sometimes excessively; nothing can be more bland than translated poetry. A convincing argument of the role of language is the existence of surviving holy languages, uncorrupted by the no-nonsense tests of daily use. Semitic religions, that is Judaism, Islam, and original Christianity understood the point: Keep a language away from the rationalization of daily use and avoid the corruption of the vernacular. Four decades ago, the Catholic church translated the services and liturgies from Latin to the local vernaculars; one may wonder if this caused a drop in religious beliefs. Suddenly religion subjected itself to being judged by intellectual and scientific, without the aesthetic, standards. We do not need to be rational and scientific when it comes to the details of our daily life—only in those that can harm us and threaten our survival. Modern life seems to invite us to do the exact opposite; become extremely realistic and intellectual when it comes to such matters as religion and personal behavior, yet as irrational as possible when it comes to matters ruled by randomness Five • SURVIVAL OF THE LEAST FIT—CAN EVOLUTION BE FOOLED BY RANDOMNESS？ 達爾文學說的適應性適用於在非常長的期間內發展的物種，而不是短期觀察到的現象。“在任何一個時間點，賺錢最多的交易員往往是最差的交易員”這個橫斷面問題（cross-sectional problem）無法證明不適者可以生存，它只是時間長度不夠導致的誤解。 And, at any point in time, the richest traders are often the worst traders. This, I will call the cross-sectional problem: At a given time in the market, the most successful traders are likely to be those that are best fit to the latest cycle. A tendency to get married to positions. The tendency to change their story. No precise game plan ahead of time as to what to do in the event of losses. Absence of critical thinking expressed in absence of revision of their stance with “stop losses.” Denial. When the losses occurred there was no clear acceptance of what had happened. Negative mutations are traits that survive in spite of being worse, from the reproductive fitness standpoint, than the ones they replaced. However, they cannot be expected to last more than a few generations (under what is called temporal aggregation). Darwinian fitness applies to species developing over a very long time, not observed over a short term—time aggregation eliminates much of the effects of randomness; things (I read noise) balance out over the long run, as people say. on average, animals will be fit, but not every single one of them, and not at all times. Just as an animal could have survived because its sample path was lucky, the “best” operators in a given business can come from a subset of operators who survived because of overfitness to a sample path—a sample path that was free of the evolutionary rare event. One vicious attribute is that the longer these animals can go without encountering the rare event, the more vulnerable they will be to it. We said that should one extend time to infinity, then, by ergodicity, that event will happen with certainty—the species will be wiped out! For evolution means fitness to one and only one time series, not the average of all the possible environments. Six • SKEWNESS AND ASYMMETRY 我們所受到的教育，都是理想化簡單化的方程式和對稱曲線，它們在現實是不存在的。正如我們從歷史數據得出的經驗和模型，也是理想化簡單化，它們可能湊巧預測未來，但更可能是誤導對將來的判斷——隨機才是未來唯一的答案。對於金融市場來說，稀有事件發生的可能性多大並不重要，而是它最終可以帶來多少利潤。 特別要警惕“這種事情以前從未發生”之類的話，如果去看更寬廣的歷史，我們會發現，某個地方從來沒有發生的事，最後往往會發生——經驗論對於無限長的歷史，沒有意義。 expected and median do not mean the same thing at all. Asymmetric odds means that probabilities are not 50% for each event, but that the probability on one side is higher than the probability on the other. Asymmetric outcomes mean that the payoffs are not equal. How could people miss such a point？ Why do they confuse probability and expectation, that is, probability and probability times the payoff？ Mainly because much of people’s schooling comes from examples in symmetric environments, like a coin toss, where such a difference does not matter. In fact, the so-called bell curve that seems to have found universal use in society is entirely symmetric. Alas, investors and businesses are not paid in probabilities; they are paid in dollars. Accordingly, it is not how likely an event is to happen that matters, it is how much is made when it happens that should be the consideration. How frequent the profit is irrelevant; it is the magnitude of the outcome that counts. But we are not sure that the world we live in is well charted. We will see that the judgment derived from the analysis of these past attributes may on occasion be relevant. But it may be meaningless; it could on occasion mislead you and take you in the opposite direction. Sometimes market data becomes a simple trap; it shows you the opposite of its nature, simply to get you to invest The problem is that we read too much into shallow recent history, with statements like “this has never happened before,” but not from history in general (things that never happened before in one area tend eventually to happen). In other words, history teaches us that things that never happened before do happen. It can teach us a lot outside of the narrowly defined time series; the broader the look, the better the lesson. In other words, history teaches us to avoid the brand of naive empiricism that consists of learning from casual historical facts. The closer he observes his performance, the more pain he will experience owing to the greater variability at a higher resolution. Accordingly investors, merely for emotional reasons, will be drawn into strategies that experience rare but large variations. It is called pushing randomness under the rug. Psychologists recently found out that people tend to be sensitive to the presence or absence of a given stimulus rather than its magnitude. This implies that a loss is first perceived as just a loss, with further implications later. The same with profits. The agent would prefer the number of losses to be low and the number of gains to be high, rather than optimizing the total performance. the more information you have, the more you are confident about the outcome. Now the problem: by how much？ Common statistical method is based on the steady augmentation of the confidence level, in nonlinear proportion to the number of observations. That is, for an n times increase in the sample size, we increase our knowledge by the square root of n. Suppose I am drawing from an urn containing red and black balls. My confidence level about the relative proportion of red and black balls after 20 drawings is not twice the one I have after 10 drawings; it is merely multiplied by the square root of 2 (that is, 1.41). Where statistics becomes complicated, and fails us, is when we have distributions that are not symmetric, like the urn above. If there is a very small probability of finding a red ball in an urn dominated by black ones, then our knowledge about the absence of red balls will increase very slowly—more slowly than at the expected square root of n rate. On the other hand, our knowledge of the presence of red balls will dramatically improve once one of them is found. This asymmetry in knowledge is not trivial; But there is even worse news. In some cases, if the incidence of red balls is itself randomly distributed, we will never get to know the composition of the urn. This is called “the problem of stationarity.” Think of an urn that is hollow at the bottom. As I am sampling from it, and without my being aware of it, some mischievous child is adding balls of one color or another. My inference thus becomes insignificant. I may infer that the red balls represent 50% of the urn while the mischievous child, hearing me, would swiftly replace all the red balls with black ones. This makes much of our knowledge derived through statistics quite shaky. If rational traders detect a pattern of stocks rising on Mondays, then, immediately such a pattern becomes detectable, it would be ironed out by people buying on Friday in anticipation of such an effect. There is no point searching for patterns that are available to everyone with a brokerage account; once detected, they would be self-canceling. Seven • THE PROBLEM OF INDUCTION 世界上只有兩類理論： 第一，經過檢驗並以適當的方法予以駁斥、已知為錯誤的理論，稱之為已被證偽（falsified）。 第二，尚未得知是否錯誤或者尚未遭否證，但將來有可能被證明為錯誤的理論。波普爾提出，我們不可能驗證理論。一味追求驗證，造成的傷害多於其他。這句話非常正確，科學本來就是在不斷地證明、推翻、再證明的過程中進步。 而根據這個道理進行投資，關鍵點在於認賠止損（stoploss）——確定犯錯時成本有限，同時在執行操作策略之前就知道哪些事件會證明自己的推測錯誤，並預先做準備；證實犯錯後，馬上結束操作。 In his Treatise on Human Nature, the Scots philosopher David Hume posed the issue in the following way (as rephrased in the now famous black swan problem by John Stuart Mill): No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion. any “testable” statement should be tested, as our minds make plenty of empirical mistakes when relying on vague impressions. A testable statement is one that can be broken down into quantitative components and subjected to statistical examination. Consider the following statements: Statement A: No swan is black, because I looked at four thousand swans and found none. Statement B: Not all swans are white. I cannot logically make statement A, no matter how many successive white swans I may have observed in my life and may observe in the future (except, of course, if I am given the privilege of observing with certainty all available swans). It is, however, possible to make Statement B merely by finding one single counterexample. Maximizing the probability of winning does not lead to maximizing the expectation from the game when one’s strategy may include skewness, i.e., a small chance of large loss and a large chance of a small win. To conclude, extreme empiricism, competitiveness, and an absence of logical structure to one’s inference can be a quite explosive combination. although Soros did not deliver anything meaningful in his writings, he knew how to handle randomness, by keeping a critical open mind and changing his opinions with minimal shame (which carries the side effect of making him treat people like napkins). He walked around calling himself fallible, but was so potent because he knew it while others had loftier ideas about themselves. I suddenly felt financially insecure and feared becoming an employee of some firm that would turn me into a corporate slave with “work ethics” (whenever I hear work ethics I interpret inefficient mediocrity). I needed the backing of my bank account so I could buy time to think and enjoy life. The last thing I needed was immediate philosophizing and work at the local McDonald’s. Philosophy, to me, became something rhetorical people did when they had plenty of time on their hands; it was an activity reserved for those who were not well versed in quantitative methods and other productive things. It was a pastime that should be limited to late hours, in bars around the campuses, when one had a few drinks and a light schedule—provided one forgot the garrulous episode as early as the next day. Too much of it can get a man in trouble, perhaps turn one into a Marxist ideologue. There are only two types of theories: 1. Theories that are known to be wrong, as they were tested and adequately rejected (he calls them falsified). Theories that have not yet been known to be wrong, not falsified yet, but are exposed to be proved wrong. Why is a theory never right？ Because we will never know if all the swans are white (Popper borrowed the Kantian idea of the flaws in our mechanisms of perception). The testing mechanism may be faulty. However, the statement that there is a black swan is possible to make. A theory cannot be verified. He refused to blindly accept the notion that knowledge can always increase with incremental information—which is the foundation of statistical inference. These are men with bold ideas, but highly critical of their own ideas; they try to find whether their ideas are right by trying first to find whether they are not perhaps wrong. They work with bold conjectures and severe attempts at refuting their own conjectures. Popper is the antidote to positivism. To him, verification is not possible. Verificationism is more dangerous than anything else. I will refrain from commonplace discourse about the divorce between those who have ideas and those who carry them in practice, except to bring out the interesting behavioral problem; we like to emit logical and rational ideas but we do not necessarily enjoy this execution. Strange as it sounds, this point has only been discovered very recently (we will see that we are not genetically fit to be rational and act rationally; we are merely fit for the maximum probability of transmitting our genes in some given unsophisticated environment). Memory in humans is a large machine to make inductive inferences. Think of memories: What is easier to remember, a collection of random facts glued together, or a story, something that offers a series of logical links？ Causality is easier to commit to memory. Our brain would have less work to do in order to retain the information. The size is smaller. What is induction exactly？ Induction is going from plenty of particulars to the general. It is very handy, as the general takes much less room in one’s memory than a collection of particulars. The effect of such compression is the reduction in the degree of detected randomness. The philosopher Pascal proclaimed that the optimal strategy for humans is to believe in the existence of God. For if God exists, then the believer would be rewarded. If he does not exist, the believer would have nothing to lose. They trade on ideas based on some observation (that includes past history) but, like the Popperian scientists, they make sure that the costs of being wrong are limited (and their probability is not derived from past data). Unlike Carlos and John, they know before getting involved in the trading strategy which events would prove their conjecture wrong and allow for it (recall that Carlos and John used past history both to make their bets and to measure their risk). They would then terminate their trade. This is called a stop loss, a predetermined exit point, a protection from the black swan. Part II • MONKEYS ON TYPEWRITERS 第二篇綜合整理目前眾多相關著作中各種關於隨機性的偏差。 這些偏差可以簡述如下：第一，存活者偏差（又稱打字機前的猴子），起於我們只看見贏家，對運氣持有的看法遭到扭曲。第二，不同凡響的成功最常見的原因是運氣。第三，我們在生物構造上缺乏瞭解概率的能力。 The major problem with inference in general is that those whose profession is to derive conclusions from data often fall into the trap faster and more confidently than others. The more data we have, the more likely we are to drown in it. For common wisdom among people with a budding knowledge of probability laws is to base their decision making on the following principle: It is very unlikely for someone to perform considerably well in a consistent fashion without his doing something right. Track records therefore become preeminent. They call on the rule of the likelihood of such a successful run and tell themselves that if someone performed better than the rest in the past then there is a great chance of his performing better than the crowd in the future—and a very great one at that. But, as usual, beware the middlebrow: A small knowledge of probability can lead to worse results than no knowledge at all. There are other aspects to the monkeys problem; in real life the other monkeys are not countable, let alone visible. They are hidden away, as one sees only the winners—it is natural for those who failed to vanish completely. Accordingly, one sees the survivors, and only the survivors, which imparts such a mistaken perception of the odds. Part I described situations where people do not understand the rare event, and do not seem to accept either the possibility of its occurrence or the dire consequences of such occurrence. The business of Part II is more pedestrian; I will rapidly provide a synthesis of the biases of randomness as discussed in the now abundant literature on the subject. These biases can be outlined as follows: (a) The survivorship biases (a.k.a. monkeys on a typewriter) arising from the fact that we see only winners and get a distorted view of the odds (Chapters 8 and 9, “Too Many Millionaires” and “Fry an Egg”), (b) the fact that luck is most frequently the reason for extreme success (Chapter 10, “Loser Takes All”), and (c) the biological handicap of our inability to understand probability (Chapter 11, “Randomness and Our Brain”). Eight • TOO MANY MILLIONAIRES NEXT DOOR 我們一直忽視存活者偏差，因為“表現最好的最容易被看見”，沒人看見輸家。整體而言，我們只關注了歷史隨機產生的唯一眼前現實，而忽略了未發生的“另類歷史”，以此得出的結論必然只是對這個歷史樣品路徑的總結，即無法推測它的未來，更不用說包含了“另類歷史”可能性的整體未來。 becoming more rational, or not feeling emotions of social slights, is not part of the human race, at least not with our current biology. we should remember that becoming rich is a purely selfish act, not a social one. The virtue of capitalism is that society can take advantage of people’s greed rather than their benevolence, but there is no need to, in addition, extol such greed as a moral (or intellectual) accomplishment The first bias comes from the fact that the rich people selected for their sample are among the lucky monkeys on typewriters. The authors made no attempt to correct their statistics with the fact that they saw only the winners. They make no mention of the “accumulators” who have accumulated the wrong things As to the second, more serious flaw, I have already discussed the problem of induction. The story focuses on an unusual episode in history; buying its thesis implies accepting that the current returns in asset values are permanent (the sort of belief that prevailed before the great crash that started in 1929). A brief summing up at this point: I showed how we tend to mistake one realization among all possible random histories as the most representative one, forgetting that there may be others. In a nutshell, the survivorship bias implies that the highest performing realization will be the most visible. Why？ Because the losers do not show up. Optimism, it is said, is predictive of success. Predictive？ It can also be predictive of failure. Optimistic people certainly take more risks as they are overconfident about the odds; those who win show up among the rich and famous, others fail and disappear from the analyses. Sadly. Nine • IT IS EASIER TO BUY AND SELL THAN FRY AN EGG 本章討論績效記錄和歷史時間序列一些有違直覺但很有名的特徵。這裏所談的觀念，名稱有幾種，如存活者偏差、數據挖掘（data mining）、數據探索（data snooping）、過度配適（overfitting）、回歸平均值（regression to the mean）等，基本上它們都是因為觀察者對隨機現象的重要性認知錯誤，因此過度誇張過去的績效。同時“即使是停住不動的時鐘，一天也有兩次正確。”也可以很好說明統計學是把雙刃劍。加上每個人都會把自己的成功歸功於實力，而把失敗歸咎於運氣不佳，我們認不清現實，就不足為怪了。 特別值得注意沙利文（R. Sullivan）、蒂默曼（A. Timmerman）及懷特（H. White）的觀點——今天使用中的法則之所以獲得成功，有可能是存活者偏差的結果。隨機使用的技術性操作法則，因為運氣成功而被模仿者，產生巨大樣品量，當然成功的個例會越多，這進一步吸引了更多人使用；在這個過程中，其他法則連被平等證明和試用的機會都沒有，也即是“另類歷史”被完全忽略。 同時隨機就是隨機，出現模式（pattern）不奇怪。完全沒有模式的隨機，反而是人為的不隨機！ 另外一方面也不可把“發現沒有事情”和“沒有去發現”混為一談。什麼事情也沒發生這個事實，可能包含重要的資訊。 Machiavelli ascribed to luck at least a 50% role in life (the rest was cunning and bravura), and that was before the creation of modern markets. The concept presented here is well-known for some of its variations under the names survivorship bias, data mining, data snooping, over-fitting, regression to the mean, etc., basically situations where the performance is exaggerated by the observer, owing to a misperception of the importance of randomness. Clearly, this concept has rather unsettling implications. It extends to more general situations where randomness may play a share, such as the choice of a medical treatment or the interpretation of coincidental events. even a broken clock is right twice a day. The manager is thus expected to earn $10,000 with 45% probability, and lose $10,000 with 55%. On average, the manager will lose $1,000 each round—but only on average. At the end of the first year, we still expect to have 4,500 managers turning a profit (45% of them), the second, 45% of that number, 2,02The third, 911; the fourth, 410; the fifth, 184. Let us give the surviving managers names and dress them in business suits. True, they represent less than 2% of the original cohort. But they will get attention. Nobody will mention the other 98%.What can we conclude？ The first counterintuitive point is that a population entirely composed of bad managers will produce a small amount of great track records. The second counterintuitive point is that the expectation of the maximum of track records, with which we are concerned, depends more on the size of the initial sample than on the individual odds per manager. This “reversion” for the large outliers is what has been observed in history and explained as regression to the mean. Note that the larger the deviation, the more important its effect. Again, one word of warning: All deviations do not come from this effect, but a disproportionately large proportion of them do. people believe that they can figure out the properties of the distribution from the sample they are witnessing. When it comes to matters that depend on the maximum, it is altogether another distribution that is being inferred, that of the best performers. We call the difference between the average of such distribution and the unconditional distribution of winners and losers the survivorship bias—here the fact that about 3% of the initial cohort discussed earlier will make money five years in a row. the properties of ergodicity, namely, that time will eliminate the annoying effects of randomness. Looking forward, in spite of the fact that these managers were profitable in the past five years, we expect them to break even in any future time period. They will fare no better than those of the initial cohort who failed earlier in the exercise. Ah, the long term. nobody accepts randomness in his own success, only his failure. Recall that the survivorship bias depends on the size of the initial population. The information that a person derived some profits in the past, just by itself, is neither meaningful nor relevant. We need to know the size of the population from which he came. The trick is as follows. The con operator pulls 10,000 names out of a phone book. He mails a bullish letter to one half of the sample, and a bearish one to the other half. The following month he selects the names of the persons to whom he mailed the letter whose prediction turned out to be right, that is, 5,000 names. The next month he does the same with the remaining 2,500 names, until the list narrows down to 500 people. Of these there will be 200 victims. An investment in a few thousand dollars’ worth of postage stamps will turn into several million. There is a high probability of the investment coming to you if its success is caused entirely by randomness. This phenomenon is what economists and insurance people call adverse selection. Judging an investment that comes to you requires more stringent standards than judging an investment you seek, owing to such selection bias. For example, by going to a cohort composed of 10,000 managers, I have 2/100 chances of finding a spurious survivor. By staying home and answering my doorbell, the chance of the soliciting party being a spurious survivor is closer to 100%. The most intuitive way to describe the data mining problem to a nonstatistician is through what is called the birthday paradox, though it is not really a paradox, simply a perceptional oddity. If you meet someone randomly, there is a one in 365.25 chance of your sharing their birthday, and a considerably smaller one of having the exact birthday of the same year. So, sharing the same birthday would be a coincidental event that you would discuss at the dinner table. Now let us look at a situation where there are 23 people in a room. What is the chance of there being 2 people with the same birthday？ About 50%. For we are not specifying which people need to share a birthday; any pair works. When the statistician looks at the data to test a given relationship, say, to ferret out the correlation between the occurrence of a given event, like a political announcement, and stock market volatility, odds are that the results can be taken seriously. But when one throws the computer at data, looking for just about any relationship, it is certain that a spurious connection will emerge, such as the fate of the stock market being linked to the length of women’s skirts. I tend to confuse a book review, which is supposed to be an assessment of the quality of the book, with the best book reviews, marred with the same survivorship biases. I mistake the distribution of the maximum of a variable with that of the variable itself. The publisher will never put on the jacket of the book anything but the best praise. Some authors go even a step beyond, taking a tepid or even unfavorable book review and selecting words in it that appear to praise the book. The exact same task of looking for the survivor within the set of rules that can possibly work. I am fitting the rule on the data. This activity is called data snooping. The more I try, the more I am likely, by mere luck, to find a rule that worked on past data. A random series will always present some detectable pattern. An outstanding recent paper by Sullivan, Timmerman, and White goes further and considers that the rules that may be in use successfully today may be the result of a survivorship bias. Suppose that, over time, investors have experimented with technical trading rules drawn from a very wide universe—in principle thousands of parameterizations of a variety of types of rules. As time progresses, the rules that happen to perform well historically receive more attention and are considered “serious contenders” by the investment community, while unsuccessful trading rules are more likely to be forgotten. . . . If enough trading rules are considered over time, some rules are bound by pure luck, even in a very large sample, to produce superior performance even if they do not genuinely possess predictive power over asset returns. Of course, inference based solely on the subset of surviving trading rules may be misleading in this context since it does not account for the full set of initial trading rules, most of which are unlikely to have underperformed. The late astronomer Carl Sagan, a devoted promoter of scientific thinking and an obsessive enemy of nonscience, examined the cures from cancer that resulted from a visit to Lourdes in France, where people were healed by simple contact with the holy waters, and found out the interesting fact that, of the total cancer patients who visited the place, the cure rate was, if anything, lower than the statistical one for spontaneous remissions. It was lower than the average for those who did not go to Lourdes! Should a statistician infer here that cancer patients’ odds of surviving deteriorates after a visit to Lourdes？ we know that there is no such thing as a pure random draw, for the outcome of the draw depends on the quality of the equipment. With enough minutiae one would be able to uncover the nonrandomness somewhere (e.g., the wheel itself may not have been perfectly balanced or perhaps the spinning ball was not comp