We often mistake luck and randomness for skill and determinism.

We are all frequently fooled by randomness, meaning that we underestimate the impact of luck and random events on our lives. We use terms like “skills”, and “determinism” when “luck” and “randomness” are called for. Nowhere is this discrepancy more evident than in the stock market, where “capable investor” should usually be substituted with “lucky idiot”.

In some professions, one simply cannot succeed without skills: A plumber or a dentist is extremely unlikely to have a long career without knowing what he’s doing.

Unfortunately, the inherent randomness of stock markets means that, just like millions of monkeys hammering on typewriters for long enough can eventually produce Shakespeare, so can unskilled investors produce great track records. In fact, it is very likely that some will.

Consider for example a cohort of 10,000 investors who, for the sake of argument, are relatively incompetent: each year they only have a 45% chance of being profitable. In other words, you would basically be better off investing based on the flip of a coin.

Nevertheless, despite their lack of skills, after 5 years based on probabilities alone we can expect almost 200 of them to have been profitable every year. They would boast flawless track records and enjoy praise for their exceptional skills.

Of course, in the long run, the randomness that sustains these “acute successful randomness fools” will turn against them. Wall Street has seen many traders, who after years of success have one devastating quarter where they lose everything in one huge blow-up.

Often their short-lived success was due to the fact that they simply happened to be at the right place at the right time, i.e. pure luck.

We can never be sure any theory is right – things constantly change and the next observation may prove us wrong.

The basis of all empirical science is a process called induction: we infer things about the nature of the world based on our observations. Thus from seeing hundreds of white swans, we might infer (mistakenly) that all swans are white.

Unfortunately, this approach carries an inherent problem, illustrated by the famous example of black swans as stated by philosopher John Stuart Mill: “No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion”.

This is known as the problem of induction, and it means no theory can ever be proved right, only wrong (by a single “black swan”). Hence theories are constantly being proved wrong and replaced by better ones.

A similar mindset can be prudent in investing as well: Always consider the possibility that your theories and assumptions may be proved wrong, and examine how such a development would affect your portfolio.

A financial risk manager who ignores this advice saying, “This has never happened before, hence it won’t tomorrow either,” may well be unpleasantly surprised one day.

In fact, he also errs by assuming the past is a relevant sample of what the future holds. What if things have changed? How could you infer anything about the color of swans if their pigmentation was constantly changing?

Yet wherever people are involved, like in the stock market, there will be constant change through adaptation. For example, if stock prices always rose on Mondays, rational investors would all buy stocks on Sundays, thus changing the market dynamic and eliminating the effect.

Life is unfair and non-linear: The best do not always win.

The popular belief is that evolution always means survival of the fittest. In fact it merely means that on average, the fittest organisms will survive. A few lucky unfit organisms will usually also endure, at least in the short term.

The same is true for many things in life. Consider the common keyboard: how did this bizarre layout of keys (known as QWERTY) end up being the near-universal standard for typing?

Rather than being the optimal solution, it was in fact designed to avoid jamming old-fashioned typewriters. Yet since people are too lazy to switch to a different kind of keyboard, this suboptimal solution has prevailed.

This is called a path dependent outcome: if we were to start from scratch, we would not wind up with a QWERTY-keyboard again.

Similarly, even inadequate products may come to dominate the market if they pass the so-called tipping point. Consider for example Microsoft: When enough people started using Microsoft products, it created a positive feedback loop, where new customers bought Microsoft products precisely because everyone they knew was already using them. After a product has passed the tipping point, it is in a very strong position.

Such non-linear events like the tipping point are hard for us to predict. Our nature is to assume that an incremental input, like adding one grain of sand to a sandcastle, will create a similarly small result. In real life though, an incremental change can have a huge impact: a single grain of sand can cause the entire castle to tumble.

Thus for example a scientist may work for years without any observable progress till suddenly a breakthrough happens. Going the extra mile is disproportionately rewarded, but without visible progress most people give up before the rewards.

Our reasoning is context-dependent and mostly based on simple heuristics.

Humans are ill-equipped to handle the probabilistic reasoning required by today’s high-information environment. Despite what we may believe, our mind is not a sophisticated thinking-machine, but rather a patchwork of rules and shortcuts called heuristics.

These heuristics have evolved to help us make quick decisions when needed instead of contemplating needlessly: If you encounter a tiger in the jungle, it may be wise just to run and not ponder the details of the situation.

Unfortunately, the price we pay for using these lazy shortcuts is that our reasoning becomes irrational and marred by what psychologists call biases. For example, due to attribution bias, we tend to disproportionately ascribe successes to our own abilities, and failures to “bad luck”.

Our thinking also becomes path dependent, meaning that the route with which we arrive at a given situation dictates how we think about it.

For instance, if you were to win $5 million today and lose $4 million tomorrow, you would likely be much less happy than if you simply won $1 million tomorrow, although the end result is identical.

Path dependency also means we cling to our existing opinions. Scientists and politicians tend to get attached to the ideas they advocate, and refuse to change their minds even in the face of contradictory information.

From an evolutionary perspective, it makes sense that we get attached to things we have invested a lot of time and effort into, e.g. our children, but this inclination can also be counterproductive. It should be possible and acceptable to change our minds and contradict ourselves at will.

Emotions can help us make decisions, but also overwhelm our capacity for rational reasoning.

Some researchers believe emotions are the true shortcuts in our decision-making process, the “lubricants of reason”. Without emotions to give us that little irrational nudge, we would agonize endlessly over the slightest decisions.

Consider the example known as Buridan’s donkey: A donkey that is equally hungry and thirsty stands equidistant between food and water. If it were purely rationally optimizing what to do, in theory it would die of both hunger and thirst, unable to decide which to pursue first. A little randomness helps it make up its mind, just like you might use the flip of a coin to help resolve an impasse. Emotions are fundamentally irrational precisely to stop us from temporizing.

Intelligent individuals also need to recognize that their capability for rational reasoning can easily be overwhelmed by emotions. In fact, neurobiologists have found evidence to support the notion that we feel emotions first, then try to rationalize an explanation for them. This means emotions have a stronger influence on rational thinking than the other way around.

When Ulysses sailed his ship past the deadly yet seductive sirens, he had his men pour wax in their ears so they would not hear their song.

Similarly, in some cases we can choose to avoid emotional input to protect our reasoning. For example, an investor who knows he is prone to act irrationally when incurring losses might choose simply not to look at the performance of his portfolio unless it triggers a certain, predefined alarm.

In retrospect, we always find patterns, causes and explanations in past events, but they are mostly useless for predicting the future.

Learning from history does not come naturally to humans.

Even after multiple “completely unexpected” stock market crashes, somehow many traders still believe the next crash will not happen or will be spotted by them in advance. This is due to hindsight bias: in retrospect, earlier events always seem more predictable than they really were at the time.

In fact, if any past data is sufficiently analyzed, it is inevitable some pattern will emerge from it: one author even claimed that he could find predictions for past world events simply by examining statistical irregularities in the Bible. Like our ancestors who divined the future by examining bird entrails, we tend to naturally find patterns and causal relationships where there may not be any.

Gambler’s ticks are manifestations of this effect, so for example a trader who wins on a day when he or she wore glasses and a green shirt will probably start wearing the same outfit more, perhaps even unconsciously.

Some traders go further in looking for patterns in the stock market, and use so-called backtesters to see how certain trading rules, e.g. “Always sell when the stock price is X% above its average”, would have performed historically.

Of course, unleashing modern computing power on a large amount of data will inevitably uncover many such rules, but the past “success” of these rules is due to pure randomness, and whoever blindly believes in them is likely to have their portfolio annihilated.

Enjoy harmless randomness and use stoicism to handle the harmful kind.

Although being fooled by randomness in the stock market is often lethal for your portfolio, there are instances where randomness can truly be enjoyable.

While generally it is sensible to be hyper-rational when dealing with science and finances, one can gladly be fooled by randomness when it comes to art and poetry. A scientist using elaborate prose as noise to disguise the fact that he has nothing of value to say is infuriating. A poet on the other hand, using similarly elaborate prose can be sublimely beautiful. Aesthetic forms appeal to our brains whether they are formed by randomness or not.

As the saying goes, “If I must eat pork, it had better be the best kind.” Similarly, if we must be fooled by randomness, it had better be the beautiful, harmless kind.

Despite our best efforts, all of us are sometimes the victims of adversity caused by harmful randomness (an unexpected cancer diagnosis is a prime example).

In such an event, the code of conduct we should follow is provided by stoicism. It encourages us to follow a dignified path of personal elegance, never showing self-pity, never blaming others and never complaining. This approach works naturally together with our personal dignity, emphasizing courage and wisdom in the face of misfortune.

Our own behavior affords us the only measure of control we have over the whims of randomness.

Both in the media and stock markets, random noise is not worth listening to.

People who obsessively read the Wall Street Journal every day expend a lot of effort for very little reward.

Today’s information environment is so cluttered with useless news, the cost of wading through all of them by far exceeds the cost of missing those few truly valuable items. It is akin to searching for a needle in a haystack for some 30 hours each month.

Similarly, stock market price movements are mostly random, unimportant noise with only very little real change in the value of the stocks. Though Bloomberg journalists may try to interpret and explain every miniscule movement, stock prices actually fluctuate quite disconnectedly from the fundamentals they are supposed to reflect. In the long-run certain stocks can perform better than others, but in the short term most movements are merely random noise.

Consider how this affects an investor following her stock portfolio, which for argument’s sake has 10% volatility and 15% expected returns.

If she checks her portfolio every minute, as many traders do these days, she will largely only see the small variance inherent to her portfolio, i.e. the natural ups and downs that are not related to the stock’s performance. Being emotional like all humans, she nevertheless rejoices at profits and agonizes over losses. Thus each year she can expect to experience 60 688 minutes of pleasure versus 60 271 of pain.

If, on the other hand, she checks her portfolio annually, the actual performance of her stock drowns out the noise-component. She can expect to feel pleasure 19 out of 20 years.

Although eventually her returns are the same either way, the minute-to-minute updates will leave the investor emotionally drained, since losses always sting more than profits.

We are all fooled by randomness, but frequently misinterpret it as something deterministic.

The questions this content may answer::

How does randomness dominate the world?
• We often mistake luck and randomness for skill and determinism.
• We can never be sure any theory is right – things constantly change and the next observation may prove us wrong.
• Life is unfair and non-linear: The best do not always win.
Why do we constantly fail to appreciate the impact of randomness?
• Our reasoning is context-dependent and mostly based on simple heuristics.
• Emotions can help us make decisions, but also overwhelm our capacity for rational reasoning.
• In retrospect, we always find patterns, causes and explanations in past events, but they are mostly useless for predicting the future.
• We are inherently poor at understanding the impact of rare events.
How can we deal with randomness?
• Enjoy harmless randomness and use stoicism to handle the harmful kind.
Both in the media and stock markets, random noise is not worth listening to.