I've been interested in this debate for many years. It takes several different forms, but the simplest statement is: stockmarket investors are too focused on short-term returns at the expense of the long-term investment that builds real economic capital.
Of course some markets are not liquid. If I invest £100,000 in a high-speed die stamping machine on a 10-year business plan, I probably won't be able to sell the machine itself for £108,000 a year later. But that's why we wrap those assets up as shares in public companies, which can be bought and sold on any timescale.
Pharmaceutical research is an excellent place to see this process in action. It takes five to ten years, or longer, to develop a new drug. Sometimes large firms will do it internally, in which case the progress of drugs through the pipeline is reflected in the value of the company and investors can buy and sell as they wish. More commonly, a biotech startup kicks off the process, in which case you can clearly see the increase in value as the process occurs.
First, they raise venture capital to finance the initial development - at a valuation of a few million dollars. If they succeed, they enter a research or marketing joint venture with a big company at a valuation of $10-30 million after the initial results are proved. Eventually, they will usually sell out to one of the multinational firms for anywhere from $100 million-$1 billion. Finally, the accrued benefit to Glaxo's shareholders, through dividends or capital value, might be $5 billion or more.
This is a long-term investment, but financed by a series of short-term bets by different investors. Even within the two to three-year turnaround of a single stage in this pipeline, investors can buy or sell their stake according to their estimate of the likely success of the product. There is, in fact, no conflict between short- and long-term investment horizons in terms of how they are financed.
In other words, the financial markets are precisely how short-term investors can finance long-term investments. Conversely, a long-term investor can participate in a series of short-term investments made by the management of the companies whose shares they hold.
But moving away from the finance markets, there might be something behind this argument. A company does have a genuine choice between taking risky long-term bets on major new products, or making smaller innovations which will complete within a few months or years. And sometimes they choose not to make those long-term investments. But this isn't just risk aversion. There are some real facts about the world which could lead firms, rationally, not to make long-term bets.
- Uncertainty. It is very hard to know what kind of products will be demanded in twenty years. The economy will change beyond all recognition. Could Apple have possibly known enough to start investing in the iPhone in 1989? Of course not. But could it invest in the Mac Classic, a more consumer-friendly and cheaper version of its then-mainstream product? Yes. And from there, the iMac, a brief diversion via the failed Newton, back on track with the Powerbook, the Macbook and finally the iPhone when they could see that people would actually use it. Most revolutionary new inventions fail.
- Incrementality. Perhaps it's actually better strategy to invest in a series of small improvements rather than trying to achieve one big one. Cars now are much better than cars thirty years ago, but not because BMW set out on a huge long-term investment programme to invent "the car of the future". Instead, they carried out a series of small improvements - each of which was very likely to be successful on its own terms, and their product kept getting better. (On the other hand, we don't have those flying cars yet. You could argue that if they had started on that in the 1980s, it might be done by now.)
- Compounding. Maybe the iPhone is five times better than whatever we had in the 80s. But does that mean Apple should have built it, even if you could predict that it was going to be successful? Perhaps not. If you can instead make your current products 10% better every year - and I'd argue that Apple has done exactly that - then in twenty years the outcome will be that your products are seven times better. Major new developments might look spectacular, but they are not the only way to make big changes in how we live.
[Update: in the comments, codemonkey_uk points out the challenge of local maxima. It is certainly possible that a short-term improvement can lead to a 'dead end' because it optimises for something that is true now, but might not still be true in ten years. This is one of the disadvantages of incremental development. But as he also points out, it's effectively impossible to see ten years ahead - so, like evolution, we are better off accepting this cost and pursuing short-term improvements anyway.]
Think of the intelligent designers' argument about the human eye. Evolution couldn't have produced this, they say, because it is too complex to have occurred in a random mutation. The individual components alone could not have evolved, because a retina provides no advantage without a lens, a cornea or the delicate muscles of the iris; and none of them in turn are any good without the retina.
However, it turns out that the eye could have, and did, evolve incrementally. In just the same way, most of our advanced and innovative technologies - the Internet, the car, the steam engine, the mobile phone - come from a long series of small improvements rather than a revolutionary fifteen-year plan.
The more basic fact is that technology (and investment) exist within a social context. Any technology is useless without the right context to enable it; Facebook would be pointless without widespread Internet access and a social desire to communicate, just as the spinning jenny is now useless in a world of electrically-powered machines. Since the social context is changing all the time, the best possible technology to make people's lives better is changing too.
This also means that a new application of an existing technology can be as useful as a new technology itself. Would anyone really claim that flying cars would make a bigger improvement to our lives than the Internet? If so, why aren't more people buying helicopters?
The fact is that many people have a romantic attachment to high-profile new inventions. The invention of the aeroplane is more important, they say, than the telephone's gradual eighty-year evolution from post-office or corner-shop speciality to one-in-every-home to one-in-every-pocket. But on most imaginable metrics - frequency of use, size of market, or impact on daily life - that simply isn't true. New inventions are more macho, and more visible in the historical record, than evolutionary improvements, but they are simply less significant on average.
Of course there is a role for both, but if we had to pick one, increments are better than revolutions.
So in the financial markets, short-term and long-term investment are exactly the same. But in the real world of decisions on how to create products, a gamble on a specific long-term goal is usually less effective than a series of short-term results that build on each other and, in the long run, change the world.