Black Swans, Algorithms, and the Human Factor

Andrew Coyne recently argued (National Post, July 25), that the Lac Megantic disaster was the kind of once-in-a-lifetime event which, however horrifying, left no real ‘lessons’ for governments and regulating bodies. He feared politically-motivated, hugely expensive, and probably futile ‘runaway regulation’, what British journalists call ‘SMBD’ [‘Something Must Be Done’] action SMBD is nearly always ill-considered, of little real value, and carrying all kinds of bad effects of its own. Coyne made some good sense, bit I think he went too far in his dismissal.

Deregulated transportation since the 1980s has been turning into something quite different from what it is still largely imagined to be, by both its mainly economist advocates and its mainly politically leftist opponents. ‘Free market’ economists have long maintained that the whole broad political wave of deregulation has been an almost unmixed blessing, replacing obsolete, inefficient, and badly-run state-imprisoned enterprises, and unleashing the innovative potentialities of competitive private enterprise. There is no doubt that the change has brought many visible successes. While especially associated with the political leadership of Ronald Reagan and Margaret Thatcher, deregulation was largely continued in the era of Bill Clinton and Tony Blair. Some journalists, Andrew Coyne among them, have also cheered on the new dispensation. In fact, Coyne has several times lambasted the Harper Conservatives for failing to go far enough in cost-cutting and shrinking the size and role of government..

Opposition to the worldwide deregulating turn has come mainly from the trade unions and the radical environmentalists. They have been raising their voices noisily again since the 2008 Crash, not only holding deregulation responsible for what went so wrong in the financial markets [where they at least have a case of sorts], but claiming that the whole development was a mistake. This more general attack has not had much impact; it is very hard to defend the idea, for example, that the monopolistic and tightly-regulated telephone companies of the first seven decades of the 20th century should have been preserved. On the other hand, economic libertarianism has sometimes acquired its own touches of ideological dogma, taking for granted the assumption that all issues, political, social, and cultural as well as economic, can be readily resolved by appealing to Hayekian or Friedmanite principles.

But as the 2008 Crash in the financial markets demonstrated, there can be major problems in ‘industry self-regulation’, not problems that can really be fixed by a tangle of new lasws and regulations, but requiring some serious ‘rethinking’ in the industries themselves. In particular, modern commercial transportation of all kinds does not now merely show the effects of competitive private ownership, but of the rise of computerized mathematical algorithms as a means of integrating land, sea, and air transport in a way that was not even possible as recently as 1980. The full implications of this change are only gradually being understood, even by the designers of the algorithms. They are as large in their own way as those from containerization for ship and land cargo.

Algorithms are step-by-step ‘recipes’ that can be applied to almost every kind of changing but partly repetitive commercial activity, from the pricing and trading of financial instruments to the scheduling and ‘optimal path’ for ocean and coastal shipping, trains, trucks, and air freighters, all now coming to be seen as part of giant interconnected networks. Until around 1980, the mathematics sometimes required looked too daunting, even for professional mathematicians. Their difficulties can be illustrated by a classic algorithmic issue, the ‘Travelling Salesman Problem’. The idea is to find the most time-efficient or ‘optimal path’ for a series of deliveries on a transportation route. The number of routes increases factorially with the number of stops. Simple enough with three stops, for which there will be 3 x 2 x 1 = 6 routes. However, for only eight, there will be 8 x 7 x 6 x 5 x 4 x 3 x 2 x 1 = 40,320. At twelve, there are already over 5 billion routes, and even at only twenty, there are over 100 quadrillion possible paths.

Nonetheless , what teams of mathematicians using superfast computers were eventually able to do was design algorithms which can produce an ‘optimal tour’, with no shorter one possible, for any specific example provided, even one involving thousands of stops. These are now being applied more and more to scheduling, to loading and unloading of goods, and to making possible communication systems between central offices and large moving collections of ships, trucks, and trains The algorithms have been spreading like cellphones. UPS, for example, was still working out its ‘same day deliveries’ by rules of thumb and pins on a map at the start of the 1980s, but after 1982, when it had to accommodate itself ton next-day air delivery, scheduling became far more complicated. The company quickly realized that it needed project managers who knew how to use algorithms, and by 1986 bought a ‘logistics’ company to help them.

This new world of computerized algorithms was a major factor making it possible for hard-driving railway executives like Paul Teller, Hunter Harrison, and Ed Burkhardt to buy, sell, and re-organize large new combinations of railway companies made out of the corpses of old giant lines. Burkhardt, who most Canadians had never heard of a month ago, and not fondly regarded at the moment, had previously been accustomed to being hailed in the transportation world as a leading star of this new world, and holds all kinds of railway companies in his fief along with the ill-fated Montreal, Maine, and Atlantic, also advising Communist on how to create their own efficient privatized lines. But it appears that Burkhardt, and railway executives in general, may have failed to learn the lessons that the algorithmic project managers at UPS have.

UPS found that getting their fully algorithmic program working was far harder than they had at first assumed. What they had first thought would take a year to work out instead took a decade, and the work is still not completed. They gradually realized that they did not merely have to be concerned with the mathematics of scheduling, numerical distances, and delivery or transfer points, but with the sometimes irrational and habitual behaviour of their clients and drivers. Emotional factors, it was found, changed things like truck driver behaviour on their routes. One major trucking company even applies ‘predictive analysis’ of when drivers have a greater risk of being involved in a crash. A divorce, a drop in pay, even a sudden and unexpected demand to make a large route change, may lead to frequent speeding and rapid lane changing. Trucks are simple, and can be stored anywhere, while drivers are complicated, and like to go home at night, or at least not sleep in their trucks. It matters whether they are happy.

At UPS, Yellow Freight, and other major transportation companies, they have been learning more and more about how to use mathematical algorithms. But equally important, they have been learning how important ordinary human elements still are. What the tragic derailment at Lac Megantic appears to have shown is that this element was being given inadequate weight in the modern system of deregulated ‘lean’ railways that replaced the old tangle of bankrupting lines. This had not just applied to MM&A, but, as Ed Burkhardt himself commented after his return to his Chicago offices, throughout the whole industry.

So Coyne was right that it is unlikely that new Canadian government regulation will produce much benefit, even in safety. A future train catastrophe could be of an entirely different kind than the Lac Megantic one, like the recent high-speed passenger train disaster in Spain. But such events are still likely to have a large impact on the assumptions previously held by railway executives and their company shareholders on the way they can direct their computerized, algorithm-directed transportation networks. The MM&A crash may also have a long-term influence on the political and economic appeal of oil pipelines, on the maximum tolerated size of trains, and on just what rolling stock and tracks can still be assumed to be useful. Whatever kind of perfect storm of possible errors by the Nantes firefighters, brake failures, and so on, it was still clearly an event that had a great deal to do with the behaviour of the missing engineer. It was the people of Lac Megantic who paid the most terriible price for the ‘systemic failure’, but Burkhardt and all those modern railmen grown too used to thinking of their railway business in terms of electronic maps in central offices have also had to learn that even the most automated and computer-sophisticated systems still have to allow for the human factor. No algorithm can eliminate that. Free market economists need to keep that in mind as well.

One Reply to “Black Swans, Algorithms, and the Human Factor”

Leave a Reply to Anonymous Cancel reply

Your email address will not be published. Required fields are marked *