All it took to trigger the widespread woe was an outwardly insignificant slip: on September 17th, Nets Estonia coordinating card transactions in Estonia forwarded a file with cards transactions to the financial institutions twice, and two days later attempted to correct the mistake by sending a file cancelling the «double» transactions.
The banks which for whatever reason only acted on the cancel-entries sent on September 19th, yesterday morning unexpectedly returned to customers the money spent on September 17th. This, for instance, was the lot of SEB clients. To our knowledge, clients of institutions like Swedbank and Citadele were less lucky. The control systems of said banks had already acted on the double file dating September 17th and brazenly pocketed the customers’ money twice.
As LHV and Nordea banks control systems pulled brakes both on the file prescribing double payments and dataset sent to cancel it, the clients of both escaped the mess.
Why LHV and Nordea engineers could implement fault tolerant algorithm while engineers of the two biggest banks SEB and Swedbank could not?
I do not know about Nordea, but the LHV team is SMALL and happened to have one of my secondary school class-mates, Rainer Tikk, as a head of their software development team. He tends to be a guy, who is not a dumb manager, but is doing the real work, software development, himself, therefore keeping him in shape from professional point of view and he seems to have also the capability to say NO and stand his ground. That helps a LOT, when it comes to assuring the quality of software, specially in a business environment. What regards to the SEB, then once upon a time they had their software made in a huge team that was lead by a general manager, who was not able to design software himself and who obeyed to its Swedish bosses very loyally. I do not know about the Swedbank, but what I’ve heard from rumors among people-I-as-an-IT-person-know, their sysadmins are basically trying to compensate software developers’ shoddy work by manual labor. There is a LIMIT, how much those sysadmins are humanely capable of compensating.