Bruce Schneier on Dual Use Technologies
Bruce Schneier, who was recently awarded CPSR’s Norbert Wiener Award for Social and Professional Responsibility, spoke at the annual conference on the topic of “Dual Use Technologies.” It’s a fascinating speech, and a good discussion afterwards. He had this to day about the August, 2003 blackout in the North American Northeast:
The genesis of that seems to have been a software problem that got out of hand. I actually blame it on a virus infection, although admittedly the report about the accident doesn’t come to that conclusion. But they’re kind of fuzzy on what happened on that computer that started the whole thing. And it was the same day that one of the big Internet worms was going across the country. So, plausibly there’s a connection. But surely that kind of thing is possible: more and more of our infrastructure that matters, is using the same software as, I guess, you know, our normal infrastructure. Emergency 911 services would be another example. [~23:00]
This, which really caught my attention (I’ve never heard of a virus hypothesis for the blackout before), came up in his discussion of the principle of “assurance” in software development, the idea that an application is presumed to be vulnerable unless it is proved not to be—a kind of “proof” that is really hard to establish. (Impossible really, but… nitpicking.) Assurance is obviously important where people’s lives are at risk, say in avionics, space travel, military applications, etc.
The point was that infrastructure applications are almost never developed using “assurance” methods. They tend to be more like Excel, they are commercially reliable, but perhaps not more than that. It crashes once in a while so you learn to save frequently. Schneier says that military technologies developed to assurance specifications could benefit the public good because they are built to a higher standards of functioning—military software as an antidote to the risks introduced through commercialisation.