When a self-driving car finds itself in a situation where a collision cannot be avoided, whose safety should take priority?
Framed as a variation on the so-called trolley problem, this question is richly suggestive for anyone thinking about the relationship between technology and ethics.
Ethical judgements were traditionally made by individual people as circumstances demanded, but with the arrival of autonomously operating machines, the criteria for judgement must now be written into a program in advance.
In other words, moral choices that had always been left tacitly to human beings must now be made explicit at the design stage.
This shift confronts us with the thorny question of where responsibility lies.
Does the blame for an accident rest with the manufacturer, with the user, or with the algorithm itself?
Existing legal systems never anticipated such a case, and rebuilding them has become urgent.
More serious still is the tendency for the technology’s judgements to become opaque.
With systems built on deep learning, even their developers may be unable to explain the grounds for a decision, and the doubt lingers as to whether we should entrust lives to something that cannot be explained.
Since the advance of technology cannot itself be halted, what is asked of us is to deepen ethical reflection at a pace commensurate with development, and to create the forums for public dialogue that sustain it.
Tap a word in the text to see its meaning. Tip: characters are not marked wrong while the IME conversion is still pending. To fix an earlier line: clear the current one, then press Backspace again to step back.