A.I. safety with Democracy?
Common path of discussion:
Alice: A.I. can already be dangerous, even though it's currently narrow intelligence only. How do we make it safe before it's general intelligence?
Bob: Democracy!
Alice: That's a sentence fragment, not an answer. What do you mean?
Bob: Vote for what you want the A.I. to do :)
Alice: But people ask for what they think they want instead of what they really want — this leads to misaligned incentives/paperclip optimisers, or pathological focus on universal instrumental goals like money or power.
Bob: Then let's give the A.I. to everyone, so we're all equal and anyone who tells their A.I. to do something daft can be countered by everyone else.
Alice: But that assumes the machines operate on the same speed we do. If we assume that an A.G.I. can be made by duplicating a human brain's connectome in silicon — mapping synapses to transistors — then even with no more Moore's Law an A.G.I. would be out-pacing our thoughts by the same margin a pack of wolves outpaces continental drift (and the volume of a few dozen grains of sand).
Because we're much too slow to respond to threats ourselves, any helpful A.G.I. working to stop a harmful A.G.I. would have to know what to do before we told it; yet if we knew how to make them work like that, then we wouldn't need to, as all A.G.I. would stop themselves from doing anything harmful in the first place.
Bob: Balance of powers, just like governments — no single A.G.I can get too big, because all the other A.G.I. want the same limited resource.
Alice: Keep reading that educational webcomic. Even in the human case (and we can't trust our intuition about the nature of an arbitrary A.G.I.), separation of powers only works if you can guarantee that those who seek power don't collude. As humans collude, an A.G.I. (even one which seeks power only as an instrumental goal for some other cause) can be expected to collude with other similar A.G.I. ("A.G.I.s"? How do you pluralise an initialism?)
There's probably something that should follow this, but I don't know what as real conversations usually go stale well before my final Alice response (and even that might have been too harsh and conversation-stopping, I'd like to dig deeper and find out what happens next).
I still think we ultimately want "do what I meant not what I said", but at the very least that's really hard to specify and at worst I'm starting to worry that some (too many?) people may be unable to cope with the possibility that some of the things they want are incoherent or self-contradictory.
Whatever the solution, I suspect that politics and economics both have a lot of lessons available to help the development of safe A.I. — both limited A.I. that currently exists and also potential future tech such as human-level general A.I. (perhaps even super-intelligence, but don't count on that).
Original post: https://kitsunesoftware.wordpress.com/2018/08/11/a-i-safety-with-democracy/
Original post timestamp: Sat, 11 Aug 2018 18:39:44 +0000
Tags: AGI, AI, Artificial intelligence, Democracy, misaligned incentives, paperclip optimiser
Categories: Minds, Philosophy, Politics