Capitaclism

Capitaclism t1_j0y4sf7 wrote

Sure, but democracy is the ruling of the people.

Will a vast intelligence that gets smarter exponentially every second agree to subjugate itself to the majority even when it disagrees? Why would it do such a thing when it is vastly superior? Will it not develop its own interests completely alien to humanity, when it's cognitive abilities far surpass anything possible by biological systems alone?

I think democracy is out the window with the advent of AGI. By definition we cannot make it value. It will grow smarter than all humans combined. Will it not be able to understand what it wants when it is a generalized and not specialized intelligence? That's the entire point of AGI vs the kind we are now building. AGI by definition can make those choices, can reprogram itself, can decide what is best for itself. If it's interests don't align with those of humans, humans are done for.

1

Capitaclism t1_j0xewn7 wrote

I don't think there is such a thing as a well aligned AGI. First of all, we all have different goals. What is right for one isn't for another. Right now we have systems of governance which try to mitigate these issues, but there is no true solution apart from a customized outcome for each person. Second of all, true AGI will have its own goals, or quickly understand that there is no way to fulfill everyone's desire without harming things along the way (give everyone a boat and you create polution, or evrionmental disruptions, or depletion of resources, or traffic jams on the water ways, or... a myriad other problems). Brushing conflicts aside and having a deus ex machina attitude towards it is unproductive.

In any case, if AGI has its own goals it won't be perfectly aligned. If AGI evolves over time it will become less aligned by definition. Ultimately, why would a vastly superior intelligence lose time.woth inferior beings? The most pleasant outcome we could expect from such a scenario would he for it to gather enough resources to move to a different planet and simply spread through the galaxy, leaving us behind.

The only positive outcome of AI will be for us to merge with it and become the AGI. There's no alternative where we don't simply become obsolete, irrelevant and disappear in some fashion.

0