Viewing a single comment thread. View all comments

MSB3000 t1_jb8o7u8 wrote

We already can't align our AI systems, or any technology for that matter. Right now it's actually a very familiar problem; machines don't do what you intend, they do what they're made to do. And this is basically fine because as of right now, there is nothing smarter in the known universe than human beings, and so we're still in charge.

But when the machines gain more intelligence than humans? Actual alignment is a totally unsolved problem, so we really do need that solved before we inadvertently create a superintelligent chatbot.

3

Yomiel94 t1_jbcxfi8 wrote

>machines don't do what you intend, they do what they're made to do.

It seems like, whether you use top-down machine-learning techniques to evolve a system according to some high-level spec or you use bottom-up conventional programming to rigorously and explicitly define behavior, what’s unspecified (ML case) or misspecified (conventional case) can bite you in the ass lol… it’s just that ML allows you to generate way more (potentially malignant) capability in the process.

There’s also possible weird inner-alignment cases where a perfectly specified optimization process still produces a misaligned agent. It seems increasingly obvious that we can’t just treat ML as some kind of black magic past a certain capability threshold.

0