Comments

You must log in or register to comment.

Adventurous-Mark2477 t1_je9njm5 wrote

How do we stop?

1

Cypher10110 OP t1_je9t7sh wrote

I guess the answer is probably don't release any more extremely powerful models for public use without extensive internal testing, and instead of quickly training ever more larger and complex models, focus more resources on safety research to ensure that AI tools are appropriately aligned.

The general idea of "slow down" seems pretty reasonable. AI safety (and potentially government regulation) may need some time to catch up.

Will it happen? Not sure, lots of conflicting incentives and perspectives. Interesting times.

1

Akimbo333 t1_je9tm3e wrote

China won't stop though!

6

Cypher10110 OP t1_je9vtqo wrote

Yea, of course. It's not an easy problem.

Personally, I don't think the correct response is to race. But I'd rather die and be right than be wrong and kill everyone else. (Obviously, all this is hyperbole. But I think it gets across my point)

But also, I don't see how "winning" or "losing" the "culture war" can be put on the same scale as potential human extinction. I know some people feel a lot more strongly that the west needs to "win" this one, and that some of the actual risks are still pretty debatable at this stage.

As it turns out, I'm a spectator with zero influence on this particular game, so I'll just do my best to deal with whatever the people with actual power decide is the best idea 🤷‍♂️

0

Saerain t1_jeaqxmu wrote

Hm. I haven't seen "the west needs to win" so much as should be free to participate. Especially if you fear the existential risk of unaligned AGI, it seems counterproductive to intentionally push its development away from your sphere of values.

2

Cypher10110 OP t1_jeax9cy wrote

So true. That's why it's such a tough issue.

In the case where 2 powers are competing to reach AGI, there is a potentially huge first move advantage if one power reaches AGI before the other. So how do they each agree to slow development on safety grounds, when they also know there is a possibility of the other side disregarding their agreement to get a competitive lead? A kind of prisoner's dilemma, they both need to refuse the easy win.

Ultimately, I think who wins is less relevant than a safe win. The pendulum of power swings, and so long as humans are "in charge" in the grand scheme of things, it doesn't really matter (in my view) if they are outside my own sphere of values. So long as they have human values! The pendulum of cultural power will hopefully continue to swing and in a few centuries, the planet will still have humans on it.

Although there is a possible future where human directed AGI changes the world in a permanent negative way (against my own values) - like some sort of totalitarian dystopic power, but as a pessimist I don't think that outcome is exclusive to "the other side" winning this race. Both sides have the power to do bad things.

1