Yep, that was also one of Bostrom arguments.

To properly align itself with our values, even in situations we could not even imagine ourselves, making a simulation of humans and test our avatars responses could be the only way of protecting us.

By harming « them » instead.