It's not absolute poverty. It's income inequality. Societies where almost everyone is poor but there isn't a huge Gini coefficient tend to be safer than societies with a medium median income but a very high Gini coefficient.

Also I'm not sure if being non-white has anything to do with it. East Asian nations tend to be less violent than Western ones once you control for income and education.