Being on a very different level from the people around you (whether you're well above or well below) is going to be inherently isolating, so there's that. But as you say, other things factor in.