You're standing in a sparse crowd of people - you can see many people, and know their positions and speeds with respect to you quite well.

Now, those folk are stars and gas clouds - does that make it seem less mysterious that you can map their position from another vantage point?