I've had success with Squeezenet for finding the origin of white circles in extremely noisy images, so maybe you could use that. Just change the last convolution in the classifier to match the desired dimensions of your output.

I was able to CPU train a solution that way. It's actually my go to for tasks like that because it seems to just do better than some larger newer networks at that sort of thing.