I used to do the same thing in public restrooms until I read a study testing the theory. The result, surprisingly, was the first stall is the least used. The majority of people thought the same as you. Interesting, right?