Not sure how big your building is, but the elevator is only used to load/move down to street level/unload. Any reason you can't work in tandem of the other people moving?

It's kind of a jerk move to commander the public elevator when it would impact so many residents.