Then I'd just make sure you don't lose it? A passport is not a difficult thing to make sure you don't lose.

It sounds like you do a lot of international travel, for reasons, and I'd expect someone who does would have particular behavior patterns, like where you put your passport when you get in a hotel, or where you put it after airport security, etc. Just make habits and you won't lose it unless it's stolen. And if it's stolen, you aren't getting it back.