Quite right. And ground sloths weren't really unusually large...they filled the same sort of large-mammal ecosystem niches that rhinos and elephants fill today. Really they were just standard-sized big mammals, the unusual part isn't that land mammals were large in the past, it's that large land mammals are so scarce in the modern world.

It's really only the giant insects that grew large due to high oxygen (and lack of competition from flying vertebrates)