When there's an uproar, just point out how everyone else around you is being rude. If they're willing to put away their phones and have an actual conversation, then you're happy to put away your book.