Lol he's always getting smokers outside of bars in Mt Vernon too

It doesn't sound like you want city life, other than to be social. But Baltimore is a city of neighborhoods, each has it's own feel and resources. For example, I'm in Mt. Vernon with what you described (no yard, parking difficult for friends, no grocery chains) - but I have 10+ restaurants, the Lyric, the Hippodrome, and the Walters within walking distance. I can even walk to the harbor in about 30 mins!

But there are other city neighborhoods offer what you're looking for. I would avoid Canton if you want a yard and parking. North Baltimore might be the best fit for you, but I second the folks recommending you come here first before deciding which neighborhood.

Also, it'll be hell to drive either into the city or out of the city at various parts of the day regardless where you are. Many of us feel trapped from 4-6pm everyday due to rush hour.

It's safe. You're fine.