Agree. The answer is first to just call/write the guy and ask. Then if it’s agreed on, you can either just document the agreement and live with the risk that he might be a dick about it later, or go through the process of actually lifting the covenant.

The only reason not to do this is if you’d rather just act first and apologize later which might be fine, or might be a huge legal nightmare, ruin goodwill with your neighbor, and worst of all make you live under the cloud of fear that you’ll run into those things (that last one is just the way my mind works. I hope you are healthier than me.)