You should definitely step away from your partners life. You didn't make a "huge mistake", if your ex didn't force herself on you, you made a choice, you have to live with it.