I'm sorry OP, but the reality is that your relationship is going backwards.

I'm a firm believer in actions. If he can walk away, he's not invested and you need to find someone at the same level of commitment and not wait on someone who is waffling.