Is my relationship truely over?

He sounds like a very confused guy. One thing I have to say, the whole break thing never works. It's always messy and confusing. Why don't you suggest not talking to him or seeing him for several days so that he can figure out what exactly he wants, and then you need to stick to the plan. Now, he loves you and wants to be with you it sounds like, but your relationship won't work if he's never there. He needs to talk to you about how he's feeling (don't pester him about it right now though, it won't help). Some time in the near future, he needs to tell you what's going on and you need to try to work through it TOGETHER. Also, you have KIDS together! His leaving and then coming back and leaving again isn't good for them. They need stability. So he needs to decide where he's going to live and stay there, for them and for you. I wish you the best of luck and I hope you guys can work it out.