Staying friends with an ex boyfriend just because of kids

I love the people that read Shine because your opinions and insight are really helpful.

Here is another dilemma I've run into so people, give me your opinions.  And I"m sorry it is so long, but I wanted you to understand how my feelings came to what they are.
I had dated a very nice guy for about 3 years.  He had a severe speech impediment and it took him along time to realize my daughter developed a liking for him just because he was who he was.  She was around 8 I believe.  He for a long time thought she and I thought him different and finally realized it was all in his mind due to his past.  He has a hard time accepting that people (compassionate/understanding ones) do not see him as any different.  He had a very rough childhood and lost his mother several months after he was born and the stepmother was always an abusive witch. So we dated,  but the issue that came up was he would not express his feelings, thoughts or desires to talk about us or himself.  The only time he let his guard down and expressed his love and passion for me outward was in the bedroom.  He even admitted he knew that.  And he never confronted his step mother for the emotional abuse he took, nor his father for allowing it...to ask for an apology.  I believe that would have made a huge difference in his outlook.  I held on for three years with hope that he would be able to bring a change about so I could feel who he was and form an unbreakable bond.  I would have dinner ready for him when he came home, etc.  We even decided to room together in his home after we had been dating 12 months or so.  I fell deeply in love with this man, despite the communication block.  I would do little surprises for him trying to make the relationship exciting and fun trying to get him to smile and see that I cared enough.  Once I was considering having a custom pool table made for him to put in his dining room as a game room as he had expressed the interest during a random conversation.  Other men, you would see the gleam in their eyes with some amount of excitment.  I was excited because I knew someone who built them.  I was going to surprise him for his birthday.  But good thinking..I mentioned the idea before making the arrangements.  This guy... there was no reaction at all.  I was just floored. The basic reaction was like, "ok."  Custom pool tables are thousands of dollars and I just got an, "ok."  Well that idea flew out the window!  At that point, I stopped the little surprises,etc to see just what creativity he had in him and I stopped cooking every night when I realized he just expected it.  Well that idea crashed and burned.  The biggest surprise was flowers when I was at my wits end expressing my desire to end it. 
So, as you see I invested very deeply my heart,soul and energy into trying to bring this man around from the protective barrier he lived in.  Several months after I broke it off, he took my daughter out to eat.  She was around 11 at that time.  She let it slip that he showed her a pic of a beautiful engagement ring he said he had purchased for me, but returned it after I broke it off.  Oh, how painful that was to have known that and hear it from my child.  He should have never said a word period.  So, the most important thing he expressed after we broke up is the need to stay friends, even after the ring thing.  I only said yes because he was good for my daughter and she was good for him.  He was her stand in father while her own was busy marrying women and leaving them.  So, I put on a face several years now that didn't show how utterly difficult it was to stay friends.  But how good is a friendship if you can't feel who the other person is?  An internal connection that never developed.  I get my other friends of both sexes and they express their most inner private thoughts to me and I just get them because I can feel who they are!  Ugh!  Now he is dating someone and just can't understand why I no longer want to pretend to be great friends.  I still have deep feelings for this man and knowing he is seeing someone, is just to painful to let ride.  I decided that totally breaking the friendship off will allow me to move on and in time those feelings will become faint allowing me to have a successful relationship with whomever I find in my future.  My daughter now 15 will continue to spend time with him, which I see no issue with that at all.  But I want to put distance between us to move ahead. 
My question is, how do I explain to this man in words that it hurts me more to stay friends than not, without him thinking I'm just mad at him???