I said leave.... but only when you are flat-out open with him and give him one more chance.
I believe that a person needs to be sexually satisfied, without that, it's somewhat of a friendship only.
and one last ditch effort, (I don't know how you'd feel about this morally), you could try an "open" relationship if he is unwilling/unable to satisfy you.
Oh, and have you thought of having a threesome FFM with him.... that might just spice it up!