I think this is how life got complicated.
'He's screwing with you so screw with him.'
Why not just be straight with him?
'Hey. I like you but feel that you're screwing with me. WTF?'
If you like the answer, then stay. If you don't like the answer, then leave. I'm sure there's a bunch of EFers who will drive down and cheer you up.