You can like the person and have great sex with them, but it doesn't necessarily mean that you have to love them.
This is where cornered the market on fuck buddy relationships.
Some of the best sex I have ever had came from people I did not care for a relationship with. And the friends with benefits scenario with some of these lasted for years with no problems or hard feelings.