Interesting question.
Quite the dilemma if you really get into the whys of flirting.
Flirting can be innocent and never actually lead to anything remotely physical, however I doubt that I have ever flirted with someone I am not attracted to on some level be it body or soul.
Attraction can go far beyond the just physical to the personality of the person. I have flirted with the physically unattractive, however their personality was what made them "flirt-worthy."