I log my workout using the Jefit app. I hate doing it in between sets, but my memory sucks so bad if I wait until after my workout I already forgot how many reps I got.
I also use a stopwatch app for timing my rests between sets sometimes.
It never occurred to me that some people would see me on my phone and think I'm a dick
And yeah, on a cardio machine, I would think that is an acceptable time to fiddle with your phone.