Ya, they can be real rough on your shoulders. But some people can do them, others cant.
Like you said, just make sure NOT to go very heavy, and pay extra attention to your form....its gotta be damn near perfect on these. Doing them behind the neck puts more emphasis on the middle delt head.
personally, I think upright rows are worse for shoulders than BTN exercises, granted you do them correctly.
The main thing is to watch for internal rotation of the humerus(upper arm). Its actually more common than you might think. People working there chest, and front delts more than they're back and rear delts can cause constant internal rotation. That why you see alot of guys walking around with there arms and back kinda bent and in front of there body, because they only workout the Mirror Muscles and its pulling there shoulders forward.
IMO, if you have balanced shoulder muscles and back muscles, btn presses can have there place. I wouldnt do them every workout tho. And IF it hurts or is uncomfortable than STOP immediately.