Change your focus. If your goal is to improve your bench, that is what you should be concentrating on. The body parts that get trained come second to this. You do not train chest, shoulders, and triceps, you train whatever is necessary to improve your bench, even if it is your left pinky toe and an ingrown hair folicle on the lower right side of your scrotum.
If you are trying to build size, I still think it is okay. However, if you train for strength, size will usually follow.