Make sure you're reading the ingredients on the bread, because some grain breads that you get at the grocery store actually have some white flour mixed in there.
I find that most of my cravings are gone if I can maintain a steady insulin level. I do make sandwiches with real chicken berates from time to time. Always buy 100% whole grain wheat bread with flax and stuff in it from this yummy German baker here. I don't think bread is that enemy just the kinda bread your eating can be the problem. I think the key is to keep things as natural as possible. I know a few male BBers that eat whole grain breads with a few of there meals.