The reason folks say its all about the diet, is simply this. Diet will determine what you get out of your workout.
Whether it is gaining muscle mass or cutting, without proper diet you will achieve neither very well.
A true indicator is look at folks in the gym. You can tell those who have diet down, they actually change their body composition in some manner. Then you have others you see that work their ass off, but never, ever change. These are the folks who do not have diet mastered.