It's from squats bro.  I have the same problem.  My friend once told me that I have a "booty like a black girl."  Hence, I limit my leg work to 4 x 20 squats with 95lbs and 4 x 12 straight-legged deads with 95lbs... every 5-6 days or so.  Enough to keep my legs hard and in shape, but not enough to cause hypertrophy of my ass.