dairy cutting

Anybody else find it alot more difficult to cut while still eating dairy? As an example I could lose 2 pounds a week easy with no dairy but only 1 pound a week while eating dairy. Both weeks eating 2000 cals every day. I would usually have about 500ml of milk in the morning and 150ml of cottage...