It only counters your efforts because using sugar has no impact on satiety and therefore you will get hungry later.
As I'm someone who'd get hungry, I'd use white potato if I wanted high GI carbs.
Otherwise, presuming you consume an appropriate amount of carbs, it shouldn't be fattening. Moreover it could be argued that high GI carbs are actually less fattening now, because whilst the insulin spike is larger, its also shorter, so insulin will return more quickyl to a lower level allowing you to burn fat more quickly again