If you never say anything about it, you can't blame him for not doing it on his own. You can tell him you don't really like hanging around everyone when they're high, because you don't smoke and it's kind of uncomfortable to be the only sober person in the room. You'll spend less time together, and one of two things will happen - either he'll start cutting down to be with you more, or he'll distance himself 'cause he prefers getting high with his buddies to hanging out with you.
Personally, if my boyfriend was getting high all the time, it would bother me, and I'd probably get a little whiny with him. I don't want to be a whiny girlfriend either. I'd do my best to tell him what was bothering me so we could fix it together... I don't want to change anyone, but this is still my relationship too! I have a right to speak up when I'm not happy, just as my boyfriend does. If he was unwilling to meet me in the middle to resolve an issue that was threatening our happiness as a couple, I'd find another boyfriend.
A close girlfriend of mine is a complete pothead,and it's so boring sitting around with all her friends when they're high. I have nothing against it, I would just prefer not to be there. When she wants to get high, she can hang out with those friends. When she wants to get off her ass and do something, she can call me.