Damn right. We got lots of little guys wearing beaters, and I got absolutely no problem with it.... the only ones that irritate me are the ones that swagger around with full blown ILS (imaginary lat spread) going. I'm a decent size, not huge (5'11", 221, ~8-9%), and I wear a tank top and sweatpants. It's comfortable, cooler, and I don't give a crap if it gets ripped to pieces.
Dress how you like, people. Sound like a buncha girls. Do you criticize what people wear when they walk by you on the street? Live and let live... obviously these little dudes don't think they're huge, so just lay your own dismorphia issues to rest and lift some damn weights.