No you are right. But most -- i.e., non-champs or non-competitors -- with early signs of negative side effects never get that far because the cost is too great to them to continue taking aas. So, you're right, top guys in 60s and 70s without anti-es had no gyno or balding, altho Franco had a comb over in Pumping Iron. (so genetics is an important factor)
Also, after Arnold retired, Franco in later pics has a unilateral case of gyno. Maybe he tried some different drugs. And Arnold, it appears, like Joe Weider, has a nice head of hair but seemingly professionally done (cosmetic hair replacement). So, eventually age, at least, caught up to the greats like Franco and Arnold, and possibly changing drugs (maybe Franco to win overall started using test or anadrol without anti-e after Arnold retired.) Who knows?
Good point tho Galaxy, just wanted to expound.