Changing your gender goes a lot deeper than appearances. It's not about "I don't like being a ____" it is much, much more. I know plenty of people who aren't crazy about their gender, or aspects of it (probably more chicks than dudes, actually) but if you ask them "Do you WISH you were a ____ instead?" They'll usually say "Well, I'd love to try it on for a few days, but to change permanently? Oh, hell no."
TG people making the change are literally to the point where their lives are utterly miserable because their bodies are almost prisons to them.
As for hormonal enhancement for cosmetic purposes. Let's be honest, there are potential dangers of taking long term exogenous hormones and those can be anything from life threatening to permanently damaging to major body systems (for heaven's sake, just look at the potential dangers of a woman taking birth control pills). But more importantly, the fact is, hormonal enhancement isn't going to work the way it should if the patient isn't going to put the sweat equity in. I could see an awful lot of patients being lazy SOBs who want an easy fix just to have abs, bis and a chest. Honestly, look at how many lazy bastards that come into the AAS forum "ready to take the plunge" and don't know the first thing about proper diet and have a totally piss poor exercise regimen AND even after you tell them, time and again what they're doing wrong, and what they need to do before making this commitment, they blow you off and just keep doing what they're doing, then spend buckets of cash on "supplements" and then don't understand why it doesn't work the way they expected it to and all of the sudden they're impotent, or have no libido, or can't get PCT to fix their gonads.
People are generally too stupid and lazy to have the option to ask for legal juice for cosmetic purposes, it would just leave the doctors in vulnerable positions in terms of liability and consider the ripples that could reach into things like amateur sports, and even school athletics. Hell, kids are stealing their parents prescription drugs to get high now, how easy would it be to steal dad's test to get onto the football team and cover yourself by refilling the vial with something out of the kitchen cabinet?