I formally studied martial arts for many years. Ishin Ryu was my base. I loved it. It was hard, heavy, and effective. It isn't flashy showy by any means, but it certainly gets the job done. I've studied some southern shaolin Kung Fu, Tai Chi, and rounded it off with multiple forms of yoga.
Studying something that trains your body, mind, and spirit, AT ANY AGE, is a positive. They're right, you probably won't be competing, but you can build a tremendous connection between mind/body/spirit, as well as learn to understand and utilize the flowing energy of this universe.
Good luck to you. And hard work.