That's another of the challenges -- this is a bit extreme, but it gives you a context - I compete in bodybuilding. I've had people tell me there is no other sport that is as demanding - because you "train" 24/7 - not meaning time in the gym, but also time modifying your diet, preparing your food, packign your food, carryign yoru food, arranging your day so that you do eat on time, as well as recovery time (sleep), cardio and generally keepign an even keel in your life to reduce the occurrence of cortisol spikes when you get stressed out.
I dont 'want to say you can't achieve your goals w/o impacting yoru daily schedule, but after you make the tweaks to your diet and get your training laid out, the next level of detail is to look at how consistently you are hitting your meal times. We base the idea of eating 5-6 small meals over the course of the day on the fact that your body metabolizes proteins, fats & carbs within 2-3 hours. If you grew up on the 3 meals/day thing - your body has adjusted to that. But for optimized "fitness" (meaning trying to most efficiently achieve specific body shaping & performance goals), we set up our activities on how our body runs. It is essentially a very fine-tuned biochemical machine. You fuel it w/ the right stuff, it burns at this rate (2-3 hours) and then you refuel. This sets up an effective burn rate if you fuel exactly what you need for whatever is the goal you want.
If you wanted to build significant muscle (I don't mean to look like Arnold Schwartszenegger, but to make a specific difference in what you have now), then you need to eat more protein to have the materials to build muscle and more carb to provide the energy for harder lifting to build more muscle & strength. But you will also have some left over carb that is deposited as bodyfat -- this reflects the idea that you can't really build muscle and burn fat at the same time. If you wanted to cut bodyfat, like for competition , there is a point where you will cut the bodyfat but you can't expect to build muscle - in fact you hit a point where your typical max lifting weights will start to drop as well simply because you arent' eatign for the purpose of maintaining strength w/ a competition cutting diet.
So where I was going with this -- if you are random about when you eat, then your body has a hard time setting up an average burn rate because it doen't really know when to expect the required about of food to come. So for example, yesterday I happened to be out running errands and hit my next meal time - since my diet is very optimized right now, if I don't feed when its feeding time, my body starts to freak out- I get dizzy in the head and feel like I'm going to die. It does pass, but it sucks. That's tellign me that I've added a small hiccup in the efficiency of fuel / burn / fuel / burn process that I'm trying to setup so I can hit that optimized rate of bodyfat burning. That will keep me from hitting optimum dieting when I have a target date in mind where I need to be a certain very lean bodyfat level and every additional time I screw up my schedule may hinder my ability to hit my date w/ my goal.
So its not saying it completely destroys my progress, but it will slow it down some. If you keep doing this --random eating time, not enough food, etc then you won't see fast progress w/ your goals. Its taking that extra step to do as much as you can to commit to eatign at the scheduled times that will get you resutls quicker.
But I think in your case, your first level of optimization is to look at your diet. I really think you aren't eating enough. Also like Bunny asked - what else is involved? Do you lift weights? Cardio? What activities do you do?