At the end of the day, the MOST important thing is your daily calorie intake and macronutrient intake at the end of the day. Meal timing is still important, but it's like 5%. Your daily totals is the most important thing.
I don't have a big appetite at breakfast either. I still consume a protein shake though.
Just think about it logically. At breakfast, you are coming off a 8-10 hour fast. Eating a high protein breakfast will prevent catabolism and get you back in a positive nitrogen balance.