Yep, improve the core. If the pain persists I would see a doctor just to make sure you don't have a mild slipped disk or something of that nature. Sit-ups are pretty low impact, so I couldn't see how it could be hurting your lower back without something being wrong.