I agree with what everybody else has said. Most important thing is to not become too obsessive with it though. I took a vacation for 10 days this summer and I didn't workout once (wanted to do all I could while traveling, so that was the focus for my time). I don't feel like I lost too much.
A little time off won't kill you as long as you aren't making poor decisions during that time (going completely sedentary, eating poorly, etc.). Again, don't obsess over it though. Like Steve mentioned above that mindset can cause you to push things and get injured.