Why would you feel terrible about it? Because you're not doing anything eventful? That's the point of a day of rest!!! Don't feel like you need to do something every day. I'm not saying be lazy all the time, but a day to do absolutely nothing once in a while is good, IMO.
Think of it like a recovery day from the gym.... you need physical breaks. You need mental ones, too. I know you're basically the busiest guy on the planet, so you're used to always up and working. Find some time to take a break once in a while, your body and mind will thank you!