Generally speaking, rest means sleeping, a total absence of exercise, or any kind of training, however, recovery is a broader term and represents techniques and actions you can take to improve your body’s physical state and adapting to the training you've given it. Rest is an important part of recovery, sleeping especially, that's why recovery days are often referred to as rest days, but in most cases, recovery/rest days refer to adapting training load.
It involves more than just resting, it involves muscular recovery, but also mental recovery allowing chemical and hormone balancing and that’s what the world of sports know as an active recovery. It could be a gentle spin or if you're a runner a gentle run, stimulating vasodilation and perfusion of (not only) skeletal muscle tissue and consequently ensures metabolite leaching and improved oxidation.
The evidence of active recovery benefits is widely supported by professional athletes, who mostly state they feel fresher after an easy spin or other well-known recovery techniques like an ice bath, massage, stretching, all stimulating perfuse. To leave for a short easy ride or just rest depends on how the athlete feels. If you wake up and feel under the weather, take the whole day off the bike.
Science might have said that active recovery is better than passive but many very good professional athletes on their recovery days wouldn’t touch the bike or running shoes as that’ part of their mental recovery. If you’d rather not ride on your recovery day, then don't, don't worry about it as the difference is marginal between going for an easy spin or not.