I exfoliate right before shaving. But there is no issue with exfoliating on off days, as far as I know. As long as you aren't using anything too harsh. IMO, of course.
Exfoliate anytime you want. I hear people say you should restrict it to a couple of times per week but I do it daily in the shower. I've never had a problem with my routine, my skin seems to benefit from it, and I swear it helps with my daily shave results.