You should forgo those things from time to time to do meditation retreats, if you can't do 9+ day retreats then even a day here or there is better than nothing. I don't think it matters what day you do it, no need to wait for an upsosatha day.
"Whenever we feel that we are definitely right, so much so that we refuse to open up to anything or anybody else, right there we are wrong. It becomes wrong view. When suffering arises, where does it arise from? The cause is wrong view, the fruit of that being suffering. If it was right view it wouldn't cause suffering." - Ajahn Chah
"Remember you dont meditate to get anything, but to get rid of things. We do it, not with desire, but with letting go. If you want anything, you wont find it." - Ajahn Chah