When you are on a diet, do you maintain the same amount of food even on days you rest from exercising?

I am wondering if my meal portions should differ since my activity levels vary between exercise days and rest days.

It is difficult to decide whether it is better to eat less than usual on days I don't exercise, or if it is better not to drastically change my meal portions for recovery.

For those of you who have lost weight through consistent exercise, how did you manage your meals on your days off? I would like to know if you also used hunger or fatigue levels as a criterion.