dropping calories may be getting you back on track but is not the idea way of making this a way of life; and that is important for this to work over the long term.

The key part to all the low carb diets is to kick into ketosis. This can be done in a fasting way. I wonder if you are not staying in a ketotic state and that is why the fasting is working for you.

It is only a speculation: Two things. How many carbs do you think you are eating? and how much cream?, and do you have those ketone pee strips to test your urine?