Fifteen years ago, a stall in the downwind turn destroyed my first RC plane - a glider - because I had no idea what I was doing. Recently I've had two incidents on the downwind turn on gusty days where airspeed has dropped low enough that a gust has stalled and then flipped the plane inverted.

Why would your airspeed downwind be 20kts less at a given power setting just because you are making a turn downwind?
I believe the problem isn't just one of ground speed / perception. The issue stems from momentum; as the longitudinal wind vector switches to a tailwind, the power setting which previously maintained airspeed now needs to increase airspeed. Once airspeed has increased to its desired level, the power setting must be returned to its previous level so as not to continue accelerating the aircraft.

I note that a lot of the guys at the club don't seem to have the issue I do - I tend to fly circuits at half throttle and close to stall speed (or even glide them for fun), whereas these guys who fly WOT all the time never come across this issue because they're always well above the stall speed as they make that turn.