Just the nature of the beast, happens all the time. Years ago I went to just a glow stick or driver and that makes the problem go away. During the winter when it gets down into the 20s and 30s I do use the driver from my power panel. I can crank it up and get more heat to the plug. Only thing I use my panel for these days is my electric fuel pump. I wouldn't use it for that but I have some big tanks in my planes and I really hate hand cranking.
Just buy a glow driver or watch the controls dance for you. I have never broken a servo because of the dancing.