If a gust caught the plane, it doesn't matter how high it is. If the AIRspeed of the plane isn't sufficient to to produce adequate lift RELATIVE to the air's motion and speed, the plane will stall (that's what airspeed is). Ground speed means absolutely nothing to the lift production of a wing. If you were close the stall speed already and then gave the plane up elevator, you accidentally contributed to the stall. If you were at full power, then the stall was a power-on stall and those are more violent than power-off stalls.

If you're positive that the servo was not reversed (and it certainly sounds like it was not), then I can't think of any other reason for the crash outside of a stall.

How fast was the wind blowing? Was it gusting? What kind of plane were you flying? Planes designed to go fast or with symmetrical wings are not going to be very forgiving in this kind of situation.

I hope you can rebuild it.