It was the hinges. The first photo shows the right elevator, the fiber material is completely gone and the hinges are slick. The second photo is the right aileron, it is a better photo of the way the hinges pulled out.
I agree, in the future, I will pin all hinges that are not hinge points. I should have known this already. However, this was my first 3D type plane and I didn't think about it. Since 1991 I have only seen a ca type hinge pull out one time. That was on an Extra 300S and it was my own fault, I had not put enough ca on the hinge. I should have learned then but it was sometime ago and I had forgotten.
No where in the instructions were there any indication that I should pin the hinges, hinges were only mentioned one time in the attachment of the rudder. I had thought it looked like the elevators were double hinged, two side by side, I thought that's a good idea so when I hinged the rudder that's what I did. But, I used actual GP hinges that I had on hand. And by the way, they did not look anything like the hinges provided with the plane for hinging the rudder, they looked like the cheap hinges I had gotten with a Seagull model.
Oh, and by the way, I had pulled on the control surfaces, not the day of the first flight but a couple of days before. They seemed fine at the time.