ORIGINAL: 2slow2matter
I didn't even know that CA hinges were actually a plastic piece with fiber around them. The fiber part that is on the outside (that actually absorbs the glue and bonds with the wood) was still glued securely to the wood on the TE of the wing. The smooth plastic inside part of the hinge was sticking out of the aileron.
That is exactly what happened to me on the elevator and rudder of a plane I purchased second hand.
I thought that the builder had installed poor plastic hinges, but now that I think of it, it is exactly as you describe...
The "inner" part of the CA hinge was still in the control surface, shiny and smooth.
The fibers were left in the elevator and rudder along with the CA...
The plane is no more.