Seems as though maybe the hole was drilled slightly closer to the hinge line on the bottom of the surface and further away on the top, is this correct? If so simply glue in a hardwood dowel in the hole, re-drill hole.
I had to do this on a rudder during assembly, since it was not glued on yet, I used my incidence meter and made sure the rudder was at 0° on my drill press and voila instant perpendicular hole.