Its usually just a bad case of nerves. Have you tried flying with an instructor using a buddy box? If you don't have to worry about crashing, it will give you a chance to relax, fly the plane, and build your confidence as to what you are capable of. A few flights like that on the same plane should give you enough confidence to go it alone and keep the nerves at bay. Even experienced flyers still get the nerves when it comes to maidening a brand new plane or the first flight at a contest.