We have one of the best instructors in my opinion and he has taught hundreds of folks of all ages to fly over 40+ years. He has said there have been a lot who take a very long time to get the basics down and retain it from flight to flight and over the off season they tend to get very rusty and almost have to start over again. He also told me there were one or two folks who simply could not get it. Even though they had the drive and determination it was just something they couldn't fully grasp.
Everyone is nervous when learning and it takes awhile to become confident. Hey we all get a little weak-kneed when taking up a new plane for the first time , especially when it's a big expensive one !
Hang in there and it will come to you and things will settle down during your flights!