I had a friend who flew choppers in the military and he said trying to hover is like rubbing your belly, patting your head, and playing the piano with your toes at the same time.
Interesting analogy, but the problem is that flying RC helis is even harder than the real deal. The reason being that when you are inside a real heli, your body becomes it's own gyroscope, AND your perspective never changes...forward is alway forward, left and right are always the same. With the RC, it's hanging out in front of you, so you are flying by sight alone, and when it turns around (nose-in), everything becomes opposite. Pushing forward on the stick brings it toward you instead of away, and right input will take the heli to your left, etc.
Learning to hover is one of the most difficult things to do, especially nose-in, but it is the fundamental building block for all other things. The real name for a helicopter is a "rotary wing aircraft". And, since the "wings" are in constant motion, the heli wants to be in constant motion, hence this is why it starts to drift when you want to hover...this is a constant. With an airplane, once you have it trimmed, you can point it in a direction, and it will go in that direction until told different, but a heli requires CONSTANT input to fly and especially hold it still in a hover. This is why I tout the sims SO much, because even though it may not react exactly like the real thing, it still teaches the "muscle memory" for what you need to do. The most common newbie mistake for hovering is overcorrecting. If the heli is set up & trimmed properly, you shouldn't need to do more than "touch" the stick to keep it still, but it is constant predictive action that almost becomes a sixth sense, where you will know what the heli is going to do before it happens, and you input the proper reactions almost before it happens. This is like the balance of riding a bike, except it's more like a unicycle on top of a beach ball...once you learn it, you never forget. It just takes a good bit of practice to become proficient enough where you don't even think about picking it up and turning the nose towards yourself, and keeping it rock steady.
I would also recommend a nitro over an elctric, especially for learning. The larger nitros are much more stable, where the little electrics can be blown over with the slightest gust of breeze.
Hope this helps, and good luck.