A standard rate gyro will initalize, and then every time it senses a movement in terms of degrees, it will tell the tail to go back that same exact ammount in degrees, sort of like a one shot deal. Tail moves 5 degrees, gyro sends tail 5 degrees back, end of story. It's cheap and effective.
However, as is common among all rate gyros, eventually the tail will begin to drift to one side or the other as the gyro only sends corrections as it senses them. Another problem with rate gyros is that during very sharp throttle changes, or in the event of a crosswind, the tail can lag behind or swing out, which of course, can cause problems if you're in fast forward flight or new to flying.

A heading hold gyro on the other hand is aware of it's position. When it's intalized, it will take note of it's heading. As the helicopter is running, the gyro will constantly give input to the servo, regardless of wether it's actually moving or not, to keep it as close as possible to it's set heading. As soon as you hit the rudder and point the heli in a new direction, the gyro memorizes the new heading and locks the tail into that position untill you tell it otherwise, and so and and so forth. In this case, crosswinds and sharp throttle changes do not affect the gyro because it's constantly giving input to the tail and so doesn't need to play "catch up" like a rate gyro would, so you don't get the swingout effect.

A good heading hold gyro can make all the difference in how a helicopter performs in the air, so it's usually wise not to go cheap on them.
Rate gyros work just fine, but you won't get the rock solid performance you get from a heading hold gyro.