Nimh cells drop voltage very quickly towards the end of their charge. When the battery is nearly depleted your headspeed will drop quite a bit making the helicopter a mess to control. It's not you, just your batteries .

To avoid it, i'd suggest going to lipo power, you can't drain the packs down like you would nimh without damaging the cells beyond repair, so you don't need to worry about losing power towards the end of your flight. Your overall headspeed will remain (for the most part) consistent throughout your flight.