Servos draw the most current when given a quick stick command, which is why you see it flicking down to red when you do. That's normal.
Your battery is probably fine. You just have to understand what a Voltwatch can and can't do for you.

Voltwatches are handy because they show you the voltage that your system actually sees and have a very fast refresh rate. So they will show you voltage drops that a traditional meter would not. That's why it flicks to red when you work the sticks. What a Voltwatch can't do is tell you the condition of your battery or how discharged it is. It also doesn't give you an actual voltage reading, but rather just "caution" or "low." I've owned one in my 13 years doing this hobby, pretty early in the process. I found it enlightening to use, but less useful than other methods of testing batteries. The real value of the Voltwatch is its convenience to do a quick check and look for changes in the way the battery behaves. You can learn what is normal and easily spot what is not.

If you want to use your Voltwatch to tell you if your battery is good, hold a control surface still and deflect the transmitter stick. That will put more load on the system than it will see in flight. Watch the Voltwatch for 2-3 seconds and release (you can overheat your servo motor by doing this too long).

If you want to really know the condition of your battery, spend a few dollars on a computerized charger. The Imax B6 or one the many clones of it is cheap and will work fine. You can do the discharge test that I recommended above and also see how your battery's voltage changes as it discharges under a constant load. To kick it up a notch, date your battery and do a discharge test every 3 months that you write in a log. That will let you spot deterioration and know if it needs to be replaced before it causes you a problem.