You didn't say what equipment you have. I'll assume you're flying a nitro heli and not an electric (no need to check the RX batt since the same batt powers the heli's motor). To check the RX battery you need a tester which places a load on the battery. A voltmeter alone won't do it since under load the voltage will be lower. Several different ones are available, some digital some analog.

The TX may have a analog meter in which case you need an external meter to check it. The meter that's on the TX may be reading RF output and if it is reading voltage they usually only give you a relative indication. The same VM you buy for your RX battery can be used for the TX. If your TX is a computer radio it will have a digital display of the voltage so you don't need to put an external meter on it.

Hope this helps,

Mike