This don't sound right as a fully charged rx battery should read in axcess of 5 volts, perhaps as much as 5.3 which would give you a much larger window which will stay mostly within that range for the life of the charge. As the running time available gets close to the end the voltage will be at or close to the 4.7 or 4.8 readings which is the time to charge This will be about 2 hours of flying time on a 600 mill battery with the radio drawing a nominal 250 to 300 mill load.
This is however the outside parimeter of operation so you shouldn't run it down that low. It is in my opinion cutting it a little too close.