Depends on how you fly and where you fly. The 450's are the biggest I'll fly in my backyard (urban area). I'd say to get the biggest you can but you're right, the cost of batteries for the bigger helis are hard on the wallet. If you do get the 500, remember to lube the tail as I've heard of static build up problems causing loss of radio signal in some cases.

Remember that the electronics will cost more on the 500, not just the kit. Mini servos (like the 9650) cost $55 vs $33 for the hs65mg... etc.