I agree it could be dangerous if you have the pits filled with cars and people, but that is not always the case. You may not even have pits if it is just a field for your own usage and in that case it is safe to fly how you want, where you want.

The limits should be able to be turned off. Just like in real life, sometimes you can fly there and sometimes you can't.