AMA Safety Code, General, Rule 2:

I will not fly my model HIGHER THAN 400 ft within 3 miles of an airport without notifying the airport operator.

You would be suprized how high 400 ft is... The plane is looking fairly small by the time yuo are that high. (it will be appx 600 ft from the pilot, at closest pass, when its 400 ft up)

Go watch a Pattern Contest. The Masters pattern has the plane MAYBE 600 ft above ground at the top of the square loops. (its WAY up there) 60 deg angle up, appx 250 ft away from the pilot (straight line on ground to be under the plane) You can do that math for altitude can't you?

Now... how often do you fly that high?