This has been an interesting discussion, but here's what I believe is the key language in Section 336 of Public Law 112-95, a.k.a. "FAA Modernization and Reform Act of 2012."

(b) STATUTORY. CONSTRUCTION. —Nothing in this section shall be construed to limit the authority of the Administrator to pursue enforcement action against persons operating model aircraft who endanger the safety of the national airspace system.

With very few exceptions, the air above most public and private land is part of the national airspace system. Regulations define the national airspace system and classifies all airspace in the US as class A, B, C, D, E, or G (F does not apply in US). Class G is defined as surface to 700 AGL (or 1200 AGL in designated mountainous areas). So even model aircraft that are operating 400 feet AGL and below over private property are indeed operating within the "national airspace system. The fact that ATC does not control class G ops is moot. So all FAA has to do is demonstrate that the aircraft is in the national airspace (i.e. airborne) and that it's being operated in a way that endangers.

That's more than enough to go after someone who may even be operating "over" private property. While some may get a lawyer and try to make a fight of it, on the issue of safety I expect the courts will defer to the FAA. Therefore, so long as the careless few continue to create media events that draw negative attention to our operations, then the FAA will continue to be under pressure to do something to address the perceived dangers.