Well in some controlled areas such as near airports the FAA control is to the surface. So why does that not exempt the pilot from trespassing?
The pilot (and UAS operation) is never exempt from trespassing, which is a civil matter and could become an FAA matter. I have worked with a drone R&D program, and we would operate it as if it were manned (except for the crashing part).
