Taken to the extreme, which is your specialty, you are essentially right. Which is why the FAA is doing what they are doing, they know that in fact there is nothing preventing a hobbyist from doing pretty much anything they please as long as it does not violate any other laws, rules, or regulations.