FAA's latest sighting data... AMA's strategy of "education" not working?
#1
Thread Starter
FAA's latest sighting data... AMA's strategy of "education" not working?
Yesterday, FAA released the latest UAS sightings: https://www.faa.gov/news/updates/?newsId=87565
With data files here: https://www.faa.gov/uas/resources/uas_sightings_report/
I combined them into a single excel file and put together some analysis, conclusions, and one opinion
Since FAA began collecting this data, AMA has contended that regulation is not required as education will solve the problem. With the trends pretty clearly established, I've come to the conclusion that the AMA's "Education" strategy is not working.
With data files here: https://www.faa.gov/uas/resources/uas_sightings_report/
I combined them into a single excel file and put together some analysis, conclusions, and one opinion
Since FAA began collecting this data, AMA has contended that regulation is not required as education will solve the problem. With the trends pretty clearly established, I've come to the conclusion that the AMA's "Education" strategy is not working.
#2
My Feedback: (3)
You cannot educate those who are not interested in what you have to say. Those not involved with the AMA most likely won't avail themselves of information offered by an organization they have no interest in. Bound to fail from the start. I am Canadian and have no dog in this fight but from the onset it seems like those not involved as members feel the AMA is attempting to interfere with they're fun. In Canada our organization was asked to help with aspects of regulation/education of all members and non-members alike. The organization turned that down and instead offered to extend the safety guidelines that have served the members so well for years, to all who were interested. Transport Canada (the FAA in the US) were more than welcome to use/adopt/advocate the safety guidelines of the organization but the organization as a not for profit entity could not become an organization for non members. At present our organization has carved out privileges for members that grandfather how we have been operating at our fields. This includes FPV, flying at altitude (gliders etc,) and so on that non members may or may not find themselves restricted from doing. Our Transport Canada is the only enforcing authority with the ability to fine/charge offenders but as long as we follow the agreed upon guidelines we are exempt from the more stringent restrictions. Our organization has gone to great lengths to educate the public and the governing/rule/law making entities that members operate at an elevated level of expected safety separating the organization and its members from the general public who may or may not be aware of what is expected of them as owner/operators.
Dennis
Dennis
#3
Thread Starter
You cannot educate those who are not interested in what you have to say. Those not involved with the AMA most likely won't avail themselves of information offered by an organization they have no interest in. Bound to fail from the start. I am Canadian and have no dog in this fight but from the onset it seems like those not involved as members feel the AMA is attempting to interfere with they're fun. In Canada our organization was asked to help with aspects of regulation/education of all members and non-members alike. The organization turned that down and instead offered to extend the safety guidelines that have served the members so well for years, to all who were interested. Transport Canada (the FAA in the US) were more than welcome to use/adopt/advocate the safety guidelines of the organization but the organization as a not for profit entity could not become an organization for non members. At present our organization has carved out privileges for members that grandfather how we have been operating at our fields. This includes FPV, flying at altitude (gliders etc,) and so on that non members may or may not find themselves restricted from doing. Our Transport Canada is the only enforcing authority with the ability to fine/charge offenders but as long as we follow the agreed upon guidelines we are exempt from the more stringent restrictions. Our organization has gone to great lengths to educate the public and the governing/rule/law making entities that members operate at an elevated level of expected safety separating the organization and its members from the general public who may or may not be aware of what is expected of them as owner/operators.
Dennis
Dennis
We now have "education" sheets included in many (if not most) sUAS sold commercially. We have "education" sheets at hobby shops. We have AMA's website "education" and their safety code. Even on their own website, the "education" and safety code are poorly executed. Site is visually VERY confusing, and both education and safety code are buried one or more layers deep - where you have to first search tabs to find the hyperlink, then follow it. If "education" and the code were so important, you'd think they'd be giant buttons on the main page. But they're not.
None-the-less, all of this "education" and safety code does not appear to be working. No way about it, the AMA needs to face the fact that their messaging is not providing results. Feds, state, and local were told AMA education would keep the numbers from going up, and that's clearly failed.
#4
My Feedback: (3)
We face the same issues and there are major sightings of drones or drone like objects in restricted airspace here as well. Why is it that we do not have the issues the AMA seems to have with our governing body? One major difference I seem to have come to understand is the AMA chose to become involved with the management of small UAV's outside its membership. Without any authority to do anything to enforce the rules/guidelines the AMA was a toothless governing body. They might better served the membership by declining and concentrating on the membership only. In Canada that is what our executive decided. Attempting to manage outside the organization without the authority to enforce or penalize offenders was an impossible task. If I am wrong about the position the AMA took I apologize but that is my understanding of the position they took.
Other than that except for when I choose to fly in the US its a moot point. What is done is done all that is left is damage control. I don't envy the executive on both sides of the border having to deal with these issues. I just hope we don't suffer as a result. Yes I do have my FAA number to fly in the US. As soon as they allowed us to register I paid my 5 bucks and did so. I enjoy the company of my US friends and don't want to loose the ability to fly at the fields I have frequented in the past.
Dennis
Other than that except for when I choose to fly in the US its a moot point. What is done is done all that is left is damage control. I don't envy the executive on both sides of the border having to deal with these issues. I just hope we don't suffer as a result. Yes I do have my FAA number to fly in the US. As soon as they allowed us to register I paid my 5 bucks and did so. I enjoy the company of my US friends and don't want to loose the ability to fly at the fields I have frequented in the past.
Dennis
#5
Junior Member
Join Date: Dec 2016
Posts: 2
Likes: 0
Received 0 Likes
on
0 Posts
FAA data is taken as gospel? Isn't the FAA the same operation that take reports from airline pilots who swear that they saw a drone but it later turns out to be a shopping bag? How do you get from increased drone sightings to blaming the AMA for failing somehow in their education programs. I think the AMA did their own review of those reports last year and debunked most of them. Are they now responsible for everyone who flies a drone? I'm not saying your premise might not have some merit, but it feels like you took one data set and backed into a predetermined conclusion, then put some sizzle in there with some fancy graphs. Doesn't seem too scenitific, nor a reasonble conclusion to reach. Propworm makes the most salient point, you can't educate those that are not interested in what you have to say.
#6
Thread Starter
But what's interesting is that the AMA report was almost immediately debunked by the Bard Center for the Study of the Drone. They did a genuine and scientific analysis of the same data set (very first release) and, not surprisingly, found much more than AMA did. In the Bard report (note 3), they found that "Of the 552 incidents in which a distance from an airport was recorded, 324, or 58.6%, occurred within five miles" (ibid.). Unlike AMA looking for specific terms only, the Bard Center evaluated reported miss distances, and they found that "Of the 219 incidents in which a near miss distance was provided, the average near miss distance was 374 ft. and the median near miss distance was 200 ft" (ibid.).And again, unlike AMA, the Bard report analyzed the altitudes at which the sightings occurred, and they found that "Of the 627 incidents in which an altitude was recorded, only 61, or 9.9%, occurred below 400 ft, which is the legal ceiling for recreational drone operations (the ceiling for commercial operations is 200 ft.). The median altitude for incidents was 2,200 ft, and the average altitude was 3,306 ft" (ibid.). We can argue all day long about how much risk that presents, but the reality is it's not our perception of risk that matters, it's the public, regulators, and the courts...just ask the club in PA (note 4).
So while AMA says there are few problems, Bard finds that there's over 300 within five miles of an airport, over 200 where the drone missed a manned aircraft by an average of 374 feet, and 560 events occurred well up in the altitude structure. That is a risk to manned aircraft by any measure ... one AMA seems content to try and ignore.
Are they now responsible for everyone who flies a drone? I'm not saying your premise might not have some merit, but it feels like you took one data set and backed into a predetermined conclusion, then put some sizzle in there with some fancy graphs. Doesn't seem too scientific, nor a reasonable conclusion to reach. Propworm makes the most salient point, you can't educate those that are not interested in what you have to say.
Regardless, their basic premise was wrong. AMA actively pursued "drone" members, and it's not working out. While drone sales grow and registrations rise, AMA adult memberships are down, AMA senior memberships are down, and the conversion rate on all those free AMA youth memberships is extremely low (Note 5 - EVP report). Some argue, and I agree, that had AMA created a clear distinction between their existing membership and the "drone" community, we might not face the problem of "painted with the same brush" as AMA itself admitted.
As for the "one data set", it's actually five, Nov14-Aug15, Aug15-Jan16, Feb16-Mar16, Apr16-Jun16, and Jul16-Sep16 (note 6). Furthermore, it's a federal reporting system of record, so it matters. If you actually go read the reports, there's 1300 of the reports that were significant enough where law enforcement was contacted. There were forty seven that occurred inside class B or class C airspace. I'm still working the data, but even being charitable, it's not a picture of success.
On the comment that the graphs add "sizzle?" Thank you. I do operational risk management and business operations consulting for a living. Much of my work involves taking abstract data and putting it in visual format for interpretation. But the numbers don't lie - unless of course you choose to believe that each one of the over 2600 reports is false, and that the Bard Center's report did not find the data they found, and all that matters is that AMA didn't find a lot of reports where a guy in a tower or ATC facility typing a free form field did not use the specific phrases "near misses" or "near mid-air collisions.”
1. 30 Jan 2015 - http://amablog.modelaircraft.org/ama...ot-regulation/
2. https://www.modelaircraft.org/gov/do...ata_091415.pdf
3. http://dronecenter.bard.edu/drone-si...d-near-misses/
4. http://www.berksmontnews.com/article...NEWS/161219966
5. http://www.modelaircraft.org/aboutam...ecminutes.aspx
6. https://www.faa.gov/uas/resources/uas_sightings_report/
Last edited by franklin_m; 02-25-2017 at 08:39 AM. Reason: Corrected notes numbering, fixed awk sentence.
#7
Thread Starter
Here's two additional slides containing 8 more charts of data. Day of week is easy. To get LEO involvement was much harder. I apologize, but it's a bit technical, so I'll explain.
Information about that is in a free text field and there isn't a lot of consistency. So I went through a couple hundred of them individually and developed a "positive list" of 75 terms that indicated LEO were involved, for example "LEO Notified". But I also developed a "negative list" 29 terms that positively indicated LEO was not engaged, for example cells containing "No LEO Notified." And therein you see the challenge. So what I did was do a test to say if a cell "hit" on the positive list AND on the negative list, I did not count it. If it hit on only the negative list, I did not count it. So all you see are ones that hit on the positive list and NOT on the negative list. Granted, this probably under-estimates how often law enforcement was involved. Regardless, it's still not a good trend we see.
I did a slightly different method for the airline / air cargo sightings. I was able to come up with 36 named carriers that seemed to show up more than others. I also came up with basic aircraft types that would capture those other few that might not match the named list, but aircraft type is pretty consistently issued. For those I used B737, B747, CRJ2, CRJ7, and CRJ9 (regional carriers tend to use these). Again, this probably vastly under-estimates the number of reports by professional aviators.
In both cases, this is a compromise between trying to get data consistently and the time it would take to go through each of the 2000+ reports individually. But no matter what, even under-estimating LEO engagement and reports by professionals, the trends are alarming (to me at least).
Information about that is in a free text field and there isn't a lot of consistency. So I went through a couple hundred of them individually and developed a "positive list" of 75 terms that indicated LEO were involved, for example "LEO Notified". But I also developed a "negative list" 29 terms that positively indicated LEO was not engaged, for example cells containing "No LEO Notified." And therein you see the challenge. So what I did was do a test to say if a cell "hit" on the positive list AND on the negative list, I did not count it. If it hit on only the negative list, I did not count it. So all you see are ones that hit on the positive list and NOT on the negative list. Granted, this probably under-estimates how often law enforcement was involved. Regardless, it's still not a good trend we see.
I did a slightly different method for the airline / air cargo sightings. I was able to come up with 36 named carriers that seemed to show up more than others. I also came up with basic aircraft types that would capture those other few that might not match the named list, but aircraft type is pretty consistently issued. For those I used B737, B747, CRJ2, CRJ7, and CRJ9 (regional carriers tend to use these). Again, this probably vastly under-estimates the number of reports by professional aviators.
In both cases, this is a compromise between trying to get data consistently and the time it would take to go through each of the 2000+ reports individually. But no matter what, even under-estimating LEO engagement and reports by professionals, the trends are alarming (to me at least).
#8
My Feedback: (3)
Franklin I attend quite a few meetings at work, budget, work shop, production, safety etc and we have a name for going on an on burying everyone in graphs, charts and other obscure but useless information. Usually done when someone is trying to justify they're position, impress your peers or make points with upper management. Its know as "Baffling with bullsh#te." May I suggest you run for office, position within the AMA since you seem to think you might do a better job.
Dennis
Dennis
#9
Thread Starter
Franklin I attend quite a few meetings at work, budget, work shop, production, safety etc and we have a name for going on an on burying everyone in graphs, charts and other obscure but useless information. Usually done when someone is trying to justify they're position, impress your peers or make points with upper management. Its know as "Baffling with bullsh#te." May I suggest you run for office, position within the AMA since you seem to think you might do a better job.
Dennis
Dennis
I spent over 22 years in operations, and I can't imagine trying to do that without data. Now I consult in operating discipline / business operations. I have noticed a curious phenomenon. That is when people resist and criticize efforts to manage better using data, it's usually in defense of "the way we've always done things." I've also found it to be a way to try and avoid accountability for lackluster performance. These are general observations of course, but they are consistent across several different workforces in several different industries.
While I'm sure that doesn't apply to you. I would say that one cannot ignore the numbers. Heck, download them yourself and do the analysis and prove me wrong! I welcome the scrutiny. But most won't do that, because the numbers don't lie. In every case, I've gone to great pains to under estimate any bad news data sets. And yet the bad trends are still there. You can argue magnitude, but you can't argue the trend.
AMA tried to debunk the data once before. They did it in a rather unscientific and slipshod way, they searched for such a narrow set of terms that of course they rarely found them. This was proven to be a poor analysis by the Bard Center's report, that found substantially more near misses than the AMA cared to acknowledge.
So, if you have difficulties with the data, I challenge you to download it and do your own analysis. I'd be curious to see if you come to different results, and of course on your methodology.
#10
My Feedback: (3)
No insult intended just a personal observation from years of attending meetings. First of all its unverified data, gathered by one source, unconfirmed for its accuracy by any recognised third party, as such it has little or no value. Second of all what is the point of collecting data such as this unless one were going to use it to improve the method of operations. Using this unverified data to attempt to come out on top in a discussion is winning by attrition. You just keep piling it on until no wants to hear it and they go on to another topic. I've been asked not to discourage you as there are a few others of the same ilk and you keep each other busy leaving other worth while threads in peace. So on that note have at it and enjoy yourselves.
Dennis
Dennis
#11
Thread Starter
"Unverified data?" Again, unless you can prove that over 2000 reports are false, you have to accept that some number of them are genuine. So what percent are completely false? 50%? 25%? or 10%? Pick a number. Even if 50% of them are false, that still leaves over 1300 valid reports. That's certainly actionable. As for using it to improve operations? You bet. Focus efforts on CA, FL, & NY. In fact, focus efforts in those top five cities. Data analysis like this is used to improve operations by helping you focus efforts on getting the most bang for the buck. This clearly points toward where to look first.
You just keep piling it on until no wants to hear it and they go on to another topic. I've been asked not to discourage you as there are a few others of the same ilk and you keep each other busy leaving other worth while threads in peace. So on that note have at it and enjoy yourselves.
As for you backing away because some unseen person said to do that...how noble. But it's convenient in that it allows one to make assertions w/o being forced to provide proof. I've provided assertions. I've backed them with analysis. Supported them with providing the source of the data. Heck I've even provided the methodology. So if you're going to say the data is bad, I think it's time to prove it and share with us why.
Last edited by franklin_m; 02-26-2017 at 03:06 PM.
#12
Join Date: Sep 2009
Location: Aguanga,
CA
Posts: 1,779
Likes: 0
Received 0 Likes
on
0 Posts
No insult intended just a personal observation from years of attending meetings. First of all its unverified data, gathered by one source, unconfirmed for its accuracy by any recognised third party, as such it has little or no value. Second of all what is the point of collecting data such as this unless one were going to use it to improve the method of operations. Using this unverified data to attempt to come out on top in a discussion is winning by attrition. You just keep piling it on until no wants to hear it and they go on to another topic. I've been asked not to discourage you as there are a few others of the same ilk and you keep each other busy leaving other worth while threads in peace. So on that note have at it and enjoy yourselves.
Dennis
Dennis
#13
My Feedback: (28)
So do people think more regulation is going to solve the issue, because that is not a path you want to travel. As with all government regulation, there is never enough regulation and soon enough there will be no RC flying by anyone except the very few that have connections.
Regulation never works. We are seeing that today in every aspect of life.
Jobs going overseas...too much regulation
Gun crimes through the roof.....too much regulation
Insurance deductibles through the roof....too much regulation
And the list goes on...
Be careful what you wish for.
Regulation never works. We are seeing that today in every aspect of life.
Jobs going overseas...too much regulation
Gun crimes through the roof.....too much regulation
Insurance deductibles through the roof....too much regulation
And the list goes on...
Be careful what you wish for.
#14
Thread Starter
So do people think more regulation is going to solve the issue, because that is not a path you want to travel. As with all government regulation, there is never enough regulation and soon enough there will be no RC flying by anyone except the very few that have connections.
Regulation never works. We are seeing that today in every aspect of life.
Jobs going overseas...too much regulation
Gun crimes through the roof.....too much regulation
Insurance deductibles through the roof....too much regulation
And the list goes on...
Be careful what you wish for.
Regulation never works. We are seeing that today in every aspect of life.
Jobs going overseas...too much regulation
Gun crimes through the roof.....too much regulation
Insurance deductibles through the roof....too much regulation
And the list goes on...
Be careful what you wish for.
What AMA (and others) are missing is that given a media that is hyping these events, and a public generally scared of them, it may only take one serious event before people are clamoring for legislators to do much more. It's not OUR perception of risk that matters, rather the media driven public, legislators, regulators, and the courts. Despite all the "influence" and AMA Government Relations activity, there's now a superior court decision that affirmed that "The record clearly demonstrates that the speed, size and weight of the model airplanes and jets have increased over the past five decades to characteristics where safety needs to be the primary concern." Keep in mind, the activities that landed them in court were all done by AMA members - you had to be AMA to fly at this particular field. AMA's needs to focus on cleaning up the operational discipline of their own members first and stop saying "it's the other guys." For this court alone demonstrates that AMA folks are indeed part of the larger safety problem.
WE don't get to decide what is and is not a risk, others get to do that, and their risk tolerance is clearly not as high as ours. Therefore I contend that we are one serious event away from draconian regulation. Yet some, including AMA, seem to be content to do more of what's already demonstrated to not work (data above). Meanwhile, these events continue to happen, there continue to be near misses with manned aircraft. Sooner or later those dice are going to come up snake eyes. By then it will be too late.
http://www.berksmontnews.com/article...NEWS/161219966
#15
My Feedback: (6)
Ah yes, the pronouncements from on high from another Washington bureaucracy and of course one must believe everything Washington tells us as they are the ones with all the data.
The real truth is Washington lies and disinforms Americans on a daily basis, why should anyone take anything the FAA says as gospel? Another bureaucracy attempting to justify more money for FY 2017.
There is an old saying the figures lie and liars figure.
The real truth is Washington lies and disinforms Americans on a daily basis, why should anyone take anything the FAA says as gospel? Another bureaucracy attempting to justify more money for FY 2017.
There is an old saying the figures lie and liars figure.
#17
Ok Dennis , since you mentioned it , a fair question here would be ; Asked by whom ?
Ah yes, the pronouncements from on high from another Washington bureaucracy and of course one must believe everything Washington tells us as they are the ones with all the data.
The real truth is Washington lies and disinforms Americans on a daily basis, why should anyone take anything the FAA says as gospel? Another bureaucracy attempting to justify more money for FY 2017.
There is an old saying the figures lie and liars figure.
The real truth is Washington lies and disinforms Americans on a daily basis, why should anyone take anything the FAA says as gospel? Another bureaucracy attempting to justify more money for FY 2017.
There is an old saying the figures lie and liars figure.