Peter Vogel's Scoring System
06-01-2014, 07:59 PM
#1
Thread Starter
My Feedback: (92)
Join Date: Dec 2001
Location: Rosamond, CA
Posts: 2,089
Likes: 0
Received 0 Likes
on
0 Posts
Peter Vogel's Scoring System
I'm coming late to this party as this weekend was the first time I was able to use the new judging/scoring system that Peter Vogel has been developing. After using it for two rounds this weekend in Victorville, here is my opinion.
Peter has put a tremendous amount of effort and money in to developing this system. He also seems extremely committed to continuing the development of the entire system.
From the standpoint of just being a judge sitting in the chair, I don't ever want to have to judge with a pencil in my hand again. Peter's game controller interface is, at least from what I can tell, just about perfect. It takes very little time to get used to using it. And you now have the ability to never have to take your eyes off the model. It seemed far less tasking both physically and mentally to judge. I personally feel that I was able to do a much better job of judging with this interface.
In all honesty, the system still has some bugs. There were times that things would lock up and Peter would have to get involved getting everything back up. So there is still work to be done. But the advantages are so significant that I feel we should support him as much as we can in getting this to completion.
I truly feel that this could be the biggest improvement to the entire pattern experience that I have ever seen. I know that sounds like a big statement, but you have to experience it to realize the possibilities.
My hats off to Peter for this work! I'm going to push the Donate button on the website to help him the little I can. I urge everyone in pattern to do the same. This truly can be a gamechanger.
Peter has put a tremendous amount of effort and money in to developing this system. He also seems extremely committed to continuing the development of the entire system.
From the standpoint of just being a judge sitting in the chair, I don't ever want to have to judge with a pencil in my hand again. Peter's game controller interface is, at least from what I can tell, just about perfect. It takes very little time to get used to using it. And you now have the ability to never have to take your eyes off the model. It seemed far less tasking both physically and mentally to judge. I personally feel that I was able to do a much better job of judging with this interface.
In all honesty, the system still has some bugs. There were times that things would lock up and Peter would have to get involved getting everything back up. So there is still work to be done. But the advantages are so significant that I feel we should support him as much as we can in getting this to completion.
I truly feel that this could be the biggest improvement to the entire pattern experience that I have ever seen. I know that sounds like a big statement, but you have to experience it to realize the possibilities.
My hats off to Peter for this work! I'm going to push the Donate button on the website to help him the little I can. I urge everyone in pattern to do the same. This truly can be a gamechanger.
Last edited by TonyF; 06-01-2014 at 08:05 PM.
06-02-2014, 04:38 AM
#2
Join Date: Jun 2012
Location: Simi Valley, Ca. CA
Posts: 28
Likes: 0
Received 0 Likes
on
0 Posts
Great job Peter! Can't wait to try your new scoring system at the next contest. Maybe Lancaster summer heat one day? Or not 
. I'll have to wait for Franks Sacramento contest. I here he is going to have a few rounds of classic too. Hope you get the bugs out of the system by then. Thanks for all the hard work.
Donald
. I'll have to wait for Franks Sacramento contest. I here he is going to have a few rounds of classic too. Hope you get the bugs out of the system by then. Thanks for all the hard work.Donald
06-02-2014, 10:20 AM
#3
I'm a believer. Once fully developed, it will be the way to go. The biggest benefit in my eyes is that the participants will receive much more accurate judging. By not having to look away from the airplane, deductions can be made as the judge sees them. This should greatly diminish "impression" judging. The next greatest benefit is the greatly reduced workload for the score keeper. By having the system automatically capture the scores, it will also eliminate input errors from manual score inputs.
Also being able to see scores in real time on the monitor on the scoring desk is just awesome!
-Robert
Also being able to see scores in real time on the monitor on the scoring desk is just awesome!
-Robert
06-02-2014, 11:53 AM
#4
Thanks Tony for the donation! As I'm still fleshing out the http://www.patternscoring.com content (as opposed to the scores that get posted there as close to real-time as internet coverage from Verizon or AT&T at the field allow) let me be clear on how I hope to use the funds received from the donate button.
At the moment, the total HW package (not counting the laptop & printer which you need to run a contest with paper anyway) comes to just over $1000 (I'm hoping to halve that through working on an Android version of the scoring SW that can run on an integrated Android game device something like this one: http://www.amazon.com/JXD-Capacitive.../dp/B00ASHURMQ). From a D7-only perspective, I believe we'll need to get to having a NorCal and a SoCal setup since it's not always possible to have me at every contest. From a broader perspective, I want this to be affordable for any district or club to get a system package for their needs. (note: it can also handle IMAC, so clubs that are heavy into Aerobatics may want a system for themselves).
So, in the short term the donations go toward me acquiring equipment with which to experiment on improvements to the system (i.e. as controllers come out, can I get a better/more reliable controller at a lower cost, as Android gaming devices come out, can I leverage them) or to flesh out a gap in the system (for example, one individual donated an iPod touch so that we had enough equipment to run 3 lines and/or have spares if something goes wrong). In the longer term, the donations will go to offsetting the cost of running the www.patternscoring.com infrastructure in Amazon Web Services.
I look at the current season here in D7 as a beta test for the system, every contest I learn something more (BTW, there's extensive logging in the software so I can pull the logs after a contest and figure out the source of any glitches we saw and fix them if it's a software problem). I'm pleased to say that we have never lost a single score in the system, though there have been a few occasions where a judge has reverted to paper due to a HW issue (most frequently a controller glitching and losing it's connection to the iPod) -- at this past weekend's contest I hand entered about 50 scores while the system processed 3954 scores automatically.
Improvements after Oakdale's 2 round test:
1. Larger fonts all around to make it easier to read in judge's chair conditions
2. Split the judge/round data entry and contestant selection screens to make the flow from contestant to contestant smoother
3. Improved the saving of data on the iPod itself (save after EVERY score entry) -- in honesty the fact that no scores were lost that contest was luck more than anything else.
4. Improved performance all around
5. message from controller to scoring UI on the PC to notify scorekeeper that scores are available to be processed.
6. LOTs of work to support different normalization schemes, prelims/finals for all classes, etc. to support experimental contest structures used at Arvin.
Improvements after Arvin:
1. Voice instructions for what to do on each screen
2. Cosmetic fixes to review screen
3. Noticed judges were pulling the trigger after the review hoping to save that way -- added the feature :-)
4. Better error handling if the connection to the DB is lost for any reason
5. Added logging for everything that happens on the device so I can analyze crashes, etc.
6. Recorded video introduction to the system to reduce the time needed to explain the system at the beginning of the contest.
7. Better handling of lost websocket connection between the scoring computer and the scoreboard display server which also handles the
scores available messaging to the scoring system, etc.
8. Built installer so that the scoring program itself can be installed and used on any PC, not just my development box.
Improvements after Sacramento:
1. Save data on a separate thread so that if save is slow we don't get in the way of contest flow to the next contestant.
2. Suggestion from Chris Atwood: Put a BIG button on the judge setup screen to advance to contestant select since not everyone knows the iOS navigation paradigm of buttons in the upper right/left of the screen to go forward/back. Accept trigger input to advance as well since that's what judges get used to during scoring.
3. Show only the classes that are being used in the contest instead of all classes.
4. Custom-made foam inserts for the box used to transport the system to reduce setup/tear down time
5. Fixes for bugs that showed up in a P-only contest (incorrect formulae in the generated standings spreadsheet trying to reference F scoresheets)
6. Make sure there's a big lipo providing power to the controllers so that we don't have glitches caused by low controller batteries.
7. Automatically correct "Not Observed" from one judge with the score from the other judge, alert when there's an NO from both judges on the same maneuver. Previously this was something I did manually when reviewing the scores, and I occasionally missed one!
There have also been a variety of fixes to the more cosmetic things like patternscoring.com itself and the local equivalent that serves up the scoreboard at the contest, particularly better support for tablet and phone size displays if you want to look at your scores on your own device, etc.
At Victorville I learned a few more things, mostly hardware/controller related:
1. When powered externally, the controllers charge themselves, but stop charging the iPods! :-( Killed the controller glitching issues for the most part, but at the cost of iPod batteries running low toward the end of the contest.
2. iPods + Heat + getting charged by the controller (when external power is disconnected) = thermal shutdown of the iPod :-(
3. There's at least one crash that happens occasionally during the score save sequence. I need to analyze the logs of the devices to understand + fix. Scores are of course retrievable, but it's disconcerting to judges and results in a support call from the flightline.
As far as I'm concerned, anything that requires the judges to call "Peter!" from the judge's chair is a defect that needs to be repaired, as well as anything that requires me to manually touch a scoring spreadsheet (i.e. at this contest judges for Advanced round 4 erroneously recorded it as Advanced round 5 so I had to manually copy the scores over to round 4 and clear round 5) or any inconveniences to me or to judges during the contest.
I'm working on using RFID card readers to automate judge entry and contestant entry to simplify things more, but round selection is a tough one given the number of variations on how we run a contest (i.e. masters round 1 on left line, round 2 on right).
We'll get there! I don't think we'll achieve perfection, but paper + pencil isn't perfect either!
Peter+
At the moment, the total HW package (not counting the laptop & printer which you need to run a contest with paper anyway) comes to just over $1000 (I'm hoping to halve that through working on an Android version of the scoring SW that can run on an integrated Android game device something like this one: http://www.amazon.com/JXD-Capacitive.../dp/B00ASHURMQ). From a D7-only perspective, I believe we'll need to get to having a NorCal and a SoCal setup since it's not always possible to have me at every contest. From a broader perspective, I want this to be affordable for any district or club to get a system package for their needs. (note: it can also handle IMAC, so clubs that are heavy into Aerobatics may want a system for themselves).
So, in the short term the donations go toward me acquiring equipment with which to experiment on improvements to the system (i.e. as controllers come out, can I get a better/more reliable controller at a lower cost, as Android gaming devices come out, can I leverage them) or to flesh out a gap in the system (for example, one individual donated an iPod touch so that we had enough equipment to run 3 lines and/or have spares if something goes wrong). In the longer term, the donations will go to offsetting the cost of running the www.patternscoring.com infrastructure in Amazon Web Services.
I look at the current season here in D7 as a beta test for the system, every contest I learn something more (BTW, there's extensive logging in the software so I can pull the logs after a contest and figure out the source of any glitches we saw and fix them if it's a software problem). I'm pleased to say that we have never lost a single score in the system, though there have been a few occasions where a judge has reverted to paper due to a HW issue (most frequently a controller glitching and losing it's connection to the iPod) -- at this past weekend's contest I hand entered about 50 scores while the system processed 3954 scores automatically.
Improvements after Oakdale's 2 round test:
1. Larger fonts all around to make it easier to read in judge's chair conditions
2. Split the judge/round data entry and contestant selection screens to make the flow from contestant to contestant smoother
3. Improved the saving of data on the iPod itself (save after EVERY score entry) -- in honesty the fact that no scores were lost that contest was luck more than anything else.
4. Improved performance all around
5. message from controller to scoring UI on the PC to notify scorekeeper that scores are available to be processed.
6. LOTs of work to support different normalization schemes, prelims/finals for all classes, etc. to support experimental contest structures used at Arvin.
Improvements after Arvin:
1. Voice instructions for what to do on each screen
2. Cosmetic fixes to review screen
3. Noticed judges were pulling the trigger after the review hoping to save that way -- added the feature :-)
4. Better error handling if the connection to the DB is lost for any reason
5. Added logging for everything that happens on the device so I can analyze crashes, etc.
6. Recorded video introduction to the system to reduce the time needed to explain the system at the beginning of the contest.
7. Better handling of lost websocket connection between the scoring computer and the scoreboard display server which also handles the
scores available messaging to the scoring system, etc.
8. Built installer so that the scoring program itself can be installed and used on any PC, not just my development box.
Improvements after Sacramento:
1. Save data on a separate thread so that if save is slow we don't get in the way of contest flow to the next contestant.
2. Suggestion from Chris Atwood: Put a BIG button on the judge setup screen to advance to contestant select since not everyone knows the iOS navigation paradigm of buttons in the upper right/left of the screen to go forward/back. Accept trigger input to advance as well since that's what judges get used to during scoring.
3. Show only the classes that are being used in the contest instead of all classes.
4. Custom-made foam inserts for the box used to transport the system to reduce setup/tear down time
5. Fixes for bugs that showed up in a P-only contest (incorrect formulae in the generated standings spreadsheet trying to reference F scoresheets)
6. Make sure there's a big lipo providing power to the controllers so that we don't have glitches caused by low controller batteries.
7. Automatically correct "Not Observed" from one judge with the score from the other judge, alert when there's an NO from both judges on the same maneuver. Previously this was something I did manually when reviewing the scores, and I occasionally missed one!
There have also been a variety of fixes to the more cosmetic things like patternscoring.com itself and the local equivalent that serves up the scoreboard at the contest, particularly better support for tablet and phone size displays if you want to look at your scores on your own device, etc.
At Victorville I learned a few more things, mostly hardware/controller related:
1. When powered externally, the controllers charge themselves, but stop charging the iPods! :-( Killed the controller glitching issues for the most part, but at the cost of iPod batteries running low toward the end of the contest.
2. iPods + Heat + getting charged by the controller (when external power is disconnected) = thermal shutdown of the iPod :-(
3. There's at least one crash that happens occasionally during the score save sequence. I need to analyze the logs of the devices to understand + fix. Scores are of course retrievable, but it's disconcerting to judges and results in a support call from the flightline.
As far as I'm concerned, anything that requires the judges to call "Peter!" from the judge's chair is a defect that needs to be repaired, as well as anything that requires me to manually touch a scoring spreadsheet (i.e. at this contest judges for Advanced round 4 erroneously recorded it as Advanced round 5 so I had to manually copy the scores over to round 4 and clear round 5) or any inconveniences to me or to judges during the contest.
I'm working on using RFID card readers to automate judge entry and contestant entry to simplify things more, but round selection is a tough one given the number of variations on how we run a contest (i.e. masters round 1 on left line, round 2 on right).
We'll get there! I don't think we'll achieve perfection, but paper + pencil isn't perfect either!
Peter+
