I believe that they were fast more due to the super low wing loading of those early pylon racing deltas. If you go back and look at the designs they were HUGE in wing area while still being quite clean due to having nothing but a wing. They were also built very light for their size thanks to the building skills of all those modelers of the day that learned their chops in the free flight field before going into radio control. Yes that's a shameless plug for free flight....

Also many of them had tip chords sizable enough to be more accurately classed as tapered flying wings. It's not that these pylon racing model delta designs were better than regular planforms. They just used the existing rules to slide around and come out better due to the circumstances of the rules. The pylon class in those days also used very small engines. .19's were the max size as I recall for that brief period. So there may well be a lot of apples to cumquats in making such a comparison.