I think you have 3 different segments of people in the hobby. You have flyers that just want to fly and have no interest in building, they buy ARF's and fly them till there wore out or crashed and throw them away without a 2nd thought of repairing them. When it comes down to it ARF's are cheaper and faster than building like it or not.
Another segment enjoys building and flying equally and they will build a lot of kits and enjoy showing up at the field with their latest creation and tearing up the skies with pride. Usually this is the younger to middle aged class with some spare time on their hands.
The last class would be your scale builder, may build from kits but there are very few scale worthy kits. Most build from plans or have a kit laser cut for a particular plan they want to build from. This will mostly be middle aged to retired modelers who really enjoy the challenge and satisfaction from completing a fine sale subject and this usually ranks higher than flying. As someone already stated a scale build is usually a long term commitment.
ARF's have their place and are not to blame for the demise of the kits, it is the current culture that is killing kits. Most people now days want it now. No patience and or pride. Now I do have to admit I have an ARF in my trailer, it is my fun-fly plane that I will rip around and do things I would not dare try with one of my scale builds. I have nothing but $$$ invested in it to risk.
People can find the time to build if they want to build. A guy in my club says he never has time to build and I know him well he spends several hours every evening watching TV. It is all about priorities and choices. Flyers will fly and builders will build. It is what ever floats your boat or flips your prop. I personally think every modeler should at least attempt to build one model to get the full experience from the hobby. You never know what your missing until you try it.
Now to answer the question of where have the builders gone, I think the 1st group has about 70% of the modelers based off of what I see show up at our Club's Fly-Ins.
Later!!
Anthony