i'll tell you a wee story....
i first started RC flying with a 2Ch glider, i knew how planes flew, i just had to teach my thumbs. a friend sold me a beat up old President Hi-fly glider, i fitted a 1400mah pack from a RC buggy, 4 cell!! i crashed a few seconds after the throw coz i was too keen and 25mph and hi-flys don't mix. one day i got two hours solid, that solved my concentration problem.
then at our club display a glider designer came with all his wares, i'd fancied this sleek little glider for ages and went to buy, he said nooooo, you want this, showing my some rudder elevator polyhedral thing, YUCK! nononno i said i want this one, pointing to a 1m V-tail aerobatic fun glider, it scored 7 on his 1-8 scale of pilot skill
after a while he gave up trying to sell me his trainer, even the small one, but said if i MUST have this, then get the standard tail coz it wont tip stall so easy, well i agreeed and i got it.
in my haste i built it wrong, putting the PACKING! foam between the ribs making it a bit heavy, i also covered the fuz in SolarTex thinking it'd take the rough landing site better.
well, what luck, i did crash a few times, it was quick, even on low rates, but the foam wing was solid! and the SolarTex held the fuz together

this glider taught me how to fly, when i got better i turned the rates up and flew faster, i then sealed my hinge gaps and now i had a mad fun plane. i've since built a new wing, fitted mini servos to fly in light winds... umm 20mph is the lightest so far, 45 the most!
this glider has given me 7 years great service, i still enjoy flying it, im not bored, far from it, i LOVE! flying it.
So dont let the nay sayers hold you back, be well aware of a steeper learning curve, but trainers arnt the only way