RE: which prop
You might be going the wrong direction with your prop size. Try an 11x5 prop on your .40 LA and see how you like it; it's actually the recommended prop size for the O.S. .40 LA according to the engine owner's manual. You can also buy a 9x6 or 9x7 prop and try that while you're at it, too. An 11x5 prop will move a larger column of air and provide quicker take-off thrust and more control authority at lower speeds.
Propellers are one area where pilots can safely experiment with their planes but seldom do so. Trying different lengths and pitches of propeller can result in very different performance characteristics at times. You can reduce your engine noise significantly by switch from a blunt-ended propeller to a "scimitar" style or tapered tip propeller. Running a 3-bladed 10x5 propeller versus a 2-bladed 11x5 propeller would be an interesting experiment as well. Common folklore states that 3-bladed props are less efficient, but I've run them on various engines and I've liked them quite well.
With bushed engines like the O.S. LA series or the Thunder Tiger GP series, you may see better overall performance with higher castor oil content in your fuel. The thick castor oil helps float the engines crankshaft on the bushings and provides smoother performance.