Hi!
I have been using many OS LA .40 engines through the years and I can assure you that your problems is not related to the 11x6 prop.
In fact the OS LA .40 have no difficulties swinging big props like a 11x6 and 12x4 if you fly at sea level .
10x6 is a prop size that is not that good for the engine, at least not in a high winged trainer like the Kyosho "Calmato" and planes like that with a span of 160cm. For smaller planes like a Q-500 racer weighting around 1700-1800g the 10x6 could be used though.
Best 11x6 prop is the RAM , Graupner Cam-prop or the APC.
A 10x4 or 10x5 is way to small for most airplanes where the OS LA .40 could be considered

For best performance use an OS 8 or Enya 3 glowplug. Other good glow plugs are Nova-Rossi 4, 4A or Rossi 3 or 4.

It is also vital that the tank is not too big. A 240cc tank (8oz) works good for a 6,5cc engine (.40).Best tank set-up is to use two clunks,one for pressure from the silencer and the other to the carb , this is called " and gives a more even fuel flow to the engine. Even better is to use a Tettra "Bubbleless" tank..but this isn't necessary for your engine.
Remember to place the tank according to the "Tank rule"!

Fuel!
The OS LA series of engines run fairly good on 0% nitro, but they will run even better on 5%nitro. 10-15% nitro is just a waste of money on these engines as they throttle and run just as good on 5% nitro.
20% Castor oil, 5% nitro and 75% Metanol is a good blend.

Placing the high speed needle 10cm from the carb is no problem.