I ran into a similar situation.
Setup was a Magnum .36, with a Nelson/PerfSpec Ultrathrust muffler.
Engine ran fine with an APC 8x8, turning low 17s. Went to 8x7 after breaking my last 8x8, and the thing turned upper 18s. But I could not get it to richen up. Went to a 9x7, turned 17s, and needle response was good.
Seems as if the engine turned outside the tuning range, muffler tank pressure went down.

So my question for you would be which muffler do you have. The Nelson Sport .40 was offered with a choice of tuned mufflers. The standard Q-500 version for the 19k range, and a lower rev muffler for spinning a 10x6 in the 15-16k range.

FWIW, I run 20% fuel with 20% oil that is all synthetic in my Nelson Sport .40 with the Q-500 muffler and ST carb. I am getting 19,200 with an APC 8.75x9 peaked. I fly it @18,700. Throttle response is fantastic. No detonation, plugs last about 15 runs.

I hope this helps.

RR