I agree with Crosswind. Sounds fairly typical.

A couple of things might be contributing to it though. When you say "doesn't seem like much difference in speed, just louder"... are you talking about in the air or on the ground? A high-drag plane will do this more than a low-drag plane. There is also some chance that you are over-propping the engine. What kind of plane, what engine, and what prop?