Without knowing the boat, it is hard to say, so guesswork appears. It might be that you have found the limit of what that hull design can do. There comes a point where just applying more power doesn't make the boat go any faster, it just misbehaves.