Before you start dismantling the brakes, just do a few checks first to see if you cant save yourself a bit of trouble.
First, are the batteries fully charged, are they 6v or just the standard AAs'. Low power will quickly affect servo performance.
Second, make sure the brakes are adjusted properly. When the brakes are applied at standstill, try to push the car to see if the brakes resist. If not, it could be that the servo arm isn't moving far enough to engage the brake fully.
The only reason I think it could be something other than the brake pads is because they wear very slowly (in my experience). I have had my car for approaching three years now and my original brake pad still works ok.