If you ever plan on towing with a car I would recommend fixing the brakes. If you will always tow with a truck you should be fine. Even after your finished with the trailer you will really notice it back there with a car, especially when stopping.
Take a look at this site. It has a lot of information on axle maintenace and brakes.
http://www.dexteraxle.com/products___literature