If its in an enclosed space, forget any automotive battery. They are rarely sealed, and are unsuited to the charge/discharge cycles that model boat use gives them, mostly because they tend to vent, what they tend to vent is a mixture of hydrogen, oxygen and bits of sulphuric acid. Between them, they make a mess of their surroundings. The acid corrodes stuff, if the H and O get a chance to recombine...
You need to know the current that the motor(s) take under load and how long they need to run. The tells you how much battery you need. Then you need to know how much weight your hull will carry (max displacement in fresh water). Effectively. this tells you how much battery you can have. Archimedes was a clever guy.