8 BALL,
A prop and motor's job is to move water in the direction you don't want to go, so that the boat will go in the direction you do want to go (equal and opposite reaction, plus a little friction and inefficiency, right?). So, if you know how much water your boat displaces, it's not ~that~ difficult to determine the diameter and pitch of the prop required to move that much water, and at what speed the motor has to be turning. The idea is to find out the volume of water moved for one revolution of the propellor. Multiply that by the RPM of the propellor to get the volume per minute (or 'per second' if you'd rather). The 'trick' is to then find a motor that has the required power to spin the prop fast enough to do the work of moving that much water.

Hey, that doesn't sound all that difficult, does it? But there are a few other things you have to consider, too. One of them is that the shape and/or 'type' of hull plays a part in how fast a boat can go. A displacement hull is 'slower' than a planing hull, it has to 'displace', or move the water out of the way. A planing hull sort of 'skips' over the water, doesn't move a lot of water out of the way of the boat, which means that less power (less work is done) is required for speed once the hull starts to 'plane'. A certain minimum amount of 'work' has to be done to achive the 'plane', though. A few of the other things that will affect speed are the density of the water, the temperature of the water (which affects density), and the friction between water and the hull (affected by the material the hull is made of, and how 'smooth' the hull is), and any movement of the water (and air, unless the boat is a submarine).

There's a practical limit to how fast a propellor can spin without cavitating, which also depends on the density of the water (also salinity and temperature), which is why deep submerged subs can go faster than surfaced subs (among other reasons). The 'shape' and diameter of the prop can make a huge difference in speed, along with the prop's finish (smooth/rough).

Depending on just how exacting your project (professor) is, there's a lot of the above that you may or may not have to get into (doesn't hurt to mention it though, it'll impress the old 'fart' and get you a better grade!).
Have fun...
- 'Doc