Unless you're going to be using the fuse in a high-voltage line, you don't need to worry about the voltage rating. The current rating is the important one.

For example, I have some .5 amp fuses from Radio Shack for my analog voltage meter. They are rated for 250 volts but will work fine for lower voltage circuits. They are tiny glass cyclinders with metal caps at the ends. You can see the fine wire element inside. Alligator clips on the metal caps would probably be the easiest connection for the test, but I think wires could be soldered to them as well. There are also dedicated fuse holders, including inline designs.

Another possibility might be mini automotive fuses, if you can find a current rating low enough to provide adequate protection. Keep the battery and test leads as short as possible. Without the prop, the motor should operate smoothly through its speed range without blowing the fuse if the windings have not been damaged.