I am rebuilding a 36" mono hull with an OS RG-M .21 wich makes apx 1.7 hp. any help with prop selection would be appreciated

check out www.impba.net, there is a propeller chart/table.

An Octura X-440/3 or Prather 215 are good places to start. You may have to play around a little to find what the boat likes.

What are some advantages of metal over plastic? Has anyone tried the same diameter and pitch in both to compare?

Metal doesn't flex and is harder to break. For anything above a .049 metal is the way to go.

metal won't bend during high load or rpm meaning less loss of power. with plastic prop advantage i think is more on safety, let say ur prop hit something then the prop will absorb the impact instead the shaft or even the hull which is less damage compare if ur using metal prop.