Technically doing so is illegal according to the FCC....

However if you do it at home to test things out, I doubt you'll hurt anything.

Switching the crystals may not work properly, but it will not damage the electronics.

Give it a try... if it works it will help pinpoint the problem.