speaking as someone who's been working as an electronics/computer technician for over 20 years, the amount of heatsink compound you use does make a difference.
Heatsink compound can't transfer heat as well as actual physical contact. It does transfer heat better than air.
What's going on is that no surface is perfectly smooth, so when you put two metal parts together, you have many little points of contact, with many little air gaps between.
The theory of using heat sink compound is that you just put on enough to barely cover the surfaces being combined, so that the compound doesn't prevent all the little spots from contacting that normally would with no compound on the parts. If you smear a heavy amount on, you're going to be holding the pieces apart, and actually slightly decrease the effectiveness of the heatsink, since it now has no actual points of metal to metal contact.
You're always better off with a heatsink than without one, no matter how well it's installed, but the heatsink will do its job better with a very minimal amount of heatsink compound on it.
I can site one example where someone had built themselves a computer with dual pentium processors that repeatedly locked up. The processors came with heatsinks pre-installed, but for some reason he'd removed the heatsink on one of them, then replaced it with a huge quantity of heatsink compound. You could easily feel the difference between the two heatsinks. The one he re-installed was barely warm, while the one that was installed at the factory was hot. In other words, it was drawing a lot more heat off its processor than the other one was.
The problem was cured by carefully cleaning all the goop off the heatsink and the chip with denatured ethyl alchohol and a lint free swap, then reapplying the correct amount of compound.

To sum up; you're best off with a heatsink that's installed with a minimal amount of compound, 2nd best is a heatsink with no compound, 3rd is a heatsink with too much compound, and last, no heatsink.