The way it was explained to me, the "poyt" sound at idle is caused at the instant the intake valve opens during the valve overlap period and can be heard through the exhaust. Just like with the popping noise one can make with their finger/mouth it requires a sudden change in pressure. It takes a valve opening with a lower pressure on one side than the other to create that popping sound. Keeping in mind that the velocity of the exhaust gasses leaving cylinder does create a lower than atmospheric pressure in the cylinder just before the intake valve opens makes sense, otherwise exhaust scavenging wouldn't work at all.

In addition, the richer the mixture, the wetter the intake valve, the more frequent the popping sound will occur.

Anyway, that's what I've heard, so please don't throw fireballs at me.