You might as well give up now while you're ahead.

But seriously, this is one of those areas where there is no perfect way to do it.

I HIGHLY recommend sheet metal screws with thin CA to harden the wood threads. It holds up very well, even after repeated removal. and if it ever gets really bad, you can fill the hole with epoxy and re-drill.