Well, Plane Jim, I think you said it all. You are an Outlaw.

You named some of the benefits but they don't seem to make you happy.

Unless you are an aviation person, like most of the members I know, you really don't need to belong. I think of you as a loner, not an outlaw. I've known a few of them myself and there's nothing wrong with it so don't feel guilty. You enjoy flying by yourself from your own statement and don't want to belong to a club and you are correct, your Homeowners Coverage will take care of the basic liability if you get sued.
AMA is in large part a "fellowship of aviation people" and for some reason you don't seem to feel that you are part of it. Either because you enjoy being on your own or because you simply haven't been part of aviation. Either way, there is no crime involved. But whatever you do, first examine your own feelings and do what you feel is right.
3dbob