Jeff, I'm assuming from this statement that you're a fairly low time pilot if this is the first time you've experienced this. What you described is typical of any aircraft of any size when the speed is allowed to drop. There's nothing "wrong" with the model at all. It's just how they feel when there's not enough airflow to fly and the controls get really mushy from the low speed. Rather than avoid it I'd suggest you play with it to better learn what it feels like and how best to control the models during low speed flight. This performance region has it's own rules about what you can do and how the model reacts and it carries it's own fun and rewards when you master it.

For example at this speed and angle of attack region you'll likely experience adverse yaw where using the ailerons causes the model to roll one direction and yaw the other. With enough aileron you may even promote a deeper stall on the aileron down side and have the model drop into a spiral dive opposite to your control input. The response to this is to use a LITTLE aileron and also coordinate in some rudder with the aileron. With that the model will respond with a more normal action. Another fun item is hovering about 5 to 10 degrees nose high into the wind at minimum airspeed and with just enough power to prevent a deep stall where the nose drops away. With the right model and wind it'll look like it's stuck there or even flying backwards.

In any event learning to fly your models in the lower speed region will not only make you a better pilot but it's fun as well. For obvious reasons lighter models will do best in this mode but any model can be used and it should be so that if something goes wrong during a landing approach you are better equipped to deal with it.