If I put in some down elevator to compensate for the diving as it starts to get close to upside down will I be able to get it to roll smoother?
Yes.
I would also increase your aileron throw. Either move the control rod in on the control horns (move it closer to the control surface), or move it out on the servo arm. More throw will increase the roll rate, but will also make all of your flying a bit more sensitive to ailerons. Just be aware at first, it won't be a dramatic change.
Have fun.
Dennis-