Think of it like it was a flying wing. Up spoilers is like up elevators (or elevons) on a wing. You'll need down elevator to counter.