What you do to check individual cells is correct; black on black and then measure either two or both remaining wires with the red-lead from the voltmeter.
Each cell reading 4.2V is a healthy voltage! That's how its supposed to be .

The lowest/least voltage they are supposed to get is 3.4V, please don't get them below this voltage, as it will be hard to recover them without proper gear.

So no worries Rice!