Fresh in from our favourite usability guy:
Tl;dr – 5 is the optimal. As you add more users, you spend more time observing same findings per new insight gained. If you just want stats, test at least 20 so the maths look good. For card sorting, test 15 or more. When eyetracking, test 39 to account for individual variations.