"In regard to the amount of difference between the races, we must make some allowance for our nice powers of discrimination gained by a long habit of observing ourselves."