"When a man takes a mistress, he doesn't turn around and divorce his wife."