How did he look when he said it? Exasperated? Envious? Admiring? Sarcastic? Annoyed? You can tell when he's annoyed, because his eyebrows are tense but the rest of his face is relaxed, as if he realizes that he's looking annoyed and should try to temper it for the sake of politeness.