I think the important thing here is that you felt hurt by what he said. IMHO, when one partner says, "when you said X I felt hurt," the other person doesn't get to pass judgement on that feeling. He can ask why, he can explain what he meant, but somewhere in there he has to accept that you felt that way and express regret that you did.

Of course, he can also say HE felt hurt that you didn't see his comment as a joke, and YOU can be sorry for the fact that you couldn't take it that way.

It's not a matter of what anyone else thinks about your feelings. You are entitled to express them (as long as you own them as your own feelings, which it seems clear you did) and the person who loves you should respect that and show they care, even if they disagree. And vice versa of course.

That's my 2 cents.