Should I invite them?

My in laws have made absolutely no effort to have a relationship with me. They are completely self absorbed and obsessed with sports, their two youngest kids, or traveling. Recently, my father in law told my husband that he should "never have kids with her" (me) and that my husband always finds girls like me with "problems and baggage". I believe he is calling my child (from prev marriage) baggage and I don't feel he is healthy to be around my child. My husband is telling me to grow up and get over it, but I'm not going to (just happened, a very fresh incident). Should I be breaking my neck to invite these imbeciles to my child's birthday party when he said something like that about my child and myself?