"To me, my X-People" just doesn't roll off the tongue

X-men is iconic, and it refers to man as in mankind, not male. I see no need to change it.