While I am very much in favor of being kind and considerate to others, I am equally opposed to political correctness and redefining words to suit ideological agendas.

From Dictionary.com, only one of the first three definitions for man is gender/age specific. The second and third are as follows:

2. a member of the species Homo sapiens or all the members of this species collectively, without regard to sex. Ex: prehistoric man.
3. the human individual as representing the species, without reference to sex; the human race; humankind. Ex: Man hopes for peace, but prepares for war.

Obviously, these are the relevant definitions for the given usage, X-Men. These definitions should inform the view we have of the intended meaning of the name X-Men. Here's why: they are doubtlessly the definitions that came to mind for the creators of the X-Men when they came up with the name. The intended meaning was obviously not gender specific on their part, and limiting yourself to the first definition, adult male, while ignoring the second and third definitions, simply shows the bias of the person doing so.