Other supers people don't worry about, because if you're a super powered person, you're either a superhero or a super villain. People don't hate superheroes because they're heroes. They're on your side. They're the good guys. Supervillains naturally get hated. Nobody makes a big deal out of because they're villains.
Mutants, however, anybody can be a mutant. That guy who cut you off in traffic? The jock who stole your girlfriend? That idiot that doesn't understand 10 items or less at the grocery store? Any of them could be a mutant. Mutants are normal people, and normal people are the worst. That's before you even get into the whole "Oh, no. We're good guys, we're just going to replace your species," stuff and the "Of course I had to poison everyone around me. That's my power! I have a right to use my power!"