Yeah, again I've kind think I've always read those comments and complaints as it annoys him that he has this weakness, not that he has any increased susceptibility or sensitivity to it. Where someone like Constantine would have the wards and whatnot to protect himself. Kind of like how Batman is sometimes written as being annoyed by it too.
Although now that I'm thinking about it, wasn't there a New 52 Batman/Superman story where they walked into a building or something and Clark was affected by the magic but Bruce wasn't?