If the racists would only shut up, any legitimate criticism wouldn't be buried underneath all the bile. When people who have never watched it are screaming at the top of their lungs, and most of the anger seems directed at the 'not white' cast members, it becomes very easy to throw all criticism into the pile with the loudest ones.
Of course, that presumes there is a criticism outside of how people look, or how things aren't perfectly lined up to the book. Because people forget that the Lord of the Rings trilogy has a lot, and I mean a LOT, of changes.