Periodically hitting his kids and allowing some of them to get killed on his watch, not to mention all the emotional manipulation/neglect/abuse, are negatives that are pretty easy to accentuate.
Especially as it wasn't what he used to be like, so he's been derailed pretty hardcore.