Sarya was just carrying those other losers. Especially Mano. Tharok (especially when he goes all Dark Man and constructs entire other supervillain teams), Persuader and Validus are at least credible threats.

Really, the biggest 'problem' with the Fatal Five is their name. Call yourself 'Fatal,' I expect to see some fatalities, yanno? That's like calling yourself the 'World's Greatest Assassin,' and then never actually managing to freaking kill one of the Teen Titans you're hunting, Slade...

The League of Super-Assassins, or the plain old League of Assassins, kinda feel the same way. Whole lotta bragging for a whole lotta failing to kill their targets...