Kang
Immortus
Ultron
Zodiac Cartel
Masters of Evil
Nebula
Lethal Legion
Grim Reaper
the Gatherers
Master Pandemonium
Alkhema
And I really could go on and on...
The thing is, unlike the X-Men or the FF, who are mostly known as a team, the Avengers are a collection of stars that have their own title, and the villians from those own titles end up carrying over to the group book, as has happened with Loki, the Olympian Gods, Red Skull and so many others... The same has been known to happen with DC's Justice League.
Peace