When think about it, it doesn't just happen with Avengers.
Remember when Thunderbolts are specifically Zemo's pretender heroes? Then it's Marvel's Suicide Squad and they shove whatever villain/Anti-hero into it.
Judging from Twitter, Reddit, etc, the general audience just see Thunderbolts, Young Avengers as any group of villains/anti-heroes, Young heroes, just like Avengers nowadays is just considered a group of any heroes.