They feared them in maximum security to turn earth into a prison planet. Even in the recentish secret empire, the reason aliens weren't willing to help was because.... Earth kinda scary. I mean they broke the infinity stones and now they'd standing up to a the chitauri wave