Yeah, but this is everything. Society over laps.
If you start lumping everyone non straight white male together, you basically get an us vs them of perceived 'norms' vs 'anyone different' not to mention problems of dividing current communities.
The Pride flag exists to help people identify and come together for a specific reason - once you start making that reason about other reasons, you dilute the point, and the impact.