Yes, something like that, although I would emphasize the extradimensional aspect of the alliance. For simplicity's sake, I probably would have the person that represented Atlantis represent the entire Earth dimension, just as I would if that person was from Russia, England, Mexico, Peru, Australia, Guam, Wakanda, Canada or wherever. It's not like you have to access a different dimension to get to Atlantis...although I have read some fantasy fiction where Atlantis was, indeed, in an adjacent dimension and not just a submerged continent on Earth proper.
I've always liked the concept of alliances, especially ones that have been in place for so long that many have forgotten that they even exist. But then a super baddie comes along and a clarion call goes out rallying all parties and reminding them that they are obligated to provide a united defense. That, to me, would be the Defenders. And if for some reason the Defenders fail (which would never happen with a team like this), then you'd call in the Avengers to, well, avenge.